2024-05-09T08:47:51.619 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T08:47:51.619 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710 --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/orig.config.yaml 2024-05-09T08:47:51.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T08:47:51.782 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698710' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF8FXeR+b6B2VVY/WMCwOBNrCkm3tB7L3TNJXX80KsokSRPKecv9qrVxwdtWiEPDLt7ayY6IO4L0kPpdY+OEGRw= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ud5JztXFNxZjHDeaed5/PMWZ3y/xJziIXiCMzpXP+Ijx/mm4TNxFt2fK9ziR3yOpxpyavmjRrVijQg5Xah6k= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjHw0qvRGp/7v1i3oL3Xsu+abJJlqUAuJEzC+67950D4+1zussDCQxuXMAoJ5qoKFwO2jAbiP4YeHRhaR89LmU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T08:47:51.782 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T08:47:51.783 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T08:47:51.783 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T08:47:51.786 INFO:teuthology.task.internal:Checking packages... 2024-05-09T08:47:51.818 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T08:47:51.819 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T08:47:51.819 INFO:teuthology.packaging:ref: None 2024-05-09T08:47:51.819 INFO:teuthology.packaging:tag: None 2024-05-09T08:47:51.819 INFO:teuthology.packaging:branch: squid 2024-05-09T08:47:51.819 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:47:51.819 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T08:47:52.055 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T08:47:52.056 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T08:47:52.065 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T08:47:52.065 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T08:47:52.079 INFO:teuthology.task.internal:Saving configuration 2024-05-09T08:47:52.105 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T08:47:52.116 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T08:47:52.142 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:42:43.247135', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF8FXeR+b6B2VVY/WMCwOBNrCkm3tB7L3TNJXX80KsokSRPKecv9qrVxwdtWiEPDLt7ayY6IO4L0kPpdY+OEGRw='} 2024-05-09T08:47:52.165 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:42:43.248662', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ud5JztXFNxZjHDeaed5/PMWZ3y/xJziIXiCMzpXP+Ijx/mm4TNxFt2fK9ziR3yOpxpyavmjRrVijQg5Xah6k='} 2024-05-09T08:47:52.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:42:43.249617', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjHw0qvRGp/7v1i3oL3Xsu+abJJlqUAuJEzC+67950D4+1zussDCQxuXMAoJ5qoKFwO2jAbiP4YeHRhaR89LmU='} 2024-05-09T08:47:52.186 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T08:47:52.194 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T08:47:52.195 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T08:47:52.195 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T08:47:52.195 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T08:47:52.305 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f054ddcd160>, signals=[15]) 2024-05-09T08:47:52.306 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T08:47:52.317 INFO:teuthology.task.internal:Opening connections... 2024-05-09T08:47:52.317 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T08:47:52.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:47:52.394 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-05-09T08:47:52.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:47:52.471 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-05-09T08:47:52.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:47:52.548 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T08:47:52.555 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-05-09T08:47:52.574 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-05-09T08:47:52.574 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-05-09T08:47:52.632 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-05-09T08:47:52.632 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-05-09T08:47:52.632 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:47:52.633 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:47:52.634 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:47:52.634 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:47:52.635 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-05-09T08:47:52.667 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-05-09T08:47:52.685 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-05-09T08:47:52.685 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-05-09T08:47:52.741 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:47:52.742 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:47:52.743 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:47:52.743 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-05-09T08:47:52.766 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-05-09T08:47:52.784 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-05-09T08:47:52.784 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:47:52.843 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:47:52.844 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:47:52.844 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-05-09T08:47:52.866 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T08:47:52.876 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T08:47:52.889 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T08:47:52.889 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:47:52.891 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:47:52.894 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:47:52.910 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T08:47:52.921 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T08:47:52.921 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:47:52.949 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:47:52.952 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:47:53.065 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T08:47:53.151 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T08:47:53.167 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-09T08:47:53.168 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-09T08:47:53.168 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:47:53.168 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:47:53.169 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:47:53.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:47:53.186 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-05-09T08:47:53.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:47:53.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:47:53.188 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-05-09T08:47:53.189 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-05-09T08:47:53.243 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-437.el9.x86_64 2024-05-09T08:47:53.244 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-437.el9.x86_64 2024-05-09T08:47:53.244 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-05-09T08:47:53.245 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-437.el9.x86_64 2024-05-09T08:47:53.245 INFO:teuthology.task.kernel:Running kernel on smithi179: 5.14.0-437.el9.x86_64 2024-05-09T08:47:53.245 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum install -y kernel 2024-05-09T08:47:53.247 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-437.el9.x86_64 2024-05-09T08:47:53.247 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-437.el9.x86_64 2024-05-09T08:47:53.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-05-09T08:47:56.637 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-09T08:47:56.644 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-05-09T08:47:57.506 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.1 MB 00:02 2024-05-09T08:47:59.238 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-05-09T08:47:59.379 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-09T08:47:59.565 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-05-09T08:48:03.489 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-05-09T08:48:03.546 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-09T08:48:03.649 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-09T08:48:05.005 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-05-09T08:48:05.075 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-05-09T08:48:06.298 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-09T08:48:07.894 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-09T08:48:08.033 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 21 MB 00:02 2024-05-09T08:48:08.834 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-05-09T08:48:12.991 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-05-09T08:48:13.339 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T08:48:14.126 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-09T08:48:14.965 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T08:48:14.965 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T08:48:15.067 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:48:15.074 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-05-09T08:48:15.075 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-05-09T08:48:15.075 INFO:teuthology.orchestra.run.smithi179.stdout:Install 4 Packages 2024-05-09T08:48:15.075 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:48:15.075 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 78 M 2024-05-09T08:48:15.075 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 123 M 2024-05-09T08:48:15.075 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-05-09T08:48:15.429 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T08:48:15.429 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T08:48:15.529 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-09T08:48:15.534 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-05-09T08:48:15.534 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-05-09T08:48:15.534 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-05-09T08:48:15.534 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-05-09T08:48:15.534 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-05-09T08:48:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:48:15.536 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 78 M 2024-05-09T08:48:15.536 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 123 M 2024-05-09T08:48:15.536 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-05-09T08:48:16.110 INFO:teuthology.orchestra.run.smithi179.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 500 kB/s | 484 kB 00:00 2024-05-09T08:48:16.131 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T08:48:16.132 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T08:48:16.234 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-05-09T08:48:16.247 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-05-09T08:48:16.248 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:48:16.249 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 78 M 2024-05-09T08:48:16.249 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 123 M 2024-05-09T08:48:16.249 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-05-09T08:48:16.374 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 697 kB/s | 484 kB 00:00 2024-05-09T08:48:16.652 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.4 MB/s | 484 kB 00:00 2024-05-09T08:48:16.768 INFO:teuthology.orchestra.run.smithi179.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 9.5 MB/s | 15 MB 00:01 2024-05-09T08:48:17.535 INFO:teuthology.orchestra.run.smithi179.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-05-09T08:48:18.744 INFO:teuthology.orchestra.run.smithi179.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 11 MB/s | 28 MB 00:02 2024-05-09T08:48:18.744 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:48:18.744 INFO:teuthology.orchestra.run.smithi179.stdout:Total 21 MB/s | 78 MB 00:03 2024-05-09T08:48:19.425 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-05-09T08:48:19.759 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-05-09T08:48:19.759 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-05-09T08:48:19.818 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 4.4 MB/s | 15 MB 00:03 2024-05-09T08:48:20.664 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-05-09T08:48:20.664 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-05-09T08:48:20.752 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 6.8 MB/s | 28 MB 00:04 2024-05-09T08:48:21.802 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 6.2 MB/s | 34 MB 00:05 2024-05-09T08:48:21.802 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:48:21.802 INFO:teuthology.orchestra.run.smithi139.stdout:Total 14 MB/s | 78 MB 00:05 2024-05-09T08:48:22.487 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-05-09T08:48:22.845 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-05-09T08:48:22.846 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-05-09T08:48:23.076 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-05-09T08:48:23.746 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:48:23.799 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-05-09T08:48:23.799 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-05-09T08:48:23.830 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:48:26.199 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-05-09T08:48:26.503 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:48:26.639 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:48:26.857 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:48:26.951 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:48:28.464 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 1.2 MB/s | 15 MB 00:12 2024-05-09T08:48:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:48:29.777 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:48:30.915 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 2.2 MB/s | 34 MB 00:15 2024-05-09T08:48:31.385 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:48:31.426 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:48:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:48:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:48:35.423 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 1.5 MB/s | 28 MB 00:19 2024-05-09T08:48:35.423 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:48:35.423 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.9 MB/s | 78 MB 00:19 2024-05-09T08:48:36.126 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:48:36.129 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-05-09T08:48:36.565 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-05-09T08:48:36.565 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-05-09T08:48:37.502 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-05-09T08:48:37.503 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-05-09T08:48:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:48:40.088 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-05-09T08:48:40.738 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:48:40.814 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:48:43.629 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:48:43.765 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:48:48.850 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:48:48.893 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:48:53.983 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:05.494 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:05.532 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:05.884 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:05.884 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:49:05.884 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:49:05.884 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T08:49:06.696 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T08:49:06.697 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:49:06.697 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-05-09T08:49:07.079 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:49:07.157 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:07.157 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:49:07.157 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T08:49:07.157 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T08:49:07.157 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T08:49:07.158 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-09T08:49:07.158 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi179.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T08:49:07.158 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:49:07.208 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:07.209 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:49:07.209 DEBUG:teuthology.orchestra.run.smithi179:> sudo rpm -qi grub2-tools 2024-05-09T08:49:07.250 INFO:teuthology.orchestra.run.smithi179.stdout:Name : grub2-tools 2024-05-09T08:49:07.250 INFO:teuthology.orchestra.run.smithi179.stdout:Epoch : 1 2024-05-09T08:49:07.250 INFO:teuthology.orchestra.run.smithi179.stdout:Version : 2.06 2024-05-09T08:49:07.250 INFO:teuthology.orchestra.run.smithi179.stdout:Release : 68.el9 2024-05-09T08:49:07.250 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: x86_64 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Group : Unspecified 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Size : 8264254 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:License : GPLv3+ 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Packager : builder@centos.org 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Vendor : CentOS 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Summary : Support tools for GRUB. 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:Description : 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi179.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T08:49:07.252 INFO:teuthology.orchestra.run.smithi179.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T08:49:07.252 INFO:teuthology.orchestra.run.smithi179.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T08:49:07.252 INFO:teuthology.orchestra.run.smithi179.stdout:hardware devices. 2024-05-09T08:49:07.252 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:49:07.252 INFO:teuthology.orchestra.run.smithi179.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T08:49:07.253 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T08:49:07.253 INFO:teuthology.task.kernel:Updating grub on smithi179 to boot 5.14.0-444.el9.x86_64 2024-05-09T08:49:07.253 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T08:49:07.876 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-05-09T08:49:08.557 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:08.595 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:08.938 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:08.939 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:49:08.939 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:49:08.939 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:49:09.266 INFO:teuthology.orchestra.run.smithi179.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T08:49:09.309 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-05-09T08:49:09.311 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-05-09T08:49:09.327 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.UQ4BQwQayN 2024-05-09T08:49:09.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UQ4BQwQayN 2024-05-09T08:49:09.393 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.UQ4BQwQayN 2024-05-09T08:49:09.531 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.UQ4BQwQayN is 6KB 2024-05-09T08:49:09.579 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.UQ4BQwQayN 2024-05-09T08:49:09.593 DEBUG:teuthology.orchestra.run.smithi179:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T08:49:09.664 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T08:49:09.665 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T08:49:09.665 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T08:49:09.665 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T08:49:09.666 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:49:09.697 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-05-09T08:49:09.820 DEBUG:teuthology.orchestra.run.smithi179:> sudo shutdown -r now 2024-05-09T08:49:10.031 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:49:10.111 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:10.112 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:49:10.112 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T08:49:10.112 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T08:49:10.112 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T08:49:10.112 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-09T08:49:10.112 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T08:49:10.112 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:49:10.162 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:10.162 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:49:10.162 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 68.el9 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8264254 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-05-09T08:49:10.193 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:49:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T08:49:10.195 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T08:49:10.195 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-444.el9.x86_64 2024-05-09T08:49:10.196 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T08:49:10.826 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-05-09T08:49:12.244 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T08:49:12.275 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-05-09T08:49:12.277 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-05-09T08:49:12.292 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.2RJ6X2spGj 2024-05-09T08:49:12.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2RJ6X2spGj 2024-05-09T08:49:12.371 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.2RJ6X2spGj 2024-05-09T08:49:12.503 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.2RJ6X2spGj is 6KB 2024-05-09T08:49:12.551 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.2RJ6X2spGj 2024-05-09T08:49:12.566 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T08:49:12.627 INFO:teuthology.orchestra.run.smithi139.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T08:49:12.627 INFO:teuthology.orchestra.run.smithi139.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T08:49:12.627 INFO:teuthology.orchestra.run.smithi139.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T08:49:12.627 INFO:teuthology.orchestra.run.smithi139.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T08:49:12.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T08:49:12.763 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-05-09T08:49:24.739 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:24.779 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:25.118 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:25.119 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:49:25.119 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:49:25.119 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:49:25.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:49:25.974 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-09T08:49:26.321 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:49:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:26.391 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:49:26.391 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T08:49:26.392 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T08:49:26.392 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T08:49:26.392 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-09T08:49:26.392 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T08:49:26.392 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:49:26.443 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:49:26.443 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:49:26.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-05-09T08:49:26.476 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-05-09T08:49:26.476 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 68.el9 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8264254 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-05-09T08:49:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:49:26.478 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T08:49:26.479 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T08:49:26.479 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-444.el9.x86_64 2024-05-09T08:49:26.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T08:49:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-05-09T08:49:28.455 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T08:49:28.502 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-05-09T08:49:28.504 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-05-09T08:49:28.519 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.J4XK7z0t00 2024-05-09T08:49:28.519 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.J4XK7z0t00 2024-05-09T08:49:28.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.J4XK7z0t00 2024-05-09T08:49:28.728 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.J4XK7z0t00 is 6KB 2024-05-09T08:49:28.778 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.J4XK7z0t00 2024-05-09T08:49:28.793 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T08:49:28.857 INFO:teuthology.orchestra.run.smithi064.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T08:49:28.857 INFO:teuthology.orchestra.run.smithi064.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T08:49:28.857 INFO:teuthology.orchestra.run.smithi064.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T08:49:28.858 INFO:teuthology.orchestra.run.smithi064.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T08:49:28.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T08:49:29.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-05-09T08:49:39.830 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-09T08:49:39.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-09T08:49:39.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:49:42.769 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-09T08:49:42.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-05-09T08:49:42.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:49:58.303 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-05-09T08:49:59.015 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-09T08:49:59.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-09T08:49:59.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:01.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-05-09T08:50:07.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-09T08:50:07.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:10.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-05-09T08:50:10.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:10.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-05-09T08:50:13.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-05-09T08:50:17.506 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-05-09T08:50:22.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-09T08:50:22.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:25.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-05-09T08:50:25.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:26.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-09T08:50:26.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:29.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-05-09T08:50:41.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-09T08:50:41.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:50:54.289 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-09T08:50:54.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-09T08:50:54.745 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-444.el9.x86_64"... 2024-05-09T08:50:54.746 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-05-09T08:50:54.771 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-444.el9.x86_64 2024-05-09T08:50:54.772 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T08:50:54.772 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T08:50:54.772 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T08:50:55.772 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-09T08:50:55.773 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T08:50:55.834 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-05-09T08:50:56.018 DEBUG:teuthology.parallel:result is None 2024-05-09T08:50:57.034 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-05-09T08:50:57.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-05-09T08:50:57.396 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-09T08:50:57.396 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-05-09T08:50:57.454 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-444.el9.x86_64 2024-05-09T08:50:57.454 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T08:50:57.454 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T08:50:57.454 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T08:50:58.455 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-09T08:50:58.455 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T08:50:58.538 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-05-09T08:50:58.569 DEBUG:teuthology.parallel:result is None 2024-05-09T08:51:13.367 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-05-09T08:51:13.751 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-09T08:51:13.751 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-09T08:51:13.751 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-05-09T08:51:13.767 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-444.el9.x86_64 2024-05-09T08:51:13.767 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T08:51:13.767 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T08:51:13.767 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T08:51:14.768 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-09T08:51:14.768 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T08:51:14.842 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-05-09T08:51:14.874 DEBUG:teuthology.parallel:result is None 2024-05-09T08:51:14.874 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T08:51:14.881 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T08:51:14.882 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:51:14.884 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:51:14.886 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:51:14.906 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T08:51:14.913 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T08:51:14.921 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T08:51:14.922 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:51:14.938 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:51:14.945 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:51:15.157 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T08:51:15.164 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T08:51:15.164 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:51:15.166 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:51:15.169 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:51:15.200 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:51:15.211 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:51:15.213 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:51:15.225 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:51:15.255 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:51:15.266 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:51:15.267 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T08:51:15.275 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T08:51:15.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:51:15.277 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:51:15.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:51:15.308 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T08:51:15.316 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T08:51:15.317 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:51:15.342 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:51:15.347 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:51:15.365 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:51:15.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:51:15.486 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:51:15.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:51:15.609 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:51:15.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:51:15.685 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:51:15.730 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:51:15.778 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:51:15.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:51:15.905 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:51:15.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:51:15.971 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:51:16.016 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:51:16.072 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:51:16.157 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:51:16.200 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:51:16.200 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:51:16.268 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-05-09T08:51:16.270 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-05-09T08:51:16.272 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-05-09T08:51:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:51:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:51:16.348 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:51:16.765 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T08:51:16.773 INFO:teuthology.task.internal:Starting timer... 2024-05-09T08:51:16.773 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T08:51:16.782 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T08:51:16.790 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T08:51:16.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-05-09T08:51:16.891 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-05-09T08:51:16.896 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-05-09T08:51:16.942 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-05-09T08:51:16.944 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-05-09T08:51:16.992 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-05-09T08:51:16.994 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T08:51:16.995 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-05-09T08:51:17.044 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-05-09T08:51:17.045 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-05-09T08:51:17.088 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-05-09T08:51:17.088 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-05-09T08:51:17.172 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-05-09T08:51:17.172 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-05-09T08:51:17.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:51:17.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:51:17.222 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:51:17.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:51:17.252 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:51:17.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:51:17.282 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T08:51:17.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:51:17.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:51:17.424 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:51:17.453 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T08:51:17.462 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T08:51:17.477 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T08:51:17.478 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi139.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T08:55:25.745 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-05-09T08:55:25.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-09T08:55:25.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:55:25.822 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-05-09T08:55:25.901 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-09T08:55:25.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-05-09T08:55:25.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:55:25.976 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-05-09T08:55:26.066 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-05-09T08:55:26.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-09T08:55:26.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:55:26.181 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-09T08:55:26.262 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-09T08:55:26.263 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T08:55:26.276 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T08:55:26.276 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:55:26.276 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:55:26.279 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:55:26.279 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:55:26.281 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:55:26.282 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:55:26.308 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:55:26.310 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:55:26.323 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:55:26.325 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:55:26.333 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:55:26.348 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:55:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-05-09T08:55:26.350 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-05-09T08:55:26.361 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-05-09T08:55:26.361 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-05-09T08:55:26.374 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-05-09T08:55:26.375 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:55:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:55:26.385 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-05-09T08:55:26.389 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:55:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:55:26.400 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:55:26.414 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:55:26.448 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:55:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:55:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-05-09T08:55:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.452 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.453 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:55:26.456 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:55:26.456 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-09T08:55:26.456 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.456 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.456 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.456 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.470 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:55:26.474 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:55:26.474 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-05-09T08:55:26.474 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.474 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.474 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.474 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:55:26.475 INFO:teuthology.run_tasks:Running task install... 2024-05-09T08:55:26.483 DEBUG:teuthology.task.install:project ceph 2024-05-09T08:55:26.483 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T08:55:26.484 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T08:55:26.484 INFO:teuthology.task.install:Using flavor: default 2024-05-09T08:55:26.490 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T08:55:26.490 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T08:55:26.490 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:55:26.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:55:26.493 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:55:26.493 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:55:26.494 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:55:26.494 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:55:26.644 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:55:26.644 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:55:26.752 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:55:26.753 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:55:26.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:55:26.778 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:55:26.779 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:55:26.805 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:55:26.805 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:55:26.823 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:55:26.823 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:55:26.844 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:55:26.910 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:55:26.910 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:55:26.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:55:26.919 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:55:26.930 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:55:26.930 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:55:26.930 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:55:26.938 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:55:26.938 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:55:26.957 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:55:26.957 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:55:26.983 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-05-09T08:55:26.985 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-05-09T08:55:27.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:55:27.023 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:55:27.081 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:55:27.098 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:55:27.145 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-05-09T08:55:27.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-05-09T08:55:27.162 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-05-09T08:55:27.163 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-05-09T08:55:27.211 INFO:teuthology.orchestra.run.smithi139.stdout:54 files removed 2024-05-09T08:55:27.238 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:55:27.385 INFO:teuthology.orchestra.run.smithi064.stdout:54 files removed 2024-05-09T08:55:27.393 INFO:teuthology.orchestra.run.smithi179.stdout:54 files removed 2024-05-09T08:55:27.412 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:55:27.419 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:55:27.967 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-05-09T08:55:28.044 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 239 kB/s | 83 kB 00:00 2024-05-09T08:55:28.125 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 196 kB/s | 83 kB 00:00 2024-05-09T08:55:28.387 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 40 kB/s | 11 kB 00:00 2024-05-09T08:55:28.401 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-09T08:55:28.459 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-05-09T08:55:28.746 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-09T08:55:28.793 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-09T08:55:28.809 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-05-09T08:55:29.438 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-09T08:55:30.418 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.1 MB 00:01 2024-05-09T08:55:30.760 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-09T08:55:31.706 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-09T08:55:32.669 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-09T08:55:33.644 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-05-09T08:55:35.691 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-09T08:55:37.334 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-09T08:55:37.762 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-05-09T08:55:39.187 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 16 kB 00:00 2024-05-09T08:55:39.353 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-09T08:55:40.202 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 21 MB 00:02 2024-05-09T08:55:40.283 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-09T08:55:40.854 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-09T08:55:43.372 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 21 MB 00:02 2024-05-09T08:55:45.377 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-05-09T08:55:45.623 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T08:55:47.738 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:55:47.738 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:55:47.749 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:55:47.750 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:55:47.751 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:55:47.752 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:55:47.754 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:55:47.756 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:55:47.757 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:55:47.817 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-05-09T08:55:47.845 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-05-09T08:55:47.845 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:55:47.846 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:55:47.847 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:55:47.848 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:55:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:55:47.850 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:55:47.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:55:47.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:55:47.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:55:47.854 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-05-09T08:55:47.855 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout:Install 148 Packages 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 8 Packages 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 225 M 2024-05-09T08:55:47.856 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-05-09T08:55:48.059 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:55:48.059 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:55:48.071 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:55:48.071 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:55:48.072 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:55:48.073 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:55:48.075 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:55:48.077 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:55:48.078 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:55:48.142 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:55:48.168 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:55:48.169 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:55:48.170 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:55:48.171 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:55:48.172 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:55:48.173 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:55:48.174 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:55:48.175 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:55:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:55:48.177 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:55:48.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 225 M 2024-05-09T08:55:48.179 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-05-09T08:55:48.555 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T08:55:48.658 INFO:teuthology.orchestra.run.smithi064.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-09T08:55:48.729 INFO:teuthology.orchestra.run.smithi179.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-05-09T08:55:48.930 INFO:teuthology.orchestra.run.smithi179.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 4.3 MB/s | 889 kB 00:00 2024-05-09T08:55:48.971 INFO:teuthology.orchestra.run.smithi179.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 3.4 MB/s | 145 kB 00:00 2024-05-09T08:55:49.122 INFO:teuthology.orchestra.run.smithi179.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 16 MB/s | 2.4 MB 00:00 2024-05-09T08:55:49.231 INFO:teuthology.orchestra.run.smithi179.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 14 MB/s | 1.5 MB 00:00 2024-05-09T08:55:49.498 INFO:teuthology.orchestra.run.smithi179.stdout:(6/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 18 MB/s | 4.7 MB 00:00 2024-05-09T08:55:50.292 INFO:teuthology.orchestra.run.smithi064.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 545 kB/s | 889 kB 00:01 2024-05-09T08:55:50.382 INFO:teuthology.orchestra.run.smithi179.stdout:(7/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 19 MB/s | 17 MB 00:00 2024-05-09T08:55:50.542 INFO:teuthology.orchestra.run.smithi064.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 581 kB/s | 145 kB 00:00 2024-05-09T08:55:50.923 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:55:50.924 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:55:50.934 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:55:50.935 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:55:50.936 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:55:50.937 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:55:50.939 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:55:50.941 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:55:50.942 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:55:50.948 INFO:teuthology.orchestra.run.smithi179.stdout:(8/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 18 MB/s | 10 MB 00:00 2024-05-09T08:55:50.990 INFO:teuthology.orchestra.run.smithi179.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 606 kB/s | 25 kB 00:00 2024-05-09T08:55:51.003 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-05-09T08:55:51.014 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-05-09T08:55:51.014 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-05-09T08:55:51.014 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-05-09T08:55:51.014 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:55:51.015 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:55:51.016 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:55:51.017 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:55:51.018 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:55:51.019 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:55:51.020 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:55:51.021 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:55:51.022 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:55:51.023 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:55:51.024 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 8 Packages 2024-05-09T08:55:51.025 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:55:51.026 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 225 M 2024-05-09T08:55:51.026 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-05-09T08:55:51.756 INFO:teuthology.orchestra.run.smithi139.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-05-09T08:55:52.273 INFO:teuthology.orchestra.run.smithi139.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.7 MB/s | 889 kB 00:00 2024-05-09T08:55:52.398 INFO:teuthology.orchestra.run.smithi139.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 1.1 MB/s | 145 kB 00:00 2024-05-09T08:55:52.915 INFO:teuthology.orchestra.run.smithi139.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 4.0 MB/s | 5.4 MB 00:01 2024-05-09T08:55:53.091 INFO:teuthology.orchestra.run.smithi064.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 948 kB/s | 2.4 MB 00:02 2024-05-09T08:55:53.224 INFO:teuthology.orchestra.run.smithi139.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-05-09T08:55:53.349 INFO:teuthology.orchestra.run.smithi139.stdout:(6/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-05-09T08:55:53.899 INFO:teuthology.orchestra.run.smithi179.stdout:(10/156): ceph-base-19.0.0-2515.g96805dcf.el9.x 1.0 MB/s | 5.4 MB 00:05 2024-05-09T08:55:54.083 INFO:teuthology.orchestra.run.smithi139.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 5.5 MB/s | 4.7 MB 00:00 2024-05-09T08:55:54.124 INFO:teuthology.orchestra.run.smithi179.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 987 kB/s | 222 kB 00:00 2024-05-09T08:55:54.224 INFO:teuthology.orchestra.run.smithi179.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 270 kB/s | 27 kB 00:00 2024-05-09T08:55:54.333 INFO:teuthology.orchestra.run.smithi179.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 293 kB/s | 32 kB 00:00 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi064.stdout:(5/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 902 kB/s | 5.4 MB 00:06 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi064.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 976 kB/s | 1.5 MB 00:01 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi139.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 6.8 MB/s | 21 MB 00:03 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi139.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 337 kB/s | 25 kB 00:00 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi179.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 1.4 MB/s | 746 kB 00:00 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi179.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 853 kB/s | 163 kB 00:00 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi179.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 563 kB/s | 127 kB 00:00 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi139.stdout:(10/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 6.3 MB/s | 10 MB 00:01 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi179.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 1.0 MB/s | 500 kB 00:00 2024-05-09T08:55:55.792 INFO:teuthology.orchestra.run.smithi139.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.6 MB/s | 222 kB 00:00 2024-05-09T08:55:55.867 INFO:teuthology.orchestra.run.smithi139.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 361 kB/s | 27 kB 00:00 2024-05-09T08:55:55.926 INFO:teuthology.orchestra.run.smithi139.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 544 kB/s | 32 kB 00:00 2024-05-09T08:55:56.068 INFO:teuthology.orchestra.run.smithi139.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 5.1 MB/s | 746 kB 00:00 2024-05-09T08:55:56.143 INFO:teuthology.orchestra.run.smithi139.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.1 MB/s | 163 kB 00:00 2024-05-09T08:55:56.218 INFO:teuthology.orchestra.run.smithi139.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 1.7 MB/s | 127 kB 00:00 2024-05-09T08:55:56.327 INFO:teuthology.orchestra.run.smithi179.stdout:(18/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 9.1 MB/s | 48 MB 00:05 2024-05-09T08:55:56.352 INFO:teuthology.orchestra.run.smithi139.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 3.7 MB/s | 500 kB 00:00 2024-05-09T08:55:56.385 INFO:teuthology.orchestra.run.smithi179.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 779 kB/s | 45 kB 00:00 2024-05-09T08:55:56.535 INFO:teuthology.orchestra.run.smithi179.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 899 kB/s | 135 kB 00:00 2024-05-09T08:55:56.628 INFO:teuthology.orchestra.run.smithi179.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.7 MB/s | 162 kB 00:00 2024-05-09T08:55:56.728 INFO:teuthology.orchestra.run.smithi179.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 3.1 MB/s | 321 kB 00:00 2024-05-09T08:55:56.828 INFO:teuthology.orchestra.run.smithi179.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 2.9 MB/s | 298 kB 00:00 2024-05-09T08:55:56.912 INFO:teuthology.orchestra.run.smithi179.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T08:55:56.996 INFO:teuthology.orchestra.run.smithi179.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-09T08:55:57.369 INFO:teuthology.orchestra.run.smithi139.stdout:(18/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 5.2 MB/s | 5.3 MB 00:01 2024-05-09T08:55:57.461 INFO:teuthology.orchestra.run.smithi139.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 492 kB/s | 45 kB 00:00 2024-05-09T08:55:57.553 INFO:teuthology.orchestra.run.smithi139.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.4 MB/s | 135 kB 00:00 2024-05-09T08:55:57.588 INFO:teuthology.orchestra.run.smithi179.stdout:(26/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 5.2 MB/s | 3.1 MB 00:00 2024-05-09T08:55:57.636 INFO:teuthology.orchestra.run.smithi139.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.9 MB/s | 162 kB 00:00 2024-05-09T08:55:57.679 INFO:teuthology.orchestra.run.smithi179.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-09T08:55:57.728 INFO:teuthology.orchestra.run.smithi139.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 3.4 MB/s | 321 kB 00:00 2024-05-09T08:55:57.772 INFO:teuthology.orchestra.run.smithi179.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 346 kB/s | 32 kB 00:00 2024-05-09T08:55:57.820 INFO:teuthology.orchestra.run.smithi139.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.2 MB/s | 298 kB 00:00 2024-05-09T08:55:57.855 INFO:teuthology.orchestra.run.smithi179.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.7 MB/s | 142 kB 00:00 2024-05-09T08:55:57.987 INFO:teuthology.orchestra.run.smithi139.stdout:(24/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 3.6 MB/s | 17 MB 00:04 2024-05-09T08:55:58.012 INFO:teuthology.orchestra.run.smithi139.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 517 kB/s | 99 kB 00:00 2024-05-09T08:55:58.071 INFO:teuthology.orchestra.run.smithi139.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-09T08:55:58.075 INFO:teuthology.orchestra.run.smithi064.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 1.3 MB/s | 4.7 MB 00:03 2024-05-09T08:55:58.171 INFO:teuthology.orchestra.run.smithi139.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.7 MB/s | 171 kB 00:00 2024-05-09T08:55:58.263 INFO:teuthology.orchestra.run.smithi139.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 347 kB/s | 32 kB 00:00 2024-05-09T08:55:58.364 INFO:teuthology.orchestra.run.smithi139.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.4 MB/s | 142 kB 00:00 2024-05-09T08:55:58.439 INFO:teuthology.orchestra.run.smithi179.stdout:(30/156): ceph-mgr-dashboard-19.0.0-2515.g96805 6.0 MB/s | 3.5 MB 00:00 2024-05-09T08:55:58.739 INFO:teuthology.orchestra.run.smithi139.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 4.3 MB/s | 3.1 MB 00:00 2024-05-09T08:55:59.606 INFO:teuthology.orchestra.run.smithi179.stdout:(31/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 1.4 MB/s | 5.3 MB 00:03 2024-05-09T08:55:59.698 INFO:teuthology.orchestra.run.smithi179.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 5.9 MB/s | 7.4 MB 00:01 2024-05-09T08:55:59.740 INFO:teuthology.orchestra.run.smithi179.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 1.2 MB/s | 49 kB 00:00 2024-05-09T08:55:59.766 INFO:teuthology.orchestra.run.smithi179.stdout:(34/156): ceph-mgr-modules-core-19.0.0-2515.g96 1.5 MB/s | 249 kB 00:00 2024-05-09T08:55:59.791 INFO:teuthology.orchestra.run.smithi179.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 326 kB/s | 16 kB 00:00 2024-05-09T08:55:59.806 INFO:teuthology.orchestra.run.smithi139.stdout:(31/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 9.6 MB/s | 48 MB 00:05 2024-05-09T08:55:59.864 INFO:teuthology.orchestra.run.smithi139.stdout:(32/156): ceph-mgr-dashboard-19.0.0-2515.g96805 2.3 MB/s | 3.5 MB 00:01 2024-05-09T08:55:59.890 INFO:teuthology.orchestra.run.smithi139.stdout:(33/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.9 MB/s | 249 kB 00:00 2024-05-09T08:55:59.891 INFO:teuthology.orchestra.run.smithi179.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 7.4 MB/s | 763 kB 00:00 2024-05-09T08:55:59.906 INFO:teuthology.orchestra.run.smithi139.stdout:(34/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 1.2 MB/s | 49 kB 00:00 2024-05-09T08:55:59.925 INFO:teuthology.orchestra.run.smithi139.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 493 kB/s | 16 kB 00:00 2024-05-09T08:55:59.926 INFO:teuthology.orchestra.run.smithi179.stdout:(37/156): ceph-volume-19.0.0-2515.g96805dcf.el9 1.6 MB/s | 264 kB 00:00 2024-05-09T08:55:59.950 INFO:teuthology.orchestra.run.smithi179.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 310 kB/s | 18 kB 00:00 2024-05-09T08:55:59.999 INFO:teuthology.orchestra.run.smithi139.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.8 MB/s | 264 kB 00:00 2024-05-09T08:56:00.032 INFO:teuthology.orchestra.run.smithi139.stdout:(37/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 6.9 MB/s | 763 kB 00:00 2024-05-09T08:56:00.049 INFO:teuthology.orchestra.run.smithi139.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 363 kB/s | 18 kB 00:00 2024-05-09T08:56:00.108 INFO:teuthology.orchestra.run.smithi139.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-05-09T08:56:00.200 INFO:teuthology.orchestra.run.smithi139.stdout:(40/156): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-09T08:56:00.242 INFO:teuthology.orchestra.run.smithi139.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-09T08:56:00.284 INFO:teuthology.orchestra.run.smithi139.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-05-09T08:56:00.309 INFO:teuthology.orchestra.run.smithi139.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-09T08:56:00.367 INFO:teuthology.orchestra.run.smithi139.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-05-09T08:56:00.409 INFO:teuthology.orchestra.run.smithi139.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-05-09T08:56:00.426 INFO:teuthology.orchestra.run.smithi139.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-09T08:56:00.485 INFO:teuthology.orchestra.run.smithi139.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-09T08:56:00.527 INFO:teuthology.orchestra.run.smithi139.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-05-09T08:56:00.642 INFO:teuthology.orchestra.run.smithi179.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 112 kB/s | 80 kB 00:00 2024-05-09T08:56:00.935 INFO:teuthology.orchestra.run.smithi179.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 816 kB/s | 803 kB 00:00 2024-05-09T08:56:00.975 INFO:teuthology.orchestra.run.smithi064.stdout:(8/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 3.5 MB/s | 10 MB 00:02 2024-05-09T08:56:01.034 INFO:teuthology.orchestra.run.smithi064.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 434 kB/s | 25 kB 00:00 2024-05-09T08:56:01.277 INFO:teuthology.orchestra.run.smithi139.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 142 kB/s | 107 kB 00:00 2024-05-09T08:56:01.419 INFO:teuthology.orchestra.run.smithi139.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 694 kB/s | 689 kB 00:00 2024-05-09T08:56:01.460 INFO:teuthology.orchestra.run.smithi179.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 239 kB/s | 195 kB 00:00 2024-05-09T08:56:01.486 INFO:teuthology.orchestra.run.smithi139.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 942 kB/s | 943 kB 00:01 2024-05-09T08:56:01.760 INFO:teuthology.orchestra.run.smithi179.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 307 kB/s | 253 kB 00:00 2024-05-09T08:56:01.870 INFO:teuthology.orchestra.run.smithi139.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-05-09T08:56:01.917 INFO:teuthology.orchestra.run.smithi064.stdout:(10/156): ceph-common-19.0.0-2515.g96805dcf.el9 1.6 MB/s | 21 MB 00:13 2024-05-09T08:56:02.051 INFO:teuthology.orchestra.run.smithi064.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 1.6 MB/s | 222 kB 00:00 2024-05-09T08:56:02.087 INFO:teuthology.orchestra.run.smithi139.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-05-09T08:56:02.118 INFO:teuthology.orchestra.run.smithi064.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 405 kB/s | 27 kB 00:00 2024-05-09T08:56:02.176 INFO:teuthology.orchestra.run.smithi064.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 545 kB/s | 32 kB 00:00 2024-05-09T08:56:02.253 INFO:teuthology.orchestra.run.smithi179.stdout:(43/156): ceph-common-19.0.0-2515.g96805dcf.el9 1.5 MB/s | 21 MB 00:13 2024-05-09T08:56:02.270 INFO:teuthology.orchestra.run.smithi139.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 369 kB/s | 313 kB 00:00 2024-05-09T08:56:02.418 INFO:teuthology.orchestra.run.smithi064.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 3.0 MB/s | 746 kB 00:00 2024-05-09T08:56:02.518 INFO:teuthology.orchestra.run.smithi064.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 1.6 MB/s | 163 kB 00:00 2024-05-09T08:56:02.585 INFO:teuthology.orchestra.run.smithi064.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 1.9 MB/s | 127 kB 00:00 2024-05-09T08:56:02.603 INFO:teuthology.orchestra.run.smithi179.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-09T08:56:02.654 INFO:teuthology.orchestra.run.smithi139.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 32 kB/s | 18 kB 00:00 2024-05-09T08:56:02.670 INFO:teuthology.orchestra.run.smithi179.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 117 kB/s | 106 kB 00:00 2024-05-09T08:56:02.769 INFO:teuthology.orchestra.run.smithi064.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 2.7 MB/s | 500 kB 00:00 2024-05-09T08:56:02.813 INFO:teuthology.orchestra.run.smithi139.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-05-09T08:56:03.021 INFO:teuthology.orchestra.run.smithi139.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-05-09T08:56:03.053 INFO:teuthology.orchestra.run.smithi179.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 169 kB/s | 135 kB 00:00 2024-05-09T08:56:03.103 INFO:teuthology.orchestra.run.smithi064.stdout:(18/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 2.0 MB/s | 17 MB 00:08 2024-05-09T08:56:03.220 INFO:teuthology.orchestra.run.smithi064.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 386 kB/s | 45 kB 00:00 2024-05-09T08:56:03.370 INFO:teuthology.orchestra.run.smithi064.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 902 kB/s | 135 kB 00:00 2024-05-09T08:56:03.479 INFO:teuthology.orchestra.run.smithi179.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 143 kB/s | 125 kB 00:00 2024-05-09T08:56:03.504 INFO:teuthology.orchestra.run.smithi064.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.2 MB/s | 162 kB 00:00 2024-05-09T08:56:03.605 INFO:teuthology.orchestra.run.smithi139.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 169 kB/s | 160 kB 00:00 2024-05-09T08:56:03.629 INFO:teuthology.orchestra.run.smithi179.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 224 kB/s | 215 kB 00:00 2024-05-09T08:56:03.630 INFO:teuthology.orchestra.run.smithi139.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 55 kB/s | 45 kB 00:00 2024-05-09T08:56:03.646 INFO:teuthology.orchestra.run.smithi064.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 2.2 MB/s | 321 kB 00:00 2024-05-09T08:56:03.746 INFO:teuthology.orchestra.run.smithi064.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 2.9 MB/s | 298 kB 00:00 2024-05-09T08:56:03.813 INFO:teuthology.orchestra.run.smithi064.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.5 MB/s | 99 kB 00:00 2024-05-09T08:56:03.888 INFO:teuthology.orchestra.run.smithi064.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-09T08:56:04.097 INFO:teuthology.orchestra.run.smithi139.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 617 kB/s | 662 kB 00:01 2024-05-09T08:56:04.130 INFO:teuthology.orchestra.run.smithi179.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 640 kB/s | 689 kB 00:01 2024-05-09T08:56:04.222 INFO:teuthology.orchestra.run.smithi139.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 43 kB/s | 25 kB 00:00 2024-05-09T08:56:04.397 INFO:teuthology.orchestra.run.smithi179.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 139 kB/s | 107 kB 00:00 2024-05-09T08:56:04.506 INFO:teuthology.orchestra.run.smithi179.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 919 kB/s | 943 kB 00:01 2024-05-09T08:56:04.564 INFO:teuthology.orchestra.run.smithi139.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 608 kB/s | 582 kB 00:00 2024-05-09T08:56:04.647 INFO:teuthology.orchestra.run.smithi064.stdout:(26/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 2.8 MB/s | 5.3 MB 00:01 2024-05-09T08:56:04.739 INFO:teuthology.orchestra.run.smithi064.stdout:(27/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 3.6 MB/s | 3.1 MB 00:00 2024-05-09T08:56:04.755 INFO:teuthology.orchestra.run.smithi179.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-05-09T08:56:04.764 INFO:teuthology.orchestra.run.smithi139.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 71 kB/s | 47 kB 00:00 2024-05-09T08:56:04.781 INFO:teuthology.orchestra.run.smithi064.stdout:(28/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.3 MB/s | 171 kB 00:00 2024-05-09T08:56:04.806 INFO:teuthology.orchestra.run.smithi064.stdout:(29/156): ceph-grafana-dashboards-19.0.0-2515.g 477 kB/s | 32 kB 00:00 2024-05-09T08:56:04.856 INFO:teuthology.orchestra.run.smithi064.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.8 MB/s | 142 kB 00:00 2024-05-09T08:56:05.073 INFO:teuthology.orchestra.run.smithi139.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 344 kB/s | 292 kB 00:00 2024-05-09T08:56:05.131 INFO:teuthology.orchestra.run.smithi179.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-05-09T08:56:05.298 INFO:teuthology.orchestra.run.smithi179.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 359 kB/s | 313 kB 00:00 2024-05-09T08:56:05.348 INFO:teuthology.orchestra.run.smithi139.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 241 kB/s | 188 kB 00:00 2024-05-09T08:56:05.365 INFO:teuthology.orchestra.run.smithi139.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-09T08:56:05.607 INFO:teuthology.orchestra.run.smithi064.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 4.4 MB/s | 3.5 MB 00:00 2024-05-09T08:56:05.681 INFO:teuthology.orchestra.run.smithi179.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 18 kB 00:00 2024-05-09T08:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-09T08:56:05.865 INFO:teuthology.orchestra.run.smithi179.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-05-09T08:56:06.065 INFO:teuthology.orchestra.run.smithi064.stdout:(32/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 9.6 MB/s | 48 MB 00:05 2024-05-09T08:56:06.090 INFO:teuthology.orchestra.run.smithi064.stdout:(33/156): ceph-mgr-modules-core-19.0.0-2515.g96 514 kB/s | 249 kB 00:00 2024-05-09T08:56:06.124 INFO:teuthology.orchestra.run.smithi064.stdout:(34/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 847 kB/s | 49 kB 00:00 2024-05-09T08:56:06.149 INFO:teuthology.orchestra.run.smithi064.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 281 kB/s | 16 kB 00:00 2024-05-09T08:56:06.225 INFO:teuthology.orchestra.run.smithi064.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.6 MB/s | 264 kB 00:00 2024-05-09T08:56:06.290 INFO:teuthology.orchestra.run.smithi139.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-05-09T08:56:06.291 INFO:teuthology.orchestra.run.smithi064.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 274 kB/s | 18 kB 00:00 2024-05-09T08:56:06.367 INFO:teuthology.orchestra.run.smithi064.stdout:(38/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 3.4 MB/s | 763 kB 00:00 2024-05-09T08:56:06.416 INFO:teuthology.orchestra.run.smithi139.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-09T08:56:06.417 INFO:teuthology.orchestra.run.smithi064.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-05-09T08:56:06.449 INFO:teuthology.orchestra.run.smithi179.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-05-09T08:56:06.484 INFO:teuthology.orchestra.run.smithi064.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-09T08:56:06.499 INFO:teuthology.orchestra.run.smithi179.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 196 kB/s | 160 kB 00:00 2024-05-09T08:56:06.534 INFO:teuthology.orchestra.run.smithi064.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-09T08:56:06.541 INFO:teuthology.orchestra.run.smithi139.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-05-09T08:56:06.568 INFO:teuthology.orchestra.run.smithi064.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-05-09T08:56:06.618 INFO:teuthology.orchestra.run.smithi064.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-09T08:56:06.660 INFO:teuthology.orchestra.run.smithi064.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-05-09T08:56:06.685 INFO:teuthology.orchestra.run.smithi064.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-09T08:56:06.711 INFO:teuthology.orchestra.run.smithi064.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-09T08:56:06.736 INFO:teuthology.orchestra.run.smithi064.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-09T08:56:06.769 INFO:teuthology.orchestra.run.smithi064.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-05-09T08:56:06.820 INFO:teuthology.orchestra.run.smithi064.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-05-09T08:56:06.845 INFO:teuthology.orchestra.run.smithi064.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-09T08:56:06.870 INFO:teuthology.orchestra.run.smithi064.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-05-09T08:56:06.884 INFO:teuthology.orchestra.run.smithi179.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 651 kB/s | 662 kB 00:01 2024-05-09T08:56:06.954 INFO:teuthology.orchestra.run.smithi064.stdout:(52/156): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-05-09T08:56:06.979 INFO:teuthology.orchestra.run.smithi064.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-05-09T08:56:07.004 INFO:teuthology.orchestra.run.smithi064.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-05-09T08:56:07.030 INFO:teuthology.orchestra.run.smithi064.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-05-09T08:56:07.055 INFO:teuthology.orchestra.run.smithi064.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 220 kB/s | 11 kB 00:00 2024-05-09T08:56:07.081 INFO:teuthology.orchestra.run.smithi064.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-09T08:56:07.106 INFO:teuthology.orchestra.run.smithi064.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-05-09T08:56:07.148 INFO:teuthology.orchestra.run.smithi064.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-09T08:56:07.167 INFO:teuthology.orchestra.run.smithi179.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-05-09T08:56:07.198 INFO:teuthology.orchestra.run.smithi064.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-09T08:56:07.223 INFO:teuthology.orchestra.run.smithi064.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-09T08:56:07.240 INFO:teuthology.orchestra.run.smithi139.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 161 kB/s | 133 kB 00:00 2024-05-09T08:56:07.299 INFO:teuthology.orchestra.run.smithi139.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 903 kB/s | 911 kB 00:01 2024-05-09T08:56:07.374 INFO:teuthology.orchestra.run.smithi064.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 271 kB/s | 47 kB 00:00 2024-05-09T08:56:07.399 INFO:teuthology.orchestra.run.smithi064.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-05-09T08:56:07.424 INFO:teuthology.orchestra.run.smithi064.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-09T08:56:07.449 INFO:teuthology.orchestra.run.smithi064.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-09T08:56:07.476 INFO:teuthology.orchestra.run.smithi179.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 568 kB/s | 582 kB 00:01 2024-05-09T08:56:07.508 INFO:teuthology.orchestra.run.smithi064.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.4 MB/s | 3.0 MB 00:00 2024-05-09T08:56:07.559 INFO:teuthology.orchestra.run.smithi179.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-05-09T08:56:07.609 INFO:teuthology.orchestra.run.smithi064.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-05-09T08:56:07.650 INFO:teuthology.orchestra.run.smithi064.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-09T08:56:07.684 INFO:teuthology.orchestra.run.smithi064.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-09T08:56:07.709 INFO:teuthology.orchestra.run.smithi064.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-09T08:56:07.768 INFO:teuthology.orchestra.run.smithi064.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T08:56:07.810 INFO:teuthology.orchestra.run.smithi064.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-09T08:56:07.835 INFO:teuthology.orchestra.run.smithi064.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T08:56:07.866 INFO:teuthology.orchestra.run.smithi139.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-05-09T08:56:07.886 INFO:teuthology.orchestra.run.smithi064.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T08:56:07.961 INFO:teuthology.orchestra.run.smithi064.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-05-09T08:56:07.986 INFO:teuthology.orchestra.run.smithi064.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-09T08:56:08.010 INFO:teuthology.orchestra.run.smithi179.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-05-09T08:56:08.011 INFO:teuthology.orchestra.run.smithi064.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-09T08:56:08.036 INFO:teuthology.orchestra.run.smithi064.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-09T08:56:08.062 INFO:teuthology.orchestra.run.smithi064.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-09T08:56:08.087 INFO:teuthology.orchestra.run.smithi064.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-09T08:56:08.152 INFO:teuthology.orchestra.run.smithi179.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-09T08:56:08.158 INFO:teuthology.orchestra.run.smithi139.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 102 kB/s | 87 kB 00:00 2024-05-09T08:56:08.171 INFO:teuthology.orchestra.run.smithi064.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-09T08:56:08.196 INFO:teuthology.orchestra.run.smithi064.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-09T08:56:08.221 INFO:teuthology.orchestra.run.smithi064.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-09T08:56:08.246 INFO:teuthology.orchestra.run.smithi064.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T08:56:08.285 INFO:teuthology.orchestra.run.smithi179.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 233 kB/s | 188 kB 00:00 2024-05-09T08:56:08.350 INFO:teuthology.orchestra.run.smithi139.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 932 kB/s | 1.0 MB 00:01 2024-05-09T08:56:08.364 INFO:teuthology.orchestra.run.smithi064.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 297 kB/s | 42 kB 00:00 2024-05-09T08:56:08.464 INFO:teuthology.orchestra.run.smithi064.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-05-09T08:56:08.489 INFO:teuthology.orchestra.run.smithi064.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-09T08:56:08.515 INFO:teuthology.orchestra.run.smithi064.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T08:56:08.540 INFO:teuthology.orchestra.run.smithi064.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T08:56:08.565 INFO:teuthology.orchestra.run.smithi064.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T08:56:08.649 INFO:teuthology.orchestra.run.smithi064.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-05-09T08:56:08.674 INFO:teuthology.orchestra.run.smithi064.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-05-09T08:56:08.699 INFO:teuthology.orchestra.run.smithi064.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-05-09T08:56:08.724 INFO:teuthology.orchestra.run.smithi064.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-09T08:56:08.750 INFO:teuthology.orchestra.run.smithi064.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-09T08:56:08.775 INFO:teuthology.orchestra.run.smithi064.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-09T08:56:08.800 INFO:teuthology.orchestra.run.smithi064.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-09T08:56:08.826 INFO:teuthology.orchestra.run.smithi064.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T08:56:08.851 INFO:teuthology.orchestra.run.smithi064.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T08:56:08.876 INFO:teuthology.orchestra.run.smithi064.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi064.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-09T08:56:08.975 INFO:teuthology.orchestra.run.smithi139.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 299 kB/s | 244 kB 00:00 2024-05-09T08:56:09.067 INFO:teuthology.orchestra.run.smithi139.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-05-09T08:56:09.160 INFO:teuthology.orchestra.run.smithi064.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-09T08:56:09.176 INFO:teuthology.orchestra.run.smithi139.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 300 kB/s | 248 kB 00:00 2024-05-09T08:56:09.185 INFO:teuthology.orchestra.run.smithi179.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-05-09T08:56:09.186 INFO:teuthology.orchestra.run.smithi064.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-05-09T08:56:09.319 INFO:teuthology.orchestra.run.smithi179.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-05-09T08:56:09.361 INFO:teuthology.orchestra.run.smithi179.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-09T08:56:09.568 INFO:teuthology.orchestra.run.smithi139.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-05-09T08:56:09.593 INFO:teuthology.orchestra.run.smithi139.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-05-09T08:56:09.636 INFO:teuthology.orchestra.run.smithi064.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 686 kB/s | 309 kB 00:00 2024-05-09T08:56:09.669 INFO:teuthology.orchestra.run.smithi064.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-05-09T08:56:09.711 INFO:teuthology.orchestra.run.smithi064.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-05-09T08:56:09.903 INFO:teuthology.orchestra.run.smithi064.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 136 kB/s | 26 kB 00:00 2024-05-09T08:56:09.928 INFO:teuthology.orchestra.run.smithi064.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 841 kB/s | 863 kB 00:01 2024-05-09T08:56:09.960 INFO:teuthology.orchestra.run.smithi139.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-05-09T08:56:09.979 INFO:teuthology.orchestra.run.smithi064.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-05-09T08:56:10.061 INFO:teuthology.orchestra.run.smithi179.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 180 kB/s | 133 kB 00:00 2024-05-09T08:56:10.079 INFO:teuthology.orchestra.run.smithi064.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-09T08:56:10.104 INFO:teuthology.orchestra.run.smithi064.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-09T08:56:10.171 INFO:teuthology.orchestra.run.smithi064.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-05-09T08:56:10.177 INFO:teuthology.orchestra.run.smithi139.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-05-09T08:56:10.238 INFO:teuthology.orchestra.run.smithi064.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-09T08:56:10.255 INFO:teuthology.orchestra.run.smithi064.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-05-09T08:56:10.280 INFO:teuthology.orchestra.run.smithi064.stdout:(115/156): libunwind-1.6.2-1.el9.x86_64.rpm 191 kB/s | 67 kB 00:00 2024-05-09T08:56:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 478 kB/s | 32 kB 00:00 2024-05-09T08:56:10.322 INFO:teuthology.orchestra.run.smithi064.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-05-09T08:56:10.381 INFO:teuthology.orchestra.run.smithi064.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-09T08:56:10.415 INFO:teuthology.orchestra.run.smithi064.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-09T08:56:10.457 INFO:teuthology.orchestra.run.smithi064.stdout:(120/156): python3-cheroot-8.6.0-4.el9.noarch.r 976 kB/s | 172 kB 00:00 2024-05-09T08:56:10.482 INFO:teuthology.orchestra.run.smithi064.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-05-09T08:56:10.507 INFO:teuthology.orchestra.run.smithi064.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-09T08:56:10.557 INFO:teuthology.orchestra.run.smithi064.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-09T08:56:10.583 INFO:teuthology.orchestra.run.smithi064.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-09T08:56:10.633 INFO:teuthology.orchestra.run.smithi064.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-09T08:56:10.658 INFO:teuthology.orchestra.run.smithi064.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-05-09T08:56:10.670 INFO:teuthology.orchestra.run.smithi179.stdout:(71/156): postgresql-server-13.14-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-05-09T08:56:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-05-09T08:56:10.734 INFO:teuthology.orchestra.run.smithi064.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-09T08:56:10.784 INFO:teuthology.orchestra.run.smithi064.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-05-09T08:56:10.810 INFO:teuthology.orchestra.run.smithi064.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-05-09T08:56:10.843 INFO:teuthology.orchestra.run.smithi064.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-09T08:56:10.852 INFO:teuthology.orchestra.run.smithi139.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-09T08:56:10.868 INFO:teuthology.orchestra.run.smithi064.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-05-09T08:56:10.877 INFO:teuthology.orchestra.run.smithi139.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 273 kB/s | 250 kB 00:00 2024-05-09T08:56:10.894 INFO:teuthology.orchestra.run.smithi064.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-05-09T08:56:10.919 INFO:teuthology.orchestra.run.smithi064.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T08:56:10.944 INFO:teuthology.orchestra.run.smithi064.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-09T08:56:10.969 INFO:teuthology.orchestra.run.smithi064.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T08:56:11.011 INFO:teuthology.orchestra.run.smithi139.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 266 kB/s | 222 kB 00:00 2024-05-09T08:56:11.011 INFO:teuthology.orchestra.run.smithi064.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-05-09T08:56:11.045 INFO:teuthology.orchestra.run.smithi064.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-09T08:56:11.070 INFO:teuthology.orchestra.run.smithi064.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T08:56:11.096 INFO:teuthology.orchestra.run.smithi064.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-05-09T08:56:11.120 INFO:teuthology.orchestra.run.smithi179.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 976 kB/s | 1.0 MB 00:01 2024-05-09T08:56:11.121 INFO:teuthology.orchestra.run.smithi064.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-09T08:56:11.146 INFO:teuthology.orchestra.run.smithi064.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-09T08:56:11.171 INFO:teuthology.orchestra.run.smithi064.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T08:56:11.197 INFO:teuthology.orchestra.run.smithi064.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T08:56:11.247 INFO:teuthology.orchestra.run.smithi064.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-09T08:56:11.304 INFO:teuthology.orchestra.run.smithi064.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.7 MB/s | 427 kB 00:00 2024-05-09T08:56:11.331 INFO:teuthology.orchestra.run.smithi064.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-09T08:56:11.373 INFO:teuthology.orchestra.run.smithi064.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi179.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 119 kB/s | 87 kB 00:00 2024-05-09T08:56:11.561 INFO:teuthology.orchestra.run.smithi139.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 62 kB/s | 42 kB 00:00 2024-05-09T08:56:11.582 INFO:teuthology.orchestra.run.smithi064.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-05-09T08:56:11.586 INFO:teuthology.orchestra.run.smithi139.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 105 kB/s | 77 kB 00:00 2024-05-09T08:56:11.657 INFO:teuthology.orchestra.run.smithi064.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-05-09T08:56:11.699 INFO:teuthology.orchestra.run.smithi064.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-09T08:56:11.749 INFO:teuthology.orchestra.run.smithi064.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-05-09T08:56:11.903 INFO:teuthology.orchestra.run.smithi139.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 302 kB/s | 269 kB 00:00 2024-05-09T08:56:11.949 INFO:teuthology.orchestra.run.smithi064.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 1.3 MB 00:00 2024-05-09T08:56:12.008 INFO:teuthology.orchestra.run.smithi064.stdout:(154/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-05-09T08:56:12.058 INFO:teuthology.orchestra.run.smithi064.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-05-09T08:56:12.254 INFO:teuthology.orchestra.run.smithi179.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 287 kB/s | 244 kB 00:00 2024-05-09T08:56:12.387 INFO:teuthology.orchestra.run.smithi139.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 192 kB/s | 159 kB 00:00 2024-05-09T08:56:12.392 INFO:teuthology.orchestra.run.smithi064.stdout:(156/156): librados2-19.0.0-2515.g96805dcf.el9. 3.1 MB/s | 3.4 MB 00:01 2024-05-09T08:56:12.398 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:56:12.398 INFO:teuthology.orchestra.run.smithi064.stdout:Total 9.3 MB/s | 225 MB 00:24 2024-05-09T08:56:12.430 INFO:teuthology.orchestra.run.smithi179.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-05-09T08:56:12.445 INFO:teuthology.orchestra.run.smithi139.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 325 kB/s | 279 kB 00:00 2024-05-09T08:56:12.545 INFO:teuthology.orchestra.run.smithi139.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-05-09T08:56:13.014 INFO:teuthology.orchestra.run.smithi179.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 26 kB 00:00 2024-05-09T08:56:13.071 INFO:teuthology.orchestra.run.smithi139.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 78 kB/s | 54 kB 00:00 2024-05-09T08:56:13.122 INFO:teuthology.orchestra.run.smithi179.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 286 kB/s | 248 kB 00:00 2024-05-09T08:56:13.213 INFO:teuthology.orchestra.run.smithi139.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 62 kB/s | 42 kB 00:00 2024-05-09T08:56:13.548 INFO:teuthology.orchestra.run.smithi179.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-05-09T08:56:13.645 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-05-09T08:56:13.713 INFO:teuthology.orchestra.run.smithi139.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-05-09T08:56:13.819 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-05-09T08:56:13.819 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-05-09T08:56:14.006 INFO:teuthology.orchestra.run.smithi179.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 195 kB/s | 172 kB 00:00 2024-05-09T08:56:14.222 INFO:teuthology.orchestra.run.smithi139.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.2 MB/s | 16 MB 00:01 2024-05-09T08:56:14.247 INFO:teuthology.orchestra.run.smithi139.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 214 kB/s | 221 kB 00:01 2024-05-09T08:56:14.273 INFO:teuthology.orchestra.run.smithi179.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 48 kB/s | 35 kB 00:00 2024-05-09T08:56:14.530 INFO:teuthology.orchestra.run.smithi139.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 44 kB/s | 36 kB 00:00 2024-05-09T08:56:14.881 INFO:teuthology.orchestra.run.smithi139.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-05-09T08:56:15.172 INFO:teuthology.orchestra.run.smithi139.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 329 kB/s | 305 kB 00:00 2024-05-09T08:56:15.181 INFO:teuthology.orchestra.run.smithi179.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 284 kB/s | 250 kB 00:00 2024-05-09T08:56:15.206 INFO:teuthology.orchestra.run.smithi139.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 86 kB/s | 58 kB 00:00 2024-05-09T08:56:15.507 INFO:teuthology.orchestra.run.smithi179.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.4 MB/s | 5.1 MB 00:01 2024-05-09T08:56:15.665 INFO:teuthology.orchestra.run.smithi139.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 120 kB/s | 94 kB 00:00 2024-05-09T08:56:15.790 INFO:teuthology.orchestra.run.smithi139.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-05-09T08:56:15.907 INFO:teuthology.orchestra.run.smithi139.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 64 kB/s | 45 kB 00:00 2024-05-09T08:56:15.932 INFO:teuthology.orchestra.run.smithi139.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 84 kB/s | 64 kB 00:00 2024-05-09T08:56:16.074 INFO:teuthology.orchestra.run.smithi139.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-05-09T08:56:16.107 INFO:teuthology.orchestra.run.smithi179.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 240 kB/s | 222 kB 00:00 2024-05-09T08:56:16.120 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-05-09T08:56:16.120 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-05-09T08:56:16.232 INFO:teuthology.orchestra.run.smithi179.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 111 kB/s | 77 kB 00:00 2024-05-09T08:56:16.366 INFO:teuthology.orchestra.run.smithi139.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-05-09T08:56:16.391 INFO:teuthology.orchestra.run.smithi179.stdout:(85/156): postgresql-contrib-13.14-1.el9.x86_64 126 kB/s | 911 kB 00:07 2024-05-09T08:56:16.391 INFO:teuthology.orchestra.run.smithi139.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 673 kB/s | 309 kB 00:00 2024-05-09T08:56:16.450 INFO:teuthology.orchestra.run.smithi139.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-09T08:56:16.475 INFO:teuthology.orchestra.run.smithi139.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-05-09T08:56:16.525 INFO:teuthology.orchestra.run.smithi139.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-05-09T08:56:16.559 INFO:teuthology.orchestra.run.smithi139.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-05-09T08:56:16.609 INFO:teuthology.orchestra.run.smithi139.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-09T08:56:16.716 INFO:teuthology.orchestra.run.smithi179.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 69 kB/s | 42 kB 00:00 2024-05-09T08:56:16.768 INFO:teuthology.orchestra.run.smithi139.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-05-09T08:56:16.810 INFO:teuthology.orchestra.run.smithi139.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-05-09T08:56:16.843 INFO:teuthology.orchestra.run.smithi139.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-05-09T08:56:16.877 INFO:teuthology.orchestra.run.smithi139.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 483 kB/s | 32 kB 00:00 2024-05-09T08:56:16.911 INFO:teuthology.orchestra.run.smithi139.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-09T08:56:16.987 INFO:teuthology.orchestra.run.smithi139.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-05-09T08:56:17.011 INFO:teuthology.orchestra.run.smithi139.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-09T08:56:17.036 INFO:teuthology.orchestra.run.smithi139.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-09T08:56:17.062 INFO:teuthology.orchestra.run.smithi139.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-09T08:56:17.087 INFO:teuthology.orchestra.run.smithi139.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-05-09T08:56:17.091 INFO:teuthology.orchestra.run.smithi179.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 313 kB/s | 269 kB 00:00 2024-05-09T08:56:17.121 INFO:teuthology.orchestra.run.smithi139.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-09T08:56:17.179 INFO:teuthology.orchestra.run.smithi139.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-05-09T08:56:17.183 INFO:teuthology.orchestra.run.smithi179.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 200 kB/s | 159 kB 00:00 2024-05-09T08:56:17.204 INFO:teuthology.orchestra.run.smithi139.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-09T08:56:17.230 INFO:teuthology.orchestra.run.smithi139.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-09T08:56:17.263 INFO:teuthology.orchestra.run.smithi139.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-05-09T08:56:17.289 INFO:teuthology.orchestra.run.smithi139.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-05-09T08:56:17.314 INFO:teuthology.orchestra.run.smithi139.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-09T08:56:17.339 INFO:teuthology.orchestra.run.smithi139.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-09T08:56:17.373 INFO:teuthology.orchestra.run.smithi139.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-09T08:56:17.398 INFO:teuthology.orchestra.run.smithi139.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 552 kB/s | 46 kB 00:00 2024-05-09T08:56:17.423 INFO:teuthology.orchestra.run.smithi139.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-05-09T08:56:17.449 INFO:teuthology.orchestra.run.smithi139.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-05-09T08:56:17.482 INFO:teuthology.orchestra.run.smithi139.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-09T08:56:17.507 INFO:teuthology.orchestra.run.smithi139.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-05-09T08:56:17.532 INFO:teuthology.orchestra.run.smithi139.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-05-09T08:56:17.558 INFO:teuthology.orchestra.run.smithi139.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 372 kB/s | 28 kB 00:00 2024-05-09T08:56:17.583 INFO:teuthology.orchestra.run.smithi139.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-05-09T08:56:17.600 INFO:teuthology.orchestra.run.smithi139.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-05-09T08:56:17.634 INFO:teuthology.orchestra.run.smithi139.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-09T08:56:17.650 INFO:teuthology.orchestra.run.smithi179.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 299 kB/s | 279 kB 00:00 2024-05-09T08:56:17.659 INFO:teuthology.orchestra.run.smithi139.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-09T08:56:17.684 INFO:teuthology.orchestra.run.smithi139.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-05-09T08:56:17.718 INFO:teuthology.orchestra.run.smithi139.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-09T08:56:17.743 INFO:teuthology.orchestra.run.smithi179.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-05-09T08:56:17.743 INFO:teuthology.orchestra.run.smithi139.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-09T08:56:17.777 INFO:teuthology.orchestra.run.smithi139.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-09T08:56:17.819 INFO:teuthology.orchestra.run.smithi139.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-05-09T08:56:17.844 INFO:teuthology.orchestra.run.smithi139.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-09T08:56:17.886 INFO:teuthology.orchestra.run.smithi139.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-09T08:56:17.901 INFO:teuthology.orchestra.run.smithi179.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 75 kB/s | 54 kB 00:00 2024-05-09T08:56:18.095 INFO:teuthology.orchestra.run.smithi139.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-05-09T08:56:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-05-09T08:56:18.279 INFO:teuthology.orchestra.run.smithi139.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 717 kB/s | 42 kB 00:00 2024-05-09T08:56:18.329 INFO:teuthology.orchestra.run.smithi139.stdout:(152/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-05-09T08:56:18.396 INFO:teuthology.orchestra.run.smithi139.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-05-09T08:56:18.427 INFO:teuthology.orchestra.run.smithi179.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 61 kB/s | 42 kB 00:00 2024-05-09T08:56:18.471 INFO:teuthology.orchestra.run.smithi139.stdout:(154/156): librados2-19.0.0-2515.g96805dcf.el9. 5.2 MB/s | 3.4 MB 00:00 2024-05-09T08:56:18.504 INFO:teuthology.orchestra.run.smithi139.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-05-09T08:56:18.568 INFO:teuthology.orchestra.run.smithi179.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-05-09T08:56:18.605 INFO:teuthology.orchestra.run.smithi139.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-09T08:56:18.612 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:56:18.612 INFO:teuthology.orchestra.run.smithi139.stdout:Total 8.1 MB/s | 225 MB 00:27 2024-05-09T08:56:19.261 INFO:teuthology.orchestra.run.smithi179.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-09T08:56:19.286 INFO:teuthology.orchestra.run.smithi179.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 50 kB/s | 36 kB 00:00 2024-05-09T08:56:19.311 INFO:teuthology.orchestra.run.smithi179.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 250 kB/s | 221 kB 00:00 2024-05-09T08:56:19.847 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-05-09T08:56:19.878 INFO:teuthology.orchestra.run.smithi179.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 77 kB/s | 47 kB 00:00 2024-05-09T08:56:19.995 INFO:teuthology.orchestra.run.smithi179.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 85 kB/s | 58 kB 00:00 2024-05-09T08:56:20.000 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-05-09T08:56:20.000 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-05-09T08:56:20.129 INFO:teuthology.orchestra.run.smithi179.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 362 kB/s | 305 kB 00:00 2024-05-09T08:56:20.671 INFO:teuthology.orchestra.run.smithi179.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 118 kB/s | 94 kB 00:00 2024-05-09T08:56:20.688 INFO:teuthology.orchestra.run.smithi179.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-05-09T08:56:20.755 INFO:teuthology.orchestra.run.smithi179.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-05-09T08:56:20.855 INFO:teuthology.orchestra.run.smithi179.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-05-09T08:56:20.897 INFO:teuthology.orchestra.run.smithi179.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-05-09T08:56:20.981 INFO:teuthology.orchestra.run.smithi179.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-09T08:56:21.098 INFO:teuthology.orchestra.run.smithi179.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-05-09T08:56:21.131 INFO:teuthology.orchestra.run.smithi179.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 806 kB/s | 26 kB 00:00 2024-05-09T08:56:21.165 INFO:teuthology.orchestra.run.smithi179.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-05-09T08:56:21.199 INFO:teuthology.orchestra.run.smithi179.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-05-09T08:56:21.232 INFO:teuthology.orchestra.run.smithi179.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-05-09T08:56:21.283 INFO:teuthology.orchestra.run.smithi179.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-05-09T08:56:21.308 INFO:teuthology.orchestra.run.smithi179.stdout:(112/156): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-05-09T08:56:21.333 INFO:teuthology.orchestra.run.smithi179.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-05-09T08:56:21.367 INFO:teuthology.orchestra.run.smithi179.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 967 kB/s | 32 kB 00:00 2024-05-09T08:56:21.392 INFO:teuthology.orchestra.run.smithi179.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 517 kB/s | 43 kB 00:00 2024-05-09T08:56:21.418 INFO:teuthology.orchestra.run.smithi179.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-09T08:56:21.435 INFO:teuthology.orchestra.run.smithi179.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-05-09T08:56:21.468 INFO:teuthology.orchestra.run.smithi179.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-05-09T08:56:21.502 INFO:teuthology.orchestra.run.smithi179.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T08:56:21.528 INFO:teuthology.orchestra.run.smithi179.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 598 kB/s | 35 kB 00:00 2024-05-09T08:56:21.553 INFO:teuthology.orchestra.run.smithi179.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-09T08:56:21.604 INFO:teuthology.orchestra.run.smithi179.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-05-09T08:56:21.629 INFO:teuthology.orchestra.run.smithi179.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-09T08:56:21.646 INFO:teuthology.orchestra.run.smithi179.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-05-09T08:56:21.663 INFO:teuthology.orchestra.run.smithi179.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-05-09T08:56:21.680 INFO:teuthology.orchestra.run.smithi179.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2024-05-09T08:56:21.697 INFO:teuthology.orchestra.run.smithi179.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 578 kB/s | 19 kB 00:00 2024-05-09T08:56:21.714 INFO:teuthology.orchestra.run.smithi179.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-05-09T08:56:21.765 INFO:teuthology.orchestra.run.smithi179.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-05-09T08:56:21.789 INFO:teuthology.orchestra.run.smithi179.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-05-09T08:56:21.832 INFO:teuthology.orchestra.run.smithi179.stdout:(131/156): protobuf-compiler-3.14.0-13.el9.x86_ 755 kB/s | 863 kB 00:01 2024-05-09T08:56:21.857 INFO:teuthology.orchestra.run.smithi179.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 854 kB/s | 79 kB 00:00 2024-05-09T08:56:21.899 INFO:teuthology.orchestra.run.smithi179.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-05-09T08:56:21.924 INFO:teuthology.orchestra.run.smithi179.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-09T08:56:21.950 INFO:teuthology.orchestra.run.smithi179.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-05-09T08:56:21.975 INFO:teuthology.orchestra.run.smithi179.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-05-09T08:56:22.000 INFO:teuthology.orchestra.run.smithi179.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-05-09T08:56:22.026 INFO:teuthology.orchestra.run.smithi179.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-05-09T08:56:22.051 INFO:teuthology.orchestra.run.smithi179.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-05-09T08:56:22.076 INFO:teuthology.orchestra.run.smithi179.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T08:56:22.110 INFO:teuthology.orchestra.run.smithi179.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-05-09T08:56:22.135 INFO:teuthology.orchestra.run.smithi179.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-05-09T08:56:22.161 INFO:teuthology.orchestra.run.smithi179.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-09T08:56:22.161 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-05-09T08:56:22.161 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-05-09T08:56:22.194 INFO:teuthology.orchestra.run.smithi179.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-09T08:56:22.220 INFO:teuthology.orchestra.run.smithi179.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 339 kB/s | 20 kB 00:00 2024-05-09T08:56:22.245 INFO:teuthology.orchestra.run.smithi179.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-09T08:56:22.270 INFO:teuthology.orchestra.run.smithi179.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-09T08:56:22.446 INFO:teuthology.orchestra.run.smithi179.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-05-09T08:56:22.646 INFO:teuthology.orchestra.run.smithi179.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 8.5 MB/s | 3.4 MB 00:00 2024-05-09T08:56:22.771 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-05-09T08:56:22.847 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:56:22.891 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:56:22.966 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:56:23.006 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:56:23.031 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:56:23.084 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:56:23.138 INFO:teuthology.orchestra.run.smithi179.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-05-09T08:56:23.439 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:56:23.540 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:56:23.555 INFO:teuthology.orchestra.run.smithi179.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-05-09T08:56:23.673 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:56:23.772 INFO:teuthology.orchestra.run.smithi179.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.6 MB/s | 1.8 MB 00:01 2024-05-09T08:56:23.797 INFO:teuthology.orchestra.run.smithi179.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 63 kB/s | 42 kB 00:00 2024-05-09T08:56:23.816 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:56:23.909 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:56:23.937 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:56:24.137 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:56:24.199 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:56:24.228 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:56:24.334 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:56:24.363 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:56:24.418 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:56:24.431 INFO:teuthology.orchestra.run.smithi179.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 284 kB/s | 248 kB 00:00 2024-05-09T08:56:24.447 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:56:24.467 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:56:24.642 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:56:24.811 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:56:24.890 INFO:teuthology.orchestra.run.smithi179.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-09T08:56:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:56:25.030 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:56:25.081 INFO:teuthology.orchestra.run.smithi179.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-05-09T08:56:25.086 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:56:25.086 INFO:teuthology.orchestra.run.smithi179.stdout:Total 6.0 MB/s | 225 MB 00:37 2024-05-09T08:56:25.162 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:25.441 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:25.559 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:56:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:56:25.704 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:56:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:56:25.824 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:56:25.863 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:56:25.906 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:56:25.939 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:56:25.966 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:56:26.048 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:56:26.160 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:56:26.205 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:56:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:56:26.307 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-05-09T08:56:26.448 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:56:26.458 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-05-09T08:56:26.458 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-05-09T08:56:26.492 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:56:26.520 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:56:26.597 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:56:26.644 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:56:26.722 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:56:26.872 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:56:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:56:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:56:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:56:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:56:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:56:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:56:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:56:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:56:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:56:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:56:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:56:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:56:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:56:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:56:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:56:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:56:28.399 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-05-09T08:56:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:56:28.485 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:56:28.523 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:56:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:56:28.595 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:56:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:56:28.634 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:56:28.644 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-05-09T08:56:28.645 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-05-09T08:56:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:56:28.660 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:56:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:56:28.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:56:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:56:28.937 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:56:28.971 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:56:29.002 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:56:29.027 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:56:29.048 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:56:29.120 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:56:29.152 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:56:29.253 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:56:29.274 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:56:29.292 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:56:29.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:56:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:56:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:56:29.388 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:56:29.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:56:29.416 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:56:29.434 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:56:29.464 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:56:29.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:56:29.532 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:56:29.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:56:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:56:29.608 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:56:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:56:29.683 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:56:29.699 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:56:29.753 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:56:29.787 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:56:29.802 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:56:29.809 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:56:29.851 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:56:29.872 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:56:29.902 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:56:29.922 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:56:29.964 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:56:30.023 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:56:30.112 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:56:30.301 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:56:30.449 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:56:30.509 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:56:30.626 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:30.702 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:30.819 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:56:30.868 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:56:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:56:30.907 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:31.010 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:56:31.025 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:56:31.069 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:56:31.119 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:56:31.157 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:56:31.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:56:31.192 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:56:31.203 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:56:31.292 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:56:31.333 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:56:31.376 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:56:31.388 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:56:31.410 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:56:31.417 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:56:31.437 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:56:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:56:31.506 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:56:31.559 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:56:31.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:56:31.633 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:56:31.709 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:56:31.743 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:56:31.788 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:56:31.816 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:56:31.886 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:56:31.930 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:56:32.004 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:56:32.144 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:56:32.199 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:56:32.205 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:56:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:56:32.730 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:56:32.813 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:56:32.841 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:56:32.884 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:56:32.921 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:56:32.964 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:56:33.112 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:56:33.166 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:56:33.221 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:56:33.292 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:56:33.324 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:56:33.361 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:56:33.408 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:56:33.452 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:56:33.495 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:56:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:56:33.673 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:56:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:56:33.726 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:56:33.756 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:56:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:56:33.797 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:56:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:56:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:56:33.893 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:56:33.940 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:56:33.987 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:56:34.021 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:56:34.046 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:56:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:56:34.165 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:56:34.211 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:56:34.257 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:56:34.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:56:34.303 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:56:34.326 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:56:34.333 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:56:34.357 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:56:34.370 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:56:34.396 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:56:34.428 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:56:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:56:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:56:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:56:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:56:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:56:34.680 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:56:34.723 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:56:34.755 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:56:34.802 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:56:34.831 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-05-09T08:56:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:56:34.901 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:56:34.906 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:56:34.912 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:56:34.949 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:56:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:56:34.961 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:56:34.991 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:56:35.020 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:56:35.054 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:56:35.080 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:56:35.128 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:56:35.468 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:56:35.573 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:56:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:56:35.687 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:56:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:56:35.724 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:56:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:56:35.808 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:56:35.837 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:56:35.860 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:56:36.007 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:56:36.025 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:56:36.055 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:56:36.077 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:56:36.208 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:56:36.261 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:56:36.293 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:56:36.315 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:56:36.336 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:56:36.348 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:56:36.535 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:56:36.709 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:56:36.847 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:56:36.908 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:56:37.000 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:56:37.014 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:37.101 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:37.126 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:56:37.295 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:56:37.417 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:56:37.419 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:37.453 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:37.501 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:56:37.559 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:56:37.601 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:56:37.674 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:56:37.710 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:56:37.752 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:56:37.785 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:56:37.812 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:56:37.881 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:56:37.960 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:56:37.991 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:56:38.035 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:56:38.055 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:56:38.113 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:56:38.173 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:56:38.173 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:56:38.173 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:56:38.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:38.190 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:56:38.208 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:56:38.218 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:56:38.287 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:56:38.333 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:56:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:56:38.407 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:56:38.481 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:56:38.508 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:56:38.533 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:56:38.548 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:56:38.603 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:56:38.846 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:56:38.888 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:56:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:56:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:56:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:56:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:56:39.199 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:56:39.228 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:56:39.272 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:56:39.309 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:56:39.351 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:56:39.502 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:56:39.519 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:56:39.549 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:56:39.555 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:56:39.609 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:56:39.620 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:56:39.651 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:56:39.682 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:56:39.714 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:56:39.751 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:56:39.798 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:56:39.842 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:56:39.886 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:56:39.927 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:56:40.063 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:56:40.109 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:56:40.140 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:56:40.181 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:56:40.307 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:56:40.349 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:56:40.381 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:56:40.406 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:56:40.483 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:56:40.607 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:56:40.639 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:56:40.670 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:56:40.702 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:56:40.741 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:56:40.773 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:56:40.803 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:56:40.837 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:56:40.871 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:56:40.912 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:56:40.947 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:56:41.034 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:56:41.090 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:56:41.126 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:56:41.173 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:56:41.271 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:56:41.320 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:56:41.927 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:42.040 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:56:42.109 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:56:42.143 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:56:42.301 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:56:42.388 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:56:42.421 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:56:42.535 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:56:42.597 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:56:42.627 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:56:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:56:42.674 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:56:42.674 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:56:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:56:42.708 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:56:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:56:43.362 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:56:43.485 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:56:44.742 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:56:45.210 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:56:45.251 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:56:45.342 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:56:45.373 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:56:45.881 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:56:45.913 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:56:48.308 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:48.348 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:48.896 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:56:48.943 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:56:49.036 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:56:49.036 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:56:49.037 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:56:49.037 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:56:49.069 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-05-09T08:56:50.296 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-05-09T08:56:50.297 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-05-09T08:56:50.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:50.471 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:56:50.489 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:56:50.489 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:50.489 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:56:50.489 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:52.016 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:56:52.045 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:56:52.045 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:52.045 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:56:52.045 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:56:52.045 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:56:52.045 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:52.283 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:56:52.308 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:56:52.308 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:52.309 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:56:52.309 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:56:52.309 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:56:52.309 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:52.784 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:56:52.809 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:56:52.809 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:52.810 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:56:52.810 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:56:52.810 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:56:52.810 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:52.915 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:56:52.965 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:56:53.029 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:56:53.227 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:56:53.279 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:56:53.362 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:56:53.603 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:56:53.654 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:56:53.695 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:56:53.721 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:56:54.174 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:56:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:56:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:56:54.458 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:54.458 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:56:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:56:55.428 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:56:55.469 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:56:55.963 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:56:55.963 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:55.964 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:56:55.964 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:56:55.964 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:56:55.964 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:56:56.178 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:56:56.202 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:56:56.202 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:56:56.202 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:56:56.489 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:56:56.519 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:56:56.629 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:56:56.654 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:56:56.677 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:56:56.678 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:56.678 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:56:56.678 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:56:56.678 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:56:56.678 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:56:56.792 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:56:56.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:56:56.830 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:56:56.840 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:56:56.855 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:56:56.856 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:56.856 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:56:56.856 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:56:56.856 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:56:56.856 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:56.904 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:56:56.905 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:56:56.918 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:56:57.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:56:57.079 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:56:57.089 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:56:57.138 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:56:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:56:57.303 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:56:57.441 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:56:57.487 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:56:57.522 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:56:57.547 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:56:58.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:56:58.030 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:56:58.050 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:56:58.062 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:56:58.075 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:56:58.075 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:58.075 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:56:58.075 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:56:58.075 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:56:58.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:58.093 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:56:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:56:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:56:58.141 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:58.141 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:56:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:58.155 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:56:58.185 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:56:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:56:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:56:58.218 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:58.218 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:56:58.218 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:56:58.218 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:56:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:56:59.219 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:56:59.261 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:57:00.224 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:57:00.254 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:57:00.356 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:57:00.524 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:57:00.556 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:00.631 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:57:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:57:00.817 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:57:00.835 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:57:00.835 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:00.835 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:57:00.835 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:01.693 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:57:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:57:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:57:01.808 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:01.808 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:57:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:57:01.885 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:57:01.885 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:01.885 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:57:01.885 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:57:01.885 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:57:01.885 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:02.186 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:57:02.210 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:57:02.210 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:02.210 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:57:02.210 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:57:02.210 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:57:02.210 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:02.320 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:57:02.347 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:57:02.347 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:02.347 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:57:02.347 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:57:02.347 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:57:02.347 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:02.579 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:57:02.602 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:57:02.602 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:02.603 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:57:02.603 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:57:02.603 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:57:02.603 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:03.054 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:57:03.078 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:57:03.078 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:03.078 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:57:03.078 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:57:03.078 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:57:03.078 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:03.176 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:57:03.225 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:57:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:57:03.473 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:57:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:57:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:57:03.521 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:57:03.593 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:57:03.689 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:57:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:57:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:57:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:57:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:57:03.829 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:57:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:57:03.881 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:57:03.916 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:57:03.941 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:57:04.380 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:57:04.411 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:57:04.459 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:04.473 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:57:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:04.503 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:57:04.519 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:05.444 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:57:05.525 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:57:05.534 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:57:05.581 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:57:05.588 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:57:05.671 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:57:05.704 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:57:05.730 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:57:05.730 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:57:05.806 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:57:05.839 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:57:05.839 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:57:05.924 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:57:05.948 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:57:05.954 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:05.954 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:06.159 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:06.159 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:57:06.243 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:57:06.312 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:06.546 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:57:06.576 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:57:06.676 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:57:06.722 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:57:06.764 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:57:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:57:06.842 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:57:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:57:06.872 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:57:06.874 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:57:06.898 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:57:06.898 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:06.898 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:57:06.898 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:57:06.899 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:57:06.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:06.950 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:57:06.962 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:57:07.059 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:57:07.064 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:57:07.110 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:57:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:57:07.140 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:57:07.170 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:57:07.207 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:08.003 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:57:08.036 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:57:08.059 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:57:08.059 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:08.059 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:57:08.059 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:57:08.059 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:57:08.060 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:08.103 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:57:08.126 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:57:08.126 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:08.126 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:57:08.126 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:08.170 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:57:08.194 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:57:08.194 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:08.194 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:57:08.194 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:57:08.194 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:57:08.194 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:08.504 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:57:08.527 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:57:08.527 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:57:08.527 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:57:08.527 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:57:08.528 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:57:08.528 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:08.723 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:57:08.810 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:57:08.859 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:57:09.099 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:57:09.134 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:57:09.134 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:57:09.199 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:57:09.224 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:57:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:09.409 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:09.410 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:57:09.567 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:11.547 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:57:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:57:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:57:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:57:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:57:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:57:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:57:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:57:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:57:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:57:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:57:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:57:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:57:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:57:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:57:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:57:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:57:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:57:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:57:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:57:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:57:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:57:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:57:13.041 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:57:13.082 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:57:13.123 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:57:13.155 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:57:13.379 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:57:13.429 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:57:13.451 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:57:13.489 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:57:13.527 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:57:14.131 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:14.177 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:14.195 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:57:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:57:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:57:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:57:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:57:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:57:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:57:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:57:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:57:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:57:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:57:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:57:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:57:14.264 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-09T08:57:14.707 DEBUG:teuthology.parallel:result is None 2024-05-09T08:57:15.029 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:57:15.118 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:15.118 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:57:15.119 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:57:15.120 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:57:15.121 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:57:15.133 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:57:15.133 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:57:15.133 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:57:15.133 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:57:15.133 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:57:15.133 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:57:15.134 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:57:15.135 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:57:15.136 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:57:15.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:57:15.138 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:57:15.139 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:57:15.140 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:57:15.141 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:57:15.142 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:57:15.170 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:57:15.262 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:57:15.409 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:57:15.451 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:57:15.451 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:57:15.535 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:57:15.560 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:57:15.566 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:15.566 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:15.745 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:57:15.745 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:57:15.902 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.088 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:18.089 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.090 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:57:18.091 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:57:18.092 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:57:18.093 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:57:18.094 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:57:18.095 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:57:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:57:18.097 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:57:18.098 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:57:18.098 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-05-09T08:57:18.098 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-05-09T08:57:18.529 DEBUG:teuthology.parallel:result is None 2024-05-09T08:57:21.522 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:57:21.523 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:57:21.524 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:57:21.525 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:57:21.536 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:57:21.537 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:57:21.538 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:57:21.539 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:57:21.540 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:57:21.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:57:21.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:57:21.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:57:21.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:57:21.545 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:57:21.546 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:57:24.125 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:57:24.125 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:24.125 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-05-09T08:57:24.125 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:24.125 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:24.125 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:57:24.128 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:57:24.129 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:57:24.130 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:57:24.131 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:57:24.132 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:57:24.133 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:57:24.134 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:57:24.135 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:57:24.136 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.137 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:57:24.138 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:57:24.139 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:57:24.139 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T08:57:24.139 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-05-09T08:57:24.575 DEBUG:teuthology.parallel:result is None 2024-05-09T08:57:24.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:57:24.908 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:57:24.937 INFO:teuthology.orchestra.run.smithi064.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:57:24.938 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:57:24.938 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:57:24.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:57:25.094 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:57:25.122 INFO:teuthology.orchestra.run.smithi139.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:57:25.122 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:57:25.122 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:57:25.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:57:25.351 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:57:25.377 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:57:25.378 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:57:25.378 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:57:25.381 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T08:57:25.381 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:57:25.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:57:25.417 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:57:25.417 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:57:25.451 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:57:25.452 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:57:25.481 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T08:57:25.481 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:57:25.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:57:25.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:57:25.584 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:57:25.584 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:57:25.614 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:57:25.684 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:57:25.684 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:57:25.712 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:57:25.779 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T08:57:25.779 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:57:25.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:57:25.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:57:25.878 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:57:25.878 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:57:25.907 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:57:25.974 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:57:25.974 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:57:26.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:57:26.068 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T08:57:26.068 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:57:26.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:57:26.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:57:26.161 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:57:26.161 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:57:26.186 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:57:26.252 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:57:26.252 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:57:26.279 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:57:26.345 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T08:57:26.447 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-09T08:57:26.447 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:57:26.448 INFO:tasks.cephadm:Cluster fsid is 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:57:26.448 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T08:57:26.448 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.b': '172.21.15.139', 'mon.c': '172.21.15.179'} 2024-05-09T08:57:26.448 INFO:tasks.cephadm:First mon is mon.a on smithi064 2024-05-09T08:57:26.448 INFO:tasks.cephadm:First mgr is x 2024-05-09T08:57:26.448 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T08:57:26.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-05-09T08:57:26.475 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-05-09T08:57:26.501 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-05-09T08:57:26.528 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T08:57:26.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:57:26.673 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55097/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2515-g96805dcf', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2515.g96805dcf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 19:00:25.721799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-09T08:57:27.055 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:57:27.056 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm 2024-05-09T08:57:27.057 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:28.146 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 08:57 /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:28.146 DEBUG:teuthology.orchestra.run.smithi139:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 08:57 /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:28.597 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:29.320 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 08:57 /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:29.320 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:29.342 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:29.362 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T08:57:29.398 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc on all hosts... 2024-05-09T08:57:29.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T08:57:29.401 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T08:57:29.404 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T08:57:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:57:31.100 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:57:31.173 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:58:04.042 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-05-09T08:58:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:58:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T08:58:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-05-09T08:58:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T08:58:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-05-09T08:58:04.043 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-05-09T08:58:16.841 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T08:58:16.841 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:58:16.841 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T08:58:16.842 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-05-09T08:58:16.842 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T08:58:16.842 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T08:58:16.842 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T08:58:22.599 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-09T08:58:22.600 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:58:22.600 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T08:58:22.600 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-05-09T08:58:22.600 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T08:58:22.600 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-09T08:58:22.600 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-09T08:58:22.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-05-09T08:58:22.659 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-05-09T08:58:22.692 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-05-09T08:58:22.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-05-09T08:58:22.752 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-05-09T08:58:22.781 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-05-09T08:58:22.805 INFO:tasks.cephadm:Writing seed config... 2024-05-09T08:58:22.806 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T08:58:22.807 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T08:58:22.808 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T08:58:22.809 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T08:58:22.810 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-09T08:58:22.810 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:58:22.810 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T08:58:22.828 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2877b902-0de2-11ef-bc98-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T08:58:22.829 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a.service 2024-05-09T08:58:22.871 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x.service 2024-05-09T08:58:22.914 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T08:58:22.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -v bootstrap --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:58:23.117 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:58:23.117 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', '-v', 'bootstrap', '--fsid', '2877b902-0de2-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-05-09T08:58:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T08:58:23.118 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2024-05-09T08:58:23.143 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T08:58:23.144 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2024-05-09T08:58:23.144 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2024-05-09T08:58:23.152 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T08:58:23.152 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T08:58:23.158 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T08:58:23.158 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-09T08:58:23.165 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-05-09T08:58:23.172 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-05-09T08:58:23.172 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-05-09T08:58:23.172 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2024-05-09T08:58:23.204 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T08:58:23.204 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T08:58:23.204 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2024-05-09T08:58:23.204 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2024-05-09T08:58:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T08:58:23.211 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T08:58:23.217 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T08:58:23.217 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-09T08:58:23.224 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-05-09T08:58:23.231 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-05-09T08:58:23.231 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-05-09T08:58:23.231 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2024-05-09T08:58:23.231 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:23.231 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 140601212714960 on /run/cephadm/2877b902-0de2-11ef-bc98-c7b262605968.lock 2024-05-09T08:58:23.231 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 140601212714960 acquired on /run/cephadm/2877b902-0de2-11ef-bc98-c7b262605968.lock 2024-05-09T08:58:23.232 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2024-05-09T08:58:23.232 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2024-05-09T08:58:23.232 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-05-09T08:58:23.235 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2024-05-09T08:58:23.236 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-05-09T08:58:23.238 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T08:58:23.238 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-09T08:58:23.239 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1750sec hoplimit 64 pref medium 2024-05-09T08:58:23.241 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T08:58:23.241 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T08:58:23.241 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T08:58:23.241 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-09T08:58:23.241 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-05-09T08:58:23.241 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T08:58:23.242 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-05-09T08:58:23.242 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-05-09T08:58:23.242 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T08:58:23.242 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T08:58:23.243 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:9852312f87792557815c3e860a5c4f8cbf2b59bbbde9b5f86ab07b438f905b2d 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 2024-05-09T08:58:23.695 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T08:58:24.643 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev) 2024-05-09T08:58:24.643 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev) 2024-05-09T08:58:24.643 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2024-05-09T08:58:26.332 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQAxkDxm0IsQMBAAThiplSaYPznpuBiS59A0jg== 2024-05-09T08:58:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQAykDxmULbpJxAA4FsH+r7cNFp7kJ1jBWFVzg== 2024-05-09T08:58:28.118 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQAzkDxmGYtJHxAAKDJKaKICjEhnuH7ucMTl5g== 2024-05-09T08:58:28.119 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = quincy 2024-05-09T08:58:28.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:28.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T08:58:28.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:58:28.900 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2024-05-09T08:58:29.797 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2024-05-09T08:58:30.022 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T08:58:30.192 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T08:58:30.378 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2877b902-0de2-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-2877b902-0de2-11ef-bc98-c7b262605968.target. 2024-05-09T08:58:30.378 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2877b902-0de2-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-2877b902-0de2-11ef-bc98-c7b262605968.target. 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a 2024-05-09T08:58:30.628 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-09T08:58:30.962 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2877b902-0de2-11ef-bc98-c7b262605968.target.wants/ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2877b902-0de2-11ef-bc98-c7b262605968@.service. 2024-05-09T08:58:31.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 podman[29755]: 2024-05-09 08:58:31.119215419 +0000 UTC m=+0.019361023 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:58:31.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 podman[29755]: 2024-05-09 08:58:31.25638273 +0000 UTC m=+0.156528320 container create 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T08:58:31.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 podman[29755]: 2024-05-09 08:58:31.459036709 +0000 UTC m=+0.359182305 container init 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T08:58:31.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 podman[29755]: 2024-05-09 08:58:31.462692146 +0000 UTC m=+0.362837738 container start 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T08:58:31.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 ceph-mon[29775]: mkfs 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:31.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 ceph-mon[29775]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T08:58:31.548 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:58:31.548 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-09T08:58:31.555 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:58:31.556 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-09T08:58:31.556 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-09T08:58:31.556 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T08:58:31.556 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2024-05-09T08:58:31.556 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2024-05-09T08:58:31.760 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 bash[29755]: 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 2024-05-09T08:58:31.760 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:31 smithi064 systemd[1]: Started Ceph mon.a for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T08:58:32.598 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T08:58:32.598 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:32.598 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T08:58:32.598 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.552657s) 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.972+0000 7fa33b562700 1 Processor -- start 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.972+0000 7fa33b562700 1 -- start start 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.973+0000 7fa33b562700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33410a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.599 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.973+0000 7fa33b562700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3340757c0 con 0x7fa3341085a0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.973+0000 7fa3392fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33410a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.973+0000 7fa3392fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33410a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59484/0 (socket says 172.21.15.64:59484) 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.973+0000 7fa3392fe700 1 -- 172.21.15.64:0/1734977942 learned_addr learned my addr 172.21.15.64:0/1734977942 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.973+0000 7fa3392fe700 1 -- 172.21.15.64:0/1734977942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33410aed0 con 0x7fa3341085a0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa3392fe700 1 --2- 172.21.15.64:0/1734977942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33410a990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa3300098a0 tx=0x7fa330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27431c6930f09cec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa32bfff700 1 -- 172.21.15.64:0/1734977942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa330004160 con 0x7fa3341085a0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa32bfff700 1 -- 172.21.15.64:0/1734977942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa330004300 con 0x7fa3341085a0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa32bfff700 1 -- 172.21.15.64:0/1734977942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3300036a0 con 0x7fa3341085a0 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 -- 172.21.15.64:0/1734977942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 msgr2=0x7fa33410a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 --2- 172.21.15.64:0/1734977942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33410a990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa3300098a0 tx=0x7fa330009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 -- 172.21.15.64:0/1734977942 shutdown_connections 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 --2- 172.21.15.64:0/1734977942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33410a990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.600 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 -- 172.21.15.64:0/1734977942 >> 172.21.15.64:0/1734977942 conn(0x7fa334101f50 msgr2=0x7fa334104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 -- 172.21.15.64:0/1734977942 shutdown_connections 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.974+0000 7fa33b562700 1 -- 172.21.15.64:0/1734977942 wait complete. 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa33b562700 1 Processor -- start 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa33b562700 1 -- start start 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa33b562700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa33b562700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410b2e0 con 0x7fa3341085a0 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa3392fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa3392fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33419d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59496/0 (socket says 172.21.15.64:59496) 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.975+0000 7fa3392fe700 1 -- 172.21.15.64:0/2765338669 learned_addr learned my addr 172.21.15.64:0/2765338669 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa3392fe700 1 -- 172.21.15.64:0/2765338669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33419d8b0 con 0x7fa3341085a0 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa3392fe700 1 --2- 172.21.15.64:0/2765338669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33419d370 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa330000c00 tx=0x7fa330004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa330003db0 con 0x7fa3341085a0 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa330011630 con 0x7fa3341085a0 2024-05-09T08:58:32.601 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33419db40 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa330011950 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a0830 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa330011af0 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.976+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa33001c450 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.977+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa334063980 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:31.980+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa330032020 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.053+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa33402dd00 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.053+0000 7fa32a7fc700 1 -- 172.21.15.64:0/2765338669 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa330004260 con 0x7fa3341085a0 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 msgr2=0x7fa33419d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 --2- 172.21.15.64:0/2765338669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33419d370 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa330000c00 tx=0x7fa330004750 comp rx=0 tx=0).stop 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 shutdown_connections 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 --2- 172.21.15.64:0/2765338669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3341085a0 0x7fa33419d370 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 >> 172.21.15.64:0/2765338669 conn(0x7fa334101f50 msgr2=0x7fa334102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 shutdown_connections 2024-05-09T08:58:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:32.054+0000 7fa33b562700 1 -- 172.21.15.64:0/2765338669 wait complete. 2024-05-09T08:58:32.603 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2024-05-09T08:58:32.603 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: monmap epoch 1 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: last_changed 2024-05-09T08:58:28.429468+0000 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: min_mon_release 19 (squid) 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: election_strategy: 1 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: fsmap 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: mgrmap e1: no daemons active 2024-05-09T08:58:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:32 smithi064 ceph-mon[29775]: from='client.? 172.21.15.64:0/2765338669' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T08:58:33.754 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:33 smithi064 ceph-mon[29775]: from='client.? 172.21.15.64:0/2830769313' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T08:58:33.754 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:33 smithi064 ceph-mon[29775]: from='client.? 172.21.15.64:0/2830769313' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T08:58:33.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.075+0000 7f3ca640b700 1 Processor -- start 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.075+0000 7f3ca640b700 1 -- start start 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.075+0000 7f3ca640b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca010a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.075+0000 7f3ca640b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca007be90 con 0x7f3ca0108520 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca010a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca010a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59498/0 (socket says 172.21.15.64:59498) 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9ffff700 1 -- 172.21.15.64:0/3621612759 learned_addr learned my addr 172.21.15.64:0/3621612759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9ffff700 1 -- 172.21.15.64:0/3621612759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca010ae50 con 0x7f3ca0108520 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9ffff700 1 --2- 172.21.15.64:0/3621612759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca010a910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3c900098a0 tx=0x7f3c90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fe2d9d18d34c33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9f7fe700 1 -- 172.21.15.64:0/3621612759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90004160 con 0x7f3ca0108520 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9f7fe700 1 -- 172.21.15.64:0/3621612759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c90004300 con 0x7f3ca0108520 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.076+0000 7f3c9f7fe700 1 -- 172.21.15.64:0/3621612759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c900036a0 con 0x7f3ca0108520 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 -- 172.21.15.64:0/3621612759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 msgr2=0x7f3ca010a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 --2- 172.21.15.64:0/3621612759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca010a910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3c900098a0 tx=0x7f3c90009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 -- 172.21.15.64:0/3621612759 shutdown_connections 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 --2- 172.21.15.64:0/3621612759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca010a910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 -- 172.21.15.64:0/3621612759 >> 172.21.15.64:0/3621612759 conn(0x7f3ca0101f50 msgr2=0x7f3ca0104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 -- 172.21.15.64:0/3621612759 shutdown_connections 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.077+0000 7f3ca640b700 1 -- 172.21.15.64:0/3621612759 wait complete. 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3ca640b700 1 Processor -- start 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3ca640b700 1 -- start start 2024-05-09T08:58:33.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3ca640b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3ca640b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010b2c0 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3c9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59512/0 (socket says 172.21.15.64:59512) 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3c9ffff700 1 -- 172.21.15.64:0/2830769313 learned_addr learned my addr 172.21.15.64:0/2830769313 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3c9ffff700 1 -- 172.21.15.64:0/2830769313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca019b660 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.078+0000 7f3c9ffff700 1 --2- 172.21.15.64:0/2830769313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca019b120 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c90000c00 tx=0x7f3c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90003db0 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c90011630 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90011950 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca019b8f0 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca019e5e0 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c90011af0 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.079+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c9001c450 con 0x7f3ca0108520 2024-05-09T08:58:33.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.080+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca0063980 con 0x7f3ca0108520 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.082+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3c90032020 con 0x7f3ca0108520 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.152+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3ca002dd00 con 0x7f3ca0108520 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.155+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f3c90004260 con 0x7f3ca0108520 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.156+0000 7f3c9dffb700 1 -- 172.21.15.64:0/2830769313 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3c9001b450 con 0x7f3ca0108520 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.156+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 msgr2=0x7f3ca019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.156+0000 7f3ca640b700 1 --2- 172.21.15.64:0/2830769313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca019b120 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c90000c00 tx=0x7f3c90004750 comp rx=0 tx=0).stop 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.156+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 shutdown_connections 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.156+0000 7f3ca640b700 1 --2- 172.21.15.64:0/2830769313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca0108520 0x7f3ca019b120 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.156+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 >> 172.21.15.64:0/2830769313 conn(0x7f3ca0101f50 msgr2=0x7f3ca0104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.157+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 shutdown_connections 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:33.157+0000 7f3ca640b700 1 -- 172.21.15.64:0/2830769313 wait complete. 2024-05-09T08:58:33.767 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2024-05-09T08:58:34.852 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:34 smithi064 ceph-mon[29775]: from='client.? 172.21.15.64:0/2398266268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2cefac700 1 Processor -- start 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2cefac700 1 -- start start 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2cefac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2cefac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c807c860 con 0x7fb2c807de20 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2ccd48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2ccd48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59514/0 (socket says 172.21.15.64:59514) 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2ccd48700 1 -- 172.21.15.64:0/8019320 learned_addr learned my addr 172.21.15.64:0/8019320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2ccd48700 1 -- 172.21.15.64:0/8019320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c807c9e0 con 0x7fb2c807de20 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.204+0000 7fb2ccd48700 1 --2- 172.21.15.64:0/8019320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c807c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0098a0 tx=0x7fb2bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c2d169857a4077 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2c77fe700 1 -- 172.21.15.64:0/8019320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2bc004160 con 0x7fb2c807de20 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2c77fe700 1 -- 172.21.15.64:0/8019320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb2bc004300 con 0x7fb2c807de20 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2c77fe700 1 -- 172.21.15.64:0/8019320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2bc011a80 con 0x7fb2c807de20 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 -- 172.21.15.64:0/8019320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 msgr2=0x7fb2c807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 --2- 172.21.15.64:0/8019320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c807c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0098a0 tx=0x7fb2bc009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 -- 172.21.15.64:0/8019320 shutdown_connections 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 --2- 172.21.15.64:0/8019320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c807c320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 -- 172.21.15.64:0/8019320 >> 172.21.15.64:0/8019320 conn(0x7fb2c81030d0 msgr2=0x7fb2c8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 -- 172.21.15.64:0/8019320 shutdown_connections 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.205+0000 7fb2cefac700 1 -- 172.21.15.64:0/8019320 wait complete. 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2cefac700 1 Processor -- start 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2cefac700 1 -- start start 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2cefac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2cefac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c810e080 con 0x7fb2c807de20 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2ccd48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2ccd48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59522/0 (socket says 172.21.15.64:59522) 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2ccd48700 1 -- 172.21.15.64:0/2398266268 learned_addr learned my addr 172.21.15.64:0/2398266268 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.206+0000 7fb2ccd48700 1 -- 172.21.15.64:0/2398266268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c819e0d0 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2ccd48700 1 --2- 172.21.15.64:0/2398266268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c819db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc011a50 tx=0x7fb2bc011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2bc003a80 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb2bc003c20 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c819e360 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2bc01a650 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a1050 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2bc01ad00 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.207+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb2bc019700 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.208+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c8063980 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.210+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb2bc024ad0 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.281+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb2c81a1590 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.281+0000 7fb2c5ffb700 1 -- 172.21.15.64:0/2398266268 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb2bc024440 con 0x7fb2c807de20 2024-05-09T08:58:34.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.282+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 msgr2=0x7fb2c819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.282+0000 7fb2cefac700 1 --2- 172.21.15.64:0/2398266268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c819db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc011a50 tx=0x7fb2bc011b30 comp rx=0 tx=0).stop 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.282+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 shutdown_connections 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.282+0000 7fb2cefac700 1 --2- 172.21.15.64:0/2398266268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c807de20 0x7fb2c819db90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.282+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 >> 172.21.15.64:0/2398266268 conn(0x7fb2c81030d0 msgr2=0x7fb2c8103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.283+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 shutdown_connections 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:34.283+0000 7fb2cefac700 1 -- 172.21.15.64:0/2398266268 wait complete. 2024-05-09T08:58:34.863 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2024-05-09T08:58:35.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:34 smithi064 systemd[1]: Stopping Ceph mon.a for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T08:58:35.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:35 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a[29770]: 2024-05-09T08:58:35.130+0000 7f2c442c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:35 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a[29770]: 2024-05-09T08:58:35.130+0000 7f2c442c9700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T08:58:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:35 smithi064 podman[30018]: 2024-05-09 08:58:35.285900603 +0000 UTC m=+0.249972313 container died 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T08:58:35.780 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:35 smithi064 podman[30018]: 2024-05-09 08:58:35.589674214 +0000 UTC m=+0.553745931 container cleanup 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T08:58:35.781 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:35 smithi064 bash[30018]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a 2024-05-09T08:58:36.031 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:35 smithi064 podman[30032]: 2024-05-09 08:58:35.782270091 +0000 UTC m=+0.494605950 container remove 1e0a19b08e3f6386d8f56e27acd5dfdc689fe36ffde48c54dcee9bd992a797b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:58:36.031 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:36 smithi064 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T08:58:36.031 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:36 smithi064 systemd[1]: Stopped Ceph mon.a for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T08:58:36.315 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:36 smithi064 systemd[1]: Starting Ceph mon.a for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T08:58:36.315 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:36 smithi064 podman[30130]: 2024-05-09 08:58:36.152436788 +0000 UTC m=+0.018503906 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:58:36.315 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:36 smithi064 podman[30130]: 2024-05-09 08:58:36.316576174 +0000 UTC m=+0.182643333 container create 8a68b30aba117b1b13482e7c4007555df3b861870b653b3e05457d2913383a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:58:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 podman[30130]: 2024-05-09 08:58:38.214912531 +0000 UTC m=+2.080979656 container init 8a68b30aba117b1b13482e7c4007555df3b861870b653b3e05457d2913383a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 podman[30130]: 2024-05-09 08:58:38.219348708 +0000 UTC m=+2.085415826 container start 8a68b30aba117b1b13482e7c4007555df3b861870b653b3e05457d2913383a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: pidfile_write: ignore empty --pid-file 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: load: jerasure load: lrc 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: RocksDB version: 7.9.2 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Git sha 0 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: DB SUMMARY 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: DB Session ID: 5ICEIUD7SVHK87I9P585 2024-05-09T08:58:38.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: CURRENT file: CURRENT 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103174 ; 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.error_if_exists: 0 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.create_if_missing: 0 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.paranoid_checks: 1 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.env: 0x55794be0dc60 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.info_log: 0x55794db53ac0 2024-05-09T08:58:38.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.statistics: (nil) 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.use_fsync: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_log_file_size: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.allow_fallocate: 1 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.use_direct_reads: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T08:58:38.656 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.db_log_dir: 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.wal_dir: 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.write_buffer_manager: 0x55794db261e0 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T08:58:38.657 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.unordered_write: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.row_cache: None 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.wal_filter: None 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.two_write_queues: 0 2024-05-09T08:58:38.658 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.wal_compression: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.atomic_flush: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.log_readahead_size: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T08:58:38.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_background_jobs: 2 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_background_compactions: -1 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_subcompactions: 1 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_open_files: -1 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T08:58:38.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_background_flushes: -1 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Compression algorithms supported: 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kZSTD supported: 0 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kXpressCompression supported: 0 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kZlibCompression supported: 1 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kSnappyCompression supported: 1 2024-05-09T08:58:38.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kLZ4Compression supported: 1 2024-05-09T08:58:38.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: kBZip2Compression supported: 0 2024-05-09T08:58:38.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T08:58:38.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.merge_operator: 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_filter: None 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T08:58:38.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55794db53bc0) 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_type: 0 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_index_type: 0 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_shortening: 1 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T08:58:38.664 INFO:journalctl@ceph.mon.a.smithi064.stdout: checksum: 4 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: no_block_cache: 0 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache: 0x55794db57090 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_options: 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: capacity : 536870912 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_shard_bits : 4 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: strict_capacity_limit : 0 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_compressed: (nil) 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: persistent_cache: (nil) 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size: 4096 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size_deviation: 10 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_restart_interval: 16 2024-05-09T08:58:38.665 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_block_restart_interval: 1 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: metadata_block_size: 4096 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: partition_filters: 0 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: use_delta_encoding: 1 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: filter_policy: bloomfilter 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: whole_key_filtering: 1 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: verify_compression: 0 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: format_version: 5 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_index_compression: 1 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_align: 0 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: max_auto_readahead_size: 262144 2024-05-09T08:58:38.666 INFO:journalctl@ceph.mon.a.smithi064.stdout: prepopulate_block_cache: 0 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout: initial_auto_readahead_size: 8192 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression: NoCompression 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.num_levels: 7 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T08:58:38.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T08:58:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T08:58:38.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T08:58:38.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.table_properties_collectors: 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.inplace_update_support: 0 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T08:58:38.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.bloom_locality: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.max_successive_merges: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.ttl: 2592000 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.enable_blob_files: false 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.min_blob_size: 0 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T08:58:38.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 405113d8-83ae-4c20-a744-ae988f1afecf 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715245118255318, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715245118257031, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96412, "index_size": 301, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13788, "raw_average_key_size": 49, "raw_value_size": 88861, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715245118, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "405113d8-83ae-4c20-a744-ae988f1afecf", "db_session_id": "5ICEIUD7SVHK87I9P585", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715245118257182, "job": 1, "event": "recovery_finished"} 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T08:58:38.673 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55794dc48000 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: DB pointer 0x55794dc32000 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** DB Stats ** 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T08:58:38.674 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: L0 2/0 97.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: Sum 2/0 97.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 97.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-09T08:58:38.675 INFO:journalctl@ceph.mon.a.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative compaction: 0.00 GB write, 17.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval compaction: 0.00 GB write, 17.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache BinnedLRUCache@0x55794db57090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T08:58:38.676 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???) e1 preinit fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).mds e1 new map 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).mds e1 print_map 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: e1 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: legacy client fscid: -1 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout: No filesystems configured 2024-05-09T08:58:38.677 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: monmap epoch 1 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: last_changed 2024-05-09T08:58:28.429468+0000 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: min_mon_release 19 (squid) 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: election_strategy: 1 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: fsmap 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T08:58:38.678 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:38 smithi064 ceph-mon[30149]: mgrmap e1: no daemons active 2024-05-09T08:58:39.055 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T08:58:39.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:39 smithi064 bash[30130]: 8a68b30aba117b1b13482e7c4007555df3b861870b653b3e05457d2913383a76 2024-05-09T08:58:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:39 smithi064 systemd[1]: Started Ceph mon.a for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T08:58:43.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:43 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3318032155' entity='client.admin' 2024-05-09T08:58:47.365 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.099+0000 7fa7b1010700 1 Processor -- start 2024-05-09T08:58:47.365 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.099+0000 7fa7b1010700 1 -- start start 2024-05-09T08:58:47.365 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.099+0000 7fa7b1010700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac10a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:47.365 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.099+0000 7fa7b1010700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10a6c0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac10a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac10a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35488/0 (socket says 172.21.15.64:35488) 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7aad9d700 1 -- 172.21.15.64:0/2527501563 learned_addr learned my addr 172.21.15.64:0/2527501563 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7aad9d700 1 -- 172.21.15.64:0/2527501563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac10aed0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7aad9d700 1 --2- 172.21.15.64:0/2527501563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac10a0f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098a0 tx=0x7fa79c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45862f42c2f9fea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7a9d9b700 1 -- 172.21.15.64:0/2527501563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79c004160 con 0x7fa7ac109cd0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.100+0000 7fa7a9d9b700 1 -- 172.21.15.64:0/2527501563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa79c004300 con 0x7fa7ac109cd0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7a9d9b700 1 -- 172.21.15.64:0/2527501563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79c011a80 con 0x7fa7ac109cd0 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 -- 172.21.15.64:0/2527501563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 msgr2=0x7fa7ac10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 --2- 172.21.15.64:0/2527501563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac10a0f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098a0 tx=0x7fa79c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 -- 172.21.15.64:0/2527501563 shutdown_connections 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 --2- 172.21.15.64:0/2527501563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac10a0f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 -- 172.21.15.64:0/2527501563 >> 172.21.15.64:0/2527501563 conn(0x7fa7ac07d080 msgr2=0x7fa7ac07d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:47.366 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 -- 172.21.15.64:0/2527501563 shutdown_connections 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.101+0000 7fa7b1010700 1 -- 172.21.15.64:0/2527501563 wait complete. 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.102+0000 7fa7b1010700 1 Processor -- start 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.102+0000 7fa7b1010700 1 -- start start 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.102+0000 7fa7b1010700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.102+0000 7fa7b1010700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10e280 con 0x7fa7ac109cd0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35500/0 (socket says 172.21.15.64:35500) 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7aad9d700 1 -- 172.21.15.64:0/3318032155 learned_addr learned my addr 172.21.15.64:0/3318032155 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7aad9d700 1 -- 172.21.15.64:0/3318032155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac19e160 con 0x7fa7ac109cd0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7aad9d700 1 --2- 172.21.15.64:0/3318032155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac19dc20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa79c000c00 tx=0x7fa79c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79c0039f0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac19e3f0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa79c003b90 con 0x7fa7ac109cd0 2024-05-09T08:58:47.367 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79c01a740 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac1a10e0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.103+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa79c02d3f0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.104+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa79c019700 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.104+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac063980 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.107+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa79c024af0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.178+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa7ac1a12b0 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.180+0000 7fa793fff700 1 -- 172.21.15.64:0/3318032155 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa79c024440 con 0x7fa7ac109cd0 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 msgr2=0x7fa7ac19dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 --2- 172.21.15.64:0/3318032155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac19dc20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa79c000c00 tx=0x7fa79c011b00 comp rx=0 tx=0).stop 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 shutdown_connections 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 --2- 172.21.15.64:0/3318032155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ac109cd0 0x7fa7ac19dc20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 >> 172.21.15.64:0/3318032155 conn(0x7fa7ac07d080 msgr2=0x7fa7ac107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 shutdown_connections 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:42.181+0000 7fa7b1010700 1 -- 172.21.15.64:0/3318032155 wait complete. 2024-05-09T08:58:47.368 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T08:58:47.381 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:58:47.381 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2024-05-09T08:58:47.382 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T08:58:47.382 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T08:58:47.637 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x 2024-05-09T08:58:47.638 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-09T08:58:47.793 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2877b902-0de2-11ef-bc98-c7b262605968.target.wants/ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2877b902-0de2-11ef-bc98-c7b262605968@.service. 2024-05-09T08:58:47.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:47 smithi064 systemd[1]: Starting Ceph mgr.x for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T08:58:48.353 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 podman[30364]: 2024-05-09 08:58:48.052138225 +0000 UTC m=+0.131218284 container create ef41d033accc09139b9c1592f15cbadfc7d4fad3d46f11575bdbacfcbb5d687b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T08:58:48.353 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 podman[30364]: 2024-05-09 08:58:47.955586082 +0000 UTC m=+0.034666161 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:58:48.354 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 podman[30364]: 2024-05-09 08:58:48.262683238 +0000 UTC m=+0.341763298 container init ef41d033accc09139b9c1592f15cbadfc7d4fad3d46f11575bdbacfcbb5d687b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T08:58:48.354 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 podman[30364]: 2024-05-09 08:58:48.265581529 +0000 UTC m=+0.344661589 container start ef41d033accc09139b9c1592f15cbadfc7d4fad3d46f11575bdbacfcbb5d687b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T08:58:48.354 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 bash[30364]: ef41d033accc09139b9c1592f15cbadfc7d4fad3d46f11575bdbacfcbb5d687b 2024-05-09T08:58:48.367 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:58:48.367 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-09T08:58:48.375 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:58:48.375 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-09T08:58:48.375 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-09T08:58:48.375 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T08:58:48.384 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:58:48.384 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-09T08:58:48.392 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:58:48.392 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-09T08:58:48.392 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-09T08:58:48.393 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T08:58:48.393 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2024-05-09T08:58:48.393 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2024-05-09T08:58:48.620 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 systemd[1]: Started Ceph mgr.x for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T08:58:48.621 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:48.425+0000 7f7fefb13200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:58:48.621 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:48.621+0000 7f7fefb13200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:58:48.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:48.732+0000 7f7fefb13200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:58:49.292 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3324162577' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:58:49.293 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:49.069+0000 7f7fefb13200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:58:49.293 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:49.158+0000 7f7fefb13200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:58:49.554 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:49.557 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:58:49.558 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:58:49.559 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:58:49.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:58:49.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:58:49.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:58:49.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:58:49.569 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.570 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:58:31.503758+0000", 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.852+0000 7fd57ce78700 1 Processor -- start 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.852+0000 7fd57ce78700 1 -- start start 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.853+0000 7fd57ce78700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.853+0000 7fd57ce78700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd578109ab0 con 0x7fd5781090c0 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.853+0000 7fd57659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.853+0000 7fd57659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51998/0 (socket says 172.21.15.64:51998) 2024-05-09T08:58:49.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.853+0000 7fd57659c700 1 -- 172.21.15.64:0/669351414 learned_addr learned my addr 172.21.15.64:0/669351414 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.853+0000 7fd57659c700 1 -- 172.21.15.64:0/669351414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd578109c30 con 0x7fd5781090c0 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.854+0000 7fd57659c700 1 --2- 172.21.15.64:0/669351414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781094e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098a0 tx=0x7fd568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bfed03c7c5023a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.854+0000 7fd57559a700 1 -- 172.21.15.64:0/669351414 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd568004160 con 0x7fd5781090c0 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.854+0000 7fd57559a700 1 -- 172.21.15.64:0/669351414 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd568004300 con 0x7fd5781090c0 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.854+0000 7fd57559a700 1 -- 172.21.15.64:0/669351414 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd568011a80 con 0x7fd5781090c0 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.854+0000 7fd57ce78700 1 -- 172.21.15.64:0/669351414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 msgr2=0x7fd5781094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.854+0000 7fd57ce78700 1 --2- 172.21.15.64:0/669351414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781094e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098a0 tx=0x7fd568009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 -- 172.21.15.64:0/669351414 shutdown_connections 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 --2- 172.21.15.64:0/669351414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781094e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 -- 172.21.15.64:0/669351414 >> 172.21.15.64:0/669351414 conn(0x7fd5781016e0 msgr2=0x7fd578103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 -- 172.21.15.64:0/669351414 shutdown_connections 2024-05-09T08:58:49.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 -- 172.21.15.64:0/669351414 wait complete. 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 Processor -- start 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.855+0000 7fd57ce78700 1 -- start start 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57ce78700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57ce78700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5781068d0 con 0x7fd5781090c0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52010/0 (socket says 172.21.15.64:52010) 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57659c700 1 -- 172.21.15.64:0/3324162577 learned_addr learned my addr 172.21.15.64:0/3324162577 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57659c700 1 -- 172.21.15.64:0/3324162577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd578108c20 con 0x7fd5781090c0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57659c700 1 --2- 172.21.15.64:0/3324162577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781086e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd568000c00 tx=0x7fd568011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5680039f0 con 0x7fd5781090c0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd578106e10 con 0x7fd5781090c0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd568003b90 con 0x7fd5781090c0 2024-05-09T08:58:49.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.856+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd56801a740 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.857+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd578107350 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.857+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd56802d3f0 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.857+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd568019700 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.858+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57810c090 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.860+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd568043dd0 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.934+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd578107aa0 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.934+0000 7fd5677fe700 1 -- 172.21.15.64:0/3324162577 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd568032020 con 0x7fd5781090c0 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.935+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 msgr2=0x7fd5781086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.935+0000 7fd57ce78700 1 --2- 172.21.15.64:0/3324162577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781086e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd568000c00 tx=0x7fd568011b00 comp rx=0 tx=0).stop 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.936+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 shutdown_connections 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.936+0000 7fd57ce78700 1 --2- 172.21.15.64:0/3324162577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5781090c0 0x7fd5781086e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:49.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.936+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 >> 172.21.15.64:0/3324162577 conn(0x7fd5781016e0 msgr2=0x7fd578102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:49.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.936+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 shutdown_connections 2024-05-09T08:58:49.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:48.936+0000 7fd57ce78700 1 -- 172.21.15.64:0/3324162577 wait complete. 2024-05-09T08:58:49.575 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2024-05-09T08:58:50.631 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:50.450+0000 7f7fefb13200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:58:50.632 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:50.632+0000 7f7fefb13200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:58:50.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:50.810+0000 7f7fefb13200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:58:51.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:51.113+0000 7f7fefb13200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:58:51.403 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:51.212+0000 7f7fefb13200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:58:51.694 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:51.428+0000 7f7fefb13200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:58:51.694 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:51.636+0000 7f7fefb13200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:58:52.280 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:52 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3900374235' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:58:52.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:52.280+0000 7f7fefb13200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:58:52.985 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:52.722+0000 7f7fefb13200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:58:52.985 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:52.823+0000 7f7fefb13200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:58:52.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:52.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-05-09T08:58:52.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:58:53.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:58:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:58:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:58:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:58:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:58:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:58:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:58:53.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:58:53.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:53.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:53.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:58:31.503758+0000", 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.993+0000 7f1913288700 1 Processor -- start 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.993+0000 7f1913288700 1 -- start start 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.993+0000 7f1913288700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c1072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.993+0000 7f1913288700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f190c107890 con 0x7f190c106ea0 2024-05-09T08:58:53.004 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1911024700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c1072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1911024700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c1072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52012/0 (socket says 172.21.15.64:52012) 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1911024700 1 -- 172.21.15.64:0/3090335132 learned_addr learned my addr 172.21.15.64:0/3090335132 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1911024700 1 -- 172.21.15.64:0/3090335132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c107bc0 con 0x7f190c106ea0 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1911024700 1 --2- 172.21.15.64:0/3090335132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c1072c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19080098a0 tx=0x7f1908009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa136227b4d8aa37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1903fff700 1 -- 172.21.15.64:0/3090335132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1908004160 con 0x7f190c106ea0 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1903fff700 1 -- 172.21.15.64:0/3090335132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1908004300 con 0x7f190c106ea0 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.994+0000 7f1903fff700 1 -- 172.21.15.64:0/3090335132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1908011a80 con 0x7f190c106ea0 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 -- 172.21.15.64:0/3090335132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 msgr2=0x7f190c1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 --2- 172.21.15.64:0/3090335132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c1072c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19080098a0 tx=0x7f1908009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 -- 172.21.15.64:0/3090335132 shutdown_connections 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 --2- 172.21.15.64:0/3090335132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c1072c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 -- 172.21.15.64:0/3090335132 >> 172.21.15.64:0/3090335132 conn(0x7f190c1016e0 msgr2=0x7f190c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:53.005 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 -- 172.21.15.64:0/3090335132 shutdown_connections 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.995+0000 7f1913288700 1 -- 172.21.15.64:0/3090335132 wait complete. 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1913288700 1 Processor -- start 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1913288700 1 -- start start 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1913288700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1913288700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f190c10af70 con 0x7f190c106ea0 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1911024700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1911024700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52026/0 (socket says 172.21.15.64:52026) 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1911024700 1 -- 172.21.15.64:0/3900374235 learned_addr learned my addr 172.21.15.64:0/3900374235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1911024700 1 -- 172.21.15.64:0/3900374235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c19bf00 con 0x7f190c106ea0 2024-05-09T08:58:53.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.996+0000 7f1911024700 1 --2- 172.21.15.64:0/3900374235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c19b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1908006af0 tx=0x7f1908011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19080039e0 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1908003b80 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c19c190 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19080196a0 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c19ee80 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1908019ce0 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.997+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1908024770 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:51.998+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c063980 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.000+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1908043dd0 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.075+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f190c02de70 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.075+0000 7f19027fc700 1 -- 172.21.15.64:0/3900374235 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f1908032020 con 0x7f190c106ea0 2024-05-09T08:58:53.007 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 msgr2=0x7f190c19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 --2- 172.21.15.64:0/3900374235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c19b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1908006af0 tx=0x7f1908011c40 comp rx=0 tx=0).stop 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 shutdown_connections 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 --2- 172.21.15.64:0/3900374235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f190c106ea0 0x7f190c19b9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 >> 172.21.15.64:0/3900374235 conn(0x7f190c1016e0 msgr2=0x7f190c1024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 shutdown_connections 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:52.076+0000 7f1913288700 1 -- 172.21.15.64:0/3900374235 wait complete. 2024-05-09T08:58:53.008 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2024-05-09T08:58:53.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:53.033+0000 7f7fefb13200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:58:53.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:53.581+0000 7f7fefb13200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:58:53.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:53.677+0000 7f7fefb13200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:58:54.289 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:53.931+0000 7f7fefb13200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:58:54.290 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:54.071+0000 7f7fefb13200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:58:54.290 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:54.169+0000 7f7fefb13200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:58:54.290 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:54.290+0000 7f7fefb13200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:58:54.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:54.602+0000 7f7fefb13200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:58:55.368 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:55.242+0000 7f7fefb13200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:58:55.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:58:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:58:55.473+0000 7f7fefb13200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3042467045' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: Activating manager daemon x 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: mgrmap e2: x(active, starting, since 0.00334125s) 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: Manager daemon x is now available 2024-05-09T08:58:55.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:55 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T08:58:56.171 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:58:56.175 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:58:56.176 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:58:56.177 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:58:56.185 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:58:56.185 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:58:56.185 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:58:56.185 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.185 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:58:56.186 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:58:31.503758+0000", 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:56.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc79bc87700 1 Processor -- start 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc79bc87700 1 -- start start 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc79bc87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc79410a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc79bc87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc79407bed0 con 0x7fc7941084f0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc799a23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc79410a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc799a23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc79410a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36560/0 (socket says 172.21.15.64:36560) 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.369+0000 7fc799a23700 1 -- 172.21.15.64:0/1672962632 learned_addr learned my addr 172.21.15.64:0/1672962632 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.370+0000 7fc799a23700 1 -- 172.21.15.64:0/1672962632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79410ae20 con 0x7fc7941084f0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.370+0000 7fc799a23700 1 --2- 172.21.15.64:0/1672962632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc79410a8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098a0 tx=0x7fc790009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f0bd03525298e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.370+0000 7fc798a21700 1 -- 172.21.15.64:0/1672962632 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc790004160 con 0x7fc7941084f0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.370+0000 7fc798a21700 1 -- 172.21.15.64:0/1672962632 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc790004300 con 0x7fc7941084f0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.370+0000 7fc798a21700 1 -- 172.21.15.64:0/1672962632 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc790011a80 con 0x7fc7941084f0 2024-05-09T08:58:56.188 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 -- 172.21.15.64:0/1672962632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 msgr2=0x7fc79410a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 --2- 172.21.15.64:0/1672962632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc79410a8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098a0 tx=0x7fc790009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 -- 172.21.15.64:0/1672962632 shutdown_connections 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 --2- 172.21.15.64:0/1672962632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc79410a8e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 -- 172.21.15.64:0/1672962632 >> 172.21.15.64:0/1672962632 conn(0x7fc794101f60 msgr2=0x7fc794104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 -- 172.21.15.64:0/1672962632 shutdown_connections 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 -- 172.21.15.64:0/1672962632 wait complete. 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.371+0000 7fc79bc87700 1 Processor -- start 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc79bc87700 1 -- start start 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc79bc87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc794107d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc79bc87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410bdf0 con 0x7fc7941084f0 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc799a23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc794107d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc799a23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc794107d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36564/0 (socket says 172.21.15.64:36564) 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc799a23700 1 -- 172.21.15.64:0/3042467045 learned_addr learned my addr 172.21.15.64:0/3042467045 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:56.189 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc799a23700 1 -- 172.21.15.64:0/3042467045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7941082b0 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.372+0000 7fc799a23700 1 --2- 172.21.15.64:0/3042467045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc794107d70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc790011a50 tx=0x7fc790011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc790003a80 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc790003c20 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941064a0 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc79001a650 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941069e0 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc79001ad00 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.373+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc790019700 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.374+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc794063980 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.376+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc790024ad0 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.450+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc794106d20 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.450+0000 7fc78affd700 1 -- 172.21.15.64:0/3042467045 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fc7900199a0 con 0x7fc7941084f0 2024-05-09T08:58:56.190 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 msgr2=0x7fc794107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 --2- 172.21.15.64:0/3042467045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc794107d70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc790011a50 tx=0x7fc790011b30 comp rx=0 tx=0).stop 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 shutdown_connections 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 --2- 172.21.15.64:0/3042467045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7941084f0 0x7fc794107d70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 >> 172.21.15.64:0/3042467045 conn(0x7fc794101f60 msgr2=0x7fc794102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 shutdown_connections 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:55.452+0000 7fc79bc87700 1 -- 172.21.15.64:0/3042467045 wait complete. 2024-05-09T08:58:56.191 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (3/15)... 2024-05-09T08:58:56.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:56 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T08:58:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:56 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' 2024-05-09T08:58:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:56 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' 2024-05-09T08:58:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:56 smithi064 ceph-mon[30149]: from='mgr.14100 172.21.15.64:0/2543819333' entity='mgr.x' 2024-05-09T08:58:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:56 smithi064 ceph-mon[30149]: mgrmap e3: x(active, since 1.00735s) 2024-05-09T08:58:58.591 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:58 smithi064 ceph-mon[30149]: mgrmap e4: x(active, since 2s) 2024-05-09T08:58:59.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:58:59.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:58:59.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:58:59.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 20, 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:58:59.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:58:59.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:58:59.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:58:59.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:58:59.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:58:59.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:58:59.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:58:31.503758+0000", 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5ae702700 1 Processor -- start 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5ae702700 1 -- start start 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5ae702700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5ae702700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a807c820 con 0x7ff5a807dde0 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36652/0 (socket says 172.21.15.64:36652) 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.592+0000 7ff5a7fff700 1 -- 172.21.15.64:0/853034161 learned_addr learned my addr 172.21.15.64:0/853034161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.593+0000 7ff5a7fff700 1 -- 172.21.15.64:0/853034161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a807c9a0 con 0x7ff5a807dde0 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.593+0000 7ff5a7fff700 1 --2- 172.21.15.64:0/853034161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a807c2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5980098a0 tx=0x7ff598009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a772142bc755394 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.593+0000 7ff5a6ffd700 1 -- 172.21.15.64:0/853034161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff598004160 con 0x7ff5a807dde0 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.593+0000 7ff5a6ffd700 1 -- 172.21.15.64:0/853034161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff598004300 con 0x7ff5a807dde0 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.593+0000 7ff5a6ffd700 1 -- 172.21.15.64:0/853034161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff598011a80 con 0x7ff5a807dde0 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 -- 172.21.15.64:0/853034161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 msgr2=0x7ff5a807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 --2- 172.21.15.64:0/853034161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a807c2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5980098a0 tx=0x7ff598009bb0 comp rx=0 tx=0).stop 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 -- 172.21.15.64:0/853034161 shutdown_connections 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 --2- 172.21.15.64:0/853034161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a807c2e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 -- 172.21.15.64:0/853034161 >> 172.21.15.64:0/853034161 conn(0x7ff5a81030d0 msgr2=0x7ff5a8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 -- 172.21.15.64:0/853034161 shutdown_connections 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.594+0000 7ff5ae702700 1 -- 172.21.15.64:0/853034161 wait complete. 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5ae702700 1 Processor -- start 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5ae702700 1 -- start start 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5ae702700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5ae702700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810e060 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36664/0 (socket says 172.21.15.64:36664) 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.595+0000 7ff5a7fff700 1 -- 172.21.15.64:0/99869187 learned_addr learned my addr 172.21.15.64:0/99869187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5a7fff700 1 -- 172.21.15.64:0/99869187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a819e0d0 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5a7fff700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a819db90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff598006af0 tx=0x7ff598011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5980039e0 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff598003b80 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a819e360 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5980196a0 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.596+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a81a1050 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.597+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a8063980 con 0x7ff5a807dde0 2024-05-09T08:58:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.598+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff59801ab50 con 0x7ff5a807dde0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.598+0000 7ff5a57fa700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7ff590039000 0x7ff59003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.598+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff59804d4f0 con 0x7ff5a807dde0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.598+0000 7ff5a77fe700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7ff590039000 0x7ff59003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.598+0000 7ff5a77fe700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7ff590039000 0x7ff59003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff59c006fd0 tx=0x7ff59c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.603+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff59804cf10 con 0x7ff5a807dde0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.894+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff5a81a1220 con 0x7ff5a807dde0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.895+0000 7ff5a57fa700 1 -- 172.21.15.64:0/99869187 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff5a81a1220 con 0x7ff5a807dde0 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7ff590039000 msgr2=0x7ff59003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7ff590039000 0x7ff59003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff59c006fd0 tx=0x7ff59c006e40 comp rx=0 tx=0).stop 2024-05-09T08:58:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 msgr2=0x7ff5a819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a819db90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff598006af0 tx=0x7ff598011c40 comp rx=0 tx=0).stop 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 shutdown_connections 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7ff590039000 0x7ff59003b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 --2- 172.21.15.64:0/99869187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a807dde0 0x7ff5a819db90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 >> 172.21.15.64:0/99869187 conn(0x7ff5a81030d0 msgr2=0x7ff5a8103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 shutdown_connections 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:58.898+0000 7ff5ae702700 1 -- 172.21.15.64:0/99869187 wait complete. 2024-05-09T08:58:59.492 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2024-05-09T08:58:59.593 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:58:59 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/99869187' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:59:00.675 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5e0dd9700 1 Processor -- start 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5e0dd9700 1 -- start start 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5e0dd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc10a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5e0dd9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc07be90 con 0x7fb5dc108590 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5da59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc10a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5da59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc10a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36668/0 (socket says 172.21.15.64:36668) 2024-05-09T08:59:00.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5da59c700 1 -- 172.21.15.64:0/2570434421 learned_addr learned my addr 172.21.15.64:0/2570434421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.867+0000 7fb5da59c700 1 -- 172.21.15.64:0/2570434421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc10aec0 con 0x7fb5dc108590 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5da59c700 1 --2- 172.21.15.64:0/2570434421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc10a980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0098a0 tx=0x7fb5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e32ecbfca0b6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5d959a700 1 -- 172.21.15.64:0/2570434421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5cc004160 con 0x7fb5dc108590 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5d959a700 1 -- 172.21.15.64:0/2570434421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb5cc004300 con 0x7fb5dc108590 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5d959a700 1 -- 172.21.15.64:0/2570434421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5cc011a80 con 0x7fb5dc108590 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/2570434421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 msgr2=0x7fb5dc10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 --2- 172.21.15.64:0/2570434421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc10a980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0098a0 tx=0x7fb5cc009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/2570434421 shutdown_connections 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 --2- 172.21.15.64:0/2570434421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc10a980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/2570434421 >> 172.21.15.64:0/2570434421 conn(0x7fb5dc101f60 msgr2=0x7fb5dc1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/2570434421 shutdown_connections 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.868+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/2570434421 wait complete. 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.869+0000 7fb5e0dd9700 1 Processor -- start 2024-05-09T08:59:00.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.869+0000 7fb5e0dd9700 1 -- start start 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.869+0000 7fb5e0dd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.869+0000 7fb5e0dd9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc10be80 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5da59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5da59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36672/0 (socket says 172.21.15.64:36672) 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5da59c700 1 -- 172.21.15.64:0/3899972273 learned_addr learned my addr 172.21.15.64:0/3899972273 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5da59c700 1 -- 172.21.15.64:0/3899972273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc19bee0 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5da59c700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc19b9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc011a50 tx=0x7fb5cc011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5cc003a80 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc19c170 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb5cc003c20 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5cc01a650 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.870+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc19ee60 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.871+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb5cc01a950 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.871+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc063980 con 0x7fb5dc108590 2024-05-09T08:59:00.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.871+0000 7fb5d37fe700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fb5c8039000 0x7fb5c803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.871+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb5cc04d650 con 0x7fb5dc108590 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.871+0000 7fb5d9d9b700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fb5c8039000 0x7fb5c803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.872+0000 7fb5d9d9b700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fb5c8039000 0x7fb5c803b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4006fd0 tx=0x7fb5c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:58:59.876+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb5cc019c70 con 0x7fb5dc108590 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.079+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb5dc19f030 con 0x7fb5dc108590 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.080+0000 7fb5d37fe700 1 -- 172.21.15.64:0/3899972273 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fb5dc19f030 con 0x7fb5dc108590 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fb5c8039000 msgr2=0x7fb5c803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fb5c8039000 0x7fb5c803b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4006fd0 tx=0x7fb5c4006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 msgr2=0x7fb5dc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc19b9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc011a50 tx=0x7fb5cc011b30 comp rx=0 tx=0).stop 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 shutdown_connections 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fb5c8039000 0x7fb5c803b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 --2- 172.21.15.64:0/3899972273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc108590 0x7fb5dc19b9a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 >> 172.21.15.64:0/3899972273 conn(0x7fb5dc101f60 msgr2=0x7fb5dc102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:00.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 shutdown_connections 2024-05-09T08:59:00.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:00.083+0000 7fb5e0dd9700 1 -- 172.21.15.64:0/3899972273 wait complete. 2024-05-09T08:59:00.680 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2024-05-09T08:59:00.700 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:00 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3899972273' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T08:59:01.552 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:01 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1352808778' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T08:59:01.804 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: ignoring --setuser ceph since I am not root 2024-05-09T08:59:01.805 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: ignoring --setgroup ceph since I am not root 2024-05-09T08:59:01.805 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:01.683+0000 7fa1e97dc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:59:02.097 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:01.875+0000 7fa1e97dc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:59:02.097 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:01.981+0000 7fa1e97dc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:59:02.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b94ba700 1 Processor -- start 2024-05-09T08:59:02.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b94ba700 1 -- start start 2024-05-09T08:59:02.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b94ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b4106da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b94ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4107370 con 0x7fd6b4106980 2024-05-09T08:59:02.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b4106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.105 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b4106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36688/0 (socket says 172.21.15.64:36688) 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b2ffd700 1 -- 172.21.15.64:0/4112162263 learned_addr learned my addr 172.21.15.64:0/4112162263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b2ffd700 1 -- 172.21.15.64:0/4112162263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b4107b80 con 0x7fd6b4106980 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.096+0000 7fd6b2ffd700 1 --2- 172.21.15.64:0/4112162263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b4106da0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40098a0 tx=0x7fd6a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32022ad4908bd585 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b1ffb700 1 -- 172.21.15.64:0/4112162263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a4004160 con 0x7fd6b4106980 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b1ffb700 1 -- 172.21.15.64:0/4112162263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd6a4004300 con 0x7fd6b4106980 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b1ffb700 1 -- 172.21.15.64:0/4112162263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a4011a80 con 0x7fd6b4106980 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 -- 172.21.15.64:0/4112162263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 msgr2=0x7fd6b4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 --2- 172.21.15.64:0/4112162263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b4106da0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40098a0 tx=0x7fd6a4009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 -- 172.21.15.64:0/4112162263 shutdown_connections 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 --2- 172.21.15.64:0/4112162263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b4106da0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 -- 172.21.15.64:0/4112162263 >> 172.21.15.64:0/4112162263 conn(0x7fd6b4101f60 msgr2=0x7fd6b4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 -- 172.21.15.64:0/4112162263 shutdown_connections 2024-05-09T08:59:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.097+0000 7fd6b94ba700 1 -- 172.21.15.64:0/4112162263 wait complete. 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.098+0000 7fd6b94ba700 1 Processor -- start 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.098+0000 7fd6b94ba700 1 -- start start 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.098+0000 7fd6b94ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b94ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b410af30 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36690/0 (socket says 172.21.15.64:36690) 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b2ffd700 1 -- 172.21.15.64:0/1352808778 learned_addr learned my addr 172.21.15.64:0/1352808778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b2ffd700 1 -- 172.21.15.64:0/1352808778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b419bf40 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b2ffd700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b419ba00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000c00 tx=0x7fd6a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a40039f0 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b419c1d0 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd6a4003b90 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b419eec0 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.099+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a401a740 con 0x7fd6b4106980 2024-05-09T08:59:02.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.100+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd6a401aa40 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.100+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b4063980 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.100+0000 7fd69bfff700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fd69c039000 0x7fd69c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.100+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6a404d4c0 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.101+0000 7fd6b27fc700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fd69c039000 0x7fd69c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.101+0000 7fd6b27fc700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fd69c039000 0x7fd69c03b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006fd0 tx=0x7fd6a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.105+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6a400ba00 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.370+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd6b40008d0 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.508+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd6a404cee0 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.510+0000 7fd69bfff700 1 -- 172.21.15.64:0/1352808778 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd6a402d910 con 0x7fd6b4106980 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fd69c039000 msgr2=0x7fd69c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fd69c039000 0x7fd69c03b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006fd0 tx=0x7fd6a8006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 msgr2=0x7fd6b419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b419ba00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000c00 tx=0x7fd6a4011b00 comp rx=0 tx=0).stop 2024-05-09T08:59:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 shutdown_connections 2024-05-09T08:59:02.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fd69c039000 0x7fd69c03b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 --2- 172.21.15.64:0/1352808778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6b4106980 0x7fd6b419ba00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 >> 172.21.15.64:0/1352808778 conn(0x7fd6b4101f60 msgr2=0x7fd6b4102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:02.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 shutdown_connections 2024-05-09T08:59:02.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:01.511+0000 7fd6b94ba700 1 -- 172.21.15.64:0/1352808778 wait complete. 2024-05-09T08:59:02.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:02 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:02.313+0000 7fa1e97dc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:59:02.734 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:02 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1352808778' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T08:59:02.734 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:02 smithi064 ceph-mon[30149]: mgrmap e5: x(active, since 6s) 2024-05-09T08:59:02.734 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:02 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:02.405+0000 7fa1e97dc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:59:03.439 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:59:03.439 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T08:59:03.439 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T08:59:03.439 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T08:59:03.439 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.528+0000 7fa1a41b1700 1 Processor -- start 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a41b1700 1 -- start start 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a41b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c10a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a41b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c07be90 con 0x7fa19c108590 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a1f4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c10a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a1f4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c10a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36726/0 (socket says 172.21.15.64:36726) 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a1f4d700 1 -- 172.21.15.64:0/2165158648 learned_addr learned my addr 172.21.15.64:0/2165158648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a1f4d700 1 -- 172.21.15.64:0/2165158648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c10aec0 con 0x7fa19c108590 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.529+0000 7fa1a1f4d700 1 --2- 172.21.15.64:0/2165158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c10a980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1980098a0 tx=0x7fa198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec197a89491c377 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a0f4b700 1 -- 172.21.15.64:0/2165158648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa198004160 con 0x7fa19c108590 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a0f4b700 1 -- 172.21.15.64:0/2165158648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa198004300 con 0x7fa19c108590 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a0f4b700 1 -- 172.21.15.64:0/2165158648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa198011a80 con 0x7fa19c108590 2024-05-09T08:59:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a41b1700 1 -- 172.21.15.64:0/2165158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 msgr2=0x7fa19c10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a41b1700 1 --2- 172.21.15.64:0/2165158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c10a980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1980098a0 tx=0x7fa198009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a41b1700 1 -- 172.21.15.64:0/2165158648 shutdown_connections 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a41b1700 1 --2- 172.21.15.64:0/2165158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c10a980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a41b1700 1 -- 172.21.15.64:0/2165158648 >> 172.21.15.64:0/2165158648 conn(0x7fa19c101f60 msgr2=0x7fa19c1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.530+0000 7fa1a41b1700 1 -- 172.21.15.64:0/2165158648 shutdown_connections 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.531+0000 7fa1a41b1700 1 -- 172.21.15.64:0/2165158648 wait complete. 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.531+0000 7fa1a41b1700 1 Processor -- start 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.531+0000 7fa1a41b1700 1 -- start start 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.531+0000 7fa1a41b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c107e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.531+0000 7fa1a41b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c10be80 con 0x7fa19c108590 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.531+0000 7fa1a1f4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa1a1f4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36738/0 (socket says 172.21.15.64:36738) 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa1a1f4d700 1 -- 172.21.15.64:0/1724389101 learned_addr learned my addr 172.21.15.64:0/1724389101 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa1a1f4d700 1 -- 172.21.15.64:0/1724389101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c108350 con 0x7fa19c108590 2024-05-09T08:59:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa1a1f4d700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c107e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa198011a50 tx=0x7fa198011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa198003a80 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa198003c20 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19c106540 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa19801a650 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.532+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19c106a80 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.533+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa19801a990 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.533+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa19c063980 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.534+0000 7fa192ffd700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 0x7fa18803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.534+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa19804d520 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.534+0000 7fa1a174c700 1 -- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 msgr2=0x7fa18803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.534+0000 7fa1a174c700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 0x7fa18803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.539+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa198020240 con 0x7fa19c108590 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.735+0000 7fa1a174c700 1 -- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 msgr2=0x7fa18803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:03.442 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.735+0000 7fa1a174c700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 0x7fa18803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.819+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa19c1073b0 con 0x7fa19c108590 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.819+0000 7fa192ffd700 1 -- 172.21.15.64:0/1724389101 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa19800fdd0 con 0x7fa19c108590 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 msgr2=0x7fa18803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 0x7fa18803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 msgr2=0x7fa19c107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c107e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa198011a50 tx=0x7fa198011b30 comp rx=0 tx=0).stop 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 shutdown_connections 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7fa188039050 0x7fa18803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 --2- 172.21.15.64:0/1724389101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa19c108590 0x7fa19c107e10 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa198011a50 tx=0x7fa198011b30 comp rx=0 tx=0).stop 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.823+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 >> 172.21.15.64:0/1724389101 conn(0x7fa19c101f60 msgr2=0x7fa19c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.825+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 shutdown_connections 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:02.826+0000 7fa1a41b1700 1 -- 172.21.15.64:0/1724389101 wait complete. 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-05-09T08:59:03.443 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2024-05-09T08:59:03.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:03 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1724389101' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T08:59:03.790 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:03 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:03.670+0000 7fa1e97dc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:59:04.101 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:03 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:03.851+0000 7fa1e97dc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:59:04.101 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:04.030+0000 7fa1e97dc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:59:04.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:04.334+0000 7fa1e97dc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:59:04.839 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:04.430+0000 7fa1e97dc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:59:04.840 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:04.640+0000 7fa1e97dc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:59:05.152 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:04.840+0000 7fa1e97dc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:59:05.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:05.469+0000 7fa1e97dc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:59:06.215 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:05.907+0000 7fa1e97dc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:59:06.215 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:06.007+0000 7fa1e97dc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:59:06.215 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:06.215+0000 7fa1e97dc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:59:07.105 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:06.757+0000 7fa1e97dc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:59:07.105 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:06.853+0000 7fa1e97dc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:59:07.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:07.105+0000 7fa1e97dc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:59:07.403 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:07.244+0000 7fa1e97dc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:59:07.403 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:07.342+0000 7fa1e97dc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:59:07.773 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:07.463+0000 7fa1e97dc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:59:07.773 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:07.774+0000 7fa1e97dc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:59:08.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:08.402+0000 7fa1e97dc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:59:08.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:08.626+0000 7fa1e97dc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:59:09.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: Active manager daemon x restarted 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: Activating manager daemon x 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: mgrmap e6: x(active, starting, since 0.00330687s) 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: Manager daemon x is now available 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:09.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:08 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T08:59:09.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:09 smithi064 ceph-mon[30149]: Found migration_current of "None". Setting to last migration. 2024-05-09T08:59:09.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:09 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T08:59:09.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:09 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:09.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:09 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:09.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:09 smithi064 ceph-mon[30149]: mgrmap e7: x(active, since 1.00591s) 2024-05-09T08:59:10.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:59:10.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T08:59:10.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.896+0000 7f8b4d67a700 1 Processor -- start 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.896+0000 7f8b4d67a700 1 -- start start 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.896+0000 7f8b4d67a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b481067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.896+0000 7f8b4d67a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4807be50 con 0x7f8b48106390 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b46ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b481067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b46ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b481067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36740/0 (socket says 172.21.15.64:36740) 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b46ffd700 1 -- 172.21.15.64:0/1390100502 learned_addr learned my addr 172.21.15.64:0/1390100502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b46ffd700 1 -- 172.21.15.64:0/1390100502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b48106cf0 con 0x7f8b48106390 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b46ffd700 1 --2- 172.21.15.64:0/1390100502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b481067b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098a0 tx=0x7f8b38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dea52df2e24c3db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b45ffb700 1 -- 172.21.15.64:0/1390100502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b38004160 con 0x7f8b48106390 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b45ffb700 1 -- 172.21.15.64:0/1390100502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8b38004300 con 0x7f8b48106390 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.897+0000 7f8b45ffb700 1 -- 172.21.15.64:0/1390100502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b38011a80 con 0x7f8b48106390 2024-05-09T08:59:10.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 -- 172.21.15.64:0/1390100502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 msgr2=0x7f8b481067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/1390100502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b481067b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098a0 tx=0x7f8b38009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 -- 172.21.15.64:0/1390100502 shutdown_connections 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/1390100502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b481067b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 -- 172.21.15.64:0/1390100502 >> 172.21.15.64:0/1390100502 conn(0x7f8b48101f70 msgr2=0x7f8b48104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 -- 172.21.15.64:0/1390100502 shutdown_connections 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.898+0000 7f8b4d67a700 1 -- 172.21.15.64:0/1390100502 wait complete. 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b4d67a700 1 Processor -- start 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b4d67a700 1 -- start start 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b4d67a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b4819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b4d67a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48107d20 con 0x7f8b48106390 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b46ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b4819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b46ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b4819dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36748/0 (socket says 172.21.15.64:36748) 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.899+0000 7f8b46ffd700 1 -- 172.21.15.64:0/779767638 learned_addr learned my addr 172.21.15.64:0/779767638 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b46ffd700 1 -- 172.21.15.64:0/779767638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4819e130 con 0x7f8b48106390 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b46ffd700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b4819dbf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8b38011a50 tx=0x7f8b38004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b380039f0 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4819e3c0 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8b38003b90 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b3801a600 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.900+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a10b0 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.901+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8b3801a900 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.901+0000 7f8b3ffff700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.901+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 --> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b3403bc20 con 0x7f8b34039050 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.901+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b3804d550 con 0x7f8b48106390 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.901+0000 7f8b467fc700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 msgr2=0x7f8b3403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:03.901+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:04.102+0000 7f8b467fc700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 msgr2=0x7f8b3403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:04.102+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:04.503+0000 7f8b467fc700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 msgr2=0x7f8b3403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:04.503+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:05.304+0000 7f8b467fc700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 msgr2=0x7f8b3403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:10.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:05.304+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:06.905+0000 7f8b467fc700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 msgr2=0x7f8b3403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/421426620 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:06.905+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:08.629+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f8b3801abf0 con 0x7f8b48106390 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:08.629+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 msgr2=0x7f8b3403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:08.629+0000 7f8b3ffff700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.631+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f8b38024a40 con 0x7f8b48106390 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.631+0000 7f8b3ffff700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f8b3403c980 0x7f8b3403ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.631+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b38012ea0 con 0x7f8b3403c980 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.631+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f8b3403c980 0x7f8b3403ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.632+0000 7f8b467fc700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f8b3403c980 0x7f8b3403ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8b30003a10 tx=0x7f8b30009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.632+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8b38012ea0 con 0x7f8b3403c980 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.639+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8b48063980 con 0x7f8b3403c980 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.639+0000 7f8b3ffff700 1 -- 172.21.15.64:0/779767638 <== mgr.14120 v2:172.21.15.64:6800/615130791 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8b48063980 con 0x7f8b3403c980 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f8b3403c980 msgr2=0x7f8b3403ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f8b3403c980 0x7f8b3403ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8b30003a10 tx=0x7f8b30009290 comp rx=0 tx=0).stop 2024-05-09T08:59:10.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 msgr2=0x7f8b4819dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b4819dbf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8b38011a50 tx=0x7f8b38004290 comp rx=0 tx=0).stop 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 shutdown_connections 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f8b3403c980 0x7f8b3403ed70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48106390 0x7f8b4819dbf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 --2- 172.21.15.64:0/779767638 >> [v2:172.21.15.64:6800/421426620,v1:172.21.15.64:6801/421426620] conn(0x7f8b34039050 0x7f8b3403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 >> 172.21.15.64:0/779767638 conn(0x7f8b48101f70 msgr2=0x7f8b48194850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 shutdown_connections 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:09.640+0000 7f8b4d67a700 1 -- 172.21.15.64:0/779767638 wait complete. 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2024-05-09T08:59:10.234 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: [09/May/2024:08:59:10] ENGINE Bus STARTING 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: [09/May/2024:08:59:10] ENGINE Serving on https://172.21.15.64:7150 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: [09/May/2024:08:59:10] ENGINE Serving on http://172.21.15.64:8765 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: [09/May/2024:08:59:10] ENGINE Bus STARTED 2024-05-09T08:59:10.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:10 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:11.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b21123700 1 Processor -- start 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b21123700 1 -- start start 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b21123700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c1061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b21123700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c1067b0 con 0x7f6b1c105dc0 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b1ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c1061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b1ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c1061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58790/0 (socket says 172.21.15.64:58790) 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b1ad9d700 1 -- 172.21.15.64:0/3958211380 learned_addr learned my addr 172.21.15.64:0/3958211380 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b1ad9d700 1 -- 172.21.15.64:0/3958211380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c108d50 con 0x7f6b1c105dc0 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.653+0000 7f6b1ad9d700 1 --2- 172.21.15.64:0/3958211380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c1061e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0098a0 tx=0x7f6b0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276ffc473477ed46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b19d9b700 1 -- 172.21.15.64:0/3958211380 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b0c004160 con 0x7f6b1c105dc0 2024-05-09T08:59:11.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b19d9b700 1 -- 172.21.15.64:0/3958211380 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6b0c004300 con 0x7f6b1c105dc0 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b19d9b700 1 -- 172.21.15.64:0/3958211380 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b0c011a80 con 0x7f6b1c105dc0 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 -- 172.21.15.64:0/3958211380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 msgr2=0x7f6b1c1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 --2- 172.21.15.64:0/3958211380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c1061e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0098a0 tx=0x7f6b0c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 -- 172.21.15.64:0/3958211380 shutdown_connections 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 --2- 172.21.15.64:0/3958211380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c1061e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 -- 172.21.15.64:0/3958211380 >> 172.21.15.64:0/3958211380 conn(0x7f6b1c1016d0 msgr2=0x7f6b1c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 -- 172.21.15.64:0/3958211380 shutdown_connections 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.654+0000 7f6b21123700 1 -- 172.21.15.64:0/3958211380 wait complete. 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.655+0000 7f6b21123700 1 Processor -- start 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.655+0000 7f6b21123700 1 -- start start 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.655+0000 7f6b21123700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.655+0000 7f6b21123700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c10c100 con 0x7f6b1c105dc0 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.655+0000 7f6b1ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b1ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58792/0 (socket says 172.21.15.64:58792) 2024-05-09T08:59:11.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b1ad9d700 1 -- 172.21.15.64:0/52616357 learned_addr learned my addr 172.21.15.64:0/52616357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b1ad9d700 1 -- 172.21.15.64:0/52616357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c19bf00 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b1ad9d700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c19b9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c00 tx=0x7f6b0c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b0c0039f0 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6b0c003b90 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c19c190 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b0c01a740 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.656+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c19ee80 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.657+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f6b0c01a9f0 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.657+0000 7f6b03fff700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f6b04038f80 0x7f6b0403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.657+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b0c04d1e0 con 0x7f6b1c105dc0 2024-05-09T08:59:11.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.657+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c063980 con 0x7f6b1c105dc0 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.657+0000 7f6b1a59c700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f6b04038f80 0x7f6b0403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.658+0000 7f6b1a59c700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f6b04038f80 0x7f6b0403b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006fd0 tx=0x7f6b10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.663+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6b0c020240 con 0x7f6b1c105dc0 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.682+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6b0c02d910 con 0x7f6b1c105dc0 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.889+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6b1c102830 con 0x7f6b04038f80 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.895+0000 7f6b03fff700 1 -- 172.21.15.64:0/52616357 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b1c102830 con 0x7f6b04038f80 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.897+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f6b04038f80 msgr2=0x7f6b0403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.897+0000 7f6b21123700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f6b04038f80 0x7f6b0403b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006fd0 tx=0x7f6b10006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.897+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 msgr2=0x7f6b1c19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.897+0000 7f6b21123700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c19b9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c00 tx=0x7f6b0c011b00 comp rx=0 tx=0).stop 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.898+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 shutdown_connections 2024-05-09T08:59:11.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.898+0000 7f6b21123700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f6b04038f80 0x7f6b0403b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:11.438 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.898+0000 7f6b21123700 1 --2- 172.21.15.64:0/52616357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b1c105dc0 0x7f6b1c19b9c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:11.438 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.898+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 >> 172.21.15.64:0/52616357 conn(0x7f6b1c1016d0 msgr2=0x7f6b1c102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:11.438 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.898+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 shutdown_connections 2024-05-09T08:59:11.438 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:10.898+0000 7f6b21123700 1 -- 172.21.15.64:0/52616357 wait complete. 2024-05-09T08:59:11.844 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:11 smithi064 ceph-mon[30149]: mgrmap e8: x(active, since 2s) 2024-05-09T08:59:11.844 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:11 smithi064 ceph-mon[30149]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:11.845 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:11 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:11.845 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:11 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.882+0000 7f434a33b700 1 Processor -- start 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.882+0000 7f434a33b700 1 -- start start 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f434a33b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434410a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f434a33b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f434407bdf0 con 0x7f43441085d0 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f4343fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434410a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f4343fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434410a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58804/0 (socket says 172.21.15.64:58804) 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f4343fff700 1 -- 172.21.15.64:0/917677807 learned_addr learned my addr 172.21.15.64:0/917677807 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f4343fff700 1 -- 172.21.15.64:0/917677807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434410af00 con 0x7f43441085d0 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.883+0000 7f4343fff700 1 --2- 172.21.15.64:0/917677807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434410a9c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43340098a0 tx=0x7f4334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feb2c5fb1b16d26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f4342ffd700 1 -- 172.21.15.64:0/917677807 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4334004160 con 0x7f43441085d0 2024-05-09T08:59:12.623 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f4342ffd700 1 -- 172.21.15.64:0/917677807 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4334004300 con 0x7f43441085d0 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f4342ffd700 1 -- 172.21.15.64:0/917677807 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4334011a80 con 0x7f43441085d0 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 -- 172.21.15.64:0/917677807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 msgr2=0x7f434410a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 --2- 172.21.15.64:0/917677807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434410a9c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43340098a0 tx=0x7f4334009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 -- 172.21.15.64:0/917677807 shutdown_connections 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 --2- 172.21.15.64:0/917677807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434410a9c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 -- 172.21.15.64:0/917677807 >> 172.21.15.64:0/917677807 conn(0x7f4344101f60 msgr2=0x7f4344104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 -- 172.21.15.64:0/917677807 shutdown_connections 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.884+0000 7f434a33b700 1 -- 172.21.15.64:0/917677807 wait complete. 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f434a33b700 1 Processor -- start 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f434a33b700 1 -- start start 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f434a33b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f434a33b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f434410be00 con 0x7f43441085d0 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f4343fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f4343fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58816/0 (socket says 172.21.15.64:58816) 2024-05-09T08:59:12.624 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.885+0000 7f4343fff700 1 -- 172.21.15.64:0/3640815189 learned_addr learned my addr 172.21.15.64:0/3640815189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f4343fff700 1 -- 172.21.15.64:0/3640815189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419beb0 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f4343fff700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434419b970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4334011a50 tx=0x7f4334011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4334003a80 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4334003c20 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434419c140 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433401a650 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.886+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434419ee30 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.887+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f433401a900 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.887+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4344063980 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.887+0000 7f43417fa700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f432c0390a0 0x7f432c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.887+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f433404d660 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.887+0000 7f43437fe700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f432c0390a0 0x7f432c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.888+0000 7f43437fe700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f432c0390a0 0x7f432c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4338006fd0 tx=0x7f4338006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:11.892+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4334019c70 con 0x7f43441085d0 2024-05-09T08:59:12.625 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.101+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4344103110 con 0x7f432c0390a0 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.101+0000 7f43417fa700 1 -- 172.21.15.64:0/3640815189 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4344103110 con 0x7f432c0390a0 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f432c0390a0 msgr2=0x7f432c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f432c0390a0 0x7f432c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4338006fd0 tx=0x7f4338006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 msgr2=0x7f434419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434419b970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4334011a50 tx=0x7f4334011b30 comp rx=0 tx=0).stop 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 shutdown_connections 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f432c0390a0 0x7f432c03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 --2- 172.21.15.64:0/3640815189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43441085d0 0x7f434419b970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 >> 172.21.15.64:0/3640815189 conn(0x7f4344101f60 msgr2=0x7f4344102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 shutdown_connections 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:12.104+0000 7f434a33b700 1 -- 172.21.15.64:0/3640815189 wait complete. 2024-05-09T08:59:12.626 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2024-05-09T08:59:12.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:12 smithi064 ceph-mon[30149]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:13.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: Generating public/private rsa key pair. 2024-05-09T08:59:13.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: Your identification has been saved in /tmp/tmp0ay3ttvo/key. 2024-05-09T08:59:13.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: Your public key has been saved in /tmp/tmp0ay3ttvo/key.pub. 2024-05-09T08:59:13.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: The key fingerprint is: 2024-05-09T08:59:13.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: SHA256:4LOZVGn4cTV7kfsmBvzJ2UP2dyAvIj/yCpmM/4T1Q/0 ceph-2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:59:13.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: The key's randomart image is: 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: +---[RSA 3072]----+ 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | o .. | 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | . . . o.. | 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | o = ... .. | 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | . = o .+.oo | 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | + S . .*o*.| 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | + X.o. ..O.B| 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | . X .oo. oEo+| 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | . o. o. | 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: | ..o+.. | 2024-05-09T08:59:13.654 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: +----[SHA256]-----+ 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.048+0000 7f98d4717700 1 Processor -- start 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.048+0000 7f98d4717700 1 -- start start 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d4717700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d4717700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1089a0 con 0x7f98cc107fb0 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d24b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d24b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58832/0 (socket says 172.21.15.64:58832) 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d24b3700 1 -- 172.21.15.64:0/948179742 learned_addr learned my addr 172.21.15.64:0/948179742 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d24b3700 1 -- 172.21.15.64:0/948179742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc106bc0 con 0x7f98cc107fb0 2024-05-09T08:59:13.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d24b3700 1 --2- 172.21.15.64:0/948179742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc1083d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098a0 tx=0x7f98c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d080c97e2f1119 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d14b1700 1 -- 172.21.15.64:0/948179742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0004160 con 0x7f98cc107fb0 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d14b1700 1 -- 172.21.15.64:0/948179742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f98c0004300 con 0x7f98cc107fb0 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.049+0000 7f98d14b1700 1 -- 172.21.15.64:0/948179742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0011a80 con 0x7f98cc107fb0 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 -- 172.21.15.64:0/948179742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 msgr2=0x7f98cc1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 --2- 172.21.15.64:0/948179742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc1083d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098a0 tx=0x7f98c0009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 -- 172.21.15.64:0/948179742 shutdown_connections 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 --2- 172.21.15.64:0/948179742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc1083d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 -- 172.21.15.64:0/948179742 >> 172.21.15.64:0/948179742 conn(0x7f98cc1016d0 msgr2=0x7f98cc103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 -- 172.21.15.64:0/948179742 shutdown_connections 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.050+0000 7f98d4717700 1 -- 172.21.15.64:0/948179742 wait complete. 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d4717700 1 Processor -- start 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d4717700 1 -- start start 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d4717700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d4717700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10af80 con 0x7f98cc107fb0 2024-05-09T08:59:13.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d24b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d24b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58836/0 (socket says 172.21.15.64:58836) 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d24b3700 1 -- 172.21.15.64:0/265847314 learned_addr learned my addr 172.21.15.64:0/265847314 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d24b3700 1 -- 172.21.15.64:0/265847314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19bf60 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.051+0000 7f98d24b3700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc19ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98c0000c00 tx=0x7f98c0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.052+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c00039f0 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.052+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f98c0003b90 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.052+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c001a740 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.052+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19c1f0 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.052+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc19eee0 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.052+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f98c0024af0 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.053+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc063980 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.053+0000 7f98bf7fe700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f98b8039050 0x7f98b803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.053+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f98c004d400 con 0x7f98cc107fb0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.053+0000 7f98d1cb2700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f98b8039050 0x7f98b803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:13.947 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.053+0000 7f98d1cb2700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f98b8039050 0x7f98b803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c8006fd0 tx=0x7f98c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.058+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f98c000f2e0 con 0x7f98cc107fb0 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.264+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f98cc0008d0 con 0x7f98b8039050 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.444+0000 7f98bf7fe700 1 -- 172.21.15.64:0/265847314 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98cc0008d0 con 0x7f98b8039050 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.446+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f98b8039050 msgr2=0x7f98b803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.446+0000 7f98d4717700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f98b8039050 0x7f98b803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c8006fd0 tx=0x7f98c8006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.446+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 msgr2=0x7f98cc19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.446+0000 7f98d4717700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc19ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98c0000c00 tx=0x7f98c0011b00 comp rx=0 tx=0).stop 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.447+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 shutdown_connections 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.447+0000 7f98d4717700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f98b8039050 0x7f98b803b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.447+0000 7f98d4717700 1 --2- 172.21.15.64:0/265847314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc107fb0 0x7f98cc19ba20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.447+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 >> 172.21.15.64:0/265847314 conn(0x7f98cc1016d0 msgr2=0x7f98cc102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.447+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 shutdown_connections 2024-05-09T08:59:13.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:13.447+0000 7f98d4717700 1 -- 172.21.15.64:0/265847314 wait complete. 2024-05-09T08:59:14.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:14 smithi064 ceph-mon[30149]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:14.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:14 smithi064 ceph-mon[30149]: Generating ssh key... 2024-05-09T08:59:14.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:14 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:14.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:14 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:15.158 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f99364d0700 1 Processor -- start 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f99364d0700 1 -- start start 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f99364d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f9930107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f99364d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9930108520 con 0x7f9930107b30 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f992ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f9930107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f992ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f9930107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58844/0 (socket says 172.21.15.64:58844) 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.389+0000 7f992ffff700 1 -- 172.21.15.64:0/1519064449 learned_addr learned my addr 172.21.15.64:0/1519064449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f992ffff700 1 -- 172.21.15.64:0/1519064449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9930108c90 con 0x7f9930107b30 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f992ffff700 1 --2- 172.21.15.64:0/1519064449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f9930107f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99200098a0 tx=0x7f9920009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c087308dd4efc760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f992effd700 1 -- 172.21.15.64:0/1519064449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9920004160 con 0x7f9930107b30 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f992effd700 1 -- 172.21.15.64:0/1519064449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9920004300 con 0x7f9930107b30 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f992effd700 1 -- 172.21.15.64:0/1519064449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9920011a80 con 0x7f9930107b30 2024-05-09T08:59:15.159 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f99364d0700 1 -- 172.21.15.64:0/1519064449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 msgr2=0x7f9930107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f99364d0700 1 --2- 172.21.15.64:0/1519064449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f9930107f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99200098a0 tx=0x7f9920009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f99364d0700 1 -- 172.21.15.64:0/1519064449 shutdown_connections 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.390+0000 7f99364d0700 1 --2- 172.21.15.64:0/1519064449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f9930107f50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.391+0000 7f99364d0700 1 -- 172.21.15.64:0/1519064449 >> 172.21.15.64:0/1519064449 conn(0x7f99301030d0 msgr2=0x7f9930105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.391+0000 7f99364d0700 1 -- 172.21.15.64:0/1519064449 shutdown_connections 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.391+0000 7f99364d0700 1 -- 172.21.15.64:0/1519064449 wait complete. 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.391+0000 7f99364d0700 1 Processor -- start 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.391+0000 7f99364d0700 1 -- start start 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f99364d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f993019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f99364d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f993010c040 con 0x7f9930107b30 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f993019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f993019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58856/0 (socket says 172.21.15.64:58856) 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992ffff700 1 -- 172.21.15.64:0/1955187186 learned_addr learned my addr 172.21.15.64:0/1955187186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992ffff700 1 -- 172.21.15.64:0/1955187186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993019bfb0 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992ffff700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f993019ba70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9920000c00 tx=0x7f9920011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99200039f0 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9920003b90 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993019c240 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f992001a740 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.392+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993019eee0 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.393+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f992001a9f0 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.393+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9930063980 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.393+0000 7f992d7fa700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f99180390a0 0x7f991803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.393+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f992004d390 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.394+0000 7f992f7fe700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f99180390a0 0x7f991803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.394+0000 7f992f7fe700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f99180390a0 0x7f991803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9924006fd0 tx=0x7f9924006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.399+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9920020240 con 0x7f9930107b30 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.605+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9930104340 con 0x7f99180390a0 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.606+0000 7f992d7fa700 1 -- 172.21.15.64:0/1955187186 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9930104340 con 0x7f99180390a0 2024-05-09T08:59:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f99180390a0 msgr2=0x7f991803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f99180390a0 0x7f991803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9924006fd0 tx=0x7f9924006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 msgr2=0x7f993019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f993019ba70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9920000c00 tx=0x7f9920011b00 comp rx=0 tx=0).stop 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 shutdown_connections 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f99180390a0 0x7f991803b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 --2- 172.21.15.64:0/1955187186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9930107b30 0x7f993019ba70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 >> 172.21.15.64:0/1955187186 conn(0x7f99301030d0 msgr2=0x7f9930103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 shutdown_connections 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:14.608+0000 7f99364d0700 1 -- 172.21.15.64:0/1955187186 wait complete. 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T08:59:15.162 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2024-05-09T08:59:15.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:15 smithi064 ceph-mon[30149]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:16.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:16 smithi064 ceph-mon[30149]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:16.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:16 smithi064 ceph-mon[30149]: Deploying cephadm binary to smithi064 2024-05-09T08:59:20.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2024-05-09T08:59:20.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06dc543700 1 Processor -- start 2024-05-09T08:59:20.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06dc543700 1 -- start start 2024-05-09T08:59:20.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06dc543700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:20.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06dc543700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06d407c880 con 0x7f06d407de40 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06da2df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06da2df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40076/0 (socket says 172.21.15.64:40076) 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.564+0000 7f06da2df700 1 -- 172.21.15.64:0/2653747091 learned_addr learned my addr 172.21.15.64:0/2653747091 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.565+0000 7f06da2df700 1 -- 172.21.15.64:0/2653747091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d407ca00 con 0x7f06d407de40 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.565+0000 7f06da2df700 1 --2- 172.21.15.64:0/2653747091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d407c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f06d00098a0 tx=0x7f06d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86db99925b127013 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.565+0000 7f06d92dd700 1 -- 172.21.15.64:0/2653747091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d0004160 con 0x7f06d407de40 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.565+0000 7f06d92dd700 1 -- 172.21.15.64:0/2653747091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f06d0004300 con 0x7f06d407de40 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.565+0000 7f06d92dd700 1 -- 172.21.15.64:0/2653747091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d0011a80 con 0x7f06d407de40 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 -- 172.21.15.64:0/2653747091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 msgr2=0x7f06d407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 --2- 172.21.15.64:0/2653747091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d407c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f06d00098a0 tx=0x7f06d0009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 -- 172.21.15.64:0/2653747091 shutdown_connections 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 --2- 172.21.15.64:0/2653747091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d407c340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 -- 172.21.15.64:0/2653747091 >> 172.21.15.64:0/2653747091 conn(0x7f06d41030d0 msgr2=0x7f06d4105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:20.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 -- 172.21.15.64:0/2653747091 shutdown_connections 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 -- 172.21.15.64:0/2653747091 wait complete. 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 Processor -- start 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.566+0000 7f06dc543700 1 -- start start 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06dc543700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06dc543700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06d410e060 con 0x7f06d407de40 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06da2df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06da2df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40088/0 (socket says 172.21.15.64:40088) 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06da2df700 1 -- 172.21.15.64:0/2449661769 learned_addr learned my addr 172.21.15.64:0/2449661769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06da2df700 1 -- 172.21.15.64:0/2449661769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d419e0f0 con 0x7f06d407de40 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06da2df700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d419dbb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006af0 tx=0x7f06d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d00039e0 con 0x7f06d407de40 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f06d0003b80 con 0x7f06d407de40 2024-05-09T08:59:20.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d419e380 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d00196a0 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.567+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d41a1070 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.568+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f06d001a430 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.568+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d4063980 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.568+0000 7f06cb7fe700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f06c0039000 0x7f06c003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.568+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f06d004d620 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.569+0000 7f06d9ade700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f06c0039000 0x7f06c003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.569+0000 7f06d9ade700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f06c0039000 0x7f06c003b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f06c4006fd0 tx=0x7f06c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.574+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f06d002de00 con 0x7f06d407de40 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:15.787+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}) -- 0x7f06d41042b0 con 0x7f06c0039000 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.086+0000 7f06cb7fe700 1 -- 172.21.15.64:0/2449661769 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f06d41042b0 con 0x7f06c0039000 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f06c0039000 msgr2=0x7f06c003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f06c0039000 0x7f06c003b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f06c4006fd0 tx=0x7f06c4006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 msgr2=0x7f06d419dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d419dbb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006af0 tx=0x7f06d0011c40 comp rx=0 tx=0).stop 2024-05-09T08:59:20.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 shutdown_connections 2024-05-09T08:59:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f06c0039000 0x7f06c003b4c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 --2- 172.21.15.64:0/2449661769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d407de40 0x7f06d419dbb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.088+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 >> 172.21.15.64:0/2449661769 conn(0x7f06d41030d0 msgr2=0x7f06d4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.089+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 shutdown_connections 2024-05-09T08:59:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:20.089+0000 7f06dc543700 1 -- 172.21.15.64:0/2449661769 wait complete. 2024-05-09T08:59:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2024-05-09T08:59:21.351 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:21 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:21.352 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:21 smithi064 ceph-mon[30149]: Added host smithi064 2024-05-09T08:59:21.352 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:21 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f9125edb700 1 Processor -- start 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f9125edb700 1 -- start start 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f9125edb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f912010a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f9125edb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f912007be70 con 0x7f9120108540 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f912010a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f912010a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40092/0 (socket says 172.21.15.64:40092) 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f911f7fe700 1 -- 172.21.15.64:0/1215915256 learned_addr learned my addr 172.21.15.64:0/1215915256 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:21.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.122+0000 7f911f7fe700 1 -- 172.21.15.64:0/1215915256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912010ae70 con 0x7f9120108540 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f911f7fe700 1 --2- 172.21.15.64:0/1215915256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f912010a930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91100098a0 tx=0x7f9110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290964d4ab509051 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f911e7fc700 1 -- 172.21.15.64:0/1215915256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9110004160 con 0x7f9120108540 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f911e7fc700 1 -- 172.21.15.64:0/1215915256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9110004300 con 0x7f9120108540 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f911e7fc700 1 -- 172.21.15.64:0/1215915256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9110011a80 con 0x7f9120108540 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 -- 172.21.15.64:0/1215915256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 msgr2=0x7f912010a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 --2- 172.21.15.64:0/1215915256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f912010a930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91100098a0 tx=0x7f9110009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 -- 172.21.15.64:0/1215915256 shutdown_connections 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 --2- 172.21.15.64:0/1215915256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f912010a930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 -- 172.21.15.64:0/1215915256 >> 172.21.15.64:0/1215915256 conn(0x7f9120101f50 msgr2=0x7f9120104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 -- 172.21.15.64:0/1215915256 shutdown_connections 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.123+0000 7f9125edb700 1 -- 172.21.15.64:0/1215915256 wait complete. 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.124+0000 7f9125edb700 1 Processor -- start 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.124+0000 7f9125edb700 1 -- start start 2024-05-09T08:59:21.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.124+0000 7f9125edb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f9120107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.124+0000 7f9125edb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f912010bde0 con 0x7f9120108540 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.124+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f9120107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f9120107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40104/0 (socket says 172.21.15.64:40104) 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911f7fe700 1 -- 172.21.15.64:0/3087343782 learned_addr learned my addr 172.21.15.64:0/3087343782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911f7fe700 1 -- 172.21.15.64:0/3087343782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120108300 con 0x7f9120108540 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911f7fe700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f9120107dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9110011a50 tx=0x7f9110011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9110003a80 con 0x7f9120108540 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9110003c20 con 0x7f9120108540 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91201064f0 con 0x7f9120108540 2024-05-09T08:59:21.963 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f911001a650 con 0x7f9120108540 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.125+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9120106a30 con 0x7f9120108540 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.126+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f911001a920 con 0x7f9120108540 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.126+0000 7f911cff9700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f91080390a0 0x7f910803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.126+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f911004d4b0 con 0x7f9120108540 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.126+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9120063980 con 0x7f9120108540 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.126+0000 7f911effd700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f91080390a0 0x7f910803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.127+0000 7f911effd700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f91080390a0 0x7f910803b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9114006fd0 tx=0x7f9114006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.131+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9110019c70 con 0x7f9120108540 2024-05-09T08:59:21.964 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.351+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f91201071b0 con 0x7f91080390a0 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.356+0000 7f911cff9700 1 -- 172.21.15.64:0/3087343782 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f91201071b0 con 0x7f91080390a0 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f91080390a0 msgr2=0x7f910803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f91080390a0 0x7f910803b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9114006fd0 tx=0x7f9114006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 msgr2=0x7f9120107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f9120107dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9110011a50 tx=0x7f9110011b30 comp rx=0 tx=0).stop 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 shutdown_connections 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f91080390a0 0x7f910803b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 --2- 172.21.15.64:0/3087343782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9120108540 0x7f9120107dc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 >> 172.21.15.64:0/3087343782 conn(0x7f9120101f50 msgr2=0x7f91201029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 shutdown_connections 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:21.358+0000 7f9125edb700 1 -- 172.21.15.64:0/3087343782 wait complete. 2024-05-09T08:59:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2024-05-09T08:59:22.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:22 smithi064 ceph-mon[30149]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:22.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:22 smithi064 ceph-mon[30149]: Saving service mon spec with placement count:5 2024-05-09T08:59:22.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:22 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:23.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:23 smithi064 ceph-mon[30149]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:23.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:23 smithi064 ceph-mon[30149]: Saving service mgr spec with placement count:2 2024-05-09T08:59:23.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:23 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:23.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:23 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.421+0000 7f90882e9700 1 Processor -- start 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f90882e9700 1 -- start start 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f90882e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908010a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f90882e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f908007be90 con 0x7f9080108590 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f9086085700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908010a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f9086085700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908010a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40110/0 (socket says 172.21.15.64:40110) 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f9086085700 1 -- 172.21.15.64:0/571485743 learned_addr learned my addr 172.21.15.64:0/571485743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.422+0000 7f9086085700 1 -- 172.21.15.64:0/571485743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908010aec0 con 0x7f9080108590 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f9086085700 1 --2- 172.21.15.64:0/571485743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908010a980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f907c0098a0 tx=0x7f907c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540b6957223554f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f9085083700 1 -- 172.21.15.64:0/571485743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c004160 con 0x7f9080108590 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f9085083700 1 -- 172.21.15.64:0/571485743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f907c004300 con 0x7f9080108590 2024-05-09T08:59:23.935 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f9085083700 1 -- 172.21.15.64:0/571485743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c011a80 con 0x7f9080108590 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f90882e9700 1 -- 172.21.15.64:0/571485743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 msgr2=0x7f908010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f90882e9700 1 --2- 172.21.15.64:0/571485743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908010a980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f907c0098a0 tx=0x7f907c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f90882e9700 1 -- 172.21.15.64:0/571485743 shutdown_connections 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f90882e9700 1 --2- 172.21.15.64:0/571485743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908010a980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.423+0000 7f90882e9700 1 -- 172.21.15.64:0/571485743 >> 172.21.15.64:0/571485743 conn(0x7f9080101f60 msgr2=0x7f90801043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f90882e9700 1 -- 172.21.15.64:0/571485743 shutdown_connections 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f90882e9700 1 -- 172.21.15.64:0/571485743 wait complete. 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f90882e9700 1 Processor -- start 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f90882e9700 1 -- start start 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f90882e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f90882e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f908010be80 con 0x7f9080108590 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f9086085700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f9086085700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40126/0 (socket says 172.21.15.64:40126) 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.424+0000 7f9086085700 1 -- 172.21.15.64:0/2606567628 learned_addr learned my addr 172.21.15.64:0/2606567628 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f9086085700 1 -- 172.21.15.64:0/2606567628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908019bee0 con 0x7f9080108590 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f9086085700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908019b9a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f907c006af0 tx=0x7f907c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:23.936 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c0039e0 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f907c003b80 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908019c170 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c0196a0 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.425+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908019ee60 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.426+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9080063980 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.426+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f907c01a430 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.426+0000 7f90777fe700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f906c039000 0x7f906c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.427+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f907c04d620 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.427+0000 7f9085884700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f906c039000 0x7f906c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.427+0000 7f9085884700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f906c039000 0x7f906c03b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9070006fd0 tx=0x7f9070006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.431+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f907c02de00 con 0x7f9080108590 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.656+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9080103130 con 0x7f906c039000 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.660+0000 7f90777fe700 1 -- 172.21.15.64:0/2606567628 <== mgr.14120 v2:172.21.15.64:6800/615130791 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9080103130 con 0x7f906c039000 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f906c039000 msgr2=0x7f906c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f906c039000 0x7f906c03b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9070006fd0 tx=0x7f9070006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 msgr2=0x7f908019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908019b9a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f907c006af0 tx=0x7f907c011c40 comp rx=0 tx=0).stop 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 shutdown_connections 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f906c039000 0x7f906c03b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 --2- 172.21.15.64:0/2606567628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9080108590 0x7f908019b9a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 >> 172.21.15.64:0/2606567628 conn(0x7f9080101f60 msgr2=0x7f9080102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 shutdown_connections 2024-05-09T08:59:23.938 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:22.662+0000 7f90882e9700 1 -- 172.21.15.64:0/2606567628 wait complete. 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2d4f69700 1 Processor -- start 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2d4f69700 1 -- start start 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2d4f69700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d0108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2d4f69700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0108a00 con 0x7ff2d0108010 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2ce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d0108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2ce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d0108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40134/0 (socket says 172.21.15.64:40134) 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2ce59c700 1 -- 172.21.15.64:0/1404225729 learned_addr learned my addr 172.21.15.64:0/1404225729 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.353+0000 7ff2ce59c700 1 -- 172.21.15.64:0/1404225729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d0105b10 con 0x7ff2d0108010 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.354+0000 7ff2ce59c700 1 --2- 172.21.15.64:0/1404225729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d0108430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00098a0 tx=0x7ff2c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d3121819463123 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.354+0000 7ff2cd59a700 1 -- 172.21.15.64:0/1404225729 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0004160 con 0x7ff2d0108010 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.354+0000 7ff2cd59a700 1 -- 172.21.15.64:0/1404225729 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2c0004300 con 0x7ff2d0108010 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.354+0000 7ff2cd59a700 1 -- 172.21.15.64:0/1404225729 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0011a50 con 0x7ff2d0108010 2024-05-09T08:59:25.106 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.354+0000 7ff2d4f69700 1 -- 172.21.15.64:0/1404225729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 msgr2=0x7ff2d0108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.354+0000 7ff2d4f69700 1 --2- 172.21.15.64:0/1404225729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d0108430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00098a0 tx=0x7ff2c0009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 -- 172.21.15.64:0/1404225729 shutdown_connections 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 --2- 172.21.15.64:0/1404225729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d0108430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 -- 172.21.15.64:0/1404225729 >> 172.21.15.64:0/1404225729 conn(0x7ff2d0101710 msgr2=0x7ff2d0103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 -- 172.21.15.64:0/1404225729 shutdown_connections 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 -- 172.21.15.64:0/1404225729 wait complete. 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 Processor -- start 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.355+0000 7ff2d4f69700 1 -- start start 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2d4f69700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2d4f69700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0106b70 con 0x7ff2d0108010 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2ce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2ce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40150/0 (socket says 172.21.15.64:40150) 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2ce59c700 1 -- 172.21.15.64:0/3425563954 learned_addr learned my addr 172.21.15.64:0/3425563954 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2ce59c700 1 -- 172.21.15.64:0/3425563954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d019e060 con 0x7ff2d0108010 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2ce59c700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d019db20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0011a20 tx=0x7ff2c0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:25.107 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0003b70 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2c001a8e0 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d019e2f0 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.356+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0024990 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.357+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d01a0fe0 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.357+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff2c0022020 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.357+0000 7ff2bf7fe700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff2b8039000 0x7ff2b803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.357+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff2c004d3d0 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.358+0000 7ff2cdd9b700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff2b8039000 0x7ff2b803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.358+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d0063980 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.358+0000 7ff2cdd9b700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff2b8039000 0x7ff2b803b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006fd0 tx=0x7ff2c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.363+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2c000fdd0 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.578+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff2d0197120 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.581+0000 7ff2bf7fe700 1 -- 172.21.15.64:0/3425563954 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff2c002d910 con 0x7ff2d0108010 2024-05-09T08:59:25.108 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff2b8039000 msgr2=0x7ff2b803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff2b8039000 0x7ff2b803b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006fd0 tx=0x7ff2c4006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 msgr2=0x7ff2d019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d019db20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0011a20 tx=0x7ff2c0011b00 comp rx=0 tx=0).stop 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 shutdown_connections 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff2b8039000 0x7ff2b803b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 --2- 172.21.15.64:0/3425563954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d0108010 0x7ff2d019db20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.583+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 >> 172.21.15.64:0/3425563954 conn(0x7ff2d0101710 msgr2=0x7ff2d0194200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.584+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 shutdown_connections 2024-05-09T08:59:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:24.584+0000 7ff2d4f69700 1 -- 172.21.15.64:0/3425563954 wait complete. 2024-05-09T08:59:25.304 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:25 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:25.304 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:25 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3425563954' entity='client.admin' 2024-05-09T08:59:26.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.535+0000 7fc0091dd700 1 Processor -- start 2024-05-09T08:59:26.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.535+0000 7fc0091dd700 1 -- start start 2024-05-09T08:59:26.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.535+0000 7fc0091dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00410a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.535+0000 7fc0091dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc00407be70 con 0x7fc004108550 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00410a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00410a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56848/0 (socket says 172.21.15.64:56848) 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc002d9d700 1 -- 172.21.15.64:0/1249360344 learned_addr learned my addr 172.21.15.64:0/1249360344 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc002d9d700 1 -- 172.21.15.64:0/1249360344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00410ae80 con 0x7fc004108550 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc002d9d700 1 --2- 172.21.15.64:0/1249360344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00410a940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbff40098a0 tx=0x7fbff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1460e8a97c025b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc001d9b700 1 -- 172.21.15.64:0/1249360344 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff4004160 con 0x7fc004108550 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc001d9b700 1 -- 172.21.15.64:0/1249360344 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff4004300 con 0x7fc004108550 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.536+0000 7fc001d9b700 1 -- 172.21.15.64:0/1249360344 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff4011a50 con 0x7fc004108550 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 -- 172.21.15.64:0/1249360344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 msgr2=0x7fc00410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 --2- 172.21.15.64:0/1249360344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00410a940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbff40098a0 tx=0x7fbff4009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 -- 172.21.15.64:0/1249360344 shutdown_connections 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 --2- 172.21.15.64:0/1249360344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00410a940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 -- 172.21.15.64:0/1249360344 >> 172.21.15.64:0/1249360344 conn(0x7fc004101f60 msgr2=0x7fc004104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 -- 172.21.15.64:0/1249360344 shutdown_connections 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.537+0000 7fc0091dd700 1 -- 172.21.15.64:0/1249360344 wait complete. 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc0091dd700 1 Processor -- start 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc0091dd700 1 -- start start 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc0091dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc0091dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc00410be30 con 0x7fc004108550 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56860/0 (socket says 172.21.15.64:56860) 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc002d9d700 1 -- 172.21.15.64:0/3320989165 learned_addr learned my addr 172.21.15.64:0/3320989165 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.538+0000 7fc002d9d700 1 -- 172.21.15.64:0/3320989165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00419be90 con 0x7fc004108550 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.539+0000 7fc002d9d700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00419b950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbff4006b20 tx=0x7fbff4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.539+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff4003b70 con 0x7fc004108550 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.539+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff401a8e0 con 0x7fc004108550 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.539+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00419c120 con 0x7fc004108550 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.539+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbff40249b0 con 0x7fc004108550 2024-05-09T08:59:26.401 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.539+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00419ee10 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.540+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbff4022020 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.540+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc004063980 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.540+0000 7fbffbfff700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbff0039050 0x7fbff003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.540+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbff404d7c0 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.540+0000 7fc00259c700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbff0039050 0x7fbff003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.541+0000 7fc00259c700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbff0039050 0x7fbff003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006fd0 tx=0x7fbfec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.545+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbff400b3e0 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.765+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc00419f430 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.767+0000 7fbffbfff700 1 -- 172.21.15.64:0/3320989165 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbff4019de0 con 0x7fc004108550 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbff0039050 msgr2=0x7fbff003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbff0039050 0x7fbff003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006fd0 tx=0x7fbfec006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 msgr2=0x7fc00419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00419b950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbff4006b20 tx=0x7fbff4011ad0 comp rx=0 tx=0).stop 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 shutdown_connections 2024-05-09T08:59:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbff0039050 0x7fbff003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:26.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 --2- 172.21.15.64:0/3320989165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc004108550 0x7fc00419b950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:26.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.769+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 >> 172.21.15.64:0/3320989165 conn(0x7fc004101f60 msgr2=0x7fc004102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:26.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.770+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 shutdown_connections 2024-05-09T08:59:26.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:25.770+0000 7fc0091dd700 1 -- 172.21.15.64:0/3320989165 wait complete. 2024-05-09T08:59:26.403 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2024-05-09T08:59:27.054 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:26 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3320989165' entity='client.admin' 2024-05-09T08:59:27.054 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:26 smithi064 ceph-mon[30149]: from='mgr.14120 172.21.15.64:0/2279437650' entity='mgr.x' 2024-05-09T08:59:28.048 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:27 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/835236370' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T08:59:28.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:27 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: ignoring --setuser ceph since I am not root 2024-05-09T08:59:28.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:27 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: ignoring --setgroup ceph since I am not root 2024-05-09T08:59:28.048 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:27 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:27.962+0000 7f4e2735b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:59:28.366 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:28 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:28.160+0000 7f4e2735b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:59:28.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:28 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:28.266+0000 7f4e2735b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:59:28.374 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b73a3700 1 Processor -- start 2024-05-09T08:59:28.374 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b73a3700 1 -- start start 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b73a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b01095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b73a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0107710 con 0x7f77b0109180 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b513f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b01095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b513f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b01095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56866/0 (socket says 172.21.15.64:56866) 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.161+0000 7f77b513f700 1 -- 172.21.15.64:0/932683936 learned_addr learned my addr 172.21.15.64:0/932683936 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.162+0000 7f77b513f700 1 -- 172.21.15.64:0/932683936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b0107890 con 0x7f77b0109180 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.162+0000 7f77b513f700 1 --2- 172.21.15.64:0/932683936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b01095a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0098a0 tx=0x7f77ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f40c7ee2f1fc94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.162+0000 7f77a7fff700 1 -- 172.21.15.64:0/932683936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ac004160 con 0x7f77b0109180 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.162+0000 7f77a7fff700 1 -- 172.21.15.64:0/932683936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77ac004300 con 0x7f77b0109180 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.162+0000 7f77a7fff700 1 -- 172.21.15.64:0/932683936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ac011a50 con 0x7f77b0109180 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 -- 172.21.15.64:0/932683936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 msgr2=0x7f77b01095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 --2- 172.21.15.64:0/932683936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b01095a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0098a0 tx=0x7f77ac009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 -- 172.21.15.64:0/932683936 shutdown_connections 2024-05-09T08:59:28.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 --2- 172.21.15.64:0/932683936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b01095a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 -- 172.21.15.64:0/932683936 >> 172.21.15.64:0/932683936 conn(0x7f77b01030b0 msgr2=0x7f77b01054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 -- 172.21.15.64:0/932683936 shutdown_connections 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.163+0000 7f77b73a3700 1 -- 172.21.15.64:0/932683936 wait complete. 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b73a3700 1 Processor -- start 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b73a3700 1 -- start start 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b73a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b73a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77b010c050 con 0x7f77b0109180 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b513f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b513f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56882/0 (socket says 172.21.15.64:56882) 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b513f700 1 -- 172.21.15.64:0/835236370 learned_addr learned my addr 172.21.15.64:0/835236370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b513f700 1 -- 172.21.15.64:0/835236370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b019c000 con 0x7f77b0109180 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77b513f700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b019bac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77ac011710 tx=0x7f77ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ac0039c0 con 0x7f77b0109180 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77ac003b60 con 0x7f77b0109180 2024-05-09T08:59:28.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.164+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ac01a690 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.165+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b019c290 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.165+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b019ef80 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.166+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f77ac024430 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.166+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b010c1d0 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.166+0000 7f77a67fc700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f779c039050 0x7f779c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.166+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77ac04d660 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.166+0000 7f77b493e700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f779c039050 0x7f779c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.167+0000 7f77b493e700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f779c039050 0x7f779c03b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77a0006fd0 tx=0x7f77a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.171+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77ac00f2e0 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.448+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f77b019f440 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.773+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f77ac024720 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.775+0000 7f77a67fc700 1 -- 172.21.15.64:0/835236370 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f77ac02d770 con 0x7f77b0109180 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f779c039050 msgr2=0x7f779c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:28.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f779c039050 0x7f779c03b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77a0006fd0 tx=0x7f77a0006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 msgr2=0x7f77b019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b019bac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77ac011710 tx=0x7f77ac011ad0 comp rx=0 tx=0).stop 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 shutdown_connections 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7f779c039050 0x7f779c03b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 --2- 172.21.15.64:0/835236370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77b0109180 0x7f77b019bac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 >> 172.21.15.64:0/835236370 conn(0x7f77b01030b0 msgr2=0x7f77b0103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 shutdown_connections 2024-05-09T08:59:28.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:27.776+0000 7f77b73a3700 1 -- 172.21.15.64:0/835236370 wait complete. 2024-05-09T08:59:28.618 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:28 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:28.594+0000 7f4e2735b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:59:28.892 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:28 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:28.688+0000 7f4e2735b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:59:28.892 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:28 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/835236370' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T08:59:28.892 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:28 smithi064 ceph-mon[30149]: mgrmap e9: x(active, since 19s) 2024-05-09T08:59:29.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:59:29.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T08:59:29.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T08:59:29.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.825+0000 7ff7506dc700 1 Processor -- start 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.825+0000 7ff7506dc700 1 -- start start 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.825+0000 7ff7506dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.825+0000 7ff7506dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff748106780 con 0x7ff748105d90 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74e478700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74e478700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56906/0 (socket says 172.21.15.64:56906) 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74e478700 1 -- 172.21.15.64:0/373655433 learned_addr learned my addr 172.21.15.64:0/373655433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74e478700 1 -- 172.21.15.64:0/373655433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff748108d20 con 0x7ff748105d90 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74e478700 1 --2- 172.21.15.64:0/373655433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481061b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098a0 tx=0x7ff73c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae8494784a9f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74d476700 1 -- 172.21.15.64:0/373655433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c004160 con 0x7ff748105d90 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74d476700 1 -- 172.21.15.64:0/373655433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff73c004300 con 0x7ff748105d90 2024-05-09T08:59:29.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.826+0000 7ff74d476700 1 -- 172.21.15.64:0/373655433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c011a50 con 0x7ff748105d90 2024-05-09T08:59:29.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 -- 172.21.15.64:0/373655433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 msgr2=0x7ff7481061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:29.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 --2- 172.21.15.64:0/373655433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481061b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098a0 tx=0x7ff73c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:29.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 -- 172.21.15.64:0/373655433 shutdown_connections 2024-05-09T08:59:29.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 --2- 172.21.15.64:0/373655433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481061b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:29.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 -- 172.21.15.64:0/373655433 >> 172.21.15.64:0/373655433 conn(0x7ff7481016e0 msgr2=0x7ff748103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 -- 172.21.15.64:0/373655433 shutdown_connections 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.827+0000 7ff7506dc700 1 -- 172.21.15.64:0/373655433 wait complete. 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff7506dc700 1 Processor -- start 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff7506dc700 1 -- start start 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff7506dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481085a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff7506dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810c0d0 con 0x7ff748105d90 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff74e478700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff74e478700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481085a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56908/0 (socket says 172.21.15.64:56908) 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.828+0000 7ff74e478700 1 -- 172.21.15.64:0/2857611421 learned_addr learned my addr 172.21.15.64:0/2857611421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff74e478700 1 -- 172.21.15.64:0/2857611421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff748108ae0 con 0x7ff748105d90 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff74e478700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481085a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006ba0 tx=0x7ff73c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c003af0 con 0x7ff748105d90 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff73c01a8e0 con 0x7ff748105d90 2024-05-09T08:59:29.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff748106cd0 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73c024900 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.829+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff748107210 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.830+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff73c022020 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.830+0000 7ff73b7fe700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 0x7ff73403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.830+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff73c04d4b0 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.830+0000 7ff74dc77700 1 -- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 msgr2=0x7ff73403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.830+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff748063980 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.830+0000 7ff74dc77700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 0x7ff73403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:28.835+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff73c020c10 con 0x7ff748105d90 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.031+0000 7ff74dc77700 1 -- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 msgr2=0x7ff73403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.031+0000 7ff74dc77700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 0x7ff73403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:59:29.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.132+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff748107a80 con 0x7ff748105d90 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.135+0000 7ff73b7fe700 1 -- 172.21.15.64:0/2857611421 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff73c00b5a0 con 0x7ff748105d90 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.138+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 msgr2=0x7ff73403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.138+0000 7ff7506dc700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 0x7ff73403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.138+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 msgr2=0x7ff7481085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.138+0000 7ff7506dc700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481085a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006ba0 tx=0x7ff73c011ad0 comp rx=0 tx=0).stop 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.139+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 shutdown_connections 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.139+0000 7ff7506dc700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7ff734039050 0x7ff73403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.139+0000 7ff7506dc700 1 --2- 172.21.15.64:0/2857611421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff748105d90 0x7ff7481085a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.139+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 >> 172.21.15.64:0/2857611421 conn(0x7ff7481016e0 msgr2=0x7ff748102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.139+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 shutdown_connections 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.139+0000 7ff7506dc700 1 -- 172.21.15.64:0/2857611421 wait complete. 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-05-09T08:59:29.578 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2024-05-09T08:59:30.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:29 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2857611421' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T08:59:30.153 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:29 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:29.996+0000 7f4e2735b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:59:30.604 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:30 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:30.178+0000 7f4e2735b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:59:30.604 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:30 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:30.356+0000 7f4e2735b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:59:30.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:30 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:30.659+0000 7f4e2735b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:59:30.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:30 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:30.755+0000 7f4e2735b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:59:31.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:30 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:30.967+0000 7f4e2735b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:59:31.403 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:31 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:31.172+0000 7f4e2735b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:59:32.152 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:31 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:31.805+0000 7f4e2735b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:59:32.562 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:32 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:32.247+0000 7f4e2735b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:59:32.563 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:32 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:32.349+0000 7f4e2735b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:59:32.563 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:32 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:32.563+0000 7f4e2735b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:59:33.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:33.120+0000 7f4e2735b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:59:33.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:33.217+0000 7f4e2735b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:59:33.833 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:33.471+0000 7f4e2735b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:59:33.833 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:33.611+0000 7f4e2735b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:59:33.833 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:33.711+0000 7f4e2735b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:59:33.833 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:33.833+0000 7f4e2735b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:59:34.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:34 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:34.147+0000 7f4e2735b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:59:35.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:34 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:34.781+0000 7f4e2735b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:59:35.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 08:59:35 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T08:59:35.009+0000 7f4e2735b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: Active manager daemon x restarted 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: Activating manager daemon x 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: mgrmap e10: x(active, starting, since 0.00329255s) 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: Manager daemon x is now available 2024-05-09T08:59:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:36.329 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T08:59:36.329 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T08:59:36.329 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:36 smithi064 ceph-mon[30149]: [09/May/2024:08:59:35] ENGINE Bus STARTING 2024-05-09T08:59:36.329 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:36 smithi064 ceph-mon[30149]: mgrmap e11: x(active, since 1.00595s) 2024-05-09T08:59:37.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:59:37.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T08:59:37.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.997+0000 7fbebbed3700 1 Processor -- start 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.997+0000 7fbebbed3700 1 -- start start 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbebbed3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb4106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbebbed3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4107390 con 0x7fbeb41069a0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbeb9c6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb4106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbeb9c6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb4106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56920/0 (socket says 172.21.15.64:56920) 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbeb9c6f700 1 -- 172.21.15.64:0/1199960303 learned_addr learned my addr 172.21.15.64:0/1199960303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbeb9c6f700 1 -- 172.21.15.64:0/1199960303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb4107ba0 con 0x7fbeb41069a0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbeb9c6f700 1 --2- 172.21.15.64:0/1199960303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb4106dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009e80 tx=0x7fbea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dafc61c07426ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.998+0000 7fbeb8c6d700 1 -- 172.21.15.64:0/1199960303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4017070 con 0x7fbeb41069a0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbeb8c6d700 1 -- 172.21.15.64:0/1199960303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbea400b8c0 con 0x7fbeb41069a0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbeb8c6d700 1 -- 172.21.15.64:0/1199960303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4004490 con 0x7fbeb41069a0 2024-05-09T08:59:37.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbebbed3700 1 -- 172.21.15.64:0/1199960303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 msgr2=0x7fbeb4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1199960303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb4106dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009e80 tx=0x7fbea400b330 comp rx=0 tx=0).stop 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbebbed3700 1 -- 172.21.15.64:0/1199960303 shutdown_connections 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1199960303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb4106dc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:29.999+0000 7fbebbed3700 1 -- 172.21.15.64:0/1199960303 >> 172.21.15.64:0/1199960303 conn(0x7fbeb4101f40 msgr2=0x7fbeb4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.000+0000 7fbebbed3700 1 -- 172.21.15.64:0/1199960303 shutdown_connections 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.000+0000 7fbebbed3700 1 -- 172.21.15.64:0/1199960303 wait complete. 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.000+0000 7fbebbed3700 1 Processor -- start 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.000+0000 7fbebbed3700 1 -- start start 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbebbed3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb407ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbebbed3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb410af50 con 0x7fbeb41069a0 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb9c6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb407ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb9c6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb407ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56928/0 (socket says 172.21.15.64:56928) 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb9c6f700 1 -- 172.21.15.64:0/1270318368 learned_addr learned my addr 172.21.15.64:0/1270318368 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb9c6f700 1 -- 172.21.15.64:0/1270318368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb419e700 con 0x7fbeb41069a0 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb9c6f700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb407ba90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbea40072a0 tx=0x7fbea4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:37.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4020020 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbea400bba0 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.001+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb419e8d0 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.002+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb419ec70 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.002+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4017040 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.003+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fbea4023020 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.003+0000 7fbeb2ffd700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.003+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 --> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbea0000d40 con 0x7fbe9c0390e0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.003+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbea400f050 con 0x7fbeb41069a0 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.003+0000 7fbeb946e700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 msgr2=0x7fbe9c03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.003+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.204+0000 7fbeb946e700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 msgr2=0x7fbe9c03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.204+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.605+0000 7fbeb946e700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 msgr2=0x7fbe9c03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:30.605+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:59:37.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:31.406+0000 7fbeb946e700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 msgr2=0x7fbe9c03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:31.406+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:33.007+0000 7fbeb946e700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 msgr2=0x7fbe9c03b5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/615130791 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:33.007+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:35.012+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fbea4018bb0 con 0x7fbeb41069a0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:35.012+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 msgr2=0x7fbe9c03b5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:35.012+0000 7fbeb2ffd700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.013+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fbea404e830 con 0x7fbeb41069a0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.014+0000 7fbeb2ffd700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe9c03c9b0 0x7fbe9c03eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.014+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbea0000d40 con 0x7fbe9c03c9b0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.014+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe9c03c9b0 0x7fbe9c03eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.015+0000 7fbeb946e700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe9c03c9b0 0x7fbe9c03eda0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbea8003a10 tx=0x7fbea80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.016+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fbea0000d40 con 0x7fbe9c03c9b0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fbea0002840 con 0x7fbe9c03c9b0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbeb2ffd700 1 -- 172.21.15.64:0/1270318368 <== mgr.14152 v2:172.21.15.64:6800/2084062196 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fbea0002840 con 0x7fbe9c03c9b0 2024-05-09T08:59:37.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe9c03c9b0 msgr2=0x7fbe9c03eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe9c03c9b0 0x7fbe9c03eda0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbea8003a10 tx=0x7fbea80092b0 comp rx=0 tx=0).stop 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 msgr2=0x7fbeb407ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb407ba90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbea40072a0 tx=0x7fbea4003d00 comp rx=0 tx=0).stop 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 shutdown_connections 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe9c03c9b0 0x7fbe9c03eda0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbeb41069a0 0x7fbeb407ba90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 --2- 172.21.15.64:0/1270318368 >> [v2:172.21.15.64:6800/615130791,v1:172.21.15.64:6801/615130791] conn(0x7fbe9c0390e0 0x7fbe9c03b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.029+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 >> 172.21.15.64:0/1270318368 conn(0x7fbeb4101f40 msgr2=0x7fbeb41029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.030+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 shutdown_connections 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:36.030+0000 7fbebbed3700 1 -- 172.21.15.64:0/1270318368 wait complete. 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2024-05-09T08:59:37.064 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2024-05-09T08:59:38.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:37 smithi064 ceph-mon[30149]: [09/May/2024:08:59:35] ENGINE Serving on https://172.21.15.64:7150 2024-05-09T08:59:38.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:37 smithi064 ceph-mon[30149]: [09/May/2024:08:59:35] ENGINE Serving on http://172.21.15.64:8765 2024-05-09T08:59:38.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:37 smithi064 ceph-mon[30149]: [09/May/2024:08:59:35] ENGINE Bus STARTED 2024-05-09T08:59:38.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:37 smithi064 ceph-mon[30149]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T08:59:38.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:37 smithi064 ceph-mon[30149]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T08:59:38.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:38.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T08:59:38.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.508+0000 7f7bcfc14700 1 Processor -- start 2024-05-09T08:59:38.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.508+0000 7f7bcfc14700 1 -- start start 2024-05-09T08:59:38.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcfc14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8106e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcfc14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81073d0 con 0x7f7bc81069e0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcd9b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcd9b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8106e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47906/0 (socket says 172.21.15.64:47906) 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcd9b0700 1 -- 172.21.15.64:0/2520038556 learned_addr learned my addr 172.21.15.64:0/2520038556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcd9b0700 1 -- 172.21.15.64:0/2520038556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc8107be0 con 0x7f7bc81069e0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcd9b0700 1 --2- 172.21.15.64:0/2520038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8106e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0098a0 tx=0x7f7bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca3233d42320de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.509+0000 7f7bcc9ae700 1 -- 172.21.15.64:0/2520038556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bbc004160 con 0x7f7bc81069e0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcc9ae700 1 -- 172.21.15.64:0/2520038556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bbc004300 con 0x7f7bc81069e0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcc9ae700 1 -- 172.21.15.64:0/2520038556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bbc011a50 con 0x7f7bc81069e0 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcfc14700 1 -- 172.21.15.64:0/2520038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 msgr2=0x7f7bc8106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:38.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcfc14700 1 --2- 172.21.15.64:0/2520038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8106e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0098a0 tx=0x7f7bbc009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcfc14700 1 -- 172.21.15.64:0/2520038556 shutdown_connections 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcfc14700 1 --2- 172.21.15.64:0/2520038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8106e00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcfc14700 1 -- 172.21.15.64:0/2520038556 >> 172.21.15.64:0/2520038556 conn(0x7f7bc8101f60 msgr2=0x7f7bc81043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.510+0000 7f7bcfc14700 1 -- 172.21.15.64:0/2520038556 shutdown_connections 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.511+0000 7f7bcfc14700 1 -- 172.21.15.64:0/2520038556 wait complete. 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.511+0000 7f7bcfc14700 1 Processor -- start 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.511+0000 7f7bcfc14700 1 -- start start 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.511+0000 7f7bcfc14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8191b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.511+0000 7f7bcfc14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc810af90 con 0x7f7bc81069e0 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.511+0000 7f7bcd9b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8191b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bcd9b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8191b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47922/0 (socket says 172.21.15.64:47922) 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bcd9b0700 1 -- 172.21.15.64:0/3829973353 learned_addr learned my addr 172.21.15.64:0/3829973353 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:38.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bcd9b0700 1 -- 172.21.15.64:0/3829973353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc8192090 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bcd9b0700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8191b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc006b20 tx=0x7f7bbc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bbc0039e0 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bbc01a8e0 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc8190280 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bbc0247e0 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.512+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc81907c0 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.513+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8063980 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.513+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f7bbc022020 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.514+0000 7f7bbaffd700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7bb4038f80 0x7f7bb403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.514+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7bbc04d530 con 0x7f7bc81069e0 2024-05-09T08:59:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.514+0000 7f7bcd1af700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7bb4038f80 0x7f7bb403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.514+0000 7f7bcd1af700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7bb4038f80 0x7f7bb403b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4006fd0 tx=0x7f7bc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.519+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7bbc00b3e0 con 0x7f7bc81069e0 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.674+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7bbc02dc90 con 0x7f7bc81069e0 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.735+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7bc8195930 con 0x7f7bb4038f80 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.890+0000 7f7bbaffd700 1 -- 172.21.15.64:0/3829973353 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7bc8195930 con 0x7f7bb4038f80 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7bb4038f80 msgr2=0x7f7bb403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7bb4038f80 0x7f7bb403b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4006fd0 tx=0x7f7bc4006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 msgr2=0x7f7bc8191b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8191b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc006b20 tx=0x7f7bbc011ad0 comp rx=0 tx=0).stop 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 shutdown_connections 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7bb4038f80 0x7f7bb403b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 --2- 172.21.15.64:0/3829973353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bc81069e0 0x7f7bc8191b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 >> 172.21.15.64:0/3829973353 conn(0x7f7bc8101f60 msgr2=0x7f7bc8102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:38.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 shutdown_connections 2024-05-09T08:59:38.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:37.892+0000 7f7bcfc14700 1 -- 172.21.15.64:0/3829973353 wait complete. 2024-05-09T08:59:38.436 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2024-05-09T08:59:38.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:38 smithi064 ceph-mon[30149]: mgrmap e12: x(active, since 2s) 2024-05-09T08:59:38.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:38 smithi064 ceph-mon[30149]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:38.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:38.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:38.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:40.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fgv66dDuGeVVeMff/yDAous4bH1o3BqS14JVxYSPz.tAFjXc2leny", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715245179, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.892+0000 7f56f6243700 1 Processor -- start 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.892+0000 7f56f6243700 1 -- start start 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56f6243700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0106da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56f6243700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0107370 con 0x7f56f0106980 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47938/0 (socket says 172.21.15.64:47938) 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56ef7fe700 1 -- 172.21.15.64:0/862738563 learned_addr learned my addr 172.21.15.64:0/862738563 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56ef7fe700 1 -- 172.21.15.64:0/862738563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f0107b80 con 0x7f56f0106980 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.893+0000 7f56ef7fe700 1 --2- 172.21.15.64:0/862738563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0106da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56e000bb40 tx=0x7f56e000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c25f8b9314247a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56ee7fc700 1 -- 172.21.15.64:0/862738563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e000dbe0 con 0x7f56f0106980 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56ee7fc700 1 -- 172.21.15.64:0/862738563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56e00044e0 con 0x7f56f0106980 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56ee7fc700 1 -- 172.21.15.64:0/862738563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0010470 con 0x7f56f0106980 2024-05-09T08:59:40.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56f6243700 1 -- 172.21.15.64:0/862738563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 msgr2=0x7f56f0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56f6243700 1 --2- 172.21.15.64:0/862738563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0106da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56e000bb40 tx=0x7f56e000be50 comp rx=0 tx=0).stop 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56f6243700 1 -- 172.21.15.64:0/862738563 shutdown_connections 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56f6243700 1 --2- 172.21.15.64:0/862738563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0106da0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.894+0000 7f56f6243700 1 -- 172.21.15.64:0/862738563 >> 172.21.15.64:0/862738563 conn(0x7f56f0101f20 msgr2=0x7f56f0104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.895+0000 7f56f6243700 1 -- 172.21.15.64:0/862738563 shutdown_connections 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.895+0000 7f56f6243700 1 -- 172.21.15.64:0/862738563 wait complete. 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.895+0000 7f56f6243700 1 Processor -- start 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56f6243700 1 -- start start 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56f6243700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56f6243700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56f007aff0 con 0x7f56f0106980 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47948/0 (socket says 172.21.15.64:47948) 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ef7fe700 1 -- 172.21.15.64:0/1141634820 learned_addr learned my addr 172.21.15.64:0/1141634820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ef7fe700 1 -- 172.21.15.64:0/1141634820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f0199cc0 con 0x7f56f0106980 2024-05-09T08:59:40.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ef7fe700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0199780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56e000b340 tx=0x7f56e0003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0027070 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.896+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56e0010bd0 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.897+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0022b70 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.897+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f0199f50 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.897+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f019a3b0 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.897+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f56e0022d10 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.898+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f0063980 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.898+0000 7f56ecff9700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f56d8039100 0x7f56d803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.898+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56e001f070 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.898+0000 7f56eeffd700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f56d8039100 0x7f56d803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.898+0000 7f56eeffd700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f56d8039100 0x7f56d803b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f56e4006fd0 tx=0x7f56e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:38.903+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f56e0015070 con 0x7f56f0106980 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.153+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f56f019cdc0 con 0x7f56d8039100 2024-05-09T08:59:40.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.400+0000 7f56ecff9700 1 -- 172.21.15.64:0/1141634820 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f56f019cdc0 con 0x7f56d8039100 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.402+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f56d8039100 msgr2=0x7f56d803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.402+0000 7f56f6243700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f56d8039100 0x7f56d803b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f56e4006fd0 tx=0x7f56e4006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.402+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 msgr2=0x7f56f0199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.402+0000 7f56f6243700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0199780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56e000b340 tx=0x7f56e0003fb0 comp rx=0 tx=0).stop 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.402+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 shutdown_connections 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.403+0000 7f56f6243700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f56d8039100 0x7f56d803b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.403+0000 7f56f6243700 1 --2- 172.21.15.64:0/1141634820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56f0106980 0x7f56f0199780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.403+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 >> 172.21.15.64:0/1141634820 conn(0x7f56f0101f20 msgr2=0x7f56f018fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.403+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 shutdown_connections 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:39.403+0000 7f56f6243700 1 -- 172.21.15.64:0/1141634820 wait complete. 2024-05-09T08:59:40.061 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2024-05-09T08:59:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:40 smithi064 ceph-mon[30149]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:41.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f549a40d700 1 Processor -- start 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f549a40d700 1 -- start start 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f549a40d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f54941061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f549a40d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54941067b0 con 0x7f5494105dc0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f5493fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f54941061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f5493fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f54941061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47950/0 (socket says 172.21.15.64:47950) 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f5493fff700 1 -- 172.21.15.64:0/4018169034 learned_addr learned my addr 172.21.15.64:0/4018169034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.517+0000 7f5493fff700 1 -- 172.21.15.64:0/4018169034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5494108d50 con 0x7f5494105dc0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f5493fff700 1 --2- 172.21.15.64:0/4018169034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f54941061e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54840098a0 tx=0x7f5484009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556448b380323c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f54937fe700 1 -- 172.21.15.64:0/4018169034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484004160 con 0x7f5494105dc0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f54937fe700 1 -- 172.21.15.64:0/4018169034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5484004300 con 0x7f5494105dc0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f54937fe700 1 -- 172.21.15.64:0/4018169034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484011a50 con 0x7f5494105dc0 2024-05-09T08:59:41.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f549a40d700 1 -- 172.21.15.64:0/4018169034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 msgr2=0x7f54941061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f549a40d700 1 --2- 172.21.15.64:0/4018169034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f54941061e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54840098a0 tx=0x7f5484009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f549a40d700 1 -- 172.21.15.64:0/4018169034 shutdown_connections 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f549a40d700 1 --2- 172.21.15.64:0/4018169034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f54941061e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.518+0000 7f549a40d700 1 -- 172.21.15.64:0/4018169034 >> 172.21.15.64:0/4018169034 conn(0x7f54941016d0 msgr2=0x7f5494103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f549a40d700 1 -- 172.21.15.64:0/4018169034 shutdown_connections 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f549a40d700 1 -- 172.21.15.64:0/4018169034 wait complete. 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f549a40d700 1 Processor -- start 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f549a40d700 1 -- start start 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f549a40d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f549419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f549a40d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f549410c100 con 0x7f5494105dc0 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f5493fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f549419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f5493fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f549419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47966/0 (socket says 172.21.15.64:47966) 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.519+0000 7f5493fff700 1 -- 172.21.15.64:0/1127069957 learned_addr learned my addr 172.21.15.64:0/1127069957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f5493fff700 1 -- 172.21.15.64:0/1127069957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549419bea0 con 0x7f5494105dc0 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f5493fff700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f549419b960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5484006ba0 tx=0x7f5484011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:41.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484003af0 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f548401a8e0 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549419c130 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484024900 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.520+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549419ee20 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.521+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5484022020 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.521+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5494063980 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.521+0000 7f5491ffb700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54800390b0 0x7f548003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.521+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f548404d690 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.521+0000 7f548bfff700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54800390b0 0x7f548003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.522+0000 7f548bfff700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54800390b0 0x7f548003b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f547c006fd0 tx=0x7f547c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.526+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f548400b3e0 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.730+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f549419f3d0 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.731+0000 7f5491ffb700 1 -- 172.21.15.64:0/1127069957 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5484020d80 con 0x7f5494105dc0 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54800390b0 msgr2=0x7f548003b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:41.436 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54800390b0 0x7f548003b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f547c006fd0 tx=0x7f547c006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 msgr2=0x7f549419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f549419b960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5484006ba0 tx=0x7f5484011ad0 comp rx=0 tx=0).stop 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 shutdown_connections 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54800390b0 0x7f548003b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 --2- 172.21.15.64:0/1127069957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5494105dc0 0x7f549419b960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 >> 172.21.15.64:0/1127069957 conn(0x7f54941016d0 msgr2=0x7f5494102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 shutdown_connections 2024-05-09T08:59:41.437 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:40.733+0000 7f549a40d700 1 -- 172.21.15.64:0/1127069957 wait complete. 2024-05-09T08:59:41.444 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:59:41.444 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-05-09T08:59:41.452 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:59:41.452 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-05-09T08:59:41.452 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-05-09T08:59:41.453 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout: Password: 4z3qhstxkl 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:41.455 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config directory 2024-05-09T08:59:41.456 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling autotune for osd_memory_target 2024-05-09T08:59:41.581 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:41 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1127069957' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.900+0000 7fe27125b700 1 Processor -- start 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.900+0000 7fe27125b700 1 -- start start 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.900+0000 7fe27125b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c106d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.900+0000 7fe27125b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c107360 con 0x7fe26c106970 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.900+0000 7fe26ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe26ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47978/0 (socket says 172.21.15.64:47978) 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe26ad9d700 1 -- 172.21.15.64:0/2456241246 learned_addr learned my addr 172.21.15.64:0/2456241246 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:42.607 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe26ad9d700 1 -- 172.21.15.64:0/2456241246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c107b70 con 0x7fe26c106970 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe26ad9d700 1 --2- 172.21.15.64:0/2456241246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c106d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00d180 tx=0x7fe25c00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372c5121a0545413 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe269d9b700 1 -- 172.21.15.64:0/2456241246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c019070 con 0x7fe26c106970 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe269d9b700 1 -- 172.21.15.64:0/2456241246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe25c00de20 con 0x7fe26c106970 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.901+0000 7fe269d9b700 1 -- 172.21.15.64:0/2456241246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c009a80 con 0x7fe26c106970 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 -- 172.21.15.64:0/2456241246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 msgr2=0x7fe26c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 --2- 172.21.15.64:0/2456241246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c106d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00d180 tx=0x7fe25c00d490 comp rx=0 tx=0).stop 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 -- 172.21.15.64:0/2456241246 shutdown_connections 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 --2- 172.21.15.64:0/2456241246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c106d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 -- 172.21.15.64:0/2456241246 >> 172.21.15.64:0/2456241246 conn(0x7fe26c101f50 msgr2=0x7fe26c104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 -- 172.21.15.64:0/2456241246 shutdown_connections 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.902+0000 7fe27125b700 1 -- 172.21.15.64:0/2456241246 wait complete. 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe27125b700 1 Processor -- start 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe27125b700 1 -- start start 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe27125b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:42.608 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe27125b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c07aff0 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe26ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe26ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47980/0 (socket says 172.21.15.64:47980) 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.903+0000 7fe26ad9d700 1 -- 172.21.15.64:0/1831032441 learned_addr learned my addr 172.21.15.64:0/1831032441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe26ad9d700 1 -- 172.21.15.64:0/1831032441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c19e040 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe26ad9d700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c19db00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00f010 tx=0x7fe25c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c019040 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c19e2d0 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe25c00b440 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c01f7a0 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.904+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1a0fc0 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.905+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe25c01faa0 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.905+0000 7fe263fff700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe258039140 0x7fe25803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.905+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe25c011050 con 0x7fe26c106970 2024-05-09T08:59:42.609 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.905+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c063980 con 0x7fe26c106970 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.905+0000 7fe26a59c700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe258039140 0x7fe25803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.906+0000 7fe26a59c700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe258039140 0x7fe25803b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe254006fd0 tx=0x7fe254006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:41.910+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe25c01d330 con 0x7fe26c106970 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.116+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe26c1a12e0 con 0x7fe26c106970 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.116+0000 7fe263fff700 1 -- 172.21.15.64:0/1831032441 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe25c0252b0 con 0x7fe26c106970 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe258039140 msgr2=0x7fe25803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe258039140 0x7fe25803b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe254006fd0 tx=0x7fe254006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 msgr2=0x7fe26c19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c19db00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00f010 tx=0x7fe25c003d00 comp rx=0 tx=0).stop 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 shutdown_connections 2024-05-09T08:59:42.610 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe258039140 0x7fe25803b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:42.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 --2- 172.21.15.64:0/1831032441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe26c106970 0x7fe26c19db00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:42.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.118+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 >> 172.21.15.64:0/1831032441 conn(0x7fe26c101f50 msgr2=0x7fe26c18fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:42.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.119+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 shutdown_connections 2024-05-09T08:59:42.611 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.119+0000 7fe27125b700 1 -- 172.21.15.64:0/1831032441 wait complete. 2024-05-09T08:59:42.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:42 smithi064 ceph-mon[30149]: mgrmap e13: x(active, since 6s) 2024-05-09T08:59:43.878 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.998+0000 7fbce422b700 1 Processor -- start 2024-05-09T08:59:43.878 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.998+0000 7fbce422b700 1 -- start start 2024-05-09T08:59:43.878 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.998+0000 7fbce422b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc10a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:43.878 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.998+0000 7fbce422b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc07be90 con 0x7fbcdc108520 2024-05-09T08:59:43.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce1fc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc10a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:43.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce1fc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc10a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47992/0 (socket says 172.21.15.64:47992) 2024-05-09T08:59:43.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce1fc7700 1 -- 172.21.15.64:0/469521262 learned_addr learned my addr 172.21.15.64:0/469521262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:43.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce1fc7700 1 -- 172.21.15.64:0/469521262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc10ae50 con 0x7fbcdc108520 2024-05-09T08:59:43.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce1fc7700 1 --2- 172.21.15.64:0/469521262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc10a910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7977b48addff1605 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:43.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce0fc5700 1 -- 172.21.15.64:0/469521262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8004160 con 0x7fbcdc108520 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce0fc5700 1 -- 172.21.15.64:0/469521262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd8004300 con 0x7fbcdc108520 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:42.999+0000 7fbce0fc5700 1 -- 172.21.15.64:0/469521262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8011a50 con 0x7fbcdc108520 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 -- 172.21.15.64:0/469521262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 msgr2=0x7fbcdc10a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 --2- 172.21.15.64:0/469521262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc10a910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).stop 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 -- 172.21.15.64:0/469521262 shutdown_connections 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 --2- 172.21.15.64:0/469521262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc10a910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:43.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 -- 172.21.15.64:0/469521262 >> 172.21.15.64:0/469521262 conn(0x7fbcdc101f50 msgr2=0x7fbcdc104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:43.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 -- 172.21.15.64:0/469521262 shutdown_connections 2024-05-09T08:59:43.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.000+0000 7fbce422b700 1 -- 172.21.15.64:0/469521262 wait complete. 2024-05-09T08:59:43.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce422b700 1 Processor -- start 2024-05-09T08:59:43.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce422b700 1 -- start start 2024-05-09T08:59:43.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce422b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:43.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce422b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10be50 con 0x7fbcdc108520 2024-05-09T08:59:43.882 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce1fc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:43.882 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce1fc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc19b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48002/0 (socket says 172.21.15.64:48002) 2024-05-09T08:59:43.882 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce1fc7700 1 -- 172.21.15.64:0/1348747349 learned_addr learned my addr 172.21.15.64:0/1348747349 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:43.882 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce1fc7700 1 -- 172.21.15.64:0/1348747349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc19be50 con 0x7fbcdc108520 2024-05-09T08:59:43.882 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.001+0000 7fbce1fc7700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc19b910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8011a20 tx=0x7fbcd8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:43.883 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.002+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8003b70 con 0x7fbcdc108520 2024-05-09T08:59:43.883 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.002+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc19c0e0 con 0x7fbcdc108520 2024-05-09T08:59:43.883 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.002+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd801a8e0 con 0x7fbcdc108520 2024-05-09T08:59:43.883 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.002+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8024990 con 0x7fbcdc108520 2024-05-09T08:59:43.883 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.002+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc19edd0 con 0x7fbcdc108520 2024-05-09T08:59:43.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.003+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbcd8022020 con 0x7fbcdc108520 2024-05-09T08:59:43.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.003+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc063980 con 0x7fbcdc108520 2024-05-09T08:59:43.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.003+0000 7fbcd2ffd700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbcc80390f0 0x7fbcc803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:43.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.003+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbcd804d9d0 con 0x7fbcdc108520 2024-05-09T08:59:43.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.003+0000 7fbce17c6700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbcc80390f0 0x7fbcc803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:43.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.003+0000 7fbce17c6700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbcc80390f0 0x7fbcc803b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:43.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.008+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbcd800f2e0 con 0x7fbcdc108520 2024-05-09T08:59:43.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.326+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fbcdc19f380 con 0x7fbcdc108520 2024-05-09T08:59:43.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.329+0000 7fbcd2ffd700 1 -- 172.21.15.64:0/1348747349 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fbcd802d3f0 con 0x7fbcdc108520 2024-05-09T08:59:43.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbcc80390f0 msgr2=0x7fbcc803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:43.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbcc80390f0 0x7fbcc803b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:43.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 msgr2=0x7fbcdc19b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc19b910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8011a20 tx=0x7fbcd8011b00 comp rx=0 tx=0).stop 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 shutdown_connections 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbcc80390f0 0x7fbcc803b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 --2- 172.21.15.64:0/1348747349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcdc108520 0x7fbcdc19b910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 >> 172.21.15.64:0/1348747349 conn(0x7fbcdc101f50 msgr2=0x7fbcdc1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 shutdown_connections 2024-05-09T08:59:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-05-09T08:59:43.331+0000 7fbce422b700 1 -- 172.21.15.64:0/1348747349 wait complete. 2024-05-09T08:59:43.887 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T08:59:43.887 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T08:59:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.887 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2024-05-09T08:59:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.889 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2024-05-09T08:59:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T08:59:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:43.889 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2024-05-09T08:59:43.915 INFO:tasks.cephadm:Fetching config... 2024-05-09T08:59:43.915 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:59:43.915 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T08:59:43.933 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T08:59:43.933 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:59:43.933 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T08:59:43.989 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T08:59:43.990 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:59:43.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T08:59:44.056 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T08:59:44.056 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T08:59:44.056 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T08:59:44.112 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T08:59:44.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T08:59:44.190 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:59:44.202 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T08:59:44.243 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:59:44.256 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T08:59:44.297 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC55DdgcfaGu4A2GA3VsD57dmu08ywTqVW4J0vnBILK0g89JeemGd17TzNFXqlSUXZsepzbvI+TeKktflGCoe1m5c+6BS1tlR5zIYSjsM1fT8J/O7n9WRY176W+5+d+qtNL7M35ivMRkm6vG7Oby+27UX3WopYGITHqqdXtiDSH4u/gE1PIH28IhkRt7dK1KD6P3+wRo8EoIXZYaM+6PkLraxYtU37Wx3QMbc8CWrJ+QzLl+GIOju92AJkZ7M78JpKH+r4z10g53YTSST+sFPsPieJlQ+9m2D/z2HfdhXR3TyXqE6OWjFO3ijgcnbw7XDbBMJ6FFF9E5tyfKcBuR9UKODCDjXCc5Ht0rzRx2fRLSzthhNPxVcez8PSvHlQ4SyR3jcp+EGoS0h0zFgU5Swpue7MCoVFVDxEzJziA2pXgZkbEPtuhqnQlrMKYJLp7CBYOrB8Wkzl7t7IQNBZA9IDKB7AJMZeMl3HLL3py0EL+tk2G/Frh0Yp1RrHi+Zh9YNE= ceph-2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T08:59:44.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T08:59:44.408 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:44 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1348747349' entity='client.admin' 2024-05-09T08:59:44.563 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:59:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.698+0000 7f854a67c700 1 -- 172.21.15.64:0/210404604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 msgr2=0x7f85441021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.698+0000 7f854a67c700 1 --2- 172.21.15.64:0/210404604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f85441021e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8534009910 tx=0x7f8534009c20 comp rx=0 tx=0).stop 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 -- 172.21.15.64:0/210404604 shutdown_connections 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 --2- 172.21.15.64:0/210404604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f85441021e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 -- 172.21.15.64:0/210404604 >> 172.21.15.64:0/210404604 conn(0x7f85440fd320 msgr2=0x7f85440ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 -- 172.21.15.64:0/210404604 shutdown_connections 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 -- 172.21.15.64:0/210404604 wait complete. 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 Processor -- start 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 -- start start 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.699+0000 7f854a67c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f8544197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f854a67c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8544107d00 con 0x7f8544101dc0 2024-05-09T08:59:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8543fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f8544197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8543fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f8544197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59402/0 (socket says 172.21.15.64:59402) 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8543fff700 1 -- 172.21.15.64:0/1758170143 learned_addr learned my addr 172.21.15.64:0/1758170143 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8543fff700 1 -- 172.21.15.64:0/1758170143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8544197950 con 0x7f8544101dc0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8543fff700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f8544197410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8534005800 tx=0x7f8534009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8534027070 con 0x7f8544101dc0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f853400be10 con 0x7f8544101dc0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853400f460 con 0x7f8544101dc0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8544197be0 con 0x7f8544101dc0 2024-05-09T08:59:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.700+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854419a880 con 0x7f8544101dc0 2024-05-09T08:59:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.701+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f853402a070 con 0x7f8544101dc0 2024-05-09T08:59:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.701+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8544108480 con 0x7f8544101dc0 2024-05-09T08:59:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.701+0000 7f8541ffb700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8530039150 0x7f853003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.701+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f853401e030 con 0x7f8544101dc0 2024-05-09T08:59:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.702+0000 7f853bfff700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8530039150 0x7f853003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.702+0000 7f853bfff700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8530039150 0x7f853003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f852c006fd0 tx=0x7f852c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.706+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8534013070 con 0x7f8544101dc0 2024-05-09T08:59:46.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.913+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f854402dc90 con 0x7f8544101dc0 2024-05-09T08:59:46.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.915+0000 7f8541ffb700 1 -- 172.21.15.64:0/1758170143 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8534031020 con 0x7f8544101dc0 2024-05-09T08:59:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8530039150 msgr2=0x7f853003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8530039150 0x7f853003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f852c006fd0 tx=0x7f852c006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 msgr2=0x7f8544197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f8544197410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8534005800 tx=0x7f8534009f60 comp rx=0 tx=0).stop 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 shutdown_connections 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8530039150 0x7f853003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 --2- 172.21.15.64:0/1758170143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8544101dc0 0x7f8544197410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 >> 172.21.15.64:0/1758170143 conn(0x7f85440fd320 msgr2=0x7f85440fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 shutdown_connections 2024-05-09T08:59:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:46.918+0000 7f854a67c700 1 -- 172.21.15.64:0/1758170143 wait complete. 2024-05-09T08:59:47.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:59:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:47.478 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T08:59:47.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T08:59:47.722 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:59:48.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1758170143' entity='client.admin' 2024-05-09T08:59:48.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:48.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 -- 172.21.15.64:0/3875462550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 msgr2=0x7f916c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 --2- 172.21.15.64:0/3875462550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c1010a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f915c009910 tx=0x7f915c009c20 comp rx=0 tx=0).stop 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 -- 172.21.15.64:0/3875462550 shutdown_connections 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 --2- 172.21.15.64:0/3875462550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c1010a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 -- 172.21.15.64:0/3875462550 >> 172.21.15.64:0/3875462550 conn(0x7f916c0fc220 msgr2=0x7f916c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 -- 172.21.15.64:0/3875462550 shutdown_connections 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.155+0000 7f917183a700 1 -- 172.21.15.64:0/3875462550 wait complete. 2024-05-09T08:59:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f917183a700 1 Processor -- start 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f917183a700 1 -- start start 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f917183a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c0fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f917183a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f916c106bc0 con 0x7f916c100c80 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f916affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c0fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f916affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c0fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59422/0 (socket says 172.21.15.64:59422) 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f916affd700 1 -- 172.21.15.64:0/1282866194 learned_addr learned my addr 172.21.15.64:0/1282866194 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.156+0000 7f916affd700 1 -- 172.21.15.64:0/1282866194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c0fe620 con 0x7f916c100c80 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.157+0000 7f916affd700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c0fe0e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f915c005800 tx=0x7f915c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.157+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915c027070 con 0x7f916c100c80 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.157+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f915c00be10 con 0x7f916c100c80 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.157+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915c00f460 con 0x7f916c100c80 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.157+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c0fc810 con 0x7f916c100c80 2024-05-09T08:59:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.157+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c0fcd50 con 0x7f916c100c80 2024-05-09T08:59:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.158+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f915c02a070 con 0x7f916c100c80 2024-05-09T08:59:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.158+0000 7f9170838700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9158039150 0x7f915803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.158+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916c107340 con 0x7f916c100c80 2024-05-09T08:59:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.158+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f915c01e030 con 0x7f916c100c80 2024-05-09T08:59:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.158+0000 7f9163fff700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9158039150 0x7f915803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.158+0000 7f9163fff700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9158039150 0x7f915803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9154006fd0 tx=0x7f9154006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.163+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f915c013070 con 0x7f916c100c80 2024-05-09T08:59:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.373+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f916c190940 con 0x7f9158039150 2024-05-09T08:59:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.376+0000 7f9170838700 1 -- 172.21.15.64:0/1282866194 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f916c190940 con 0x7f9158039150 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9158039150 msgr2=0x7f915803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9158039150 0x7f915803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9154006fd0 tx=0x7f9154006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 msgr2=0x7f916c0fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c0fe0e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f915c005800 tx=0x7f915c009f60 comp rx=0 tx=0).stop 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 shutdown_connections 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9158039150 0x7f915803b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 --2- 172.21.15.64:0/1282866194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c100c80 0x7f916c0fe0e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 >> 172.21.15.64:0/1282866194 conn(0x7f916c0fc220 msgr2=0x7f916c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 shutdown_connections 2024-05-09T08:59:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:49.379+0000 7f917183a700 1 -- 172.21.15.64:0/1282866194 wait complete. 2024-05-09T08:59:50.103 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi139 2024-05-09T08:59:50.103 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:59:50.103 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.conf 2024-05-09T08:59:50.122 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T08:59:50.122 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:50.181 INFO:tasks.cephadm:Adding host smithi139 to orchestrator... 2024-05-09T08:59:50.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch host add smithi139 2024-05-09T08:59:50.435 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:59:50.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:59:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:59:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-09T08:59:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:50 smithi064 ceph-mon[30149]: Updating smithi064:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:59:51.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:51 smithi064 ceph-mon[30149]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:51.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:51 smithi064 ceph-mon[30149]: Updating smithi064:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T08:59:51.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:51 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:51.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:51 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:51.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:51 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.859+0000 7f76fe51f700 1 -- 172.21.15.64:0/1171579830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 msgr2=0x7f76f8102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.859+0000 7f76fe51f700 1 --2- 172.21.15.64:0/1171579830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8102370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009910 tx=0x7f76e8009c20 comp rx=0 tx=0).stop 2024-05-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 -- 172.21.15.64:0/1171579830 shutdown_connections 2024-05-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 --2- 172.21.15.64:0/1171579830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8102370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 -- 172.21.15.64:0/1171579830 >> 172.21.15.64:0/1171579830 conn(0x7f76f80fd860 msgr2=0x7f76f80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 -- 172.21.15.64:0/1171579830 shutdown_connections 2024-05-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 -- 172.21.15.64:0/1171579830 wait complete. 2024-05-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 Processor -- start 2024-05-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.860+0000 7f76fe51f700 1 -- start start 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76fe51f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76fe51f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76f810c210 con 0x7f76f8101f90 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59440/0 (socket says 172.21.15.64:59440) 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f7fff700 1 -- 172.21.15.64:0/2890250453 learned_addr learned my addr 172.21.15.64:0/2890250453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f7fff700 1 -- 172.21.15.64:0/2890250453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f8197c90 con 0x7f76f8101f90 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f7fff700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8197750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76e80056d0 tx=0x7f76e80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76e8026070 con 0x7f76f8101f90 2024-05-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f8197f80 con 0x7f76f8101f90 2024-05-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e800bba0 con 0x7f76f8101f90 2024-05-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76e80037f0 con 0x7f76f8101f90 2024-05-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.861+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f819c020 con 0x7f76f8101f90 2024-05-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.862+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f76e8029070 con 0x7f76f8101f90 2024-05-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.862+0000 7f76f57fa700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76e00390f0 0x7f76e003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.863+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f76e801e070 con 0x7f76f8101f90 2024-05-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.863+0000 7f76f77fe700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76e00390f0 0x7f76e003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.863+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f8103470 con 0x7f76f8101f90 2024-05-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.863+0000 7f76f77fe700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76e00390f0 0x7f76e003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006fd0 tx=0x7f76ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:51.868+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76e8030070 con 0x7f76f8101f90 2024-05-09T08:59:52.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:52.080+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}) -- 0x7f76f8198ab0 con 0x7f76e00390f0 2024-05-09T08:59:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:52 smithi064 ceph-mon[30149]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:59:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:52 smithi064 ceph-mon[30149]: Deploying cephadm binary to smithi139 2024-05-09T08:59:56.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:56 smithi064 ceph-mon[30149]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:59:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.351+0000 7f76f57fa700 1 -- 172.21.15.64:0/2890250453 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f76f8198ab0 con 0x7f76e00390f0 2024-05-09T08:59:56.351 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi139' with addr '172.21.15.139' 2024-05-09T08:59:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76e00390f0 msgr2=0x7f76e003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76e00390f0 0x7f76e003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006fd0 tx=0x7f76ec006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 msgr2=0x7f76f8197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8197750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76e80056d0 tx=0x7f76e80057b0 comp rx=0 tx=0).stop 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 shutdown_connections 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76e00390f0 0x7f76e003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 --2- 172.21.15.64:0/2890250453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f8101f90 0x7f76f8197750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 >> 172.21.15.64:0/2890250453 conn(0x7f76f80fd860 msgr2=0x7f76f8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.353+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 shutdown_connections 2024-05-09T08:59:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:56.354+0000 7f76fe51f700 1 -- 172.21.15.64:0/2890250453 wait complete. 2024-05-09T08:59:56.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T08:59:57.208 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:59:57.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:57.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:57 smithi064 ceph-mon[30149]: Added host smithi139 2024-05-09T08:59:57.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:59:57.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:58.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:58 smithi064 ceph-mon[30149]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:59:58.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T08:59:58.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 -- 172.21.15.64:0/2328971646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 msgr2=0x7f98841032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:58.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 --2- 172.21.15.64:0/2328971646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f98841032b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f987400b0a0 tx=0x7f987400b3b0 comp rx=0 tx=0).stop 2024-05-09T08:59:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 -- 172.21.15.64:0/2328971646 shutdown_connections 2024-05-09T08:59:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 --2- 172.21.15.64:0/2328971646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f98841032b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 -- 172.21.15.64:0/2328971646 >> 172.21.15.64:0/2328971646 conn(0x7f98840fc720 msgr2=0x7f98840feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 -- 172.21.15.64:0/2328971646 shutdown_connections 2024-05-09T08:59:58.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.702+0000 7f9889947700 1 -- 172.21.15.64:0/2328971646 wait complete. 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9889947700 1 Processor -- start 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9889947700 1 -- start start 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9889947700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f9884197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9889947700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f988410c270 con 0x7f9884102ed0 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9882ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f9884197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9882ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f9884197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46118/0 (socket says 172.21.15.64:46118) 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9882ffd700 1 -- 172.21.15.64:0/1438147503 learned_addr learned my addr 172.21.15.64:0/1438147503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.703+0000 7f9882ffd700 1 -- 172.21.15.64:0/1438147503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9884197c80 con 0x7f9884102ed0 2024-05-09T08:59:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.704+0000 7f9882ffd700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f9884197740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98740076a0 tx=0x7f9874009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.704+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9874017040 con 0x7f9884102ed0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.704+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9874004040 con 0x7f9884102ed0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.704+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9884197f10 con 0x7f9884102ed0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.704+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f987401d450 con 0x7f9884102ed0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.704+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9884198370 con 0x7f9884102ed0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.705+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9874024020 con 0x7f9884102ed0 2024-05-09T08:59:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.705+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988406a8b0 con 0x7f9884102ed0 2024-05-09T08:59:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.705+0000 7f9888945700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f98700390f0 0x7f987003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.705+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9874022020 con 0x7f9884102ed0 2024-05-09T08:59:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.705+0000 7f98827fc700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f98700390f0 0x7f987003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.706+0000 7f98827fc700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f98700390f0 0x7f987003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f986c006fd0 tx=0x7f986c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:58.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.710+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f987402b020 con 0x7f9884102ed0 2024-05-09T08:59:58.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.922+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9884198aa0 con 0x7f98700390f0 2024-05-09T08:59:58.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.923+0000 7f9888945700 1 -- 172.21.15.64:0/1438147503 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f9884198aa0 con 0x7f98700390f0 2024-05-09T08:59:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T08:59:58.923 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}] 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f98700390f0 msgr2=0x7f987003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f98700390f0 0x7f987003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f986c006fd0 tx=0x7f986c006e40 comp rx=0 tx=0).stop 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 msgr2=0x7f9884197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f9884197740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98740076a0 tx=0x7f9874009800 comp rx=0 tx=0).stop 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 shutdown_connections 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f98700390f0 0x7f987003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 --2- 172.21.15.64:0/1438147503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9884102ed0 0x7f9884197740 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 >> 172.21.15.64:0/1438147503 conn(0x7f98840fc720 msgr2=0x7f9884103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 shutdown_connections 2024-05-09T08:59:58.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T08:59:58.926+0000 7f9889947700 1 -- 172.21.15.64:0/1438147503 wait complete. 2024-05-09T08:59:59.535 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-05-09T08:59:59.536 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:59:59.536 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-05-09T08:59:59.557 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T08:59:59.557 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:59.615 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-05-09T08:59:59.616 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch host add smithi179 2024-05-09T08:59:59.864 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:00:00.310 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:59 smithi064 ceph-mon[30149]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:00:00.310 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:59 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:00.310 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 08:59:59 smithi064 ceph-mon[30149]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.582+0000 7f7628ca3700 1 -- 172.21.15.64:0/687391374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 msgr2=0x7f76241032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.582+0000 7f7628ca3700 1 --2- 172.21.15.64:0/687391374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f76241032d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7614009910 tx=0x7f7614009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.582+0000 7f7628ca3700 1 -- 172.21.15.64:0/687391374 shutdown_connections 2024-05-09T09:00:01.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.582+0000 7f7628ca3700 1 --2- 172.21.15.64:0/687391374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f76241032d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:01.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.582+0000 7f7628ca3700 1 -- 172.21.15.64:0/687391374 >> 172.21.15.64:0/687391374 conn(0x7f76240fc740 msgr2=0x7f76240feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:01.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.583+0000 7f7628ca3700 1 -- 172.21.15.64:0/687391374 shutdown_connections 2024-05-09T09:00:01.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.583+0000 7f7628ca3700 1 -- 172.21.15.64:0/687391374 wait complete. 2024-05-09T09:00:01.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.583+0000 7f7628ca3700 1 Processor -- start 2024-05-09T09:00:01.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f7628ca3700 1 -- start start 2024-05-09T09:00:01.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f7628ca3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f7624100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:01.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f7628ca3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f762410c290 con 0x7f7624102ef0 2024-05-09T09:00:01.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f762259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f7624100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f762259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f7624100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46124/0 (socket says 172.21.15.64:46124) 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f762259c700 1 -- 172.21.15.64:0/196977201 learned_addr learned my addr 172.21.15.64:0/196977201 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f762259c700 1 -- 172.21.15.64:0/196977201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7624102a40 con 0x7f7624102ef0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.584+0000 7f762259c700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f7624100b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7614006010 tx=0x7f7614005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.585+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7614026070 con 0x7f7624102ef0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.585+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7624101090 con 0x7f7624102ef0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.585+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f761400bc00 con 0x7f7624102ef0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.585+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7614003910 con 0x7f7624102ef0 2024-05-09T09:00:01.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.585+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7624101550 con 0x7f7624102ef0 2024-05-09T09:00:01.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.586+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762406aab0 con 0x7f7624102ef0 2024-05-09T09:00:01.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.586+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7614029070 con 0x7f7624102ef0 2024-05-09T09:00:01.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.586+0000 7f761b7fe700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76100390f0 0x7f761003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:01.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.586+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f761401e070 con 0x7f7624102ef0 2024-05-09T09:00:01.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.586+0000 7f7621d9b700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76100390f0 0x7f761003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:01.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.587+0000 7f7621d9b700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76100390f0 0x7f761003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f760c006fd0 tx=0x7f760c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.591+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7614013070 con 0x7f7624102ef0 2024-05-09T09:00:01.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:01.802+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) -- 0x7f7624102080 con 0x7f76100390f0 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:03.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:03 smithi064 ceph-mon[30149]: Updating smithi139:/etc/ceph/ceph.conf 2024-05-09T09:00:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:03 smithi064 ceph-mon[30149]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:03 smithi064 ceph-mon[30149]: Deploying cephadm binary to smithi179 2024-05-09T09:00:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:03 smithi064 ceph-mon[30149]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:04.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:04 smithi064 ceph-mon[30149]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:00:04.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:04 smithi064 ceph-mon[30149]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:06.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.084+0000 7f761b7fe700 1 -- 172.21.15.64:0/196977201 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7624102080 con 0x7f76100390f0 2024-05-09T09:00:06.083 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76100390f0 msgr2=0x7f761003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76100390f0 0x7f761003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f760c006fd0 tx=0x7f760c006e40 comp rx=0 tx=0).stop 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 msgr2=0x7f7624100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f7624100b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7614006010 tx=0x7f7614005780 comp rx=0 tx=0).stop 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 shutdown_connections 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76100390f0 0x7f761003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 --2- 172.21.15.64:0/196977201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7624102ef0 0x7f7624100b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:06.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 >> 172.21.15.64:0/196977201 conn(0x7f76240fc740 msgr2=0x7f7624103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:06.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 shutdown_connections 2024-05-09T09:00:06.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:06.087+0000 7f7628ca3700 1 -- 172.21.15.64:0/196977201 wait complete. 2024-05-09T09:00:06.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:06 smithi064 ceph-mon[30149]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:06.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:06.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:06.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:06.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:06.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:06.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T09:00:06.985 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:00:07.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:07 smithi064 ceph-mon[30149]: Added host smithi179 2024-05-09T09:00:07.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:07.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 -- 172.21.15.64:0/3408682849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 msgr2=0x7f22bc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 --2- 172.21.15.64:0/3408682849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1012b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009910 tx=0x7f22ac009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 -- 172.21.15.64:0/3408682849 shutdown_connections 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 --2- 172.21.15.64:0/3408682849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1012b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 -- 172.21.15.64:0/3408682849 >> 172.21.15.64:0/3408682849 conn(0x7f22bc076780 msgr2=0x7f22bc076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 -- 172.21.15.64:0/3408682849 shutdown_connections 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.468+0000 7f22c240d700 1 -- 172.21.15.64:0/3408682849 wait complete. 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22c240d700 1 Processor -- start 2024-05-09T09:00:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22c240d700 1 -- start start 2024-05-09T09:00:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22c240d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22c240d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc10b100 con 0x7f22bc100ed0 2024-05-09T09:00:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50016/0 (socket says 172.21.15.64:50016) 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22bbfff700 1 -- 172.21.15.64:0/2292804417 learned_addr learned my addr 172.21.15.64:0/2292804417 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.469+0000 7f22bbfff700 1 -- 172.21.15.64:0/2292804417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc198b00 con 0x7f22bc100ed0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.470+0000 7f22bbfff700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1985c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0056d0 tx=0x7f22ac0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.470+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22ac026070 con 0x7f22bc100ed0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.470+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22ac00bba0 con 0x7f22bc100ed0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.470+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22ac0037f0 con 0x7f22bc100ed0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.470+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc198df0 con 0x7f22bc100ed0 2024-05-09T09:00:08.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.470+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc1992b0 con 0x7f22bc100ed0 2024-05-09T09:00:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.471+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f22ac029070 con 0x7f22bc100ed0 2024-05-09T09:00:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.471+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc102360 con 0x7f22bc100ed0 2024-05-09T09:00:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.471+0000 7f22b9ffb700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22a8039150 0x7f22a803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.471+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f22ac01e070 con 0x7f22bc100ed0 2024-05-09T09:00:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.471+0000 7f22b3fff700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22a8039150 0x7f22a803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.472+0000 7f22b3fff700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22a8039150 0x7f22a803b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f22a4006fd0 tx=0x7f22a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.476+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f22ac013070 con 0x7f22bc100ed0 2024-05-09T09:00:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.688+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22bc069d50 con 0x7f22a8039150 2024-05-09T09:00:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.689+0000 7f22b9ffb700 1 -- 172.21.15.64:0/2292804417 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f22bc069d50 con 0x7f22a8039150 2024-05-09T09:00:08.688 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:00:08.688 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-05-09T09:00:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22a8039150 msgr2=0x7f22a803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22a8039150 0x7f22a803b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f22a4006fd0 tx=0x7f22a4006e40 comp rx=0 tx=0).stop 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 msgr2=0x7f22bc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1985c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0056d0 tx=0x7f22ac0057b0 comp rx=0 tx=0).stop 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 shutdown_connections 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22a8039150 0x7f22a803b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 --2- 172.21.15.64:0/2292804417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22bc100ed0 0x7f22bc1985c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 >> 172.21.15.64:0/2292804417 conn(0x7f22bc076780 msgr2=0x7f22bc102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 shutdown_connections 2024-05-09T09:00:08.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:08.691+0000 7f22c240d700 1 -- 172.21.15.64:0/2292804417 wait complete. 2024-05-09T09:00:08.981 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:08 smithi064 ceph-mon[30149]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:09.275 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T09:00:09.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-09T09:00:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:00:09.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:09 smithi064 ceph-mon[30149]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:00:09.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:10.791 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:10 smithi064 ceph-mon[30149]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:10.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 -- 172.21.15.64:0/4157781273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 msgr2=0x7fec1c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:10.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 --2- 172.21.15.64:0/4157781273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c101270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00b0a0 tx=0x7fec0c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:00:10.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 -- 172.21.15.64:0/4157781273 shutdown_connections 2024-05-09T09:00:10.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 --2- 172.21.15.64:0/4157781273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c101270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:10.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 -- 172.21.15.64:0/4157781273 >> 172.21.15.64:0/4157781273 conn(0x7fec1c0fc720 msgr2=0x7fec1c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:10.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 -- 172.21.15.64:0/4157781273 shutdown_connections 2024-05-09T09:00:10.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.992+0000 7fec20946700 1 -- 172.21.15.64:0/4157781273 wait complete. 2024-05-09T09:00:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec20946700 1 Processor -- start 2024-05-09T09:00:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec20946700 1 -- start start 2024-05-09T09:00:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec20946700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:10.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec20946700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c10b110 con 0x7fec1c100e90 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec1a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec1a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50036/0 (socket says 172.21.15.64:50036) 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.993+0000 7fec1a59c700 1 -- 172.21.15.64:0/2961850510 learned_addr learned my addr 172.21.15.64:0/2961850510 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec1a59c700 1 -- 172.21.15.64:0/2961850510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c198ac0 con 0x7fec1c100e90 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec1a59c700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c198580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0076a0 tx=0x7fec0c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec0c017040 con 0x7fec1c100e90 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c198d50 con 0x7fec1c100e90 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec0c022070 con 0x7fec1c100e90 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec0c01d5e0 con 0x7fec1c100e90 2024-05-09T09:00:10.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.994+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c199210 con 0x7fec1c100e90 2024-05-09T09:00:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.995+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fec0c01d8a0 con 0x7fec1c100e90 2024-05-09T09:00:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.995+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c102370 con 0x7fec1c100e90 2024-05-09T09:00:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.995+0000 7fec137fe700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec080390f0 0x7fec0803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.995+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec0c00f050 con 0x7fec1c100e90 2024-05-09T09:00:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.995+0000 7fec19d9b700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec080390f0 0x7fec0803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:10.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:10.996+0000 7fec19d9b700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec080390f0 0x7fec0803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec04006fd0 tx=0x7fec04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:10.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.000+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec0c025c00 con 0x7fec1c100e90 2024-05-09T09:00:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.212+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fec1c063980 con 0x7fec1c100e90 2024-05-09T09:00:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.707+0000 7fec137fe700 1 -- 172.21.15.64:0/2961850510 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fec0c01c3b0 con 0x7fec1c100e90 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec080390f0 msgr2=0x7fec0803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec080390f0 0x7fec0803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec04006fd0 tx=0x7fec04006e40 comp rx=0 tx=0).stop 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 msgr2=0x7fec1c198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c198580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0076a0 tx=0x7fec0c003ce0 comp rx=0 tx=0).stop 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 shutdown_connections 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec080390f0 0x7fec0803b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 --2- 172.21.15.64:0/2961850510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1c100e90 0x7fec1c198580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 >> 172.21.15.64:0/2961850510 conn(0x7fec1c0fc720 msgr2=0x7fec1c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 shutdown_connections 2024-05-09T09:00:11.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:11.710+0000 7fec20946700 1 -- 172.21.15.64:0/2961850510 wait complete. 2024-05-09T09:00:11.711 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-05-09T09:00:12.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2961850510' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T09:00:12.408 INFO:tasks.cephadm:Adding mon.a on smithi064 2024-05-09T09:00:12.408 INFO:tasks.cephadm:Adding mon.b on smithi139 2024-05-09T09:00:12.408 INFO:tasks.cephadm:Adding mon.c on smithi179 2024-05-09T09:00:12.408 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c' 2024-05-09T09:00:12.645 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:13.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2961850510' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:12 smithi064 ceph-mon[30149]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/3096043010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 msgr2=0x7fb5e41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 --2- 172.21.15.179:0/3096043010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e41012b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009910 tx=0x7fb5d4009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/3096043010 shutdown_connections 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 --2- 172.21.15.179:0/3096043010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e41012b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/3096043010 >> 172.21.15.179:0/3096043010 conn(0x7fb5e4076780 msgr2=0x7fb5e4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/3096043010 shutdown_connections 2024-05-09T09:00:14.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.059+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/3096043010 wait complete. 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.060+0000 7fb5e8bc6700 1 Processor -- start 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.060+0000 7fb5e8bc6700 1 -- start start 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.060+0000 7fb5e8bc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.060+0000 7fb5e8bc6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e410b100 con 0x7fb5e4100ed0 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e419d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.179:37042/0 (socket says 172.21.15.179:37042) 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e259c700 1 -- 172.21.15.179:0/1790817789 learned_addr learned my addr 172.21.15.179:0/1790817789 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e259c700 1 -- 172.21.15.179:0/1790817789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e419d920 con 0x7fb5e4100ed0 2024-05-09T09:00:14.062 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e259c700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e419d3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40056d0 tx=0x7fb5d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d4026070 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e419dc10 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5d400bba0 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d40037f0 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.061+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e41975a0 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.062+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4102360 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.062+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb5d4029070 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.063+0000 7fb5dbfff700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5d00390f0 0x7fb5d003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.063+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb5d401e070 con 0x7fb5e4100ed0 2024-05-09T09:00:14.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.063+0000 7fb5d9d9b700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5d00390f0 0x7fb5d003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:14.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.063+0000 7fb5d9d9b700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5d00390f0 0x7fb5d003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006fd0 tx=0x7fb5cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:14.067 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.068+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb5d4013070 con 0x7fb5e4100ed0 2024-05-09T09:00:14.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:13 smithi064 ceph-mon[30149]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:13 smithi064 ceph-mon[30149]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:13 smithi064 ceph-mon[30149]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:00:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:14.273 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.274+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) -- 0x7fb5e4198ad0 con 0x7fb5d00390f0 2024-05-09T09:00:14.280 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.281+0000 7fb5dbfff700 1 -- 172.21.15.179:0/1790817789 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb5e4198ad0 con 0x7fb5d00390f0 2024-05-09T09:00:14.280 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-05-09T09:00:14.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5d00390f0 msgr2=0x7fb5d003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5d00390f0 0x7fb5d003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006fd0 tx=0x7fb5cc006e40 comp rx=0 tx=0).stop 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 msgr2=0x7fb5e419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e419d3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40056d0 tx=0x7fb5d40057b0 comp rx=0 tx=0).stop 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 shutdown_connections 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5d00390f0 0x7fb5d003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 --2- 172.21.15.179:0/1790817789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5e4100ed0 0x7fb5e419d3e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 >> 172.21.15.179:0/1790817789 conn(0x7fb5e4076780 msgr2=0x7fb5e4069740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 shutdown_connections 2024-05-09T09:00:14.283 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:14.283+0000 7fb5e8bc6700 1 -- 172.21.15.179:0/1790817789 wait complete. 2024-05-09T09:00:14.920 DEBUG:teuthology.orchestra.run.smithi139:mon.b> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.b.service 2024-05-09T09:00:14.923 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.c.service 2024-05-09T09:00:14.925 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T09:00:14.925 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:15.708 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:16.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:15 smithi064 ceph-mon[30149]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:15 smithi064 ceph-mon[30149]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c;count:3 2024-05-09T09:00:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:15 smithi064 ceph-mon[30149]: Deploying daemon mon.c on smithi179 2024-05-09T09:00:17.054 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:16 smithi064 ceph-mon[30149]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:18.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.472+0000 7f9fa77c1700 1 -- 172.21.15.179:0/1570019506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 msgr2=0x7f9fa01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.472+0000 7f9fa77c1700 1 --2- 172.21.15.179:0/1570019506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa01023a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009910 tx=0x7f9f9c009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.472+0000 7f9fa77c1700 1 -- 172.21.15.179:0/1570019506 shutdown_connections 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.472+0000 7f9fa77c1700 1 --2- 172.21.15.179:0/1570019506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa01023a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.472+0000 7f9fa77c1700 1 -- 172.21.15.179:0/1570019506 >> 172.21.15.179:0/1570019506 conn(0x7f9fa00fd870 msgr2=0x7f9fa00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa77c1700 1 -- 172.21.15.179:0/1570019506 shutdown_connections 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa77c1700 1 -- 172.21.15.179:0/1570019506 wait complete. 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa77c1700 1 Processor -- start 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa77c1700 1 -- start start 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa77c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa77c1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa010c240 con 0x7f9fa0101fc0 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa555d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa555d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.179:37056/0 (socket says 172.21.15.179:37056) 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.473+0000 7f9fa555d700 1 -- 172.21.15.179:0/750625151 learned_addr learned my addr 172.21.15.179:0/750625151 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:00:18.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9fa555d700 1 -- 172.21.15.179:0/750625151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa019d990 con 0x7f9fa0101fc0 2024-05-09T09:00:18.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9fa555d700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa019d450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0056d0 tx=0x7f9f9c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:18.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f9c026070 con 0x7f9fa0101fc0 2024-05-09T09:00:18.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f9c00bba0 con 0x7f9fa0101fc0 2024-05-09T09:00:18.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa019dc80 con 0x7f9fa0101fc0 2024-05-09T09:00:18.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f9c0037f0 con 0x7f9fa0101fc0 2024-05-09T09:00:18.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.474+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa0197610 con 0x7f9fa0101fc0 2024-05-09T09:00:18.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.475+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa01034a0 con 0x7f9fa0101fc0 2024-05-09T09:00:18.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.475+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9f9c029070 con 0x7f9fa0101fc0 2024-05-09T09:00:18.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.476+0000 7f9f967fc700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9f8c039150 0x7f9f8c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:18.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.476+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9f9c01e070 con 0x7f9fa0101fc0 2024-05-09T09:00:18.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.476+0000 7f9fa4d5c700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9f8c039150 0x7f9f8c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:18.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.476+0000 7f9fa4d5c700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9f8c039150 0x7f9f8c03b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f90006fd0 tx=0x7f9f90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:18.480 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.481+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f9c013070 con 0x7f9fa0101fc0 2024-05-09T09:00:18.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.784+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9fa0063980 con 0x7f9fa0101fc0 2024-05-09T09:00:18.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.784+0000 7f9f967fc700 1 -- 172.21.15.179:0/750625151 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9f9c030020 con 0x7f9fa0101fc0 2024-05-09T09:00:18.784 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T09:00:18.784 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","modified":"2024-05-09T08:58:28.429468Z","created":"2024-05-09T08:58:28.429468Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9f8c039150 msgr2=0x7f9f8c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9f8c039150 0x7f9f8c03b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f90006fd0 tx=0x7f9f90006e40 comp rx=0 tx=0).stop 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 msgr2=0x7f9fa019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa019d450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0056d0 tx=0x7f9f9c0057b0 comp rx=0 tx=0).stop 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 shutdown_connections 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9f8c039150 0x7f9f8c03b610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:18.786 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 --2- 172.21.15.179:0/750625151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fa0101fc0 0x7f9fa019d450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:18.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 >> 172.21.15.179:0/750625151 conn(0x7f9fa00fd870 msgr2=0x7f9fa0107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:18.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 shutdown_connections 2024-05-09T09:00:18.787 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:18.787+0000 7f9fa77c1700 1 -- 172.21.15.179:0/750625151 wait complete. 2024-05-09T09:00:18.787 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-05-09T09:00:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:18 smithi064 ceph-mon[30149]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout: e1 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout: legacy client fscid: -1 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout: No filesystems configured 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T09:00:19.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T09:00:19.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T09:00:19.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T09:00:19.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T09:00:20.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:19 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/750625151' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:20.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi139:/etc/ceph/ceph.conf 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Deploying cephadm binary to smithi179 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Added host smithi179 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2961850510' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2961850510' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c;count:3 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: Deploying daemon mon.c on smithi179 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/750625151' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 ceph-mon[34086]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 bash[34066]: 915ca2680a39e78ad55277d5e62d16cf8b5667269e07f1b49f531ad5bf4089b8 2024-05-09T09:00:20.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:19 smithi179 systemd[1]: Started Ceph mon.c for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:00:20.498 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T09:00:20.498 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T09:00:20.746 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:00:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 -- 172.21.15.179:0/1091836438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c0035c0 msgr2=0x7f542c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 --2- 172.21.15.179:0/1091836438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c0035c0 0x7f542c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5444069ce0 tx=0x7f543400feb0 comp rx=0 tx=0).stop 2024-05-09T09:00:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 -- 172.21.15.179:0/1091836438 shutdown_connections 2024-05-09T09:00:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 --2- 172.21.15.179:0/1091836438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c0035c0 0x7f542c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:25.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 -- 172.21.15.179:0/1091836438 >> 172.21.15.179:0/1091836438 conn(0x7f54440fc800 msgr2=0x7f54440fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 -- 172.21.15.179:0/1091836438 shutdown_connections 2024-05-09T09:00:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.218+0000 7f54499ea700 1 -- 172.21.15.179:0/1091836438 wait complete. 2024-05-09T09:00:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.219+0000 7f54499ea700 1 Processor -- start 2024-05-09T09:00:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.219+0000 7f54499ea700 1 -- start start 2024-05-09T09:00:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.219+0000 7f54499ea700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:25.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.219+0000 7f54499ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5444101210 0x7f5444101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:25.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.219+0000 7f54499ea700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5444109d60 con 0x7f5444101210 2024-05-09T09:00:25.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.219+0000 7f54499ea700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5444109be0 con 0x7f542c0035c0 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40748/0 (socket says 172.21.15.179:40748) 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 -- 172.21.15.179:0/3117138670 learned_addr learned my addr 172.21.15.179:0/3117138670 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 -- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 msgr2=0x7f5444100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 -- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 msgr2=0x7f5444100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f54427fc700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5444101210 0x7f5444101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f5442ffd700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:00:25.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f54427fc700 1 -- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 msgr2=0x7f5444100cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f54427fc700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f54427fc700 1 -- 172.21.15.179:0/3117138670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5444102ba0 con 0x7f5444101210 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.220+0000 7f54427fc700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5444101210 0x7f5444101690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f543800b4d0 tx=0x7f543800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.221+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5438011630 con 0x7f5444101210 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.221+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5438011c80 con 0x7f5444101210 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.221+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54441a3bb0 con 0x7f5444101210 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.221+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5438010da0 con 0x7f5444101210 2024-05-09T09:00:25.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.221+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54441a4110 con 0x7f5444101210 2024-05-09T09:00:25.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.222+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5438018020 con 0x7f5444101210 2024-05-09T09:00:25.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.222+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544406a4e0 con 0x7f5444101210 2024-05-09T09:00:25.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.222+0000 7f54489e8700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5430038ee0 0x7f543003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:25.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.222+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f543804bba0 con 0x7f5444101210 2024-05-09T09:00:25.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.222+0000 7f5442ffd700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5430038ee0 0x7f543003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:25.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.223+0000 7f5442ffd700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5430038ee0 0x7f543003b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f543400fa60 tx=0x7f5434005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:25.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.227+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f543801f020 con 0x7f5444101210 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: mon.a calling monitor election 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: mon.c calling monitor election 2024-05-09T09:00:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: monmap epoch 2 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: last_changed 2024-05-09T09:00:19.939664+0000 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: min_mon_release 19 (squid) 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: election_strategy: 1 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-09T09:00:25.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: fsmap 2024-05-09T09:00:25.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:25.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: mgrmap e13: x(active, since 49s) 2024-05-09T09:00:25.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: overall HEALTH_OK 2024-05-09T09:00:25.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:25.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:25.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:00:25.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: mon.a calling monitor election 2024-05-09T09:00:25.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: mon.c calling monitor election 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: monmap epoch 2 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: last_changed 2024-05-09T09:00:19.939664+0000 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: min_mon_release 19 (squid) 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: election_strategy: 1 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T09:00:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-09T09:00:25.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: fsmap 2024-05-09T09:00:25.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:25.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: mgrmap e13: x(active, since 49s) 2024-05-09T09:00:25.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: overall HEALTH_OK 2024-05-09T09:00:25.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:25.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:25.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.505+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f54441a4490 con 0x7f5444101210 2024-05-09T09:00:25.504 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.505+0000 7f54489e8700 1 -- 172.21.15.179:0/3117138670 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5438018310 con 0x7f5444101210 2024-05-09T09:00:25.505 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T09:00:25.505 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","modified":"2024-05-09T09:00:19.939664Z","created":"2024-05-09T08:58:28.429468Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T09:00:25.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5430038ee0 msgr2=0x7f543003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:25.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5430038ee0 0x7f543003b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f543400fa60 tx=0x7f5434005c20 comp rx=0 tx=0).stop 2024-05-09T09:00:25.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5444101210 msgr2=0x7f5444101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:25.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5444101210 0x7f5444101690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f543800b4d0 tx=0x7f543800b890 comp rx=0 tx=0).stop 2024-05-09T09:00:25.507 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 shutdown_connections 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5430038ee0 0x7f543003b3a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542c0035c0 0x7f5444100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 --2- 172.21.15.179:0/3117138670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5444101210 0x7f5444101690 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 >> 172.21.15.179:0/3117138670 conn(0x7f54440fc800 msgr2=0x7f5444103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 shutdown_connections 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:25.508+0000 7f54499ea700 1 -- 172.21.15.179:0/3117138670 wait complete. 2024-05-09T09:00:25.508 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-05-09T09:00:26.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:26.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:26.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:26.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:26.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: Deploying daemon mon.b on smithi139 2024-05-09T09:00:26.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/3117138670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:26.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:26.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: Deploying daemon mon.b on smithi139 2024-05-09T09:00:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/3117138670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:27.079 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T09:00:27.079 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T09:00:27.326 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:00:27.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:00:26 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:00:26.939+0000 7f4de47be700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T09:00:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:28 smithi064 ceph-mon[30149]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:28.412 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:28 smithi179 ceph-mon[34086]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 -- 172.21.15.179:0/489994415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 msgr2=0x7f04b8101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 --2- 172.21.15.179:0/489994415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b8101c60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009910 tx=0x7f04a8009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 -- 172.21.15.179:0/489994415 shutdown_connections 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 --2- 172.21.15.179:0/489994415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b8101c60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 -- 172.21.15.179:0/489994415 >> 172.21.15.179:0/489994415 conn(0x7f04b80773d0 msgr2=0x7f04b80777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 -- 172.21.15.179:0/489994415 shutdown_connections 2024-05-09T09:00:28.722 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 -- 172.21.15.179:0/489994415 wait complete. 2024-05-09T09:00:28.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 Processor -- start 2024-05-09T09:00:28.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.723+0000 7f04bea50700 1 -- start start 2024-05-09T09:00:28.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04bea50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:28.724 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04bea50700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04bea50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04b810b2b0 con 0x7f04b8101880 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04bea50700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04b810b130 con 0x7f04b8198e20 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40750/0 (socket says 172.21.15.179:40750) 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 -- 172.21.15.179:0/566481719 learned_addr learned my addr 172.21.15.179:0/566481719 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04b7fff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 -- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 msgr2=0x7f04b81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 -- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 msgr2=0x7f04b81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04affff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04b7fff700 1 -- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 msgr2=0x7f04b81992a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04b7fff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.724+0000 7f04b7fff700 1 -- 172.21.15.179:0/566481719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b8101450 con 0x7f04b8101880 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.725+0000 7f04b7fff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b81988e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f04a8014000 tx=0x7f04a8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:28.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.725+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f04a8005dc0 con 0x7f04b8101880 2024-05-09T09:00:28.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.725+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b81a1a70 con 0x7f04b8101880 2024-05-09T09:00:28.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.725+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a800b6f0 con 0x7f04b8101880 2024-05-09T09:00:28.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.725+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f04a80038f0 con 0x7f04b8101880 2024-05-09T09:00:28.726 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.725+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b81a2000 con 0x7f04b8101880 2024-05-09T09:00:28.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.726+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8102630 con 0x7f04b8101880 2024-05-09T09:00:28.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.726+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f04a8029070 con 0x7f04b8101880 2024-05-09T09:00:28.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.726+0000 7f04b5ffb700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04a4038e90 0x7f04a403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:28.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.726+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f04a801e070 con 0x7f04b8101880 2024-05-09T09:00:28.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.726+0000 7f04affff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04a4038e90 0x7f04a403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:28.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.727+0000 7f04affff700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04a4038e90 0x7f04a403b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009180 tx=0x7f04a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:28.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.731+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f04a8013070 con 0x7f04b8101880 2024-05-09T09:00:28.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:28.895+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8029360 con 0x7f04b8101880 2024-05-09T09:00:29.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:29.011+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f04b81a2360 con 0x7f04b8101880 2024-05-09T09:00:29.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:28 smithi139 systemd[1]: Started Ceph mon.b for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:00:33.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.905+0000 7f04b5ffb700 1 -- 172.21.15.179:0/566481719 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f04a800b8a0 con 0x7f04b8101880 2024-05-09T09:00:33.905 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-09T09:00:33.905 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","modified":"2024-05-09T09:00:28.894715Z","created":"2024-05-09T08:58:28.429468Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04a4038e90 msgr2=0x7f04a403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04a4038e90 0x7f04a403b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009180 tx=0x7f04a0008040 comp rx=0 tx=0).stop 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 msgr2=0x7f04b81988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b81988e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f04a8014000 tx=0x7f04a8005030 comp rx=0 tx=0).stop 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 shutdown_connections 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04b8101880 0x7f04b81988e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04a4038e90 0x7f04a403b350 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 --2- 172.21.15.179:0/566481719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b8198e20 0x7f04b81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 >> 172.21.15.179:0/566481719 conn(0x7f04b80773d0 msgr2=0x7f04b81029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.909+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 shutdown_connections 2024-05-09T09:00:33.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:33.910+0000 7f04bea50700 1 -- 172.21.15.179:0/566481719 wait complete. 2024-05-09T09:00:33.910 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-05-09T09:00:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:00:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: mon.a calling monitor election 2024-05-09T09:00:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: mon.c calling monitor election 2024-05-09T09:00:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/566481719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: mon.b calling monitor election 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: monmap epoch 3 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: last_changed 2024-05-09T09:00:28.894715+0000 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: min_mon_release 19 (squid) 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: election_strategy: 1 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: fsmap 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: mgrmap e13: x(active, since 58s) 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: overall HEALTH_OK 2024-05-09T09:00:34.231 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:34.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: mon.a calling monitor election 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: mon.c calling monitor election 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/566481719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: mon.b calling monitor election 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: monmap epoch 3 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: last_changed 2024-05-09T09:00:28.894715+0000 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T09:00:34.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: min_mon_release 19 (squid) 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: election_strategy: 1 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: fsmap 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: mgrmap e13: x(active, since 58s) 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: overall HEALTH_OK 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:34.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: mon.a calling monitor election 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: mon.c calling monitor election 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/566481719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: mon.b calling monitor election 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: monmap epoch 3 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: last_changed 2024-05-09T09:00:28.894715+0000 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: created 2024-05-09T08:58:28.429468+0000 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: min_mon_release 19 (squid) 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: election_strategy: 1 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-09T09:00:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: 2: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-05-09T09:00:34.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: fsmap 2024-05-09T09:00:34.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T09:00:34.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: mgrmap e13: x(active, since 58s) 2024-05-09T09:00:34.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: overall HEALTH_OK 2024-05-09T09:00:34.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:34.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:34.535 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T09:00:34.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T09:00:34.803 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:00:35.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:35.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:35.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:35.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:00:36.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:36.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: Updating smithi139:/etc/ceph/ceph.conf 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: Updating smithi064:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:36 smithi064 ceph-mon[30149]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 -- 172.21.15.64:0/1521937935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc107dc0 msgr2=0x7f4bdc1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 --2- 172.21.15.64:0/1521937935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc107dc0 0x7f4bdc1081a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009910 tx=0x7f4bcc009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 -- 172.21.15.64:0/1521937935 shutdown_connections 2024-05-09T09:00:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 --2- 172.21.15.64:0/1521937935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc107dc0 0x7f4bdc1081a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 -- 172.21.15.64:0/1521937935 >> 172.21.15.64:0/1521937935 conn(0x7f4bdc076780 msgr2=0x7f4bdc076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 -- 172.21.15.64:0/1521937935 shutdown_connections 2024-05-09T09:00:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.360+0000 7f4be24f3700 1 -- 172.21.15.64:0/1521937935 wait complete. 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 Processor -- start 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 -- start start 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bdc107dc0 0x7f4bdc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 0x7f4bdc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bdc197d40 0x7f4bdc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc10c200 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc10c080 con 0x7f4bdc107dc0 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4be24f3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc10c380 con 0x7f4bdc197d40 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.361+0000 7f4bdb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 0x7f4bdc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 0x7f4bdc197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55168/0 (socket says 172.21.15.64:55168) 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 -- 172.21.15.64:0/435695667 learned_addr learned my addr 172.21.15.64:0/435695667 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4be0a90700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bdc197d40 0x7f4bdc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdbfff700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bdc107dc0 0x7f4bdc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 -- 172.21.15.64:0/435695667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bdc197d40 msgr2=0x7f4bdc1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bdc197d40 0x7f4bdc1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 -- 172.21.15.64:0/435695667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bdc107dc0 msgr2=0x7f4bdc19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bdc107dc0 0x7f4bdc19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 -- 172.21.15.64:0/435695667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc1a44e0 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bdb7fe700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 0x7f4bdc197800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b780 tx=0x7f4bd000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0004230 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bdc1a4770 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bd00043d0 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0011b60 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.362+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bdc1a4cd0 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.363+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4bd001a400 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.363+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bdc103470 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.363+0000 7f4bd97fa700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4bc4039580 0x7f4bc403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.364+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4bd004cc40 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.364+0000 7f4bdbfff700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4bc4039580 0x7f4bc403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:36.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.364+0000 7f4bdbfff700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4bc4039580 0x7f4bc403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009750 tx=0x7f4bcc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:36.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.369+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4bd0020500 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: Updating smithi139:/etc/ceph/ceph.conf 2024-05-09T09:00:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-09T09:00:36.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: Updating smithi064:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:36 smithi179 ceph-mon[34086]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: Updating smithi139:/etc/ceph/ceph.conf 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-09T09:00:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: Updating smithi064:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: Updating smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:36 smithi139 ceph-mon[33599]: Updating smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T09:00:36.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.578+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4bdc1a4fc0 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.578+0000 7f4bd97fa700 1 -- 172.21.15.64:0/435695667 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f4bd00202a0 con 0x7f4bdc19dbc0 2024-05-09T09:00:36.577 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:36.577 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-05-09T09:00:36.578 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:00:36.578 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.580+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4bc4039580 msgr2=0x7f4bc403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.580+0000 7f4be24f3700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4bc4039580 0x7f4bc403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009750 tx=0x7f4bcc01a040 comp rx=0 tx=0).stop 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.580+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 msgr2=0x7f4bdc197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.580+0000 7f4be24f3700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 0x7f4bdc197800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b780 tx=0x7f4bd000ba90 comp rx=0 tx=0).stop 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 shutdown_connections 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4bc4039580 0x7f4bc403ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bdc197d40 0x7f4bdc1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bdc107dc0 0x7f4bdc19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 --2- 172.21.15.64:0/435695667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bdc19dbc0 0x7f4bdc197800 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 >> 172.21.15.64:0/435695667 conn(0x7f4bdc076780 msgr2=0x7f4bdc0fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 shutdown_connections 2024-05-09T09:00:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:36.581+0000 7f4be24f3700 1 -- 172.21.15.64:0/435695667 wait complete. 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.176 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: Reconfiguring daemon mon.a on smithi064 2024-05-09T09:00:37.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:37 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/435695667' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:37.203 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T09:00:37.203 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:00:37.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T09:00:37.233 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:00:37.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:37.299 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T09:00:37.299 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: Reconfiguring daemon mon.a on smithi064 2024-05-09T09:00:37.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:37 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/435695667' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:37.335 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T09:00:37.335 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:37.403 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T09:00:37.403 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T09:00:37.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: Reconfiguring daemon mon.a on smithi064 2024-05-09T09:00:37.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:37 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/435695667' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:37.436 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T09:00:37.437 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T09:00:37.506 INFO:tasks.cephadm:Adding mgr.x on smithi064 2024-05-09T09:00:37.506 INFO:tasks.cephadm:Adding mgr.y on smithi139 2024-05-09T09:00:37.506 INFO:tasks.cephadm:Adding mgr.z on smithi179 2024-05-09T09:00:37.507 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi064=x;smithi139=y;smithi179=z' 2024-05-09T09:00:37.805 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:00:38.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:38 smithi139 ceph-mon[33599]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:38.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:38 smithi064 ceph-mon[30149]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:38.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:38 smithi179 ceph-mon[34086]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:39.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 -- 172.21.15.179:0/2924495021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80100d90 msgr2=0x7f6f801011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:39.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 --2- 172.21.15.179:0/2924495021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80100d90 0x7f6f801011b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009910 tx=0x7f6f74009c20 comp rx=0 tx=0).stop 2024-05-09T09:00:39.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 -- 172.21.15.179:0/2924495021 shutdown_connections 2024-05-09T09:00:39.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 --2- 172.21.15.179:0/2924495021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80100d90 0x7f6f801011b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 -- 172.21.15.179:0/2924495021 >> 172.21.15.179:0/2924495021 conn(0x7f6f800fc330 msgr2=0x7f6f800fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 -- 172.21.15.179:0/2924495021 shutdown_connections 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 -- 172.21.15.179:0/2924495021 wait complete. 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.324+0000 7f6f86be2700 1 Processor -- start 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 -- start start 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80100d90 0x7f6f801975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80197b10 0x7f6f8019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 0x7f6f8019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80106cd0 con 0x7f6f80197b10 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80106b50 con 0x7f6f80100d90 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f86be2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80106e50 con 0x7f6f80197f90 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 0x7f6f8019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:39.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 0x7f6f8019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53494/0 (socket says 172.21.15.179:53494) 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f7ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80197b10 0x7f6f8019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8497e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80100d90 0x7f6f801975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 -- 172.21.15.179:0/143545394 learned_addr learned my addr 172.21.15.179:0/143545394 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 -- 172.21.15.179:0/143545394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80100d90 msgr2=0x7f6f801975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80100d90 0x7f6f801975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 -- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80197b10 msgr2=0x7f6f8019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80197b10 0x7f6f8019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 -- 172.21.15.179:0/143545394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019f9e0 con 0x7f6f80197f90 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f7ffff700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80197b10 0x7f6f8019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f8517f700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 0x7f6f8019f110 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800ba50 tx=0x7f6f7800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.325+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78019070 con 0x7f6f80197f90 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.326+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019fcd0 con 0x7f6f80197f90 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.326+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f7800cde0 con 0x7f6f80197f90 2024-05-09T09:00:39.327 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.326+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78007350 con 0x7f6f80197f90 2024-05-09T09:00:39.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.326+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801a0240 con 0x7f6f80197f90 2024-05-09T09:00:39.328 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.327+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6f7801c070 con 0x7f6f80197f90 2024-05-09T09:00:39.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.327+0000 7f6f7dffb700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f70039580 0x7f6f7003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:39.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.327+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f7804b820 con 0x7f6f80197f90 2024-05-09T09:00:39.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.327+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80063980 con 0x7f6f80197f90 2024-05-09T09:00:39.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.327+0000 7f6f8497e700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f70039580 0x7f6f7003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:39.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.328+0000 7f6f8497e700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f70039580 0x7f6f7003ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005190 tx=0x7f6f7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:39.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.332+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f78014440 con 0x7f6f80197f90 2024-05-09T09:00:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:39 smithi179 ceph-mon[34086]: Reconfiguring mon.b (monmap changed)... 2024-05-09T09:00:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:39 smithi179 ceph-mon[34086]: Reconfiguring daemon mon.b on smithi139 2024-05-09T09:00:39.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.540+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi139=y;smithi179=z", "target": ["mon-mgr", ""]}) -- 0x7f6f80062760 con 0x7f6f70039580 2024-05-09T09:00:39.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.547+0000 7f6f7dffb700 1 -- 172.21.15.179:0/143545394 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6f80062760 con 0x7f6f70039580 2024-05-09T09:00:39.545 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f70039580 msgr2=0x7f6f7003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f70039580 0x7f6f7003ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005190 tx=0x7f6f7401a040 comp rx=0 tx=0).stop 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 msgr2=0x7f6f8019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 0x7f6f8019f110 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800ba50 tx=0x7f6f7800bd60 comp rx=0 tx=0).stop 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 shutdown_connections 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f70039580 0x7f6f7003ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f80197f90 0x7f6f8019f110 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f80100d90 0x7f6f801975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 --2- 172.21.15.179:0/143545394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f80197b10 0x7f6f8019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 >> 172.21.15.179:0/143545394 conn(0x7f6f800fc330 msgr2=0x7f6f8018e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 shutdown_connections 2024-05-09T09:00:39.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:00:39.549+0000 7f6f86be2700 1 -- 172.21.15.179:0/143545394 wait complete. 2024-05-09T09:00:39.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:39 smithi139 ceph-mon[33599]: Reconfiguring mon.b (monmap changed)... 2024-05-09T09:00:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:39 smithi139 ceph-mon[33599]: Reconfiguring daemon mon.b on smithi139 2024-05-09T09:00:39.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:39 smithi064 ceph-mon[30149]: Reconfiguring mon.b (monmap changed)... 2024-05-09T09:00:39.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:39 smithi064 ceph-mon[30149]: Reconfiguring daemon mon.b on smithi139 2024-05-09T09:00:40.149 DEBUG:teuthology.orchestra.run.smithi139:mgr.y> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.y.service 2024-05-09T09:00:40.152 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.z.service 2024-05-09T09:00:40.154 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T09:00:40.155 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:00:40.155 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T09:00:40.175 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T09:00:40.175 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-09 08:59:44.845519048 +0000 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-09 08:55:06.903705081 +0000 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-09 08:55:06.903705081 +0000 2024-05-09T09:00:40.237 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-09 08:55:06.903705081 +0000 2024-05-09T09:00:40.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T09:00:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-09T09:00:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-09T09:00:40.307 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000354151 s, 1.4 MB/s 2024-05-09T09:00:40.309 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T09:00:40.368 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-09 08:59:44.881518341 +0000 2024-05-09T09:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-09 08:55:07.251698022 +0000 2024-05-09T09:00:40.428 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-09 08:55:07.251698022 +0000 2024-05-09T09:00:40.428 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-09 08:55:07.251698022 +0000 2024-05-09T09:00:40.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T09:00:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-09T09:00:40.494 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-09T09:00:40.494 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000208317 s, 2.5 MB/s 2024-05-09T09:00:40.495 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T09:00:40.555 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-09 08:59:44.915517673 +0000 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-09 08:55:07.604690861 +0000 2024-05-09T09:00:40.612 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-09 08:55:07.604690861 +0000 2024-05-09T09:00:40.613 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-09 08:55:07.604690861 +0000 2024-05-09T09:00:40.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi139=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: Saving service mgr spec with placement smithi064=x;smithi139=y;smithi179=z;count:3 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: Reconfiguring mon.c (monmap changed)... 2024-05-09T09:00:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:40.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:40.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:40.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:40 smithi064 ceph-mon[30149]: Reconfiguring daemon mon.c on smithi179 2024-05-09T09:00:40.679 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-09T09:00:40.679 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-09T09:00:40.679 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000223941 s, 2.3 MB/s 2024-05-09T09:00:40.680 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T09:00:40.738 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-05-09T09:00:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T09:00:40.795 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:40.795 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T09:00:40.795 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:40.795 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:40.795 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-09 08:59:44.951516966 +0000 2024-05-09T09:00:40.796 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-09 08:55:07.947683903 +0000 2024-05-09T09:00:40.796 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-09 08:55:07.947683903 +0000 2024-05-09T09:00:40.796 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-09 08:55:07.947683903 +0000 2024-05-09T09:00:40.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T09:00:40.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi139=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: Saving service mgr spec with placement smithi064=x;smithi139=y;smithi179=z;count:3 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: Reconfiguring mon.c (monmap changed)... 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:40 smithi139 ceph-mon[33599]: Reconfiguring daemon mon.c on smithi179 2024-05-09T09:00:40.861 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-09T09:00:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-09T09:00:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000287677 s, 1.8 MB/s 2024-05-09T09:00:40.863 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T09:00:40.920 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-05-09T09:00:40.920 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T09:00:40.941 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T09:00:40.941 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-05-09T09:00:40.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:40.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi139=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:40.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: Saving service mgr spec with placement smithi064=x;smithi139=y;smithi179=z;count:3 2024-05-09T09:00:40.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: Reconfiguring mon.c (monmap changed)... 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:40 smithi179 ceph-mon[34086]: Reconfiguring daemon mon.c on smithi179 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-05-09 09:00:28.337668900 +0000 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-05-09 08:55:00.531349068 +0000 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-05-09 08:55:00.531349068 +0000 2024-05-09T09:00:41.000 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-05-09 08:55:00.531349068 +0000 2024-05-09T09:00:41.000 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T09:00:41.068 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-05-09T09:00:41.068 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-05-09T09:00:41.068 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000218772 s, 2.3 MB/s 2024-05-09T09:00:41.070 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T09:00:41.129 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-05-09 09:00:28.337668900 +0000 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-05-09 08:55:00.870341769 +0000 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-05-09 08:55:00.870341769 +0000 2024-05-09T09:00:41.186 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-05-09 08:55:00.870341769 +0000 2024-05-09T09:00:41.186 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T09:00:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-05-09T09:00:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-05-09T09:00:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000208734 s, 2.5 MB/s 2024-05-09T09:00:41.253 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T09:00:41.311 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-05-09T09:00:41.368 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T09:00:41.368 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-05-09 09:00:28.337668900 +0000 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-05-09 08:55:01.217334296 +0000 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-05-09 08:55:01.217334296 +0000 2024-05-09T09:00:41.369 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-05-09 08:55:01.217334296 +0000 2024-05-09T09:00:41.369 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T09:00:41.434 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-05-09T09:00:41.434 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-05-09T09:00:41.434 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000229295 s, 2.2 MB/s 2024-05-09T09:00:41.436 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T09:00:41.494 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-05-09 09:00:28.338668879 +0000 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-05-09 08:55:01.533327491 +0000 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-05-09 08:55:01.533327491 +0000 2024-05-09T09:00:41.551 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-05-09 08:55:01.533327491 +0000 2024-05-09T09:00:41.551 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T09:00:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-05-09T09:00:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-05-09T09:00:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000202834 s, 2.5 MB/s 2024-05-09T09:00:41.618 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T09:00:41.675 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-09T09:00:41.675 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T09:00:41.693 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T09:00:41.693 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-05-09T09:00:41.749 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T09:00:41.749 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-09 09:00:19.214459526 +0000 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-09 08:55:03.856016448 +0000 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-09 08:55:03.856016448 +0000 2024-05-09T09:00:41.750 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-09 08:55:03.856016448 +0000 2024-05-09T09:00:41.750 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T09:00:41.814 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-09T09:00:41.814 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-09T09:00:41.815 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000210555 s, 2.4 MB/s 2024-05-09T09:00:41.816 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T09:00:41.874 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-05-09T09:00:41.929 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-09 09:00:19.214459526 +0000 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-09 08:55:04.205009258 +0000 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-09 08:55:04.205009258 +0000 2024-05-09T09:00:41.930 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-09 08:55:04.205009258 +0000 2024-05-09T09:00:41.930 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T09:00:41.994 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-09T09:00:41.994 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-09T09:00:41.994 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000194553 s, 2.6 MB/s 2024-05-09T09:00:41.995 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T09:00:42.053 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-05-09T09:00:42.108 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-09 09:00:19.215459505 +0000 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-09 08:55:04.538002398 +0000 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-09 08:55:04.538002398 +0000 2024-05-09T09:00:42.109 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-09 08:55:04.538002398 +0000 2024-05-09T09:00:42.109 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T09:00:42.172 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-09T09:00:42.172 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-09T09:00:42.172 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000251775 s, 2.0 MB/s 2024-05-09T09:00:42.173 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T09:00:42.230 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-05-09T09:00:42.286 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T09:00:42.286 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T09:00:42.286 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T09:00:42.286 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T09:00:42.286 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T09:00:42.286 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-09 09:00:19.215459505 +0000 2024-05-09T09:00:42.287 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-09 08:55:04.872995496 +0000 2024-05-09T09:00:42.287 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-09 08:55:04.872995496 +0000 2024-05-09T09:00:42.287 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-09 08:55:04.872995496 +0000 2024-05-09T09:00:42.287 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.348 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:42.349 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:00:42.349 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:42.349 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:42.349 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:42 smithi179 ceph-mon[34086]: Deploying daemon mgr.z on smithi179 2024-05-09T09:00:42.350 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-09T09:00:42.350 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-09T09:00:42.350 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000191833 s, 2.7 MB/s 2024-05-09T09:00:42.352 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:42 smithi064 ceph-mon[30149]: Deploying daemon mgr.z on smithi179 2024-05-09T09:00:42.408 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2024-05-09T09:00:42.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:42.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:00:42.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:42.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:42.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:42 smithi139 ceph-mon[33599]: Deploying daemon mgr.z on smithi179 2024-05-09T09:00:42.661 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:00:43.870 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:43 smithi179 systemd[1]: Starting Ceph mgr.z for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:00:44.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:44 smithi179 ceph-mon[34086]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:44.131 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:43 smithi179 podman[35920]: 2024-05-09 09:00:43.871485153 +0000 UTC m=+0.096216875 container create 5170f2b17a6448f422403346fed04b4fe1e33e68e7cc39a1170e036951332223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T09:00:44.131 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:43 smithi179 podman[35920]: 2024-05-09 09:00:43.792912878 +0000 UTC m=+0.017644618 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:00:44.131 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 podman[35920]: 2024-05-09 09:00:44.057145152 +0000 UTC m=+0.281876882 container init 5170f2b17a6448f422403346fed04b4fe1e33e68e7cc39a1170e036951332223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T09:00:44.131 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 podman[35920]: 2024-05-09 09:00:44.059806927 +0000 UTC m=+0.284538649 container start 5170f2b17a6448f422403346fed04b4fe1e33e68e7cc39a1170e036951332223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T09:00:44.131 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 ceph-mgr[35939]: -- 172.21.15.179:0/2703726993 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56114890e5a0 con 0x5611488e2800 2024-05-09T09:00:44.131 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 bash[35920]: 5170f2b17a6448f422403346fed04b4fe1e33e68e7cc39a1170e036951332223 2024-05-09T09:00:44.300 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:44 smithi064 ceph-mon[30149]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:44.397 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 systemd[1]: Started Ceph mgr.z for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:00:44.398 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:44.212+0000 7f31dce84200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T09:00:44.398 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:44.397+0000 7f31dce84200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:00:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:44 smithi139 ceph-mon[33599]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:44.655 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T09:00:44.655 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T09:00:44.655 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-09T09:00:44.655 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-09T09:00:44.655 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011299 s, 928 MB/s 2024-05-09T09:00:44.656 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-09T09:00:44.729 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:44.498+0000 7f31dce84200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T09:00:45.202 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:44.809+0000 7f31dce84200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:00:45.202 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:44.895+0000 7f31dce84200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:45.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:45 smithi064 ceph-mon[30149]: Deploying daemon mgr.y on smithi139 2024-05-09T09:00:45.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:45 smithi179 ceph-mon[34086]: Deploying daemon mgr.y on smithi139 2024-05-09T09:00:45.495 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:00:45.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:45 smithi139 ceph-mon[33599]: Deploying daemon mgr.y on smithi139 2024-05-09T09:00:46.235 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 systemd[1]: Starting Ceph mgr.y for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:00:46.471 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:46 smithi179 ceph-mon[34086]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:46.471 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:46.128+0000 7f31dce84200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:00:46.471 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:46.302+0000 7f31dce84200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:00:46.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:46 smithi064 ceph-mon[30149]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:46.521 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:46 smithi139 ceph-mon[33599]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:46.522 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 podman[34960]: 2024-05-09 09:00:46.321630986 +0000 UTC m=+0.104546745 container create 92f936f5be87ace2a0483581b84c6ce49d1cc03ef98b5d20149c254911e63b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T09:00:46.522 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 podman[34960]: 2024-05-09 09:00:46.235059241 +0000 UTC m=+0.017975013 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:00:46.522 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 podman[34960]: 2024-05-09 09:00:46.523706111 +0000 UTC m=+0.306621876 container init 92f936f5be87ace2a0483581b84c6ce49d1cc03ef98b5d20149c254911e63b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:00:46.729 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:46.480+0000 7f31dce84200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:00:46.822 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 podman[34960]: 2024-05-09 09:00:46.526488624 +0000 UTC m=+0.309404384 container start 92f936f5be87ace2a0483581b84c6ce49d1cc03ef98b5d20149c254911e63b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T09:00:46.823 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 ceph-mgr[34979]: -- 172.21.15.139:0/2735670804 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d9496a5a0 con 0x559d9493ec00 2024-05-09T09:00:46.823 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 bash[34960]: 92f936f5be87ace2a0483581b84c6ce49d1cc03ef98b5d20149c254911e63b5a 2024-05-09T09:00:46.823 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 systemd[1]: Started Ceph mgr.y for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:00:46.823 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:46.685+0000 7f75edfca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 -- 172.21.15.64:0/577549991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103f20 msgr2=0x7f130810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 --2- 172.21.15.64:0/577549991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103f20 0x7f130810a7d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f130000b0d0 tx=0x7f130000b3e0 comp rx=0 tx=0).stop 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 -- 172.21.15.64:0/577549991 shutdown_connections 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 --2- 172.21.15.64:0/577549991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103f20 0x7f130810a7d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 --2- 172.21.15.64:0/577549991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1308102380 0x7f13081027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 --2- 172.21.15.64:0/577549991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1308103580 0x7f13081039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 -- 172.21.15.64:0/577549991 >> 172.21.15.64:0/577549991 conn(0x7f13080fd920 msgr2=0x7f13080ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 -- 172.21.15.64:0/577549991 shutdown_connections 2024-05-09T09:00:47.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.031+0000 7f130cd26700 1 -- 172.21.15.64:0/577549991 wait complete. 2024-05-09T09:00:47.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 Processor -- start 2024-05-09T09:00:47.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 -- start start 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1308102380 0x7f13080760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 0x7f1308074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1308103f20 0x7f1308074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f130810de20 con 0x7f1308103580 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f130810dca0 con 0x7f1308103f20 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130cd26700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f130810dfa0 con 0x7f1308102380 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f1305d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 0x7f1308074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f1305d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 0x7f1308074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46184/0 (socket says 172.21.15.64:46184) 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f1305d9b700 1 -- 172.21.15.64:0/3172349429 learned_addr learned my addr 172.21.15.64:0/3172349429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f130659c700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1308102380 0x7f13080760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.032+0000 7f1306d9d700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1308103f20 0x7f1308074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:47.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f1305d9b700 1 -- 172.21.15.64:0/3172349429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1308102380 msgr2=0x7f13080760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f1305d9b700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1308102380 0x7f13080760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f1305d9b700 1 -- 172.21.15.64:0/3172349429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1308103f20 msgr2=0x7f1308074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f1305d9b700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1308103f20 0x7f1308074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f1305d9b700 1 -- 172.21.15.64:0/3172349429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1308075500 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f1305d9b700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 0x7f1308074710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00e9a0 tx=0x7f12fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc00ca10 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12fc00cbb0 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13081a25f0 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc018680 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.033+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13081a2b00 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.034+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f12fc018820 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.034+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1308069010 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.034+0000 7f12f77fe700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12f0039330 0x7f12f003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.034+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f12fc04c4e0 con 0x7f1308103580 2024-05-09T09:00:47.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.035+0000 7f130659c700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12f0039330 0x7f12f003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:00:47.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.035+0000 7f130659c700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12f0039330 0x7f12f003b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f13081033e0 tx=0x7f12f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:00:47.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.039+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12fc00fc70 con 0x7f1308103580 2024-05-09T09:00:47.072 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:46.768+0000 7f31dce84200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:00:47.072 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:46.860+0000 7f31dce84200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:00:47.072 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:47.073+0000 7f31dce84200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:00:47.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:47.257+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1308062760 con 0x7f12f0039330 2024-05-09T09:00:47.287 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:46.877+0000 7f75edfca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:00:47.287 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:46.981+0000 7f75edfca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T09:00:47.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:47.274+0000 7f31dce84200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:00:47.572 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:47.317+0000 7f75edfca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T09:00:47.573 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:47.410+0000 7f75edfca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:47.911+0000 7f31dce84200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:47.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:00:47.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:00:47.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:48.039 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:00:48.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:48.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:48.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:00:48.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:00:48.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:48.643 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:48.344+0000 7f31dce84200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:00:48.644 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:48.442+0000 7f31dce84200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:00:48.644 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:48.644+0000 7f31dce84200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:48.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:49.028 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:49.029 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:49.032 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:48.691+0000 7f75edfca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T09:00:49.032 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:48.861+0000 7f75edfca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T09:00:49.032 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:49.033+0000 7f75edfca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T09:00:49.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T09:00:49.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:49.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:49.187+0000 7f31dce84200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:00:49.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:49.279+0000 7f31dce84200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:00:49.572 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:49.319+0000 7f75edfca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T09:00:49.573 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:49.412+0000 7f75edfca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T09:00:49.862 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:49 smithi179 ceph-mon[34086]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T09:00:49.863 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:49 smithi179 ceph-mon[34086]: Reconfiguring daemon mgr.x on smithi064 2024-05-09T09:00:49.863 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:49.518+0000 7f31dce84200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:00:49.863 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:49.650+0000 7f31dce84200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:00:49.863 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:49.747+0000 7f31dce84200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:00:49.863 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:49.863+0000 7f31dce84200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:00:50.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:49 smithi139 ceph-mon[33599]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T09:00:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:49 smithi139 ceph-mon[33599]: Reconfiguring daemon mgr.x on smithi064 2024-05-09T09:00:50.073 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:49.617+0000 7f75edfca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T09:00:50.073 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:49.809+0000 7f75edfca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T09:00:50.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:49 smithi064 ceph-mon[30149]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T09:00:50.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:49 smithi064 ceph-mon[30149]: Reconfiguring daemon mgr.x on smithi064 2024-05-09T09:00:50.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:50.170+0000 7f31dce84200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:00:50.764 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:50 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:50.427+0000 7f75edfca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:50 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:50.849+0000 7f75edfca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:50 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:50.947+0000 7f75edfca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2749421252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17d19a2-cc4a-48e7-a9ab-39baba537bd3"}]: dispatch 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2749421252' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17d19a2-cc4a-48e7-a9ab-39baba537bd3"}]': finished 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:00:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:51.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:50 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1555780649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2749421252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17d19a2-cc4a-48e7-a9ab-39baba537bd3"}]: dispatch 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2749421252' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17d19a2-cc4a-48e7-a9ab-39baba537bd3"}]': finished 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:50 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1555780649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:50.768+0000 7f31dce84200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:00:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z[35935]: 2024-05-09T09:00:50.986+0000 7f31dce84200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2749421252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17d19a2-cc4a-48e7-a9ab-39baba537bd3"}]: dispatch 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2749421252' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17d19a2-cc4a-48e7-a9ab-39baba537bd3"}]': finished 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:00:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:51.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:50 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1555780649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:00:51.572 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:51.149+0000 7f75edfca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T09:00:51.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:51.773+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f12fc0178f0 con 0x7f1308103580 2024-05-09T09:00:52.016 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:51.680+0000 7f75edfca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T09:00:52.017 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:51.772+0000 7f75edfca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T09:00:52.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:51 smithi139 ceph-mon[33599]: Standby manager daemon z started 2024-05-09T09:00:52.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:51 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T09:00:52.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:51 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:00:52.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:51 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T09:00:52.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:51 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:00:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:51 smithi064 ceph-mon[30149]: Standby manager daemon z started 2024-05-09T09:00:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:51 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T09:00:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:51 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:00:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:51 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T09:00:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:51 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:00:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:51 smithi179 ceph-mon[34086]: Standby manager daemon z started 2024-05-09T09:00:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:51 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T09:00:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:51 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:00:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:51 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T09:00:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:51 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.179:0/1968988623' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:00:52.322 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:52.017+0000 7f75edfca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T09:00:52.323 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:52.151+0000 7f75edfca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T09:00:52.323 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:52.247+0000 7f75edfca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T09:00:52.681 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:52.362+0000 7f75edfca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T09:00:52.681 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:52.679+0000 7f75edfca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T09:00:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:52 smithi139 ceph-mon[33599]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:52 smithi139 ceph-mon[33599]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-09T09:00:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T09:00:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:52 smithi064 ceph-mon[30149]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:52 smithi064 ceph-mon[30149]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-09T09:00:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T09:00:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:52 smithi179 ceph-mon[34086]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:52 smithi179 ceph-mon[34086]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-09T09:00:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T09:00:53.572 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:53.281+0000 7f75edfca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T09:00:53.572 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:00:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y[34975]: 2024-05-09T09:00:53.501+0000 7f75edfca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T09:00:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:00:53.782+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f12fc018c10 con 0x7f1308103580 2024-05-09T09:00:53.979 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:53 smithi064 ceph-mon[30149]: Standby manager daemon y started 2024-05-09T09:00:53.979 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:53 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T09:00:53.979 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:53 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:00:53.980 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:53 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T09:00:53.980 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:53 smithi064 ceph-mon[30149]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:00:54.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:53 smithi139 ceph-mon[33599]: Standby manager daemon y started 2024-05-09T09:00:54.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:53 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T09:00:54.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:53 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:00:54.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:53 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T09:00:54.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:53 smithi139 ceph-mon[33599]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:00:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:53 smithi179 ceph-mon[34086]: Standby manager daemon y started 2024-05-09T09:00:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:53 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T09:00:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:53 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T09:00:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:53 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T09:00:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:53 smithi179 ceph-mon[34086]: from='mgr.? 172.21.15.139:0/2595667203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T09:00:54.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:54 smithi064 ceph-mon[30149]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:54.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:54 smithi064 ceph-mon[30149]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-09T09:00:54.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T09:00:55.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:54 smithi139 ceph-mon[33599]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:54 smithi139 ceph-mon[33599]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-09T09:00:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T09:00:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:54 smithi179 ceph-mon[34086]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:54 smithi179 ceph-mon[34086]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-05-09T09:00:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T09:00:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:56.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:57.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:00:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:00:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:00:58.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:58 smithi064 ceph-mon[30149]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:58.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T09:00:58.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:58.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:58 smithi139 ceph-mon[33599]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:58.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T09:00:58.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:58 smithi179 ceph-mon[34086]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:00:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T09:00:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:00:59.825 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:00:59 smithi064 ceph-mon[30149]: Deploying daemon osd.0 on smithi064 2024-05-09T09:00:59.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:00:59 smithi179 ceph-mon[34086]: Deploying daemon osd.0 on smithi064 2024-05-09T09:01:00.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:00:59 smithi139 ceph-mon[33599]: Deploying daemon osd.0 on smithi064 2024-05-09T09:01:00.886 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:00 smithi064 ceph-mon[30149]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:00 smithi179 ceph-mon[34086]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:00 smithi139 ceph-mon[33599]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:02.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:02 smithi064 ceph-mon[30149]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:02 smithi179 ceph-mon[34086]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:03.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:02 smithi139 ceph-mon[33599]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:04.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:04 smithi064 ceph-mon[30149]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:04.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:04.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:04.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:04 smithi179 ceph-mon[34086]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:04.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:04 smithi139 ceph-mon[33599]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:06.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:06 smithi064 ceph-mon[30149]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:06.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:06 smithi179 ceph-mon[34086]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:06.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:06 smithi139 ceph-mon[33599]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:06.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.836+0000 7f12f77fe700 1 -- 172.21.15.64:0/3172349429 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1308062760 con 0x7f12f0039330 2024-05-09T09:01:06.836 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2024-05-09T09:01:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12f0039330 msgr2=0x7f12f003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12f0039330 0x7f12f003b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f13081033e0 tx=0x7f12f8009260 comp rx=0 tx=0).stop 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 msgr2=0x7f1308074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 0x7f1308074710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00e9a0 tx=0x7f12fc00ecb0 comp rx=0 tx=0).stop 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 shutdown_connections 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1308103f20 0x7f1308074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12f0039330 0x7f12f003b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1308102380 0x7f13080760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 --2- 172.21.15.64:0/3172349429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1308103580 0x7f1308074710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.838+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 >> 172.21.15.64:0/3172349429 conn(0x7f13080fd920 msgr2=0x7f1308106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.839+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 shutdown_connections 2024-05-09T09:01:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:06.839+0000 7f130cd26700 1 -- 172.21.15.64:0/3172349429 wait complete. 2024-05-09T09:01:07.453 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.0.service 2024-05-09T09:01:07.455 INFO:tasks.cephadm:Deploying osd.1 on smithi139 with /dev/vg_nvme/lv_4... 2024-05-09T09:01:07.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T09:01:07.704 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:01:08.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:07 smithi139 ceph-mon[33599]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T09:01:08.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:07 smithi064 ceph-mon[30149]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T09:01:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:07 smithi179 ceph-mon[34086]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T09:01:08.882 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:08 smithi064 ceph-mon[30149]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:08.882 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:08 smithi064 ceph-mon[30149]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T09:01:08.882 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:08 smithi064 ceph-mon[30149]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T09:01:08.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:08 smithi064 ceph-mon[30149]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:01:08.883 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:08 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:08 smithi139 ceph-mon[33599]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:08 smithi139 ceph-mon[33599]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T09:01:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:08 smithi139 ceph-mon[33599]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T09:01:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:08 smithi139 ceph-mon[33599]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:01:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:08 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:09.153 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:01:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0[42705]: 2024-05-09T09:01:08.882+0000 7fa3c74ad700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T09:01:09.153 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:01:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0[42705]: 2024-05-09T09:01:08.885+0000 7fa3c32a0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:01:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:08 smithi179 ceph-mon[34086]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:08 smithi179 ceph-mon[34086]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T09:01:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:08 smithi179 ceph-mon[34086]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T09:01:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:08 smithi179 ceph-mon[34086]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:01:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:08 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T09:01:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T09:01:09.550 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-05-09T09:01:09.551 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-05-09T09:01:09.551 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113857 s, 921 MB/s 2024-05-09T09:01:09.551 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-05-09T09:01:10.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:09 smithi064 ceph-mon[30149]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:01:10.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:09 smithi064 ceph-mon[30149]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T09:01:10.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:10.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:10.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:09 smithi064 ceph-mon[30149]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:10.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:09 smithi139 ceph-mon[33599]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:01:10.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:09 smithi139 ceph-mon[33599]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T09:01:10.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:10.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:10.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:09 smithi139 ceph-mon[33599]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:10.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_4 2024-05-09T09:01:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:09 smithi179 ceph-mon[34086]: from='osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:01:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:09 smithi179 ceph-mon[34086]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T09:01:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:09 smithi179 ceph-mon[34086]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T09:01:10.387 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] boot 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi064 2024-05-09T09:01:11.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] boot 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi064 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-09T09:01:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:11.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:11.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: osd.0 [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] boot 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi064 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-09T09:01:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 -- 172.21.15.139:0/684357166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102e20 msgr2=0x7fcd241096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 --2- 172.21.15.139:0/684357166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102e20 0x7fcd241096d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b0d0 tx=0x7fcd1c00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 -- 172.21.15.139:0/684357166 shutdown_connections 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 --2- 172.21.15.139:0/684357166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102e20 0x7fcd241096d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 --2- 172.21.15.139:0/684357166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd24101280 0x7fcd241016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 --2- 172.21.15.139:0/684357166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd24102480 0x7fcd241028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 -- 172.21.15.139:0/684357166 >> 172.21.15.139:0/684357166 conn(0x7fcd240fc820 msgr2=0x7fcd240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:01:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.749+0000 7fcd29223700 1 -- 172.21.15.139:0/684357166 shutdown_connections 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.750+0000 7fcd29223700 1 -- 172.21.15.139:0/684357166 wait complete. 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.750+0000 7fcd29223700 1 Processor -- start 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 -- start start 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd24101280 0x7fcd2406abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 0x7fcd2419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd24102e20 0x7fcd2419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410cde0 con 0x7fcd24101280 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410cc60 con 0x7fcd24102480 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd29223700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410cf60 con 0x7fcd24102e20 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 0x7fcd2419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 0x7fcd2419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53650/0 (socket says 172.21.15.139:53650) 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 -- 172.21.15.139:0/2863530563 learned_addr learned my addr 172.21.15.139:0/2863530563 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2359e700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd24102e20 0x7fcd2419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd22d9d700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd24101280 0x7fcd2406abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 -- 172.21.15.139:0/2863530563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd24102e20 msgr2=0x7fcd2419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd24102e20 0x7fcd2419f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 -- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd24101280 msgr2=0x7fcd2406abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd24101280 0x7fcd2406abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 -- 172.21.15.139:0/2863530563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241a01d0 con 0x7fcd24102480 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd2259c700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 0x7fcd2419d270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b530 tx=0x7fcd1800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.751+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18015020 con 0x7fcd24102480 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.752+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd18011ce0 con 0x7fcd24102480 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.752+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241a04c0 con 0x7fcd24102480 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.752+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18010dd0 con 0x7fcd24102480 2024-05-09T09:01:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.752+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241a0a80 con 0x7fcd24102480 2024-05-09T09:01:11.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.753+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd24069010 con 0x7fcd24102480 2024-05-09T09:01:11.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.754+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fcd18018020 con 0x7fcd24102480 2024-05-09T09:01:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.754+0000 7fcd0bfff700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd0c0a2b90 0x7fcd0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.754+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fcd180cd1e0 con 0x7fcd24102480 2024-05-09T09:01:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.754+0000 7fcd22d9d700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd0c0a2b90 0x7fcd0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.755+0000 7fcd22d9d700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd0c0a2b90 0x7fcd0c0a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd241022e0 tx=0x7fcd1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:01:11.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.758+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcd1801f020 con 0x7fcd24102480 2024-05-09T09:01:11.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:11.963+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcd24062760 con 0x7fcd0c0a2b90 2024-05-09T09:01:11.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:11 smithi139 ceph-mon[33599]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T09:01:11.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:11 smithi139 ceph-mon[33599]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:11.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:11 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:11.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:11 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:11.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:11 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:12.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:11 smithi064 ceph-mon[30149]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T09:01:12.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:11 smithi064 ceph-mon[30149]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:12.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:11 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:12.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:11 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:12.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:11 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:11 smithi179 ceph-mon[34086]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T09:01:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:11 smithi179 ceph-mon[34086]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:11 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:11 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:11 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:13.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:12 smithi064 ceph-mon[30149]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:01:13.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:01:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:01:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:13.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:12 smithi139 ceph-mon[33599]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:01:13.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:01:13.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:01:13.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:12 smithi179 ceph-mon[34086]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:01:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:01:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:01:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:14.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:13 smithi139 ceph-mon[33599]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:14.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:13 smithi064 ceph-mon[30149]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:14.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:13 smithi179 ceph-mon[34086]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:15.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:14 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/1261487086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]: dispatch 2024-05-09T09:01:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:14 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]: dispatch 2024-05-09T09:01:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:14 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]': finished 2024-05-09T09:01:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:14 smithi064 ceph-mon[30149]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T09:01:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:14 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:14 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/1261487086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]: dispatch 2024-05-09T09:01:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:14 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]: dispatch 2024-05-09T09:01:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:14 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]': finished 2024-05-09T09:01:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:14 smithi179 ceph-mon[34086]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T09:01:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:14 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:14 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/1261487086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]: dispatch 2024-05-09T09:01:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:14 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]: dispatch 2024-05-09T09:01:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:14 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa7e401d-8122-43c1-9c90-c5ec8f78e1c6"}]': finished 2024-05-09T09:01:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:14 smithi139 ceph-mon[33599]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T09:01:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:14 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:15 smithi179 ceph-mon[34086]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:15 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/4107869682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:01:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:15 smithi139 ceph-mon[33599]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:15 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/4107869682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:01:16.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:15 smithi064 ceph-mon[30149]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:16.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:15 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/4107869682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:01:18.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:18 smithi064 ceph-mon[30149]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:18 smithi179 ceph-mon[34086]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:18 smithi139 ceph-mon[33599]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:20.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:20 smithi139 ceph-mon[33599]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:20.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:20 smithi064 ceph-mon[30149]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:20.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:20 smithi179 ceph-mon[34086]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:22.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:22 smithi139 ceph-mon[33599]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:22.353 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T09:01:22.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:22.354 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:22 smithi139 ceph-mon[33599]: Deploying daemon osd.1 on smithi139 2024-05-09T09:01:22.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:22 smithi064 ceph-mon[30149]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:22.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T09:01:22.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:22.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:22 smithi064 ceph-mon[30149]: Deploying daemon osd.1 on smithi139 2024-05-09T09:01:22.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:22 smithi179 ceph-mon[34086]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:22.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T09:01:22.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:22.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:22 smithi179 ceph-mon[34086]: Deploying daemon osd.1 on smithi139 2024-05-09T09:01:24.396 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:24 smithi139 ceph-mon[33599]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:24.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:24 smithi064 ceph-mon[30149]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:24 smithi179 ceph-mon[34086]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:26.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:26 smithi139 ceph-mon[33599]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:26.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:26 smithi064 ceph-mon[30149]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:26.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:26 smithi179 ceph-mon[34086]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:28.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:27 smithi064 ceph-mon[30149]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:28.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:28.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:28.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:28.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:27 smithi139 ceph-mon[33599]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:27 smithi179 ceph-mon[34086]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:30.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.178+0000 7fcd0bfff700 1 -- 172.21.15.139:0/2863530563 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcd24062760 con 0x7fcd0c0a2b90 2024-05-09T09:01:30.178 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 1 on host 'smithi139' 2024-05-09T09:01:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd0c0a2b90 msgr2=0x7fcd0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd0c0a2b90 0x7fcd0c0a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd241022e0 tx=0x7fcd1400b410 comp rx=0 tx=0).stop 2024-05-09T09:01:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 msgr2=0x7fcd2419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 0x7fcd2419d270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b530 tx=0x7fcd1800b840 comp rx=0 tx=0).stop 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 shutdown_connections 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd0c0a2b90 0x7fcd0c0a5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd24102e20 0x7fcd2419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd24101280 0x7fcd2406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 --2- 172.21.15.139:0/2863530563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd24102480 0x7fcd2419d270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 >> 172.21.15.139:0/2863530563 conn(0x7fcd240fc820 msgr2=0x7fcd24105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 shutdown_connections 2024-05-09T09:01:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:01:30.180+0000 7fcd29223700 1 -- 172.21.15.139:0/2863530563 wait complete. 2024-05-09T09:01:30.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:30 smithi064 ceph-mon[30149]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:30.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:30 smithi179 ceph-mon[34086]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:30.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:30 smithi139 ceph-mon[33599]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:30.804 DEBUG:teuthology.orchestra.run.smithi139:osd.1> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.1.service 2024-05-09T09:01:30.806 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2024-05-09T09:01:30.806 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T09:01:31.056 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:01:31.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:31 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:31 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:31 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:31 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:31 smithi179 ceph-mon[34086]: from='osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:01:31.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:31 smithi179 ceph-mon[34086]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:01:31.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:31 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:31 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:31 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:31 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:31 smithi064 ceph-mon[30149]: from='osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:01:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:31 smithi064 ceph-mon[30149]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:01:31.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:31 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:31 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:31 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:31 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:31 smithi139 ceph-mon[33599]: from='osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:01:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:31 smithi139 ceph-mon[33599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T09:01:32.238 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:32 smithi179 ceph-mon[34086]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:32.238 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:32 smithi179 ceph-mon[34086]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T09:01:32.238 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:32 smithi179 ceph-mon[34086]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T09:01:32.239 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:32.239 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:32 smithi179 ceph-mon[34086]: from='osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:01:32.239 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:32 smithi179 ceph-mon[34086]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:01:32.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:32 smithi139 ceph-mon[33599]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:32.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:32 smithi139 ceph-mon[33599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T09:01:32.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:32 smithi139 ceph-mon[33599]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T09:01:32.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:32.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:32 smithi139 ceph-mon[33599]: from='osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:01:32.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:32 smithi139 ceph-mon[33599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:01:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:32 smithi064 ceph-mon[30149]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:32 smithi064 ceph-mon[30149]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T09:01:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:32 smithi064 ceph-mon[30149]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T09:01:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:32 smithi064 ceph-mon[30149]: from='osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:01:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:32 smithi064 ceph-mon[30149]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:01:32.764 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T09:01:32.764 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T09:01:32.764 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-09T09:01:32.764 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-09T09:01:32.764 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122719 s, 854 MB/s 2024-05-09T09:01:32.765 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-09T09:01:33.177 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:01:32 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1[38199]: 2024-05-09T09:01:32.860+0000 7f2cd1c1e700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T09:01:33.177 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:01:32 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1[38199]: 2024-05-09T09:01:32.864+0000 7f2cc9a09700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:01:33.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:33 smithi179 ceph-mon[34086]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:01:33.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:33 smithi179 ceph-mon[34086]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T09:01:33.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:33 smithi179 ceph-mon[34086]: from='osd.1 ' entity='osd.1' 2024-05-09T09:01:33.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.387 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-05-09T09:01:33.514 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:33 smithi139 ceph-mon[33599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:01:33.514 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:33 smithi139 ceph-mon[33599]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T09:01:33.514 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.514 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.514 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:33 smithi139 ceph-mon[33599]: from='osd.1 ' entity='osd.1' 2024-05-09T09:01:33.514 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.629 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:01:33.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:33 smithi064 ceph-mon[30149]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:01:33.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:33 smithi064 ceph-mon[30149]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T09:01:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:33 smithi064 ceph-mon[30149]: from='osd.1 ' entity='osd.1' 2024-05-09T09:01:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi139 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] boot 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T09:01:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi139 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] boot 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:34.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:34.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:34.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:34.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:34.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi139 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: osd.1 [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] boot 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:34.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:34.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:34.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:34.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 -- 172.21.15.179:0/1229386387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101f20 msgr2=0x7ffb541097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/1229386387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101f20 0x7ffb541097e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00b0d0 tx=0x7ffb4c00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 -- 172.21.15.179:0/1229386387 shutdown_connections 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/1229386387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101f20 0x7ffb541097e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/1229386387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb54102840 0x7ffb54102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/1229386387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54101580 0x7ffb541019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 -- 172.21.15.179:0/1229386387 >> 172.21.15.179:0/1229386387 conn(0x7ffb540fbfd0 msgr2=0x7ffb540fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 -- 172.21.15.179:0/1229386387 shutdown_connections 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 -- 172.21.15.179:0/1229386387 wait complete. 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 Processor -- start 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.167+0000 7ffb5aad5700 1 -- start start 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 0x7ffb54100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb54101f20 0x7ffb5419d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102840 0x7ffb5419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5410cda0 con 0x7ffb54102840 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5410cc20 con 0x7ffb54101f20 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5410cf20 con 0x7ffb54101580 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 0x7ffb54100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 0x7ffb54100ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44770/0 (socket says 172.21.15.179:44770) 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 -- 172.21.15.179:0/3784049913 learned_addr learned my addr 172.21.15.179:0/3784049913 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb53fff700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb54101f20 0x7ffb5419d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb59072700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102840 0x7ffb5419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 -- 172.21.15.179:0/3784049913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb54101f20 msgr2=0x7ffb5419d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb54101f20 0x7ffb5419d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 -- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102840 msgr2=0x7ffb5419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102840 0x7ffb5419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 -- 172.21.15.179:0/3784049913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb541a0030 con 0x7ffb54101580 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb59072700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102840 0x7ffb5419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb58871700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 0x7ffb54100ff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c990 tx=0x7ffb4800cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48007b00 con 0x7ffb54101580 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb541a0320 con 0x7ffb54101580 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb48007ca0 con 0x7ffb54101580 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.168+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48018660 con 0x7ffb54101580 2024-05-09T09:01:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.169+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb541a0930 con 0x7ffb54101580 2024-05-09T09:01:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.170+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb54069010 con 0x7ffb54101580 2024-05-09T09:01:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.171+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ffb4800f450 con 0x7ffb54101580 2024-05-09T09:01:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.171+0000 7ffb51ffb700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.171+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7ffb480cd710 con 0x7ffb54101580 2024-05-09T09:01:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.172+0000 7ffb53fff700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.172+0000 7ffb53fff700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb440a2b90 0x7ffb440a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb54101550 tx=0x7ffb40009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:01:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.176+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb48014030 con 0x7ffb54101580 2024-05-09T09:01:35.378 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:35.379+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ffb54062760 con 0x7ffb440a2b90 2024-05-09T09:01:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:35 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi139 to 17088M 2024-05-09T09:01:35.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:35 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi139 to 17088M 2024-05-09T09:01:35.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:35 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi139 to 17088M 2024-05-09T09:01:36.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:36 smithi179 ceph-mon[34086]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:36.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:36 smithi179 ceph-mon[34086]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T09:01:36.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:36 smithi179 ceph-mon[34086]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:01:36.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:01:36.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:01:36.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:36.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:36 smithi139 ceph-mon[33599]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:36 smithi139 ceph-mon[33599]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T09:01:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:36 smithi139 ceph-mon[33599]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:01:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:01:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:01:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:36 smithi064 ceph-mon[30149]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:36 smithi064 ceph-mon[30149]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T09:01:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:36 smithi064 ceph-mon[30149]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:01:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:01:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:01:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:38 smithi179 ceph-mon[34086]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/4077578989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]: dispatch 2024-05-09T09:01:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:38 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]: dispatch 2024-05-09T09:01:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:38 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]': finished 2024-05-09T09:01:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:38 smithi179 ceph-mon[34086]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T09:01:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:38.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:38 smithi139 ceph-mon[33599]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/4077578989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]: dispatch 2024-05-09T09:01:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:38 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]: dispatch 2024-05-09T09:01:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:38 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]': finished 2024-05-09T09:01:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:38 smithi139 ceph-mon[33599]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T09:01:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:38 smithi064 ceph-mon[30149]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/4077578989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]: dispatch 2024-05-09T09:01:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:38 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]: dispatch 2024-05-09T09:01:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:38 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b8bc7a9-9231-4f53-ae83-151cc4898360"}]': finished 2024-05-09T09:01:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:38 smithi064 ceph-mon[30149]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T09:01:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/2634107096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:01:39.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/2634107096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:01:39.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/2634107096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:01:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:40 smithi179 ceph-mon[34086]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:40 smithi139 ceph-mon[33599]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:40 smithi064 ceph-mon[30149]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:42 smithi179 ceph-mon[34086]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:42.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:42 smithi139 ceph-mon[33599]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:42.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:42 smithi064 ceph-mon[30149]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:44 smithi179 ceph-mon[34086]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:44 smithi139 ceph-mon[33599]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:44.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:44 smithi064 ceph-mon[30149]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:45.468 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T09:01:45.468 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:45.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T09:01:45.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:45.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T09:01:45.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:46.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:46 smithi179 ceph-mon[34086]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:46.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:46 smithi179 ceph-mon[34086]: Deploying daemon osd.2 on smithi179 2024-05-09T09:01:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:46 smithi139 ceph-mon[33599]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:46 smithi139 ceph-mon[33599]: Deploying daemon osd.2 on smithi179 2024-05-09T09:01:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:46 smithi064 ceph-mon[30149]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:46.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:46 smithi064 ceph-mon[30149]: Deploying daemon osd.2 on smithi179 2024-05-09T09:01:48.487 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:48 smithi179 ceph-mon[34086]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:48.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:48 smithi139 ceph-mon[33599]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:48.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:48 smithi064 ceph-mon[30149]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:50.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:50 smithi139 ceph-mon[33599]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:50.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:50 smithi064 ceph-mon[30149]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:50 smithi179 ceph-mon[34086]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:51.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:51 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:51.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:51 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:51.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:51 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:51.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:51 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:51.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:51 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:51.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:51 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:51.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:51 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:51.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:51 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:51.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:51 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:52.621 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:52 smithi179 ceph-mon[34086]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:52.825 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:52 smithi139 ceph-mon[33599]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:52.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:52 smithi064 ceph-mon[30149]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:53.846 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.846+0000 7ffb51ffb700 1 -- 172.21.15.179:0/3784049913 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb54062760 con 0x7ffb440a2b90 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb440a2b90 msgr2=0x7ffb440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb440a2b90 0x7ffb440a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb54101550 tx=0x7ffb40009190 comp rx=0 tx=0).stop 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 msgr2=0x7ffb54100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 0x7ffb54100ff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c990 tx=0x7ffb4800cca0 comp rx=0 tx=0).stop 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 shutdown_connections 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102840 0x7ffb5419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb54101580 0x7ffb54100ff0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 --2- 172.21.15.179:0/3784049913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb54101f20 0x7ffb5419d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 >> 172.21.15.179:0/3784049913 conn(0x7ffb540fbfd0 msgr2=0x7ffb54105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 shutdown_connections 2024-05-09T09:01:53.850 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:01:53.849+0000 7ffb5aad5700 1 -- 172.21.15.179:0/3784049913 wait complete. 2024-05-09T09:01:54.391 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.2.service 2024-05-09T09:01:54.393 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/vg_nvme/lv_3... 2024-05-09T09:01:54.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T09:01:54.651 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:01:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: from='osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:01:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:01:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: from='osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: from='osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:55.079 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:56.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:55 smithi064 ceph-mon[30149]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T09:01:56.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:55 smithi064 ceph-mon[30149]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T09:01:56.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:55 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:56.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:55 smithi064 ceph-mon[30149]: from='osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:01:56.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:55 smithi064 ceph-mon[30149]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:01:56.115 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:55 smithi179 ceph-mon[34086]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T09:01:56.115 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:55 smithi179 ceph-mon[34086]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T09:01:56.115 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:55 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:56.115 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:55 smithi179 ceph-mon[34086]: from='osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:01:56.115 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:55 smithi179 ceph-mon[34086]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:01:56.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:55 smithi139 ceph-mon[33599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T09:01:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:55 smithi139 ceph-mon[33599]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T09:01:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:55 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:55 smithi139 ceph-mon[33599]: from='osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:01:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:55 smithi139 ceph-mon[33599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:01:56.729 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:01:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2[39241]: 2024-05-09T09:01:56.527+0000 7f7624e05700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T09:01:56.729 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:01:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2[39241]: 2024-05-09T09:01:56.532+0000 7f761cbf0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:01:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T09:01:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T09:01:56.971 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-09T09:01:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-09T09:01:56.971 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111372 s, 942 MB/s 2024-05-09T09:01:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-09T09:01:57.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:56 smithi064 ceph-mon[30149]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:56 smithi064 ceph-mon[30149]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:01:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:56 smithi064 ceph-mon[30149]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T09:01:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:56 smithi064 ceph-mon[30149]: from='osd.2 ' entity='osd.2' 2024-05-09T09:01:57.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:56 smithi179 ceph-mon[34086]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:57.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:56 smithi179 ceph-mon[34086]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:01:57.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:56 smithi179 ceph-mon[34086]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T09:01:57.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:56 smithi179 ceph-mon[34086]: from='osd.2 ' entity='osd.2' 2024-05-09T09:01:57.195 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:56 smithi139 ceph-mon[33599]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:57.195 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:56 smithi139 ceph-mon[33599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:01:57.195 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:56 smithi139 ceph-mon[33599]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T09:01:57.195 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.195 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.195 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:56 smithi139 ceph-mon[33599]: from='osd.2 ' entity='osd.2' 2024-05-09T09:01:57.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-05-09T09:01:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:57 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:01:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:57 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:01:57.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:57 smithi064 ceph-mon[30149]: osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] boot 2024-05-09T09:01:57.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:57 smithi064 ceph-mon[30149]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T09:01:57.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:57.946 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:01:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:57 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:01:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:57 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:01:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:57 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:57 smithi139 ceph-mon[33599]: osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] boot 2024-05-09T09:01:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:57 smithi139 ceph-mon[33599]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T09:01:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:57 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:57 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:01:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:57 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:01:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:57 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:57 smithi179 ceph-mon[34086]: osd.2 [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] boot 2024-05-09T09:01:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:57 smithi179 ceph-mon[34086]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T09:01:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:57 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi179 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:59.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.056 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:59.056 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:59.056 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:59.056 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi179 2024-05-09T09:01:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi179 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:01:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:01:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 -- 172.21.15.64:0/279355550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102e10 msgr2=0x7facac1096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 --2- 172.21.15.64:0/279355550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102e10 0x7facac1096c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faca400b0d0 tx=0x7faca400b3e0 comp rx=0 tx=0).stop 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 -- 172.21.15.64:0/279355550 shutdown_connections 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 --2- 172.21.15.64:0/279355550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102e10 0x7facac1096c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 --2- 172.21.15.64:0/279355550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac101270 0x7facac101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 --2- 172.21.15.64:0/279355550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102470 0x7facac1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 -- 172.21.15.64:0/279355550 >> 172.21.15.64:0/279355550 conn(0x7facac0fc810 msgr2=0x7facac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 -- 172.21.15.64:0/279355550 shutdown_connections 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.527+0000 7facb4628700 1 -- 172.21.15.64:0/279355550 wait complete. 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 Processor -- start 2024-05-09T09:01:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 -- start start 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac101270 0x7facac1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102470 0x7facac197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 0x7facac19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7facac10cce0 con 0x7facac102e10 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facac10cb60 con 0x7facac102470 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb4628700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facac10ce60 con 0x7facac101270 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb2bc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 0x7facac19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb2bc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 0x7facac19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53618/0 (socket says 172.21.15.64:53618) 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.529+0000 7facb2bc5700 1 -- 172.21.15.64:0/357965185 learned_addr learned my addr 172.21.15.64:0/357965185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:01:59.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb23c4700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac101270 0x7facac1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb1bc3700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102470 0x7facac197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb2bc5700 1 -- 172.21.15.64:0/357965185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac101270 msgr2=0x7facac1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb2bc5700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac101270 0x7facac1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb2bc5700 1 -- 172.21.15.64:0/357965185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102470 msgr2=0x7facac197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb2bc5700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102470 0x7facac197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb2bc5700 1 -- 172.21.15.64:0/357965185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac19ef50 con 0x7facac102e10 2024-05-09T09:01:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.530+0000 7facb2bc5700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 0x7facac19e8b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faca4009fd0 tx=0x7faca4004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:01:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.548+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca401f030 con 0x7facac102e10 2024-05-09T09:01:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.548+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faca4003a00 con 0x7facac102e10 2024-05-09T09:01:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.548+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac19f1e0 con 0x7facac102e10 2024-05-09T09:01:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.548+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca40046a0 con 0x7facac102e10 2024-05-09T09:01:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.549+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac19f690 con 0x7facac102e10 2024-05-09T09:01:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.549+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac069010 con 0x7facac102e10 2024-05-09T09:01:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.551+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7faca4003bb0 con 0x7facac102e10 2024-05-09T09:01:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.551+0000 7faca37fe700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac980a2c10 0x7fac980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:01:59.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.552+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7faca400f020 con 0x7facac102e10 2024-05-09T09:01:59.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.552+0000 7facb23c4700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac980a2c10 0x7fac980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:01:59.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.552+0000 7facb23c4700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac980a2c10 0x7fac980a50d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facac1022d0 tx=0x7faca800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:01:59.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.555+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faca4029020 con 0x7facac102e10 2024-05-09T09:01:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:01:59.772+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7facac062760 con 0x7fac980a2c10 2024-05-09T09:02:00.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:59 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi179 to 17088M 2024-05-09T09:02:00.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:59 smithi064 ceph-mon[30149]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T09:02:00.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:59 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:02:00.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:59 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:00.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:59 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:00.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:01:59 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:59 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi179 to 17088M 2024-05-09T09:02:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:59 smithi179 ceph-mon[34086]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T09:02:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:59 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:02:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:59 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:59 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:00.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:01:59 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:00.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:59 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi179 to 17088M 2024-05-09T09:02:00.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:59 smithi139 ceph-mon[33599]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T09:02:00.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:59 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:02:00.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:59 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:00.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:59 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:00.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:01:59 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:01.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:00 smithi064 ceph-mon[30149]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:01.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:00 smithi064 ceph-mon[30149]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:01.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T09:02:01.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:00 smithi064 ceph-mon[30149]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T09:02:01.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:02:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:00 smithi179 ceph-mon[34086]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:00 smithi179 ceph-mon[34086]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T09:02:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:00 smithi179 ceph-mon[34086]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T09:02:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:02:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:00 smithi139 ceph-mon[33599]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:00 smithi139 ceph-mon[33599]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T09:02:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:00 smithi139 ceph-mon[33599]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T09:02:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:02:01.724 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:01 smithi064 sudo[47541]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:02:01.724 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:01 smithi064 sudo[47541]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:01.724 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:01 smithi064 sudo[47541]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:01.724 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:01 smithi064 sudo[47541]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:02.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:02 smithi139 ceph-mon[33599]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:02.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:02 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:02:02.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:02 smithi139 ceph-mon[33599]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T09:02:02.350 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:02 smithi139 ceph-mon[33599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:02:02.402 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:02 smithi064 sudo[47553]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:02:02.403 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:02 smithi064 sudo[47553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:02.403 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:02 smithi064 sudo[47553]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:02.403 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:02:02 smithi064 sudo[47553]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:02 smithi064 ceph-mon[30149]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:02:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:02 smithi064 ceph-mon[30149]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T09:02:02.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:02 smithi064 ceph-mon[30149]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:02:02.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:02 smithi179 ceph-mon[34086]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:02.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:02 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:02:02.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:02 smithi179 ceph-mon[34086]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T09:02:02.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:02 smithi179 ceph-mon[34086]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:02:02.605 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:02:02.605 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:02.605 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:02.605 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42311]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:03.072 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:02:03.073 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:03.073 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:03.073 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:02:02 smithi139 sudo[42314]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:03.137 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:02 smithi179 sudo[42825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:02:03.137 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:02 smithi179 sudo[42825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:03.137 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:02 smithi179 sudo[42825]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:03.137 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:02 smithi179 sudo[42825]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:03.137 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T09:02:03.137 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:03.137 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3706192136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd6b939-e38b-417b-86af-bf815725ee86"}]: dispatch 2024-05-09T09:02:03.137 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3706192136' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cd6b939-e38b-417b-86af-bf815725ee86"}]': finished 2024-05-09T09:02:03.137 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T09:02:03.138 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:03.138 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:03 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3706192136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd6b939-e38b-417b-86af-bf815725ee86"}]: dispatch 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3706192136' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cd6b939-e38b-417b-86af-bf815725ee86"}]': finished 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:03.479 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:03 smithi179 sudo[42828]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:02:03.479 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:03 smithi179 sudo[42828]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:03.479 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:03 smithi179 sudo[42828]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:03.479 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:02:03 smithi179 sudo[42828]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:03.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T09:02:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3706192136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd6b939-e38b-417b-86af-bf815725ee86"}]: dispatch 2024-05-09T09:02:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3706192136' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cd6b939-e38b-417b-86af-bf815725ee86"}]': finished 2024-05-09T09:02:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T09:02:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:03 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:03.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 sudo[47632]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:02:03.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 sudo[47632]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:03.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 sudo[47632]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:03.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:03 smithi064 sudo[47632]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:04.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:04.083+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faca40995a0 con 0x7facac102e10 2024-05-09T09:02:04.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 sudo[47635]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:02:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 sudo[47635]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 sudo[47635]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 sudo[47635]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:04.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42831]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:02:04.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42831]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:04.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42831]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:04.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42831]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:04.833 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:04.833 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:04.833 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:04.833 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1220623500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 sudo[42317]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 sudo[42317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 sudo[42317]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:04.834 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:04 smithi139 sudo[42317]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1220623500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T09:02:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:04.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:04.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:04.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42834]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42834]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42834]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 sudo[42834]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1220623500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T09:02:04.981 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:04.981 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:04.981 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 sudo[42320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T09:02:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 sudo[42320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T09:02:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 sudo[42320]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T09:02:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 sudo[42320]: pam_unix(sudo:session): session closed for user root 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:05 smithi064 ceph-mon[30149]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:05 smithi179 ceph-mon[34086]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:06.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T09:02:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T09:02:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T09:02:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T09:02:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:05 smithi139 ceph-mon[33599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T09:02:06.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:06 smithi064 ceph-mon[30149]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:06 smithi179 ceph-mon[34086]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:06 smithi139 ceph-mon[33599]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:08.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:08 smithi064 ceph-mon[30149]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:08.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:08 smithi179 ceph-mon[34086]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:08 smithi139 ceph-mon[33599]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:10.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:10 smithi064 ceph-mon[30149]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:10 smithi179 ceph-mon[34086]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:11.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:10 smithi139 ceph-mon[33599]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:11.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:11 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T09:02:11.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:11 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:11.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:11 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T09:02:11.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:11 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:12.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:11 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T09:02:12.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:11 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:12.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:12 smithi064 ceph-mon[30149]: Deploying daemon osd.3 on smithi064 2024-05-09T09:02:12.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:12 smithi064 ceph-mon[30149]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-09T09:02:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:12 smithi179 ceph-mon[34086]: Deploying daemon osd.3 on smithi064 2024-05-09T09:02:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:12 smithi179 ceph-mon[34086]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-09T09:02:13.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:12 smithi139 ceph-mon[33599]: Deploying daemon osd.3 on smithi064 2024-05-09T09:02:13.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:12 smithi139 ceph-mon[33599]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-09T09:02:14.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:14 smithi064 ceph-mon[30149]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-09T09:02:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:14 smithi179 ceph-mon[34086]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-09T09:02:15.082 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:14 smithi139 ceph-mon[33599]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-09T09:02:16.875 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:16 smithi139 ceph-mon[33599]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T09:02:16.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:16 smithi064 ceph-mon[30149]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T09:02:16.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:16 smithi179 ceph-mon[34086]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T09:02:17.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:17 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:17.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:17 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:17.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:17 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:18.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:17 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:18.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:17 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:18.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:17 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:18.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:17 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:18.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:17 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:18.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:17 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:18.893 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:18 smithi064 ceph-mon[30149]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:18 smithi179 ceph-mon[34086]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:19.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:18 smithi139 ceph-mon[33599]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:20.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.132+0000 7faca37fe700 1 -- 172.21.15.64:0/357965185 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7facac062760 con 0x7fac980a2c10 2024-05-09T09:02:20.132 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2024-05-09T09:02:20.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac980a2c10 msgr2=0x7fac980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac980a2c10 0x7fac980a50d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facac1022d0 tx=0x7faca800b410 comp rx=0 tx=0).stop 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 msgr2=0x7facac19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 0x7facac19e8b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faca4009fd0 tx=0x7faca4004d50 comp rx=0 tx=0).stop 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 shutdown_connections 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facac102e10 0x7facac19e8b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac980a2c10 0x7fac980a50d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac101270 0x7facac1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.135+0000 7facb4628700 1 --2- 172.21.15.64:0/357965185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac102470 0x7facac197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.136+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 >> 172.21.15.64:0/357965185 conn(0x7facac0fc810 msgr2=0x7facac105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.136+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 shutdown_connections 2024-05-09T09:02:20.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:02:20.136+0000 7facb4628700 1 -- 172.21.15.64:0/357965185 wait complete. 2024-05-09T09:02:20.799 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:20 smithi064 ceph-mon[30149]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:20.799 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:20.799 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:20.799 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:20 smithi064 ceph-mon[30149]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T09:02:20.853 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.3.service 2024-05-09T09:02:20.856 INFO:tasks.cephadm:Deploying osd.4 on smithi139 with /dev/vg_nvme/lv_3... 2024-05-09T09:02:20.856 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T09:02:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:20 smithi179 ceph-mon[34086]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:20 smithi179 ceph-mon[34086]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T09:02:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:20 smithi139 ceph-mon[33599]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:20 smithi139 ceph-mon[33599]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T09:02:21.102 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:02:22.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T09:02:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T09:02:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:02:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:22.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:22.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:22.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:22.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T09:02:22.208 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T09:02:22.209 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:02:22.209 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:02:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T09:02:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T09:02:23.061 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-05-09T09:02:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-05-09T09:02:23.061 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116284 s, 902 MB/s 2024-05-09T09:02:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-05-09T09:02:23.135 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:02:22 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:02:22.864+0000 7f67f9cc4700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T09:02:23.135 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:02:22 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:02:22.872+0000 7f67f12ae700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:02:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:23 smithi139 ceph-mon[33599]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:02:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:23 smithi139 ceph-mon[33599]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T09:02:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:23 smithi139 ceph-mon[33599]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' 2024-05-09T09:02:23.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:23 smithi064 ceph-mon[30149]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:02:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:23 smithi064 ceph-mon[30149]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T09:02:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:23 smithi064 ceph-mon[30149]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' 2024-05-09T09:02:23.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:23 smithi179 ceph-mon[34086]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:02:23.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:23 smithi179 ceph-mon[34086]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T09:02:23.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:23.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:23 smithi179 ceph-mon[34086]: from='osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430]' entity='osd.3' 2024-05-09T09:02:23.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_3 2024-05-09T09:02:23.875 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] boot 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:24 smithi139 ceph-mon[33599]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T09:02:24.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:02:24.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:02:24.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:24.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] boot 2024-05-09T09:02:24.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T09:02:24.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:24.375 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:24 smithi064 ceph-mon[30149]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: osd.3 [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] boot 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T09:02:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:24 smithi179 ceph-mon[34086]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 -- 172.21.15.139:0/1283470065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec1024a0 msgr2=0x7f3bec102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/1283470065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec1024a0 0x7f3bec102900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40099c0 tx=0x7f3bd4009cd0 comp rx=0 tx=0).stop 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 -- 172.21.15.139:0/1283470065 shutdown_connections 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/1283470065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec102e40 0x7f3bec1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/1283470065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1012a0 0x7f3bec1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/1283470065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec1024a0 0x7f3bec102900 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 -- 172.21.15.139:0/1283470065 >> 172.21.15.139:0/1283470065 conn(0x7f3bec0fc840 msgr2=0x7f3bec0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 -- 172.21.15.139:0/1283470065 shutdown_connections 2024-05-09T09:02:25.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.396+0000 7f3bf2a50700 1 -- 172.21.15.139:0/1283470065 wait complete. 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 Processor -- start 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 -- start start 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 0x7f3bec19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1024a0 0x7f3bec19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec102e40 0x7f3bec1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec10cd30 con 0x7f3bec1012a0 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec10cbb0 con 0x7f3bec102e40 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bf2a50700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec10ceb0 con 0x7f3bec1024a0 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 0x7f3bec19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 0x7f3bec19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.139:45274/0 (socket says 172.21.15.139:45274) 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.397+0000 7f3bebfff700 1 -- 172.21.15.139:0/936199841 learned_addr learned my addr 172.21.15.139:0/936199841 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bf0fed700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec102e40 0x7f3bec1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3be3fff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1024a0 0x7f3bec19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:25.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bebfff700 1 -- 172.21.15.139:0/936199841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1024a0 msgr2=0x7f3bec19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bebfff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1024a0 0x7f3bec19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bebfff700 1 -- 172.21.15.139:0/936199841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec102e40 msgr2=0x7f3bec1a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bebfff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec102e40 0x7f3bec1a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bebfff700 1 -- 172.21.15.139:0/936199841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec1a22a0 con 0x7f3bec1012a0 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bf0fed700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec102e40 0x7f3bec1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3be3fff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1024a0 0x7f3bec19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bebfff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 0x7f3bec19a640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00d9b0 tx=0x7f3bdc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc017070 con 0x7f3bec1012a0 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec1a2590 con 0x7f3bec1012a0 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bdc009de0 con 0x7f3bec1012a0 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00f3c0 con 0x7f3bec1012a0 2024-05-09T09:02:25.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.398+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec1a2a80 con 0x7f3bec1012a0 2024-05-09T09:02:25.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.400+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bec069010 con 0x7f3bec1012a0 2024-05-09T09:02:25.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.401+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3bdc009930 con 0x7f3bec1012a0 2024-05-09T09:02:25.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.401+0000 7f3be9ffb700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bd80a2c20 0x7f3bd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:25.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.402+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f3bdc0cdab0 con 0x7f3bec1012a0 2024-05-09T09:02:25.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.402+0000 7f3be3fff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bd80a2c20 0x7f3bd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:25.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.402+0000 7f3be3fff700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bd80a2c20 0x7f3bd80a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40038f0 tx=0x7f3bd4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:02:25.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.407+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3bdc09bb70 con 0x7f3bec1012a0 2024-05-09T09:02:25.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:25.610+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3bec06b160 con 0x7f3bd80a2c20 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi064 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:25.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:25.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:25.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:25.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:25.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:25 smithi179 ceph-mon[34086]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T09:02:25.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi064 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:25 smithi139 ceph-mon[33599]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi064 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:25.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:25.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:25.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:25.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:25.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:25.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:25 smithi064 ceph-mon[30149]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T09:02:26.563 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:26 smithi139 ceph-mon[33599]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:26.563 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:26 smithi139 ceph-mon[33599]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:26.563 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:26.563 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:26.563 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:26.563 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:26 smithi139 ceph-mon[33599]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T09:02:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:26 smithi179 ceph-mon[34086]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:26 smithi179 ceph-mon[34086]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:26 smithi179 ceph-mon[34086]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T09:02:26.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:26 smithi064 ceph-mon[30149]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:26 smithi064 ceph-mon[30149]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:26 smithi064 ceph-mon[30149]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T09:02:28.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:28 smithi064 ceph-mon[30149]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:28.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:28 smithi064 ceph-mon[30149]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T09:02:28.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:28 smithi179 ceph-mon[34086]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:28.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:28 smithi179 ceph-mon[34086]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T09:02:28.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:28 smithi139 ceph-mon[33599]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:28.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:28 smithi139 ceph-mon[33599]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T09:02:29.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:29 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/2559886554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]: dispatch 2024-05-09T09:02:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:29 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]: dispatch 2024-05-09T09:02:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:29 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]': finished 2024-05-09T09:02:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:29 smithi064 ceph-mon[30149]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T09:02:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:29 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/1083549792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:29 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/2559886554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]: dispatch 2024-05-09T09:02:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:29 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]: dispatch 2024-05-09T09:02:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:29 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]': finished 2024-05-09T09:02:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:29 smithi179 ceph-mon[34086]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T09:02:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:29 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/1083549792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:29.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:29 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/2559886554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]: dispatch 2024-05-09T09:02:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:29 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]: dispatch 2024-05-09T09:02:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:29 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01787529-c26f-4e8e-aca7-3254a587b036"}]': finished 2024-05-09T09:02:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:29 smithi139 ceph-mon[33599]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T09:02:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:29 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/1083549792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:30.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:30 smithi064 ceph-mon[30149]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:30.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:30 smithi139 ceph-mon[33599]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:30 smithi179 ceph-mon[34086]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:32 smithi139 ceph-mon[33599]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-09T09:02:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:32 smithi064 ceph-mon[30149]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-09T09:02:32.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:32 smithi179 ceph-mon[34086]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-09T09:02:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:34 smithi139 ceph-mon[33599]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:02:34.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:34 smithi064 ceph-mon[30149]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:02:34.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:34 smithi179 ceph-mon[34086]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:02:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:36 smithi139 ceph-mon[33599]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:02:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T09:02:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:36 smithi139 ceph-mon[33599]: Deploying daemon osd.4 on smithi139 2024-05-09T09:02:36.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:36 smithi064 ceph-mon[30149]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:02:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T09:02:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:36.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:36 smithi064 ceph-mon[30149]: Deploying daemon osd.4 on smithi139 2024-05-09T09:02:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:36 smithi179 ceph-mon[34086]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:02:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T09:02:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:36 smithi179 ceph-mon[34086]: Deploying daemon osd.4 on smithi139 2024-05-09T09:02:38.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:38 smithi139 ceph-mon[33599]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:02:38.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:38 smithi064 ceph-mon[30149]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:02:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:38 smithi179 ceph-mon[34086]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:02:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:40 smithi064 ceph-mon[30149]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-09T09:02:40.710 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:40 smithi139 ceph-mon[33599]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-09T09:02:40.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:40 smithi179 ceph-mon[34086]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-09T09:02:42.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:42 smithi139 ceph-mon[33599]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:02:42.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:42.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:42.326 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:42.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:42 smithi064 ceph-mon[30149]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:02:42.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:42.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:42.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:42.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:42 smithi179 ceph-mon[34086]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:02:42.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:42.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:42.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:44.387 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:44 smithi139 ceph-mon[33599]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:44.477 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 4 on host 'smithi139' 2024-05-09T09:02:44.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.478+0000 7f3be9ffb700 1 -- 172.21.15.139:0/936199841 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bec06b160 con 0x7f3bd80a2c20 2024-05-09T09:02:44.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bd80a2c20 msgr2=0x7f3bd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:44.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bd80a2c20 0x7f3bd80a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40038f0 tx=0x7f3bd4005fd0 comp rx=0 tx=0).stop 2024-05-09T09:02:44.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 msgr2=0x7f3bec19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 0x7f3bec19a640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00d9b0 tx=0x7f3bdc00dcc0 comp rx=0 tx=0).stop 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 shutdown_connections 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bec102e40 0x7f3bec1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec1012a0 0x7f3bec19a640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bd80a2c20 0x7f3bd80a50e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 --2- 172.21.15.139:0/936199841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bec1024a0 0x7f3bec19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 >> 172.21.15.139:0/936199841 conn(0x7f3bec0fc840 msgr2=0x7f3bec06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 shutdown_connections 2024-05-09T09:02:44.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:02:44.480+0000 7f3bf2a50700 1 -- 172.21.15.139:0/936199841 wait complete. 2024-05-09T09:02:44.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:44 smithi064 ceph-mon[30149]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:44.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:44 smithi179 ceph-mon[34086]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:45.411 DEBUG:teuthology.orchestra.run.smithi139:osd.4> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.4.service 2024-05-09T09:02:45.413 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-05-09T09:02:45.413 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T09:02:45.661 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:02:45.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:45.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:45.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:45 smithi179 ceph-mon[34086]: from='osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:02:45.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:45 smithi179 ceph-mon[34086]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:02:45.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:45.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:45.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:45 smithi139 ceph-mon[33599]: from='osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:02:45.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:45 smithi139 ceph-mon[33599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:02:45.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:45.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:45 smithi064 ceph-mon[30149]: from='osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:02:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:45 smithi064 ceph-mon[30149]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: from='osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:02:46.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: from='osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:46.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:46 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:46.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:46 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:46.823 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:02:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:02:46.442+0000 7f5684ea7700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T09:02:46.823 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:02:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:02:46.451+0000 7f5680c9a700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: from='osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T09:02:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T09:02:47.560 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-09T09:02:47.560 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-09T09:02:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126523 s, 829 MB/s 2024-05-09T09:02:47.561 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 2024-05-09T09:02:47.561 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-09T09:02:47.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:47 smithi179 ceph-mon[34086]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:02:47.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:47 smithi179 ceph-mon[34086]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T09:02:47.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:47.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:47.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:47 smithi139 ceph-mon[33599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:02:47.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:47 smithi139 ceph-mon[33599]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T09:02:47.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:47.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:47.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:47 smithi064 ceph-mon[30149]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:02:47.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:47 smithi064 ceph-mon[30149]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T09:02:47.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:47.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:48.204 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-05-09T09:02:48.447 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] boot 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T09:02:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:48.714 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:02:48.714 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:02:48.714 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:48.714 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:48.715 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] boot 2024-05-09T09:02:48.715 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T09:02:48.715 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: osd.4 [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] boot 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T09:02:48.804 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:49.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:49.602 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:49.602 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:02:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:02:50.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 -- 172.21.15.179:0/853406883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114101280 msgr2=0x7f21141016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:50.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 --2- 172.21.15.179:0/853406883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114101280 0x7f21141016a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2108009990 tx=0x7f2108009ca0 comp rx=0 tx=0).stop 2024-05-09T09:02:50.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 -- 172.21.15.179:0/853406883 shutdown_connections 2024-05-09T09:02:50.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 --2- 172.21.15.179:0/853406883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2114102e20 0x7f21141096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:50.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 --2- 172.21.15.179:0/853406883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114101280 0x7f21141016a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 --2- 172.21.15.179:0/853406883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114102480 0x7f21141028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 -- 172.21.15.179:0/853406883 >> 172.21.15.179:0/853406883 conn(0x7f21140fc820 msgr2=0x7f21140fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 -- 172.21.15.179:0/853406883 shutdown_connections 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.069+0000 7f211adf9700 1 -- 172.21.15.179:0/853406883 wait complete. 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 Processor -- start 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 -- start start 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114101280 0x7f21141051c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 0x7f2114103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2114102e20 0x7f2114103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f211410cd20 con 0x7f2114101280 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f211410cba0 con 0x7f2114102e20 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f211adf9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f211410cea0 con 0x7f2114102480 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 0x7f2114103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:50.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 0x7f2114103860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59678/0 (socket says 172.21.15.179:59678) 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 -- 172.21.15.179:0/2751921657 learned_addr learned my addr 172.21.15.179:0/2751921657 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2118b95700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114101280 0x7f21141051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2119396700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2114102e20 0x7f2114103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 -- 172.21.15.179:0/2751921657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2114102e20 msgr2=0x7f2114103da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2114102e20 0x7f2114103da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 -- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114101280 msgr2=0x7f21141051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114101280 0x7f21141051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 -- 172.21.15.179:0/2751921657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2114104670 con 0x7f2114102480 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.070+0000 7f2113fff700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 0x7f2114103860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f210000e9a0 tx=0x7f210000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.071+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210000ca10 con 0x7f2114102480 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.071+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f210000cbb0 con 0x7f2114102480 2024-05-09T09:02:50.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.071+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100018680 con 0x7f2114102480 2024-05-09T09:02:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.071+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21141a26f0 con 0x7f2114102480 2024-05-09T09:02:50.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.071+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21141a2c50 con 0x7f2114102480 2024-05-09T09:02:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.072+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2114069010 con 0x7f2114102480 2024-05-09T09:02:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.074+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2100010430 con 0x7f2114102480 2024-05-09T09:02:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.074+0000 7f2111ffb700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f21040a2ca0 0x7f21040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:02:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.075+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f21000ce2c0 con 0x7f2114102480 2024-05-09T09:02:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.075+0000 7f2118b95700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f21040a2ca0 0x7f21040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:02:50.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.075+0000 7f2118b95700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f21040a2ca0 0x7f21040a5160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f210800b2e0 tx=0x7f210800b7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:02:50.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.078+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f210009c290 con 0x7f2114102480 2024-05-09T09:02:50.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:02:50.281+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f211406a740 con 0x7f21040a2ca0 2024-05-09T09:02:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:50 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi139 2024-05-09T09:02:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:50 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi139 to 8544M 2024-05-09T09:02:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:50 smithi179 ceph-mon[34086]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:50.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:50 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi139 2024-05-09T09:02:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:50 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi139 to 8544M 2024-05-09T09:02:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:50 smithi139 ceph-mon[33599]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:50.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:50 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi139 2024-05-09T09:02:50.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:50 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi139 to 8544M 2024-05-09T09:02:50.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:50 smithi064 ceph-mon[30149]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:50.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:02:50.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:02:50.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:02:51.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:51 smithi179 ceph-mon[34086]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:51.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:51 smithi139 ceph-mon[33599]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:51.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:51 smithi064 ceph-mon[30149]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:02:52.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:52 smithi179 ceph-mon[34086]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:52.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:52 smithi139 ceph-mon[33599]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:52.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:52 smithi064 ceph-mon[30149]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:53.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:53 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/436673403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]: dispatch 2024-05-09T09:02:53.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:53 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]: dispatch 2024-05-09T09:02:53.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:53 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]': finished 2024-05-09T09:02:53.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:53 smithi179 ceph-mon[34086]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T09:02:53.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:53 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:02:53.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:53 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/436673403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]: dispatch 2024-05-09T09:02:53.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:53 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]: dispatch 2024-05-09T09:02:53.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:53 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]': finished 2024-05-09T09:02:53.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:53 smithi139 ceph-mon[33599]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T09:02:53.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:53 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:02:53.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:53 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/436673403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]: dispatch 2024-05-09T09:02:53.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:53 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]: dispatch 2024-05-09T09:02:53.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:53 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f02944f-d0d9-423c-8747-06be792aa16d"}]': finished 2024-05-09T09:02:53.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:53 smithi064 ceph-mon[30149]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T09:02:53.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:53 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:02:54.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:54 smithi179 ceph-mon[34086]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:54.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:54 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/2964070962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:54.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:54 smithi139 ceph-mon[33599]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:54.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:54 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/2964070962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:54.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:54 smithi064 ceph-mon[30149]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:54.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:54 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/2964070962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:02:56.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:56 smithi179 ceph-mon[34086]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:56.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:56 smithi139 ceph-mon[33599]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:56.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:56 smithi064 ceph-mon[30149]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:58.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:02:58 smithi179 ceph-mon[34086]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:58.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:02:58 smithi139 ceph-mon[33599]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:02:58.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:02:58 smithi064 ceph-mon[30149]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:00.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:00 smithi179 ceph-mon[34086]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:00.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T09:03:00.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:00.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:00 smithi139 ceph-mon[33599]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T09:03:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:00 smithi064 ceph-mon[30149]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T09:03:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:01.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:01 smithi179 ceph-mon[34086]: Deploying daemon osd.5 on smithi179 2024-05-09T09:03:01.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:01 smithi139 ceph-mon[33599]: Deploying daemon osd.5 on smithi179 2024-05-09T09:03:01.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:01 smithi064 ceph-mon[30149]: Deploying daemon osd.5 on smithi179 2024-05-09T09:03:02.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:02 smithi179 ceph-mon[34086]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:02.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:02 smithi139 ceph-mon[33599]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:02.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:02 smithi064 ceph-mon[30149]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:04 smithi179 ceph-mon[34086]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:04.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:04 smithi139 ceph-mon[33599]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:04.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:04 smithi064 ceph-mon[30149]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:06.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:06 smithi064 ceph-mon[30149]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:06 smithi179 ceph-mon[34086]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:06 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:06 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:06 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:06 smithi139 ceph-mon[33599]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:06 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:06 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:06 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:08.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:08 smithi064 ceph-mon[30149]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:08.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:08 smithi179 ceph-mon[34086]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:08.997 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:08.998+0000 7f2111ffb700 1 -- 172.21.15.179:0/2751921657 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f211406a740 con 0x7f21040a2ca0 2024-05-09T09:03:08.998 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-05-09T09:03:09.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f21040a2ca0 msgr2=0x7f21040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:09.000 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f21040a2ca0 0x7f21040a5160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f210800b2e0 tx=0x7f210800b7e0 comp rx=0 tx=0).stop 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 msgr2=0x7f2114103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 0x7f2114103860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f210000e9a0 tx=0x7f210000ecb0 comp rx=0 tx=0).stop 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 shutdown_connections 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2114102e20 0x7f2114103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114101280 0x7f21141051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f21040a2ca0 0x7f21040a5160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 --2- 172.21.15.179:0/2751921657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2114102480 0x7f2114103860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 >> 172.21.15.179:0/2751921657 conn(0x7f21140fc820 msgr2=0x7f211406a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 shutdown_connections 2024-05-09T09:03:09.001 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:03:09.001+0000 7f211adf9700 1 -- 172.21.15.179:0/2751921657 wait complete. 2024-05-09T09:03:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:08 smithi139 ceph-mon[33599]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:09.649 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.5.service 2024-05-09T09:03:09.652 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2024-05-09T09:03:09.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T09:03:09.912 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: from='osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:03:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:09 smithi179 ceph-mon[34086]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: from='osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:03:10.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:09 smithi064 ceph-mon[30149]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:03:10.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:10.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:10.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: from='osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:03:10.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:09 smithi139 ceph-mon[33599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T09:03:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:11 smithi179 ceph-mon[34086]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T09:03:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:11 smithi179 ceph-mon[34086]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T09:03:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:11 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:11 smithi179 ceph-mon[34086]: from='osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:03:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:11 smithi179 ceph-mon[34086]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:03:11.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:11 smithi139 ceph-mon[33599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T09:03:11.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:11 smithi139 ceph-mon[33599]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T09:03:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:11 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:11 smithi139 ceph-mon[33599]: from='osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:03:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:11 smithi139 ceph-mon[33599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:03:11.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:11 smithi064 ceph-mon[30149]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T09:03:11.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:11 smithi064 ceph-mon[30149]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T09:03:11.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:11 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:11.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:11 smithi064 ceph-mon[30149]: from='osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:03:11.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:11 smithi064 ceph-mon[30149]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:03:11.920 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T09:03:11.920 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T09:03:11.920 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-09T09:03:11.920 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-09T09:03:11.921 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114721 s, 914 MB/s 2024-05-09T09:03:11.921 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-09T09:03:12.047 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:03:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:03:11.778+0000 7f2e321f8700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T09:03:12.047 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:03:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:03:11.787+0000 7f2e297e2700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:03:12.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:12 smithi064 ceph-mon[30149]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:03:12.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:12 smithi064 ceph-mon[30149]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T09:03:12.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:12.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:12.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:12 smithi064 ceph-mon[30149]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:12.248 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:12 smithi064 ceph-mon[30149]: from='osd.5 ' entity='osd.5' 2024-05-09T09:03:12.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:12 smithi139 ceph-mon[33599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:03:12.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:12 smithi139 ceph-mon[33599]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T09:03:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:12 smithi139 ceph-mon[33599]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:12 smithi139 ceph-mon[33599]: from='osd.5 ' entity='osd.5' 2024-05-09T09:03:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:12 smithi179 ceph-mon[34086]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:03:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:12 smithi179 ceph-mon[34086]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T09:03:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:12 smithi179 ceph-mon[34086]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-09T09:03:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:12 smithi179 ceph-mon[34086]: from='osd.5 ' entity='osd.5' 2024-05-09T09:03:12.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-05-09T09:03:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:03:13.243 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:03:13.243 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] boot 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi179 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi179 to 8544M 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:13.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:13.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:13.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:13.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] boot 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi179 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi179 to 8544M 2024-05-09T09:03:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:13.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: osd.5 [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] boot 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi179 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi179 to 8544M 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:13.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:14.089 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:14 smithi064 ceph-mon[30149]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:14.089 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:14 smithi064 ceph-mon[30149]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T09:03:14.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:14 smithi139 ceph-mon[33599]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:14.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:14 smithi139 ceph-mon[33599]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T09:03:14.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:14 smithi179 ceph-mon[34086]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:14.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:14 smithi179 ceph-mon[34086]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 -- 172.21.15.64:0/3686399608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac103730 msgr2=0x7f15ac103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 --2- 172.21.15.64:0/3686399608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac103730 0x7f15ac103b90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15940099c0 tx=0x7f1594009cd0 comp rx=0 tx=0).stop 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 -- 172.21.15.64:0/3686399608 shutdown_connections 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 --2- 172.21.15.64:0/3686399608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ac1040d0 0x7f15ac10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 --2- 172.21.15.64:0/3686399608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ac1006b0 0x7f15ac100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 --2- 172.21.15.64:0/3686399608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac103730 0x7f15ac103b90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 -- 172.21.15.64:0/3686399608 >> 172.21.15.64:0/3686399608 conn(0x7f15ac0fbff0 msgr2=0x7f15ac0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:03:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.419+0000 7f15b246e700 1 -- 172.21.15.64:0/3686399608 shutdown_connections 2024-05-09T09:03:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 -- 172.21.15.64:0/3686399608 wait complete. 2024-05-09T09:03:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 Processor -- start 2024-05-09T09:03:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 -- start start 2024-05-09T09:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 0x7f15ac103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ac103730 0x7f15ac1016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ac1040d0 0x7f15ac101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10df00 con 0x7f15ac1006b0 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.420+0000 7f15b246e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10dd80 con 0x7f15ac103730 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15b246e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10e080 con 0x7f15ac1040d0 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 0x7f15ac103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 0x7f15ac103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46030/0 (socket says 172.21.15.64:46030) 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 -- 172.21.15.64:0/2739476648 learned_addr learned my addr 172.21.15.64:0/2739476648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15b0a0b700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ac1040d0 0x7f15ac101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15ab7fe700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ac103730 0x7f15ac1016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 -- 172.21.15.64:0/2739476648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ac1040d0 msgr2=0x7f15ac101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ac1040d0 0x7f15ac101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 -- 172.21.15.64:0/2739476648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ac103730 msgr2=0x7f15ac1016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ac103730 0x7f15ac1016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 -- 172.21.15.64:0/2739476648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac1024d0 con 0x7f15ac1006b0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15abfff700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 0x7f15ac103090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f159c00b500 tx=0x7f159c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c015020 con 0x7f15ac1006b0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac1a2680 con 0x7f15ac1006b0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f159c011ce0 con 0x7f15ac1006b0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c010c80 con 0x7f15ac1006b0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.421+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a2c10 con 0x7f15ac1006b0 2024-05-09T09:03:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.422+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac069010 con 0x7f15ac1006b0 2024-05-09T09:03:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.423+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f159c018020 con 0x7f15ac1006b0 2024-05-09T09:03:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.424+0000 7f15a97fa700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15980a2c20 0x7f15980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.424+0000 7f15ab7fe700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15980a2c20 0x7f15980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.424+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f159c0cd8d0 con 0x7f15ac1006b0 2024-05-09T09:03:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.425+0000 7f15ab7fe700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15980a2c20 0x7f15980a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:03:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.429+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f159c01f020 con 0x7f15ac1006b0 2024-05-09T09:03:14.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:14.645+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f15ac062760 con 0x7f15980a2c20 2024-05-09T09:03:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:15 smithi139 ceph-mon[33599]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T09:03:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:15 smithi139 ceph-mon[33599]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:03:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:03:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:03:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:15 smithi064 ceph-mon[30149]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T09:03:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:15 smithi064 ceph-mon[30149]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:03:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:03:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:03:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:15 smithi179 ceph-mon[34086]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T09:03:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:15 smithi179 ceph-mon[34086]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:03:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:03:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:03:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:16 smithi064 ceph-mon[30149]: pgmap v141: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:16 smithi064 ceph-mon[30149]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T09:03:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:16 smithi064 ceph-mon[30149]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T09:03:16.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:16 smithi139 ceph-mon[33599]: pgmap v141: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:16.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:16 smithi139 ceph-mon[33599]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T09:03:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:16 smithi139 ceph-mon[33599]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T09:03:16.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:16 smithi179 ceph-mon[34086]: pgmap v141: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:16.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:16 smithi179 ceph-mon[34086]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T09:03:16.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:16 smithi179 ceph-mon[34086]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T09:03:17.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:17 smithi064 ceph-mon[30149]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T09:03:17.438 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:17 smithi139 ceph-mon[33599]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T09:03:17.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:17 smithi179 ceph-mon[34086]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T09:03:18.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:18 smithi064 ceph-mon[30149]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:18.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:18 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3957593927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df6bd66-57d2-4efd-927e-75def737bed2"}]: dispatch 2024-05-09T09:03:18.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:18 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3957593927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df6bd66-57d2-4efd-927e-75def737bed2"}]': finished 2024-05-09T09:03:18.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:18 smithi064 ceph-mon[30149]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T09:03:18.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:18 smithi179 ceph-mon[34086]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:18 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3957593927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df6bd66-57d2-4efd-927e-75def737bed2"}]: dispatch 2024-05-09T09:03:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:18 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3957593927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df6bd66-57d2-4efd-927e-75def737bed2"}]': finished 2024-05-09T09:03:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:18 smithi179 ceph-mon[34086]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T09:03:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:18 smithi139 ceph-mon[33599]: pgmap v144: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:18 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3957593927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df6bd66-57d2-4efd-927e-75def737bed2"}]: dispatch 2024-05-09T09:03:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:18 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3957593927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df6bd66-57d2-4efd-927e-75def737bed2"}]': finished 2024-05-09T09:03:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:18 smithi139 ceph-mon[33599]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T09:03:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:19.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:19 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3494522667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:03:19.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:19 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3494522667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:03:19.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:19 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3494522667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:03:20.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:20 smithi064 ceph-mon[30149]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:20.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:20 smithi179 ceph-mon[34086]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:20.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:20 smithi139 ceph-mon[33599]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T09:03:21.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:21 smithi064 ceph-mon[30149]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T09:03:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:21 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:03:21.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:21 smithi179 ceph-mon[34086]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T09:03:21.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:21 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:03:21.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:21 smithi139 ceph-mon[33599]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T09:03:21.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:21 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:03:22.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:22 smithi064 ceph-mon[30149]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:03:22.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:22 smithi179 ceph-mon[34086]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:03:22.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:22 smithi139 ceph-mon[33599]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:03:24.360 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:24 smithi064 ceph-mon[30149]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:03:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:24 smithi179 ceph-mon[34086]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:03:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:24 smithi139 ceph-mon[33599]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:03:26.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:26 smithi064 ceph-mon[30149]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:03:26.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T09:03:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:26.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:26 smithi064 ceph-mon[30149]: Deploying daemon osd.6 on smithi064 2024-05-09T09:03:26.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:26 smithi179 ceph-mon[34086]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:03:26.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T09:03:26.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:26.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:26 smithi179 ceph-mon[34086]: Deploying daemon osd.6 on smithi064 2024-05-09T09:03:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:26 smithi139 ceph-mon[33599]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:03:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T09:03:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:26 smithi139 ceph-mon[33599]: Deploying daemon osd.6 on smithi064 2024-05-09T09:03:28.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:28 smithi064 ceph-mon[30149]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:03:28.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:28 smithi179 ceph-mon[34086]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:03:28.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:28 smithi139 ceph-mon[33599]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:03:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:30 smithi064 ceph-mon[30149]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-09T09:03:30.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:30 smithi179 ceph-mon[34086]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-09T09:03:30.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:30 smithi139 ceph-mon[33599]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-09T09:03:32.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:32 smithi064 ceph-mon[30149]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:03:32.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:32.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:32.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:32 smithi179 ceph-mon[34086]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:03:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:32 smithi139 ceph-mon[33599]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T09:03:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:34.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:34 smithi064 ceph-mon[30149]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:34.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:34 smithi179 ceph-mon[34086]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:34 smithi139 ceph-mon[33599]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:35.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.014+0000 7f15a97fa700 1 -- 172.21.15.64:0/2739476648 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f15ac062760 con 0x7f15980a2c20 2024-05-09T09:03:35.014 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15980a2c20 msgr2=0x7f15980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15980a2c20 0x7f15980a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594005fb0 comp rx=0 tx=0).stop 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 msgr2=0x7f15ac103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 0x7f15ac103090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f159c00b500 tx=0x7f159c00b810 comp rx=0 tx=0).stop 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 shutdown_connections 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15980a2c20 0x7f15980a50e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ac1040d0 0x7f15ac101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac1006b0 0x7f15ac103090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:35.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 --2- 172.21.15.64:0/2739476648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ac103730 0x7f15ac1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:35.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 >> 172.21.15.64:0/2739476648 conn(0x7f15ac0fbff0 msgr2=0x7f15ac106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:03:35.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 shutdown_connections 2024-05-09T09:03:35.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:03:35.016+0000 7f15b246e700 1 -- 172.21.15.64:0/2739476648 wait complete. 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:03:35.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.378 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.378 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:35.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:03:35.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T09:03:35.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:35.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:35.773 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.6.service 2024-05-09T09:03:35.775 INFO:tasks.cephadm:Deploying osd.7 on smithi139 with /dev/vg_nvme/lv_2... 2024-05-09T09:03:35.776 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T09:03:36.036 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:03:36.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:36 smithi064 ceph-mon[30149]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:36.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:36 smithi064 ceph-mon[30149]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T09:03:36.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:36 smithi064 ceph-mon[30149]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T09:03:36.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:36.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:36 smithi064 ceph-mon[30149]: from='osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:03:36.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:36 smithi064 ceph-mon[30149]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:03:36.317 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:03:36 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:03:36.137+0000 7fe896762700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T09:03:36.317 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:03:36 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:03:36.146+0000 7fe892555700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:03:36.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:36 smithi139 ceph-mon[33599]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:36.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:36 smithi139 ceph-mon[33599]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T09:03:36.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:36 smithi139 ceph-mon[33599]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T09:03:36.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:36.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:36 smithi139 ceph-mon[33599]: from='osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:03:36.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:36 smithi139 ceph-mon[33599]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:03:36.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:36 smithi179 ceph-mon[34086]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:36 smithi179 ceph-mon[34086]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T09:03:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:36 smithi179 ceph-mon[34086]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T09:03:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:36 smithi179 ceph-mon[34086]: from='osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:03:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:36 smithi179 ceph-mon[34086]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:03:37.324 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:37 smithi064 ceph-mon[30149]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:03:37.324 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:37 smithi064 ceph-mon[30149]: osdmap e46: 7 total, 6 up, 7 in 2024-05-09T09:03:37.324 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:37.324 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:37.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:37 smithi139 ceph-mon[33599]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:03:37.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:37 smithi139 ceph-mon[33599]: osdmap e46: 7 total, 6 up, 7 in 2024-05-09T09:03:37.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:37.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:37 smithi179 ceph-mon[34086]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:03:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:37 smithi179 ceph-mon[34086]: osdmap e46: 7 total, 6 up, 7 in 2024-05-09T09:03:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:38.307 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T09:03:38.307 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T09:03:38.307 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-05-09T09:03:38.307 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-05-09T09:03:38.307 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112248 s, 934 MB/s 2024-05-09T09:03:38.307 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-05-09T09:03:38.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:38 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:03:38.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:38 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:03:38.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:38 smithi139 ceph-mon[33599]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:38.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:38 smithi139 ceph-mon[33599]: osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] boot 2024-05-09T09:03:38.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:38 smithi139 ceph-mon[33599]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T09:03:38.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:38.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:38 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:03:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:38 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:03:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:38 smithi064 ceph-mon[30149]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:38 smithi064 ceph-mon[30149]: osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] boot 2024-05-09T09:03:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:38 smithi064 ceph-mon[30149]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T09:03:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:38 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:03:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:38 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:03:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:38 smithi179 ceph-mon[34086]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-05-09T09:03:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:38 smithi179 ceph-mon[34086]: osd.6 [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] boot 2024-05-09T09:03:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:38 smithi179 ceph-mon[34086]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T09:03:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T09:03:38.998 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_2 2024-05-09T09:03:39.249 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:03:39.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: osdmap e48: 7 total, 7 up, 7 in 2024-05-09T09:03:39.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: osdmap e48: 7 total, 7 up, 7 in 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: osdmap e48: 7 total, 7 up, 7 in 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:39.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:03:39.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:40.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:40 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi064 2024-05-09T09:03:40.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:40 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-09T09:03:40.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:40 smithi139 ceph-mon[33599]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:40 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi064 2024-05-09T09:03:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:40 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-09T09:03:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:40 smithi064 ceph-mon[30149]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:40 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi064 2024-05-09T09:03:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:40 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-09T09:03:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:40 smithi179 ceph-mon[34086]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:40.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 -- 172.21.15.139:0/2857499229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103f20 msgr2=0x7f581810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:40.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 --2- 172.21.15.139:0/2857499229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103f20 0x7f581810a7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58100099c0 tx=0x7f5810009cd0 comp rx=0 tx=0).stop 2024-05-09T09:03:40.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 -- 172.21.15.139:0/2857499229 shutdown_connections 2024-05-09T09:03:40.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 --2- 172.21.15.139:0/2857499229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103f20 0x7f581810a7d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:40.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 --2- 172.21.15.139:0/2857499229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102380 0x7f58181027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:40.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 --2- 172.21.15.139:0/2857499229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5818103580 0x7f58181039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:40.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 -- 172.21.15.139:0/2857499229 >> 172.21.15.139:0/2857499229 conn(0x7f58180fd920 msgr2=0x7f58180ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:03:40.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 -- 172.21.15.139:0/2857499229 shutdown_connections 2024-05-09T09:03:40.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 -- 172.21.15.139:0/2857499229 wait complete. 2024-05-09T09:03:40.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.793+0000 7f581db4e700 1 Processor -- start 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 -- start start 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102380 0x7f5818198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 0x7f5818198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5818103f20 0x7f581819f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f581810de70 con 0x7f5818102380 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f581810dcf0 con 0x7f5818103580 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f581db4e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f581810dff0 con 0x7f5818103f20 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 0x7f5818198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 0x7f5818198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36752/0 (socket says 172.21.15.139:36752) 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 -- 172.21.15.139:0/2195006447 learned_addr learned my addr 172.21.15.139:0/2195006447 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f58177fe700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102380 0x7f5818198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5817fff700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5818103f20 0x7f581819f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 -- 172.21.15.139:0/2195006447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5818103f20 msgr2=0x7f581819f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5818103f20 0x7f581819f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 -- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102380 msgr2=0x7f5818198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102380 0x7f5818198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 -- 172.21.15.139:0/2195006447 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58181a0080 con 0x7f5818103580 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5816ffd700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 0x7f5818198960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b530 tx=0x7f580c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.794+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c015020 con 0x7f5818103580 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.795+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181a0370 con 0x7f5818103580 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.795+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f580c011ce0 con 0x7f5818103580 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.795+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c010e00 con 0x7f5818103580 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.795+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a0980 con 0x7f5818103580 2024-05-09T09:03:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.796+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818069010 con 0x7f5818103580 2024-05-09T09:03:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.797+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f580c018020 con 0x7f5818103580 2024-05-09T09:03:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.797+0000 7f5814ff9700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f58000a2c20 0x7f58000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:03:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.798+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f580c0cdf40 con 0x7f5818103580 2024-05-09T09:03:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.798+0000 7f58177fe700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f58000a2c20 0x7f58000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:03:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.798+0000 7f58177fe700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f58000a2c20 0x7f58000a50e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58181033e0 tx=0x7f5808007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:03:40.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:40.801+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f580c01f020 con 0x7f5818103580 2024-05-09T09:03:41.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:03:41.003+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5818062760 con 0x7f58000a2c20 2024-05-09T09:03:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:41 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:03:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:41 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:03:41.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:41 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:41 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:03:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:41 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:03:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:41 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:41.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:41 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:03:41.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:41 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:03:41.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:41 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:42.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:42 smithi139 ceph-mon[33599]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:03:42.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:42 smithi139 ceph-mon[33599]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:42.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:42 smithi179 ceph-mon[34086]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:03:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:42 smithi179 ceph-mon[34086]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:42.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:42 smithi064 ceph-mon[30149]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:03:42.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:42 smithi064 ceph-mon[30149]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:44 smithi179 ceph-mon[34086]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:44 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/2088639061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]: dispatch 2024-05-09T09:03:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:44 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]: dispatch 2024-05-09T09:03:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:44 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]': finished 2024-05-09T09:03:44.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:44 smithi179 ceph-mon[34086]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T09:03:44.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:44 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:03:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:44 smithi139 ceph-mon[33599]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:44 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/2088639061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]: dispatch 2024-05-09T09:03:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:44 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]: dispatch 2024-05-09T09:03:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:44 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]': finished 2024-05-09T09:03:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:44 smithi139 ceph-mon[33599]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T09:03:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:44 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:03:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:44 smithi064 ceph-mon[30149]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:44 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/2088639061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]: dispatch 2024-05-09T09:03:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:44 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]: dispatch 2024-05-09T09:03:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:44 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e0802fb-8046-46e9-9a0a-923e400299d1"}]': finished 2024-05-09T09:03:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:44 smithi064 ceph-mon[30149]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T09:03:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:44 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:03:45.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:45 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/92595122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:03:45.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:45 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/92595122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:03:45.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:45 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/92595122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:03:46.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:46 smithi139 ceph-mon[33599]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:46.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:46 smithi179 ceph-mon[34086]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:46 smithi064 ceph-mon[30149]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:48.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:48 smithi139 ceph-mon[33599]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:48.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:48 smithi064 ceph-mon[30149]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:48.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:48 smithi179 ceph-mon[34086]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:50.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:50 smithi139 ceph-mon[33599]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:50.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:50 smithi179 ceph-mon[34086]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:50 smithi064 ceph-mon[30149]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:52 smithi179 ceph-mon[34086]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T09:03:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:52 smithi179 ceph-mon[34086]: Deploying daemon osd.7 on smithi139 2024-05-09T09:03:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:52 smithi139 ceph-mon[33599]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T09:03:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:52 smithi139 ceph-mon[33599]: Deploying daemon osd.7 on smithi139 2024-05-09T09:03:52.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:52 smithi064 ceph-mon[30149]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:52.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T09:03:52.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:03:52.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:52 smithi064 ceph-mon[30149]: Deploying daemon osd.7 on smithi139 2024-05-09T09:03:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:54 smithi179 ceph-mon[34086]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:54.564 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:54 smithi139 ceph-mon[33599]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:54.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:54 smithi064 ceph-mon[30149]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:56.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:56 smithi139 ceph-mon[33599]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:56 smithi179 ceph-mon[34086]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:56.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:56 smithi064 ceph-mon[30149]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:57 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:57 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:57 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:58.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:58.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:58.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:57 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:57 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:57 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:03:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:57 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:03:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:03:58 smithi139 ceph-mon[33599]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:03:58 smithi064 ceph-mon[30149]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:03:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:03:58 smithi179 ceph-mon[34086]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:00.768 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 7 on host 'smithi139' 2024-05-09T09:04:00.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.769+0000 7f5814ff9700 1 -- 172.21.15.139:0/2195006447 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5818062760 con 0x7f58000a2c20 2024-05-09T09:04:00.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f58000a2c20 msgr2=0x7f58000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:00.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f58000a2c20 0x7f58000a50e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58181033e0 tx=0x7f5808007a30 comp rx=0 tx=0).stop 2024-05-09T09:04:00.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 msgr2=0x7f5818198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:00.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 0x7f5818198960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b530 tx=0x7f580c00b840 comp rx=0 tx=0).stop 2024-05-09T09:04:00.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 shutdown_connections 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f58000a2c20 0x7f58000a50e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5818103f20 0x7f581819f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102380 0x7f5818198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 --2- 172.21.15.139:0/2195006447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818103580 0x7f5818198960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 >> 172.21.15.139:0/2195006447 conn(0x7f58180fd920 msgr2=0x7f5818106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 shutdown_connections 2024-05-09T09:04:00.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:00.772+0000 7f581db4e700 1 -- 172.21.15.139:0/2195006447 wait complete. 2024-05-09T09:04:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:04:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:00 smithi139 ceph-mon[33599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:04:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:04:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:00 smithi064 ceph-mon[30149]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:01.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:01.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:04:01.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:00 smithi179 ceph-mon[34086]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T09:04:01.409 DEBUG:teuthology.orchestra.run.smithi139:osd.7> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.7.service 2024-05-09T09:04:01.412 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2024-05-09T09:04:01.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T09:04:01.659 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:04:02.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:02 smithi139 ceph-mon[33599]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:02.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:02 smithi139 ceph-mon[33599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T09:04:02.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:02 smithi139 ceph-mon[33599]: from='osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:04:02.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:02 smithi139 ceph-mon[33599]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T09:04:02.893 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:02 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:02.893 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:02 smithi139 ceph-mon[33599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:04:02.893 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:04:02 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:04:02.752+0000 7f19003c7700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T09:04:02.893 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:04:02 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:04:02.761+0000 7f18fc1ba700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:04:03.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:02 smithi179 ceph-mon[34086]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:03.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:02 smithi179 ceph-mon[34086]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T09:04:03.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:02 smithi179 ceph-mon[34086]: from='osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:04:03.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:02 smithi179 ceph-mon[34086]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T09:04:03.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:02 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:03.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:02 smithi179 ceph-mon[34086]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:04:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:02 smithi064 ceph-mon[30149]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:02 smithi064 ceph-mon[30149]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T09:04:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:02 smithi064 ceph-mon[30149]: from='osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:04:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:02 smithi064 ceph-mon[30149]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T09:04:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:02 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:02 smithi064 ceph-mon[30149]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:04:03.607 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T09:04:03.607 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T09:04:03.607 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-09T09:04:03.607 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-09T09:04:03.607 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124955 s, 839 MB/s 2024-05-09T09:04:03.607 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-09T09:04:04.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:03 smithi139 ceph-mon[33599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:04:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:03 smithi139 ceph-mon[33599]: osdmap e51: 8 total, 7 up, 8 in 2024-05-09T09:04:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:03 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:03 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:03 smithi064 ceph-mon[30149]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:04:04.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:03 smithi064 ceph-mon[30149]: osdmap e51: 8 total, 7 up, 8 in 2024-05-09T09:04:04.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:03 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:03 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:03 smithi179 ceph-mon[34086]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:04:04.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:03 smithi179 ceph-mon[34086]: osdmap e51: 8 total, 7 up, 8 in 2024-05-09T09:04:04.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:03 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:03 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.235 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-05-09T09:04:04.488 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] boot 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:04.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:04.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:04.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:04.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] boot 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:04.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:04.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:04.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:04.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:04.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:04.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: osd.7 [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] boot 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:05.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:05.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:05 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi139 2024-05-09T09:04:05.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:05 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi139 to 5696M 2024-05-09T09:04:05.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:05 smithi179 ceph-mon[34086]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T09:04:06.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:05 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi139 2024-05-09T09:04:06.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:05 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi139 to 5696M 2024-05-09T09:04:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:05 smithi139 ceph-mon[33599]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T09:04:06.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 -- 172.21.15.179:0/4002932980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102e10 msgr2=0x7fe11c1096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 --2- 172.21.15.179:0/4002932980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102e10 0x7fe11c1096c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b0a0 tx=0x7fe11400b3b0 comp rx=0 tx=0).stop 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 -- 172.21.15.179:0/4002932980 shutdown_connections 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 --2- 172.21.15.179:0/4002932980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102e10 0x7fe11c1096c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 --2- 172.21.15.179:0/4002932980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c101270 0x7fe11c101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 --2- 172.21.15.179:0/4002932980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe11c102470 0x7fe11c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 -- 172.21.15.179:0/4002932980 >> 172.21.15.179:0/4002932980 conn(0x7fe11c0fc810 msgr2=0x7fe11c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 -- 172.21.15.179:0/4002932980 shutdown_connections 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.124+0000 7fe12449b700 1 -- 172.21.15.179:0/4002932980 wait complete. 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 Processor -- start 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 -- start start 2024-05-09T09:04:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe11c101270 0x7fe11c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 0x7fe11c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c102e10 0x7fe11c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c10cd80 con 0x7fe11c101270 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c10cc00 con 0x7fe11c102e10 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe12449b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c10cf00 con 0x7fe11c102470 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe121a36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 0x7fe11c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe121a36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 0x7fe11c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49460/0 (socket says 172.21.15.179:49460) 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe121a36700 1 -- 172.21.15.179:0/1169074139 learned_addr learned my addr 172.21.15.179:0/1169074139 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.125+0000 7fe122237700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe11c101270 0x7fe11c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe122a38700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c102e10 0x7fe11c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe121a36700 1 -- 172.21.15.179:0/1169074139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c102e10 msgr2=0x7fe11c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe121a36700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c102e10 0x7fe11c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe121a36700 1 -- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe11c101270 msgr2=0x7fe11c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe121a36700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe11c101270 0x7fe11c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe121a36700 1 -- 172.21.15.179:0/1169074139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c1a00b0 con 0x7fe11c102470 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe122a38700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c102e10 0x7fe11c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe121a36700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 0x7fe11c198990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe11000c9e0 tx=0x7fe11000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110007ac0 con 0x7fe11c102470 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c1a03a0 con 0x7fe11c102470 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe110007c60 con 0x7fe11c102470 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110018660 con 0x7fe11c102470 2024-05-09T09:04:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.126+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c1a0960 con 0x7fe11c102470 2024-05-09T09:04:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.127+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11c069010 con 0x7fe11c102470 2024-05-09T09:04:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.128+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe11000f450 con 0x7fe11c102470 2024-05-09T09:04:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.129+0000 7fe10f7fe700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1080a2c20 0x7fe1080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.129+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7fe1100ce4c0 con 0x7fe11c102470 2024-05-09T09:04:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.129+0000 7fe122237700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1080a2c20 0x7fe1080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.130+0000 7fe122237700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1080a2c20 0x7fe1080a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe11c1022d0 tx=0x7fe118009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:04:06.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.133+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe110014030 con 0x7fe11c102470 2024-05-09T09:04:06.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:05 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi139 2024-05-09T09:04:06.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:05 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi139 to 5696M 2024-05-09T09:04:06.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:05 smithi064 ceph-mon[30149]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T09:04:06.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:06.336+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe11c06b1e0 con 0x7fe1080a2c20 2024-05-09T09:04:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:06 smithi139 ceph-mon[33599]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:06 smithi139 ceph-mon[33599]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T09:04:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:06 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:06 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:06 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:07.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:06 smithi179 ceph-mon[34086]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:07.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:06 smithi179 ceph-mon[34086]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T09:04:07.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:06 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:07.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:06 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:07.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:06 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:06 smithi064 ceph-mon[30149]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:06 smithi064 ceph-mon[30149]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T09:04:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:06 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:08.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:07 smithi139 ceph-mon[33599]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:04:08.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:07 smithi139 ceph-mon[33599]: osdmap e56: 8 total, 8 up, 8 in 2024-05-09T09:04:08.098 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:07 smithi179 ceph-mon[34086]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:04:08.098 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:07 smithi179 ceph-mon[34086]: osdmap e56: 8 total, 8 up, 8 in 2024-05-09T09:04:08.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:07 smithi064 ceph-mon[30149]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:04:08.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:07 smithi064 ceph-mon[30149]: osdmap e56: 8 total, 8 up, 8 in 2024-05-09T09:04:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:08 smithi139 ceph-mon[33599]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:09.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:08 smithi064 ceph-mon[30149]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:08 smithi179 ceph-mon[34086]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:10.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/2971802521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]: dispatch 2024-05-09T09:04:10.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:09 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]: dispatch 2024-05-09T09:04:10.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:09 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]': finished 2024-05-09T09:04:10.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:09 smithi139 ceph-mon[33599]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T09:04:10.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:10.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/3498174370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:04:10.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/2971802521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]: dispatch 2024-05-09T09:04:10.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:09 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]: dispatch 2024-05-09T09:04:10.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:09 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]': finished 2024-05-09T09:04:10.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:09 smithi064 ceph-mon[30149]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T09:04:10.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:10.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/3498174370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:04:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/2971802521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]: dispatch 2024-05-09T09:04:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:09 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]: dispatch 2024-05-09T09:04:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:09 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35bee3f0-b62d-4743-a492-5b91e6d58126"}]': finished 2024-05-09T09:04:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:09 smithi179 ceph-mon[34086]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T09:04:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/3498174370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:04:11.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:10 smithi139 ceph-mon[33599]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:11.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:10 smithi064 ceph-mon[30149]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:11.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:10 smithi179 ceph-mon[34086]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:13.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:12 smithi139 ceph-mon[33599]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:13.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:12 smithi064 ceph-mon[30149]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:13.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:12 smithi179 ceph-mon[34086]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:15.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:14 smithi139 ceph-mon[33599]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:15.111 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:14 smithi179 ceph-mon[34086]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:15.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:14 smithi064 ceph-mon[30149]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:16.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:16 smithi179 ceph-mon[34086]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:16.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:16 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T09:04:16.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:16 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:17.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:16 smithi139 ceph-mon[33599]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:17.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:16 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T09:04:17.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:16 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:17.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:16 smithi064 ceph-mon[30149]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:17.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:16 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T09:04:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:16 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:18.054 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:17 smithi064 ceph-mon[30149]: Deploying daemon osd.8 on smithi179 2024-05-09T09:04:18.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:17 smithi139 ceph-mon[33599]: Deploying daemon osd.8 on smithi179 2024-05-09T09:04:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:17 smithi179 ceph-mon[34086]: Deploying daemon osd.8 on smithi179 2024-05-09T09:04:19.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:18 smithi139 ceph-mon[33599]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:19.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:18 smithi064 ceph-mon[30149]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:19.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:18 smithi179 ceph-mon[34086]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:20.915 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:20 smithi179 ceph-mon[34086]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:20 smithi139 ceph-mon[33599]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:20 smithi064 ceph-mon[30149]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:22.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:22 smithi179 ceph-mon[34086]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:23.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:22 smithi139 ceph-mon[33599]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:23.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:23.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:23.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:23.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:22 smithi064 ceph-mon[30149]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:23.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:23.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:23.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:24 smithi139 ceph-mon[33599]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:24 smithi064 ceph-mon[30149]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:24 smithi179 ceph-mon[34086]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:25.558 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.559+0000 7fe10f7fe700 1 -- 172.21.15.179:0/1169074139 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe11c06b1e0 con 0x7fe1080a2c20 2024-05-09T09:04:25.558 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2024-05-09T09:04:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1080a2c20 msgr2=0x7fe1080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1080a2c20 0x7fe1080a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe11c1022d0 tx=0x7fe118009190 comp rx=0 tx=0).stop 2024-05-09T09:04:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 msgr2=0x7fe11c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:25.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 0x7fe11c198990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe11000c9e0 tx=0x7fe11000ccf0 comp rx=0 tx=0).stop 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 shutdown_connections 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe11c102e10 0x7fe11c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe11c101270 0x7fe11c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1080a2c20 0x7fe1080a50e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 --2- 172.21.15.179:0/1169074139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c102470 0x7fe11c198990 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 >> 172.21.15.179:0/1169074139 conn(0x7fe11c0fc810 msgr2=0x7fe11c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 shutdown_connections 2024-05-09T09:04:25.561 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:04:25.561+0000 7fe12449b700 1 -- 172.21.15.179:0/1169074139 wait complete. 2024-05-09T09:04:26.189 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.8.service 2024-05-09T09:04:26.191 INFO:tasks.cephadm:Deploying osd.9 on smithi064 with /dev/vg_nvme/lv_1... 2024-05-09T09:04:26.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T09:04:26.450 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:04:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:26 smithi139 ceph-mon[33599]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:26 smithi139 ceph-mon[33599]: from='osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:04:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:26 smithi139 ceph-mon[33599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:04:26.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:26 smithi064 ceph-mon[30149]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:26.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:26.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:26.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:26.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:26 smithi064 ceph-mon[30149]: from='osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:04:26.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:26 smithi064 ceph-mon[30149]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:04:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:26 smithi179 ceph-mon[34086]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:26.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:26 smithi179 ceph-mon[34086]: from='osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:04:26.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:26 smithi179 ceph-mon[34086]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T09:04:27.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:27 smithi179 ceph-mon[34086]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T09:04:27.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:27 smithi179 ceph-mon[34086]: from='osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:04:27.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:27 smithi179 ceph-mon[34086]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T09:04:27.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:27.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:27 smithi179 ceph-mon[34086]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:04:27.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:27 smithi139 ceph-mon[33599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T09:04:27.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:27 smithi139 ceph-mon[33599]: from='osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:04:27.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:27 smithi139 ceph-mon[33599]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T09:04:27.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:27.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:27 smithi139 ceph-mon[33599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:04:27.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:27 smithi064 ceph-mon[30149]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T09:04:27.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:27 smithi064 ceph-mon[30149]: from='osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:04:27.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:27 smithi064 ceph-mon[30149]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T09:04:27.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:27.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:27 smithi064 ceph-mon[30149]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:04:28.562 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:04:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:04:28.294+0000 7f3dcd6d9700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T09:04:28.562 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:04:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:04:28.305+0000 7f3dc54c4700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:04:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T09:04:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T09:04:28.751 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-09T09:04:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-09T09:04:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114307 s, 917 MB/s 2024-05-09T09:04:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-09T09:04:28.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:28 smithi139 ceph-mon[33599]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:28 smithi139 ceph-mon[33599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:04:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:28 smithi139 ceph-mon[33599]: osdmap e59: 9 total, 8 up, 9 in 2024-05-09T09:04:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:28 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:28 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:28 smithi139 ceph-mon[33599]: from='osd.8 ' entity='osd.8' 2024-05-09T09:04:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:28 smithi179 ceph-mon[34086]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:28 smithi179 ceph-mon[34086]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:04:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:28 smithi179 ceph-mon[34086]: osdmap e59: 9 total, 8 up, 9 in 2024-05-09T09:04:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:28 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:28 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:28 smithi179 ceph-mon[34086]: from='osd.8 ' entity='osd.8' 2024-05-09T09:04:28.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:28 smithi064 ceph-mon[30149]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:28.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:28 smithi064 ceph-mon[30149]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:04:28.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:28 smithi064 ceph-mon[30149]: osdmap e59: 9 total, 8 up, 9 in 2024-05-09T09:04:28.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:28 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:28.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:28 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:28.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:28 smithi064 ceph-mon[30149]: from='osd.8 ' entity='osd.8' 2024-05-09T09:04:29.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-05-09T09:04:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:04:29.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:04:29.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] boot 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:29.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] boot 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:29.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:29.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: osd.8 [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] boot 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:30.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:30 smithi139 ceph-mon[33599]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:30.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:30 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi179 2024-05-09T09:04:30.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:30 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi179 to 5696M 2024-05-09T09:04:30.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:30 smithi139 ceph-mon[33599]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T09:04:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:30 smithi064 ceph-mon[30149]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:30 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi179 2024-05-09T09:04:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:30 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi179 to 5696M 2024-05-09T09:04:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:30 smithi064 ceph-mon[30149]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T09:04:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:30 smithi179 ceph-mon[34086]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-05-09T09:04:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:30 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi179 2024-05-09T09:04:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:30 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi179 to 5696M 2024-05-09T09:04:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:30 smithi179 ceph-mon[34086]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 -- 172.21.15.64:0/2799246924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 msgr2=0x7f07141027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 --2- 172.21.15.64:0/2799246924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f07141027a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f070400b0d0 tx=0x7f070400b3e0 comp rx=0 tx=0).stop 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 -- 172.21.15.64:0/2799246924 shutdown_connections 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 --2- 172.21.15.64:0/2799246924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0714103f20 0x7f071410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 --2- 172.21.15.64:0/2799246924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f07141027a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 --2- 172.21.15.64:0/2799246924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0714103580 0x7f07141039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.434+0000 7f0718ffe700 1 -- 172.21.15.64:0/2799246924 >> 172.21.15.64:0/2799246924 conn(0x7f07140fd920 msgr2=0x7f07140ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 -- 172.21.15.64:0/2799246924 shutdown_connections 2024-05-09T09:04:31.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 -- 172.21.15.64:0/2799246924 wait complete. 2024-05-09T09:04:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 Processor -- start 2024-05-09T09:04:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 -- start start 2024-05-09T09:04:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f0714198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0714103580 0x7f0714198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0714103f20 0x7f071419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.435+0000 7f0718ffe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f071410de10 con 0x7f0714102380 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f0718ffe700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f071410dc90 con 0x7f0714103f20 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f0718ffe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f071410df90 con 0x7f0714103580 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f0714198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f0714198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54014/0 (socket says 172.21.15.64:54014) 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 -- 172.21.15.64:0/717262083 learned_addr learned my addr 172.21.15.64:0/717262083 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f0711d9b700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0714103580 0x7f0714198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f0712d9d700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0714103f20 0x7f071419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 -- 172.21.15.64:0/717262083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0714103580 msgr2=0x7f0714198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0714103580 0x7f0714198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 -- 172.21.15.64:0/717262083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0714103f20 msgr2=0x7f071419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0714103f20 0x7f071419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 -- 172.21.15.64:0/717262083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07141a00b0 con 0x7f0714102380 2024-05-09T09:04:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f071259c700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f0714198450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0704000f80 tx=0x7f0704004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070401f030 con 0x7f0714102380 2024-05-09T09:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07141a0340 con 0x7f0714102380 2024-05-09T09:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0704007bc0 con 0x7f0714102380 2024-05-09T09:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07040047a0 con 0x7f0714102380 2024-05-09T09:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.436+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07141a07a0 con 0x7f0714102380 2024-05-09T09:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.437+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0714069010 con 0x7f0714102380 2024-05-09T09:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.439+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0704003a00 con 0x7f0714102380 2024-05-09T09:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.439+0000 7f070b7fe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07000a2c20 0x7f07000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.439+0000 7f0711d9b700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07000a2c20 0x7f07000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.440+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f070400f020 con 0x7f0714102380 2024-05-09T09:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.440+0000 7f0711d9b700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07000a2c20 0x7f07000a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06fc005950 tx=0x7f06fc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:04:31.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.444+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0704029020 con 0x7f0714102380 2024-05-09T09:04:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:31.660+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0714062760 con 0x7f07000a2c20 2024-05-09T09:04:32.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:32 smithi064 ceph-mon[30149]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:32.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:32 smithi064 ceph-mon[30149]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T09:04:32.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:32 smithi064 ceph-mon[30149]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:04:32.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:32.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:32.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:32 smithi179 ceph-mon[34086]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:32 smithi179 ceph-mon[34086]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T09:04:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:32 smithi179 ceph-mon[34086]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:04:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:32.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:32 smithi139 ceph-mon[33599]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:32 smithi139 ceph-mon[33599]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T09:04:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:32 smithi139 ceph-mon[33599]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:04:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:33.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:33 smithi064 ceph-mon[30149]: osdmap e64: 9 total, 9 up, 9 in 2024-05-09T09:04:33.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:33 smithi179 ceph-mon[34086]: osdmap e64: 9 total, 9 up, 9 in 2024-05-09T09:04:33.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:33 smithi139 ceph-mon[33599]: osdmap e64: 9 total, 9 up, 9 in 2024-05-09T09:04:34.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:34 smithi064 ceph-mon[30149]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:34.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:34 smithi179 ceph-mon[34086]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:34 smithi139 ceph-mon[33599]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:35.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:35 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2883346128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9f9fbad-4a3a-4dd6-91c1-b856e1b53018"}]: dispatch 2024-05-09T09:04:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:35 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2883346128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9f9fbad-4a3a-4dd6-91c1-b856e1b53018"}]': finished 2024-05-09T09:04:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:35 smithi064 ceph-mon[30149]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T09:04:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:35 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2883346128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9f9fbad-4a3a-4dd6-91c1-b856e1b53018"}]: dispatch 2024-05-09T09:04:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:35 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2883346128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9f9fbad-4a3a-4dd6-91c1-b856e1b53018"}]': finished 2024-05-09T09:04:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:35 smithi179 ceph-mon[34086]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T09:04:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:35.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:35 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2883346128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9f9fbad-4a3a-4dd6-91c1-b856e1b53018"}]: dispatch 2024-05-09T09:04:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:35 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2883346128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9f9fbad-4a3a-4dd6-91c1-b856e1b53018"}]': finished 2024-05-09T09:04:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:35 smithi139 ceph-mon[33599]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T09:04:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:36.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:36 smithi064 ceph-mon[30149]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:04:36.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:36 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/861847161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:04:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:36 smithi179 ceph-mon[34086]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:04:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:36 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/861847161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:04:36.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:36 smithi139 ceph-mon[33599]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-09T09:04:36.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:36 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/861847161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:04:38.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:38 smithi064 ceph-mon[30149]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:04:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:38 smithi179 ceph-mon[34086]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:04:38.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:38 smithi139 ceph-mon[33599]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T09:04:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:40 smithi064 ceph-mon[30149]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:04:40.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:40 smithi179 ceph-mon[34086]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:04:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:40 smithi139 ceph-mon[33599]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:04:42.363 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:42 smithi064 ceph-mon[30149]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:04:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:42 smithi179 ceph-mon[34086]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:04:42.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:42 smithi139 ceph-mon[33599]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-09T09:04:43.308 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:43 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T09:04:43.308 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:43 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:43.308 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:43 smithi064 ceph-mon[30149]: Deploying daemon osd.9 on smithi064 2024-05-09T09:04:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:43 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T09:04:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:43 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:43 smithi179 ceph-mon[34086]: Deploying daemon osd.9 on smithi064 2024-05-09T09:04:43.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:43 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T09:04:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:43 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:43 smithi139 ceph-mon[33599]: Deploying daemon osd.9 on smithi064 2024-05-09T09:04:44.394 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:44 smithi179 ceph-mon[34086]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:04:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:44 smithi064 ceph-mon[30149]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:04:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:44 smithi139 ceph-mon[33599]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:04:46.384 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:46 smithi064 ceph-mon[30149]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:46 smithi179 ceph-mon[34086]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:46 smithi139 ceph-mon[33599]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:48.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:48 smithi064 ceph-mon[30149]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:48.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:48 smithi179 ceph-mon[34086]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:48.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:48 smithi139 ceph-mon[33599]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:49.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:49.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:49.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:49.776 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:49.776 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:49.776 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:49.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:49.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:50.780 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:50 smithi064 ceph-mon[30149]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:50.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:50 smithi139 ceph-mon[33599]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:50.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:50 smithi179 ceph-mon[34086]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:52.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.566+0000 7f070b7fe700 1 -- 172.21.15.64:0/717262083 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0714062760 con 0x7f07000a2c20 2024-05-09T09:04:52.565 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 9 on host 'smithi064' 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07000a2c20 msgr2=0x7f07000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07000a2c20 0x7f07000a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06fc005950 tx=0x7f06fc00a300 comp rx=0 tx=0).stop 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 msgr2=0x7f0714198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f0714198450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0704000f80 tx=0x7f0704004e40 comp rx=0 tx=0).stop 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 shutdown_connections 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0714103f20 0x7f071419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:52.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714102380 0x7f0714198450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07000a2c20 0x7f07000a50e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 --2- 172.21.15.64:0/717262083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0714103580 0x7f0714198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 >> 172.21.15.64:0/717262083 conn(0x7f07140fd920 msgr2=0x7f07141068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:04:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 shutdown_connections 2024-05-09T09:04:52.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:04:52.568+0000 7f0718ffe700 1 -- 172.21.15.64:0/717262083 wait complete. 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:52 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:52 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:53.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:53.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T09:04:53.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:53.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:52 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:04:53.320 DEBUG:teuthology.orchestra.run.smithi064:osd.9> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.9.service 2024-05-09T09:04:53.322 INFO:tasks.cephadm:Deploying osd.10 on smithi139 with /dev/vg_nvme/lv_1... 2024-05-09T09:04:53.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T09:04:53.570 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:04:53.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:53 smithi064 ceph-mon[30149]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T09:04:53.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:53 smithi064 ceph-mon[30149]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T09:04:53.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:53 smithi064 ceph-mon[30149]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:04:53.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:53 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:53.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:53 smithi139 ceph-mon[33599]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T09:04:53.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:53 smithi139 ceph-mon[33599]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T09:04:53.957 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:53 smithi139 ceph-mon[33599]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:04:53.957 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:53 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:53 smithi179 ceph-mon[34086]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T09:04:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:53 smithi179 ceph-mon[34086]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T09:04:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:53 smithi179 ceph-mon[34086]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-09T09:04:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:53 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:54.784 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:04:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:04:54.485+0000 7f40c38c0700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T09:04:54.784 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:04:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:04:54.497+0000 7f40bb6ab700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:04:55.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:54 smithi139 ceph-mon[33599]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:55.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:54 smithi139 ceph-mon[33599]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:04:55.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:54 smithi139 ceph-mon[33599]: osdmap e67: 10 total, 9 up, 10 in 2024-05-09T09:04:55.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:54 smithi139 ceph-mon[33599]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' 2024-05-09T09:04:55.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:54 smithi179 ceph-mon[34086]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:55.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:54 smithi179 ceph-mon[34086]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:04:55.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:54 smithi179 ceph-mon[34086]: osdmap e67: 10 total, 9 up, 10 in 2024-05-09T09:04:55.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:54 smithi179 ceph-mon[34086]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' 2024-05-09T09:04:55.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:54 smithi064 ceph-mon[30149]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:55.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:54 smithi064 ceph-mon[30149]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-09T09:04:55.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:54 smithi064 ceph-mon[30149]: osdmap e67: 10 total, 9 up, 10 in 2024-05-09T09:04:55.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:54 smithi064 ceph-mon[30149]: from='osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163]' entity='osd.9' 2024-05-09T09:04:55.620 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T09:04:55.621 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T09:04:55.621 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-05-09T09:04:55.621 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-05-09T09:04:55.621 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145602 s, 720 MB/s 2024-05-09T09:04:55.621 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-05-09T09:04:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:55 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:04:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:55 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:04:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:55 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:55 smithi179 ceph-mon[34086]: osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] boot 2024-05-09T09:04:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:55 smithi179 ceph-mon[34086]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T09:04:55.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:55 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:56.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:55 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:04:56.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:55 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:04:56.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:55 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:56.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:55 smithi139 ceph-mon[33599]: osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] boot 2024-05-09T09:04:56.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:55 smithi139 ceph-mon[33599]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T09:04:56.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:55 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:56.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:55 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:04:56.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:55 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:04:56.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:55 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:56.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:55 smithi064 ceph-mon[30149]: osd.9 [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] boot 2024-05-09T09:04:56.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:55 smithi064 ceph-mon[30149]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T09:04:56.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:55 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T09:04:56.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_1 2024-05-09T09:04:56.580 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:04:56.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:56.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:56 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:56.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:56.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:56.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:56.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:56 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:04:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:56 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:04:58.028 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:57 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi064 2024-05-09T09:04:58.028 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:57 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-09T09:04:58.028 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:57 smithi139 ceph-mon[33599]: osdmap e69: 10 total, 10 up, 10 in 2024-05-09T09:04:58.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:57 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi064 2024-05-09T09:04:58.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:57 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-09T09:04:58.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:57 smithi064 ceph-mon[30149]: osdmap e69: 10 total, 10 up, 10 in 2024-05-09T09:04:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:57 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi064 2024-05-09T09:04:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:57 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-09T09:04:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:57 smithi179 ceph-mon[34086]: osdmap e69: 10 total, 10 up, 10 in 2024-05-09T09:04:58.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 -- 172.21.15.139:0/3292643406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41022a0 msgr2=0x7fd2a41026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:58.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/3292643406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41022a0 0x7fd2a41026c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00b0d0 tx=0x7fd29c00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:04:58.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 -- 172.21.15.139:0/3292643406 shutdown_connections 2024-05-09T09:04:58.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/3292643406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a4103e40 0x7fd2a410a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:58.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/3292643406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41022a0 0x7fd2a41026c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:58.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/3292643406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a41034a0 0x7fd2a4103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 -- 172.21.15.139:0/3292643406 >> 172.21.15.139:0/3292643406 conn(0x7fd2a40fd860 msgr2=0x7fd2a40ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 -- 172.21.15.139:0/3292643406 shutdown_connections 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.287+0000 7fd2aa39d700 1 -- 172.21.15.139:0/3292643406 wait complete. 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 Processor -- start 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 -- start start 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a41022a0 0x7fd2a40760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 0x7fd2a4074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a4103e40 0x7fd2a4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a410dd30 con 0x7fd2a4103e40 2024-05-09T09:04:58.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a410dbb0 con 0x7fd2a41034a0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2aa39d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a410deb0 con 0x7fd2a41022a0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 0x7fd2a4074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 0x7fd2a4074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45112/0 (socket says 172.21.15.139:45112) 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 -- 172.21.15.139:0/2904091662 learned_addr learned my addr 172.21.15.139:0/2904091662 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a9b9c700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a4103e40 0x7fd2a4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a939b700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a41022a0 0x7fd2a40760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 -- 172.21.15.139:0/2904091662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a41022a0 msgr2=0x7fd2a40760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a41022a0 0x7fd2a40760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 -- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a4103e40 msgr2=0x7fd2a4074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a4103e40 0x7fd2a4074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.288+0000 7fd2a8b9a700 1 -- 172.21.15.139:0/2904091662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a4075500 con 0x7fd2a41034a0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.289+0000 7fd2a8b9a700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 0x7fd2a4074710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd29800d980 tx=0x7fd29800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.289+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298017070 con 0x7fd2a41034a0 2024-05-09T09:04:58.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.289+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a41a2560 con 0x7fd2a41034a0 2024-05-09T09:04:58.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.289+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd298005020 con 0x7fd2a41034a0 2024-05-09T09:04:58.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.289+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29800fa90 con 0x7fd2a41034a0 2024-05-09T09:04:58.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.289+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a41a2a70 con 0x7fd2a41034a0 2024-05-09T09:04:58.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.290+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a4069010 con 0x7fd2a41034a0 2024-05-09T09:04:58.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.291+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd2980051d0 con 0x7fd2a41034a0 2024-05-09T09:04:58.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.292+0000 7fd2a27fc700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd28c0a2c20 0x7fd28c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:04:58.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.292+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd2980ceac0 con 0x7fd2a41034a0 2024-05-09T09:04:58.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.292+0000 7fd2a939b700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd28c0a2c20 0x7fd28c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:04:58.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.292+0000 7fd2a939b700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd28c0a2c20 0x7fd28c0a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009040 tx=0x7fd29c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:04:58.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.295+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd298020020 con 0x7fd2a41034a0 2024-05-09T09:04:58.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:04:58.498+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd2a4062760 con 0x7fd28c0a2c20 2024-05-09T09:04:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:58 smithi139 ceph-mon[33599]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:04:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:59.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:58 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:58 smithi064 ceph-mon[30149]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:04:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:58 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:04:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:58 smithi179 ceph-mon[34086]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:04:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:04:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:04:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:58 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:00.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:04:59 smithi139 ceph-mon[33599]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:05:00.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:04:59 smithi064 ceph-mon[30149]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:05:00.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:04:59 smithi179 ceph-mon[34086]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:05:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:00 smithi139 ceph-mon[33599]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:00 smithi064 ceph-mon[30149]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:01.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:00 smithi179 ceph-mon[34086]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:02.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:01 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/681936433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]: dispatch 2024-05-09T09:05:02.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:01 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]: dispatch 2024-05-09T09:05:02.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:01 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]': finished 2024-05-09T09:05:02.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:01 smithi139 ceph-mon[33599]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T09:05:02.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:01 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:02.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:01 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/681936433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]: dispatch 2024-05-09T09:05:02.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:01 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]: dispatch 2024-05-09T09:05:02.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:01 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]': finished 2024-05-09T09:05:02.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:01 smithi064 ceph-mon[30149]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T09:05:02.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:01 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:02.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:01 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/681936433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]: dispatch 2024-05-09T09:05:02.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:01 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]: dispatch 2024-05-09T09:05:02.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:01 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38196d80-7f0b-4dd2-af5f-5c7341bcd2e8"}]': finished 2024-05-09T09:05:02.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:01 smithi179 ceph-mon[34086]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T09:05:02.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:01 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:03.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:02 smithi064 ceph-mon[30149]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:03.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:02 smithi064 ceph-mon[30149]: from='client.? 172.21.15.139:0/1120931385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:05:03.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:02 smithi179 ceph-mon[34086]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:03.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:02 smithi179 ceph-mon[34086]: from='client.? 172.21.15.139:0/1120931385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:05:03.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:02 smithi139 ceph-mon[33599]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:03.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:02 smithi139 ceph-mon[33599]: from='client.? 172.21.15.139:0/1120931385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:05:05.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:04 smithi064 ceph-mon[30149]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:05.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:04 smithi179 ceph-mon[34086]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:05.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:04 smithi139 ceph-mon[33599]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:06 smithi139 ceph-mon[33599]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:06 smithi064 ceph-mon[30149]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:06 smithi179 ceph-mon[34086]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:09.057 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:08 smithi139 ceph-mon[33599]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:09.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:08 smithi064 ceph-mon[30149]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:09.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:08 smithi179 ceph-mon[34086]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:09.977 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T09:05:09.978 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:10.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T09:05:10.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T09:05:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:11.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:10 smithi139 ceph-mon[33599]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:11.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:10 smithi139 ceph-mon[33599]: Deploying daemon osd.10 on smithi139 2024-05-09T09:05:11.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:10 smithi064 ceph-mon[30149]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:11.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:10 smithi064 ceph-mon[30149]: Deploying daemon osd.10 on smithi139 2024-05-09T09:05:11.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:10 smithi179 ceph-mon[34086]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:10 smithi179 ceph-mon[34086]: Deploying daemon osd.10 on smithi139 2024-05-09T09:05:13.119 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:12 smithi139 ceph-mon[33599]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:13.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:12 smithi064 ceph-mon[30149]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:13.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:12 smithi179 ceph-mon[34086]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:15.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:14 smithi139 ceph-mon[33599]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:15.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:14 smithi064 ceph-mon[30149]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:15.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:14 smithi179 ceph-mon[34086]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:15 smithi179 ceph-mon[34086]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:16.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:16.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:16.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:15 smithi139 ceph-mon[33599]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:16.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:16.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:15 smithi064 ceph-mon[30149]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:18.387 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:18 smithi139 ceph-mon[33599]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:18.388 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.388 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.388 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:18.388 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:18.388 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:18 smithi064 ceph-mon[30149]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:18.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:18.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:18.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:18 smithi179 ceph-mon[34086]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:18.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.818+0000 7fd2a27fc700 1 -- 172.21.15.139:0/2904091662 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd2a4062760 con 0x7fd28c0a2c20 2024-05-09T09:05:18.817 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 10 on host 'smithi139' 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd28c0a2c20 msgr2=0x7fd28c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd28c0a2c20 0x7fd28c0a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009040 tx=0x7fd29c00b040 comp rx=0 tx=0).stop 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 msgr2=0x7fd2a4074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 0x7fd2a4074710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd29800d980 tx=0x7fd29800dc90 comp rx=0 tx=0).stop 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 shutdown_connections 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a4103e40 0x7fd2a4074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd28c0a2c20 0x7fd28c0a50e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a41022a0 0x7fd2a40760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.820+0000 7fd2aa39d700 1 --2- 172.21.15.139:0/2904091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a41034a0 0x7fd2a4074710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.821+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 >> 172.21.15.139:0/2904091662 conn(0x7fd2a40fd860 msgr2=0x7fd2a41067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.821+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 shutdown_connections 2024-05-09T09:05:18.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-05-09T09:05:18.821+0000 7fd2aa39d700 1 -- 172.21.15.139:0/2904091662 wait complete. 2024-05-09T09:05:19.542 DEBUG:teuthology.orchestra.run.smithi139:osd.10> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.10.service 2024-05-09T09:05:19.545 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2024-05-09T09:05:19.545 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T09:05:19.797 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:05:19.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:19 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:19.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:19 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:19.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:19 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:19.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:19 smithi139 ceph-mon[33599]: from='osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:05:19.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:19 smithi139 ceph-mon[33599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:05:19.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:19.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:19.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:19 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:19.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:19 smithi179 ceph-mon[34086]: from='osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:05:19.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:19 smithi179 ceph-mon[34086]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:05:20.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:19 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:19 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:19 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:19 smithi064 ceph-mon[30149]: from='osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:05:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:19 smithi064 ceph-mon[30149]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T09:05:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:20 smithi139 ceph-mon[33599]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:20 smithi139 ceph-mon[33599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T09:05:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:20 smithi139 ceph-mon[33599]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T09:05:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:20 smithi139 ceph-mon[33599]: from='osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:05:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:20 smithi139 ceph-mon[33599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:05:21.073 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:05:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:05:20.779+0000 7fc1e38f2700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T09:05:21.073 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:05:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:05:20.797+0000 7fc1df6e5700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:05:21.146 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:20 smithi179 ceph-mon[34086]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:21.146 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:20 smithi179 ceph-mon[34086]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T09:05:21.146 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:20 smithi179 ceph-mon[34086]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T09:05:21.146 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:20 smithi179 ceph-mon[34086]: from='osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:05:21.146 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:21.146 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:20 smithi179 ceph-mon[34086]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:05:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:20 smithi064 ceph-mon[30149]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:20 smithi064 ceph-mon[30149]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T09:05:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:20 smithi064 ceph-mon[30149]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T09:05:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:20 smithi064 ceph-mon[30149]: from='osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:05:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:20 smithi064 ceph-mon[30149]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-05-09T09:05:21.886 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T09:05:21.886 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T09:05:21.887 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-09T09:05:21.887 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-09T09:05:21.887 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114975 s, 912 MB/s 2024-05-09T09:05:21.887 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 2024-05-09T09:05:21.887 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-09T09:05:22.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:21 smithi139 ceph-mon[33599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:05:22.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:21 smithi139 ceph-mon[33599]: osdmap e72: 11 total, 10 up, 11 in 2024-05-09T09:05:22.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:21 smithi064 ceph-mon[30149]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:05:22.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:21 smithi064 ceph-mon[30149]: osdmap e72: 11 total, 10 up, 11 in 2024-05-09T09:05:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:21 smithi179 ceph-mon[34086]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-05-09T09:05:22.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:21 smithi179 ceph-mon[34086]: osdmap e72: 11 total, 10 up, 11 in 2024-05-09T09:05:22.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.639 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-05-09T09:05:22.892 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:05:22.927 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:05:22.928 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:05:22.928 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:22.928 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:22.928 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] boot 2024-05-09T09:05:22.928 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T09:05:22.928 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:22 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:23.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:05:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:05:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] boot 2024-05-09T09:05:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T09:05:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:22 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:05:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:05:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-05-09T09:05:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: osd.10 [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] boot 2024-05-09T09:05:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T09:05:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:22 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T09:05:23.893 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T09:05:23.893 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi139 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi139 to 4272M 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:23.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi139 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi139 to 4272M 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:24.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:24.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi139 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi139 to 4272M 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 -- 172.21.15.179:0/2478041903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1057a0 msgr2=0x7f693c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 --2- 172.21.15.179:0/2478041903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1057a0 0x7f693c107b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6930009990 tx=0x7f6930009ca0 comp rx=0 tx=0).stop 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 -- 172.21.15.179:0/2478041903 shutdown_connections 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 --2- 172.21.15.179:0/2478041903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c1080d0 0x7f693c10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 --2- 172.21.15.179:0/2478041903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c102e70 0x7f693c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 --2- 172.21.15.179:0/2478041903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1057a0 0x7f693c107b90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:24.463 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 -- 172.21.15.179:0/2478041903 >> 172.21.15.179:0/2478041903 conn(0x7f693c0fc820 msgr2=0x7f693c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:24.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 -- 172.21.15.179:0/2478041903 shutdown_connections 2024-05-09T09:05:24.464 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 -- 172.21.15.179:0/2478041903 wait complete. 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.464+0000 7f6940a36700 1 Processor -- start 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 -- start start 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c102e70 0x7f693c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c1057a0 0x7f693c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 0x7f693c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10de80 con 0x7f693c1057a0 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10dd00 con 0x7f693c102e70 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6940a36700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10e000 con 0x7f693c1080d0 2024-05-09T09:05:24.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 0x7f693c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693a59c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c102e70 0x7f693c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 0x7f693c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53314/0 (socket says 172.21.15.179:53314) 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 -- 172.21.15.179:0/2185918523 learned_addr learned my addr 172.21.15.179:0/2185918523 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693a59c700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c102e70 0x7f693c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.179:49134/0 (socket says 172.21.15.179:49134) 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f6939d9b700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c1057a0 0x7f693c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 -- 172.21.15.179:0/2185918523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c102e70 msgr2=0x7f693c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c102e70 0x7f693c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 -- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c1057a0 msgr2=0x7f693c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c1057a0 0x7f693c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 -- 172.21.15.179:0/2185918523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1a0160 con 0x7f693c1080d0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f693ad9d700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 0x7f693c19fac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f693400ba10 tx=0x7f693400bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.465+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693400d8e0 con 0x7f693c1080d0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.466+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6934015420 con 0x7f693c1080d0 2024-05-09T09:05:24.467 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.466+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1a0450 con 0x7f693c1080d0 2024-05-09T09:05:24.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.466+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934014630 con 0x7f693c1080d0 2024-05-09T09:05:24.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.466+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a0990 con 0x7f693c1080d0 2024-05-09T09:05:24.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.466+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c069010 con 0x7f693c1080d0 2024-05-09T09:05:24.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.468+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6934015980 con 0x7f693c1080d0 2024-05-09T09:05:24.468 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.468+0000 7f692b7fe700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69240a2ca0 0x7f69240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:24.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.468+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f69340ce460 con 0x7f693c1080d0 2024-05-09T09:05:24.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.469+0000 7f693a59c700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69240a2ca0 0x7f69240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:24.469 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.469+0000 7f693a59c700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69240a2ca0 0x7f69240a5160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f692c005950 tx=0x7f692c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:24.470 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.472+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f693409ba10 con 0x7f693c1080d0 2024-05-09T09:05:24.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:24.676+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f693c062760 con 0x7f69240a2ca0 2024-05-09T09:05:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:24 smithi179 ceph-mon[34086]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:24 smithi179 ceph-mon[34086]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T09:05:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:24 smithi179 ceph-mon[34086]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T09:05:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:05:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:05:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:24 smithi139 ceph-mon[33599]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:25.136 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:24 smithi139 ceph-mon[33599]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T09:05:25.136 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:24 smithi139 ceph-mon[33599]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T09:05:25.136 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:05:25.136 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:05:25.136 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:24 smithi064 ceph-mon[30149]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:25.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:24 smithi064 ceph-mon[30149]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T09:05:25.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:24 smithi064 ceph-mon[30149]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T09:05:25.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T09:05:25.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T09:05:25.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:25 smithi179 ceph-mon[34086]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:05:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:25 smithi179 ceph-mon[34086]: osdmap e77: 11 total, 11 up, 11 in 2024-05-09T09:05:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:25 smithi139 ceph-mon[33599]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:05:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:25 smithi139 ceph-mon[33599]: osdmap e77: 11 total, 11 up, 11 in 2024-05-09T09:05:26.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:25 smithi064 ceph-mon[30149]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:05:26.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:25 smithi064 ceph-mon[30149]: osdmap e77: 11 total, 11 up, 11 in 2024-05-09T09:05:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:26 smithi139 ceph-mon[33599]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:26 smithi064 ceph-mon[30149]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:26 smithi179 ceph-mon[34086]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:27.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:27 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/4048885554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]: dispatch 2024-05-09T09:05:27.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:27 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]: dispatch 2024-05-09T09:05:27.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:27 smithi179 ceph-mon[34086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]': finished 2024-05-09T09:05:27.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:27 smithi179 ceph-mon[34086]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T09:05:27.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:28.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:27 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/4048885554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]: dispatch 2024-05-09T09:05:28.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:27 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]: dispatch 2024-05-09T09:05:28.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:27 smithi139 ceph-mon[33599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]': finished 2024-05-09T09:05:28.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:27 smithi139 ceph-mon[33599]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T09:05:28.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:27 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/4048885554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]: dispatch 2024-05-09T09:05:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:27 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]: dispatch 2024-05-09T09:05:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:27 smithi064 ceph-mon[30149]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998"}]': finished 2024-05-09T09:05:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:27 smithi064 ceph-mon[30149]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T09:05:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:29.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:28 smithi139 ceph-mon[33599]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:29.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:28 smithi139 ceph-mon[33599]: from='client.? 172.21.15.179:0/2213264566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:05:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:28 smithi064 ceph-mon[30149]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:28 smithi064 ceph-mon[30149]: from='client.? 172.21.15.179:0/2213264566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:05:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:28 smithi179 ceph-mon[34086]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:28 smithi179 ceph-mon[34086]: from='client.? 172.21.15.179:0/2213264566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T09:05:31.147 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:30 smithi179 ceph-mon[34086]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-09T09:05:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:30 smithi064 ceph-mon[30149]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-09T09:05:31.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:30 smithi139 ceph-mon[33599]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-09T09:05:32.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:32 smithi179 ceph-mon[34086]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:05:33.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:32 smithi064 ceph-mon[30149]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:05:33.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:32 smithi139 ceph-mon[33599]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T09:05:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:34 smithi064 ceph-mon[30149]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:05:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T09:05:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:35.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:34 smithi179 ceph-mon[34086]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:05:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T09:05:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:35.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:34 smithi139 ceph-mon[33599]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T09:05:35.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T09:05:35.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:35.972 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:35 smithi179 ceph-mon[34086]: Deploying daemon osd.11 on smithi179 2024-05-09T09:05:36.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:35 smithi064 ceph-mon[30149]: Deploying daemon osd.11 on smithi179 2024-05-09T09:05:36.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:35 smithi139 ceph-mon[33599]: Deploying daemon osd.11 on smithi179 2024-05-09T09:05:36.961 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:36 smithi179 ceph-mon[34086]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:05:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:36 smithi064 ceph-mon[30149]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:05:37.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:36 smithi139 ceph-mon[33599]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:05:39.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:38 smithi064 ceph-mon[30149]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:05:39.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:38 smithi179 ceph-mon[34086]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:05:39.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:38 smithi139 ceph-mon[33599]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T09:05:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:40 smithi064 ceph-mon[30149]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-09T09:05:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:41.223 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:40 smithi179 ceph-mon[34086]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-09T09:05:41.223 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:41.223 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:41.223 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:40 smithi139 ceph-mon[33599]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-09T09:05:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:42.872 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:42 smithi179 ceph-mon[34086]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:43.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:42 smithi064 ceph-mon[30149]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:43.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:42 smithi139 ceph-mon[33599]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:43.950 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2024-05-09T09:05:43.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.950+0000 7f692b7fe700 1 -- 172.21.15.179:0/2185918523 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f693c062760 con 0x7f69240a2ca0 2024-05-09T09:05:43.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69240a2ca0 msgr2=0x7f69240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:43.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69240a2ca0 0x7f69240a5160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f692c005950 tx=0x7f692c00b410 comp rx=0 tx=0).stop 2024-05-09T09:05:43.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 msgr2=0x7f693c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:43.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 0x7f693c19fac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f693400ba10 tx=0x7f693400bd20 comp rx=0 tx=0).stop 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 shutdown_connections 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69240a2ca0 0x7f69240a5160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f693c1080d0 0x7f693c19fac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f693c102e70 0x7f693c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 --2- 172.21.15.179:0/2185918523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f693c1057a0 0x7f693c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 >> 172.21.15.179:0/2185918523 conn(0x7f693c0fc820 msgr2=0x7f693c1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 shutdown_connections 2024-05-09T09:05:43.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:43.952+0000 7f6940a36700 1 -- 172.21.15.179:0/2185918523 wait complete. 2024-05-09T09:05:44.561 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.11.service 2024-05-09T09:05:44.563 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T09:05:44.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-09T09:05:44.824 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:05:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:44 smithi064 ceph-mon[30149]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:44 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:44 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:44 smithi064 ceph-mon[30149]: from='osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:05:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:44 smithi064 ceph-mon[30149]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:05:45.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:44 smithi179 ceph-mon[34086]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:44 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:44 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:44 smithi179 ceph-mon[34086]: from='osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:05:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:44 smithi179 ceph-mon[34086]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:05:45.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:44 smithi139 ceph-mon[33599]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:45.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:44 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:44 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:44 smithi139 ceph-mon[33599]: from='osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:05:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:44 smithi139 ceph-mon[33599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: from='osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:46.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: from='osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:46.198 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: from='osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:46.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 -- 172.21.15.64:0/2479170482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978102480 msgr2=0x7f39781028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 --2- 172.21.15.64:0/2479170482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978102480 0x7f39781028e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f396c0099c0 tx=0x7f396c009cd0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 -- 172.21.15.64:0/2479170482 shutdown_connections 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 --2- 172.21.15.64:0/2479170482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3978102e20 0x7f39781096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 --2- 172.21.15.64:0/2479170482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3978101280 0x7f39781016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 --2- 172.21.15.64:0/2479170482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978102480 0x7f39781028e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 -- 172.21.15.64:0/2479170482 >> 172.21.15.64:0/2479170482 conn(0x7f39780fc820 msgr2=0x7f39780fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 -- 172.21.15.64:0/2479170482 shutdown_connections 2024-05-09T09:05:46.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.539+0000 7f397cd2f700 1 -- 172.21.15.64:0/2479170482 wait complete. 2024-05-09T09:05:46.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 Processor -- start 2024-05-09T09:05:46.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 -- start start 2024-05-09T09:05:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 0x7f397806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3978102480 0x7f397819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3978102e20 0x7f397819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f397810ccf0 con 0x7f3978101280 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f397810cb70 con 0x7f3978102480 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397cd2f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f397810ce70 con 0x7f3978102e20 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 0x7f397806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 0x7f397806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34440/0 (socket says 172.21.15.64:34440) 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f397659c700 1 -- 172.21.15.64:0/3783615689 learned_addr learned my addr 172.21.15.64:0/3783615689 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f3976d9d700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3978102e20 0x7f397819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.540+0000 7f3975d9b700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3978102480 0x7f397819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397659c700 1 -- 172.21.15.64:0/3783615689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3978102e20 msgr2=0x7f397819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397659c700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3978102e20 0x7f397819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397659c700 1 -- 172.21.15.64:0/3783615689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3978102480 msgr2=0x7f397819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397659c700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3978102480 0x7f397819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397659c700 1 -- 172.21.15.64:0/3783615689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781a00d0 con 0x7f3978101280 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397659c700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 0x7f397806ab60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f396800d980 tx=0x7f396800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968017030 con 0x7f3978101280 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781a03c0 con 0x7f3978101280 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39680044d0 con 0x7f3978101280 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680039e0 con 0x7f3978101280 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.541+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a0950 con 0x7f3978101280 2024-05-09T09:05:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.542+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3978069010 con 0x7f3978101280 2024-05-09T09:05:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.544+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f396801a070 con 0x7f3978101280 2024-05-09T09:05:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.545+0000 7f39677fe700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39600a2c20 0x7f39600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.545+0000 7f3975d9b700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39600a2c20 0x7f39600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.545+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f39680ce730 con 0x7f3978101280 2024-05-09T09:05:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.545+0000 7f3975d9b700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39600a2c20 0x7f39600a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f397806b0e0 tx=0x7f396c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:46.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.548+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f396809bc50 con 0x7f3978101280 2024-05-09T09:05:46.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.755+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f397802db80 con 0x7f3978101280 2024-05-09T09:05:46.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.756+0000 7f39677fe700 1 -- 172.21.15.64:0/3783615689 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f396800fe80 con 0x7f3978101280 2024-05-09T09:05:46.755 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:05:46.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39600a2c20 msgr2=0x7f39600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:46.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39600a2c20 0x7f39600a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f397806b0e0 tx=0x7f396c005fd0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 msgr2=0x7f397806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 0x7f397806ab60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f396800d980 tx=0x7f396800dc90 comp rx=0 tx=0).stop 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 shutdown_connections 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39600a2c20 0x7f39600a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3978102e20 0x7f397819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3978101280 0x7f397806ab60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 --2- 172.21.15.64:0/3783615689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3978102480 0x7f397819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 >> 172.21.15.64:0/3783615689 conn(0x7f39780fc820 msgr2=0x7f39781057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:46.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 shutdown_connections 2024-05-09T09:05:46.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:46.758+0000 7f397cd2f700 1 -- 172.21.15.64:0/3783615689 wait complete. 2024-05-09T09:05:46.912 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:05:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:05:46.637+0000 7fef5e9cd700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T09:05:46.912 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:05:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:05:46.650+0000 7fef567b8700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T09:05:47.221 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:46 smithi179 ceph-mon[34086]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:05:47.221 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:46 smithi179 ceph-mon[34086]: osdmap e80: 12 total, 11 up, 12 in 2024-05-09T09:05:47.221 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:46 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:47.221 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:46 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:47.221 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:46 smithi179 ceph-mon[34086]: from='osd.11 ' entity='osd.11' 2024-05-09T09:05:47.221 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:46 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3783615689' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:05:47.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:46 smithi064 ceph-mon[30149]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:05:47.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:46 smithi064 ceph-mon[30149]: osdmap e80: 12 total, 11 up, 12 in 2024-05-09T09:05:47.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:47.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:47.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:46 smithi064 ceph-mon[30149]: from='osd.11 ' entity='osd.11' 2024-05-09T09:05:47.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:46 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3783615689' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:05:47.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:46 smithi139 ceph-mon[33599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-09T09:05:47.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:46 smithi139 ceph-mon[33599]: osdmap e80: 12 total, 11 up, 12 in 2024-05-09T09:05:47.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:46 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:47.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:46 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:47.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:46 smithi139 ceph-mon[33599]: from='osd.11 ' entity='osd.11' 2024-05-09T09:05:47.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:46 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3783615689' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:05:47.510 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1715245521,"num_in_osds":12,"osd_in_since":1715245527,"num_remapped_pgs":0} 2024-05-09T09:05:48.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: purged_snaps scrub starts 2024-05-09T09:05:48.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: purged_snaps scrub ok 2024-05-09T09:05:48.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:48.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:48.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] boot 2024-05-09T09:05:48.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T09:05:48.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: purged_snaps scrub starts 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: purged_snaps scrub ok 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] boot 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T09:05:48.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:48.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: purged_snaps scrub starts 2024-05-09T09:05:48.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: purged_snaps scrub ok 2024-05-09T09:05:48.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:48.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-05-09T09:05:48.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: osd.11 [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] boot 2024-05-09T09:05:48.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T09:05:48.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T09:05:48.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-09T09:05:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: Detected new or changed devices on smithi179 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi179 to 4272M 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: Detected new or changed devices on smithi179 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi179 to 4272M 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T09:05:50.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:50.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:50.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:50.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: Detected new or changed devices on smithi179 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:05:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi179 to 4272M 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:05:50.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:05:50.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 -- 172.21.15.64:0/2822066597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 msgr2=0x7fe8a81016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:50.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 --2- 172.21.15.64:0/2822066597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a81016a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe8980099c0 tx=0x7fe898009cd0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 -- 172.21.15.64:0/2822066597 shutdown_connections 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 --2- 172.21.15.64:0/2822066597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a8102e20 0x7fe8a81096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 --2- 172.21.15.64:0/2822066597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a81016a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 --2- 172.21.15.64:0/2822066597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8102480 0x7fe8a81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 -- 172.21.15.64:0/2822066597 >> 172.21.15.64:0/2822066597 conn(0x7fe8a80fc820 msgr2=0x7fe8a80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 -- 172.21.15.64:0/2822066597 shutdown_connections 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.534+0000 7fe8ad254700 1 -- 172.21.15.64:0/2822066597 wait complete. 2024-05-09T09:05:50.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 Processor -- start 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 -- start start 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a8105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a8102480 0x7fe8a8103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8102e20 0x7fe8a8103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810cdc0 con 0x7fe8a8101280 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810cc40 con 0x7fe8a8102480 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8ad254700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810cf40 con 0x7fe8a8102e20 2024-05-09T09:05:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8a6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a8105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8a6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a8105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34462/0 (socket says 172.21.15.64:34462) 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8a6d9d700 1 -- 172.21.15.64:0/1716714366 learned_addr learned my addr 172.21.15.64:0/1716714366 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe89e59c700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a8102480 0x7fe8a8103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.535+0000 7fe8a759e700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8102e20 0x7fe8a8103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a6d9d700 1 -- 172.21.15.64:0/1716714366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8102e20 msgr2=0x7fe8a8103da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a6d9d700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8102e20 0x7fe8a8103da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a6d9d700 1 -- 172.21.15.64:0/1716714366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a8102480 msgr2=0x7fe8a8103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a6d9d700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a8102480 0x7fe8a8103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a6d9d700 1 -- 172.21.15.64:0/1716714366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a8104650 con 0x7fe8a8101280 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a6d9d700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a8105210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe89800c010 tx=0x7fe898009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898027070 con 0x7fe8a8101280 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81a2730 con 0x7fe8a8101280 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe898004050 con 0x7fe8a8101280 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898022d30 con 0x7fe8a8101280 2024-05-09T09:05:50.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.536+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a2c00 con 0x7fe8a8101280 2024-05-09T09:05:50.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.537+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8069010 con 0x7fe8a8101280 2024-05-09T09:05:50.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.538+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe89802a070 con 0x7fe8a8101280 2024-05-09T09:05:50.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.539+0000 7fe8a4d99700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8940a2c20 0x7fe8940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:50.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.539+0000 7fe89e59c700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8940a2c20 0x7fe8940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:50.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.539+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8980d0300 con 0x7fe8a8101280 2024-05-09T09:05:50.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.539+0000 7fe89e59c700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8940a2c20 0x7fe8940a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe890009a10 tx=0x7fe890009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:50.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.542+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe898031070 con 0x7fe8a8101280 2024-05-09T09:05:50.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.748+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe8a81a3060 con 0x7fe8a8101280 2024-05-09T09:05:50.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.748+0000 7fe8a4d99700 1 -- 172.21.15.64:0/1716714366 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fe898013070 con 0x7fe8a8101280 2024-05-09T09:05:50.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:05:50.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8940a2c20 msgr2=0x7fe8940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:50.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8940a2c20 0x7fe8940a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe890009a10 tx=0x7fe890009190 comp rx=0 tx=0).stop 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 msgr2=0x7fe8a8105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a8105210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe89800c010 tx=0x7fe898009580 comp rx=0 tx=0).stop 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 shutdown_connections 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8940a2c20 0x7fe8940a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8102e20 0x7fe8a8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8a8101280 0x7fe8a8105210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.750+0000 7fe8ad254700 1 --2- 172.21.15.64:0/1716714366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a8102480 0x7fe8a8103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.751+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 >> 172.21.15.64:0/1716714366 conn(0x7fe8a80fc820 msgr2=0x7fe8a806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.751+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 shutdown_connections 2024-05-09T09:05:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:50.751+0000 7fe8ad254700 1 -- 172.21.15.64:0/1716714366 wait complete. 2024-05-09T09:05:51.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:50 smithi139 ceph-mon[33599]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:51.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:50 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1716714366' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:05:51.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:50 smithi064 ceph-mon[30149]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:51.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:50 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1716714366' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:05:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:50 smithi179 ceph-mon[34086]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:50 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1716714366' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T09:05:51.432 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1715245547,"num_in_osds":12,"osd_in_since":1715245527,"num_remapped_pgs":0} 2024-05-09T09:05:51.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T09:05:51.686 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:05:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:52 smithi139 ceph-mon[33599]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:53.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:52 smithi064 ceph-mon[30149]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:53.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:52 smithi179 ceph-mon[34086]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 -- 172.21.15.64:0/465631832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488101220 msgr2=0x7f74881016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 --2- 172.21.15.64:0/465631832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488101220 0x7f74881016a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f747c009990 tx=0x7f747c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 -- 172.21.15.64:0/465631832 shutdown_connections 2024-05-09T09:05:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 --2- 172.21.15.64:0/465631832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 0x7f748810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 --2- 172.21.15.64:0/465631832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7488107290 0x7f7488107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 --2- 172.21.15.64:0/465631832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488101220 0x7f74881016a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 -- 172.21.15.64:0/465631832 >> 172.21.15.64:0/465631832 conn(0x7f74880781d0 msgr2=0x7f74880785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 -- 172.21.15.64:0/465631832 shutdown_connections 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.370+0000 7f748d53f700 1 -- 172.21.15.64:0/465631832 wait complete. 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 Processor -- start 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 -- start start 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7488101220 0x7f74881977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 0x7f7488197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 0x7f7488198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:53.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f74881114b0 con 0x7f7488107290 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111330 con 0x7f7488101220 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f748d53f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111630 con 0x7f7488101be0 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f74877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 0x7f7488198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f74877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 0x7f7488198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34474/0 (socket says 172.21.15.64:34474) 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f74877fe700 1 -- 172.21.15.64:0/1330912432 learned_addr learned my addr 172.21.15.64:0/1330912432 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.371+0000 7f7486ffd700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7488101220 0x7f74881977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74867fc700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 0x7f7488197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74877fe700 1 -- 172.21.15.64:0/1330912432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 msgr2=0x7f7488197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74877fe700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 0x7f7488197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74877fe700 1 -- 172.21.15.64:0/1330912432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7488101220 msgr2=0x7f74881977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74877fe700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7488101220 0x7f74881977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74877fe700 1 -- 172.21.15.64:0/1330912432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881a6fd0 con 0x7f7488107290 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74867fc700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 0x7f7488197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:05:53.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f74877fe700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 0x7f7488198240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f748000b970 tx=0x7f748000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480019070 con 0x7f7488107290 2024-05-09T09:05:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74881a72c0 con 0x7f7488107290 2024-05-09T09:05:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f748000cde0 con 0x7f7488107290 2024-05-09T09:05:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480007370 con 0x7f7488107290 2024-05-09T09:05:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.372+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a7850 con 0x7f7488107290 2024-05-09T09:05:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.373+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488109800 con 0x7f7488107290 2024-05-09T09:05:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.374+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f748001c070 con 0x7f7488107290 2024-05-09T09:05:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.375+0000 7f746ffff700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74700a2c20 0x7f74700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.375+0000 7f7486ffd700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74700a2c20 0x7f74700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.375+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f74800ce010 con 0x7f7488107290 2024-05-09T09:05:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.375+0000 7f7486ffd700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74700a2c20 0x7f74700a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7478009a10 tx=0x7f7478009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.379+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f748009b410 con 0x7f7488107290 2024-05-09T09:05:53.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.588+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74881091e0 con 0x7f7488107290 2024-05-09T09:05:53.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.589+0000 7f746ffff700 1 -- 172.21.15.64:0/1330912432 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18454 (secure 0 0 0) 0x7f748009ad10 con 0x7f7488107290 2024-05-09T09:05:53.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:05:53.589 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":82,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:05:48.950167+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74700a2c20 msgr2=0x7f74700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74700a2c20 0x7f74700a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7478009a10 tx=0x7f7478009190 comp rx=0 tx=0).stop 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 msgr2=0x7f7488198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 0x7f7488198240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f748000b970 tx=0x7f748000bc80 comp rx=0 tx=0).stop 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 shutdown_connections 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7488107290 0x7f7488198240 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7488101220 0x7f74881977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74700a2c20 0x7f74700a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 --2- 172.21.15.64:0/1330912432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7488101be0 0x7f7488197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 >> 172.21.15.64:0/1330912432 conn(0x7f74880781d0 msgr2=0x7f74880fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:53.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 shutdown_connections 2024-05-09T09:05:53.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:53.592+0000 7f748d53f700 1 -- 172.21.15.64:0/1330912432 wait complete. 2024-05-09T09:05:53.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:53 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1330912432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:05:54.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:53 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1330912432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:05:54.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:53 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1330912432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:05:54.379 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T09:05:54.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T09:05:54.627 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:05:55.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:54 smithi064 ceph-mon[30149]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:55.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:54 smithi179 ceph-mon[34086]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:55.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:54 smithi139 ceph-mon[33599]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 -- 172.21.15.64:0/2443275594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 msgr2=0x7f2c40101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2443275594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c40101890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0099c0 tx=0x7f2c3c009cd0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 -- 172.21.15.64:0/2443275594 shutdown_connections 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2443275594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c40106540 0x7f2c4010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2443275594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c40101890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2443275594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c40101e60 0x7f2c40105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 -- 172.21.15.64:0/2443275594 >> 172.21.15.64:0/2443275594 conn(0x7f2c400781d0 msgr2=0x7f2c400785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 -- 172.21.15.64:0/2443275594 shutdown_connections 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 -- 172.21.15.64:0/2443275594 wait complete. 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 Processor -- start 2024-05-09T09:05:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.400+0000 7f2c4806c700 1 -- start start 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c4806c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c401998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c4806c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c40101e60 0x7f2c40199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c4806c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c40106540 0x7f2c401a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c4806c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401113f0 con 0x7f2c401014b0 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c4806c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40111270 con 0x7f2c40101e60 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c4806c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40111570 con 0x7f2c40106540 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c401998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c401998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45076/0 (socket says 172.21.15.64:45076) 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 -- 172.21.15.64:0/2467061912 learned_addr learned my addr 172.21.15.64:0/2467061912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45607700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c40101e60 0x7f2c40199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c46609700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c40106540 0x7f2c401a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 -- 172.21.15.64:0/2467061912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c40106540 msgr2=0x7f2c401a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c40106540 0x7f2c401a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 -- 172.21.15.64:0/2467061912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c40101e60 msgr2=0x7f2c40199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c40101e60 0x7f2c40199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 -- 172.21.15.64:0/2467061912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c401a4eb0 con 0x7f2c401014b0 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c45e08700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c401998f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c006010 tx=0x7f2c3c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.401+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c027070 con 0x7f2c401014b0 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.402+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c401a5080 con 0x7f2c401014b0 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.402+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c3c003fd0 con 0x7f2c401014b0 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.402+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c022d70 con 0x7f2c401014b0 2024-05-09T09:05:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.402+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401a54d0 con 0x7f2c401014b0 2024-05-09T09:05:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.403+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c40069010 con 0x7f2c401014b0 2024-05-09T09:05:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.404+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c3c02a070 con 0x7f2c401014b0 2024-05-09T09:05:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.404+0000 7f2c36ffd700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c2c0a2c20 0x7f2c2c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.405+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2c3c0cfe40 con 0x7f2c401014b0 2024-05-09T09:05:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.405+0000 7f2c45607700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c2c0a2c20 0x7f2c2c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.405+0000 7f2c45607700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c2c0a2c20 0x7f2c2c0a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005950 tx=0x7f2c300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.408+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c3c031070 con 0x7f2c401014b0 2024-05-09T09:05:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.611+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c4002dd40 con 0x7f2c401014b0 2024-05-09T09:05:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.612+0000 7f2c36ffd700 1 -- 172.21.15.64:0/2467061912 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f2c3c013070 con 0x7f2c401014b0 2024-05-09T09:05:56.611 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:05:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c2c0a2c20 msgr2=0x7f2c2c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c2c0a2c20 0x7f2c2c0a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005950 tx=0x7f2c300058e0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 msgr2=0x7f2c401998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c401998f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c006010 tx=0x7f2c3c009f80 comp rx=0 tx=0).stop 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 shutdown_connections 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c2c0a2c20 0x7f2c2c0a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c40106540 0x7f2c401a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c401014b0 0x7f2c401998f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 --2- 172.21.15.64:0/2467061912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c40101e60 0x7f2c40199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 >> 172.21.15.64:0/2467061912 conn(0x7f2c400781d0 msgr2=0x7f2c40105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 shutdown_connections 2024-05-09T09:05:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:05:56.614+0000 7f2c4806c700 1 -- 172.21.15.64:0/2467061912 wait complete. 2024-05-09T09:05:56.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:56 smithi064 ceph-mon[30149]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2467061912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:05:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:56 smithi179 ceph-mon[34086]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2467061912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:05:57.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:56 smithi139 ceph-mon[33599]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:57.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2467061912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.a on smithi064 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.d on smithi064 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.g on smithi064 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.j on smithi064 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.b on smithi139 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.e on smithi139 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.h on smithi139 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.k on smithi139 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.c on smithi179 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.f on smithi179 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.i on smithi179 2024-05-09T09:05:57.394 INFO:tasks.cephadm:Adding mds.l on smithi179 2024-05-09T09:05:57.395 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2024-05-09T09:05:57.650 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:05:59.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:05:58 smithi179 ceph-mon[34086]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:05:58 smithi064 ceph-mon[30149]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:59.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 -- 172.21.15.179:0/3263151617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210103670 msgr2=0x7fc210103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:59.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 --2- 172.21.15.179:0/3263151617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210103670 0x7fc210103ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc204009990 tx=0x7fc204009ca0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 -- 172.21.15.179:0/3263151617 shutdown_connections 2024-05-09T09:05:59.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 --2- 172.21.15.179:0/3263151617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc210104010 0x7fc21010a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 --2- 172.21.15.179:0/3263151617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc210100690 0x7fc210100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 --2- 172.21.15.179:0/3263151617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210103670 0x7fc210103ad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 -- 172.21.15.179:0/3263151617 >> 172.21.15.179:0/3263151617 conn(0x7fc2100fbff0 msgr2=0x7fc2100fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:59.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 -- 172.21.15.179:0/3263151617 shutdown_connections 2024-05-09T09:05:59.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 -- 172.21.15.179:0/3263151617 wait complete. 2024-05-09T09:05:59.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.306+0000 7fc215c67700 1 Processor -- start 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 -- start start 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 0x7fc210198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc210103670 0x7fc2101989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc210104010 0x7fc21019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc21010de30 con 0x7fc210103670 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc21010dcb0 con 0x7fc210104010 2024-05-09T09:05:59.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc215c67700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc21010dfb0 con 0x7fc210100690 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 0x7fc210198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 0x7fc210198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56074/0 (socket says 172.21.15.179:56074) 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 -- 172.21.15.179:0/812769834 learned_addr learned my addr 172.21.15.179:0/812769834 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20effd700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc210103670 0x7fc2101989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20ffff700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc210104010 0x7fc21019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 -- 172.21.15.179:0/812769834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc210104010 msgr2=0x7fc21019fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc210104010 0x7fc21019fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 -- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc210103670 msgr2=0x7fc2101989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc210103670 0x7fc2101989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 -- 172.21.15.179:0/812769834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2101a00d0 con 0x7fc210100690 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20f7fe700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 0x7fc210198470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc200007f40 tx=0x7fc20000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.307+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20000db20 con 0x7fc210100690 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.308+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2101a03c0 con 0x7fc210100690 2024-05-09T09:05:59.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.308+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc20000dcc0 con 0x7fc210100690 2024-05-09T09:05:59.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.308+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200004820 con 0x7fc210100690 2024-05-09T09:05:59.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.308+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2101a0900 con 0x7fc210100690 2024-05-09T09:05:59.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.309+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc210069010 con 0x7fc210100690 2024-05-09T09:05:59.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.310+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc20001a020 con 0x7fc210100690 2024-05-09T09:05:59.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.310+0000 7fc20cff9700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc1f80a2c20 0x7fc1f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:05:59.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.310+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2000ce1e0 con 0x7fc210100690 2024-05-09T09:05:59.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.311+0000 7fc20effd700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc1f80a2c20 0x7fc1f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:05:59.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.311+0000 7fc20effd700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc1f80a2c20 0x7fc1f80a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc20400b5c0 tx=0x7fc204005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:05:59.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.315+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc200010070 con 0x7fc210100690 2024-05-09T09:05:59.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:05:58 smithi139 ceph-mon[33599]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:05:59.517 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.518+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) -- 0x7fc210062a20 con 0x7fc1f80a2c20 2024-05-09T09:05:59.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.527+0000 7fc20cff9700 1 -- 172.21.15.179:0/812769834 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc210062a20 con 0x7fc1f80a2c20 2024-05-09T09:05:59.526 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc1f80a2c20 msgr2=0x7fc1f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc1f80a2c20 0x7fc1f80a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc20400b5c0 tx=0x7fc204005fd0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 msgr2=0x7fc210198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 0x7fc210198470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc200007f40 tx=0x7fc20000d3b0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 shutdown_connections 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc210104010 0x7fc21019fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc1f80a2c20 0x7fc1f80a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc210100690 0x7fc210198470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 --2- 172.21.15.179:0/812769834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc210103670 0x7fc2101989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 >> 172.21.15.179:0/812769834 conn(0x7fc2100fbff0 msgr2=0x7fc210106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:05:59.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 shutdown_connections 2024-05-09T09:05:59.529 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-09T09:05:59.529+0000 7fc215c67700 1 -- 172.21.15.179:0/812769834 wait complete. 2024-05-09T09:06:00.331 DEBUG:teuthology.orchestra.run.smithi064:mds.a> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.a.service 2024-05-09T09:06:00.334 DEBUG:teuthology.orchestra.run.smithi064:mds.d> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.d.service 2024-05-09T09:06:00.377 DEBUG:teuthology.orchestra.run.smithi064:mds.g> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.g.service 2024-05-09T09:06:00.420 DEBUG:teuthology.orchestra.run.smithi064:mds.j> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.j.service 2024-05-09T09:06:00.462 DEBUG:teuthology.orchestra.run.smithi139:mds.b> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.b.service 2024-05-09T09:06:00.465 DEBUG:teuthology.orchestra.run.smithi139:mds.e> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.e.service 2024-05-09T09:06:00.508 DEBUG:teuthology.orchestra.run.smithi139:mds.h> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.h.service 2024-05-09T09:06:00.550 DEBUG:teuthology.orchestra.run.smithi139:mds.k> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.k.service 2024-05-09T09:06:00.593 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.c.service 2024-05-09T09:06:00.596 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.f.service 2024-05-09T09:06:00.616 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:00.616 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:06:00.616 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-09T09:06:00.616 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:00.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:00 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:00.618 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.i.service 2024-05-09T09:06:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:06:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-09T09:06:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:00.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:00.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:00.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:00.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:00.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:00 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:00.661 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.l.service 2024-05-09T09:06:00.703 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T09:06:00.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 -- 172.21.15.64:0/1404972996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c102310 msgr2=0x7fed5c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 --2- 172.21.15.64:0/1404972996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c102310 0x7fed5c104700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009900 tx=0x7fed4c02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 -- 172.21.15.64:0/1404972996 shutdown_connections 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 --2- 172.21.15.64:0/1404972996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed5c107570 0x7fed5c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 --2- 172.21.15.64:0/1404972996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c104c40 0x7fed5c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 --2- 172.21.15.64:0/1404972996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c102310 0x7fed5c104700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 -- 172.21.15.64:0/1404972996 >> 172.21.15.64:0/1404972996 conn(0x7fed5c0fbd20 msgr2=0x7fed5c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 -- 172.21.15.64:0/1404972996 shutdown_connections 2024-05-09T09:06:00.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.816+0000 7fed60dc1640 1 -- 172.21.15.64:0/1404972996 wait complete. 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 Processor -- start 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 -- start start 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102310 0x7fed5c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 0x7fed5c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed5c107570 0x7fed5c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c10d3e0 con 0x7fed5c104c40 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c10d260 con 0x7fed5c102310 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed60dc1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c10d560 con 0x7fed5c107570 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 0x7fed5c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 0x7fed5c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45088/0 (socket says 172.21.15.64:45088) 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 -- 172.21.15.64:0/13405863 learned_addr learned my addr 172.21.15.64:0/13405863 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:00.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed5ad9d640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed5c107570 0x7fed5c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed5a59c640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102310 0x7fed5c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 -- 172.21.15.64:0/13405863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed5c107570 msgr2=0x7fed5c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed5c107570 0x7fed5c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 -- 172.21.15.64:0/13405863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102310 msgr2=0x7fed5c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102310 0x7fed5c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 -- 172.21.15.64:0/13405863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c19f730 con 0x7fed5c104c40 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed5a59c640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102310 0x7fed5c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed59d9b640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 0x7fed5c198050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d770 tx=0x7fed4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.817+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44017070 con 0x7fed5c104c40 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.818+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed44009c30 con 0x7fed5c104c40 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.818+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4400f620 con 0x7fed5c104c40 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.818+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c19fa20 con 0x7fed5c104c40 2024-05-09T09:06:00.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.818+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c19ff70 con 0x7fed5c104c40 2024-05-09T09:06:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.819+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c0fd810 con 0x7fed5c104c40 2024-05-09T09:06:00.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.820+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed4401a070 con 0x7fed5c104c40 2024-05-09T09:06:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.820+0000 7fed537fe640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed480a2b90 0x7fed480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.820+0000 7fed5a59c640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed480a2b90 0x7fed480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.821+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fed440ce780 con 0x7fed5c104c40 2024-05-09T09:06:00.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.821+0000 7fed5a59c640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed480a2b90 0x7fed480a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006010 tx=0x7fed4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:00.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:00.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:00 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:00.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:00.825+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed44009de0 con 0x7fed5c104c40 2024-05-09T09:06:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.010+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed5c02d410 con 0x7fed5c104c40 2024-05-09T09:06:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.011+0000 7fed537fe640 1 -- 172.21.15.64:0/13405863 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18454 (secure 0 0 0) 0x7fed4409bb80 con 0x7fed5c104c40 2024-05-09T09:06:01.011 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:01.011 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":82,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:05:48.950167+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed480a2b90 msgr2=0x7fed480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed480a2b90 0x7fed480a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006010 tx=0x7fed4c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 msgr2=0x7fed5c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 0x7fed5c198050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d770 tx=0x7fed4400dc30 comp rx=0 tx=0).stop 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 shutdown_connections 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed480a2b90 0x7fed480a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed5c107570 0x7fed5c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c104c40 0x7fed5c198050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 --2- 172.21.15.64:0/13405863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102310 0x7fed5c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.013+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 >> 172.21.15.64:0/13405863 conn(0x7fed5c0fbd20 msgr2=0x7fed5c102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.014+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 shutdown_connections 2024-05-09T09:06:01.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.014+0000 7fed60dc1640 1 -- 172.21.15.64:0/13405863 wait complete. 2024-05-09T09:06:01.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T09:06:01.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:06:01.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 -- 172.21.15.64:0/1482421025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 msgr2=0x7f16d8101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 --2- 172.21.15.64:0/1482421025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8101cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009900 tx=0x7f16cc02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 -- 172.21.15.64:0/1482421025 shutdown_connections 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 --2- 172.21.15.64:0/1482421025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d8103440 0x7f16d8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 --2- 172.21.15.64:0/1482421025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d8102a80 0x7f16d8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 --2- 172.21.15.64:0/1482421025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8101cb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 -- 172.21.15.64:0/1482421025 >> 172.21.15.64:0/1482421025 conn(0x7f16d80fce00 msgr2=0x7f16d80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 -- 172.21.15.64:0/1482421025 shutdown_connections 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.134+0000 7f16df038640 1 -- 172.21.15.64:0/1482421025 wait complete. 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 Processor -- start 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 -- start start 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d8102a80 0x7f16d819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d8103440 0x7f16d81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16d810d3b0 con 0x7f16d8101890 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16d810d230 con 0x7f16d8103440 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16df038640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16d810d530 con 0x7f16d8102a80 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dcdd4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dcdd4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45104/0 (socket says 172.21.15.64:45104) 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dcdd4640 1 -- 172.21.15.64:0/1385861174 learned_addr learned my addr 172.21.15.64:0/1385861174 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16d7fff640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d8102a80 0x7f16d819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dcdd4640 1 -- 172.21.15.64:0/1385861174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d8102a80 msgr2=0x7f16d819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dcdd4640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d8102a80 0x7f16d819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dcdd4640 1 -- 172.21.15.64:0/1385861174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d8103440 msgr2=0x7f16d81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.135+0000 7f16dd5d5640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d8103440 0x7f16d81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16dcdd4640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d8103440 0x7f16d81a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16dcdd4640 1 -- 172.21.15.64:0/1385861174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81a1940 con 0x7f16d8101890 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16dcdd4640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8199d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0314e0 tx=0x7f16cc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc03f030 con 0x7f16d8101890 2024-05-09T09:06:01.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81a1bd0 con 0x7f16d8101890 2024-05-09T09:06:01.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16cc003cb0 con 0x7f16d8101890 2024-05-09T09:06:01.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc035070 con 0x7f16d8101890 2024-05-09T09:06:01.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.136+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81a20c0 con 0x7f16d8101890 2024-05-09T09:06:01.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.137+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d81a2290 con 0x7f16d8101890 2024-05-09T09:06:01.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.138+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f16cc042030 con 0x7f16d8101890 2024-05-09T09:06:01.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.139+0000 7f16d5ffb640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16c80a2b90 0x7f16c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.139+0000 7f16d7fff640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16c80a2b90 0x7f16c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.139+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f16cc0ef250 con 0x7f16d8101890 2024-05-09T09:06:01.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.139+0000 7f16d7fff640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16c80a2b90 0x7f16c80a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16c4005fd0 tx=0x7f16c400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:01.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.143+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16cc0bc650 con 0x7f16d8101890 2024-05-09T09:06:01.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.328+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16d81a2460 con 0x7f16d8101890 2024-05-09T09:06:01.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.328+0000 7f16d5ffb640 1 -- 172.21.15.64:0/1385861174 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f16cc0bbf50 con 0x7f16d8101890 2024-05-09T09:06:01.327 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16c80a2b90 msgr2=0x7f16c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16c80a2b90 0x7f16c80a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16c4005fd0 tx=0x7f16c400b390 comp rx=0 tx=0).stop 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 msgr2=0x7f16d8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8199d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0314e0 tx=0x7f16cc0042e0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 shutdown_connections 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16c80a2b90 0x7f16c80a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d8103440 0x7f16d81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d8102a80 0x7f16d819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 --2- 172.21.15.64:0/1385861174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d8101890 0x7f16d8199d30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 >> 172.21.15.64:0/1385861174 conn(0x7f16d80fce00 msgr2=0x7f16d8108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 shutdown_connections 2024-05-09T09:06:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.330+0000 7f16df038640 1 -- 172.21.15.64:0/1385861174 wait complete. 2024-05-09T09:06:01.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:01.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 -- 172.21.15.64:0/2929889478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 msgr2=0x7f1820104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 --2- 172.21.15.64:0/2929889478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820104700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f181c009900 tx=0x7f181c02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:01.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 -- 172.21.15.64:0/2929889478 shutdown_connections 2024-05-09T09:06:01.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 --2- 172.21.15.64:0/2929889478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1820107570 0x7f18201099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 --2- 172.21.15.64:0/2929889478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1820104c40 0x7f1820107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 --2- 172.21.15.64:0/2929889478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820104700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 -- 172.21.15.64:0/2929889478 >> 172.21.15.64:0/2929889478 conn(0x7f18200fbd20 msgr2=0x7f18200fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 -- 172.21.15.64:0/2929889478 shutdown_connections 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.446+0000 7f182729e640 1 -- 172.21.15.64:0/2929889478 wait complete. 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 Processor -- start 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 -- start start 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1820104c40 0x7f18201002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1820107570 0x7f1820100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f182010d3c0 con 0x7f1820102310 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f182010d240 con 0x7f1820104c40 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182729e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f182010d540 con 0x7f1820107570 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45120/0 (socket says 172.21.15.64:45120) 2024-05-09T09:06:01.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 -- 172.21.15.64:0/1521211214 learned_addr learned my addr 172.21.15.64:0/1521211214 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182583b640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1820107570 0x7f1820100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f1824839640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1820104c40 0x7f18201002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 -- 172.21.15.64:0/1521211214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1820107570 msgr2=0x7f1820100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1820107570 0x7f1820100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 -- 172.21.15.64:0/1521211214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1820104c40 msgr2=0x7f18201002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1820104c40 0x7f18201002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.447+0000 7f182503a640 1 -- 172.21.15.64:0/1521211214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1820101090 con 0x7f1820102310 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.448+0000 7f182503a640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820101c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f181c004770 tx=0x7f181c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.448+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c03f030 con 0x7f1820102310 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.448+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201a1c40 con 0x7f1820102310 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.448+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f181c003cb0 con 0x7f1820102310 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.448+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c035070 con 0x7f1820102310 2024-05-09T09:06:01.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.448+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201a2190 con 0x7f1820102310 2024-05-09T09:06:01.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.449+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18200fd810 con 0x7f1820102310 2024-05-09T09:06:01.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.450+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f181c042030 con 0x7f1820102310 2024-05-09T09:06:01.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.451+0000 7f18167fc640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f180c0a2b90 0x7f180c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:01.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.451+0000 7f1824839640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f180c0a2b90 0x7f180c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:01.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.451+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f181c0ef580 con 0x7f1820102310 2024-05-09T09:06:01.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.451+0000 7f1824839640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f180c0a2b90 0x7f180c0a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18201019f0 tx=0x7f181000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:01.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.456+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f181c0bc980 con 0x7f1820102310 2024-05-09T09:06:01.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:01 smithi064 ceph-mon[30149]: Deploying daemon mds.l on smithi179 2024-05-09T09:06:01.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:01 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/13405863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:01.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:01 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1385861174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:01.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.682+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f182002d1c0 con 0x7f1820102310 2024-05-09T09:06:01.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.682+0000 7f18167fc640 1 -- 172.21.15.64:0/1521211214 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f181c0bc280 con 0x7f1820102310 2024-05-09T09:06:01.681 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:01.681 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T09:06:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 1 2024-05-09T09:06:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f180c0a2b90 msgr2=0x7f180c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f180c0a2b90 0x7f180c0a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18201019f0 tx=0x7f181000a4c0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 msgr2=0x7f1820101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820101c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f181c004770 tx=0x7f181c0042f0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 shutdown_connections 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f180c0a2b90 0x7f180c0a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1820107570 0x7f1820100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1820104c40 0x7f18201002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 --2- 172.21.15.64:0/1521211214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1820102310 0x7f1820101c80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 >> 172.21.15.64:0/1521211214 conn(0x7f18200fbd20 msgr2=0x7f1820102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 shutdown_connections 2024-05-09T09:06:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:01.684+0000 7f182729e640 1 -- 172.21.15.64:0/1521211214 wait complete. 2024-05-09T09:06:01.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:01 smithi139 ceph-mon[33599]: Deploying daemon mds.l on smithi179 2024-05-09T09:06:01.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:01 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/13405863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:01.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:01 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1385861174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:01.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:01 smithi179 ceph-mon[34086]: Deploying daemon mds.l on smithi179 2024-05-09T09:06:01.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:01 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/13405863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:01.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:01 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1385861174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:02.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:02 smithi139 ceph-mon[33599]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:02.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:02 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1521211214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:02.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:02 smithi064 ceph-mon[30149]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:02.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:02 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1521211214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:02.906 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:02 smithi179 ceph-mon[34086]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:02.906 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:02 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1521211214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:02.906 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:02 smithi179 systemd[1]: Starting Ceph mds.l for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:03.169 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:02 smithi179 podman[63737]: 2024-05-09 09:06:02.907795472 +0000 UTC m=+0.107645563 container create 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T09:06:03.169 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:02 smithi179 podman[63737]: 2024-05-09 09:06:02.818254127 +0000 UTC m=+0.018104232 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:03.169 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:03 smithi179 podman[63737]: 2024-05-09 09:06:03.144047688 +0000 UTC m=+0.343897782 container init 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T09:06:03.169 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:03 smithi179 podman[63737]: 2024-05-09 09:06:03.147032422 +0000 UTC m=+0.346882523 container start 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:06:03.479 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:03 smithi179 ceph-mds[63756]: -- 172.21.15.179:0/1755672553 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644ce251a40 con 0x5644ce224c00 2024-05-09T09:06:03.479 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l[63752]: starting mds.l at 2024-05-09T09:06:03.479 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:03 smithi179 bash[63737]: 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca 2024-05-09T09:06:03.479 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:06:03 smithi179 systemd[1]: Started Ceph mds.l for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:03.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:03.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 -- 172.21.15.64:0/326018897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48103450 msgr2=0x7f1c48109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/326018897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48103450 0x7f1c48109ce0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009ca0 tx=0x7f1c4002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 -- 172.21.15.64:0/326018897 shutdown_connections 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/326018897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48103450 0x7f1c48109ce0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/326018897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48102a90 0x7f1c48102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/326018897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 -- 172.21.15.64:0/326018897 >> 172.21.15.64:0/326018897 conn(0x7f1c480fce30 msgr2=0x7f1c480ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 -- 172.21.15.64:0/326018897 shutdown_connections 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.815+0000 7f1c4fd89640 1 -- 172.21.15.64:0/326018897 wait complete. 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 Processor -- start 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 -- start start 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 0x7f1c48197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48103450 0x7f1c4819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810d3d0 con 0x7f1c48102a90 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810d250 con 0x7f1c481018a0 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4fd89640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810d550 con 0x7f1c48103450 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 0x7f1c48197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4db25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4db25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36600/0 (socket says 172.21.15.64:36600) 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 0x7f1c48197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45134/0 (socket says 172.21.15.64:45134) 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4db25640 1 -- 172.21.15.64:0/854045257 learned_addr learned my addr 172.21.15.64:0/854045257 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4e326640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48103450 0x7f1c4819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 -- 172.21.15.64:0/854045257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48103450 msgr2=0x7f1c4819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48103450 0x7f1c4819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 -- 172.21.15.64:0/854045257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 msgr2=0x7f1c48197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4d324640 1 -- 172.21.15.64:0/854045257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4819f6a0 con 0x7f1c48102a90 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4db25640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.816+0000 7f1c4e326640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48103450 0x7f1c4819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.817+0000 7f1c4d324640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 0x7f1c48197fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400e9f0 tx=0x7f1c4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.817+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400ccc0 con 0x7f1c48102a90 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.817+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4819f990 con 0x7f1c48102a90 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.817+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c44005020 con 0x7f1c48102a90 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.817+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44004a00 con 0x7f1c48102a90 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.817+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4819fe90 con 0x7f1c48102a90 2024-05-09T09:06:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.818+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c48108c30 con 0x7f1c48102a90 2024-05-09T09:06:03.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.819+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1c44004ba0 con 0x7f1c48102a90 2024-05-09T09:06:03.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.820+0000 7f1c3affd640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c340a2b90 0x7f1c340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:03.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.820+0000 7f1c4db25640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c340a2b90 0x7f1c340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:03.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.820+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1c440ceb40 con 0x7f1c48102a90 2024-05-09T09:06:03.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.820+0000 7f1c4db25640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c340a2b90 0x7f1c340a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c005fd0 tx=0x7f1c3c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:03.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:03.824+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c4409bf40 con 0x7f1c48102a90 2024-05-09T09:06:04.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.048+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c4802d1c0 con 0x7f1c48102a90 2024-05-09T09:06:04.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.048+0000 7f1c3affd640 1 -- 172.21.15.64:0/854045257 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f1c4409b840 con 0x7f1c48102a90 2024-05-09T09:06:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:04.047 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:06:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c340a2b90 msgr2=0x7f1c340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c340a2b90 0x7f1c340a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c005fd0 tx=0x7f1c3c00b4a0 comp rx=0 tx=0).stop 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 msgr2=0x7f1c48197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 0x7f1c48197fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400e9f0 tx=0x7f1c4400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 shutdown_connections 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c340a2b90 0x7f1c340a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c48103450 0x7f1c4819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c48102a90 0x7f1c48197fc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 --2- 172.21.15.64:0/854045257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c481018a0 0x7f1c48197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 >> 172.21.15.64:0/854045257 conn(0x7f1c480fce30 msgr2=0x7f1c48105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 shutdown_connections 2024-05-09T09:06:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:04.051+0000 7f1c4fd89640 1 -- 172.21.15.64:0/854045257 wait complete. 2024-05-09T09:06:04.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:04.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6832/3794352368,v1:172.21.15.179:6833/3794352368] up:boot 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: fsmap 1 up:standby 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: Deploying daemon mds.k on smithi139 2024-05-09T09:06:04.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:04 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/854045257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6832/3794352368,v1:172.21.15.179:6833/3794352368] up:boot 2024-05-09T09:06:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: fsmap 1 up:standby 2024-05-09T09:06:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:06:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:04.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:04.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:04.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: Deploying daemon mds.k on smithi139 2024-05-09T09:06:04.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:04 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/854045257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6832/3794352368,v1:172.21.15.179:6833/3794352368] up:boot 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: fsmap 1 up:standby 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: Deploying daemon mds.k on smithi139 2024-05-09T09:06:04.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:04 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/854045257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:05.807 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:05 smithi139 systemd[1]: Starting Ceph mds.k for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:06.060 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:05 smithi139 podman[63321]: 2024-05-09 09:06:05.711799158 +0000 UTC m=+0.018565740 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:06.060 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:05 smithi139 podman[63321]: 2024-05-09 09:06:05.817996921 +0000 UTC m=+0.124763501 container create 6290d5fab2919328abab466c239c69ce61f31a6703b63bc1fadee39afaf3ff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T09:06:06.060 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:06 smithi139 podman[63321]: 2024-05-09 09:06:06.062297816 +0000 UTC m=+0.369064393 container init 6290d5fab2919328abab466c239c69ce61f31a6703b63bc1fadee39afaf3ff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:06:06.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:06.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 -- 172.21.15.64:0/766569568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4103440 msgr2=0x7fb8c4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:06.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/766569568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4103440 0x7fb8c4109cd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009ca0 tx=0x7fb8b402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 -- 172.21.15.64:0/766569568 shutdown_connections 2024-05-09T09:06:06.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/766569568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4103440 0x7fb8c4109cd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/766569568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8c4102a80 0x7fb8c4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/766569568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c4101890 0x7fb8c4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 -- 172.21.15.64:0/766569568 >> 172.21.15.64:0/766569568 conn(0x7fb8c40fcde0 msgr2=0x7fb8c40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 -- 172.21.15.64:0/766569568 shutdown_connections 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.185+0000 7fb8cc01c640 1 -- 172.21.15.64:0/766569568 wait complete. 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 Processor -- start 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 -- start start 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c4101890 0x7fb8c4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 0x7fb8c4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8c4103440 0x7fb8c419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:06.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410d410 con 0x7fb8c4102a80 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410d290 con 0x7fb8c4101890 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8cc01c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410d590 con 0x7fb8c4103440 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 0x7fb8c4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 0x7fb8c4197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34770/0 (socket says 172.21.15.64:34770) 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 -- 172.21.15.64:0/3940789107 learned_addr learned my addr 172.21.15.64:0/3940789107 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c9db8640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c4101890 0x7fb8c4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8ca5b9640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8c4103440 0x7fb8c419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 -- 172.21.15.64:0/3940789107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8c4103440 msgr2=0x7fb8c419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:06.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8c4103440 0x7fb8c419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 -- 172.21.15.64:0/3940789107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c4101890 msgr2=0x7fb8c4197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c4101890 0x7fb8c4197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.186+0000 7fb8c95b7640 1 -- 172.21.15.64:0/3940789107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c419f670 con 0x7fb8c4102a80 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.187+0000 7fb8c95b7640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 0x7fb8c4197f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.188+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800cd80 con 0x7fb8c4102a80 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.188+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8b80054d0 con 0x7fb8c4102a80 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.188+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8010490 con 0x7fb8c4102a80 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.188+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c419f960 con 0x7fb8c4102a80 2024-05-09T09:06:06.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.188+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c419fe60 con 0x7fb8c4102a80 2024-05-09T09:06:06.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.189+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c4108b50 con 0x7fb8c4102a80 2024-05-09T09:06:06.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.190+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8b8005000 con 0x7fb8c4102a80 2024-05-09T09:06:06.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.191+0000 7fb8c2ffd640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb8ac0a2b90 0x7fb8ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:06.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.191+0000 7fb8c9db8640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb8ac0a2b90 0x7fb8ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:06.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.191+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb8b80cf120 con 0x7fb8c4102a80 2024-05-09T09:06:06.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.191+0000 7fb8c9db8640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb8ac0a2b90 0x7fb8ac0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8c41028e0 tx=0x7fb8bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:06.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.194+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb8b8053740 con 0x7fb8c4102a80 2024-05-09T09:06:06.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:06 smithi139 ceph-mon[33599]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:06.322 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:06 smithi139 podman[63321]: 2024-05-09 09:06:06.065343539 +0000 UTC m=+0.372110124 container start 6290d5fab2919328abab466c239c69ce61f31a6703b63bc1fadee39afaf3ff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:06:06.323 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:06 smithi139 ceph-mds[63340]: -- 172.21.15.139:0/1495399919 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a0dba3a40 con 0x558a0db76c00 2024-05-09T09:06:06.323 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:06 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k[63336]: starting mds.k at 2024-05-09T09:06:06.323 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:06 smithi139 bash[63321]: 6290d5fab2919328abab466c239c69ce61f31a6703b63bc1fadee39afaf3ff4b 2024-05-09T09:06:06.323 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:06:06 smithi139 systemd[1]: Started Ceph mds.k for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:06.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:06 smithi064 ceph-mon[30149]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:06.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.422+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c402d1c0 con 0x7fb8c4102a80 2024-05-09T09:06:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.423+0000 7fb8c2ffd640 1 -- 172.21.15.64:0/3940789107 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fb8b80052f0 con 0x7fb8c4102a80 2024-05-09T09:06:06.422 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:06.422 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:06:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb8ac0a2b90 msgr2=0x7fb8ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb8ac0a2b90 0x7fb8ac0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8c41028e0 tx=0x7fb8bc005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 msgr2=0x7fb8c4197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 0x7fb8c4197f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800edf0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 shutdown_connections 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb8ac0a2b90 0x7fb8ac0a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8c4103440 0x7fb8c419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8c4102a80 0x7fb8c4197f90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 --2- 172.21.15.64:0/3940789107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c4101890 0x7fb8c4197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 >> 172.21.15.64:0/3940789107 conn(0x7fb8c40fcde0 msgr2=0x7fb8c4105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:06.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 shutdown_connections 2024-05-09T09:06:06.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:06.425+0000 7fb8cc01c640 1 -- 172.21.15.64:0/3940789107 wait complete. 2024-05-09T09:06:06.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:06 smithi179 ceph-mon[34086]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6832/3830149624,v1:172.21.15.139:6833/3830149624] up:boot 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: fsmap 2 up:standby 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: Deploying daemon mds.j on smithi064 2024-05-09T09:06:07.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3940789107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:07.480 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6832/3830149624,v1:172.21.15.139:6833/3830149624] up:boot 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: fsmap 2 up:standby 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: Deploying daemon mds.j on smithi064 2024-05-09T09:06:07.481 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3940789107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6832/3830149624,v1:172.21.15.139:6833/3830149624] up:boot 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: fsmap 2 up:standby 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:07.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:07.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: Deploying daemon mds.j on smithi064 2024-05-09T09:06:07.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3940789107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:08.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.553+0000 7fa8163f9640 1 -- 172.21.15.64:0/3028168237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 msgr2=0x7fa810108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3028168237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa810108bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa808009ca0 tx=0x7fa80802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 -- 172.21.15.64:0/3028168237 shutdown_connections 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3028168237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa810108bd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3028168237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa810101980 0x7fa810101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3028168237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 0x7fa810100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 -- 172.21.15.64:0/3028168237 >> 172.21.15.64:0/3028168237 conn(0x7fa8100fbd20 msgr2=0x7fa8100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 -- 172.21.15.64:0/3028168237 shutdown_connections 2024-05-09T09:06:08.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 -- 172.21.15.64:0/3028168237 wait complete. 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.554+0000 7fa8163f9640 1 Processor -- start 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 -- start start 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 0x7fa810199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa810101980 0x7fa81019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa8101a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa81010c2c0 con 0x7fa810102340 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa81010c140 con 0x7fa810100790 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa8163f9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa81010c440 con 0x7fa810101980 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa814996640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa8101a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa814996640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa8101a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34796/0 (socket says 172.21.15.64:34796) 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa814996640 1 -- 172.21.15.64:0/3108412154 learned_addr learned my addr 172.21.15.64:0/3108412154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa80ffff640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 0x7fa810199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.555+0000 7fa80f7fe640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa810101980 0x7fa81019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa814996640 1 -- 172.21.15.64:0/3108412154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa810101980 msgr2=0x7fa81019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa814996640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa810101980 0x7fa81019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa814996640 1 -- 172.21.15.64:0/3108412154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 msgr2=0x7fa810199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa814996640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 0x7fa810199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa814996640 1 -- 172.21.15.64:0/3108412154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8101a1860 con 0x7fa810102340 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa80ffff640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 0x7fa810199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa814996640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa8101a11c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa808009dc0 tx=0x7fa808003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808032ec0 con 0x7fa810102340 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8080044f0 con 0x7fa810102340 2024-05-09T09:06:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808030d40 con 0x7fa810102340 2024-05-09T09:06:08.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8101a1af0 con 0x7fa810102340 2024-05-09T09:06:08.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.556+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101a1ff0 con 0x7fa810102340 2024-05-09T09:06:08.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.557+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8101a21c0 con 0x7fa810102340 2024-05-09T09:06:08.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.559+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa808004040 con 0x7fa810102340 2024-05-09T09:06:08.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.560+0000 7fa80d7fa640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7f80a2d30 0x7fa7f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:08.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.560+0000 7fa80ffff640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7f80a2d30 0x7fa7f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:08.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.560+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa8080f0500 con 0x7fa810102340 2024-05-09T09:06:08.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.561+0000 7fa80ffff640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7f80a2d30 0x7fa7f80a51f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8000058c0 tx=0x7fa800005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:08.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.563+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa8080bd8a0 con 0x7fa810102340 2024-05-09T09:06:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:08 smithi139 ceph-mon[33599]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:08.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:08 smithi064 ceph-mon[30149]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:08.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:08 smithi179 ceph-mon[34086]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:08.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.792+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8101a2390 con 0x7fa810102340 2024-05-09T09:06:08.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.792+0000 7fa80d7fa640 1 -- 172.21.15.64:0/3108412154 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fa8080bd1a0 con 0x7fa810102340 2024-05-09T09:06:08.792 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:08.792 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:06:08.793 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-05-09T09:06:08.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7f80a2d30 msgr2=0x7fa7f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:08.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7f80a2d30 0x7fa7f80a51f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8000058c0 tx=0x7fa800005850 comp rx=0 tx=0).stop 2024-05-09T09:06:08.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 msgr2=0x7fa8101a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:08.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa8101a11c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa808009dc0 tx=0x7fa808003ba0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 shutdown_connections 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7f80a2d30 0x7fa7f80a51f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa810102340 0x7fa8101a11c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa810101980 0x7fa81019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 --2- 172.21.15.64:0/3108412154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa810100790 0x7fa810199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 >> 172.21.15.64:0/3108412154 conn(0x7fa8100fbd20 msgr2=0x7fa8100fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 shutdown_connections 2024-05-09T09:06:08.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:08.794+0000 7fa8163f9640 1 -- 172.21.15.64:0/3108412154 wait complete. 2024-05-09T09:06:09.274 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:06:09 smithi064 systemd[1]: Started Ceph mds.j for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:09.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3108412154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:09 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3108412154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:09.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:09.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:09 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3108412154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:09.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:09 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:10 smithi064 ceph-mon[30149]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:10 smithi064 ceph-mon[30149]: Deploying daemon mds.d on smithi064 2024-05-09T09:06:10.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:10 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.64:6834/201840839,v1:172.21.15.64:6835/201840839] up:boot 2024-05-09T09:06:10.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:10 smithi064 ceph-mon[30149]: fsmap 3 up:standby 2024-05-09T09:06:10.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:10 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T09:06:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:10 smithi139 ceph-mon[33599]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:10.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:10 smithi139 ceph-mon[33599]: Deploying daemon mds.d on smithi064 2024-05-09T09:06:10.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:10 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.64:6834/201840839,v1:172.21.15.64:6835/201840839] up:boot 2024-05-09T09:06:10.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:10 smithi139 ceph-mon[33599]: fsmap 3 up:standby 2024-05-09T09:06:10.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:10 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T09:06:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:10 smithi179 ceph-mon[34086]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:10 smithi179 ceph-mon[34086]: Deploying daemon mds.d on smithi064 2024-05-09T09:06:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:10 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.64:6834/201840839,v1:172.21.15.64:6835/201840839] up:boot 2024-05-09T09:06:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:10 smithi179 ceph-mon[34086]: fsmap 3 up:standby 2024-05-09T09:06:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:10 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T09:06:10.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 -- 172.21.15.64:0/562094596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 msgr2=0x7fd504107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 --2- 172.21.15.64:0/562094596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504107070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009900 tx=0x7fd4f802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 -- 172.21.15.64:0/562094596 shutdown_connections 2024-05-09T09:06:10.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 --2- 172.21.15.64:0/562094596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5041075b0 0x7fd5041099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:10.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 --2- 172.21.15.64:0/562094596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504107070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:10.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 --2- 172.21.15.64:0/562094596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd504102350 0x7fd504104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:10.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 -- 172.21.15.64:0/562094596 >> 172.21.15.64:0/562094596 conn(0x7fd5040fbce0 msgr2=0x7fd5040fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:10.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 -- 172.21.15.64:0/562094596 shutdown_connections 2024-05-09T09:06:10.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.926+0000 7fd508b5a640 1 -- 172.21.15.64:0/562094596 wait complete. 2024-05-09T09:06:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 Processor -- start 2024-05-09T09:06:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 -- start start 2024-05-09T09:06:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd504102350 0x7fd504101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5041075b0 0x7fd504100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410d390 con 0x7fd504104c80 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410d210 con 0x7fd504102350 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd508b5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410d510 con 0x7fd5041075b0 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd501d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd501d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504100300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34846/0 (socket says 172.21.15.64:34846) 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd501d9b640 1 -- 172.21.15.64:0/3251760847 learned_addr learned my addr 172.21.15.64:0/3251760847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd50259c640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd504102350 0x7fd504101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.927+0000 7fd502d9d640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5041075b0 0x7fd504100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd501d9b640 1 -- 172.21.15.64:0/3251760847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5041075b0 msgr2=0x7fd504100840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd501d9b640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5041075b0 0x7fd504100840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd501d9b640 1 -- 172.21.15.64:0/3251760847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd504102350 msgr2=0x7fd504101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd501d9b640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd504102350 0x7fd504101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd501d9b640 1 -- 172.21.15.64:0/3251760847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5041010d0 con 0x7fd504104c80 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd50259c640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd504102350 0x7fd504101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd501d9b640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504100300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80314e0 tx=0x7fd4f80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f803f030 con 0x7fd504104c80 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4f8003cb0 con 0x7fd504104c80 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5041a1bb0 con 0x7fd504104c80 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8035070 con 0x7fd504104c80 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.928+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041a2100 con 0x7fd504104c80 2024-05-09T09:06:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.929+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5040fd7d0 con 0x7fd504104c80 2024-05-09T09:06:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.930+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd4f8042030 con 0x7fd504104c80 2024-05-09T09:06:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.931+0000 7fd4f37fe640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4ec0a2b90 0x7fd4ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.931+0000 7fd50259c640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4ec0a2b90 0x7fd4ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.931+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd4f80ef260 con 0x7fd504104c80 2024-05-09T09:06:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.931+0000 7fd50259c640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4ec0a2b90 0x7fd4ec0a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc005fd0 tx=0x7fd4fc009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:10.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:10.935+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4f80bc660 con 0x7fd504104c80 2024-05-09T09:06:11.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.183+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd50402d370 con 0x7fd504104c80 2024-05-09T09:06:11.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.184+0000 7fd4f37fe640 1 -- 172.21.15.64:0/3251760847 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fd4f80bbf60 con 0x7fd504104c80 2024-05-09T09:06:11.184 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:11.184 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:06:11.185 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 4 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4ec0a2b90 msgr2=0x7fd4ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4ec0a2b90 0x7fd4ec0a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc005fd0 tx=0x7fd4fc009520 comp rx=0 tx=0).stop 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 msgr2=0x7fd504100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504100300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80314e0 tx=0x7fd4f80042e0 comp rx=0 tx=0).stop 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 shutdown_connections 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4ec0a2b90 0x7fd4ec0a5050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:11.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5041075b0 0x7fd504100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd504104c80 0x7fd504100300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 --2- 172.21.15.64:0/3251760847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd504102350 0x7fd504101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 >> 172.21.15.64:0/3251760847 conn(0x7fd5040fbce0 msgr2=0x7fd5041041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 shutdown_connections 2024-05-09T09:06:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:11.187+0000 7fd508b5a640 1 -- 172.21.15.64:0/3251760847 wait complete. 2024-05-09T09:06:11.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3251760847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:11.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:11 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3251760847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:11.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:11 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3251760847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:12.152 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:06:11 smithi064 systemd[1]: Started Ceph mds.d for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:12.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:12.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:12.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:12.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:12 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:12.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:12 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:12 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:13.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 -- 172.21.15.64:0/232278736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740102310 msgr2=0x7f2740104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 --2- 172.21.15.64:0/232278736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740102310 0x7f2740104700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2730009900 tx=0x7f273002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 -- 172.21.15.64:0/232278736 shutdown_connections 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 --2- 172.21.15.64:0/232278736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2740107570 0x7f27401099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 --2- 172.21.15.64:0/232278736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2740104c40 0x7f2740107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 --2- 172.21.15.64:0/232278736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740102310 0x7f2740104700 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 -- 172.21.15.64:0/232278736 >> 172.21.15.64:0/232278736 conn(0x7f27400fbd20 msgr2=0x7f27400fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 -- 172.21.15.64:0/232278736 shutdown_connections 2024-05-09T09:06:13.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.320+0000 7f27461b7640 1 -- 172.21.15.64:0/232278736 wait complete. 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 Processor -- start 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 -- start start 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2740102310 0x7f2740197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 0x7f2740198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2740107570 0x7f274019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f274010d410 con 0x7f2740104c40 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f274010d290 con 0x7f2740102310 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f27461b7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f274010d590 con 0x7f2740107570 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.321+0000 7f273effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 0x7f2740198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 0x7f2740198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34878/0 (socket says 172.21.15.64:34878) 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 -- 172.21.15.64:0/711817551 learned_addr learned my addr 172.21.15.64:0/711817551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273ffff640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2740107570 0x7f274019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273f7fe640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2740102310 0x7f2740197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 -- 172.21.15.64:0/711817551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2740107570 msgr2=0x7f274019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:13.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2740107570 0x7f274019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 -- 172.21.15.64:0/711817551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2740102310 msgr2=0x7f2740197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2740102310 0x7f2740197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 -- 172.21.15.64:0/711817551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274019f740 con 0x7f2740104c40 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273effd640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 0x7f2740198060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2734005fd0 tx=0x7f273400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734017070 con 0x7f2740104c40 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2734009c30 con 0x7f2740104c40 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274019fa30 con 0x7f2740104c40 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734003d40 con 0x7f2740104c40 2024-05-09T09:06:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.322+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274019ffd0 con 0x7f2740104c40 2024-05-09T09:06:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.323+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27400fd810 con 0x7f2740104c40 2024-05-09T09:06:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.324+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2734003ee0 con 0x7f2740104c40 2024-05-09T09:06:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.325+0000 7f273cff9640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f27280a2c10 0x7f27280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.326+0000 7f273f7fe640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f27280a2c10 0x7f27280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.326+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27340ce850 con 0x7f2740104c40 2024-05-09T09:06:13.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.326+0000 7f273f7fe640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f27280a2c10 0x7f27280a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2730000c00 tx=0x7f27300314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.331+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2734020030 con 0x7f2740104c40 2024-05-09T09:06:13.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.555+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f274002d3c0 con 0x7f2740104c40 2024-05-09T09:06:13.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.555+0000 7f273cff9640 1 -- 172.21.15.64:0/711817551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f273409bc50 con 0x7f2740104c40 2024-05-09T09:06:13.556 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:13.556 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T09:06:13.556 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 5 2024-05-09T09:06:13.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f27280a2c10 msgr2=0x7f27280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f27280a2c10 0x7f27280a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2730000c00 tx=0x7f27300314a0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 msgr2=0x7f2740198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 0x7f2740198060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2734005fd0 tx=0x7f273400dde0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 shutdown_connections 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f27280a2c10 0x7f27280a50d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2740107570 0x7f274019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2740104c40 0x7f2740198060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 --2- 172.21.15.64:0/711817551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2740102310 0x7f2740197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.557+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 >> 172.21.15.64:0/711817551 conn(0x7f27400fbd20 msgr2=0x7f2740102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.558+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 shutdown_connections 2024-05-09T09:06:13.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:13.558+0000 7f27461b7640 1 -- 172.21.15.64:0/711817551 wait complete. 2024-05-09T09:06:13.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:13 smithi064 ceph-mon[30149]: Deploying daemon mds.e on smithi139 2024-05-09T09:06:13.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:13 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.64:6836/4065753229,v1:172.21.15.64:6837/4065753229] up:boot 2024-05-09T09:06:13.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:13 smithi064 ceph-mon[30149]: fsmap 4 up:standby 2024-05-09T09:06:13.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:13 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T09:06:13.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:13 smithi139 ceph-mon[33599]: Deploying daemon mds.e on smithi139 2024-05-09T09:06:13.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:13 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.64:6836/4065753229,v1:172.21.15.64:6837/4065753229] up:boot 2024-05-09T09:06:13.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:13 smithi139 ceph-mon[33599]: fsmap 4 up:standby 2024-05-09T09:06:13.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:13 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T09:06:13.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:13 smithi179 ceph-mon[34086]: Deploying daemon mds.e on smithi139 2024-05-09T09:06:13.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:13 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.64:6836/4065753229,v1:172.21.15.64:6837/4065753229] up:boot 2024-05-09T09:06:13.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:13 smithi179 ceph-mon[34086]: fsmap 4 up:standby 2024-05-09T09:06:13.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:13 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T09:06:14.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:14 smithi139 ceph-mon[33599]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:14.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:14 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/711817551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:14.529 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 systemd[1]: Starting Ceph mds.e for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:14.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:14 smithi064 ceph-mon[30149]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:14.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:14 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/711817551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:14 smithi179 ceph-mon[34086]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:14 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/711817551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:14.807 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 podman[63648]: 2024-05-09 09:06:14.433254155 +0000 UTC m=+0.017738580 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:14.807 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 podman[63648]: 2024-05-09 09:06:14.55574748 +0000 UTC m=+0.140231892 container create 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T09:06:14.807 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 podman[63648]: 2024-05-09 09:06:14.808798656 +0000 UTC m=+0.393283067 container init 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:06:15.072 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 podman[63648]: 2024-05-09 09:06:14.811548206 +0000 UTC m=+0.396032619 container start 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:06:15.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 ceph-mds[63667]: -- 172.21.15.139:0/2033457606 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56264c7e1a40 con 0x56264c7b4c00 2024-05-09T09:06:15.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e[63663]: starting mds.e at 2024-05-09T09:06:15.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 bash[63648]: 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 2024-05-09T09:06:15.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:06:14 smithi139 systemd[1]: Started Ceph mds.e for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:15.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 -- 172.21.15.64:0/2769358146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 msgr2=0x7f8790108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 --2- 172.21.15.64:0/2769358146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790108bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8788009ca0 tx=0x7f878802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 -- 172.21.15.64:0/2769358146 shutdown_connections 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 --2- 172.21.15.64:0/2769358146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790108bd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 --2- 172.21.15.64:0/2769358146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 0x7f8790101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 --2- 172.21.15.64:0/2769358146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 0x7f8790100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 -- 172.21.15.64:0/2769358146 >> 172.21.15.64:0/2769358146 conn(0x7f87900fbd20 msgr2=0x7f87900fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 -- 172.21.15.64:0/2769358146 shutdown_connections 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.689+0000 7f8797541640 1 -- 172.21.15.64:0/2769358146 wait complete. 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 Processor -- start 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 -- start start 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 0x7f879010b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 0x7f8790109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f879010d3e0 con 0x7f8790102340 2024-05-09T09:06:15.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f879010d260 con 0x7f8790101980 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8797541640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f879010d560 con 0x7f8790100790 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54456/0 (socket says 172.21.15.64:54456) 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 -- 172.21.15.64:0/4064182854 learned_addr learned my addr 172.21.15.64:0/4064182854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f87952dd640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 0x7f879010b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8794adc640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 0x7f8790109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 -- 172.21.15.64:0/4064182854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 msgr2=0x7f879010b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:15.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 0x7f879010b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 -- 172.21.15.64:0/4064182854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 msgr2=0x7f8790109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 0x7f8790109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 -- 172.21.15.64:0/4064182854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879010a520 con 0x7f8790102340 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8794adc640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 0x7f8790109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f87952dd640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 0x7f879010b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.690+0000 7f8795ade640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790109c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8788031970 tx=0x7f8788003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.691+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788004040 con 0x7f8790102340 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.691+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8788004690 con 0x7f8790102340 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.691+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788030c60 con 0x7f8790102340 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.691+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87901a1b00 con 0x7f8790102340 2024-05-09T09:06:15.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.691+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87901a20a0 con 0x7f8790102340 2024-05-09T09:06:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.692+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8790107ab0 con 0x7f8790102340 2024-05-09T09:06:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.693+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f87880041e0 con 0x7f8790102340 2024-05-09T09:06:15.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.694+0000 7f87827fc640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f877c0a2c10 0x7f877c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.694+0000 7f87952dd640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f877c0a2c10 0x7f877c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.694+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f87880f00c0 con 0x7f8790102340 2024-05-09T09:06:15.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.695+0000 7f87952dd640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f877c0a2c10 0x7f877c0a50d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87901017e0 tx=0x7f8784005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:15.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.699+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f87880bd5e0 con 0x7f8790102340 2024-05-09T09:06:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.925+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8790062430 con 0x7f8790102340 2024-05-09T09:06:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.926+0000 7f87827fc640 1 -- 172.21.15.64:0/4064182854 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f8788039e10 con 0x7f8790102340 2024-05-09T09:06:15.925 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:15.925 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T09:06:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2024-05-09T09:06:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f877c0a2c10 msgr2=0x7f877c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f877c0a2c10 0x7f877c0a50d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87901017e0 tx=0x7f8784005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 msgr2=0x7f8790109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790109c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8788031970 tx=0x7f8788003ba0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 shutdown_connections 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f877c0a2c10 0x7f877c0a50d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8790102340 0x7f8790109c90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8790101980 0x7f8790109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 --2- 172.21.15.64:0/4064182854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790100790 0x7f879010b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 >> 172.21.15.64:0/4064182854 conn(0x7f87900fbd20 msgr2=0x7f8790104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 shutdown_connections 2024-05-09T09:06:15.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:15.928+0000 7f8797541640 1 -- 172.21.15.64:0/4064182854 wait complete. 2024-05-09T09:06:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: Deploying daemon mds.f on smithi179 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6834/3760508086,v1:172.21.15.139:6835/3760508086] up:boot 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: fsmap 5 up:standby 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:06:16.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:15 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4064182854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:16.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: Deploying daemon mds.f on smithi179 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6834/3760508086,v1:172.21.15.139:6835/3760508086] up:boot 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: fsmap 5 up:standby 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:06:16.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:15 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4064182854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:16.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: Deploying daemon mds.f on smithi179 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6834/3760508086,v1:172.21.15.139:6835/3760508086] up:boot 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: fsmap 5 up:standby 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:06:16.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:15 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4064182854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:17.229 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 systemd[1]: Starting Ceph mds.f for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:17.666 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 podman[64047]: 2024-05-09 09:06:17.295134088 +0000 UTC m=+0.020988038 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:17.666 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 podman[64047]: 2024-05-09 09:06:17.414001182 +0000 UTC m=+0.139855115 container create 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T09:06:17.666 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 podman[64047]: 2024-05-09 09:06:17.667154746 +0000 UTC m=+0.393008672 container init 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:06:17.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:17.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 podman[64047]: 2024-05-09 09:06:17.67038181 +0000 UTC m=+0.396235736 container start 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T09:06:17.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 ceph-mds[64074]: -- 172.21.15.179:0/2223589319 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558771559a40 con 0x55877152c800 2024-05-09T09:06:17.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f[64064]: starting mds.f at 2024-05-09T09:06:17.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 bash[64047]: 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 2024-05-09T09:06:17.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:06:17 smithi179 systemd[1]: Started Ceph mds.f for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:18.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 -- 172.21.15.64:0/2983358478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc101960 msgr2=0x7f86dc101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:18.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 --2- 172.21.15.64:0/2983358478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc101960 0x7f86dc101de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c402a410 comp rx=0 tx=0).stop 2024-05-09T09:06:18.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 -- 172.21.15.64:0/2983358478 shutdown_connections 2024-05-09T09:06:18.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 --2- 172.21.15.64:0/2983358478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86dc102320 0x7f86dc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 --2- 172.21.15.64:0/2983358478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc101960 0x7f86dc101de0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 --2- 172.21.15.64:0/2983358478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86dc100770 0x7f86dc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 -- 172.21.15.64:0/2983358478 >> 172.21.15.64:0/2983358478 conn(0x7f86dc0fbce0 msgr2=0x7f86dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 -- 172.21.15.64:0/2983358478 shutdown_connections 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 -- 172.21.15.64:0/2983358478 wait complete. 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 Processor -- start 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.057+0000 7f86e18fc640 1 -- start start 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86e18fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 0x7f86dc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86e18fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86dc101960 0x7f86dc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86e18fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86dc102320 0x7f86dc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86e18fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10c2f0 con 0x7f86dc100770 2024-05-09T09:06:18.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86e18fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10c170 con 0x7f86dc102320 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86e18fc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10c470 con 0x7f86dc101960 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 0x7f86dc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 0x7f86dc069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54484/0 (socket says 172.21.15.64:54484) 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 -- 172.21.15.64:0/1462039960 learned_addr learned my addr 172.21.15.64:0/1462039960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86d27fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86dc101960 0x7f86dc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86db7fe640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86dc102320 0x7f86dc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 -- 172.21.15.64:0/1462039960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86dc101960 msgr2=0x7f86dc19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86dc101960 0x7f86dc19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 -- 172.21.15.64:0/1462039960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86dc102320 msgr2=0x7f86dc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:18.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86dc102320 0x7f86dc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 -- 172.21.15.64:0/1462039960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc19f630 con 0x7f86dc100770 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.058+0000 7f86daffd640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 0x7f86dc069550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c970 tx=0x7f86cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.059+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc007c10 con 0x7f86dc100770 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.059+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86cc007db0 con 0x7f86dc100770 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.059+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc19f920 con 0x7f86dc100770 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.059+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00f870 con 0x7f86dc100770 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.059+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc19fe40 con 0x7f86dc100770 2024-05-09T09:06:18.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.060+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc107a90 con 0x7f86dc100770 2024-05-09T09:06:18.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.061+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f86cc00fa10 con 0x7f86dc100770 2024-05-09T09:06:18.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.062+0000 7f86e08fa640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f86c80a2b90 0x7f86c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:18.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.062+0000 7f86d27fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f86c80a2b90 0x7f86c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:18.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.062+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f86cc0cf1d0 con 0x7f86dc100770 2024-05-09T09:06:18.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.062+0000 7f86d27fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f86c80a2b90 0x7f86c80a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f86c40314e0 tx=0x7f86c4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:18.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.065+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86cc09c6f0 con 0x7f86dc100770 2024-05-09T09:06:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.294+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86dc02d1c0 con 0x7f86dc100770 2024-05-09T09:06:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.295+0000 7f86e08fa640 1 -- 172.21.15.64:0/1462039960 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f86cc09bff0 con 0x7f86dc100770 2024-05-09T09:06:18.294 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:18.294 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-09T09:06:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f86c80a2b90 msgr2=0x7f86c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f86c80a2b90 0x7f86c80a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f86c40314e0 tx=0x7f86c4005a20 comp rx=0 tx=0).stop 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 msgr2=0x7f86dc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 0x7f86dc069550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c970 tx=0x7f86cc00ce30 comp rx=0 tx=0).stop 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 shutdown_connections 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f86c80a2b90 0x7f86c80a5050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86dc102320 0x7f86dc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86dc101960 0x7f86dc19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 --2- 172.21.15.64:0/1462039960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc100770 0x7f86dc069550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 >> 172.21.15.64:0/1462039960 conn(0x7f86dc0fbce0 msgr2=0x7f86dc104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 shutdown_connections 2024-05-09T09:06:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:18.297+0000 7f86e18fc640 1 -- 172.21.15.64:0/1462039960 wait complete. 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:18.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: Deploying daemon mds.c on smithi179 2024-05-09T09:06:18.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6834/502414567,v1:172.21.15.179:6835/502414567] up:boot 2024-05-09T09:06:18.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: fsmap 6 up:standby 2024-05-09T09:06:18.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:06:18.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:18 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1462039960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: Deploying daemon mds.c on smithi179 2024-05-09T09:06:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6834/502414567,v1:172.21.15.179:6835/502414567] up:boot 2024-05-09T09:06:19.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: fsmap 6 up:standby 2024-05-09T09:06:19.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:06:19.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:18 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1462039960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:19.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:19.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: Deploying daemon mds.c on smithi179 2024-05-09T09:06:19.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6834/502414567,v1:172.21.15.179:6835/502414567] up:boot 2024-05-09T09:06:19.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: fsmap 6 up:standby 2024-05-09T09:06:19.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:06:19.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:18 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1462039960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:20.229 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 systemd[1]: Starting Ceph mds.c for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:20.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.428+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/625353083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 msgr2=0x7f1c68101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.428+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/625353083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68101cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009900 tx=0x7f1c5802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/625353083 shutdown_connections 2024-05-09T09:06:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/625353083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c68103450 0x7f1c68109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/625353083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c68102a90 0x7f1c68102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/625353083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68101cc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/625353083 >> 172.21.15.64:0/625353083 conn(0x7f1c680fce30 msgr2=0x7f1c680ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/625353083 shutdown_connections 2024-05-09T09:06:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/625353083 wait complete. 2024-05-09T09:06:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 Processor -- start 2024-05-09T09:06:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.429+0000 7f1c6e9f9640 1 -- start start 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6e9f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6e9f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c68102a90 0x7f1c681980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6e9f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c68103450 0x7f1c6819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6e9f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810d420 con 0x7f1c681018a0 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6e9f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810d2a0 con 0x7f1c68102a90 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6e9f9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810d5a0 con 0x7f1c68103450 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54498/0 (socket says 172.21.15.64:54498) 2024-05-09T09:06:20.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 -- 172.21.15.64:0/4268866234 learned_addr learned my addr 172.21.15.64:0/4268866234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c677fe640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c68102a90 0x7f1c681980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c6cf96640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c68103450 0x7f1c6819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 -- 172.21.15.64:0/4268866234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c68103450 msgr2=0x7f1c6819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c68103450 0x7f1c6819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 -- 172.21.15.64:0/4268866234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c68102a90 msgr2=0x7f1c681980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c68102a90 0x7f1c681980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 -- 172.21.15.64:0/4268866234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6819f790 con 0x7f1c681018a0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.430+0000 7f1c67fff640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68197b70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c580314e0 tx=0x7f1c580042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.431+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5803f030 con 0x7f1c681018a0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.431+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6819fa20 con 0x7f1c681018a0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.431+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c58003cb0 con 0x7f1c681018a0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.431+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58035070 con 0x7f1c681018a0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.431+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6819ff10 con 0x7f1c681018a0 2024-05-09T09:06:20.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.432+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c68108ce0 con 0x7f1c681018a0 2024-05-09T09:06:20.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.433+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1c58042030 con 0x7f1c681018a0 2024-05-09T09:06:20.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.433+0000 7f1c657fa640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c500a2b90 0x7f1c500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:20.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.434+0000 7f1c677fe640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c500a2b90 0x7f1c500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:20.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.434+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1c580ef110 con 0x7f1c681018a0 2024-05-09T09:06:20.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.434+0000 7f1c677fe640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c500a2b90 0x7f1c500a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c68199080 tx=0x7f1c5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:20.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.438+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c580bbec0 con 0x7f1c681018a0 2024-05-09T09:06:20.515 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 podman[64366]: 2024-05-09 09:06:20.157167774 +0000 UTC m=+0.018131513 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:20.515 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 podman[64366]: 2024-05-09 09:06:20.280797905 +0000 UTC m=+0.141761669 container create 0fccc11519eb5f2b56d68b76bcca685cee0e464ae6212f84331449bd81faea2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T09:06:20.516 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 podman[64366]: 2024-05-09 09:06:20.517218582 +0000 UTC m=+0.378182307 container init 0fccc11519eb5f2b56d68b76bcca685cee0e464ae6212f84331449bd81faea2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T09:06:20.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.664+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c6802d1c0 con 0x7f1c681018a0 2024-05-09T09:06:20.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.664+0000 7f1c657fa640 1 -- 172.21.15.64:0/4268866234 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f1c580c01c0 con 0x7f1c681018a0 2024-05-09T09:06:20.664 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:20.664 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-09T09:06:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-05-09T09:06:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.666+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c500a2b90 msgr2=0x7f1c500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.666+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c500a2b90 0x7f1c500a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c68199080 tx=0x7f1c5c00b410 comp rx=0 tx=0).stop 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.666+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 msgr2=0x7f1c68197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.666+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68197b70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c580314e0 tx=0x7f1c580042e0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 shutdown_connections 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c500a2b90 0x7f1c500a5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c68103450 0x7f1c6819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c68102a90 0x7f1c681980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 --2- 172.21.15.64:0/4268866234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c681018a0 0x7f1c68197b70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 >> 172.21.15.64:0/4268866234 conn(0x7f1c680fce30 msgr2=0x7f1c68105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 shutdown_connections 2024-05-09T09:06:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:20.667+0000 7f1c6e9f9640 1 -- 172.21.15.64:0/4268866234 wait complete. 2024-05-09T09:06:20.822 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 podman[64366]: 2024-05-09 09:06:20.520083476 +0000 UTC m=+0.381047207 container start 0fccc11519eb5f2b56d68b76bcca685cee0e464ae6212f84331449bd81faea2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T09:06:20.822 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mds[64385]: -- 172.21.15.179:0/762831820 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c66ec5fa40 con 0x55c66ec32800 2024-05-09T09:06:20.822 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c[64381]: starting mds.c at 2024-05-09T09:06:20.823 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 bash[64366]: 0fccc11519eb5f2b56d68b76bcca685cee0e464ae6212f84331449bd81faea2f 2024-05-09T09:06:20.823 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:06:20 smithi179 systemd[1]: Started Ceph mds.c for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4268866234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4268866234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:21.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:21.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4268866234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:22.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:21 smithi139 ceph-mon[33599]: Deploying daemon mds.h on smithi139 2024-05-09T09:06:22.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:21 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] up:boot 2024-05-09T09:06:22.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:21 smithi139 ceph-mon[33599]: fsmap 7 up:standby 2024-05-09T09:06:22.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:06:22.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:21 smithi064 ceph-mon[30149]: Deploying daemon mds.h on smithi139 2024-05-09T09:06:22.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:21 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] up:boot 2024-05-09T09:06:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:21 smithi064 ceph-mon[30149]: fsmap 7 up:standby 2024-05-09T09:06:22.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:06:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:21 smithi179 ceph-mon[34086]: Deploying daemon mds.h on smithi139 2024-05-09T09:06:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:21 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] up:boot 2024-05-09T09:06:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:21 smithi179 ceph-mon[34086]: fsmap 7 up:standby 2024-05-09T09:06:22.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:06:22.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 -- 172.21.15.64:0/664439626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 msgr2=0x7f79e8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 --2- 172.21.15.64:0/664439626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8101de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009900 tx=0x7f79dc02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 -- 172.21.15.64:0/664439626 shutdown_connections 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 --2- 172.21.15.64:0/664439626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 0x7f79e8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 --2- 172.21.15.64:0/664439626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8101de0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 --2- 172.21.15.64:0/664439626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 0x7f79e8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 -- 172.21.15.64:0/664439626 >> 172.21.15.64:0/664439626 conn(0x7f79e80fbce0 msgr2=0x7f79e80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:22.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 -- 172.21.15.64:0/664439626 shutdown_connections 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.796+0000 7f79f06bd640 1 -- 172.21.15.64:0/664439626 wait complete. 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 Processor -- start 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 -- start start 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 0x7f79e8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 0x7f79e819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810d3c0 con 0x7f79e8101960 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810d240 con 0x7f79e8100770 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79f06bd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810d540 con 0x7f79e8102320 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79ee459640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 0x7f79e8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54514/0 (socket says 172.21.15.64:54514) 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 -- 172.21.15.64:0/2516135158 learned_addr learned my addr 172.21.15.64:0/2516135158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79eec5a640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 0x7f79e819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 -- 172.21.15.64:0/2516135158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 msgr2=0x7f79e819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 0x7f79e819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 -- 172.21.15.64:0/2516135158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 msgr2=0x7f79e8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 0x7f79e8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 -- 172.21.15.64:0/2516135158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e819f640 con 0x7f79e8101960 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79eec5a640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 0x7f79e819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79ee459640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 0x7f79e8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.797+0000 7f79edc58640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8197f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005940 tx=0x7f79dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.798+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc03f030 con 0x7f79e8101960 2024-05-09T09:06:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.798+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79dc003cb0 con 0x7f79e8101960 2024-05-09T09:06:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.798+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e819f8d0 con 0x7f79e8101960 2024-05-09T09:06:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.798+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc035070 con 0x7f79e8101960 2024-05-09T09:06:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.798+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e819fdc0 con 0x7f79e8101960 2024-05-09T09:06:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.799+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e8107b10 con 0x7f79e8101960 2024-05-09T09:06:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.800+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f79dc042030 con 0x7f79e8101960 2024-05-09T09:06:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.801+0000 7f79e77fe640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79d00a2b90 0x7f79d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.801+0000 7f79ee459640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79d00a2b90 0x7f79d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.801+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f79dc0ef1d0 con 0x7f79e8101960 2024-05-09T09:06:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.801+0000 7f79ee459640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79d00a2b90 0x7f79d00a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f79e81017c0 tx=0x7f79e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:22.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:22.806+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79dc0bc5d0 con 0x7f79e8101960 2024-05-09T09:06:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.029+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e802d3e0 con 0x7f79e8101960 2024-05-09T09:06:23.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.030+0000 7f79e77fe640 1 -- 172.21.15.64:0/2516135158 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f79dc0bbed0 con 0x7f79e8101960 2024-05-09T09:06:23.029 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:23.029 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T09:06:23.030 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-05-09T09:06:23.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79d00a2b90 msgr2=0x7f79d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:23.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79d00a2b90 0x7f79d00a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f79e81017c0 tx=0x7f79e0005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 msgr2=0x7f79e8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8197f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005940 tx=0x7f79dc0042f0 comp rx=0 tx=0).stop 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 shutdown_connections 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79d00a2b90 0x7f79d00a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e8102320 0x7f79e819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79e8101960 0x7f79e8197f60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 --2- 172.21.15.64:0/2516135158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e8100770 0x7f79e8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 >> 172.21.15.64:0/2516135158 conn(0x7f79e80fbce0 msgr2=0x7f79e8104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 shutdown_connections 2024-05-09T09:06:23.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:23.032+0000 7f79f06bd640 1 -- 172.21.15.64:0/2516135158 wait complete. 2024-05-09T09:06:23.041 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:22 smithi064 ceph-mon[30149]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:23.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:22 smithi139 ceph-mon[33599]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:23.073 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:22 smithi139 systemd[1]: Starting Ceph mds.h for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:23.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:22 smithi179 ceph-mon[34086]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:23.470 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 podman[63958]: 2024-05-09 09:06:23.084793278 +0000 UTC m=+0.018234981 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:23.470 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 podman[63958]: 2024-05-09 09:06:23.210899721 +0000 UTC m=+0.144341414 container create 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T09:06:23.470 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 podman[63958]: 2024-05-09 09:06:23.471761616 +0000 UTC m=+0.405203308 container init 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T09:06:23.823 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 podman[63958]: 2024-05-09 09:06:23.475007776 +0000 UTC m=+0.408449480 container start 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T09:06:23.823 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mds[63977]: -- 172.21.15.139:0/2755151555 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd26031a40 con 0x55fd26004c00 2024-05-09T09:06:23.823 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h[63973]: starting mds.h at 2024-05-09T09:06:23.823 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 bash[63958]: 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 2024-05-09T09:06:23.823 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:06:23 smithi139 systemd[1]: Started Ceph mds.h for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:24.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2516135158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:24.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:24.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:23 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2516135158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:24.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:23 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:24.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2516135158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:24.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:24.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:23 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:24.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:24 smithi179 ceph-mon[34086]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:24 smithi179 ceph-mon[34086]: Deploying daemon mds.i on smithi179 2024-05-09T09:06:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:24 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6836/2410946724,v1:172.21.15.139:6837/2410946724] up:boot 2024-05-09T09:06:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:24 smithi179 ceph-mon[34086]: fsmap 8 up:standby 2024-05-09T09:06:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:06:25.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:24 smithi064 ceph-mon[30149]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:24 smithi064 ceph-mon[30149]: Deploying daemon mds.i on smithi179 2024-05-09T09:06:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:24 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6836/2410946724,v1:172.21.15.139:6837/2410946724] up:boot 2024-05-09T09:06:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:24 smithi064 ceph-mon[30149]: fsmap 8 up:standby 2024-05-09T09:06:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 -- 172.21.15.64:0/365059150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 msgr2=0x7f2d54100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 --2- 172.21.15.64:0/365059150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54100520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009900 tx=0x7f2d5002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 -- 172.21.15.64:0/365059150 shutdown_connections 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 --2- 172.21.15.64:0/365059150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54101400 0x7f2d54101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 --2- 172.21.15.64:0/365059150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d54100a60 0x7f2d54100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 --2- 172.21.15.64:0/365059150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54100520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 -- 172.21.15.64:0/365059150 >> 172.21.15.64:0/365059150 conn(0x7f2d540fbd20 msgr2=0x7f2d540fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 -- 172.21.15.64:0/365059150 shutdown_connections 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.163+0000 7f2d5b529640 1 -- 172.21.15.64:0/365059150 wait complete. 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 Processor -- start 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 -- start start 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:25.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54100a60 0x7f2d5419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d54101400 0x7f2d541a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410f680 con 0x7f2d54100100 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410f500 con 0x7f2d54100a60 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d5b529640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5410f800 con 0x7f2d54101400 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d592c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d592c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54214/0 (socket says 172.21.15.64:54214) 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d592c5640 1 -- 172.21.15.64:0/1789123994 learned_addr learned my addr 172.21.15.64:0/1789123994 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d59ac6640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d54101400 0x7f2d541a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d592c5640 1 -- 172.21.15.64:0/1789123994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d54101400 msgr2=0x7f2d541a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d592c5640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d54101400 0x7f2d541a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d58ac4640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54100a60 0x7f2d5419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.164+0000 7f2d592c5640 1 -- 172.21.15.64:0/1789123994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54100a60 msgr2=0x7f2d5419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d592c5640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54100a60 0x7f2d5419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d592c5640 1 -- 172.21.15.64:0/1789123994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d541a1a60 con 0x7f2d54100100 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d58ac4640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54100a60 0x7f2d5419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d592c5640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54199e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d50005940 tx=0x7f2d50004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5003f030 con 0x7f2d54100100 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d50003cb0 con 0x7f2d54100100 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d541a1cf0 con 0x7f2d54100100 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50035070 con 0x7f2d54100100 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.165+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d541a2190 con 0x7f2d54100100 2024-05-09T09:06:25.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.166+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5410ae50 con 0x7f2d54100100 2024-05-09T09:06:25.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.167+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2d50042030 con 0x7f2d54100100 2024-05-09T09:06:25.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.168+0000 7f2d4a7fc640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d400a2b90 0x7f2d400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:25.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.168+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2d500ef700 con 0x7f2d54100100 2024-05-09T09:06:25.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.168+0000 7f2d58ac4640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d400a2b90 0x7f2d400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:25.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.168+0000 7f2d58ac4640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d400a2b90 0x7f2d400a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d44006fd0 tx=0x7f2d44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:25.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.173+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d500bcb00 con 0x7f2d54100100 2024-05-09T09:06:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:24 smithi139 ceph-mon[33599]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:24 smithi139 ceph-mon[33599]: Deploying daemon mds.i on smithi179 2024-05-09T09:06:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:24 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6836/2410946724,v1:172.21.15.139:6837/2410946724] up:boot 2024-05-09T09:06:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:24 smithi139 ceph-mon[33599]: fsmap 8 up:standby 2024-05-09T09:06:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:06:25.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.400+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d5402d410 con 0x7f2d54100100 2024-05-09T09:06:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.401+0000 7f2d4a7fc640 1 -- 172.21.15.64:0/1789123994 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f2d500bc400 con 0x7f2d54100100 2024-05-09T09:06:25.400 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:25.400 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T09:06:25.401 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 9 2024-05-09T09:06:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d400a2b90 msgr2=0x7f2d400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d400a2b90 0x7f2d400a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d44006fd0 tx=0x7f2d44008040 comp rx=0 tx=0).stop 2024-05-09T09:06:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 msgr2=0x7f2d54199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54199e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d50005940 tx=0x7f2d50004430 comp rx=0 tx=0).stop 2024-05-09T09:06:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 shutdown_connections 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d400a2b90 0x7f2d400a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d54101400 0x7f2d541a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d54100a60 0x7f2d5419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 --2- 172.21.15.64:0/1789123994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d54100100 0x7f2d54199e40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 >> 172.21.15.64:0/1789123994 conn(0x7f2d540fbd20 msgr2=0x7f2d540ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 shutdown_connections 2024-05-09T09:06:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:25.403+0000 7f2d5b529640 1 -- 172.21.15.64:0/1789123994 wait complete. 2024-05-09T09:06:25.849 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:25 smithi179 systemd[1]: Starting Ceph mds.i for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:26.107 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:25 smithi179 podman[64671]: 2024-05-09 09:06:25.976622955 +0000 UTC m=+0.102143928 container create d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T09:06:26.107 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:25 smithi179 podman[64671]: 2024-05-09 09:06:25.89257345 +0000 UTC m=+0.018094451 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:26.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:25 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1789123994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:26.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:25 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1789123994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:26.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:25 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1789123994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:26.479 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:26 smithi179 podman[64671]: 2024-05-09 09:06:26.204337822 +0000 UTC m=+0.329858817 container init d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T09:06:26.479 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:26 smithi179 podman[64671]: 2024-05-09 09:06:26.207122403 +0000 UTC m=+0.332643378 container start d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T09:06:26.479 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mds[64690]: -- 172.21.15.179:0/1511398492 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9145d1a40 con 0x55d9145a5000 2024-05-09T09:06:26.479 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i[64686]: starting mds.i at 2024-05-09T09:06:26.479 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:26 smithi179 bash[64671]: d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 2024-05-09T09:06:26.479 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:06:26 smithi179 systemd[1]: Started Ceph mds.i for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:27.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:27.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 -- 172.21.15.64:0/2052025107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 msgr2=0x7f613c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 --2- 172.21.15.64:0/2052025107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c100bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f612c009900 tx=0x7f612c02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 -- 172.21.15.64:0/2052025107 shutdown_connections 2024-05-09T09:06:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 --2- 172.21.15.64:0/2052025107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c102340 0x7f613c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 --2- 172.21.15.64:0/2052025107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c101980 0x7f613c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 --2- 172.21.15.64:0/2052025107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c100bb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 -- 172.21.15.64:0/2052025107 >> 172.21.15.64:0/2052025107 conn(0x7f613c0fbd20 msgr2=0x7f613c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 -- 172.21.15.64:0/2052025107 shutdown_connections 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.524+0000 7f6141d9e640 1 -- 172.21.15.64:0/2052025107 wait complete. 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 Processor -- start 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 -- start start 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c101980 0x7f613c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c102340 0x7f613c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f613c10c390 con 0x7f613c100790 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f613c10c210 con 0x7f613c102340 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f6141d9e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f613c10c510 con 0x7f613c101980 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f613b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f613b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54222/0 (socket says 172.21.15.64:54222) 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.525+0000 7f613b7fe640 1 -- 172.21.15.64:0/3421801733 learned_addr learned my addr 172.21.15.64:0/3421801733 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613affd640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c101980 0x7f613c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613bfff640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c102340 0x7f613c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613b7fe640 1 -- 172.21.15.64:0/3421801733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c101980 msgr2=0x7f613c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613b7fe640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c101980 0x7f613c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613b7fe640 1 -- 172.21.15.64:0/3421801733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c102340 msgr2=0x7f613c19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613b7fe640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c102340 0x7f613c19f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613b7fe640 1 -- 172.21.15.64:0/3421801733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c19f8b0 con 0x7f613c100790 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f613b7fe640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c197c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f612c009230 tx=0x7f612c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c03f030 con 0x7f613c100790 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f612c003cb0 con 0x7f613c100790 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c19fb40 con 0x7f613c100790 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c035070 con 0x7f613c100790 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.526+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1a0030 con 0x7f613c100790 2024-05-09T09:06:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.527+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613c1a0200 con 0x7f613c100790 2024-05-09T09:06:27.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.529+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f612c042030 con 0x7f613c100790 2024-05-09T09:06:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.529+0000 7f6138ff9640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61240a2b90 0x7f61240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.529+0000 7f613affd640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61240a2b90 0x7f61240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.529+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f612c0ef6a0 con 0x7f613c100790 2024-05-09T09:06:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.530+0000 7f613affd640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61240a2b90 0x7f61240a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f613c199170 tx=0x7f6130008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:27.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.532+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f612c0bcaa0 con 0x7f613c100790 2024-05-09T09:06:27.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.757+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f613c1a03d0 con 0x7f613c100790 2024-05-09T09:06:27.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.757+0000 7f6138ff9640 1 -- 172.21.15.64:0/3421801733 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f612c0bc3a0 con 0x7f613c100790 2024-05-09T09:06:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:27.756 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:06:27.757 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 10 2024-05-09T09:06:27.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61240a2b90 msgr2=0x7f61240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:27.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61240a2b90 0x7f61240a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f613c199170 tx=0x7f6130008040 comp rx=0 tx=0).stop 2024-05-09T09:06:27.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 msgr2=0x7f613c197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:27.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c197c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f612c009230 tx=0x7f612c004340 comp rx=0 tx=0).stop 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 shutdown_connections 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61240a2b90 0x7f61240a5050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c102340 0x7f613c19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c101980 0x7f613c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 --2- 172.21.15.64:0/3421801733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c100790 0x7f613c197c70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 >> 172.21.15.64:0/3421801733 conn(0x7f613c0fbd20 msgr2=0x7f613c069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 shutdown_connections 2024-05-09T09:06:27.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:27.759+0000 7f6141d9e640 1 -- 172.21.15.64:0/3421801733 wait complete. 2024-05-09T09:06:28.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:27 smithi064 ceph-mon[30149]: Deploying daemon mds.g on smithi064 2024-05-09T09:06:28.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:27 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] up:boot 2024-05-09T09:06:28.043 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:27 smithi064 ceph-mon[30149]: fsmap 9 up:standby 2024-05-09T09:06:28.043 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:06:28.043 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:27 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3421801733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:27 smithi179 ceph-mon[34086]: Deploying daemon mds.g on smithi064 2024-05-09T09:06:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:27 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] up:boot 2024-05-09T09:06:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:27 smithi179 ceph-mon[34086]: fsmap 9 up:standby 2024-05-09T09:06:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:06:28.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:27 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3421801733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:28.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:27 smithi139 ceph-mon[33599]: Deploying daemon mds.g on smithi064 2024-05-09T09:06:28.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:27 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] up:boot 2024-05-09T09:06:28.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:27 smithi139 ceph-mon[33599]: fsmap 9 up:standby 2024-05-09T09:06:28.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:06:28.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:27 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3421801733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:28.653 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:28 smithi064 systemd[1]: Starting Ceph mds.g for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:28 smithi064 ceph-mon[30149]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:29.012 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:28 smithi064 podman[69773]: 2024-05-09 09:06:28.641908651 +0000 UTC m=+0.018475523 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:29.012 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:28 smithi064 podman[69773]: 2024-05-09 09:06:28.762785998 +0000 UTC m=+0.139352917 container create 05109ada3bef95caa97e64f2479b6f01c7c2adb4889cc0fce2f5d811af5ddf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:06:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:28 smithi179 ceph-mon[34086]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:29.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:28 smithi139 ceph-mon[33599]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:29.402 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:29 smithi064 podman[69773]: 2024-05-09 09:06:29.149517864 +0000 UTC m=+0.526084727 container init 05109ada3bef95caa97e64f2479b6f01c7c2adb4889cc0fce2f5d811af5ddf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T09:06:29.403 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:29 smithi064 podman[69773]: 2024-05-09 09:06:29.15272678 +0000 UTC m=+0.529293634 container start 05109ada3bef95caa97e64f2479b6f01c7c2adb4889cc0fce2f5d811af5ddf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.vendor=CentOS) 2024-05-09T09:06:29.403 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:29 smithi064 ceph-mds[69793]: -- 172.21.15.64:0/3600380299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ad8035a40 con 0x561ad8009000 2024-05-09T09:06:29.403 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:29 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g[69789]: starting mds.g at 2024-05-09T09:06:29.403 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:29 smithi064 bash[69773]: 05109ada3bef95caa97e64f2479b6f01c7c2adb4889cc0fce2f5d811af5ddf31 2024-05-09T09:06:29.403 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:06:29 smithi064 systemd[1]: Started Ceph mds.g for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:29.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 -- 172.21.15.64:0/4240509473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838104c20 msgr2=0x7fa838107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 --2- 172.21.15.64:0/4240509473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838104c20 0x7fa838107010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa828009900 tx=0x7fa82802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 -- 172.21.15.64:0/4240509473 shutdown_connections 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 --2- 172.21.15.64:0/4240509473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa838107550 0x7fa8381099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 --2- 172.21.15.64:0/4240509473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838104c20 0x7fa838107010 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 --2- 172.21.15.64:0/4240509473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8381022f0 0x7fa8381046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 -- 172.21.15.64:0/4240509473 >> 172.21.15.64:0/4240509473 conn(0x7fa8380fbce0 msgr2=0x7fa8380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 -- 172.21.15.64:0/4240509473 shutdown_connections 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 -- 172.21.15.64:0/4240509473 wait complete. 2024-05-09T09:06:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 Processor -- start 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.890+0000 7fa840570640 1 -- start start 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa840570640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8381022f0 0x7fa838101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa840570640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa838104c20 0x7fa8381002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa840570640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 0x7fa8381007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa840570640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d3b0 con 0x7fa838107550 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa840570640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d230 con 0x7fa8381022f0 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa840570640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d530 con 0x7fa838104c20 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 0x7fa8381007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 0x7fa8381007e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54250/0 (socket says 172.21.15.64:54250) 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 -- 172.21.15.64:0/3051516128 learned_addr learned my addr 172.21.15.64:0/3051516128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83db0b640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa838104c20 0x7fa8381002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83e30c640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8381022f0 0x7fa838101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 -- 172.21.15.64:0/3051516128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa838104c20 msgr2=0x7fa8381002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa838104c20 0x7fa8381002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 -- 172.21.15.64:0/3051516128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8381022f0 msgr2=0x7fa838101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8381022f0 0x7fa838101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 -- 172.21.15.64:0/3051516128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8381010a0 con 0x7fa838107550 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.891+0000 7fa83eb0d640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 0x7fa8381007e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa83000eb10 tx=0x7fa83000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.892+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004b20 con 0x7fa838107550 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.892+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8381a1c70 con 0x7fa838107550 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.892+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa830004cc0 con 0x7fa838107550 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.892+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830010690 con 0x7fa838107550 2024-05-09T09:06:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.892+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8381a2190 con 0x7fa838107550 2024-05-09T09:06:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.893+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8380fd7d0 con 0x7fa838107550 2024-05-09T09:06:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.894+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa830005000 con 0x7fa838107550 2024-05-09T09:06:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.895+0000 7fa82f7fe640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8240a2b90 0x7fa8240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.895+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa8300cf300 con 0x7fa838107550 2024-05-09T09:06:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.895+0000 7fa83e30c640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8240a2b90 0x7fa8240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.895+0000 7fa83e30c640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8240a2b90 0x7fa8240a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa834009bd0 tx=0x7fa834005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:29.898+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa830053740 con 0x7fa838107550 2024-05-09T09:06:30.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.132+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa83802d3c0 con 0x7fa838107550 2024-05-09T09:06:30.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.133+0000 7fa82f7fe640 1 -- 172.21.15.64:0/3051516128 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fa83009c700 con 0x7fa838107550 2024-05-09T09:06:30.132 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:30.132 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:06:30.133 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-05-09T09:06:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8240a2b90 msgr2=0x7fa8240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8240a2b90 0x7fa8240a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa834009bd0 tx=0x7fa834005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 msgr2=0x7fa8381007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 0x7fa8381007e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa83000eb10 tx=0x7fa83000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:06:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 shutdown_connections 2024-05-09T09:06:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8240a2b90 0x7fa8240a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:30.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa838107550 0x7fa8381007e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:30.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa838104c20 0x7fa8381002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:30.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 --2- 172.21.15.64:0/3051516128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8381022f0 0x7fa838101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:30.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 >> 172.21.15.64:0/3051516128 conn(0x7fa8380fbce0 msgr2=0x7fa838102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:30.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 shutdown_connections 2024-05-09T09:06:30.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:30.135+0000 7fa840570640 1 -- 172.21.15.64:0/3051516128 wait complete. 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: Deploying daemon mds.a on smithi064 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.64:6838/437565752,v1:172.21.15.64:6839/437565752] up:boot 2024-05-09T09:06:30.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: fsmap 10 up:standby 2024-05-09T09:06:30.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:06:30.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:30 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3051516128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:30.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: Deploying daemon mds.a on smithi064 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.64:6838/437565752,v1:172.21.15.64:6839/437565752] up:boot 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: fsmap 10 up:standby 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:06:30.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:30 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3051516128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:30.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: Deploying daemon mds.a on smithi064 2024-05-09T09:06:30.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.64:6838/437565752,v1:172.21.15.64:6839/437565752] up:boot 2024-05-09T09:06:30.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: fsmap 10 up:standby 2024-05-09T09:06:30.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T09:06:30.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:30 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3051516128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:31.652 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:31 smithi064 systemd[1]: Starting Ceph mds.a for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:32.030 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:31 smithi064 podman[70134]: 2024-05-09 09:06:31.662185119 +0000 UTC m=+0.018750510 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:32.030 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:31 smithi064 podman[70134]: 2024-05-09 09:06:31.787418412 +0000 UTC m=+0.143983840 container create 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:06:32.030 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:32 smithi064 podman[70134]: 2024-05-09 09:06:32.031838189 +0000 UTC m=+0.388403589 container init 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0) 2024-05-09T09:06:32.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:32.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 -- 172.21.15.64:0/3001740485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c107570 msgr2=0x7fca3c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 --2- 172.21.15.64:0/3001740485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c107570 0x7fca3c1099c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca34009ca0 tx=0x7fca3402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 -- 172.21.15.64:0/3001740485 shutdown_connections 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 --2- 172.21.15.64:0/3001740485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c107570 0x7fca3c1099c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 --2- 172.21.15.64:0/3001740485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c104c40 0x7fca3c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 --2- 172.21.15.64:0/3001740485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca3c102310 0x7fca3c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 -- 172.21.15.64:0/3001740485 >> 172.21.15.64:0/3001740485 conn(0x7fca3c0fbd20 msgr2=0x7fca3c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 -- 172.21.15.64:0/3001740485 shutdown_connections 2024-05-09T09:06:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.262+0000 7fca40fee640 1 -- 172.21.15.64:0/3001740485 wait complete. 2024-05-09T09:06:32.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 Processor -- start 2024-05-09T09:06:32.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 -- start start 2024-05-09T09:06:32.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca3c102310 0x7fca3c101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 0x7fca3c1002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c107570 0x7fca3c100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c10d430 con 0x7fca3c104c40 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c10d2b0 con 0x7fca3c107570 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca40fee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c10d5b0 con 0x7fca3c102310 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca39d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 0x7fca3c1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca39d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 0x7fca3c1002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54280/0 (socket says 172.21.15.64:54280) 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.263+0000 7fca39d9b640 1 -- 172.21.15.64:0/4252491087 learned_addr learned my addr 172.21.15.64:0/4252491087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca3ad9d640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c107570 0x7fca3c100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca3a59c640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca3c102310 0x7fca3c101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca39d9b640 1 -- 172.21.15.64:0/4252491087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca3c102310 msgr2=0x7fca3c101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca39d9b640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca3c102310 0x7fca3c101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca39d9b640 1 -- 172.21.15.64:0/4252491087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c107570 msgr2=0x7fca3c100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca39d9b640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c107570 0x7fca3c100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca39d9b640 1 -- 172.21.15.64:0/4252491087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c101090 con 0x7fca3c104c40 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca39d9b640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 0x7fca3c1002c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e9c0 tx=0x7fca2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00cd80 con 0x7fca3c104c40 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca2c0054d0 con 0x7fca3c104c40 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c1a1bd0 con 0x7fca3c104c40 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c010430 con 0x7fca3c104c40 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.264+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c1a2120 con 0x7fca3c104c40 2024-05-09T09:06:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.265+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3c0fd810 con 0x7fca3c104c40 2024-05-09T09:06:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.266+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca2c004fd0 con 0x7fca3c104c40 2024-05-09T09:06:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.267+0000 7fca337fe640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca280a2b90 0x7fca280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.267+0000 7fca3a59c640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca280a2b90 0x7fca280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.267+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fca2c0cf0b0 con 0x7fca3c104c40 2024-05-09T09:06:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.268+0000 7fca3a59c640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca280a2b90 0x7fca280a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca24005fd0 tx=0x7fca24005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:32.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.272+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fca2c09c4b0 con 0x7fca3c104c40 2024-05-09T09:06:32.311 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:32 smithi064 podman[70134]: 2024-05-09 09:06:32.034702648 +0000 UTC m=+0.391268038 container start 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:06:32.312 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mds[70153]: -- 172.21.15.64:0/2612003147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff99715a40 con 0x55ff996e8c00 2024-05-09T09:06:32.312 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a[70149]: starting mds.a at 2024-05-09T09:06:32.312 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:32 smithi064 bash[70134]: 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d 2024-05-09T09:06:32.312 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:06:32 smithi064 systemd[1]: Started Ceph mds.a for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.498+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca3c02d3e0 con 0x7fca3c104c40 2024-05-09T09:06:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.499+0000 7fca337fe640 1 -- 172.21.15.64:0/4252491087 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fca2c09bdb0 con 0x7fca3c104c40 2024-05-09T09:06:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:06:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-05-09T09:06:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca280a2b90 msgr2=0x7fca280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca280a2b90 0x7fca280a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca24005fd0 tx=0x7fca24005850 comp rx=0 tx=0).stop 2024-05-09T09:06:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 msgr2=0x7fca3c1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 0x7fca3c1002c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e9c0 tx=0x7fca2c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 shutdown_connections 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca280a2b90 0x7fca280a5050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca3c107570 0x7fca3c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca3c104c40 0x7fca3c1002c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 --2- 172.21.15.64:0/4252491087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca3c102310 0x7fca3c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 >> 172.21.15.64:0/4252491087 conn(0x7fca3c0fbd20 msgr2=0x7fca3c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 shutdown_connections 2024-05-09T09:06:32.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:32.501+0000 7fca40fee640 1 -- 172.21.15.64:0/4252491087 wait complete. 2024-05-09T09:06:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T09:06:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:33.549 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:33 smithi139 ceph-mon[33599]: Deploying daemon mds.b on smithi139 2024-05-09T09:06:33.549 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:33 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.64:6840/4047794572,v1:172.21.15.64:6841/4047794572] up:boot 2024-05-09T09:06:33.549 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:33 smithi139 ceph-mon[33599]: fsmap 11 up:standby 2024-05-09T09:06:33.549 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:06:33.549 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:33 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4252491087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:33.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:33 smithi064 ceph-mon[30149]: Deploying daemon mds.b on smithi139 2024-05-09T09:06:33.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:33 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.64:6840/4047794572,v1:172.21.15.64:6841/4047794572] up:boot 2024-05-09T09:06:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:33 smithi064 ceph-mon[30149]: fsmap 11 up:standby 2024-05-09T09:06:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:06:33.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:33 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4252491087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:33.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:33 smithi179 ceph-mon[34086]: Deploying daemon mds.b on smithi139 2024-05-09T09:06:33.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:33 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.64:6840/4047794572,v1:172.21.15.64:6841/4047794572] up:boot 2024-05-09T09:06:33.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:33 smithi179 ceph-mon[34086]: fsmap 11 up:standby 2024-05-09T09:06:33.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T09:06:33.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:33 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4252491087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:34.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:34 smithi139 ceph-mon[33599]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:34.573 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:34 smithi139 systemd[1]: Starting Ceph mds.b for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:06:34.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 -- 172.21.15.64:0/3882282085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9064102160 msgr2=0x7f90641089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 --2- 172.21.15.64:0/3882282085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9064102160 0x7f90641089f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f905c009ca0 tx=0x7f905c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 -- 172.21.15.64:0/3882282085 shutdown_connections 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 --2- 172.21.15.64:0/3882282085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9064102160 0x7f90641089f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 --2- 172.21.15.64:0/3882282085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90641017a0 0x7f9064101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 --2- 172.21.15.64:0/3882282085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90641005b0 0x7f90641009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 -- 172.21.15.64:0/3882282085 >> 172.21.15.64:0/3882282085 conn(0x7f90640fbb00 msgr2=0x7f90640fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 -- 172.21.15.64:0/3882282085 shutdown_connections 2024-05-09T09:06:34.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.632+0000 7f906a59e640 1 -- 172.21.15.64:0/3882282085 wait complete. 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 Processor -- start 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 -- start start 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90641005b0 0x7f90641044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 0x7f9064102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9064102160 0x7f90641030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f906410c0e0 con 0x7f90641017a0 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f906410bf60 con 0x7f9064102160 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.633+0000 7f906a59e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f906410c260 con 0x7f90641005b0 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 0x7f9064102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 0x7f9064102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40018/0 (socket says 172.21.15.64:40018) 2024-05-09T09:06:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 -- 172.21.15.64:0/1430243630 learned_addr learned my addr 172.21.15.64:0/1430243630 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9069d9d640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9064102160 0x7f90641030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f906959c640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90641005b0 0x7f90641044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 -- 172.21.15.64:0/1430243630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90641005b0 msgr2=0x7f90641044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90641005b0 0x7f90641044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 -- 172.21.15.64:0/1430243630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9064102160 msgr2=0x7f90641030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9064102160 0x7f90641030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 -- 172.21.15.64:0/1430243630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9064103980 con 0x7f90641017a0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f9068d9b640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 0x7f9064102b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f905400d770 tx=0x7f905400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054017070 con 0x7f90641017a0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90540054d0 con 0x7f90641017a0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90641a1a20 con 0x7f90641017a0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054004cf0 con 0x7f90641017a0 2024-05-09T09:06:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.634+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641a1ef0 con 0x7f90641017a0 2024-05-09T09:06:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.635+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90641a2120 con 0x7f90641017a0 2024-05-09T09:06:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.637+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9054004e90 con 0x7f90641017a0 2024-05-09T09:06:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.637+0000 7f905a7fc640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90500a2b90 0x7f90500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:34.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.638+0000 7f906959c640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90500a2b90 0x7f90500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:34.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.638+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f90540ce8d0 con 0x7f90641017a0 2024-05-09T09:06:34.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.638+0000 7f906959c640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90500a2b90 0x7f90500a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9064101600 tx=0x7f9060005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.641+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f905409b680 con 0x7f90641017a0 2024-05-09T09:06:34.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:34 smithi064 ceph-mon[30149]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:34.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:34 smithi179 ceph-mon[34086]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.864+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90641a22f0 con 0x7f90641017a0 2024-05-09T09:06:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.865+0000 7f905a7fc640 1 -- 172.21.15.64:0/1430243630 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f905409f980 con 0x7f90641017a0 2024-05-09T09:06:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:34.864 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T09:06:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-05-09T09:06:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90500a2b90 msgr2=0x7f90500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90500a2b90 0x7f90500a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9064101600 tx=0x7f9060005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 msgr2=0x7f9064102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 0x7f9064102b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f905400d770 tx=0x7f905400dc30 comp rx=0 tx=0).stop 2024-05-09T09:06:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 shutdown_connections 2024-05-09T09:06:34.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90500a2b90 0x7f90500a5050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9064102160 0x7f90641030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90641017a0 0x7f9064102b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 --2- 172.21.15.64:0/1430243630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90641005b0 0x7f90641044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:34.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 >> 172.21.15.64:0/1430243630 conn(0x7f90640fbb00 msgr2=0x7f90640fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:34.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 shutdown_connections 2024-05-09T09:06:34.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:34.867+0000 7f906a59e640 1 -- 172.21.15.64:0/1430243630 wait complete. 2024-05-09T09:06:35.040 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:34 smithi139 podman[64264]: 2024-05-09 09:06:34.639932396 +0000 UTC m=+0.018311993 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:06:35.040 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:34 smithi139 podman[64264]: 2024-05-09 09:06:34.762737406 +0000 UTC m=+0.141116989 container create 7b9d727b4326ef2679654afe0b266c522bad469bbb18b075214a76d5394ba3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T09:06:35.322 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:35 smithi139 podman[64264]: 2024-05-09 09:06:35.041968431 +0000 UTC m=+0.420348016 container init 7b9d727b4326ef2679654afe0b266c522bad469bbb18b075214a76d5394ba3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-96805dc) 2024-05-09T09:06:35.322 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:35 smithi139 podman[64264]: 2024-05-09 09:06:35.044752401 +0000 UTC m=+0.423131985 container start 7b9d727b4326ef2679654afe0b266c522bad469bbb18b075214a76d5394ba3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T09:06:35.323 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mds[64284]: -- 172.21.15.139:0/1562219949 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5a6a69a40 con 0x55f5a6a3c800 2024-05-09T09:06:35.323 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: starting mds.b at 2024-05-09T09:06:35.323 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:35 smithi139 bash[64264]: 7b9d727b4326ef2679654afe0b266c522bad469bbb18b075214a76d5394ba3ae 2024-05-09T09:06:35.323 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:35 smithi139 systemd[1]: Started Ceph mds.b for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:06:35.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:35 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1430243630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:35.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:35 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1430243630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1430243630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:35.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:35.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:36 smithi179 ceph-mon[34086]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:36 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] up:boot 2024-05-09T09:06:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:36 smithi179 ceph-mon[34086]: fsmap 12 up:standby 2024-05-09T09:06:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:06:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:36.526 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:36 smithi064 ceph-mon[30149]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:36.526 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:36 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] up:boot 2024-05-09T09:06:36.526 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:36 smithi064 ceph-mon[30149]: fsmap 12 up:standby 2024-05-09T09:06:36.526 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:06:36.526 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:36 smithi139 ceph-mon[33599]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:36 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] up:boot 2024-05-09T09:06:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:36 smithi139 ceph-mon[33599]: fsmap 12 up:standby 2024-05-09T09:06:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:06:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:36.878 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 -- 172.21.15.64:0/2055014294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8102330 msgr2=0x7f22e8108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 --2- 172.21.15.64:0/2055014294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8102330 0x7f22e8108bc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009ca0 tx=0x7f22e002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 -- 172.21.15.64:0/2055014294 shutdown_connections 2024-05-09T09:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 --2- 172.21.15.64:0/2055014294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8102330 0x7f22e8108bc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 --2- 172.21.15.64:0/2055014294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e8101970 0x7f22e8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 --2- 172.21.15.64:0/2055014294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22e8100780 0x7f22e8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 -- 172.21.15.64:0/2055014294 >> 172.21.15.64:0/2055014294 conn(0x7f22e80fbd10 msgr2=0x7f22e80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 -- 172.21.15.64:0/2055014294 shutdown_connections 2024-05-09T09:06:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.997+0000 7f22ecd89640 1 -- 172.21.15.64:0/2055014294 wait complete. 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.998+0000 7f22ecd89640 1 Processor -- start 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.998+0000 7f22ecd89640 1 -- start start 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.998+0000 7f22ecd89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 0x7f22e81047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.998+0000 7f22ecd89640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22e8101970 0x7f22e8102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.998+0000 7f22ecd89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e8102330 0x7f22e8103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.998+0000 7f22ecd89640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22e810c2b0 con 0x7f22e8100780 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22ecd89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22e810c130 con 0x7f22e8102330 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22ecd89640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f22e810c430 con 0x7f22e8101970 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 0x7f22e81047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 0x7f22e81047b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40036/0 (socket says 172.21.15.64:40036) 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 -- 172.21.15.64:0/857251655 learned_addr learned my addr 172.21.15.64:0/857251655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e5d9b640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22e8101970 0x7f22e8102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e6d9d640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e8102330 0x7f22e8103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 -- 172.21.15.64:0/857251655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22e8101970 msgr2=0x7f22e8102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22e8101970 0x7f22e8102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 -- 172.21.15.64:0/857251655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e8102330 msgr2=0x7f22e8103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e8102330 0x7f22e8103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 -- 172.21.15.64:0/857251655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e8103c40 con 0x7f22e8100780 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22e659c640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 0x7f22e81047b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22d800e9c0 tx=0x7f22d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d800cd80 con 0x7f22e8100780 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e81a1bc0 con 0x7f22e8100780 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22d80035f0 con 0x7f22e8100780 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8010690 con 0x7f22e8100780 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:36.999+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e81a2090 con 0x7f22e8100780 2024-05-09T09:06:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.001+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e81a2260 con 0x7f22e8100780 2024-05-09T09:06:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.002+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f22d80037a0 con 0x7f22e8100780 2024-05-09T09:06:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.003+0000 7f22d77fe640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22d00a2b90 0x7f22d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.003+0000 7f22e5d9b640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22d00a2b90 0x7f22d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.003+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f22d80cf0d0 con 0x7f22e8100780 2024-05-09T09:06:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.003+0000 7f22e5d9b640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22d00a2b90 0x7f22d00a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f22e8104520 tx=0x7f22dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.007+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f22d809c4d0 con 0x7f22e8100780 2024-05-09T09:06:37.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.281+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22e81a2550 con 0x7f22e8100780 2024-05-09T09:06:37.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.281+0000 7f22d77fe640 1 -- 172.21.15.64:0/857251655 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7f22d809bdd0 con 0x7f22e8100780 2024-05-09T09:06:37.280 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:37.280 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-09T09:06:37.281 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-05-09T09:06:37.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22d00a2b90 msgr2=0x7f22d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22d00a2b90 0x7f22d00a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f22e8104520 tx=0x7f22dc005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:37.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 msgr2=0x7f22e81047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 0x7f22e81047b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22d800e9c0 tx=0x7f22d800ee80 comp rx=0 tx=0).stop 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 shutdown_connections 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22d00a2b90 0x7f22d00a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e8102330 0x7f22e8103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22e8101970 0x7f22e8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 --2- 172.21.15.64:0/857251655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22e8100780 0x7f22e81047b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 >> 172.21.15.64:0/857251655 conn(0x7f22e80fbd10 msgr2=0x7f22e8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 shutdown_connections 2024-05-09T09:06:37.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.283+0000 7f22ecd89640 1 -- 172.21.15.64:0/857251655 wait complete. 2024-05-09T09:06:37.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 -- 172.21.15.64:0/2178512343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c102300 msgr2=0x7f9e5c1046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 --2- 172.21.15.64:0/2178512343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c102300 0x7f9e5c1046f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402a410 comp rx=0 tx=0).stop 2024-05-09T09:06:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 -- 172.21.15.64:0/2178512343 shutdown_connections 2024-05-09T09:06:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 --2- 172.21.15.64:0/2178512343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c107560 0x7f9e5c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 --2- 172.21.15.64:0/2178512343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c104c30 0x7f9e5c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 --2- 172.21.15.64:0/2178512343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c102300 0x7f9e5c1046f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 -- 172.21.15.64:0/2178512343 >> 172.21.15.64:0/2178512343 conn(0x7f9e5c0fbd10 msgr2=0x7f9e5c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 -- 172.21.15.64:0/2178512343 shutdown_connections 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 -- 172.21.15.64:0/2178512343 wait complete. 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.400+0000 7f9e64309640 1 Processor -- start 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 -- start start 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c102300 0x7f9e5c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c104c30 0x7f9e5c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 0x7f9e5c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c10d3e0 con 0x7f9e5c107560 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c10d260 con 0x7f9e5c104c30 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e64309640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c10d560 con 0x7f9e5c102300 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 0x7f9e5c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 0x7f9e5c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40054/0 (socket says 172.21.15.64:40054) 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 -- 172.21.15.64:0/3374890335 learned_addr learned my addr 172.21.15.64:0/3374890335 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e620a5640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c102300 0x7f9e5c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e618a4640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c104c30 0x7f9e5c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 -- 172.21.15.64:0/3374890335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c102300 msgr2=0x7f9e5c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c102300 0x7f9e5c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 -- 172.21.15.64:0/3374890335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c104c30 msgr2=0x7f9e5c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c104c30 0x7f9e5c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 -- 172.21.15.64:0/3374890335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c19f740 con 0x7f9e5c107560 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e628a6640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 0x7f9e5c19f0d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00eaa0 tx=0x7f9e4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.401+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00cdb0 con 0x7f9e5c107560 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.402+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c19fa30 con 0x7f9e5c107560 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.402+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c005020 con 0x7f9e5c107560 2024-05-09T09:06:37.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.402+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0116d0 con 0x7f9e5c107560 2024-05-09T09:06:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.402+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c19ff80 con 0x7f9e5c107560 2024-05-09T09:06:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.402+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c0fd800 con 0x7f9e5c107560 2024-05-09T09:06:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.404+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e4c0051d0 con 0x7f9e5c107560 2024-05-09T09:06:37.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.404+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2b90 0x7f9e440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.405+0000 7f9e620a5640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2b90 0x7f9e440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.405+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9e4c0cf090 con 0x7f9e5c107560 2024-05-09T09:06:37.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.405+0000 7f9e620a5640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2b90 0x7f9e440a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006010 tx=0x7f9e54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:37.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.408+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e4c016030 con 0x7f9e5c107560 2024-05-09T09:06:37.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.602+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e5c062430 con 0x7f9e5c107560 2024-05-09T09:06:37.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.603+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/3374890335 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18454 (secure 0 0 0) 0x7f9e4c09be40 con 0x7f9e5c107560 2024-05-09T09:06:37.602 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:37.602 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":82,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:05:48.950167+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:37.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2b90 msgr2=0x7f9e440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2b90 0x7f9e440a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006010 tx=0x7f9e54000bc0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 msgr2=0x7f9e5c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 0x7f9e5c19f0d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00eaa0 tx=0x7f9e4c00ef60 comp rx=0 tx=0).stop 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 shutdown_connections 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2b90 0x7f9e440a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c107560 0x7f9e5c19f0d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c104c30 0x7f9e5c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 --2- 172.21.15.64:0/3374890335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c102300 0x7f9e5c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 >> 172.21.15.64:0/3374890335 conn(0x7f9e5c0fbd10 msgr2=0x7f9e5c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 shutdown_connections 2024-05-09T09:06:37.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.605+0000 7f9e64309640 1 -- 172.21.15.64:0/3374890335 wait complete. 2024-05-09T09:06:37.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T09:06:37.616 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:06:37.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 -- 172.21.15.64:0/1878695588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8102320 msgr2=0x7fb0b8108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 --2- 172.21.15.64:0/1878695588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8102320 0x7fb0b8108bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009ca0 tx=0x7fb0b002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 -- 172.21.15.64:0/1878695588 shutdown_connections 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 --2- 172.21.15.64:0/1878695588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8102320 0x7fb0b8108bb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 --2- 172.21.15.64:0/1878695588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8101960 0x7fb0b8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 --2- 172.21.15.64:0/1878695588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8100770 0x7fb0b8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 -- 172.21.15.64:0/1878695588 >> 172.21.15.64:0/1878695588 conn(0x7fb0b80fbce0 msgr2=0x7fb0b80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 -- 172.21.15.64:0/1878695588 shutdown_connections 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.727+0000 7fb0be34f640 1 -- 172.21.15.64:0/1878695588 wait complete. 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 Processor -- start 2024-05-09T09:06:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 -- start start 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8100770 0x7fb0b8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 0x7fb0b8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8102320 0x7fb0b819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810c2d0 con 0x7fb0b8101960 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810c150 con 0x7fb0b8102320 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0be34f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810c450 con 0x7fb0b8100770 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 0x7fb0b8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 0x7fb0b8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40076/0 (socket says 172.21.15.64:40076) 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 -- 172.21.15.64:0/2837947057 learned_addr learned my addr 172.21.15.64:0/2837947057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b7fff640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8100770 0x7fb0b8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0bc8ec640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8102320 0x7fb0b819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 -- 172.21.15.64:0/2837947057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8100770 msgr2=0x7fb0b8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8100770 0x7fb0b8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 -- 172.21.15.64:0/2837947057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8102320 msgr2=0x7fb0b819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.728+0000 7fb0b77fe640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8102320 0x7fb0b819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0b77fe640 1 -- 172.21.15.64:0/2837947057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b819f6a0 con 0x7fb0b8101960 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0b77fe640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 0x7fb0b8197fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000d950 tx=0x7fb0a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0017030 con 0x7fb0b8101960 2024-05-09T09:06:37.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a0004020 con 0x7fb0b8101960 2024-05-09T09:06:37.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00047a0 con 0x7fb0b8101960 2024-05-09T09:06:37.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b819f990 con 0x7fb0b8101960 2024-05-09T09:06:37.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.729+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b819ff30 con 0x7fb0b8101960 2024-05-09T09:06:37.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.730+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b81a0220 con 0x7fb0b8101960 2024-05-09T09:06:37.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.732+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb0a00041d0 con 0x7fb0b8101960 2024-05-09T09:06:37.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.733+0000 7fb0b57fa640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:37.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.733+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb0a00ce500 con 0x7fb0b8101960 2024-05-09T09:06:37.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.733+0000 7fb0b7fff640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:37.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.733+0000 7fb0b7fff640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0b81017c0 tx=0x7fb0a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:37.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.738+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb0a009b900 con 0x7fb0b8101960 2024-05-09T09:06:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.930+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0b81a03f0 con 0x7fb0b8101960 2024-05-09T09:06:37.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.930+0000 7fb0b57fa640 1 -- 172.21.15.64:0/2837947057 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb0a009b200 con 0x7fb0b8101960 2024-05-09T09:06:37.929 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2c10 msgr2=0x7fb0a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0b81017c0 tx=0x7fb0a8005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 msgr2=0x7fb0b8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 0x7fb0b8197fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000d950 tx=0x7fb0a000de10 comp rx=0 tx=0).stop 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 shutdown_connections 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8102320 0x7fb0b819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8101960 0x7fb0b8197fc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 --2- 172.21.15.64:0/2837947057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8100770 0x7fb0b8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 >> 172.21.15.64:0/2837947057 conn(0x7fb0b80fbce0 msgr2=0x7fb0b80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 shutdown_connections 2024-05-09T09:06:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:37.933+0000 7fb0be34f640 1 -- 172.21.15.64:0/2837947057 wait complete. 2024-05-09T09:06:37.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T09:06:38.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 -- 172.21.15.64:0/1581834238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4102340 msgr2=0x7f7eb4108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 --2- 172.21.15.64:0/1581834238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4102340 0x7f7eb4108bd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009ca0 tx=0x7f7eac02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 -- 172.21.15.64:0/1581834238 shutdown_connections 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 --2- 172.21.15.64:0/1581834238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4102340 0x7f7eb4108bd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 --2- 172.21.15.64:0/1581834238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4101980 0x7f7eb4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 --2- 172.21.15.64:0/1581834238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb4100790 0x7f7eb4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 -- 172.21.15.64:0/1581834238 >> 172.21.15.64:0/1581834238 conn(0x7f7eb40fbd20 msgr2=0x7f7eb40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 -- 172.21.15.64:0/1581834238 shutdown_connections 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.058+0000 7f7ebb916640 1 -- 172.21.15.64:0/1581834238 wait complete. 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 Processor -- start 2024-05-09T09:06:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 -- start start 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4100790 0x7f7eb406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 0x7f7eb419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb4102340 0x7f7eb419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb410c2a0 con 0x7f7eb4101980 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb410c120 con 0x7f7eb4100790 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7ebb916640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb410c420 con 0x7f7eb4102340 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7eb8eb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 0x7f7eb419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7eb8eb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 0x7f7eb419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40106/0 (socket says 172.21.15.64:40106) 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7eb8eb1640 1 -- 172.21.15.64:0/660393218 learned_addr learned my addr 172.21.15.64:0/660393218 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.059+0000 7f7eb96b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4100790 0x7f7eb406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb9eb3640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb4102340 0x7f7eb419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb8eb1640 1 -- 172.21.15.64:0/660393218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb4102340 msgr2=0x7f7eb419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb8eb1640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb4102340 0x7f7eb419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb8eb1640 1 -- 172.21.15.64:0/660393218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4100790 msgr2=0x7f7eb406a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb8eb1640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4100790 0x7f7eb406a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb8eb1640 1 -- 172.21.15.64:0/660393218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb419f610 con 0x7f7eb4101980 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb96b2640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4100790 0x7f7eb406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eb8eb1640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 0x7f7eb419c6e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400d950 tx=0x7f7ea400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4017030 con 0x7f7eb4101980 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb419f900 con 0x7f7eb4101980 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea4004020 con 0x7f7eb4101980 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea40047a0 con 0x7f7eb4101980 2024-05-09T09:06:38.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.060+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb419fe50 con 0x7f7eb4101980 2024-05-09T09:06:38.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.061+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb4107ab0 con 0x7f7eb4101980 2024-05-09T09:06:38.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.062+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7ea40041d0 con 0x7f7eb4101980 2024-05-09T09:06:38.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.063+0000 7f7eaa7fc640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7ea00a2b90 0x7f7ea00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.063+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7ea40ce540 con 0x7f7eb4101980 2024-05-09T09:06:38.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.063+0000 7f7eb96b2640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7ea00a2b90 0x7f7ea00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.064+0000 7f7eb96b2640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7ea00a2b90 0x7f7ea00a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7eb41017e0 tx=0x7f7eb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:38.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.068+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ea409b940 con 0x7f7eb4101980 2024-05-09T09:06:38.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.259+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7eb402d3c0 con 0x7f7eb4101980 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.260+0000 7f7eaa7fc640 1 -- 172.21.15.64:0/660393218 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f7ea409b240 con 0x7f7eb4101980 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:06:38.259 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-05-09T09:06:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7ea00a2b90 msgr2=0x7f7ea00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7ea00a2b90 0x7f7ea00a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7eb41017e0 tx=0x7f7eb0005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 msgr2=0x7f7eb419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 0x7f7eb419c6e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400d950 tx=0x7f7ea400de10 comp rx=0 tx=0).stop 2024-05-09T09:06:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 shutdown_connections 2024-05-09T09:06:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7ea00a2b90 0x7f7ea00a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb4102340 0x7f7eb419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb4101980 0x7f7eb419c6e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 --2- 172.21.15.64:0/660393218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb4100790 0x7f7eb406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 >> 172.21.15.64:0/660393218 conn(0x7f7eb40fbd20 msgr2=0x7f7eb4104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 shutdown_connections 2024-05-09T09:06:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.262+0000 7f7ebb916640 1 -- 172.21.15.64:0/660393218 wait complete. 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/857251655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.268 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.268 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3374890335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.268 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2837947057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:38.274 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T09:06:38.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T09:06:38.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:38.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/857251655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3374890335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2837947057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 -- 172.21.15.64:0/1217917737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc102a70 msgr2=0x7f85bc104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1217917737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc102a70 0x7f85bc104ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009900 tx=0x7f85b002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 -- 172.21.15.64:0/1217917737 shutdown_connections 2024-05-09T09:06:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1217917737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 0x7f85bc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1217917737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc102a70 0x7f85bc104ee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1217917737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc100100 0x7f85bc102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 -- 172.21.15.64:0/1217917737 >> 172.21.15.64:0/1217917737 conn(0x7f85bc0fbd00 msgr2=0x7f85bc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.387+0000 7f85c1de7640 1 -- 172.21.15.64:0/1217917737 shutdown_connections 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.388+0000 7f85c1de7640 1 -- 172.21.15.64:0/1217917737 wait complete. 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.388+0000 7f85c1de7640 1 Processor -- start 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.388+0000 7f85c1de7640 1 -- start start 2024-05-09T09:06:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85c1de7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 0x7f85bc19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85c1de7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 0x7f85bc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85c1de7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 0x7f85bc1a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85c1de7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc113a00 con 0x7f85bc100100 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85c1de7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc113880 con 0x7f85bc102a70 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85c1de7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc113b80 con 0x7f85bc105420 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 0x7f85bc19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85baffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 0x7f85bc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85baffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 0x7f85bc19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:56028/0 (socket says 172.21.15.64:56028) 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85baffd640 1 -- 172.21.15.64:0/1634819599 learned_addr learned my addr 172.21.15.64:0/1634819599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 0x7f85bc19e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40132/0 (socket says 172.21.15.64:40132) 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bbfff640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 0x7f85bc1a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 -- 172.21.15.64:0/1634819599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 msgr2=0x7f85bc1a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 0x7f85bc1a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 -- 172.21.15.64:0/1634819599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 msgr2=0x7f85bc19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 0x7f85bc19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bb7fe640 1 -- 172.21.15.64:0/1634819599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc1a5cc0 con 0x7f85bc100100 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85baffd640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 0x7f85bc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.389+0000 7f85bbfff640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 0x7f85bc1a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.390+0000 7f85bb7fe640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 0x7f85bc19e0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00eb00 tx=0x7f85ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.390+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac010430 con 0x7f85bc100100 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.390+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ac010a80 con 0x7f85bc100100 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.390+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc1a5fb0 con 0x7f85bc100100 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.390+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017ad0 con 0x7f85bc100100 2024-05-09T09:06:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.390+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc1a6420 con 0x7f85bc100100 2024-05-09T09:06:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.391+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc0fd7f0 con 0x7f85bc100100 2024-05-09T09:06:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.392+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85ac010c30 con 0x7f85bc100100 2024-05-09T09:06:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.393+0000 7f85b8ff9640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f85a40a2c10 0x7f85a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.393+0000 7f85baffd640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f85a40a2c10 0x7f85a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.393+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f85ac0cf490 con 0x7f85bc100100 2024-05-09T09:06:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.393+0000 7f85baffd640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f85a40a2c10 0x7f85a40a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f85b00096c0 tx=0x7f85b00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.397+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f85ac09c890 con 0x7f85bc100100 2024-05-09T09:06:38.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:38.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/857251655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3374890335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:38.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2837947057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:38.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:38.584+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f85bc02d3b0 con 0x7f85bc100100 2024-05-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.088+0000 7f85b8ff9640 1 -- 172.21.15.64:0/1634819599 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f85ac09c190 con 0x7f85bc100100 2024-05-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-05-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f85a40a2c10 msgr2=0x7f85a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f85a40a2c10 0x7f85a40a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f85b00096c0 tx=0x7f85b00314a0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 msgr2=0x7f85bc19e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 0x7f85bc19e0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00eb00 tx=0x7f85ac00c2d0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 shutdown_connections 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f85a40a2c10 0x7f85a40a50d0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f85b00096c0 tx=0x7f85b00314a0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85bc105420 0x7f85bc1a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc102a70 0x7f85bc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 --2- 172.21.15.64:0/1634819599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100100 0x7f85bc19e0a0 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00eb00 tx=0x7f85ac00c2d0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.091+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 >> 172.21.15.64:0/1634819599 conn(0x7f85bc0fbd00 msgr2=0x7f85bc1009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.092+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 shutdown_connections 2024-05-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.092+0000 7f85c1de7640 1 -- 172.21.15.64:0/1634819599 wait complete. 2024-05-09T09:06:39.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 -- 172.21.15.64:0/2365465668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0104ca0 msgr2=0x7f3cb0107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/2365465668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0107090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009900 tx=0x7f3ca402a410 comp rx=0 tx=0).stop 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 -- 172.21.15.64:0/2365465668 shutdown_connections 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/2365465668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb01075d0 0x7f3cb01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/2365465668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0107090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/2365465668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb0102370 0x7f3cb0104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 -- 172.21.15.64:0/2365465668 >> 172.21.15.64:0/2365465668 conn(0x7f3cb00fbd00 msgr2=0x7f3cb00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 -- 172.21.15.64:0/2365465668 shutdown_connections 2024-05-09T09:06:39.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.233+0000 7f3cb50ee640 1 -- 172.21.15.64:0/2365465668 wait complete. 2024-05-09T09:06:39.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.234+0000 7f3cb50ee640 1 Processor -- start 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.234+0000 7f3cb50ee640 1 -- start start 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cb50ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 0x7f3cb0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cb50ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cb50ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb01075d0 0x7f3cb019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cb50ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb010d3c0 con 0x7f3cb0102370 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cb50ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb010d240 con 0x7f3cb01075d0 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cb50ee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb010d540 con 0x7f3cb0104ca0 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 0x7f3cb0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 0x7f3cb0197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40156/0 (socket says 172.21.15.64:40156) 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 -- 172.21.15.64:0/3504596662 learned_addr learned my addr 172.21.15.64:0/3504596662 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cae59c640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:39.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caf59e640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb01075d0 0x7f3cb019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 -- 172.21.15.64:0/3504596662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb0104ca0 msgr2=0x7f3cb0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 -- 172.21.15.64:0/3504596662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb01075d0 msgr2=0x7f3cb019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb01075d0 0x7f3cb019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 -- 172.21.15.64:0/3504596662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb019f620 con 0x7f3cb0102370 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caf59e640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb01075d0 0x7f3cb019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3cae59c640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.235+0000 7f3caed9d640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 0x7f3cb0197a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000c920 tx=0x7f3ca000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.236+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0007c70 con 0x7f3cb0102370 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.236+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ca0007e10 con 0x7f3cb0102370 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.236+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb019f910 con 0x7f3cb0102370 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.236+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0018820 con 0x7f3cb0102370 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.236+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb019fe30 con 0x7f3cb0102370 2024-05-09T09:06:39.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.237+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb00fd7f0 con 0x7f3cb0102370 2024-05-09T09:06:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.238+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ca00189c0 con 0x7f3cb0102370 2024-05-09T09:06:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.239+0000 7f3c97fff640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c980a2b90 0x7f3c980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.239+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f3ca00cf130 con 0x7f3cb0102370 2024-05-09T09:06:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.239+0000 7f3cae59c640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c980a2b90 0x7f3c980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:39.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.240+0000 7f3cae59c640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c980a2b90 0x7f3c980a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4004770 tx=0x7f3ca4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.243+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ca0014030 con 0x7f3cb0102370 2024-05-09T09:06:39.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/660393218' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T09:06:39.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1634819599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:06:39.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:39.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:39 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:39.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:39.430+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3cb0062430 con 0x7f3cb0102370 2024-05-09T09:06:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/660393218' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T09:06:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1634819599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:06:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:39 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:39.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/660393218' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T09:06:39.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1634819599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T09:06:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:39 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.091+0000 7f3c97fff640 1 -- 172.21.15.64:0/3504596662 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f3ca009c450 con 0x7f3cb0102370 2024-05-09T09:06:40.090 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-05-09T09:06:40.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c980a2b90 msgr2=0x7f3c980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:40.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c980a2b90 0x7f3c980a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4004770 tx=0x7f3ca4000bc0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 msgr2=0x7f3cb0197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:40.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 0x7f3cb0197a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000c920 tx=0x7f3ca000cde0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 shutdown_connections 2024-05-09T09:06:40.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c980a2b90 0x7f3c980a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cb01075d0 0x7f3cb019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb0104ca0 0x7f3cb0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 --2- 172.21.15.64:0/3504596662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb0102370 0x7f3cb0197a00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 >> 172.21.15.64:0/3504596662 conn(0x7f3cb00fbd00 msgr2=0x7f3cb0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:40.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 shutdown_connections 2024-05-09T09:06:40.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.093+0000 7f3cb50ee640 1 -- 172.21.15.64:0/3504596662 wait complete. 2024-05-09T09:06:40.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T09:06:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 -- 172.21.15.64:0/2044144908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0107570 msgr2=0x7f34d01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 --2- 172.21.15.64:0/2044144908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0107570 0x7f34d01099c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009ca0 tx=0x7f34c802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 -- 172.21.15.64:0/2044144908 shutdown_connections 2024-05-09T09:06:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 --2- 172.21.15.64:0/2044144908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0107570 0x7f34d01099c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 --2- 172.21.15.64:0/2044144908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0104c40 0x7f34d0107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 --2- 172.21.15.64:0/2044144908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0102310 0x7f34d0104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 -- 172.21.15.64:0/2044144908 >> 172.21.15.64:0/2044144908 conn(0x7f34d00fbd20 msgr2=0x7f34d00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 -- 172.21.15.64:0/2044144908 shutdown_connections 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.224+0000 7f34d61c1640 1 -- 172.21.15.64:0/2044144908 wait complete. 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 Processor -- start 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 -- start start 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0102310 0x7f34d0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 0x7f34d0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0107570 0x7f34d019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34d010d3e0 con 0x7f34d0104c40 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34d010d260 con 0x7f34d0107570 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34d61c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34d010d560 con 0x7f34d0102310 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 0x7f34d0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 0x7f34d0197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40176/0 (socket says 172.21.15.64:40176) 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 -- 172.21.15.64:0/1145552177 learned_addr learned my addr 172.21.15.64:0/1145552177 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34cf7fe640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0102310 0x7f34d0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34cffff640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0107570 0x7f34d019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 -- 172.21.15.64:0/1145552177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0102310 msgr2=0x7f34d0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0102310 0x7f34d0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 -- 172.21.15.64:0/1145552177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0107570 msgr2=0x7f34d019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0107570 0x7f34d019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34ceffd640 1 -- 172.21.15.64:0/1145552177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d019f6d0 con 0x7f34d0104c40 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34cffff640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0107570 0x7f34d019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.225+0000 7f34cf7fe640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0102310 0x7f34d0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.226+0000 7f34ceffd640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 0x7f34d0197ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f34c400d740 tx=0x7f34c400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.226+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4017070 con 0x7f34d0104c40 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.226+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34c400de50 con 0x7f34d0104c40 2024-05-09T09:06:40.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.226+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d019f9c0 con 0x7f34d0104c40 2024-05-09T09:06:40.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.226+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4004d60 con 0x7f34d0104c40 2024-05-09T09:06:40.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.226+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d019ff10 con 0x7f34d0104c40 2024-05-09T09:06:40.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.227+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d00fd810 con 0x7f34d0104c40 2024-05-09T09:06:40.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.228+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f34c4004f00 con 0x7f34d0104c40 2024-05-09T09:06:40.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.229+0000 7f34ccff9640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34b80a2c10 0x7f34b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:40.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.229+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f34c40cebf0 con 0x7f34d0104c40 2024-05-09T09:06:40.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.229+0000 7f34cf7fe640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34b80a2c10 0x7f34b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:40.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.229+0000 7f34cf7fe640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34b80a2c10 0x7f34b80a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005d00 tx=0x7f34c0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:40.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.233+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f34c409bd80 con 0x7f34d0104c40 2024-05-09T09:06:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1634819599' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T09:06:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:40 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3504596662' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T09:06:40.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:40.456+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f34d00624a0 con 0x7f34d0104c40 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1634819599' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:40 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3504596662' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T09:06:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1634819599' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T09:06:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:40 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3504596662' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T09:06:41.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.117+0000 7f34ccff9640 1 -- 172.21.15.64:0/1145552177 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f34c409b680 con 0x7f34d0104c40 2024-05-09T09:06:41.116 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T09:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.121+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34b80a2c10 msgr2=0x7f34b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.121+0000 7f34d61c1640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34b80a2c10 0x7f34b80a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005d00 tx=0x7f34c0005c70 comp rx=0 tx=0).stop 2024-05-09T09:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.121+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 msgr2=0x7f34d0197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:41.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.121+0000 7f34d61c1640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 0x7f34d0197ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f34c400d740 tx=0x7f34c400dc00 comp rx=0 tx=0).stop 2024-05-09T09:06:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 shutdown_connections 2024-05-09T09:06:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34b80a2c10 0x7f34b80a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34d0107570 0x7f34d019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34d0104c40 0x7f34d0197ff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 --2- 172.21.15.64:0/1145552177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d0102310 0x7f34d0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 >> 172.21.15.64:0/1145552177 conn(0x7f34d00fbd20 msgr2=0x7f34d0102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 shutdown_connections 2024-05-09T09:06:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.124+0000 7f34d61c1640 1 -- 172.21.15.64:0/1145552177 wait complete. 2024-05-09T09:06:41.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:41 smithi064 ceph-mon[30149]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:06:41.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:41 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3504596662' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T09:06:41.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:41 smithi064 ceph-mon[30149]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T09:06:41.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:41 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:41.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:41 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1145552177' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T09:06:41.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:41 smithi064 ceph-mon[30149]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T09:06:41.145 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T09:06:41.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T09:06:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 -- 172.21.15.64:0/373517938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100aa0 msgr2=0x7fdcdc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 --2- 172.21.15.64:0/373517938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc100f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 -- 172.21.15.64:0/373517938 shutdown_connections 2024-05-09T09:06:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 --2- 172.21.15.64:0/373517938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 0x7fdcdc1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 --2- 172.21.15.64:0/373517938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc100f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 --2- 172.21.15.64:0/373517938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100140 0x7fdcdc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 -- 172.21.15.64:0/373517938 >> 172.21.15.64:0/373517938 conn(0x7fdcdc0fbce0 msgr2=0x7fdcdc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 -- 172.21.15.64:0/373517938 shutdown_connections 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 -- 172.21.15.64:0/373517938 wait complete. 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.258+0000 7fdce2732640 1 Processor -- start 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 -- start start 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 0x7fdcdc199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 0x7fdcdc1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10f560 con 0x7fdcdc100140 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10f3e0 con 0x7fdcdc101440 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce2732640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10f6e0 con 0x7fdcdc100aa0 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 0x7fdcdc199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 0x7fdcdc199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40190/0 (socket says 172.21.15.64:40190) 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 -- 172.21.15.64:0/2766632645 learned_addr learned my addr 172.21.15.64:0/2766632645 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdb7fe640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce0ccf640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 0x7fdcdc1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 -- 172.21.15.64:0/2766632645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100aa0 msgr2=0x7fdcdc19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 -- 172.21.15.64:0/2766632645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 msgr2=0x7fdcdc1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 0x7fdcdc1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdbfff640 1 -- 172.21.15.64:0/2766632645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1a1860 con 0x7fdcdc100140 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdce0ccf640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 0x7fdcdc1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.259+0000 7fdcdb7fe640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.260+0000 7fdcdbfff640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 0x7fdcdc199c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c920 tx=0x7fdccc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.260+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc007c70 con 0x7fdcdc100140 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.260+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1a1b50 con 0x7fdcdc100140 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.260+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdccc007e10 con 0x7fdcdc100140 2024-05-09T09:06:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.260+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc018820 con 0x7fdcdc100140 2024-05-09T09:06:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.260+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a20a0 con 0x7fdcdc100140 2024-05-09T09:06:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.261+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc10b080 con 0x7fdcdc100140 2024-05-09T09:06:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.262+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdccc0189c0 con 0x7fdcdc100140 2024-05-09T09:06:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.263+0000 7fdcd97fa640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcc40a2b90 0x7fdcc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.263+0000 7fdcdb7fe640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcc40a2b90 0x7fdcc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.263+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdccc0cf180 con 0x7fdcdc100140 2024-05-09T09:06:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.263+0000 7fdcdb7fe640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcc40a2b90 0x7fdcc40a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00314e0 tx=0x7fdcd0032d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.268+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdccc014030 con 0x7fdcdc100140 2024-05-09T09:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:41.462+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fdcdc02d430 con 0x7fdcdc100140 2024-05-09T09:06:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:41 smithi179 ceph-mon[34086]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:06:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:41 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3504596662' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T09:06:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:41 smithi179 ceph-mon[34086]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T09:06:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:41 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:41 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1145552177' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T09:06:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:41 smithi179 ceph-mon[34086]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T09:06:41.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:41 smithi139 ceph-mon[33599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:06:41.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:41 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3504596662' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T09:06:41.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:41 smithi139 ceph-mon[33599]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T09:06:41.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:41 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:06:41.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:41 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1145552177' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T09:06:41.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:41 smithi139 ceph-mon[33599]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T09:06:42.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.118+0000 7fdcd97fa640 1 -- 172.21.15.64:0/2766632645 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fdccc00de20 con 0x7fdcdc100140 2024-05-09T09:06:42.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.121+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcc40a2b90 msgr2=0x7fdcc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:42.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.121+0000 7fdce2732640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcc40a2b90 0x7fdcc40a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00314e0 tx=0x7fdcd0032d40 comp rx=0 tx=0).stop 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.121+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 msgr2=0x7fdcdc199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.121+0000 7fdce2732640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 0x7fdcdc199c50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c920 tx=0x7fdccc00cde0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 shutdown_connections 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcc40a2b90 0x7fdcc40a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcdc101440 0x7fdcdc1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc100aa0 0x7fdcdc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 --2- 172.21.15.64:0/2766632645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcdc100140 0x7fdcdc199c50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 >> 172.21.15.64:0/2766632645 conn(0x7fdcdc0fbce0 msgr2=0x7fdcdc0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 shutdown_connections 2024-05-09T09:06:42.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.122+0000 7fdce2732640 1 -- 172.21.15.64:0/2766632645 wait complete. 2024-05-09T09:06:42.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T09:06:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 -- 172.21.15.64:0/595885498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 msgr2=0x7f068c109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 --2- 172.21.15.64:0/595885498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c109ce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0684009ca0 tx=0x7f068402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 -- 172.21.15.64:0/595885498 shutdown_connections 2024-05-09T09:06:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 --2- 172.21.15.64:0/595885498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c109ce0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 --2- 172.21.15.64:0/595885498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c102a90 0x7f068c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 --2- 172.21.15.64:0/595885498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c1018a0 0x7f068c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 -- 172.21.15.64:0/595885498 >> 172.21.15.64:0/595885498 conn(0x7f068c0fce30 msgr2=0x7f068c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 -- 172.21.15.64:0/595885498 shutdown_connections 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.257+0000 7f06924e1640 1 -- 172.21.15.64:0/595885498 wait complete. 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 Processor -- start 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 -- start start 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c1018a0 0x7f068c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c102a90 0x7f068c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10d4c0 con 0x7f068c103450 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10d340 con 0x7f068c1018a0 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f06924e1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10d640 con 0x7f068c102a90 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c074160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40200/0 (socket says 172.21.15.64:40200) 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 -- 172.21.15.64:0/3045992049 learned_addr learned my addr 172.21.15.64:0/3045992049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f068bfff640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c1018a0 0x7f068c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0683fff640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c102a90 0x7f068c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 -- 172.21.15.64:0/3045992049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c102a90 msgr2=0x7f068c073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c102a90 0x7f068c073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 -- 172.21.15.64:0/3045992049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c1018a0 msgr2=0x7f068c0755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c1018a0 0x7f068c0755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 -- 172.21.15.64:0/3045992049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c0749f0 con 0x7f068c103450 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.258+0000 7f0690a7e640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c074160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0684009dc0 tx=0x7f0684003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.259+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684004040 con 0x7f068c103450 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.259+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c1a1c90 con 0x7f068c103450 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.259+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0684004690 con 0x7f068c103450 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.259+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068402f4c0 con 0x7f068c103450 2024-05-09T09:06:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.259+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1a2200 con 0x7f068c103450 2024-05-09T09:06:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.260+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068c108bc0 con 0x7f068c103450 2024-05-09T09:06:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.261+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f06840305c0 con 0x7f068c103450 2024-05-09T09:06:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.262+0000 7f0689ffb640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06780a2b90 0x7f06780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.262+0000 7f068bfff640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06780a2b90 0x7f06780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.262+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f06840f0880 con 0x7f068c103450 2024-05-09T09:06:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.262+0000 7f068bfff640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06780a2b90 0x7f06780a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f068c1028f0 tx=0x7f067c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.266+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f068404d3a0 con 0x7f068c103450 2024-05-09T09:06:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:42 smithi064 ceph-mon[30149]: pgmap v287: 97 pgs: 17 creating+peering, 4 active+clean, 76 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:42 smithi064 ceph-mon[30149]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T09:06:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:42 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1145552177' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T09:06:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:42 smithi064 ceph-mon[30149]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T09:06:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:42 smithi064 ceph-mon[30149]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T09:06:42.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:42 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2766632645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T09:06:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:42.450+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f068c062430 con 0x7f068c103450 2024-05-09T09:06:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:42 smithi179 ceph-mon[34086]: pgmap v287: 97 pgs: 17 creating+peering, 4 active+clean, 76 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:42 smithi179 ceph-mon[34086]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T09:06:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:42 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1145552177' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T09:06:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:42 smithi179 ceph-mon[34086]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T09:06:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:42 smithi179 ceph-mon[34086]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T09:06:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:42 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2766632645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T09:06:42.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:42 smithi139 ceph-mon[33599]: pgmap v287: 97 pgs: 17 creating+peering, 4 active+clean, 76 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:06:42.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:42 smithi139 ceph-mon[33599]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T09:06:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:42 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1145552177' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T09:06:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:42 smithi139 ceph-mon[33599]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T09:06:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:42 smithi139 ceph-mon[33599]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T09:06:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:42 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2766632645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T09:06:43.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:43 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2766632645' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:43 smithi064 ceph-mon[30149]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:43 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] up:active 2024-05-09T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:43 smithi064 ceph-mon[30149]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:43 smithi064 ceph-mon[30149]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:43 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3045992049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T09:06:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:43 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2766632645' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T09:06:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:43 smithi179 ceph-mon[34086]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T09:06:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:43 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] up:active 2024-05-09T09:06:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:43 smithi179 ceph-mon[34086]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:43 smithi179 ceph-mon[34086]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:43 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3045992049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T09:06:43.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:43 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2766632645' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T09:06:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:43 smithi139 ceph-mon[33599]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T09:06:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:43 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] up:active 2024-05-09T09:06:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:43 smithi139 ceph-mon[33599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:43 smithi139 ceph-mon[33599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:43 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3045992049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T09:06:44.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.144+0000 7f0689ffb640 1 -- 172.21.15.64:0/3045992049 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f06840bd7b0 con 0x7f068c103450 2024-05-09T09:06:44.146 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data_ec' created 2024-05-09T09:06:44.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06780a2b90 msgr2=0x7f06780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:44.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06780a2b90 0x7f06780a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f068c1028f0 tx=0x7f067c009110 comp rx=0 tx=0).stop 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 msgr2=0x7f068c074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c074160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0684009dc0 tx=0x7f0684003ba0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 shutdown_connections 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06780a2b90 0x7f06780a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c103450 0x7f068c074160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c102a90 0x7f068c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 --2- 172.21.15.64:0/3045992049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c1018a0 0x7f068c0755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 >> 172.21.15.64:0/3045992049 conn(0x7f068c0fce30 msgr2=0x7f068c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 shutdown_connections 2024-05-09T09:06:44.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.147+0000 7f06924e1640 1 -- 172.21.15.64:0/3045992049 wait complete. 2024-05-09T09:06:44.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T09:06:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 -- 172.21.15.64:0/1018464182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8107570 msgr2=0x7ff4c81099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1018464182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8107570 0x7ff4c81099c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009ca0 tx=0x7ff4c002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 -- 172.21.15.64:0/1018464182 shutdown_connections 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1018464182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8107570 0x7ff4c81099c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1018464182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8104c40 0x7ff4c8107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1018464182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8102310 0x7ff4c8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 -- 172.21.15.64:0/1018464182 >> 172.21.15.64:0/1018464182 conn(0x7ff4c80fbd20 msgr2=0x7ff4c80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 -- 172.21.15.64:0/1018464182 shutdown_connections 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.281+0000 7ff4d0761640 1 -- 172.21.15.64:0/1018464182 wait complete. 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 Processor -- start 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 -- start start 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 0x7ff4c8101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8104c40 0x7ff4c81002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8107570 0x7ff4c8100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c810d410 con 0x7ff4c8102310 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c810d290 con 0x7ff4c8104c40 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4d0761640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c810d590 con 0x7ff4c8107570 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 0x7ff4c8101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 0x7ff4c8101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40232/0 (socket says 172.21.15.64:40232) 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 -- 172.21.15.64:0/1730305851 learned_addr learned my addr 172.21.15.64:0/1730305851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4cecfe640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8107570 0x7ff4c8100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4cdcfc640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8104c40 0x7ff4c81002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 -- 172.21.15.64:0/1730305851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8107570 msgr2=0x7ff4c8100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8107570 0x7ff4c8100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 -- 172.21.15.64:0/1730305851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8104c40 msgr2=0x7ff4c81002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8104c40 0x7ff4c81002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4ce4fd640 1 -- 172.21.15.64:0/1730305851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c8101090 con 0x7ff4c8102310 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.282+0000 7ff4cdcfc640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8104c40 0x7ff4c81002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.283+0000 7ff4ce4fd640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 0x7ff4c8101c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400d950 tx=0x7ff4c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.283+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4017070 con 0x7ff4c8102310 2024-05-09T09:06:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.283+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4c4009c30 con 0x7ff4c8102310 2024-05-09T09:06:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.283+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400f620 con 0x7ff4c8102310 2024-05-09T09:06:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.283+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c81a1ba0 con 0x7ff4c8102310 2024-05-09T09:06:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.283+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c81a2140 con 0x7ff4c8102310 2024-05-09T09:06:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.284+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c80fd810 con 0x7ff4c8102310 2024-05-09T09:06:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.286+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4c40109a0 con 0x7ff4c8102310 2024-05-09T09:06:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.286+0000 7ff4bf7fe640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff4b40a2c10 0x7ff4b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.287+0000 7ff4cdcfc640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff4b40a2c10 0x7ff4b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.287+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7ff4c40cee80 con 0x7ff4c8102310 2024-05-09T09:06:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.287+0000 7ff4cdcfc640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff4b40a2c10 0x7ff4b40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff4c81019f0 tx=0x7ff4b8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.290+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff4c4009de0 con 0x7ff4c8102310 2024-05-09T09:06:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:44 smithi064 ceph-mon[30149]: pgmap v290: 97 pgs: 17 creating+peering, 44 active+clean, 36 unknown; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-05-09T09:06:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:44 smithi064 ceph-mon[30149]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T09:06:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:44 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:06:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:44 smithi064 ceph-mon[30149]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T09:06:44.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:44.476+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff4c802d1c0 con 0x7ff4c8102310 2024-05-09T09:06:44.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:44 smithi179 ceph-mon[34086]: pgmap v290: 97 pgs: 17 creating+peering, 44 active+clean, 36 unknown; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-05-09T09:06:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:44 smithi179 ceph-mon[34086]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T09:06:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:44 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:06:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:44 smithi179 ceph-mon[34086]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T09:06:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:44 smithi139 ceph-mon[33599]: pgmap v290: 97 pgs: 17 creating+peering, 44 active+clean, 36 unknown; 578 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-05-09T09:06:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:44 smithi139 ceph-mon[33599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T09:06:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:44 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:06:44.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:44 smithi139 ceph-mon[33599]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T09:06:45.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.151+0000 7ff4bf7fe640 1 -- 172.21.15.64:0/1730305851 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7ff4c409bc40 con 0x7ff4c8102310 2024-05-09T09:06:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff4b40a2c10 msgr2=0x7ff4b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff4b40a2c10 0x7ff4b40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff4c81019f0 tx=0x7ff4b8005d30 comp rx=0 tx=0).stop 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 msgr2=0x7ff4c8101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 0x7ff4c8101c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400d950 tx=0x7ff4c400de10 comp rx=0 tx=0).stop 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 shutdown_connections 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff4b40a2c10 0x7ff4b40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8107570 0x7ff4c8100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c8104c40 0x7ff4c81002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 --2- 172.21.15.64:0/1730305851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8102310 0x7ff4c8101c80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 >> 172.21.15.64:0/1730305851 conn(0x7ff4c80fbd20 msgr2=0x7ff4c8102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 shutdown_connections 2024-05-09T09:06:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.155+0000 7ff4d0761640 1 -- 172.21.15.64:0/1730305851 wait complete. 2024-05-09T09:06:45.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 -- 172.21.15.64:0/2215838592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4103440 msgr2=0x7f07f4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 --2- 172.21.15.64:0/2215838592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4103440 0x7f07f4109cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009ca0 tx=0x7f07ec02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 -- 172.21.15.64:0/2215838592 shutdown_connections 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 --2- 172.21.15.64:0/2215838592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4103440 0x7f07f4109cd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 --2- 172.21.15.64:0/2215838592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4102a80 0x7f07f4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 --2- 172.21.15.64:0/2215838592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4101890 0x7f07f4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 -- 172.21.15.64:0/2215838592 >> 172.21.15.64:0/2215838592 conn(0x7f07f40fce00 msgr2=0x7f07f40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:45.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 -- 172.21.15.64:0/2215838592 shutdown_connections 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 -- 172.21.15.64:0/2215838592 wait complete. 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.294+0000 7f07f9d34640 1 Processor -- start 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 -- start start 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 0x7f07f4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4102a80 0x7f07f4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4103440 0x7f07f419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410d490 con 0x7f07f4101890 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410d310 con 0x7f07f4103440 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f9d34640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410d610 con 0x7f07f4102a80 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 0x7f07f4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 0x7f07f4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57162/0 (socket says 172.21.15.64:57162) 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 -- 172.21.15.64:0/4238129446 learned_addr learned my addr 172.21.15.64:0/4238129446 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f2ffd640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4102a80 0x7f07f4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f3fff640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4103440 0x7f07f419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:45.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 -- 172.21.15.64:0/4238129446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4102a80 msgr2=0x7f07f4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4102a80 0x7f07f4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 -- 172.21.15.64:0/4238129446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4103440 msgr2=0x7f07f419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4103440 0x7f07f419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 -- 172.21.15.64:0/4238129446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f419f730 con 0x7f07f4101890 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f3fff640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4103440 0x7f07f419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f37fe640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 0x7f07f4197b10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c970 tx=0x7f07e400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.295+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4004220 con 0x7f07f4101890 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.296+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f419fa20 con 0x7f07f4101890 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.296+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07e40043c0 con 0x7f07f4101890 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.296+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40047c0 con 0x7f07f4101890 2024-05-09T09:06:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.296+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f419ff20 con 0x7f07f4101890 2024-05-09T09:06:45.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.297+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f4108c80 con 0x7f07f4101890 2024-05-09T09:06:45.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.298+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f07e40079b0 con 0x7f07f4101890 2024-05-09T09:06:45.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.299+0000 7f07f0ff9640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2b90 0x7f07dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:45.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.299+0000 7f07f2ffd640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2b90 0x7f07dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:45.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.299+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f07e40ceff0 con 0x7f07f4101890 2024-05-09T09:06:45.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.299+0000 7f07f2ffd640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2b90 0x7f07dc0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f07f4199020 tx=0x7f07e8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:45.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.303+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07e4014030 con 0x7f07f4101890 2024-05-09T09:06:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:45 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3045992049' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T09:06:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:45 smithi179 ceph-mon[34086]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T09:06:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:45 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1730305851' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T09:06:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:45.534+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f07f402d1c0 con 0x7f07f4101890 2024-05-09T09:06:45.572 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:06:45 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:06:45.332+0000 7f3d1c2fe700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T09:06:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:45 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3045992049' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T09:06:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:45 smithi139 ceph-mon[33599]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T09:06:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:45 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1730305851' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T09:06:45.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:45 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3045992049' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T09:06:45.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:45 smithi064 ceph-mon[30149]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T09:06:45.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:45 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1730305851' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T09:06:46.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.161+0000 7f07f0ff9640 1 -- 172.21.15.64:0/4238129446 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f07e409bd30 con 0x7f07f4101890 2024-05-09T09:06:46.160 INFO:teuthology.orchestra.run.smithi064.stderr:added data pool 4 to fsmap 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.164+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2b90 msgr2=0x7f07dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.164+0000 7f07f9d34640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2b90 0x7f07dc0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f07f4199020 tx=0x7f07e8009110 comp rx=0 tx=0).stop 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.164+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 msgr2=0x7f07f4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.164+0000 7f07f9d34640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 0x7f07f4197b10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c970 tx=0x7f07e400cd70 comp rx=0 tx=0).stop 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.164+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 shutdown_connections 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.164+0000 7f07f9d34640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2b90 0x7f07dc0a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.165+0000 7f07f9d34640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4103440 0x7f07f419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.165+0000 7f07f9d34640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4102a80 0x7f07f4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.165+0000 7f07f9d34640 1 --2- 172.21.15.64:0/4238129446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4101890 0x7f07f4197b10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.165+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 >> 172.21.15.64:0/4238129446 conn(0x7f07f40fce00 msgr2=0x7f07f4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.165+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 shutdown_connections 2024-05-09T09:06:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.165+0000 7f07f9d34640 1 -- 172.21.15.64:0/4238129446 wait complete. 2024-05-09T09:06:46.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/357307194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38100790 msgr2=0x7fcc38100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/357307194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38100790 0x7fcc38100bb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a410 comp rx=0 tx=0).stop 2024-05-09T09:06:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/357307194 shutdown_connections 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/357307194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc38102340 0x7fcc38108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/357307194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38101980 0x7fcc38101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/357307194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38100790 0x7fcc38100bb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/357307194 >> 172.21.15.64:0/357307194 conn(0x7fcc380fbd20 msgr2=0x7fcc380fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/357307194 shutdown_connections 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/357307194 wait complete. 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.291+0000 7fcc3f2d4640 1 Processor -- start 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 -- start start 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc38100790 0x7fcc3806aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 0x7fcc3819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38102340 0x7fcc3819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3810c330 con 0x7fcc38101980 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3810c1b0 con 0x7fcc38100790 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3f2d4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3810c4b0 con 0x7fcc38102340 2024-05-09T09:06:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 0x7fcc3819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 0x7fcc3819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57178/0 (socket says 172.21.15.64:57178) 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 -- 172.21.15.64:0/1860791873 learned_addr learned my addr 172.21.15.64:0/1860791873 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3d070640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc38100790 0x7fcc3806aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3d871640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38102340 0x7fcc3819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 -- 172.21.15.64:0/1860791873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38102340 msgr2=0x7fcc3819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38102340 0x7fcc3819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 -- 172.21.15.64:0/1860791873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc38100790 msgr2=0x7fcc3806aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc38100790 0x7fcc3806aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 -- 172.21.15.64:0/1860791873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3819f780 con 0x7fcc38101980 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3d871640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38102340 0x7fcc3819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc3c86f640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 0x7fcc3819c850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800e9f0 tx=0x7fcc2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2800cd80 con 0x7fcc38101980 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.292+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc280054d0 con 0x7fcc38101980 2024-05-09T09:06:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.293+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3819fa70 con 0x7fcc38101980 2024-05-09T09:06:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.293+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28010490 con 0x7fcc38101980 2024-05-09T09:06:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.293+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a0040 con 0x7fcc38101980 2024-05-09T09:06:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.294+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38107b80 con 0x7fcc38101980 2024-05-09T09:06:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.295+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc28005020 con 0x7fcc38101980 2024-05-09T09:06:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.296+0000 7fcc2e7fc640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcc240a2c10 0x7fcc240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.296+0000 7fcc3d070640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcc240a2c10 0x7fcc240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.296+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcc280cf6c0 con 0x7fcc38101980 2024-05-09T09:06:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.296+0000 7fcc3d070640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcc240a2c10 0x7fcc240a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc34000c00 tx=0x7fcc340314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.300+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc2809c3e0 con 0x7fcc38101980 2024-05-09T09:06:46.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:46 smithi139 ceph-mon[33599]: pgmap v293: 129 pgs: 12 creating+peering, 87 active+clean, 30 unknown; 578 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-09T09:06:46.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:46 smithi139 ceph-mon[33599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:06:46.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:46 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1730305851' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T09:06:46.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:46 smithi139 ceph-mon[33599]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T09:06:46.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:46 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4238129446' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T09:06:46.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:46 smithi179 ceph-mon[34086]: pgmap v293: 129 pgs: 12 creating+peering, 87 active+clean, 30 unknown; 578 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-09T09:06:46.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:46 smithi179 ceph-mon[34086]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:06:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:46 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1730305851' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T09:06:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:46 smithi179 ceph-mon[34086]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T09:06:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:46 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4238129446' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T09:06:46.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.524+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc3802d1c0 con 0x7fcc38101980 2024-05-09T09:06:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.525+0000 7fcc2e7fc640 1 -- 172.21.15.64:0/1860791873 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7fcc2809bce0 con 0x7fcc38101980 2024-05-09T09:06:46.524 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:46.524 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:06:45.536413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24503,"qdb_cluster":[24503]},"id":1}]} 2024-05-09T09:06:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 17 2024-05-09T09:06:46.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcc240a2c10 msgr2=0x7fcc240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcc240a2c10 0x7fcc240a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc34000c00 tx=0x7fcc340314a0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 msgr2=0x7fcc3819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 0x7fcc3819c850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800e9f0 tx=0x7fcc2800edf0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 shutdown_connections 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcc240a2c10 0x7fcc240a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc38102340 0x7fcc3819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38101980 0x7fcc3819c850 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 --2- 172.21.15.64:0/1860791873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc38100790 0x7fcc3806aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 >> 172.21.15.64:0/1860791873 conn(0x7fcc380fbd20 msgr2=0x7fcc38104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 shutdown_connections 2024-05-09T09:06:46.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.527+0000 7fcc3f2d4640 1 -- 172.21.15.64:0/1860791873 wait complete. 2024-05-09T09:06:46.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:46 smithi064 ceph-mon[30149]: pgmap v293: 129 pgs: 12 creating+peering, 87 active+clean, 30 unknown; 578 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-09T09:06:46.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:46 smithi064 ceph-mon[30149]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T09:06:46.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:46 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1730305851' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T09:06:46.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:46 smithi064 ceph-mon[30149]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T09:06:46.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:46 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4238129446' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T09:06:46.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 -- 172.21.15.64:0/1973227993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 msgr2=0x7fa824104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 --2- 172.21.15.64:0/1973227993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824104700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa818009ca0 tx=0x7fa81802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 -- 172.21.15.64:0/1973227993 shutdown_connections 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 --2- 172.21.15.64:0/1973227993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824107570 0x7fa8241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 --2- 172.21.15.64:0/1973227993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824104c40 0x7fa824107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 --2- 172.21.15.64:0/1973227993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824104700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 -- 172.21.15.64:0/1973227993 >> 172.21.15.64:0/1973227993 conn(0x7fa8240fbd20 msgr2=0x7fa8240fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 -- 172.21.15.64:0/1973227993 shutdown_connections 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.643+0000 7fa82c231640 1 -- 172.21.15.64:0/1973227993 wait complete. 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 Processor -- start 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 -- start start 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824104c40 0x7fa8241980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824107570 0x7fa82419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410d3f0 con 0x7fa824102310 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410d270 con 0x7fa824104c40 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82c231640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410d570 con 0x7fa824107570 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57204/0 (socket says 172.21.15.64:57204) 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 -- 172.21.15.64:0/2465757486 learned_addr learned my addr 172.21.15.64:0/2465757486 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:46.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa82a7ce640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824107570 0x7fa82419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa8297cc640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824104c40 0x7fa8241980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 -- 172.21.15.64:0/2465757486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824107570 msgr2=0x7fa82419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824107570 0x7fa82419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 -- 172.21.15.64:0/2465757486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824104c40 msgr2=0x7fa8241980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824104c40 0x7fa8241980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 -- 172.21.15.64:0/2465757486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419f780 con 0x7fa824102310 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.644+0000 7fa829fcd640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824197b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa818009dc0 tx=0x7fa818009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.645+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818003db0 con 0x7fa824102310 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.645+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419fa10 con 0x7fa824102310 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.645+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8180044f0 con 0x7fa824102310 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.645+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818030d90 con 0x7fa824102310 2024-05-09T09:06:46.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.645+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82419ff60 con 0x7fa824102310 2024-05-09T09:06:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.646+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8240fd810 con 0x7fa824102310 2024-05-09T09:06:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.647+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa8180305c0 con 0x7fa824102310 2024-05-09T09:06:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.647+0000 7fa816ffd640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8100a2b90 0x7fa8100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.647+0000 7fa8297cc640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8100a2b90 0x7fa8100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.648+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa8180f0760 con 0x7fa824102310 2024-05-09T09:06:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.648+0000 7fa8297cc640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8100a2b90 0x7fa8100a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa824199070 tx=0x7fa820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:46.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.652+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa8180bd480 con 0x7fa824102310 2024-05-09T09:06:46.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.837+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa824062430 con 0x7fa824102310 2024-05-09T09:06:46.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.839+0000 7fa816ffd640 1 -- 172.21.15.64:0/2465757486 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7fa8180bcd80 con 0x7fa824102310 2024-05-09T09:06:46.838 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:46.838 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:46.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8100a2b90 msgr2=0x7fa8100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8100a2b90 0x7fa8100a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa824199070 tx=0x7fa820005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 msgr2=0x7fa824197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824197b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa818009dc0 tx=0x7fa818009380 comp rx=0 tx=0).stop 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 shutdown_connections 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8100a2b90 0x7fa8100a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824107570 0x7fa82419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824104c40 0x7fa8241980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 --2- 172.21.15.64:0/2465757486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824102310 0x7fa824197b60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 >> 172.21.15.64:0/2465757486 conn(0x7fa8240fbd20 msgr2=0x7fa824102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 shutdown_connections 2024-05-09T09:06:46.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.841+0000 7fa82c231640 1 -- 172.21.15.64:0/2465757486 wait complete. 2024-05-09T09:06:46.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.957+0000 7f49d30bd640 1 -- 172.21.15.64:0/4267987778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc104c20 msgr2=0x7f49cc107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.957+0000 7f49d30bd640 1 --2- 172.21.15.64:0/4267987778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc104c20 0x7f49cc107010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009900 tx=0x7f49b802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- 172.21.15.64:0/4267987778 shutdown_connections 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 --2- 172.21.15.64:0/4267987778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49cc107550 0x7f49cc1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 --2- 172.21.15.64:0/4267987778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc104c20 0x7f49cc107010 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 --2- 172.21.15.64:0/4267987778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc1022f0 0x7f49cc1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- 172.21.15.64:0/4267987778 >> 172.21.15.64:0/4267987778 conn(0x7f49cc0fbce0 msgr2=0x7f49cc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- 172.21.15.64:0/4267987778 shutdown_connections 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- 172.21.15.64:0/4267987778 wait complete. 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 Processor -- start 2024-05-09T09:06:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- start start 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 0x7f49cc101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49cc104c20 0x7f49cc1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc107550 0x7f49cc1007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10d3a0 con 0x7f49cc1022f0 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10d220 con 0x7f49cc107550 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d30bd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10d520 con 0x7f49cc104c20 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.958+0000 7f49d0e59640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 0x7f49cc101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 0x7f49cc101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57230/0 (socket says 172.21.15.64:57230) 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 -- 172.21.15.64:0/3980013010 learned_addr learned my addr 172.21.15.64:0/3980013010 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49cbfff640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49cc104c20 0x7f49cc1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d165a640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc107550 0x7f49cc1007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 -- 172.21.15.64:0/3980013010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49cc104c20 msgr2=0x7f49cc1002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49cc104c20 0x7f49cc1002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 -- 172.21.15.64:0/3980013010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc107550 msgr2=0x7f49cc1007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc107550 0x7f49cc1007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 -- 172.21.15.64:0/3980013010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc101070 con 0x7f49cc1022f0 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d165a640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc107550 0x7f49cc1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d0e59640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 0x7f49cc101c60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d950 tx=0x7f49c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0017030 con 0x7f49cc1022f0 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49c0004020 con 0x7f49cc1022f0 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00047a0 con 0x7f49cc1022f0 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc1a1bf0 con 0x7f49cc1022f0 2024-05-09T09:06:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.959+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1a2110 con 0x7f49cc1022f0 2024-05-09T09:06:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.960+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc068650 con 0x7f49cc1022f0 2024-05-09T09:06:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.961+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49c00041d0 con 0x7f49cc1022f0 2024-05-09T09:06:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.962+0000 7f49c9ffb640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49bc0a2b90 0x7f49bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.962+0000 7f49cbfff640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49bc0a2b90 0x7f49bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.962+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f49c00cedb0 con 0x7f49cc1022f0 2024-05-09T09:06:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.963+0000 7f49cbfff640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49bc0a2b90 0x7f49bc0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f49b8004770 tx=0x7f49b8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:46.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:46.966+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f49c009bad0 con 0x7f49cc1022f0 2024-05-09T09:06:47.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.152+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49cc062460 con 0x7f49cc1022f0 2024-05-09T09:06:47.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.154+0000 7f49c9ffb640 1 -- 172.21.15.64:0/3980013010 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7f49c009b3d0 con 0x7f49cc1022f0 2024-05-09T09:06:47.153 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:47.153 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:47.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49bc0a2b90 msgr2=0x7f49bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49bc0a2b90 0x7f49bc0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f49b8004770 tx=0x7f49b8000bc0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 msgr2=0x7f49cc101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 0x7f49cc101c60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d950 tx=0x7f49c000de10 comp rx=0 tx=0).stop 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 shutdown_connections 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49bc0a2b90 0x7f49bc0a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49cc107550 0x7f49cc1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49cc104c20 0x7f49cc1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 --2- 172.21.15.64:0/3980013010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49cc1022f0 0x7f49cc101c60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 >> 172.21.15.64:0/3980013010 conn(0x7f49cc0fbce0 msgr2=0x7f49cc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 shutdown_connections 2024-05-09T09:06:47.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.156+0000 7f49d30bd640 1 -- 172.21.15.64:0/3980013010 wait complete. 2024-05-09T09:06:47.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 -- 172.21.15.64:0/3991557587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 msgr2=0x7ff06c100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 --2- 172.21.15.64:0/3991557587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c100330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff06800aa10 tx=0x7ff06802b720 comp rx=0 tx=0).stop 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 -- 172.21.15.64:0/3991557587 shutdown_connections 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 --2- 172.21.15.64:0/3991557587 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff06c101a20 0x7ff06c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 --2- 172.21.15.64:0/3991557587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101060 0x7ff06c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 --2- 172.21.15.64:0/3991557587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c100330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 -- 172.21.15.64:0/3991557587 >> 172.21.15.64:0/3991557587 conn(0x7ff06c0fb4a0 msgr2=0x7ff06c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 -- 172.21.15.64:0/3991557587 shutdown_connections 2024-05-09T09:06:47.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.287+0000 7ff073e9e640 1 -- 172.21.15.64:0/3991557587 wait complete. 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 Processor -- start 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 -- start start 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c103e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff06c101060 0x7ff06c1024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101a20 0x7ff06c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10b1e0 con 0x7ff06c0fff10 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10b060 con 0x7ff06c101060 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff073e9e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10b360 con 0x7ff06c101a20 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c103e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c103e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57248/0 (socket says 172.21.15.64:57248) 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 -- 172.21.15.64:0/1112182869 learned_addr learned my addr 172.21.15.64:0/1112182869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff07243b640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101a20 0x7ff06c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 -- 172.21.15.64:0/1112182869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101a20 msgr2=0x7ff06c102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101a20 0x7ff06c102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071439640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff06c101060 0x7ff06c1024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 -- 172.21.15.64:0/1112182869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff06c101060 msgr2=0x7ff06c1024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff06c101060 0x7ff06c1024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff071c3a640 1 -- 172.21.15.64:0/1112182869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c1032a0 con 0x7ff06c0fff10 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.288+0000 7ff07243b640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101a20 0x7ff06c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.289+0000 7ff071c3a640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c103e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff068004770 tx=0x7ff068030b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.289+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06803f070 con 0x7ff06c0fff10 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.289+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff068003ca0 con 0x7ff06c0fff10 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.289+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c1a1c10 con 0x7ff06c0fff10 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.289+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068036040 con 0x7ff06c0fff10 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.289+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c1a21b0 con 0x7ff06c0fff10 2024-05-09T09:06:47.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.290+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c1a2380 con 0x7ff06c0fff10 2024-05-09T09:06:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.291+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff068042070 con 0x7ff06c0fff10 2024-05-09T09:06:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.292+0000 7ff062ffd640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff0580a2c10 0x7ff0580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.292+0000 7ff071439640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff0580a2c10 0x7ff0580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.292+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0680efa70 con 0x7ff06c0fff10 2024-05-09T09:06:47.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.293+0000 7ff071439640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff0580a2c10 0x7ff0580a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff06c103c00 tx=0x7ff05c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.297+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0680bc790 con 0x7ff06c0fff10 2024-05-09T09:06:47.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:47 smithi179 ceph-mon[34086]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T09:06:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:47 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4238129446' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T09:06:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:47 smithi179 ceph-mon[34086]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:47 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1860791873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:47 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2465757486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:47.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:47 smithi139 ceph-mon[33599]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T09:06:47.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:47 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4238129446' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T09:06:47.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:47 smithi139 ceph-mon[33599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:47.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:47 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1860791873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:47.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:47 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2465757486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:47.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.520+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff06c1a2550 con 0x7ff06c0fff10 2024-05-09T09:06:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.521+0000 7ff062ffd640 1 -- 172.21.15.64:0/1112182869 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7ff0680bc090 con 0x7ff06c0fff10 2024-05-09T09:06:47.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:47.520 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:06:45.536413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24503,"qdb_cluster":[24503]},"id":1}]} 2024-05-09T09:06:47.520 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 17 2024-05-09T09:06:47.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff0580a2c10 msgr2=0x7ff0580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff0580a2c10 0x7ff0580a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff06c103c00 tx=0x7ff05c005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 msgr2=0x7ff06c103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c103e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff068004770 tx=0x7ff068030b40 comp rx=0 tx=0).stop 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 shutdown_connections 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff0580a2c10 0x7ff0580a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff06c101a20 0x7ff06c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff06c101060 0x7ff06c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 --2- 172.21.15.64:0/1112182869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff06c0fff10 0x7ff06c103e90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 >> 172.21.15.64:0/1112182869 conn(0x7ff06c0fb4a0 msgr2=0x7ff06c106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 shutdown_connections 2024-05-09T09:06:47.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.523+0000 7ff073e9e640 1 -- 172.21.15.64:0/1112182869 wait complete. 2024-05-09T09:06:47.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:47 smithi064 ceph-mon[30149]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T09:06:47.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4238129446' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T09:06:47.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:47 smithi064 ceph-mon[30149]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:06:47.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1860791873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:47.532 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2465757486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:47.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 -- 172.21.15.64:0/169937711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc102330 msgr2=0x7f2bbc108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/169937711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc102330 0x7f2bbc108bc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009ca0 tx=0x7f2bb402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 -- 172.21.15.64:0/169937711 shutdown_connections 2024-05-09T09:06:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/169937711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc102330 0x7f2bbc108bc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/169937711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 0x7f2bbc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/169937711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bbc100780 0x7f2bbc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 -- 172.21.15.64:0/169937711 >> 172.21.15.64:0/169937711 conn(0x7f2bbc0fbd10 msgr2=0x7f2bbc0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 -- 172.21.15.64:0/169937711 shutdown_connections 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 -- 172.21.15.64:0/169937711 wait complete. 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.639+0000 7f2bc1e95640 1 Processor -- start 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 -- start start 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 0x7f2bbc06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 0x7f2bbc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bbc102330 0x7f2bbc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc10c300 con 0x7f2bbc100780 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc10c180 con 0x7f2bbc102330 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bc1e95640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc10c480 con 0x7f2bbc101970 2024-05-09T09:06:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 0x7f2bbc06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 0x7f2bbc06aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57278/0 (socket says 172.21.15.64:57278) 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/1254139444 learned_addr learned my addr 172.21.15.64:0/1254139444 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbbfff640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bbc102330 0x7f2bbc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbaffd640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 0x7f2bbc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/1254139444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 msgr2=0x7f2bbc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 0x7f2bbc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/1254139444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bbc102330 msgr2=0x7f2bbc19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bbc102330 0x7f2bbc19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/1254139444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc19f6f0 con 0x7f2bbc100780 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.640+0000 7f2bbaffd640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 0x7f2bbc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.641+0000 7f2bbb7fe640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 0x7f2bbc06aa00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9c0 tx=0x7f2bac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.641+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00cd80 con 0x7f2bbc100780 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.641+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bac0035f0 con 0x7f2bbc100780 2024-05-09T09:06:47.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.641+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc19f980 con 0x7f2bbc100780 2024-05-09T09:06:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.641+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac010690 con 0x7f2bbc100780 2024-05-09T09:06:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.641+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc19ff20 con 0x7f2bbc100780 2024-05-09T09:06:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.642+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc107aa0 con 0x7f2bbc100780 2024-05-09T09:06:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.643+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2bac0037a0 con 0x7f2bbc100780 2024-05-09T09:06:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.644+0000 7f2bb8ff9640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ba40a2c10 0x7f2ba40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.644+0000 7f2bbaffd640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ba40a2c10 0x7f2ba40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.644+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bac0cf660 con 0x7f2bbc100780 2024-05-09T09:06:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.644+0000 7f2bbaffd640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ba40a2c10 0x7f2ba40a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc06a770 tx=0x7f2bb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.648+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2bac09c380 con 0x7f2bbc100780 2024-05-09T09:06:47.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.834+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bbc0624d0 con 0x7f2bbc100780 2024-05-09T09:06:47.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.836+0000 7f2bb8ff9640 1 -- 172.21.15.64:0/1254139444 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7f2bac09bc80 con 0x7f2bbc100780 2024-05-09T09:06:47.835 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:47.835 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ba40a2c10 msgr2=0x7f2ba40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ba40a2c10 0x7f2ba40a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc06a770 tx=0x7f2bb0005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 msgr2=0x7f2bbc06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 0x7f2bbc06aa00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9c0 tx=0x7f2bac00ee80 comp rx=0 tx=0).stop 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 shutdown_connections 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ba40a2c10 0x7f2ba40a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bbc102330 0x7f2bbc19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bbc101970 0x7f2bbc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 --2- 172.21.15.64:0/1254139444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bbc100780 0x7f2bbc06aa00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 >> 172.21.15.64:0/1254139444 conn(0x7f2bbc0fbd10 msgr2=0x7f2bbc104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 shutdown_connections 2024-05-09T09:06:47.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.838+0000 7f2bc1e95640 1 -- 172.21.15.64:0/1254139444 wait complete. 2024-05-09T09:06:47.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T09:06:40.458278+0000', 'modified': '2024-05-09T09:06:45.536413+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6839/1823707244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 1823707244}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 1823707244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24503, 'qdb_cluster': [24503]} 2024-05-09T09:06:47.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T09:06:47.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 -- 172.21.15.64:0/3640791915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950100790 msgr2=0x7f4950100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 --2- 172.21.15.64:0/3640791915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950100790 0x7f4950100bb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 -- 172.21.15.64:0/3640791915 shutdown_connections 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 --2- 172.21.15.64:0/3640791915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4950102340 0x7f4950108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 --2- 172.21.15.64:0/3640791915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4950101980 0x7f4950101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 --2- 172.21.15.64:0/3640791915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950100790 0x7f4950100bb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 -- 172.21.15.64:0/3640791915 >> 172.21.15.64:0/3640791915 conn(0x7f49500fbd20 msgr2=0x7f49500fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 -- 172.21.15.64:0/3640791915 shutdown_connections 2024-05-09T09:06:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.955+0000 7f4958095640 1 -- 172.21.15.64:0/3640791915 wait complete. 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 Processor -- start 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 -- start start 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4950100790 0x7f4950197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 0x7f4950198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4950102340 0x7f495019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f495010c2e0 con 0x7f4950101980 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f495010c160 con 0x7f4950102340 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4958095640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f495010c460 con 0x7f4950100790 2024-05-09T09:06:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 0x7f4950198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 0x7f4950198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57298/0 (socket says 172.21.15.64:57298) 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 -- 172.21.15.64:0/1432346874 learned_addr learned my addr 172.21.15.64:0/1432346874 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4956632640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4950102340 0x7f495019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955e31640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4950100790 0x7f4950197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 -- 172.21.15.64:0/1432346874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4950100790 msgr2=0x7f4950197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4950100790 0x7f4950197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 -- 172.21.15.64:0/1432346874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4950102340 msgr2=0x7f495019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4950102340 0x7f495019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.956+0000 7f4955630640 1 -- 172.21.15.64:0/1432346874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495019f6d0 con 0x7f4950101980 2024-05-09T09:06:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.957+0000 7f4955630640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 0x7f4950198040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f494000d950 tx=0x7f494000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.957+0000 7f4946ffd640 1 -- 172.21.15.64:0/1432346874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940017070 con 0x7f4950101980 2024-05-09T09:06:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.957+0000 7f4946ffd640 1 -- 172.21.15.64:0/1432346874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4940009c30 con 0x7f4950101980 2024-05-09T09:06:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.957+0000 7f4946ffd640 1 -- 172.21.15.64:0/1432346874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000f620 con 0x7f4950101980 2024-05-09T09:06:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.957+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495019f9c0 con 0x7f4950101980 2024-05-09T09:06:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.957+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495019ff10 con 0x7f4950101980 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 Processor -- start 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f4946ffd640 1 -- 172.21.15.64:0/1432346874 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49400109a0 con 0x7f4950101980 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 -- start start 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x5611584420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x5611584471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x561158449cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5611583a14d0 con 0x5611584426c0 2024-05-09T09:06:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611583493e0 con 0x561158441cd0 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f495ad9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56115839d490 con 0x561158447830 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.960+0000 7f4955e31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x5611584471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x5611584471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57310/0 (socket says 172.21.15.64:57310) 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 -- 172.21.15.64:0/4220256278 learned_addr learned my addr 172.21.15.64:0/4220256278 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4956632640 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x5611584420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955630640 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x561158449cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 -- 172.21.15.64:0/4220256278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 msgr2=0x561158449cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x561158449cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 -- 172.21.15.64:0/4220256278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 msgr2=0x5611584420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x5611584420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 -- 172.21.15.64:0/4220256278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611580b52d0 con 0x5611584426c0 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4946ffd640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f493c0a2b90 0x7f493c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4955e31640 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x5611584471c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919611956062f698 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4956632640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f493c0a2b90 0x7f493c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f4946ffd640 1 -- 172.21.15.64:0/1432346874 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f49400cee00 con 0x7f4950101980 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f493bfff640 1 -- 172.21.15.64:0/4220256278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c035030 con 0x5611584426c0 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f493bfff640 1 -- 172.21.15.64:0/4220256278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f494c040070 con 0x5611584426c0 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.961+0000 7f493bfff640 1 -- 172.21.15.64:0/4220256278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c031b20 con 0x5611584426c0 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 -- 172.21.15.64:0/4220256278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 msgr2=0x5611584471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x5611584471c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c004390 comp rx=0 tx=0).stop 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f4956632640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f493c0a2b90 0x7f493c0a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x561158442f20 tx=0x7f4948007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 -- 172.21.15.64:0/4220256278 shutdown_connections 2024-05-09T09:06:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x561158449cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x5611584471c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 --2- 172.21.15.64:0/4220256278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x5611584420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 -- 172.21.15.64:0/4220256278 >> 172.21.15.64:0/4220256278 conn(0x561158438f30 msgr2=0x561158439330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 -- 172.21.15.64:0/4220256278 shutdown_connections 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 -- 172.21.15.64:0/4220256278 wait complete. 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.962+0000 7f495ad9c740 1 -- start start 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 Processor -- start 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x561158453590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56115844a780 con 0x5611584426c0 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56115835c8e0 con 0x561158447830 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56115844a900 con 0x561158441cd0 2024-05-09T09:06:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f495ad9c740 10 client.0 _ll_register_callbacks cb 0x5611583299c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57322/0 (socket says 172.21.15.64:57322) 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 -- 172.21.15.64:0/2898505681 learned_addr learned my addr 172.21.15.64:0/2898505681 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4956632640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x561158453590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955e31640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 -- 172.21.15.64:0/2898505681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 msgr2=0x56115844c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 -- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 msgr2=0x561158453590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x561158453590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.963+0000 7f4955630640 1 -- 172.21.15.64:0/2898505681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115817f6b0 con 0x5611584426c0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f4955630640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844c550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4940000c00 tx=0x7f49400d24d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a056cdaec45a7199 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f49397fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:06:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f49397fa640 1 -- 172.21.15.64:0/2898505681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940020070 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f49397fa640 1 -- 172.21.15.64:0/2898505681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f49400ccb80 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f494c040070 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f49397fa640 1 -- 172.21.15.64:0/2898505681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f494009b010 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 10 client.14763 fetch_fsmap learned FSMap version 17 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x561158456630 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f49397fa640 1 -- 172.21.15.64:0/2898505681 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f494c040070 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f49397fa640 1 -- 172.21.15.64:0/2898505681 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f4940097220 con 0x5611584426c0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 10 client.14763 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 20 client.14763 populate_metadata read hostname 'smithi064' 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x56115845a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 4 client.14763 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.964+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611583aec00 con 0x561158458ca0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.965+0000 7f4955e31640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x56115845a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.965+0000 7f4955e31640 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x56115845a110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0a940dbad7f7c server_cookie=2f8352b3ef48b811 in_seq=0 out_seq=0 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.965+0000 7f49397fa640 10 client.14763 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f49397fa640 1 -- 172.21.15.64:0/2898505681 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611583aec00 con 0x561158458ca0 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f49397fa640 10 client.14763 handle_command_reply: tid=0 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 client.14763 shutdown 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 msgr2=0x56115844c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844c550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4940000c00 tx=0x7f49400d24d0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 shutdown_connections 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x56115845a110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x561158453590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f49397fa640 0 client.14763 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844c550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2898505681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.966+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 >> 172.21.15.64:0/2898505681 conn(0x561158438f30 msgr2=0x561158438000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.967+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 shutdown_connections 2024-05-09T09:06:47.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.967+0000 7f495ad9c740 1 -- 172.21.15.64:0/2898505681 wait complete. 2024-05-09T09:06:47.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.967+0000 7f495ad9c740 20 client.14763 trim_cache size 0 max 16384 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 Processor -- start 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 -- start start 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x561158453550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x56115844b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561158458450 con 0x5611584426c0 2024-05-09T09:06:47.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561158458b20 con 0x561158447830 2024-05-09T09:06:47.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f495ad9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5611584591f0 con 0x561158441cd0 2024-05-09T09:06:47.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x561158453550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x561158453550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57336/0 (socket says 172.21.15.64:57336) 2024-05-09T09:06:47.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 -- 172.21.15.64:0/3097760775 learned_addr learned my addr 172.21.15.64:0/3097760775 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:47.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955e31640 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4956632640 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x56115844b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 -- 172.21.15.64:0/3097760775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 msgr2=0x56115844b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 -- 172.21.15.64:0/3097760775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 msgr2=0x56115844b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x56115844b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 -- 172.21.15.64:0/3097760775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561158442f50 con 0x5611584426c0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f4955630640 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x561158453550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f494009b460 tx=0x7f49400999e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4547902ba714e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.985+0000 7f493b7fe640 1 -- 172.21.15.64:0/3097760775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940014020 con 0x5611584426c0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f493b7fe640 1 -- 172.21.15.64:0/3097760775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49400cce40 con 0x5611584426c0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f493b7fe640 1 -- 172.21.15.64:0/3097760775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940020070 con 0x5611584426c0 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 -- 172.21.15.64:0/3097760775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 msgr2=0x561158453550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:47.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x561158453550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f494009b460 tx=0x7f49400999e0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 -- 172.21.15.64:0/3097760775 shutdown_connections 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158447830 0x56115844b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x561158453550 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 --2- 172.21.15.64:0/3097760775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158441cd0 0x56115844b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 -- 172.21.15.64:0/3097760775 >> 172.21.15.64:0/3097760775 conn(0x561158438f30 msgr2=0x56115844c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 -- 172.21.15.64:0/3097760775 shutdown_connections 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 -- 172.21.15.64:0/3097760775 wait complete. 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.986+0000 7f495ad9c740 1 -- start start 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 Processor -- start 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x56115844f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x5611584527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561158459f90 con 0x5611584426c0 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611584598c0 con 0x561158441cd0 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56115845a660 con 0x561158447830 2024-05-09T09:06:48.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f495ad9c740 10 client.0 _ll_register_callbacks cb 0x7f49400970b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57342/0 (socket says 172.21.15.64:57342) 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 -- 172.21.15.64:0/2502574986 learned_addr learned my addr 172.21.15.64:0/2502574986 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955e31640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x56115844f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4956632640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x5611584527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 -- 172.21.15.64:0/2502574986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 msgr2=0x5611584527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x5611584527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 -- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 msgr2=0x56115844f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x56115844f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4955630640 1 -- 172.21.15.64:0/2502574986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561158452e20 con 0x5611584426c0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.987+0000 7f4956632640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x5611584527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f4955630640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844fc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49400a3b60 tx=0x7f494009f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc683fd24edf68f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f49397fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:06:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f49397fa640 1 -- 172.21.15.64:0/2502574986 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400d7060 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f49397fa640 1 -- 172.21.15.64:0/2502574986 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f49400104b0 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611583a8f50 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f49397fa640 1 -- 172.21.15.64:0/2502574986 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4940014070 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 10 client.14769 fetch_fsmap learned FSMap version 17 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5611584bf2b0 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f49397fa640 1 -- 172.21.15.64:0/2502574986 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5611583a8f50 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f49397fa640 1 -- 172.21.15.64:0/2502574986 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f494009a390 con 0x5611584426c0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 10 client.14769 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 10 client.14769 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 20 client.14769 populate_metadata read hostname 'smithi064' 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x5611584c0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 4 client.14769 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "status"}) -- 0x561158326b80 con 0x561158458ca0 2024-05-09T09:06:48.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.988+0000 7f4955e31640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x5611584c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f4955e31640 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x5611584c0530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ae62de2b017d45 server_cookie=2c49b3de8459cc4 in_seq=0 out_seq=0 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f49397fa640 10 client.14769 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f49397fa640 1 -- 172.21.15.64:0/2502574986 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561158326b80 con 0x561158458ca0 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f49397fa640 10 client.14769 handle_command_reply: tid=0 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 client.14769 shutdown 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 msgr2=0x56115844fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844fc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49400a3b60 tx=0x7f494009f7d0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 shutdown_connections 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561158458ca0 0x5611584c0530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561158447830 0x5611584527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611584426c0 0x56115844fc20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f49397fa640 0 client.14769 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 --2- 172.21.15.64:0/2502574986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561158441cd0 0x56115844f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 >> 172.21.15.64:0/2502574986 conn(0x561158438f30 msgr2=0x56115844b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:48.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 shutdown_connections 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 1 -- 172.21.15.64:0/2502574986 wait complete. 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.989+0000 7f495ad9c740 20 client.14769 trim_cache size 0 max 16384 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24503, 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 6.8679393280000003, 2024-05-09T09:06:48.004 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 17, 2024-05-09T09:06:48.005 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 90, 2024-05-09T09:06:48.005 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:06:48.005 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 12.919386991 2024-05-09T09:06:48.005 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f493c0a2b90 msgr2=0x7f493c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f493c0a2b90 0x7f493c0a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x561158442f20 tx=0x7f4948007520 comp rx=0 tx=0).stop 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 msgr2=0x7f4950198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 0x7f4950198040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f494000d950 tx=0x7f494000de10 comp rx=0 tx=0).stop 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 shutdown_connections 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f493c0a2b90 0x7f493c0a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4950102340 0x7f495019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101980 0x7f4950198040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 --2- 172.21.15.64:0/1432346874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4950100790 0x7f4950197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:48.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 >> 172.21.15.64:0/1432346874 conn(0x7f49500fbd20 msgr2=0x7f49500fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:48.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 shutdown_connections 2024-05-09T09:06:48.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:47.990+0000 7f4958095640 1 -- 172.21.15.64:0/1432346874 wait complete. 2024-05-09T09:06:48.007 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T09:06:48.007 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:48.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-09T09:06:48.272 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:06:48.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:48 smithi064 ceph-mon[30149]: pgmap v296: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-05-09T09:06:48.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3980013010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:48 smithi064 ceph-mon[30149]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:06:48.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:48 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:06:48.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1112182869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1254139444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:48 smithi179 ceph-mon[34086]: pgmap v296: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-05-09T09:06:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:48 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3980013010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:48 smithi179 ceph-mon[34086]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:06:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:48 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:06:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:48 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1112182869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:48 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1254139444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:48 smithi139 ceph-mon[33599]: pgmap v296: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-05-09T09:06:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:48 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3980013010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:48 smithi139 ceph-mon[33599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T09:06:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:48 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:06:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:48 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1112182869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:06:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:48 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1254139444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:50.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 -- 172.21.15.64:0/3355913851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101be0 msgr2=0x7f80d810e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:50.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 --2- 172.21.15.64:0/3355913851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101be0 0x7f80d810e0d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009990 tx=0x7f80d0009ca0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 -- 172.21.15.64:0/3355913851 shutdown_connections 2024-05-09T09:06:50.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 --2- 172.21.15.64:0/3355913851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101be0 0x7f80d810e0d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 --2- 172.21.15.64:0/3355913851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80d8107290 0x7f80d8107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 --2- 172.21.15.64:0/3355913851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d8101220 0x7f80d81016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 -- 172.21.15.64:0/3355913851 >> 172.21.15.64:0/3355913851 conn(0x7f80d80781d0 msgr2=0x7f80d80785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 -- 172.21.15.64:0/3355913851 shutdown_connections 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.021+0000 7f80dde56700 1 -- 172.21.15.64:0/3355913851 wait complete. 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 Processor -- start 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 -- start start 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 0x7f80d8102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80d8101be0 0x7f80d8103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d8107290 0x7f80d8103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8111420 con 0x7f80d8101220 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80d81112a0 con 0x7f80d8107290 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80dde56700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f80d81115a0 con 0x7f80d8101be0 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.022+0000 7f80d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 0x7f80d8102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:50.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 0x7f80d8102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57370/0 (socket says 172.21.15.64:57370) 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 -- 172.21.15.64:0/712902697 learned_addr learned my addr 172.21.15.64:0/712902697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d7fff700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d8107290 0x7f80d8103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80ceffd700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80d8101be0 0x7f80d8103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 -- 172.21.15.64:0/712902697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80d8101be0 msgr2=0x7f80d8103440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80d8101be0 0x7f80d8103440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 -- 172.21.15.64:0/712902697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d8107290 msgr2=0x7f80d8103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d8107290 0x7f80d8103980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 -- 172.21.15.64:0/712902697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d81a6f00 con 0x7f80d8101220 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d77fe700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 0x7f80d8102f00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80c8007f10 tx=0x7f80c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c800dc90 con 0x7f80d8101220 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d81a71f0 con 0x7f80d8101220 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80c800de30 con 0x7f80d8101220 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80133f0 con 0x7f80d8101220 2024-05-09T09:06:50.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.023+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d81a7730 con 0x7f80d8101220 2024-05-09T09:06:50.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.024+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d8069010 con 0x7f80d8101220 2024-05-09T09:06:50.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.026+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80c801a020 con 0x7f80d8101220 2024-05-09T09:06:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.027+0000 7f80d57fa700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f80c40a2c20 0x7f80c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.027+0000 7f80ceffd700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f80c40a2c20 0x7f80c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.027+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f80c80ceab0 con 0x7f80d8101220 2024-05-09T09:06:50.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.027+0000 7f80ceffd700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f80c40a2c20 0x7f80c40a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005fd0 tx=0x7f80c000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:50.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.031+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80c8010070 con 0x7f80d8101220 2024-05-09T09:06:50.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.284+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80d81a7990 con 0x7f80d8101220 2024-05-09T09:06:50.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.284+0000 7f80d57fa700 1 -- 172.21.15.64:0/712902697 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f80c809b7d0 con 0x7f80d8101220 2024-05-09T09:06:50.283 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T09:06:50.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f80c40a2c20 msgr2=0x7f80c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:50.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f80c40a2c20 0x7f80c40a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005fd0 tx=0x7f80c000a560 comp rx=0 tx=0).stop 2024-05-09T09:06:50.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 msgr2=0x7f80d8102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:50.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 0x7f80d8102f00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80c8007f10 tx=0x7f80c800d3b0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 shutdown_connections 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d8107290 0x7f80d8103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80d8101220 0x7f80d8102f00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f80c40a2c20 0x7f80c40a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 --2- 172.21.15.64:0/712902697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80d8101be0 0x7f80d8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 >> 172.21.15.64:0/712902697 conn(0x7f80d80781d0 msgr2=0x7f80d8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 shutdown_connections 2024-05-09T09:06:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:50.286+0000 7f80dde56700 1 -- 172.21.15.64:0/712902697 wait complete. 2024-05-09T09:06:50.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:50 smithi179 ceph-mon[34086]: pgmap v297: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 344 B/s wr, 4 op/s 2024-05-09T09:06:50.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:50 smithi139 ceph-mon[33599]: pgmap v297: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 344 B/s wr, 4 op/s 2024-05-09T09:06:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:50 smithi064 ceph-mon[30149]: pgmap v297: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 344 B/s wr, 4 op/s 2024-05-09T09:06:51.130 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T09:06:51.130 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T09:06:51.130 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-05-09T09:06:51.131 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T09:06:51.131 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:51.131 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T09:06:51.131 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T09:06:51.131 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T09:06:51.131 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T09:06:51.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:51.132 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-05-09T09:06:51.152 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T09:06:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T09:06:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T09:06:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:06:51.152 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T09:06:51.153 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42253sec preferred_lft 42253sec 2024-05-09T09:06:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-05-09T09:06:51.154 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:06:51.154 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T09:06:51.154 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:06:51.154 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:06:51.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-05-09T09:06:51.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-05-09T09:06:51.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-05-09T09:06:51.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T09:06:51.155 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:06:51.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:06:51.349 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:51 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/712902697' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:06:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:51.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:51.360 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T09:06:51.385 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-05-09T09:06:51.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:51.387 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-05-09T09:06:51.464 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-05-09T09:06:51.464 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T09:06:51.464 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T09:06:51.464 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T09:06:51.464 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T09:06:51.464 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:06:51.465 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:06:51.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T09:06:51.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T09:06:51.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T09:06:51.465 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:06:51.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:51 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/712902697' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:06:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:06:51.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:51 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/712902697' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:06:51.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:51.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:51.782 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-09T09:06:51.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:51.798 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-05-09T09:06:51.854 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:06:51.854 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:06:51.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:06:51.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T09:06:51.854 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:06:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:06:51.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:51.975 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T09:06:51.975 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:06:51.975 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:06:51.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T09:06:51.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T09:06:51.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T09:06:51.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T09:06:51.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T09:06:51.976 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:06:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:06:52.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:52.129 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:06:52.130 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:06:52.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-05-09T09:06:52.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T09:06:52.130 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:06:52.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:06:52.203 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:52 smithi064 ceph-mon[30149]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 294 B/s wr, 3 op/s 2024-05-09T09:06:52.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:52.213 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T09:06:52.213 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:06:52.214 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:52.229 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T09:06:52.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:06:52.229 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:52.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-05-09T09:06:52.353 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-05-09T09:06:52.410 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:06:52.411 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/772ca47141e89a435b1d4995ca7ca0255eeb3e7e3688439228116871ac4d9c3b/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5fd57eec47dd9362180c164c9948523587125bcef039d506fd3669295e466963/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/cc30f61bab2b46c2740ffa4e83d1465ed8121ce8a19bdf5854792c73278a4b99/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/682bf516c81da3d3ea12bc411010a33978c3943b58d077694c76780c78065648/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/66eeb9b9f88ac7c0171f72bc86ef7dd1bd98829e92f8b1170ec7e931d97a22bf/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/39d3a895bf51eac157ab546c2811481aed20bb20e2d1f7f2b06d93af2db82269/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/efb2205fd00fd12c2bb8b9f0f71d4d65e45a88d143b0e0a79ab4a9c2dd4f6a0e/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7b5fd69a89f54593dcb5250ec9a0353112a929e54486fbf644a57c5b2bc1057d/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a69532b418b1328df84cbcdd5bca8a0a218491254d483d92be609490f02b12ee/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/aca1353679f4b3104cf0788d46b2eb0aa5c85c46745ddf8949a56baadf7ec1d6/merged 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:06:52.412 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:06:52.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:52.413 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-05-09T09:06:52.465 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T09:06:52.466 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T09:06:52.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:52 smithi179 ceph-mon[34086]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 294 B/s wr, 3 op/s 2024-05-09T09:06:52.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-05-09T09:06:52.534 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:06:52.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:52 smithi139 ceph-mon[33599]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 294 B/s wr, 3 op/s 2024-05-09T09:06:52.591 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-05-09T09:06:52.591 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-05-09T09:06:52.591 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-05-09T09:06:52.591 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-05-09T09:06:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/772ca47141e89a435b1d4995ca7ca0255eeb3e7e3688439228116871ac4d9c3b/merged 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5fd57eec47dd9362180c164c9948523587125bcef039d506fd3669295e466963/merged 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/cc30f61bab2b46c2740ffa4e83d1465ed8121ce8a19bdf5854792c73278a4b99/merged 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/682bf516c81da3d3ea12bc411010a33978c3943b58d077694c76780c78065648/merged 2024-05-09T09:06:52.593 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/66eeb9b9f88ac7c0171f72bc86ef7dd1bd98829e92f8b1170ec7e931d97a22bf/merged 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/39d3a895bf51eac157ab546c2811481aed20bb20e2d1f7f2b06d93af2db82269/merged 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/efb2205fd00fd12c2bb8b9f0f71d4d65e45a88d143b0e0a79ab4a9c2dd4f6a0e/merged 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7b5fd69a89f54593dcb5250ec9a0353112a929e54486fbf644a57c5b2bc1057d/merged 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a69532b418b1328df84cbcdd5bca8a0a218491254d483d92be609490f02b12ee/merged 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/aca1353679f4b3104cf0788d46b2eb0aa5c85c46745ddf8949a56baadf7ec1d6/merged 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:06:52.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:52.594 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-05-09T09:06:52.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:2024-05-09T09:06:52.623+0000 7fcdaeb8b480 -1 init, newargv = 0x560d6b5ba250 newargc=15 2024-05-09T09:06:52.623 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[72326]: starting ceph client 2024-05-09T09:06:52.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[72326]: starting fuse 2024-05-09T09:06:52.639 INFO:teuthology.orchestra.run.smithi064.stdout:114 2024-05-09T09:06:52.639 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:06:52.639 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-05-09T09:06:52.640 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-05-09T09:06:52.641 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T09:06:52.641 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:06:52.641 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.admin")) 2024-05-09T09:06:52.641 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-05-09T09:06:52.760 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.admin.72326.asok 2024-05-09T09:06:52.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:52.771 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72326.asok 2024-05-09T09:06:52.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:52.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72326.asok status 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "admin", 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T09:06:52.909 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "72326", 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 14781, 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 14781 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:06:52.910 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3996013170 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3996013170 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.14781 192.168.0.1:0/3996013170", 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3996013170", 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 17, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 90, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-05-09T09:06:52.911 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:06:52.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:52.917 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-05-09T09:06:52.917 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:52.934 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-05-09T09:06:52.934 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:52.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:52.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:53.001 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T09:06:53.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T09:06:53.110 INFO:teuthology.orchestra.run.smithi064.stdout:# file: . 2024-05-09T09:06:53.111 INFO:teuthology.orchestra.run.smithi064.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T09:06:53.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:53.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:06:53.117 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:06:53.117 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:06:53.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 -- 172.21.15.64:0/2892248981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 msgr2=0x7fb30c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 --2- 172.21.15.64:0/2892248981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c104750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009900 tx=0x7fb2fc02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 -- 172.21.15.64:0/2892248981 shutdown_connections 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 --2- 172.21.15.64:0/2892248981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 0x7fb30c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 --2- 172.21.15.64:0/2892248981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 0x7fb30c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 --2- 172.21.15.64:0/2892248981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c104750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 -- 172.21.15.64:0/2892248981 >> 172.21.15.64:0/2892248981 conn(0x7fb30c0fbd10 msgr2=0x7fb30c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 -- 172.21.15.64:0/2892248981 shutdown_connections 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.280+0000 7fb311df4640 1 -- 172.21.15.64:0/2892248981 wait complete. 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 Processor -- start 2024-05-09T09:06:53.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 -- start start 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 0x7fb30c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 0x7fb30c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10d3d0 con 0x7fb30c102360 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10d250 con 0x7fb30c104c90 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb311df4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10d550 con 0x7fb30c1075c0 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.281+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57384/0 (socket says 172.21.15.64:57384) 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 -- 172.21.15.64:0/144045433 learned_addr learned my addr 172.21.15.64:0/144045433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30bfff640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 0x7fb30c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30affd640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 0x7fb30c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 -- 172.21.15.64:0/144045433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 msgr2=0x7fb30c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 0x7fb30c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 -- 172.21.15.64:0/144045433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 msgr2=0x7fb30c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 0x7fb30c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 -- 172.21.15.64:0/144045433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c19f720 con 0x7fb30c102360 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30affd640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 0x7fb30c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30bfff640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 0x7fb30c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb30b7fe640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c197b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc004770 tx=0x7fb2fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc03f030 con 0x7fb30c102360 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2fc003cb0 con 0x7fb30c102360 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c19f9b0 con 0x7fb30c102360 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc035070 con 0x7fb30c102360 2024-05-09T09:06:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.282+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c19fea0 con 0x7fb30c102360 2024-05-09T09:06:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.283+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c0fd800 con 0x7fb30c102360 2024-05-09T09:06:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.285+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb2fc042030 con 0x7fb30c102360 2024-05-09T09:06:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.286+0000 7fb308ff9640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2f40a2b90 0x7fb2f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.286+0000 7fb30affd640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2f40a2b90 0x7fb2f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.286+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb2fc0f0030 con 0x7fb30c102360 2024-05-09T09:06:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.286+0000 7fb30affd640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2f40a2b90 0x7fb2f40a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb30c199010 tx=0x7fb300005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.289+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb2fc0bcd50 con 0x7fb30c102360 2024-05-09T09:06:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.474+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb30c02d390 con 0x7fb30c102360 2024-05-09T09:06:53.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.476+0000 7fb308ff9640 1 -- 172.21.15.64:0/144045433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7fb2fc0bc650 con 0x7fb30c102360 2024-05-09T09:06:53.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:53.475 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:53.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2f40a2b90 msgr2=0x7fb2f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2f40a2b90 0x7fb2f40a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb30c199010 tx=0x7fb300005e50 comp rx=0 tx=0).stop 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 msgr2=0x7fb30c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c197b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc004770 tx=0x7fb2fc0042f0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 shutdown_connections 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2f40a2b90 0x7fb2f40a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb30c1075c0 0x7fb30c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb30c104c90 0x7fb30c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 --2- 172.21.15.64:0/144045433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30c102360 0x7fb30c197b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 >> 172.21.15.64:0/144045433 conn(0x7fb30c0fbd10 msgr2=0x7fb30c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:53.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 shutdown_connections 2024-05-09T09:06:53.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.478+0000 7fb311df4640 1 -- 172.21.15.64:0/144045433 wait complete. 2024-05-09T09:06:53.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:06:53.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 -- 172.21.15.64:0/2227159130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84107570 msgr2=0x7fac841099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/2227159130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84107570 0x7fac841099c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009ca0 tx=0x7fac7c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 -- 172.21.15.64:0/2227159130 shutdown_connections 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/2227159130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84107570 0x7fac841099c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/2227159130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84104c40 0x7fac84107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/2227159130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac84102310 0x7fac84104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 -- 172.21.15.64:0/2227159130 >> 172.21.15.64:0/2227159130 conn(0x7fac840fbd00 msgr2=0x7fac840fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 -- 172.21.15.64:0/2227159130 shutdown_connections 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.599+0000 7fac8a1c4640 1 -- 172.21.15.64:0/2227159130 wait complete. 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 Processor -- start 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 -- start start 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84102310 0x7fac84197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 0x7fac84197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac84107570 0x7fac8419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fac8410d3c0 con 0x7fac84104c40 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fac8410d240 con 0x7fac84102310 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac8a1c4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fac8410d540 con 0x7fac84107570 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 0x7fac84197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 0x7fac84197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57394/0 (socket says 172.21.15.64:57394) 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac82ffd640 1 -- 172.21.15.64:0/1008518370 learned_addr learned my addr 172.21.15.64:0/1008518370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:53.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.600+0000 7fac837fe640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84102310 0x7fac84197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac83fff640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac84107570 0x7fac8419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac82ffd640 1 -- 172.21.15.64:0/1008518370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac84107570 msgr2=0x7fac8419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac82ffd640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac84107570 0x7fac8419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac82ffd640 1 -- 172.21.15.64:0/1008518370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84102310 msgr2=0x7fac84197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac82ffd640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84102310 0x7fac84197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac82ffd640 1 -- 172.21.15.64:0/1008518370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8419f6c0 con 0x7fac84104c40 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac837fe640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84102310 0x7fac84197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac82ffd640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 0x7fac84197fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac7800d770 tx=0x7fac7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78017070 con 0x7fac84104c40 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8419f9b0 con 0x7fac84104c40 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac78009c30 con 0x7fac84104c40 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7800f620 con 0x7fac84104c40 2024-05-09T09:06:53.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.601+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8419ff00 con 0x7fac84104c40 2024-05-09T09:06:53.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.602+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84068aa0 con 0x7fac84104c40 2024-05-09T09:06:53.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.603+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fac7801a070 con 0x7fac84104c40 2024-05-09T09:06:53.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.604+0000 7fac80ff9640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac6c0a2b90 0x7fac6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.604+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fac780cee60 con 0x7fac84104c40 2024-05-09T09:06:53.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.604+0000 7fac837fe640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac6c0a2b90 0x7fac6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.605+0000 7fac837fe640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac6c0a2b90 0x7fac6c0a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c20 tx=0x7fac74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:53.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.608+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fac78009de0 con 0x7fac84104c40 2024-05-09T09:06:53.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.793+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac840624f0 con 0x7fac84104c40 2024-05-09T09:06:53.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.793+0000 7fac80ff9640 1 -- 172.21.15.64:0/1008518370 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fac7809bb80 con 0x7fac84104c40 2024-05-09T09:06:53.792 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:06:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac6c0a2b90 msgr2=0x7fac6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac6c0a2b90 0x7fac6c0a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c20 tx=0x7fac74005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 msgr2=0x7fac84197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 0x7fac84197fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac7800d770 tx=0x7fac7800dc30 comp rx=0 tx=0).stop 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 shutdown_connections 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac6c0a2b90 0x7fac6c0a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac84107570 0x7fac8419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac84104c40 0x7fac84197fe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 --2- 172.21.15.64:0/1008518370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84102310 0x7fac84197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 >> 172.21.15.64:0/1008518370 conn(0x7fac840fbd00 msgr2=0x7fac84102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 shutdown_connections 2024-05-09T09:06:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.795+0000 7fac8a1c4640 1 -- 172.21.15.64:0/1008518370 wait complete. 2024-05-09T09:06:53.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:06:53.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 -- 172.21.15.64:0/3157877121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c100790 msgr2=0x7f3b8c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 --2- 172.21.15.64:0/3157877121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c100790 0x7f3b8c100bb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009900 tx=0x7f3b8802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:53.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 -- 172.21.15.64:0/3157877121 shutdown_connections 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 --2- 172.21.15.64:0/3157877121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c102340 0x7f3b8c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 --2- 172.21.15.64:0/3157877121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8c101980 0x7f3b8c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 --2- 172.21.15.64:0/3157877121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c100790 0x7f3b8c100bb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 -- 172.21.15.64:0/3157877121 >> 172.21.15.64:0/3157877121 conn(0x7f3b8c0fbd20 msgr2=0x7f3b8c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 -- 172.21.15.64:0/3157877121 shutdown_connections 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.914+0000 7f3b94664640 1 -- 172.21.15.64:0/3157877121 wait complete. 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 Processor -- start 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 -- start start 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8c100790 0x7f3b8c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c101980 0x7f3b8c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 0x7f3b8c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10d400 con 0x7f3b8c102340 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10d280 con 0x7f3b8c100790 2024-05-09T09:06:53.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b94664640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10d580 con 0x7f3b8c101980 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b92c01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 0x7f3b8c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b92c01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 0x7f3b8c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57398/0 (socket says 172.21.15.64:57398) 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b92c01640 1 -- 172.21.15.64:0/4148717212 learned_addr learned my addr 172.21.15.64:0/4148717212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.915+0000 7f3b92400640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8c100790 0x7f3b8c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b91bff640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c101980 0x7f3b8c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b92c01640 1 -- 172.21.15.64:0/4148717212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c101980 msgr2=0x7f3b8c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b92c01640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c101980 0x7f3b8c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b92c01640 1 -- 172.21.15.64:0/4148717212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8c100790 msgr2=0x7f3b8c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b92c01640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8c100790 0x7f3b8c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b92c01640 1 -- 172.21.15.64:0/4148717212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c19f730 con 0x7f3b8c102340 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b91bff640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c101980 0x7f3b8c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b92c01640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 0x7f3b8c19f0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400cad0 tx=0x7f3b8400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400ae20 con 0x7f3b8c102340 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b84005020 con 0x7f3b8c102340 2024-05-09T09:06:53.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c19fa20 con 0x7f3b8c102340 2024-05-09T09:06:53.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400f6b0 con 0x7f3b8c102340 2024-05-09T09:06:53.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.916+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c19ff20 con 0x7f3b8c102340 2024-05-09T09:06:53.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.917+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c107b80 con 0x7f3b8c102340 2024-05-09T09:06:53.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.918+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3b840051d0 con 0x7f3b8c102340 2024-05-09T09:06:53.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.919+0000 7f3b837fe640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b780a2b90 0x7f3b780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:53.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.919+0000 7f3b92400640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b780a2b90 0x7f3b780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:53.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.919+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3b840cf360 con 0x7f3b8c102340 2024-05-09T09:06:53.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.919+0000 7f3b92400640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b780a2b90 0x7f3b780a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005ab0 tx=0x7f3b88005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:53.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:53.923+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b84014030 con 0x7f3b8c102340 2024-05-09T09:06:54.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.107+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b8c062430 con 0x7f3b8c102340 2024-05-09T09:06:54.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.107+0000 7f3b837fe640 1 -- 172.21.15.64:0/4148717212 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f3b8409c080 con 0x7f3b8c102340 2024-05-09T09:06:54.107 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:06:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b780a2b90 msgr2=0x7f3b780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b780a2b90 0x7f3b780a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005ab0 tx=0x7f3b88005a20 comp rx=0 tx=0).stop 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 msgr2=0x7f3b8c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 0x7f3b8c19f0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400cad0 tx=0x7f3b8400cf90 comp rx=0 tx=0).stop 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 shutdown_connections 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b780a2b90 0x7f3b780a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c102340 0x7f3b8c19f0c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b8c101980 0x7f3b8c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 --2- 172.21.15.64:0/4148717212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8c100790 0x7f3b8c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 >> 172.21.15.64:0/4148717212 conn(0x7f3b8c0fbd20 msgr2=0x7f3b8c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 shutdown_connections 2024-05-09T09:06:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.109+0000 7f3b94664640 1 -- 172.21.15.64:0/4148717212 wait complete. 2024-05-09T09:06:54.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:06:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 -- 172.21.15.64:0/2624050029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 msgr2=0x7f43ec102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2624050029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec102f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009900 tx=0x7f43e002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 -- 172.21.15.64:0/2624050029 shutdown_connections 2024-05-09T09:06:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2624050029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec103440 0x7f43ec109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2624050029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec102f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2624050029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43ec101890 0x7f43ec101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 -- 172.21.15.64:0/2624050029 >> 172.21.15.64:0/2624050029 conn(0x7f43ec0fce00 msgr2=0x7f43ec0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 -- 172.21.15.64:0/2624050029 shutdown_connections 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 -- 172.21.15.64:0/2624050029 wait complete. 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.229+0000 7f43f17ee640 1 Processor -- start 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 -- start start 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec101890 0x7f43ec197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43ec103440 0x7f43ec19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10d3c0 con 0x7f43ec102a80 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10d240 con 0x7f43ec103440 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43f17ee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10d540 con 0x7f43ec101890 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57422/0 (socket says 172.21.15.64:57422) 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 -- 172.21.15.64:0/2541226541 learned_addr learned my addr 172.21.15.64:0/2541226541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43eaffd640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec101890 0x7f43ec197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43eb7fe640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43ec103440 0x7f43ec19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 -- 172.21.15.64:0/2541226541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec101890 msgr2=0x7f43ec197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec101890 0x7f43ec197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 -- 172.21.15.64:0/2541226541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43ec103440 msgr2=0x7f43ec19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43ec103440 0x7f43ec19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 -- 172.21.15.64:0/2541226541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec19f6f0 con 0x7f43ec102a80 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43eaffd640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec101890 0x7f43ec197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.230+0000 7f43ea7fc640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec198060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005940 tx=0x7f43e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.231+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e003f030 con 0x7f43ec102a80 2024-05-09T09:06:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.231+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43e0003cb0 con 0x7f43ec102a80 2024-05-09T09:06:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.231+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec19f980 con 0x7f43ec102a80 2024-05-09T09:06:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.231+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0035070 con 0x7f43ec102a80 2024-05-09T09:06:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.231+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec19fe70 con 0x7f43ec102a80 2024-05-09T09:06:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.232+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec108cd0 con 0x7f43ec102a80 2024-05-09T09:06:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.233+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f43e0042030 con 0x7f43ec102a80 2024-05-09T09:06:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.233+0000 7f43d3fff640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43d40a2b90 0x7f43d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.234+0000 7f43eaffd640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43d40a2b90 0x7f43d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.234+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f43e00ef940 con 0x7f43ec102a80 2024-05-09T09:06:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.234+0000 7f43eaffd640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43d40a2b90 0x7f43d40a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0058c0 tx=0x7f43dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:54.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.238+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43e00bc660 con 0x7f43ec102a80 2024-05-09T09:06:54.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:54 smithi064 ceph-mon[30149]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-09T09:06:54.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:54 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/144045433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:54.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:54 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1008518370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:54 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4148717212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:06:54.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.422+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43ec062430 con 0x7f43ec102a80 2024-05-09T09:06:54.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.423+0000 7f43d3fff640 1 -- 172.21.15.64:0/2541226541 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f43e00bbf60 con 0x7f43ec102a80 2024-05-09T09:06:54.422 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43d40a2b90 msgr2=0x7f43d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43d40a2b90 0x7f43d40a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0058c0 tx=0x7f43dc005850 comp rx=0 tx=0).stop 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 msgr2=0x7f43ec198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec198060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005940 tx=0x7f43e00042f0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 shutdown_connections 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43d40a2b90 0x7f43d40a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43ec103440 0x7f43ec19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43ec102a80 0x7f43ec198060 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 --2- 172.21.15.64:0/2541226541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ec101890 0x7f43ec197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 >> 172.21.15.64:0/2541226541 conn(0x7f43ec0fce00 msgr2=0x7f43ec105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:54.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 shutdown_connections 2024-05-09T09:06:54.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.425+0000 7f43f17ee640 1 -- 172.21.15.64:0/2541226541 wait complete. 2024-05-09T09:06:54.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:06:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:54 smithi179 ceph-mon[34086]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-09T09:06:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:54 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/144045433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:54 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1008518370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:54 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4148717212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:06:54.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 -- 172.21.15.64:0/910618113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8102a80 msgr2=0x7f46b8102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 --2- 172.21.15.64:0/910618113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8102a80 0x7f46b8102f00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009900 tx=0x7f46a802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:54.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 -- 172.21.15.64:0/910618113 shutdown_connections 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 --2- 172.21.15.64:0/910618113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8103440 0x7f46b8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 --2- 172.21.15.64:0/910618113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8102a80 0x7f46b8102f00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 --2- 172.21.15.64:0/910618113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46b8101890 0x7f46b8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 -- 172.21.15.64:0/910618113 >> 172.21.15.64:0/910618113 conn(0x7f46b80fce00 msgr2=0x7f46b80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 -- 172.21.15.64:0/910618113 shutdown_connections 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.543+0000 7f46bf37d640 1 -- 172.21.15.64:0/910618113 wait complete. 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 Processor -- start 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 -- start start 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46b8101890 0x7f46b8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8102a80 0x7f46b8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 0x7f46b819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810d3c0 con 0x7f46b8103440 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810d240 con 0x7f46b8102a80 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bf37d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810d540 con 0x7f46b8101890 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bd91a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 0x7f46b819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bd91a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 0x7f46b819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57438/0 (socket says 172.21.15.64:57438) 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bd91a640 1 -- 172.21.15.64:0/2917012152 learned_addr learned my addr 172.21.15.64:0/2917012152 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.544+0000 7f46bd119640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46b8101890 0x7f46b8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bc918640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8102a80 0x7f46b8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bd91a640 1 -- 172.21.15.64:0/2917012152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46b8101890 msgr2=0x7f46b8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bd91a640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46b8101890 0x7f46b8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bd91a640 1 -- 172.21.15.64:0/2917012152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8102a80 msgr2=0x7f46b8197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bd91a640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8102a80 0x7f46b8197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bd91a640 1 -- 172.21.15.64:0/2917012152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b819f6a0 con 0x7f46b8103440 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bc918640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8102a80 0x7f46b8197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bd91a640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 0x7f46b819f030 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c970 tx=0x7f46b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0007bb0 con 0x7f46b8103440 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46b0007d50 con 0x7f46b8103440 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b819f990 con 0x7f46b8103440 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0018700 con 0x7f46b8103440 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.545+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b819fe90 con 0x7f46b8103440 2024-05-09T09:06:54.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.546+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b8108c80 con 0x7f46b8103440 2024-05-09T09:06:54.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.547+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f46b000fa20 con 0x7f46b8103440 2024-05-09T09:06:54.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.548+0000 7f46ae7fc640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46a40a2b90 0x7f46a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.548+0000 7f46bd119640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46a40a2b90 0x7f46a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.548+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f46b00cf340 con 0x7f46b8103440 2024-05-09T09:06:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.549+0000 7f46bd119640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46a40a2b90 0x7f46a40a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46b81028e0 tx=0x7f46b4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:54.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.553+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46b0014030 con 0x7f46b8103440 2024-05-09T09:06:54.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:54 smithi139 ceph-mon[33599]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-09T09:06:54.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:54 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/144045433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:54.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:54 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1008518370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:54.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:54 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4148717212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:06:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.737+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f46b8062430 con 0x7f46b8103440 2024-05-09T09:06:54.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.738+0000 7f46ae7fc640 1 -- 172.21.15.64:0/2917012152 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f46b009c060 con 0x7f46b8103440 2024-05-09T09:06:54.737 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46a40a2b90 msgr2=0x7f46a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46a40a2b90 0x7f46a40a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46b81028e0 tx=0x7f46b4009680 comp rx=0 tx=0).stop 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 msgr2=0x7f46b819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 0x7f46b819f030 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c970 tx=0x7f46b000ce30 comp rx=0 tx=0).stop 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 shutdown_connections 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46a40a2b90 0x7f46a40a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46b8103440 0x7f46b819f030 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46b8102a80 0x7f46b8197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 --2- 172.21.15.64:0/2917012152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46b8101890 0x7f46b8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 >> 172.21.15.64:0/2917012152 conn(0x7f46b80fce00 msgr2=0x7f46b8105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 shutdown_connections 2024-05-09T09:06:54.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.740+0000 7f46bf37d640 1 -- 172.21.15.64:0/2917012152 wait complete. 2024-05-09T09:06:54.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:06:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.857+0000 7fb5c4215640 1 -- 172.21.15.64:0/1158870079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 msgr2=0x7fb5bc101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.857+0000 7fb5c4215640 1 --2- 172.21.15.64:0/1158870079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc101cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009900 tx=0x7fb5b802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 -- 172.21.15.64:0/1158870079 shutdown_connections 2024-05-09T09:06:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 --2- 172.21.15.64:0/1158870079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5bc103440 0x7fb5bc109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 --2- 172.21.15.64:0/1158870079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5bc102a80 0x7fb5bc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 --2- 172.21.15.64:0/1158870079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc101cb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 -- 172.21.15.64:0/1158870079 >> 172.21.15.64:0/1158870079 conn(0x7fb5bc0fce00 msgr2=0x7fb5bc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 -- 172.21.15.64:0/1158870079 shutdown_connections 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 -- 172.21.15.64:0/1158870079 wait complete. 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 Processor -- start 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 -- start start 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5bc102a80 0x7fb5bc073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5bc103440 0x7fb5bc074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.858+0000 7fb5c4215640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10d3b0 con 0x7fb5bc101890 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c4215640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10d230 con 0x7fb5bc102a80 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c4215640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10d530 con 0x7fb5bc103440 2024-05-09T09:06:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc0755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60888/0 (socket says 172.21.15.64:60888) 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 -- 172.21.15.64:0/2487135347 learned_addr learned my addr 172.21.15.64:0/2487135347 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c17b0640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5bc102a80 0x7fb5bc073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c27b2640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5bc103440 0x7fb5bc074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 -- 172.21.15.64:0/2487135347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5bc103440 msgr2=0x7fb5bc074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5bc103440 0x7fb5bc074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 -- 172.21.15.64:0/2487135347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5bc102a80 msgr2=0x7fb5bc073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5bc102a80 0x7fb5bc073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 -- 172.21.15.64:0/2487135347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc0749f0 con 0x7fb5bc101890 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c1fb1640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc0755e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80314e0 tx=0x7fb5b8031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b803f030 con 0x7fb5bc101890 2024-05-09T09:06:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5b80044b0 con 0x7fb5bc101890 2024-05-09T09:06:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc1a1bb0 con 0x7fb5bc101890 2024-05-09T09:06:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8035070 con 0x7fb5bc101890 2024-05-09T09:06:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.859+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1a2150 con 0x7fb5bc101890 2024-05-09T09:06:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.860+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc108c20 con 0x7fb5bc101890 2024-05-09T09:06:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.861+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb5b8042030 con 0x7fb5bc101890 2024-05-09T09:06:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.862+0000 7fb5b2ffd640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5a80a2c10 0x7fb5a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.862+0000 7fb5c17b0640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5a80a2c10 0x7fb5a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.862+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb5b80ef6d0 con 0x7fb5bc101890 2024-05-09T09:06:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.863+0000 7fb5c17b0640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5a80a2c10 0x7fb5a80a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc075350 tx=0x7fb5ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:54.866+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb5b80bbda0 con 0x7fb5bc101890 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.064+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5bc02d1c0 con 0x7fb5bc101890 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.064+0000 7fb5b2ffd640 1 -- 172.21.15.64:0/2487135347 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+588 (secure 0 0 0) 0x7fb5b80c00a0 con 0x7fb5bc101890 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1152515804 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3043598500 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2011681658 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3326248924 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:55.064 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1070396506 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:55.064 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2526964267 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:55.064 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:55.064 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5a80a2c10 msgr2=0x7fb5a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5a80a2c10 0x7fb5a80a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc075350 tx=0x7fb5ac005f90 comp rx=0 tx=0).stop 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 msgr2=0x7fb5bc0755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc0755e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80314e0 tx=0x7fb5b8031a30 comp rx=0 tx=0).stop 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 shutdown_connections 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5a80a2c10 0x7fb5a80a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5bc103440 0x7fb5bc074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5bc102a80 0x7fb5bc073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 --2- 172.21.15.64:0/2487135347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc101890 0x7fb5bc0755e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 >> 172.21.15.64:0/2487135347 conn(0x7fb5bc0fce00 msgr2=0x7fb5bc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 shutdown_connections 2024-05-09T09:06:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.066+0000 7fb5c4215640 1 -- 172.21.15.64:0/2487135347 wait complete. 2024-05-09T09:06:55.076 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:06:55.076 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:06:55.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:06:55.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 -- 172.21.15.64:0/1888670363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4102a70 msgr2=0x7f36d4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/1888670363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4102a70 0x7f36d4102ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009900 tx=0x7f36c802a410 comp rx=0 tx=0).stop 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 -- 172.21.15.64:0/1888670363 shutdown_connections 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/1888670363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d4103430 0x7f36d4109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/1888670363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4102a70 0x7f36d4102ef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/1888670363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36d4101880 0x7f36d4101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 -- 172.21.15.64:0/1888670363 >> 172.21.15.64:0/1888670363 conn(0x7f36d40fce10 msgr2=0x7f36d40ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 -- 172.21.15.64:0/1888670363 shutdown_connections 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.244+0000 7f36d9ddb640 1 -- 172.21.15.64:0/1888670363 wait complete. 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 Processor -- start 2024-05-09T09:06:55.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 -- start start 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 0x7f36d40755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36d4102a70 0x7f36d4073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d4103430 0x7f36d4074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410d3b0 con 0x7f36d4101880 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410d230 con 0x7f36d4102a70 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d9ddb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410d530 con 0x7f36d4103430 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 0x7f36d40755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 0x7f36d40755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60898/0 (socket says 172.21.15.64:60898) 2024-05-09T09:06:55.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 -- 172.21.15.64:0/4054469314 learned_addr learned my addr 172.21.15.64:0/4054469314 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d2ffd640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36d4102a70 0x7f36d4073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d3fff640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d4103430 0x7f36d4074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 -- 172.21.15.64:0/4054469314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d4103430 msgr2=0x7f36d4074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d4103430 0x7f36d4074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 -- 172.21.15.64:0/4054469314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36d4102a70 msgr2=0x7f36d4073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36d4102a70 0x7f36d4073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.245+0000 7f36d37fe640 1 -- 172.21.15.64:0/4054469314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d40749f0 con 0x7f36d4101880 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.246+0000 7f36d37fe640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 0x7f36d40755e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36c400d950 tx=0x7f36c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.246+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4017020 con 0x7f36d4101880 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.246+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36c4009c30 con 0x7f36d4101880 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.246+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c400f6b0 con 0x7f36d4101880 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.246+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d41a1bd0 con 0x7f36d4101880 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.246+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a20f0 con 0x7f36d4101880 2024-05-09T09:06:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.247+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d4068aa0 con 0x7f36d4101880 2024-05-09T09:06:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.249+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36c401a020 con 0x7f36d4101880 2024-05-09T09:06:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.250+0000 7f36d0ff9640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f36bc0a2b90 0x7f36bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.250+0000 7f36d2ffd640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f36bc0a2b90 0x7f36bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.250+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f36c40cf290 con 0x7f36d4101880 2024-05-09T09:06:55.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.250+0000 7f36d2ffd640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f36bc0a2b90 0x7f36bc0a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006010 tx=0x7f36c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.253+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36c4009de0 con 0x7f36d4101880 2024-05-09T09:06:55.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2541226541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:06:55.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2917012152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:06:55.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2487135347' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:06:55.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.439+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36d4062460 con 0x7f36d4101880 2024-05-09T09:06:55.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.441+0000 7f36d0ff9640 1 -- 172.21.15.64:0/4054469314 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7f36c409bf50 con 0x7f36d4101880 2024-05-09T09:06:55.440 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:06:55.440 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:06:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f36bc0a2b90 msgr2=0x7f36bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f36bc0a2b90 0x7f36bc0a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006010 tx=0x7f36c8000bc0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 msgr2=0x7f36d40755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 0x7f36d40755e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36c400d950 tx=0x7f36c400de10 comp rx=0 tx=0).stop 2024-05-09T09:06:55.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 shutdown_connections 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f36bc0a2b90 0x7f36bc0a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36d4103430 0x7f36d4074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36d4102a70 0x7f36d4073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 --2- 172.21.15.64:0/4054469314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36d4101880 0x7f36d40755e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 >> 172.21.15.64:0/4054469314 conn(0x7f36d40fce10 msgr2=0x7f36d4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 shutdown_connections 2024-05-09T09:06:55.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.443+0000 7f36d9ddb640 1 -- 172.21.15.64:0/4054469314 wait complete. 2024-05-09T09:06:55.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:06:55.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:06:55.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:55 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2541226541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:06:55.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:55 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2917012152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:06:55.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:55 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2487135347' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:06:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 -- 172.21.15.64:0/1351122428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100790 msgr2=0x7ff894100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 --2- 172.21.15.64:0/1351122428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100790 0x7ff894100bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff890009900 tx=0x7ff89002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:55.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 -- 172.21.15.64:0/1351122428 shutdown_connections 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 --2- 172.21.15.64:0/1351122428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff894102340 0x7ff894108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 --2- 172.21.15.64:0/1351122428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894101980 0x7ff894101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 --2- 172.21.15.64:0/1351122428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100790 0x7ff894100bb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.562+0000 7ff89beca640 1 -- 172.21.15.64:0/1351122428 >> 172.21.15.64:0/1351122428 conn(0x7ff8940fbd20 msgr2=0x7ff8940fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 -- 172.21.15.64:0/1351122428 shutdown_connections 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 -- 172.21.15.64:0/1351122428 wait complete. 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 Processor -- start 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 -- start start 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff894100790 0x7ff894196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 0x7ff894196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894102340 0x7ff89419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410c2c0 con 0x7ff894101980 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410c140 con 0x7ff894102340 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff89beca640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410c440 con 0x7ff894100790 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff899465640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 0x7ff894196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff899465640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 0x7ff894196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60924/0 (socket says 172.21.15.64:60924) 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.563+0000 7ff899465640 1 -- 172.21.15.64:0/1237546247 learned_addr learned my addr 172.21.15.64:0/1237546247 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899c66640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff894100790 0x7ff894196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff89a467640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894102340 0x7ff89419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899465640 1 -- 172.21.15.64:0/1237546247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff894100790 msgr2=0x7ff894196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899465640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff894100790 0x7ff894196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899465640 1 -- 172.21.15.64:0/1237546247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894102340 msgr2=0x7ff89419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899465640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894102340 0x7ff89419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899465640 1 -- 172.21.15.64:0/1237546247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89419e630 con 0x7ff894101980 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff89a467640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894102340 0x7ff89419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff899465640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 0x7ff894196f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff88400d950 tx=0x7ff88400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884017030 con 0x7ff894101980 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff884004020 con 0x7ff894101980 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8840047a0 con 0x7ff894101980 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89419e920 con 0x7ff894101980 2024-05-09T09:06:55.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.564+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89419ee20 con 0x7ff894101980 2024-05-09T09:06:55.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.565+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff894107b80 con 0x7ff894101980 2024-05-09T09:06:55.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.567+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8840041d0 con 0x7ff894101980 2024-05-09T09:06:55.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.567+0000 7ff88affd640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8800a2c10 0x7ff8800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.567+0000 7ff899c66640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8800a2c10 0x7ff8800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.567+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff8840cf400 con 0x7ff894101980 2024-05-09T09:06:55.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.568+0000 7ff899c66640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8800a2c10 0x7ff8800a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff890000c00 tx=0x7ff8900314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:55.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.570+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff88409c1a0 con 0x7ff894101980 2024-05-09T09:06:55.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:55 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2541226541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:06:55.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:55 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2917012152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:06:55.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:55 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2487135347' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:06:55.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.755+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff89402d430 con 0x7ff894101980 2024-05-09T09:06:55.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.755+0000 7ff88affd640 1 -- 172.21.15.64:0/1237546247 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff88409baa0 con 0x7ff894101980 2024-05-09T09:06:55.754 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:06:55.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8800a2c10 msgr2=0x7ff8800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8800a2c10 0x7ff8800a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff890000c00 tx=0x7ff8900314a0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 msgr2=0x7ff894196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 0x7ff894196f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff88400d950 tx=0x7ff88400de10 comp rx=0 tx=0).stop 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 shutdown_connections 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8800a2c10 0x7ff8800a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff894102340 0x7ff89419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101980 0x7ff894196f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 --2- 172.21.15.64:0/1237546247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff894100790 0x7ff894196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 >> 172.21.15.64:0/1237546247 conn(0x7ff8940fbd20 msgr2=0x7ff894104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 shutdown_connections 2024-05-09T09:06:55.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.757+0000 7ff89beca640 1 -- 172.21.15.64:0/1237546247 wait complete. 2024-05-09T09:06:55.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 -- 172.21.15.64:0/2157642053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408100770 msgr2=0x7fc408100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 --2- 172.21.15.64:0/2157642053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408100770 0x7fc408100b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a410 comp rx=0 tx=0).stop 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 -- 172.21.15.64:0/2157642053 shutdown_connections 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 --2- 172.21.15.64:0/2157642053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc408102320 0x7fc408108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 --2- 172.21.15.64:0/2157642053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc408101960 0x7fc408101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 --2- 172.21.15.64:0/2157642053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408100770 0x7fc408100b90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 -- 172.21.15.64:0/2157642053 >> 172.21.15.64:0/2157642053 conn(0x7fc4080fbce0 msgr2=0x7fc4080fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 -- 172.21.15.64:0/2157642053 shutdown_connections 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 -- 172.21.15.64:0/2157642053 wait complete. 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 Processor -- start 2024-05-09T09:06:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 -- start start 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc408100770 0x7fc408197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.876+0000 7fc40f637640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 0x7fc408198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40f637640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc408102320 0x7fc40819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40f637640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810c290 con 0x7fc408101960 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40f637640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810c110 con 0x7fc408102320 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40f637640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810c410 con 0x7fc408100770 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 0x7fc408198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 0x7fc408198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60948/0 (socket says 172.21.15.64:60948) 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 -- 172.21.15.64:0/2801474295 learned_addr learned my addr 172.21.15.64:0/2801474295 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40d3d3640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc408100770 0x7fc408197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40dbd4640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc408102320 0x7fc40819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 -- 172.21.15.64:0/2801474295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc408100770 msgr2=0x7fc408197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc408100770 0x7fc408197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 -- 172.21.15.64:0/2801474295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc408102320 msgr2=0x7fc40819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:55.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc408102320 0x7fc40819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 -- 172.21.15.64:0/2801474295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40819f730 con 0x7fc408101960 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40cbd2640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 0x7fc408198050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00e9f0 tx=0x7fc3fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc00cd80 con 0x7fc408101960 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3fc0054d0 con 0x7fc408101960 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40819fa20 con 0x7fc408101960 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc010490 con 0x7fc408101960 2024-05-09T09:06:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.877+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40819ff20 con 0x7fc408101960 2024-05-09T09:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.878+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4081a01b0 con 0x7fc408101960 2024-05-09T09:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.880+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc3fc005020 con 0x7fc408101960 2024-05-09T09:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.881+0000 7fc4067fc640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3f00a2b90 0x7fc3f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.881+0000 7fc40d3d3640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3f00a2b90 0x7fc3f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.881+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc3fc0cf5e0 con 0x7fc408101960 2024-05-09T09:06:55.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.881+0000 7fc40d3d3640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3f00a2b90 0x7fc3f00a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc400004770 tx=0x7fc400000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:55.885+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3fc09c300 con 0x7fc408101960 2024-05-09T09:06:56.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.067+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc4081a0380 con 0x7fc408101960 2024-05-09T09:06:56.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.067+0000 7fc4067fc640 1 -- 172.21.15.64:0/2801474295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fc3fc09bc00 con 0x7fc408101960 2024-05-09T09:06:56.066 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:06:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3f00a2b90 msgr2=0x7fc3f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3f00a2b90 0x7fc3f00a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc400004770 tx=0x7fc400000bc0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 msgr2=0x7fc408198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 0x7fc408198050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00e9f0 tx=0x7fc3fc00edf0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 shutdown_connections 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3f00a2b90 0x7fc3f00a5050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc408102320 0x7fc40819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc408101960 0x7fc408198050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 --2- 172.21.15.64:0/2801474295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc408100770 0x7fc408197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 >> 172.21.15.64:0/2801474295 conn(0x7fc4080fbce0 msgr2=0x7fc4080689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 shutdown_connections 2024-05-09T09:06:56.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.069+0000 7fc40f637640 1 -- 172.21.15.64:0/2801474295 wait complete. 2024-05-09T09:06:56.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:06:56.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/1962301760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb681075b0 msgr2=0x7ffb681099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/1962301760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb681075b0 0x7ffb681099a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009ca0 tx=0x7ffb6002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/1962301760 shutdown_connections 2024-05-09T09:06:56.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/1962301760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb681075b0 0x7ffb681099a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/1962301760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 0x7ffb68107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/1962301760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb68102350 0x7ffb68104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/1962301760 >> 172.21.15.64:0/1962301760 conn(0x7ffb680fbce0 msgr2=0x7ffb680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:56.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/1962301760 shutdown_connections 2024-05-09T09:06:56.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/1962301760 wait complete. 2024-05-09T09:06:56.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 Processor -- start 2024-05-09T09:06:56.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.190+0000 7ffb6e4ba640 1 -- start start 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6e4ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 0x7ffb68197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6e4ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 0x7ffb68198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6e4ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb681075b0 0x7ffb6819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6e4ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810d3f0 con 0x7ffb68102350 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6e4ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810d270 con 0x7ffb68104c80 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6e4ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810d570 con 0x7ffb681075b0 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 0x7ffb68197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 0x7ffb68197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60966/0 (socket says 172.21.15.64:60966) 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 -- 172.21.15.64:0/589701020 learned_addr learned my addr 172.21.15.64:0/589701020 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6ca57640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb681075b0 0x7ffb6819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb677fe640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 0x7ffb68198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 -- 172.21.15.64:0/589701020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb681075b0 msgr2=0x7ffb6819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb681075b0 0x7ffb6819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 -- 172.21.15.64:0/589701020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 msgr2=0x7ffb68198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 0x7ffb68198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 -- 172.21.15.64:0/589701020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6819f710 con 0x7ffb68102350 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb6ca57640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb681075b0 0x7ffb6819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb677fe640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 0x7ffb68198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb67fff640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 0x7ffb68197af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800d950 tx=0x7ffb5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.191+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58017070 con 0x7ffb68102350 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.192+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb58009c30 con 0x7ffb68102350 2024-05-09T09:06:56.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.192+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6819fa00 con 0x7ffb68102350 2024-05-09T09:06:56.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.192+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800f620 con 0x7ffb68102350 2024-05-09T09:06:56.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.192+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6819ff50 con 0x7ffb68102350 2024-05-09T09:06:56.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.193+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb680fd7d0 con 0x7ffb68102350 2024-05-09T09:06:56.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.194+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb580109a0 con 0x7ffb68102350 2024-05-09T09:06:56.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.195+0000 7ffb657fa640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb500a2b90 0x7ffb500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.195+0000 7ffb677fe640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb500a2b90 0x7ffb500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.195+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffb580cef60 con 0x7ffb68102350 2024-05-09T09:06:56.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.195+0000 7ffb677fe640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb500a2b90 0x7ffb500a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffb68199000 tx=0x7ffb5c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:56.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.199+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb58009de0 con 0x7ffb68102350 2024-05-09T09:06:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.382+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb6802d1c0 con 0x7ffb68102350 2024-05-09T09:06:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.382+0000 7ffb657fa640 1 -- 172.21.15.64:0/589701020 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ffb6802d1c0 con 0x7ffb68102350 2024-05-09T09:06:56.381 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb500a2b90 msgr2=0x7ffb500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb500a2b90 0x7ffb500a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffb68199000 tx=0x7ffb5c005d30 comp rx=0 tx=0).stop 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 msgr2=0x7ffb68197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 0x7ffb68197af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800d950 tx=0x7ffb5800de10 comp rx=0 tx=0).stop 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 shutdown_connections 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb500a2b90 0x7ffb500a5050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb681075b0 0x7ffb6819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68104c80 0x7ffb68198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 --2- 172.21.15.64:0/589701020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb68102350 0x7ffb68197af0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 >> 172.21.15.64:0/589701020 conn(0x7ffb680fbce0 msgr2=0x7ffb68102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 shutdown_connections 2024-05-09T09:06:56.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.384+0000 7ffb6e4ba640 1 -- 172.21.15.64:0/589701020 wait complete. 2024-05-09T09:06:56.393 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:56 smithi064 ceph-mon[30149]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 206 B/s wr, 0 op/s 2024-05-09T09:06:56.393 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4054469314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:56.393 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1237546247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:56.393 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2801474295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:06:56.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:06:56.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:56 smithi179 ceph-mon[34086]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 206 B/s wr, 0 op/s 2024-05-09T09:06:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4054469314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1237546247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2801474295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:06:56.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.502+0000 7f425071b640 1 -- 172.21.15.64:0/3463897814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 msgr2=0x7f42481099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.502+0000 7f425071b640 1 --2- 172.21.15.64:0/3463897814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f42481099c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4240009ca0 tx=0x7f424002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- 172.21.15.64:0/3463897814 shutdown_connections 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 --2- 172.21.15.64:0/3463897814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f42481099c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 --2- 172.21.15.64:0/3463897814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248104c40 0x7f4248107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 --2- 172.21.15.64:0/3463897814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4248102310 0x7f4248104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- 172.21.15.64:0/3463897814 >> 172.21.15.64:0/3463897814 conn(0x7f42480fbd20 msgr2=0x7f42480fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- 172.21.15.64:0/3463897814 shutdown_connections 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- 172.21.15.64:0/3463897814 wait complete. 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 Processor -- start 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- start start 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248102310 0x7f4248197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4248104c40 0x7f4248197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f424819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f424810d3c0 con 0x7f4248107570 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f424810d240 con 0x7f4248102310 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f425071b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f424810d540 con 0x7f4248104c40 2024-05-09T09:06:56.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.503+0000 7f424ecb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f424819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f424819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60978/0 (socket says 172.21.15.64:60978) 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 -- 172.21.15.64:0/4105479861 learned_addr learned my addr 172.21.15.64:0/4105479861 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424e4b7640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248102310 0x7f4248197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424dcb6640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4248104c40 0x7f4248197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 -- 172.21.15.64:0/4105479861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4248104c40 msgr2=0x7f4248197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4248104c40 0x7f4248197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 -- 172.21.15.64:0/4105479861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248102310 msgr2=0x7f4248197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248102310 0x7f4248197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 -- 172.21.15.64:0/4105479861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424819f6b0 con 0x7f4248107570 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424e4b7640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248102310 0x7f4248197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f424ecb8640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f424819f040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4240031970 tx=0x7f4240003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240004040 con 0x7f4248107570 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4240004690 con 0x7f4248107570 2024-05-09T09:06:56.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240030c60 con 0x7f4248107570 2024-05-09T09:06:56.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424819f940 con 0x7f4248107570 2024-05-09T09:06:56.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.504+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424819fee0 con 0x7f4248107570 2024-05-09T09:06:56.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.505+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42480fd810 con 0x7f4248107570 2024-05-09T09:06:56.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.506+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f42400041e0 con 0x7f4248107570 2024-05-09T09:06:56.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.507+0000 7f423f7fe640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42340a2c10 0x7f42340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.507+0000 7f424e4b7640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42340a2c10 0x7f42340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.507+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f42400f0960 con 0x7f4248107570 2024-05-09T09:06:56.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.508+0000 7f424e4b7640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42340a2c10 0x7f42340a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4244005d00 tx=0x7f4244005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:56.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.510+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f42400bd680 con 0x7f4248107570 2024-05-09T09:06:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:56 smithi139 ceph-mon[33599]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 206 B/s wr, 0 op/s 2024-05-09T09:06:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4054469314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:06:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1237546247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:06:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2801474295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:06:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.694+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f424802d1c0 con 0x7f4248107570 2024-05-09T09:06:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.694+0000 7f423f7fe640 1 -- 172.21.15.64:0/4105479861 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f42400bcf80 con 0x7f4248107570 2024-05-09T09:06:56.693 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:06:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42340a2c10 msgr2=0x7f42340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42340a2c10 0x7f42340a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4244005d00 tx=0x7f4244005c70 comp rx=0 tx=0).stop 2024-05-09T09:06:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 msgr2=0x7f424819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f424819f040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4240031970 tx=0x7f4240003ba0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 shutdown_connections 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42340a2c10 0x7f42340a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4248107570 0x7f424819f040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4248104c40 0x7f4248197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 --2- 172.21.15.64:0/4105479861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4248102310 0x7f4248197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 >> 172.21.15.64:0/4105479861 conn(0x7f42480fbd20 msgr2=0x7f4248102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 shutdown_connections 2024-05-09T09:06:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.696+0000 7f425071b640 1 -- 172.21.15.64:0/4105479861 wait complete. 2024-05-09T09:06:56.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:06:56.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 -- 172.21.15.64:0/3362608056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4102340 msgr2=0x7ff6c4108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 --2- 172.21.15.64:0/3362608056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4102340 0x7ff6c4108bd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009900 tx=0x7ff6bc02a410 comp rx=0 tx=0).stop 2024-05-09T09:06:56.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 -- 172.21.15.64:0/3362608056 shutdown_connections 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 --2- 172.21.15.64:0/3362608056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4102340 0x7ff6c4108bd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 --2- 172.21.15.64:0/3362608056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4101980 0x7ff6c4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 --2- 172.21.15.64:0/3362608056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6c4100790 0x7ff6c4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 -- 172.21.15.64:0/3362608056 >> 172.21.15.64:0/3362608056 conn(0x7ff6c40fbd20 msgr2=0x7ff6c40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 -- 172.21.15.64:0/3362608056 shutdown_connections 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.813+0000 7ff6cb012640 1 -- 172.21.15.64:0/3362608056 wait complete. 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 Processor -- start 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 -- start start 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 0x7ff6c4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6c4101980 0x7ff6c4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4102340 0x7ff6c419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410c2c0 con 0x7ff6c4100790 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410c140 con 0x7ff6c4102340 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6cb012640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410c440 con 0x7ff6c4101980 2024-05-09T09:06:56.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 0x7ff6c4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 0x7ff6c4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60996/0 (socket says 172.21.15.64:60996) 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 -- 172.21.15.64:0/1078531719 learned_addr learned my addr 172.21.15.64:0/1078531719 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c3fff640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6c4101980 0x7ff6c4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c95af640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4102340 0x7ff6c419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 -- 172.21.15.64:0/1078531719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6c4101980 msgr2=0x7ff6c4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6c4101980 0x7ff6c4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 -- 172.21.15.64:0/1078531719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4102340 msgr2=0x7ff6c419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4102340 0x7ff6c419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c8dae640 1 -- 172.21.15.64:0/1078531719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c419f6d0 con 0x7ff6c4100790 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.814+0000 7ff6c95af640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4102340 0x7ff6c419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.815+0000 7ff6c8dae640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 0x7ff6c4197ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00d950 tx=0x7ff6ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.815+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac017070 con 0x7ff6c4100790 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.815+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ac009c30 con 0x7ff6c4100790 2024-05-09T09:06:56.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.815+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c419f9c0 con 0x7ff6c4100790 2024-05-09T09:06:56.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.815+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00f620 con 0x7ff6c4100790 2024-05-09T09:06:56.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.815+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c419fec0 con 0x7ff6c4100790 2024-05-09T09:06:56.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.816+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c41a0090 con 0x7ff6c4100790 2024-05-09T09:06:56.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.817+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff6ac01a070 con 0x7ff6c4100790 2024-05-09T09:06:56.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.818+0000 7ff6c1ffb640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6b00a2b90 0x7ff6b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:06:56.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.818+0000 7ff6c3fff640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6b00a2b90 0x7ff6b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:06:56.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.818+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff6ac0cece0 con 0x7ff6c4100790 2024-05-09T09:06:56.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.818+0000 7ff6c3fff640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6b00a2b90 0x7ff6b00a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4198fc0 tx=0x7ff6b4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:06:56.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:56.822+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6ac009de0 con 0x7ff6c4100790 2024-05-09T09:06:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.018+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6c41a0260 con 0x7ff6c4100790 2024-05-09T09:06:57.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.019+0000 7ff6c1ffb640 1 -- 172.21.15.64:0/1078531719 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+588 (secure 0 0 0) 0x7ff6ac09ba00 con 0x7ff6c4100790 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1152515804 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3043598500 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2011681658 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3326248924 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1070396506 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2526964267 2024-05-10T08:59:35.011155+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:57.018 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:06:57.019 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6b00a2b90 msgr2=0x7ff6b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6b00a2b90 0x7ff6b00a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4198fc0 tx=0x7ff6b4009680 comp rx=0 tx=0).stop 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 msgr2=0x7ff6c4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 0x7ff6c4197ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00d950 tx=0x7ff6ac00de10 comp rx=0 tx=0).stop 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 shutdown_connections 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6b00a2b90 0x7ff6b00a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6c4102340 0x7ff6c419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6c4101980 0x7ff6c4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 --2- 172.21.15.64:0/1078531719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6c4100790 0x7ff6c4197ab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 >> 172.21.15.64:0/1078531719 conn(0x7ff6c40fbd20 msgr2=0x7ff6c4069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 shutdown_connections 2024-05-09T09:06:57.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:06:57.021+0000 7ff6cb012640 1 -- 172.21.15.64:0/1078531719 wait complete. 2024-05-09T09:06:57.031 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-05-09T09:06:57.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:06:57.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T09:06:57.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[72326]: fuse finished with error 0 and tester_r 0 2024-05-09T09:06:57.057 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T09:06:57.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/589701020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:06:57.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4105479861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:06:57.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1078531719' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:06:57.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/589701020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:06:57.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4105479861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:06:57.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1078531719' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:06:57.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/589701020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:06:57.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4105479861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:06:57.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1078531719' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:06:58.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:06:58 smithi179 ceph-mon[34086]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-05-09T09:06:58.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:06:58 smithi139 ceph-mon[33599]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-05-09T09:06:58.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:06:58 smithi064 ceph-mon[30149]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s 2024-05-09T09:07:00.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:00 smithi179 ceph-mon[34086]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:00.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:00 smithi139 ceph-mon[33599]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:00 smithi064 ceph-mon[30149]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:02.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:02 smithi179 ceph-mon[34086]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:02.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:02 smithi139 ceph-mon[33599]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:02.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:02 smithi064 ceph-mon[30149]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:03.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:07:03.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:07:03.159 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T09:07:03.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:07:03.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 -- 172.21.15.64:0/1349985114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 msgr2=0x7f106c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 --2- 172.21.15.64:0/1349985114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1099c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1064009ca0 tx=0x7f106402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 -- 172.21.15.64:0/1349985114 shutdown_connections 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 --2- 172.21.15.64:0/1349985114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1099c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 --2- 172.21.15.64:0/1349985114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 0x7f106c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 --2- 172.21.15.64:0/1349985114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106c102310 0x7f106c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 -- 172.21.15.64:0/1349985114 >> 172.21.15.64:0/1349985114 conn(0x7f106c0fbd20 msgr2=0x7f106c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 -- 172.21.15.64:0/1349985114 shutdown_connections 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.349+0000 7f107456a640 1 -- 172.21.15.64:0/1349985114 wait complete. 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 Processor -- start 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 -- start start 2024-05-09T09:07:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106c102310 0x7f106c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 0x7f106c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f106c10d3c0 con 0x7f106c107570 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f106c10d240 con 0x7f106c104c40 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f107456a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f106c10d540 con 0x7f106c102310 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f1072b07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f1072b07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32794/0 (socket says 172.21.15.64:32794) 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f1072b07640 1 -- 172.21.15.64:0/3673936762 learned_addr learned my addr 172.21.15.64:0/3673936762 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.350+0000 7f1072306640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106c102310 0x7f106c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1071b05640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 0x7f106c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1072b07640 1 -- 172.21.15.64:0/3673936762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106c102310 msgr2=0x7f106c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1072b07640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106c102310 0x7f106c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1072b07640 1 -- 172.21.15.64:0/3673936762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 msgr2=0x7f106c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1072b07640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 0x7f106c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1072b07640 1 -- 172.21.15.64:0/3673936762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c1a1840 con 0x7f106c107570 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1071b05640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 0x7f106c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f1072b07640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1a11a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1064009dc0 tx=0x7f1064003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064004040 con 0x7f106c107570 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1064004690 con 0x7f106c107570 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106402f4c0 con 0x7f106c107570 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c1a1ad0 con 0x7f106c107570 2024-05-09T09:07:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.351+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c1a2020 con 0x7f106c107570 2024-05-09T09:07:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.352+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c109060 con 0x7f106c107570 2024-05-09T09:07:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.354+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1064030650 con 0x7f106c107570 2024-05-09T09:07:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.355+0000 7f105f7fe640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10580a2b90 0x7f10580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.355+0000 7f1072306640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10580a2b90 0x7f10580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.355+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f10640f0d60 con 0x7f106c107570 2024-05-09T09:07:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.355+0000 7f1072306640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10580a2b90 0x7f10580a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1068005e10 tx=0x7f1068005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.358+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f10640bda80 con 0x7f106c107570 2024-05-09T09:07:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.546+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f106c02d380 con 0x7f106c107570 2024-05-09T09:07:03.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.547+0000 7f105f7fe640 1 -- 172.21.15.64:0/3673936762 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7f10640bd380 con 0x7f106c107570 2024-05-09T09:07:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:03.548 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:07:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10580a2b90 msgr2=0x7f10580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10580a2b90 0x7f10580a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1068005e10 tx=0x7f1068005da0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 msgr2=0x7f106c1a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1a11a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1064009dc0 tx=0x7f1064003ba0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 shutdown_connections 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10580a2b90 0x7f10580a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f106c107570 0x7f106c1a11a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f106c104c40 0x7f106c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 --2- 172.21.15.64:0/3673936762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f106c102310 0x7f106c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 >> 172.21.15.64:0/3673936762 conn(0x7f106c0fbd20 msgr2=0x7f106c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 shutdown_connections 2024-05-09T09:07:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.549+0000 7f107456a640 1 -- 172.21.15.64:0/3673936762 wait complete. 2024-05-09T09:07:03.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:07:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 -- 172.21.15.64:0/804386166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758101da0 msgr2=0x7fd7581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 --2- 172.21.15.64:0/804386166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758101da0 0x7fd7581021c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd754009900 tx=0x7fd75402a410 comp rx=0 tx=0).stop 2024-05-09T09:07:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 -- 172.21.15.64:0/804386166 shutdown_connections 2024-05-09T09:07:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 --2- 172.21.15.64:0/804386166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7581013b0 0x7fd758108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 --2- 172.21.15.64:0/804386166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7581009f0 0x7fd758100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 --2- 172.21.15.64:0/804386166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758101da0 0x7fd7581021c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 -- 172.21.15.64:0/804386166 >> 172.21.15.64:0/804386166 conn(0x7fd7580fb4a0 msgr2=0x7fd7580fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 -- 172.21.15.64:0/804386166 shutdown_connections 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.666+0000 7fd76075a640 1 -- 172.21.15.64:0/804386166 wait complete. 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 Processor -- start 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 -- start start 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 0x7fd75810b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7581013b0 0x7fd758109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd758101da0 0x7fd758109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd75810d440 con 0x7fd7581009f0 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd75810d2c0 con 0x7fd7581013b0 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd76075a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd75810d5c0 con 0x7fd758101da0 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 0x7fd75810b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 0x7fd75810b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32814/0 (socket says 172.21.15.64:32814) 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 -- 172.21.15.64:0/3855973649 learned_addr learned my addr 172.21.15.64:0/3855973649 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75ecf7640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd758101da0 0x7fd758109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 -- 172.21.15.64:0/3855973649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd758101da0 msgr2=0x7fd758109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd758101da0 0x7fd758109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 -- 172.21.15.64:0/3855973649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7581013b0 msgr2=0x7fd758109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7581013b0 0x7fd758109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75e4f6640 1 -- 172.21.15.64:0/3855973649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75810a550 con 0x7fd7581009f0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.667+0000 7fd75ecf7640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd758101da0 0x7fd758109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.668+0000 7fd75e4f6640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 0x7fd75810b140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7540314e0 tx=0x7fd754004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.668+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75403f030 con 0x7fd7581009f0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.668+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7581a1bd0 con 0x7fd7581009f0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.668+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd754003cb0 con 0x7fd7581009f0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.668+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754035070 con 0x7fd7581009f0 2024-05-09T09:07:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.668+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7581a2120 con 0x7fd7581009f0 2024-05-09T09:07:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.669+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd758107ca0 con 0x7fd7581009f0 2024-05-09T09:07:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.670+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd754042030 con 0x7fd7581009f0 2024-05-09T09:07:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.671+0000 7fd74f7fe640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd7440a2b90 0x7fd7440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.671+0000 7fd75dcf5640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd7440a2b90 0x7fd7440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.671+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd7540ef940 con 0x7fd7581009f0 2024-05-09T09:07:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.671+0000 7fd75dcf5640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd7440a2b90 0x7fd7440a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd748005fd0 tx=0x7fd74800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.675+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd7540bc660 con 0x7fd7581009f0 2024-05-09T09:07:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.861+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd758062460 con 0x7fd7581009f0 2024-05-09T09:07:03.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.862+0000 7fd74f7fe640 1 -- 172.21.15.64:0/3855973649 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7fd7540bbf60 con 0x7fd7581009f0 2024-05-09T09:07:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:03.863 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd7440a2b90 msgr2=0x7fd7440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd7440a2b90 0x7fd7440a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd748005fd0 tx=0x7fd74800b410 comp rx=0 tx=0).stop 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 msgr2=0x7fd75810b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 0x7fd75810b140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7540314e0 tx=0x7fd754004270 comp rx=0 tx=0).stop 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 shutdown_connections 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd7440a2b90 0x7fd7440a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd758101da0 0x7fd758109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7581013b0 0x7fd758109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 --2- 172.21.15.64:0/3855973649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7581009f0 0x7fd75810b140 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 >> 172.21.15.64:0/3855973649 conn(0x7fd7580fb4a0 msgr2=0x7fd758104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 shutdown_connections 2024-05-09T09:07:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.865+0000 7fd76075a640 1 -- 172.21.15.64:0/3855973649 wait complete. 2024-05-09T09:07:03.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T09:07:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.984+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3722828007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18101880 msgr2=0x7fbf18101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.984+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3722828007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18101880 0x7fbf18101ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009900 tx=0x7fbf1402a410 comp rx=0 tx=0).stop 2024-05-09T09:07:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3722828007 shutdown_connections 2024-05-09T09:07:03.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3722828007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf18103430 0x7fbf18109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3722828007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18102a70 0x7fbf18102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3722828007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18101880 0x7fbf18101ca0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3722828007 >> 172.21.15.64:0/3722828007 conn(0x7fbf180fce10 msgr2=0x7fbf180ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3722828007 shutdown_connections 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3722828007 wait complete. 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 Processor -- start 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- start start 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18101880 0x7fbf18197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf18102a70 0x7fbf18197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 0x7fbf1819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1810d3b0 con 0x7fbf18103430 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1810d230 con 0x7fbf18101880 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.985+0000 7fbf1fdc1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1810d530 con 0x7fbf18102a70 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 0x7fbf1819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 0x7fbf1819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32840/0 (socket says 172.21.15.64:32840) 2024-05-09T09:07:03.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 -- 172.21.15.64:0/3470855980 learned_addr learned my addr 172.21.15.64:0/3470855980 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1d35c640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf18102a70 0x7fbf18197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1db5d640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18101880 0x7fbf18197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 -- 172.21.15.64:0/3470855980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf18102a70 msgr2=0x7fbf18197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf18102a70 0x7fbf18197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 -- 172.21.15.64:0/3470855980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18101880 msgr2=0x7fbf18197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18101880 0x7fbf18197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 -- 172.21.15.64:0/3470855980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1819f6d0 con 0x7fbf18103430 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1db5d640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18101880 0x7fbf18197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1e35e640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 0x7fbf1819f060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000eaa0 tx=0x7fbf1000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10004d20 con 0x7fbf18103430 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf10005020 con 0x7fbf18103430 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1819f9c0 con 0x7fbf18103430 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100116b0 con 0x7fbf18103430 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.986+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1819ff10 con 0x7fbf18103430 2024-05-09T09:07:03.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.987+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf18108c20 con 0x7fbf18103430 2024-05-09T09:07:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.989+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf100051d0 con 0x7fbf18103430 2024-05-09T09:07:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.990+0000 7fbf0effd640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbf040a2b90 0x7fbf040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.990+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbf100cf820 con 0x7fbf18103430 2024-05-09T09:07:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.990+0000 7fbf1db5d640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbf040a2b90 0x7fbf040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:03.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.990+0000 7fbf1db5d640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbf040a2b90 0x7fbf040a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf14000c00 tx=0x7fbf14032d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:03.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:03.993+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbf10016030 con 0x7fbf18103430 2024-05-09T09:07:04.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:04.220+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbf1802d1c0 con 0x7fbf18103430 2024-05-09T09:07:04.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:04 smithi179 ceph-mon[34086]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:04 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3673936762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:04.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:04 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3855973649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:04 smithi139 ceph-mon[33599]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:04 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3673936762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:04 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3855973649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:04.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:04 smithi064 ceph-mon[30149]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T09:07:04.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:04 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3673936762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:04.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:04 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3855973649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:05.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.203+0000 7fbf0effd640 1 -- 172.21.15.64:0/3470855980 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fbf1009c540 con 0x7fbf18103430 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbf040a2b90 msgr2=0x7fbf040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbf040a2b90 0x7fbf040a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf14000c00 tx=0x7fbf14032d20 comp rx=0 tx=0).stop 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 msgr2=0x7fbf1819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 0x7fbf1819f060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000eaa0 tx=0x7fbf1000eea0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 shutdown_connections 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbf040a2b90 0x7fbf040a5050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf18103430 0x7fbf1819f060 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf18102a70 0x7fbf18197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 --2- 172.21.15.64:0/3470855980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf18101880 0x7fbf18197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 >> 172.21.15.64:0/3470855980 conn(0x7fbf180fce10 msgr2=0x7fbf18105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:05.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 shutdown_connections 2024-05-09T09:07:05.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.206+0000 7fbf1fdc1640 1 -- 172.21.15.64:0/3470855980 wait complete. 2024-05-09T09:07:05.222 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-09T09:07:05.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 -- 172.21.15.64:0/3316368589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0103440 msgr2=0x7f26c0109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/3316368589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0103440 0x7f26c0109cd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009ca0 tx=0x7f26b802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 -- 172.21.15.64:0/3316368589 shutdown_connections 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/3316368589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0103440 0x7f26c0109cd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/3316368589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26c0102a80 0x7f26c0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/3316368589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0101890 0x7f26c0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 -- 172.21.15.64:0/3316368589 >> 172.21.15.64:0/3316368589 conn(0x7f26c00fce00 msgr2=0x7f26c00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 -- 172.21.15.64:0/3316368589 shutdown_connections 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.339+0000 7f26c7d5a640 1 -- 172.21.15.64:0/3316368589 wait complete. 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 Processor -- start 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 -- start start 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26c0101890 0x7f26c00755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 0x7f26c0073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0103440 0x7f26c0074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010d3c0 con 0x7f26c0102a80 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010d240 con 0x7f26c0103440 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c7d5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010d540 con 0x7f26c0101890 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 0x7f26c0073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 0x7f26c0073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37638/0 (socket says 172.21.15.64:37638) 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 -- 172.21.15.64:0/4170449241 learned_addr learned my addr 172.21.15.64:0/4170449241 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c5af6640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26c0101890 0x7f26c00755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c62f7640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0103440 0x7f26c0074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 -- 172.21.15.64:0/4170449241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26c0101890 msgr2=0x7f26c00755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26c0101890 0x7f26c00755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 -- 172.21.15.64:0/4170449241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0103440 msgr2=0x7f26c0074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0103440 0x7f26c0074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 -- 172.21.15.64:0/4170449241 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c00749f0 con 0x7f26c0102a80 2024-05-09T09:07:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c62f7640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0103440 0x7f26c0074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.340+0000 7f26c52f5640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 0x7f26c0073c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f26b000c970 tx=0x7f26b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.341+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0007bb0 con 0x7f26c0102a80 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.341+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26b0007d50 con 0x7f26c0102a80 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.341+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c01a1ba0 con 0x7f26c0102a80 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.341+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b000f450 con 0x7f26c0102a80 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.341+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c01a20f0 con 0x7f26c0102a80 2024-05-09T09:07:05.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.342+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c0108c80 con 0x7f26c0102a80 2024-05-09T09:07:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.343+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f26b0004f20 con 0x7f26c0102a80 2024-05-09T09:07:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.344+0000 7f26b6ffd640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26ac0a2c10 0x7f26ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.344+0000 7f26c5af6640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26ac0a2c10 0x7f26ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:05.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.344+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f26b00cefe0 con 0x7f26c0102a80 2024-05-09T09:07:05.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.344+0000 7f26c5af6640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26ac0a2c10 0x7f26ac0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26c01028e0 tx=0x7f26bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:05.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.348+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26b0014030 con 0x7f26c0102a80 2024-05-09T09:07:05.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:05 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3470855980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T09:07:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:05.572+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f26c00624d0 con 0x7f26c0102a80 2024-05-09T09:07:05.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:05 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3470855980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T09:07:05.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:05 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3470855980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T09:07:06.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.204+0000 7f26b6ffd640 1 -- 172.21.15.64:0/4170449241 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f26b009bd00 con 0x7f26c0102a80 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26ac0a2c10 msgr2=0x7f26ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26ac0a2c10 0x7f26ac0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26c01028e0 tx=0x7f26bc00b390 comp rx=0 tx=0).stop 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 msgr2=0x7f26c0073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 0x7f26c0073c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f26b000c970 tx=0x7f26b000ce30 comp rx=0 tx=0).stop 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 shutdown_connections 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26ac0a2c10 0x7f26ac0a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26c0103440 0x7f26c0074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26c0102a80 0x7f26c0073c20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 --2- 172.21.15.64:0/4170449241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26c0101890 0x7f26c00755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 >> 172.21.15.64:0/4170449241 conn(0x7f26c00fce00 msgr2=0x7f26c0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.208+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 shutdown_connections 2024-05-09T09:07:06.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.209+0000 7f26c7d5a640 1 -- 172.21.15.64:0/4170449241 wait complete. 2024-05-09T09:07:06.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-09T09:07:06.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 -- 172.21.15.64:0/962251823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0100770 msgr2=0x7f63c0100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:06.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 --2- 172.21.15.64:0/962251823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0100770 0x7f63c0100b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009900 tx=0x7f63b802a410 comp rx=0 tx=0).stop 2024-05-09T09:07:06.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 -- 172.21.15.64:0/962251823 shutdown_connections 2024-05-09T09:07:06.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 --2- 172.21.15.64:0/962251823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63c0102320 0x7f63c0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 --2- 172.21.15.64:0/962251823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63c0101960 0x7f63c0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 --2- 172.21.15.64:0/962251823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0100770 0x7f63c0100b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 -- 172.21.15.64:0/962251823 >> 172.21.15.64:0/962251823 conn(0x7f63c00fbce0 msgr2=0x7f63c00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 -- 172.21.15.64:0/962251823 shutdown_connections 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 -- 172.21.15.64:0/962251823 wait complete. 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 Processor -- start 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 -- start start 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63c0100770 0x7f63c0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.341+0000 7f63c83b7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63c0101960 0x7f63c019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c83b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 0x7f63c019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c83b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c2f0 con 0x7f63c0102320 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c83b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c170 con 0x7f63c0100770 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c83b7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c470 con 0x7f63c0101960 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 0x7f63c019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 0x7f63c019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37652/0 (socket says 172.21.15.64:37652) 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 -- 172.21.15.64:0/1992443709 learned_addr learned my addr 172.21.15.64:0/1992443709 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c5952640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63c0101960 0x7f63c019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6153640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63c0100770 0x7f63c0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 -- 172.21.15.64:0/1992443709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63c0101960 msgr2=0x7f63c019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63c0101960 0x7f63c019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 -- 172.21.15.64:0/1992443709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63c0100770 msgr2=0x7f63c0069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63c0100770 0x7f63c0069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 -- 172.21.15.64:0/1992443709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c019f7d0 con 0x7f63c0102320 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c6954640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 0x7f63c019eec0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b970 tx=0x7f63b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0019070 con 0x7f63c0102320 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63b0014440 con 0x7f63c0102320 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b00075d0 con 0x7f63c0102320 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c019fa60 con 0x7f63c0102320 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.342+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c019ffe0 con 0x7f63c0102320 2024-05-09T09:07:06.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.343+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0107b80 con 0x7f63c0102320 2024-05-09T09:07:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.344+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63b001c070 con 0x7f63c0102320 2024-05-09T09:07:06.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.345+0000 7f63bf7fe640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f63a80a2b90 0x7f63a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:06.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.345+0000 7f63c6153640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f63a80a2b90 0x7f63a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:06.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.345+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f63b00ce900 con 0x7f63c0102320 2024-05-09T09:07:06.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.346+0000 7f63c6153640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f63a80a2b90 0x7f63a80a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000c00 tx=0x7f63b80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.350+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63b009b620 con 0x7f63c0102320 2024-05-09T09:07:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:06 smithi179 ceph-mon[34086]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T09:07:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3470855980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T09:07:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:06 smithi179 ceph-mon[34086]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:07:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4170449241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T09:07:06.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:06.572+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f63c0062430 con 0x7f63c0102320 2024-05-09T09:07:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:06 smithi139 ceph-mon[33599]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T09:07:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3470855980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T09:07:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:06 smithi139 ceph-mon[33599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:07:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4170449241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T09:07:06.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:06 smithi064 ceph-mon[30149]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T09:07:06.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3470855980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T09:07:06.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:06 smithi064 ceph-mon[30149]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:07:06.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4170449241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T09:07:07.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.217+0000 7f63bf7fe640 1 -- 172.21.15.64:0/1992443709 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f63b009af20 con 0x7f63c0102320 2024-05-09T09:07:07.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f63a80a2b90 msgr2=0x7f63a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:07.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f63a80a2b90 0x7f63a80a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000c00 tx=0x7f63b80314a0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 msgr2=0x7f63c019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:07.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 0x7f63c019eec0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b970 tx=0x7f63b000be30 comp rx=0 tx=0).stop 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 shutdown_connections 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f63a80a2b90 0x7f63a80a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63c0102320 0x7f63c019eec0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63c0101960 0x7f63c019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 --2- 172.21.15.64:0/1992443709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63c0100770 0x7f63c0069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 >> 172.21.15.64:0/1992443709 conn(0x7f63c00fbce0 msgr2=0x7f63c0104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 shutdown_connections 2024-05-09T09:07:07.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.220+0000 7f63c83b7640 1 -- 172.21.15.64:0/1992443709 wait complete. 2024-05-09T09:07:07.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- 172.21.15.64:0/393588370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c107550 msgr2=0x7f5a7c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- 172.21.15.64:0/393588370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c107550 0x7f5a7c1099a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009ca0 tx=0x7f5a6c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- 172.21.15.64:0/393588370 shutdown_connections 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- 172.21.15.64:0/393588370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c107550 0x7f5a7c1099a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- 172.21.15.64:0/393588370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a7c104c20 0x7f5a7c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- 172.21.15.64:0/393588370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- 172.21.15.64:0/393588370 >> 172.21.15.64:0/393588370 conn(0x7f5a7c0fbce0 msgr2=0x7f5a7c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:07.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- 172.21.15.64:0/393588370 shutdown_connections 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- 172.21.15.64:0/393588370 wait complete. 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 Processor -- start 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- start start 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a7c104c20 0x7f5a7c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a7c107550 0x7f5a7c1007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c10d3c0 con 0x7f5a7c1022f0 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c10d240 con 0x7f5a7c107550 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.351+0000 7f5a8309a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c10d540 con 0x7f5a7c104c20 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37664/0 (socket says 172.21.15.64:37664) 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 -- 172.21.15.64:0/1067136652 learned_addr learned my addr 172.21.15.64:0/1067136652 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a81637640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a7c107550 0x7f5a7c1007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:07.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a7bfff640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a7c104c20 0x7f5a7c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 -- 172.21.15.64:0/1067136652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a7c104c20 msgr2=0x7f5a7c1002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a7c104c20 0x7f5a7c1002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 -- 172.21.15.64:0/1067136652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a7c107550 msgr2=0x7f5a7c1007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a7c107550 0x7f5a7c1007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 -- 172.21.15.64:0/1067136652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c101070 con 0x7f5a7c1022f0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a80e36640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c101c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400d770 tx=0x7f5a7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74017070 con 0x7f5a7c1022f0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a740054d0 con 0x7f5a7c1022f0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c1a1bf0 con 0x7f5a7c1022f0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.352+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74004cf0 con 0x7f5a7c1022f0 2024-05-09T09:07:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.353+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c1a20f0 con 0x7f5a7c1022f0 2024-05-09T09:07:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.354+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7c068650 con 0x7f5a7c1022f0 2024-05-09T09:07:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.354+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a74004e90 con 0x7f5a7c1022f0 2024-05-09T09:07:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.355+0000 7f5a79ffb640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a680a2b90 0x7f5a680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.355+0000 7f5a7bfff640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a680a2b90 0x7f5a680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.355+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5a740cec80 con 0x7f5a7c1022f0 2024-05-09T09:07:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.356+0000 7f5a7bfff640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a680a2b90 0x7f5a680a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c1019d0 tx=0x7f5a64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:07.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.360+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a7409b350 con 0x7f5a7c1022f0 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4170449241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-09T09:07:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1992443709' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T09:07:07.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T09:07:07.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4170449241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T09:07:07.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T09:07:07.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:07:07.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-09T09:07:07.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-09T09:07:07.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1992443709' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T09:07:07.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:07.584+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5a7c062430 con 0x7f5a7c1022f0 2024-05-09T09:07:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T09:07:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4170449241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T09:07:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T09:07:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T09:07:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-09T09:07:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-09T09:07:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1992443709' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T09:07:08.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.226+0000 7f5a79ffb640 1 -- 172.21.15.64:0/1067136652 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f5a7409f650 con 0x7f5a7c1022f0 2024-05-09T09:07:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a680a2b90 msgr2=0x7f5a680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a680a2b90 0x7f5a680a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c1019d0 tx=0x7f5a64005f90 comp rx=0 tx=0).stop 2024-05-09T09:07:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 msgr2=0x7f5a7c101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c101c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400d770 tx=0x7f5a7400dc30 comp rx=0 tx=0).stop 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 shutdown_connections 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a680a2b90 0x7f5a680a5050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a7c107550 0x7f5a7c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a7c104c20 0x7f5a7c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 --2- 172.21.15.64:0/1067136652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7c1022f0 0x7f5a7c101c60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 >> 172.21.15.64:0/1067136652 conn(0x7f5a7c0fbce0 msgr2=0x7f5a7c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 shutdown_connections 2024-05-09T09:07:08.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.229+0000 7f5a8309a640 1 -- 172.21.15.64:0/1067136652 wait complete. 2024-05-09T09:07:08.243 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-09T09:07:08.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- 172.21.15.64:0/4032253472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 msgr2=0x7f795c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- 172.21.15.64:0/4032253472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c100b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7954009900 tx=0x7f795402a410 comp rx=0 tx=0).stop 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- 172.21.15.64:0/4032253472 shutdown_connections 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- 172.21.15.64:0/4032253472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 0x7f795c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- 172.21.15.64:0/4032253472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c101960 0x7f795c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- 172.21.15.64:0/4032253472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c100b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- 172.21.15.64:0/4032253472 >> 172.21.15.64:0/4032253472 conn(0x7f795c0fbce0 msgr2=0x7f795c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- 172.21.15.64:0/4032253472 shutdown_connections 2024-05-09T09:07:08.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- 172.21.15.64:0/4032253472 wait complete. 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 Processor -- start 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- start start 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c101960 0x7f795c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 0x7f795c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10c2c0 con 0x7f795c100770 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10c140 con 0x7f795c101960 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.356+0000 7f7964734640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10c440 con 0x7f795c102320 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37694/0 (socket says 172.21.15.64:37694) 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 -- 172.21.15.64:0/3850258305 learned_addr learned my addr 172.21.15.64:0/3850258305 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f7961ccf640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c101960 0x7f795c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f7962cd1640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 0x7f795c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 -- 172.21.15.64:0/3850258305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 msgr2=0x7f795c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 0x7f795c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 -- 172.21.15.64:0/3850258305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c101960 msgr2=0x7f795c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c101960 0x7f795c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 -- 172.21.15.64:0/3850258305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c19f6e0 con 0x7f795c100770 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f7962cd1640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 0x7f795c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f79624d0640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c197ac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79540314e0 tx=0x7f7954003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795403f070 con 0x7f795c100770 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c19f970 con 0x7f795c100770 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7954031c40 con 0x7f795c100770 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954035030 con 0x7f795c100770 2024-05-09T09:07:08.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.357+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c19fe60 con 0x7f795c100770 2024-05-09T09:07:08.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.358+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c1a0090 con 0x7f795c100770 2024-05-09T09:07:08.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.360+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7954042070 con 0x7f795c100770 2024-05-09T09:07:08.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.360+0000 7f795b7fe640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79440a2b90 0x7f79440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.360+0000 7f7961ccf640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79440a2b90 0x7f79440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.360+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f79540f0070 con 0x7f795c100770 2024-05-09T09:07:08.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.361+0000 7f7961ccf640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79440a2b90 0x7f79440a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f795c198fd0 tx=0x7f7950005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:08.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.364+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79540bcd90 con 0x7f795c100770 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1992443709' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] up:active 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T09:07:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:08 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1067136652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T09:07:08.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.571+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f795c069100 con 0x7f79440a2b90 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1992443709' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] up:active 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:07:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-09T09:07:08.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-09T09:07:08.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T09:07:08.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:08 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1067136652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T09:07:08.575 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T09:07:08.575 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1992443709' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] up:active 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T09:07:08.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:08 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1067136652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T09:07:08.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.590+0000 7f795b7fe640 1 -- 172.21.15.64:0/3850258305 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f795c069100 con 0x7f79440a2b90 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79440a2b90 msgr2=0x7f79440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79440a2b90 0x7f79440a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f795c198fd0 tx=0x7f7950005f90 comp rx=0 tx=0).stop 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 msgr2=0x7f795c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c197ac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79540314e0 tx=0x7f7954003c10 comp rx=0 tx=0).stop 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 shutdown_connections 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f79440a2b90 0x7f79440a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f795c102320 0x7f795c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c101960 0x7f795c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 --2- 172.21.15.64:0/3850258305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f795c100770 0x7f795c197ac0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 >> 172.21.15.64:0/3850258305 conn(0x7f795c0fbce0 msgr2=0x7f795c0689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:08.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 shutdown_connections 2024-05-09T09:07:08.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.592+0000 7f7964734640 1 -- 172.21.15.64:0/3850258305 wait complete. 2024-05-09T09:07:08.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T09:07:08.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 -- 172.21.15.64:0/2096104546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0107570 msgr2=0x7f83a01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 --2- 172.21.15.64:0/2096104546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0107570 0x7f83a01099c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8398009ca0 tx=0x7f839802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 -- 172.21.15.64:0/2096104546 shutdown_connections 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 --2- 172.21.15.64:0/2096104546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0107570 0x7f83a01099c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 --2- 172.21.15.64:0/2096104546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a0104c40 0x7f83a0107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 --2- 172.21.15.64:0/2096104546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83a0102310 0x7f83a0104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 -- 172.21.15.64:0/2096104546 >> 172.21.15.64:0/2096104546 conn(0x7f83a00fbd20 msgr2=0x7f83a00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 -- 172.21.15.64:0/2096104546 shutdown_connections 2024-05-09T09:07:08.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 -- 172.21.15.64:0/2096104546 wait complete. 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.713+0000 7f83a6790640 1 Processor -- start 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 -- start start 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83a0102310 0x7f83a0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 0x7f83a0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a0107570 0x7f83a019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010d3e0 con 0x7f83a0104c40 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010d260 con 0x7f83a0107570 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010d560 con 0x7f83a0102310 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 0x7f83a0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 0x7f83a0197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37728/0 (socket says 172.21.15.64:37728) 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 -- 172.21.15.64:0/3222780284 learned_addr learned my addr 172.21.15.64:0/3222780284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a4d2d640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a0107570 0x7f83a019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f839ffff640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83a0102310 0x7f83a0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 -- 172.21.15.64:0/3222780284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83a0102310 msgr2=0x7f83a0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83a0102310 0x7f83a0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 -- 172.21.15.64:0/3222780284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a0107570 msgr2=0x7f83a019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a0107570 0x7f83a019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 -- 172.21.15.64:0/3222780284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a019f6d0 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f8397fff640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 0x7f83a0197ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9c0 tx=0x7f838800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838800cd80 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83880054d0 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a019f9c0 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.714+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388010430 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.715+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a019ff10 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.715+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a00fd810 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.717+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8388004fd0 con 0x7f83a0104c40 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.717+0000 7f839dffb640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f838c0a2b90 0x7f838c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:08.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.717+0000 7f839ffff640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f838c0a2b90 0x7f838c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.717+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f83880cf720 con 0x7f83a0104c40 2024-05-09T09:07:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.718+0000 7f839ffff640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f838c0a2b90 0x7f838c0a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8390005fd0 tx=0x7f8390005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:08.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.722+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f838809c440 con 0x7f83a0104c40 2024-05-09T09:07:08.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:07:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:07:08.575+0000 7f4d89705700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:07:08.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:07:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:07:08.575+0000 7f4d89705700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:07:08.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:07:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:07:08.575+0000 7f4d89705700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:07:08.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:07:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:07:08.575+0000 7f4d89705700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:07:08.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:07:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:07:08.575+0000 7f4d89705700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T09:07:08.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.934+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f83a0061c70 con 0x7f838c0a2b90 2024-05-09T09:07:08.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.958+0000 7f839dffb640 1 -- 172.21.15.64:0/3222780284 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83a0061c70 con 0x7f838c0a2b90 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f838c0a2b90 msgr2=0x7f838c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f838c0a2b90 0x7f838c0a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8390005fd0 tx=0x7f8390005850 comp rx=0 tx=0).stop 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 msgr2=0x7f83a0197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 0x7f83a0197ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9c0 tx=0x7f838800ee80 comp rx=0 tx=0).stop 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 shutdown_connections 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f838c0a2b90 0x7f838c0a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a0107570 0x7f83a019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83a0104c40 0x7f83a0197ff0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 --2- 172.21.15.64:0/3222780284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83a0102310 0x7f83a0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:08.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 >> 172.21.15.64:0/3222780284 conn(0x7f83a00fbd20 msgr2=0x7f83a0102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:08.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 shutdown_connections 2024-05-09T09:07:08.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:08.960+0000 7f83a6790640 1 -- 172.21.15.64:0/3222780284 wait complete. 2024-05-09T09:07:08.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T09:07:09.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 -- 172.21.15.64:0/3811840749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 msgr2=0x7feb2c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 --2- 172.21.15.64:0/3811840749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c101de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb20009900 tx=0x7feb2002a410 comp rx=0 tx=0).stop 2024-05-09T09:07:09.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 -- 172.21.15.64:0/3811840749 shutdown_connections 2024-05-09T09:07:09.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 --2- 172.21.15.64:0/3811840749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2c102320 0x7feb2c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 --2- 172.21.15.64:0/3811840749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c101de0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 --2- 172.21.15.64:0/3811840749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c100770 0x7feb2c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 -- 172.21.15.64:0/3811840749 >> 172.21.15.64:0/3811840749 conn(0x7feb2c0fbce0 msgr2=0x7feb2c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 -- 172.21.15.64:0/3811840749 shutdown_connections 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.081+0000 7feb3275a640 1 -- 172.21.15.64:0/3811840749 wait complete. 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 Processor -- start 2024-05-09T09:07:09.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 -- start start 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c100770 0x7feb2c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2c102320 0x7feb2c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c10d3c0 con 0x7feb2c101960 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c10d240 con 0x7feb2c100770 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb3275a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c10d540 con 0x7feb2c102320 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37738/0 (socket says 172.21.15.64:37738) 2024-05-09T09:07:09.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 -- 172.21.15.64:0/3802484552 learned_addr learned my addr 172.21.15.64:0/3802484552 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2bfff640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c100770 0x7feb2c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb30cf7640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2c102320 0x7feb2c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 -- 172.21.15.64:0/3802484552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2c102320 msgr2=0x7feb2c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2c102320 0x7feb2c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 -- 172.21.15.64:0/3802484552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c100770 msgr2=0x7feb2c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c100770 0x7feb2c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.082+0000 7feb2b7fe640 1 -- 172.21.15.64:0/3802484552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c19f750 con 0x7feb2c101960 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.083+0000 7feb2b7fe640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c198070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feb20004770 tx=0x7feb200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.083+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2003f030 con 0x7feb2c101960 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.083+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c19f9e0 con 0x7feb2c101960 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.083+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb20003cb0 con 0x7feb2c101960 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.083+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20035070 con 0x7feb2c101960 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.083+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c19ff20 con 0x7feb2c101960 2024-05-09T09:07:09.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.084+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c107ba0 con 0x7feb2c101960 2024-05-09T09:07:09.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.085+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feb20042030 con 0x7feb2c101960 2024-05-09T09:07:09.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.086+0000 7feb297fa640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb140a2b90 0x7feb140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.086+0000 7feb2bfff640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb140a2b90 0x7feb140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.086+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7feb200efb60 con 0x7feb2c101960 2024-05-09T09:07:09.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.086+0000 7feb2bfff640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb140a2b90 0x7feb140a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005fd0 tx=0x7feb1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:09.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.090+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feb200bc230 con 0x7feb2c101960 2024-05-09T09:07:09.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.249+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feb200b88a0 con 0x7feb2c101960 2024-05-09T09:07:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.313+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7feb2c061d00 con 0x7feb140a2b90 2024-05-09T09:07:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.319+0000 7feb297fa640 1 -- 172.21.15.64:0/3802484552 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7feb2c061d00 con 0x7feb140a2b90 2024-05-09T09:07:09.318 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85 2024-05-09T09:07:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb140a2b90 msgr2=0x7feb140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb140a2b90 0x7feb140a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005fd0 tx=0x7feb1c00b410 comp rx=0 tx=0).stop 2024-05-09T09:07:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 msgr2=0x7feb2c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c198070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feb20004770 tx=0x7feb200042f0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 shutdown_connections 2024-05-09T09:07:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb140a2b90 0x7feb140a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2c102320 0x7feb2c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb2c101960 0x7feb2c198070 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 --2- 172.21.15.64:0/3802484552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c100770 0x7feb2c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 >> 172.21.15.64:0/3802484552 conn(0x7feb2c0fbce0 msgr2=0x7feb2c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 shutdown_connections 2024-05-09T09:07:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.321+0000 7feb3275a640 1 -- 172.21.15.64:0/3802484552 wait complete. 2024-05-09T09:07:09.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T09:07:09.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 -- 172.21.15.64:0/1971345103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24104c40 msgr2=0x7fcf24107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 --2- 172.21.15.64:0/1971345103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24104c40 0x7fcf24107030 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009900 tx=0x7fcf1802a410 comp rx=0 tx=0).stop 2024-05-09T09:07:09.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 -- 172.21.15.64:0/1971345103 shutdown_connections 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 --2- 172.21.15.64:0/1971345103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24107570 0x7fcf241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 --2- 172.21.15.64:0/1971345103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24104c40 0x7fcf24107030 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 --2- 172.21.15.64:0/1971345103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf24102310 0x7fcf24104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 -- 172.21.15.64:0/1971345103 >> 172.21.15.64:0/1971345103 conn(0x7fcf240fbd20 msgr2=0x7fcf240fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 -- 172.21.15.64:0/1971345103 shutdown_connections 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.443+0000 7fcf29c09640 1 -- 172.21.15.64:0/1971345103 wait complete. 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 Processor -- start 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 -- start start 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 0x7fcf24197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24104c40 0x7fcf24198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf24107570 0x7fcf2419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410d3b0 con 0x7fcf24102310 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410d230 con 0x7fcf24107570 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf29c09640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410d530 con 0x7fcf24104c40 2024-05-09T09:07:09.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 0x7fcf24197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 0x7fcf24197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37752/0 (socket says 172.21.15.64:37752) 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 -- 172.21.15.64:0/2260391605 learned_addr learned my addr 172.21.15.64:0/2260391605 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf23fff640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf24107570 0x7fcf2419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24104c40 0x7fcf24198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 -- 172.21.15.64:0/2260391605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24104c40 msgr2=0x7fcf24198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24104c40 0x7fcf24198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 -- 172.21.15.64:0/2260391605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf24107570 msgr2=0x7fcf2419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf24107570 0x7fcf2419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.444+0000 7fcf237fe640 1 -- 172.21.15.64:0/2260391605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2419f720 con 0x7fcf24102310 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24104c40 0x7fcf24198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf237fe640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 0x7fcf24197b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400d950 tx=0x7fcf1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14017070 con 0x7fcf24102310 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf14009c30 con 0x7fcf24102310 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2419fa10 con 0x7fcf24102310 2024-05-09T09:07:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400f620 con 0x7fcf24102310 2024-05-09T09:07:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.445+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2419ff30 con 0x7fcf24102310 2024-05-09T09:07:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.446+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf240fd810 con 0x7fcf24102310 2024-05-09T09:07:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.447+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcf140109a0 con 0x7fcf24102310 2024-05-09T09:07:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.448+0000 7fcf20ff9640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf0c0a2c20 0x7fcf0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.448+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf0c0a2c20 0x7fcf0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.448+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcf140cf060 con 0x7fcf24102310 2024-05-09T09:07:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.448+0000 7fcf22ffd640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf0c0a2c20 0x7fcf0c0a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf18000c00 tx=0x7fcf18032c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.453+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcf14009de0 con 0x7fcf24102310 2024-05-09T09:07:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1067136652' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T09:07:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:09 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] up:active 2024-05-09T09:07:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:09 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-09T09:07:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:09 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:07:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:09 smithi179 ceph-mon[34086]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:09.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1067136652' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T09:07:09.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:09 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] up:active 2024-05-09T09:07:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:09 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-09T09:07:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:09 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:07:09.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:09 smithi139 ceph-mon[33599]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1067136652' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T09:07:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:09 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] up:active 2024-05-09T09:07:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:09 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-09T09:07:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:09 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:07:09.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:09 smithi064 ceph-mon[30149]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:09.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.666+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcf24061c70 con 0x7fcf0c0a2c20 2024-05-09T09:07:09.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.691+0000 7fcf20ff9640 1 -- 172.21.15.64:0/2260391605 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcf24061c70 con 0x7fcf0c0a2c20 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf0c0a2c20 msgr2=0x7fcf0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf0c0a2c20 0x7fcf0c0a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf18000c00 tx=0x7fcf18032c40 comp rx=0 tx=0).stop 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 msgr2=0x7fcf24197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 0x7fcf24197b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400d950 tx=0x7fcf1400de10 comp rx=0 tx=0).stop 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 shutdown_connections 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf0c0a2c20 0x7fcf0c0a50e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf24107570 0x7fcf2419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf24104c40 0x7fcf24198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 --2- 172.21.15.64:0/2260391605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf24102310 0x7fcf24197b50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 >> 172.21.15.64:0/2260391605 conn(0x7fcf240fbd20 msgr2=0x7fcf24102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:09.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 shutdown_connections 2024-05-09T09:07:09.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.693+0000 7fcf29c09640 1 -- 172.21.15.64:0/2260391605 wait complete. 2024-05-09T09:07:09.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 -- 172.21.15.64:0/2417555056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 msgr2=0x7ff160100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 --2- 172.21.15.64:0/2417555056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160100bb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009900 tx=0x7ff15c02a410 comp rx=0 tx=0).stop 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 -- 172.21.15.64:0/2417555056 shutdown_connections 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 --2- 172.21.15.64:0/2417555056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 0x7ff160108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 --2- 172.21.15.64:0/2417555056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff160101980 0x7ff160101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 --2- 172.21.15.64:0/2417555056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160100bb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 -- 172.21.15.64:0/2417555056 >> 172.21.15.64:0/2417555056 conn(0x7ff1600fbd20 msgr2=0x7ff1600fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 -- 172.21.15.64:0/2417555056 shutdown_connections 2024-05-09T09:07:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.814+0000 7ff167f29640 1 -- 172.21.15.64:0/2417555056 wait complete. 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 Processor -- start 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 -- start start 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff160101980 0x7ff160196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 0x7ff16019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff16010c290 con 0x7ff160100790 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff16010c110 con 0x7ff160102340 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff167f29640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff16010c410 con 0x7ff160101980 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37762/0 (socket says 172.21.15.64:37762) 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 -- 172.21.15.64:0/3177158398 learned_addr learned my addr 172.21.15.64:0/3177158398 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff1654c4640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff160101980 0x7ff160196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff1664c6640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 0x7ff16019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 -- 172.21.15.64:0/3177158398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff160101980 msgr2=0x7ff160196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff160101980 0x7ff160196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 -- 172.21.15.64:0/3177158398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 msgr2=0x7ff16019dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 0x7ff16019dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 -- 172.21.15.64:0/3177158398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16019e640 con 0x7ff160100790 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff1664c6640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 0x7ff16019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.815+0000 7ff165cc5640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160196a20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0314e0 tx=0x7ff15c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.816+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c03f030 con 0x7ff160100790 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.816+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff15c003cb0 con 0x7ff160100790 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.816+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16019e8d0 con 0x7ff160100790 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.816+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c035070 con 0x7ff160100790 2024-05-09T09:07:09.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.816+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16019ee10 con 0x7ff160100790 2024-05-09T09:07:09.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.817+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff160107c20 con 0x7ff160100790 2024-05-09T09:07:09.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.818+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff15c042030 con 0x7ff160100790 2024-05-09T09:07:09.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.818+0000 7ff156ffd640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff14c0a2ca0 0x7ff14c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:09.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.818+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff15c0efa80 con 0x7ff160100790 2024-05-09T09:07:09.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.819+0000 7ff1654c4640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff14c0a2ca0 0x7ff14c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:09.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.819+0000 7ff1654c4640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff14c0a2ca0 0x7ff14c0a5160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff150005d00 tx=0x7ff150005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:09.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:09.823+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff15c0bc640 con 0x7ff160100790 2024-05-09T09:07:10.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.035+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff160061d00 con 0x7ff14c0a2ca0 2024-05-09T09:07:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.040+0000 7ff156ffd640 1 -- 172.21.15.64:0/3177158398 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff160061d00 con 0x7ff14c0a2ca0 2024-05-09T09:07:10.040 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_1/f0228f2a-0afc-4524-b9a8-ab2c0af2d559 2024-05-09T09:07:10.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff14c0a2ca0 msgr2=0x7ff14c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff14c0a2ca0 0x7ff14c0a5160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff150005d00 tx=0x7ff150005c50 comp rx=0 tx=0).stop 2024-05-09T09:07:10.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 msgr2=0x7ff160196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160196a20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0314e0 tx=0x7ff15c0042e0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 shutdown_connections 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff14c0a2ca0 0x7ff14c0a5160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff160102340 0x7ff16019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff160101980 0x7ff160196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 --2- 172.21.15.64:0/3177158398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff160100790 0x7ff160196a20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 >> 172.21.15.64:0/3177158398 conn(0x7ff1600fbd20 msgr2=0x7ff160104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:10.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.042+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 shutdown_connections 2024-05-09T09:07:10.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.043+0000 7ff167f29640 1 -- 172.21.15.64:0/3177158398 wait complete. 2024-05-09T09:07:10.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:07:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.158+0000 7f6db6eba640 1 -- 172.21.15.64:0/1098258009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 msgr2=0x7f6db0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.158+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1098258009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0101de0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009900 tx=0x7f6da002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- 172.21.15.64:0/1098258009 shutdown_connections 2024-05-09T09:07:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1098258009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 0x7f6db0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1098258009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0101de0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1098258009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0100770 0x7f6db0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- 172.21.15.64:0/1098258009 >> 172.21.15.64:0/1098258009 conn(0x7f6db00fbce0 msgr2=0x7f6db00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- 172.21.15.64:0/1098258009 shutdown_connections 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- 172.21.15.64:0/1098258009 wait complete. 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 Processor -- start 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- start start 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0100770 0x7f6db010b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 0x7f6db0109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010d420 con 0x7f6db0101960 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010d2a0 con 0x7f6db0102320 2024-05-09T09:07:10.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6db6eba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010d5a0 con 0x7f6db0100770 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.159+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37784/0 (socket says 172.21.15.64:37784) 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 -- 172.21.15.64:0/1977513569 learned_addr learned my addr 172.21.15.64:0/1977513569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6db4c56640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0100770 0x7f6db010b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6db5457640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 0x7f6db0109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 -- 172.21.15.64:0/1977513569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0100770 msgr2=0x7f6db010b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0100770 0x7f6db010b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 -- 172.21.15.64:0/1977513569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 msgr2=0x7f6db0109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 0x7f6db0109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 -- 172.21.15.64:0/1977513569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db010a500 con 0x7f6db0101960 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6db5457640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 0x7f6db0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da7fff640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0109730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6da00314e0 tx=0x7f6da0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da003f070 con 0x7f6db0101960 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da0031e40 con 0x7f6db0101960 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0035030 con 0x7f6db0101960 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db01a1a60 con 0x7f6db0101960 2024-05-09T09:07:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.160+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db01a1f60 con 0x7f6db0101960 2024-05-09T09:07:10.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.162+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db0107a30 con 0x7f6db0101960 2024-05-09T09:07:10.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.163+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6da0042070 con 0x7f6db0101960 2024-05-09T09:07:10.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.164+0000 7f6da5ffb640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d980a2ca0 0x7f6d980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.164+0000 7f6db4c56640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d980a2ca0 0x7f6d980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.164+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6da00efe60 con 0x7f6db0101960 2024-05-09T09:07:10.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.164+0000 7f6db4c56640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d980a2ca0 0x7f6d980a5160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0058c0 tx=0x7f6dac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:10.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.168+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6da00bca20 con 0x7f6db0101960 2024-05-09T09:07:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.396+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6db002d1c0 con 0x7f6db0101960 2024-05-09T09:07:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.397+0000 7f6da5ffb640 1 -- 172.21.15.64:0/1977513569 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f6da00bc320 con 0x7f6db0101960 2024-05-09T09:07:10.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:10.396 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:07:08.230747+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:07:10.396 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:07:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d980a2ca0 msgr2=0x7f6d980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d980a2ca0 0x7f6d980a5160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0058c0 tx=0x7f6dac005850 comp rx=0 tx=0).stop 2024-05-09T09:07:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 msgr2=0x7f6db0109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0109730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6da00314e0 tx=0x7f6da0003d50 comp rx=0 tx=0).stop 2024-05-09T09:07:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 shutdown_connections 2024-05-09T09:07:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d980a2ca0 0x7f6d980a5160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6db0102320 0x7f6db0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db0101960 0x7f6db0109730 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 --2- 172.21.15.64:0/1977513569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0100770 0x7f6db010b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 >> 172.21.15.64:0/1977513569 conn(0x7f6db00fbce0 msgr2=0x7f6db0104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 shutdown_connections 2024-05-09T09:07:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.399+0000 7f6db6eba640 1 -- 172.21.15.64:0/1977513569 wait complete. 2024-05-09T09:07:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:10 smithi064 ceph-mon[30149]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:10 smithi064 ceph-mon[30149]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-09T09:07:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:10 smithi064 ceph-mon[30149]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T09:07:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:10 smithi064 ceph-mon[30149]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:10 smithi064 ceph-mon[30149]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:07:10.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 -- 172.21.15.64:0/1461100370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 msgr2=0x7ff00c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 --2- 172.21.15.64:0/1461100370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c1099c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff004009900 tx=0x7ff00402a410 comp rx=0 tx=0).stop 2024-05-09T09:07:10.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 -- 172.21.15.64:0/1461100370 shutdown_connections 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 --2- 172.21.15.64:0/1461100370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c1099c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 --2- 172.21.15.64:0/1461100370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 0x7ff00c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 --2- 172.21.15.64:0/1461100370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 0x7ff00c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 -- 172.21.15.64:0/1461100370 >> 172.21.15.64:0/1461100370 conn(0x7ff00c0fbd20 msgr2=0x7ff00c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 -- 172.21.15.64:0/1461100370 shutdown_connections 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.515+0000 7ff012b8b640 1 -- 172.21.15.64:0/1461100370 wait complete. 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 Processor -- start 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 -- start start 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 0x7ff00c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 0x7ff00c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d3c0 con 0x7ff00c107570 2024-05-09T09:07:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d240 con 0x7ff00c102310 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff012b8b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d540 con 0x7ff00c104c40 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff011128640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff011128640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37804/0 (socket says 172.21.15.64:37804) 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.516+0000 7ff011128640 1 -- 172.21.15.64:0/3219190589 learned_addr learned my addr 172.21.15.64:0/3219190589 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff00bfff640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 0x7ff00c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff010927640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 0x7ff00c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff011128640 1 -- 172.21.15.64:0/3219190589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 msgr2=0x7ff00c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff011128640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 0x7ff00c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff011128640 1 -- 172.21.15.64:0/3219190589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 msgr2=0x7ff00c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff011128640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 0x7ff00c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff011128640 1 -- 172.21.15.64:0/3219190589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c19f680 con 0x7ff00c107570 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff00bfff640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 0x7ff00c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff010927640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 0x7ff00c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:07:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff011128640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c19f010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff004005940 tx=0x7ff004031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00403f030 con 0x7ff00c107570 2024-05-09T09:07:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c19f910 con 0x7ff00c107570 2024-05-09T09:07:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff004003aa0 con 0x7ff00c107570 2024-05-09T09:07:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004035070 con 0x7ff00c107570 2024-05-09T09:07:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.517+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c19fe00 con 0x7ff00c107570 2024-05-09T09:07:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.518+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c0fd810 con 0x7ff00c107570 2024-05-09T09:07:10.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.520+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff004042030 con 0x7ff00c107570 2024-05-09T09:07:10.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.520+0000 7ff009ffb640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feff80a2c20 0x7feff80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.520+0000 7ff010927640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feff80a2c20 0x7feff80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.520+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0040eff20 con 0x7ff00c107570 2024-05-09T09:07:10.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.521+0000 7ff010927640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feff80a2c20 0x7feff80a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feff40058c0 tx=0x7feff4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:10.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.523+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0040bcae0 con 0x7ff00c107570 2024-05-09T09:07:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-mon[33599]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-mon[33599]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-09T09:07:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-mon[33599]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T09:07:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-mon[33599]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-mon[33599]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.573 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:07:10.334+0000 7f3d1c2fe700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T09:07:10.573 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:07:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:07:10.334+0000 7f3d1c2fe700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-09T09:07:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.707+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff00c02d340 con 0x7ff00c107570 2024-05-09T09:07:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.709+0000 7ff009ffb640 1 -- 172.21.15.64:0/3219190589 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7ff0040bc3e0 con 0x7ff00c107570 2024-05-09T09:07:10.708 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:10.709 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:07:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feff80a2c20 msgr2=0x7feff80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feff80a2c20 0x7feff80a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feff40058c0 tx=0x7feff4005d30 comp rx=0 tx=0).stop 2024-05-09T09:07:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 msgr2=0x7ff00c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c19f010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff004005940 tx=0x7ff004031a30 comp rx=0 tx=0).stop 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 shutdown_connections 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feff80a2c20 0x7feff80a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff00c107570 0x7ff00c19f010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff00c104c40 0x7ff00c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 --2- 172.21.15.64:0/3219190589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff00c102310 0x7ff00c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.711+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 >> 172.21.15.64:0/3219190589 conn(0x7ff00c0fbd20 msgr2=0x7ff00c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.712+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 shutdown_connections 2024-05-09T09:07:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.712+0000 7ff012b8b640 1 -- 172.21.15.64:0/3219190589 wait complete. 2024-05-09T09:07:10.722 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T09:07:10.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:07:10.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:10 smithi179 ceph-mon[34086]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:10 smithi179 ceph-mon[34086]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-09T09:07:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:10 smithi179 ceph-mon[34086]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T09:07:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:10 smithi179 ceph-mon[34086]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:10 smithi179 ceph-mon[34086]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 -- 172.21.15.64:0/211963891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4102340 msgr2=0x7fdeb4108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 --2- 172.21.15.64:0/211963891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4102340 0x7fdeb4108bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009ca0 tx=0x7fdeac02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 -- 172.21.15.64:0/211963891 shutdown_connections 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 --2- 172.21.15.64:0/211963891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4102340 0x7fdeb4108bd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 --2- 172.21.15.64:0/211963891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 0x7fdeb4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 --2- 172.21.15.64:0/211963891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdeb4100790 0x7fdeb4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 -- 172.21.15.64:0/211963891 >> 172.21.15.64:0/211963891 conn(0x7fdeb40fbd20 msgr2=0x7fdeb40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 -- 172.21.15.64:0/211963891 shutdown_connections 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.831+0000 7fdebada5640 1 -- 172.21.15.64:0/211963891 wait complete. 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 Processor -- start 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 -- start start 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 0x7fdeb4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 0x7fdeb4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdeb4102340 0x7fdeb419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410c2c0 con 0x7fdeb4100790 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410c140 con 0x7fdeb4102340 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdebada5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410c440 con 0x7fdeb4101980 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 0x7fdeb4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 0x7fdeb4197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37824/0 (socket says 172.21.15.64:37824) 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 -- 172.21.15.64:0/2952234456 learned_addr learned my addr 172.21.15.64:0/2952234456 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeabfff640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 0x7fdeb4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 -- 172.21.15.64:0/2952234456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 msgr2=0x7fdeb4197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 0x7fdeb4197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 -- 172.21.15.64:0/2952234456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdeb4102340 msgr2=0x7fdeb419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdeb4102340 0x7fdeb419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeb8b41640 1 -- 172.21.15.64:0/2952234456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb419f6b0 con 0x7fdeb4100790 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.832+0000 7fdeabfff640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 0x7fdeb4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.833+0000 7fdeb8b41640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 0x7fdeb4197a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c970 tx=0x7fdeb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.833+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0004220 con 0x7fdeb4100790 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.833+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb00043c0 con 0x7fdeb4100790 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.833+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb419f9a0 con 0x7fdeb4100790 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.833+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000f450 con 0x7fdeb4100790 2024-05-09T09:07:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.833+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb419ff40 con 0x7fdeb4100790 2024-05-09T09:07:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.834+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb41a01d0 con 0x7fdeb4100790 2024-05-09T09:07:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.835+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdeb00079b0 con 0x7fdeb4100790 2024-05-09T09:07:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.836+0000 7fdea9ffb640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde9c0a2ca0 0x7fde9c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.836+0000 7fdeabfff640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde9c0a2ca0 0x7fde9c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.836+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdeb00cf1e0 con 0x7fdeb4100790 2024-05-09T09:07:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.836+0000 7fdeabfff640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde9c0a2ca0 0x7fde9c0a5160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4198fa0 tx=0x7fdea4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:10.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:10.840+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdeb0014030 con 0x7fdeb4100790 2024-05-09T09:07:11.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.023+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdeb41a03a0 con 0x7fde9c0a2ca0 2024-05-09T09:07:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.033+0000 7fdea9ffb640 1 -- 172.21.15.64:0/2952234456 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340275 (secure 0 0 0) 0x7fdeb41a03a0 con 0x7fde9c0a2ca0 2024-05-09T09:07:11.037 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:11.039 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde9c0a2ca0 msgr2=0x7fde9c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde9c0a2ca0 0x7fde9c0a5160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4198fa0 tx=0x7fdea4005e50 comp rx=0 tx=0).stop 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 msgr2=0x7fdeb4197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 0x7fdeb4197a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c970 tx=0x7fdeb000ce30 comp rx=0 tx=0).stop 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 shutdown_connections 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde9c0a2ca0 0x7fde9c0a5160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdeb4102340 0x7fdeb419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdeb4101980 0x7fdeb4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 --2- 172.21.15.64:0/2952234456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb4100790 0x7fdeb4197a90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 >> 172.21.15.64:0/2952234456 conn(0x7fdeb40fbd20 msgr2=0x7fdeb40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 shutdown_connections 2024-05-09T09:07:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.037+0000 7fdebada5640 1 -- 172.21.15.64:0/2952234456 wait complete. 2024-05-09T09:07:11.047 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-05-09T09:07:09.101746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597596,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":133,"num_read_kb":115,"num_write":275,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387676,"kb_used_data":13540,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554148,"statfs":{"total":1151940427776,"available":1151543447552,"internally_reserved":0,"allocated":13864960,"data_stored":7554804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4369,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004169"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339280+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:06:46.339280+0000","last_peered":"2024-05-09T09:06:46.339280+0000","last_clean":"2024-05-09T09:06:46.339280+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:06:46.339280+0000","last_undegraded":"2024-05-09T09:06:46.339280+0000","last_fullsized":"2024-05-09T09:06:46.339280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502876+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:06:46.502876+0000","last_peered":"2024-05-09T09:06:46.502876+0000","last_clean":"2024-05-09T09:06:46.502876+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:06:46.502876+0000","last_undegraded":"2024-05-09T09:06:46.502876+0000","last_fullsized":"2024-05-09T09:06:46.502876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502762+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:06:46.502762+0000","last_peered":"2024-05-09T09:06:46.502762+0000","last_clean":"2024-05-09T09:06:46.502762+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:06:46.502762+0000","last_undegraded":"2024-05-09T09:06:46.502762+0000","last_fullsized":"2024-05-09T09:06:46.502762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340076+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:06:46.340076+0000","last_peered":"2024-05-09T09:06:46.340076+0000","last_clean":"2024-05-09T09:06:46.340076+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:06:46.340076+0000","last_undegraded":"2024-05-09T09:06:46.340076+0000","last_fullsized":"2024-05-09T09:06:46.340076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339799+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:06:46.339799+0000","last_peered":"2024-05-09T09:06:46.339799+0000","last_clean":"2024-05-09T09:06:46.339799+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:06:46.339799+0000","last_undegraded":"2024-05-09T09:06:46.339799+0000","last_fullsized":"2024-05-09T09:06:46.339799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339742+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:06:46.339742+0000","last_peered":"2024-05-09T09:06:46.339742+0000","last_clean":"2024-05-09T09:06:46.339742+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:06:46.339742+0000","last_undegraded":"2024-05-09T09:06:46.339742+0000","last_fullsized":"2024-05-09T09:06:46.339742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157288+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:06:46.157288+0000","last_peered":"2024-05-09T09:06:46.157288+0000","last_clean":"2024-05-09T09:06:46.157288+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:06:46.157288+0000","last_undegraded":"2024-05-09T09:06:46.157288+0000","last_fullsized":"2024-05-09T09:06:46.157288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338094+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:06:46.338094+0000","last_peered":"2024-05-09T09:06:46.338094+0000","last_clean":"2024-05-09T09:06:46.338094+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:06:46.338094+0000","last_undegraded":"2024-05-09T09:06:46.338094+0000","last_fullsized":"2024-05-09T09:06:46.338094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339450+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:06:46.339450+0000","last_peered":"2024-05-09T09:06:46.339450+0000","last_clean":"2024-05-09T09:06:46.339450+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:06:46.339450+0000","last_undegraded":"2024-05-09T09:06:46.339450+0000","last_fullsized":"2024-05-09T09:06:46.339450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340239+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:06:46.340239+0000","last_peered":"2024-05-09T09:06:46.340239+0000","last_clean":"2024-05-09T09:06:46.340239+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:06:46.340239+0000","last_undegraded":"2024-05-09T09:06:46.340239+0000","last_fullsized":"2024-05-09T09:06:46.340239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.158034+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:06:46.158034+0000","last_peered":"2024-05-09T09:06:46.158034+0000","last_clean":"2024-05-09T09:06:46.158034+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:06:46.158034+0000","last_undegraded":"2024-05-09T09:06:46.158034+0000","last_fullsized":"2024-05-09T09:06:46.158034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340060+0000","last_change":"2024-05-09T09:06:41.137455+0000","last_active":"2024-05-09T09:06:46.340060+0000","last_peered":"2024-05-09T09:06:46.340060+0000","last_clean":"2024-05-09T09:06:46.340060+0000","last_became_active":"2024-05-09T09:06:41.136139+0000","last_became_peered":"2024-05-09T09:06:41.136139+0000","last_unstale":"2024-05-09T09:06:46.340060+0000","last_undegraded":"2024-05-09T09:06:46.340060+0000","last_fullsized":"2024-05-09T09:06:46.340060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:10.693831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338938+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:06:46.338938+0000","last_peered":"2024-05-09T09:06:46.338938+0000","last_clean":"2024-05-09T09:06:46.338938+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:06:46.338938+0000","last_undegraded":"2024-05-09T09:06:46.338938+0000","last_fullsized":"2024-05-09T09:06:46.338938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502463+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:06:46.502463+0000","last_peered":"2024-05-09T09:06:46.502463+0000","last_clean":"2024-05-09T09:06:46.502463+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:06:46.502463+0000","last_undegraded":"2024-05-09T09:06:46.502463+0000","last_fullsized":"2024-05-09T09:06:46.502463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502512+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:06:46.502512+0000","last_peered":"2024-05-09T09:06:46.502512+0000","last_clean":"2024-05-09T09:06:46.502512+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:06:46.502512+0000","last_undegraded":"2024-05-09T09:06:46.502512+0000","last_fullsized":"2024-05-09T09:06:46.502512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338292+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:06:46.338292+0000","last_peered":"2024-05-09T09:06:46.338292+0000","last_clean":"2024-05-09T09:06:46.338292+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:06:46.338292+0000","last_undegraded":"2024-05-09T09:06:46.338292+0000","last_fullsized":"2024-05-09T09:06:46.338292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339235+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:06:46.339235+0000","last_peered":"2024-05-09T09:06:46.339235+0000","last_clean":"2024-05-09T09:06:46.339235+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:06:46.339235+0000","last_undegraded":"2024-05-09T09:06:46.339235+0000","last_fullsized":"2024-05-09T09:06:46.339235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157064+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:06:46.157064+0000","last_peered":"2024-05-09T09:06:46.157064+0000","last_clean":"2024-05-09T09:06:46.157064+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:06:46.157064+0000","last_undegraded":"2024-05-09T09:06:46.157064+0000","last_fullsized":"2024-05-09T09:06:46.157064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339066+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:06:46.339066+0000","last_peered":"2024-05-09T09:06:46.339066+0000","last_clean":"2024-05-09T09:06:46.339066+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:06:46.339066+0000","last_undegraded":"2024-05-09T09:06:46.339066+0000","last_fullsized":"2024-05-09T09:06:46.339066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339206+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:06:46.339206+0000","last_peered":"2024-05-09T09:06:46.339206+0000","last_clean":"2024-05-09T09:06:46.339206+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:06:46.339206+0000","last_undegraded":"2024-05-09T09:06:46.339206+0000","last_fullsized":"2024-05-09T09:06:46.339206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157297+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:06:46.157297+0000","last_peered":"2024-05-09T09:06:46.157297+0000","last_clean":"2024-05-09T09:06:46.157297+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:06:46.157297+0000","last_undegraded":"2024-05-09T09:06:46.157297+0000","last_fullsized":"2024-05-09T09:06:46.157297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338179+0000","last_change":"2024-05-09T09:06:41.137679+0000","last_active":"2024-05-09T09:06:46.338179+0000","last_peered":"2024-05-09T09:06:46.338179+0000","last_clean":"2024-05-09T09:06:46.338179+0000","last_became_active":"2024-05-09T09:06:41.137353+0000","last_became_peered":"2024-05-09T09:06:41.137353+0000","last_unstale":"2024-05-09T09:06:46.338179+0000","last_undegraded":"2024-05-09T09:06:46.338179+0000","last_fullsized":"2024-05-09T09:06:46.338179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:36.080021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338646+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:06:46.338646+0000","last_peered":"2024-05-09T09:06:46.338646+0000","last_clean":"2024-05-09T09:06:46.338646+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:06:46.338646+0000","last_undegraded":"2024-05-09T09:06:46.338646+0000","last_fullsized":"2024-05-09T09:06:46.338646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338583+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:06:46.338583+0000","last_peered":"2024-05-09T09:06:46.338583+0000","last_clean":"2024-05-09T09:06:46.338583+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:06:46.338583+0000","last_undegraded":"2024-05-09T09:06:46.338583+0000","last_fullsized":"2024-05-09T09:06:46.338583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502915+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:06:46.502915+0000","last_peered":"2024-05-09T09:06:46.502915+0000","last_clean":"2024-05-09T09:06:46.502915+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:06:46.502915+0000","last_undegraded":"2024-05-09T09:06:46.502915+0000","last_fullsized":"2024-05-09T09:06:46.502915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339294+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:06:46.339294+0000","last_peered":"2024-05-09T09:06:46.339294+0000","last_clean":"2024-05-09T09:06:46.339294+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:06:46.339294+0000","last_undegraded":"2024-05-09T09:06:46.339294+0000","last_fullsized":"2024-05-09T09:06:46.339294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339354+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:06:46.339354+0000","last_peered":"2024-05-09T09:06:46.339354+0000","last_clean":"2024-05-09T09:06:46.339354+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:06:46.339354+0000","last_undegraded":"2024-05-09T09:06:46.339354+0000","last_fullsized":"2024-05-09T09:06:46.339354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338732+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:06:46.338732+0000","last_peered":"2024-05-09T09:06:46.338732+0000","last_clean":"2024-05-09T09:06:46.338732+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:06:46.338732+0000","last_undegraded":"2024-05-09T09:06:46.338732+0000","last_fullsized":"2024-05-09T09:06:46.338732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339401+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:06:46.339401+0000","last_peered":"2024-05-09T09:06:46.339401+0000","last_clean":"2024-05-09T09:06:46.339401+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:06:46.339401+0000","last_undegraded":"2024-05-09T09:06:46.339401+0000","last_fullsized":"2024-05-09T09:06:46.339401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339157+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:06:46.339157+0000","last_peered":"2024-05-09T09:06:46.339157+0000","last_clean":"2024-05-09T09:06:46.339157+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:06:46.339157+0000","last_undegraded":"2024-05-09T09:06:46.339157+0000","last_fullsized":"2024-05-09T09:06:46.339157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160309+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:06:46.160309+0000","last_peered":"2024-05-09T09:06:46.160309+0000","last_clean":"2024-05-09T09:06:46.160309+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:06:46.160309+0000","last_undegraded":"2024-05-09T09:06:46.160309+0000","last_fullsized":"2024-05-09T09:06:46.160309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338983+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:06:46.338983+0000","last_peered":"2024-05-09T09:06:46.338983+0000","last_clean":"2024-05-09T09:06:46.338983+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:06:46.338983+0000","last_undegraded":"2024-05-09T09:06:46.338983+0000","last_fullsized":"2024-05-09T09:06:46.338983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.226235+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:07:06.226235+0000","last_peered":"2024-05-09T09:07:06.226235+0000","last_clean":"2024-05-09T09:07:06.226235+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:07:06.226235+0000","last_undegraded":"2024-05-09T09:07:06.226235+0000","last_fullsized":"2024-05-09T09:07:06.226235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339646+0000","last_change":"2024-05-09T09:06:41.131157+0000","last_active":"2024-05-09T09:06:46.339646+0000","last_peered":"2024-05-09T09:06:46.339646+0000","last_clean":"2024-05-09T09:06:46.339646+0000","last_became_active":"2024-05-09T09:06:41.130708+0000","last_became_peered":"2024-05-09T09:06:41.130708+0000","last_unstale":"2024-05-09T09:06:46.339646+0000","last_undegraded":"2024-05-09T09:06:46.339646+0000","last_fullsized":"2024-05-09T09:06:46.339646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:09.418840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339865+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:06:46.339865+0000","last_peered":"2024-05-09T09:06:46.339865+0000","last_clean":"2024-05-09T09:06:46.339865+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:06:46.339865+0000","last_undegraded":"2024-05-09T09:06:46.339865+0000","last_fullsized":"2024-05-09T09:06:46.339865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.235658+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:07:07.235658+0000","last_peered":"2024-05-09T09:07:07.235658+0000","last_clean":"2024-05-09T09:07:07.235658+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:07:07.235658+0000","last_undegraded":"2024-05-09T09:07:07.235658+0000","last_fullsized":"2024-05-09T09:07:07.235658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160249+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:06:46.160249+0000","last_peered":"2024-05-09T09:06:46.160249+0000","last_clean":"2024-05-09T09:06:46.160249+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:06:46.160249+0000","last_undegraded":"2024-05-09T09:06:46.160249+0000","last_fullsized":"2024-05-09T09:06:46.160249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339218+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:06:46.339218+0000","last_peered":"2024-05-09T09:06:46.339218+0000","last_clean":"2024-05-09T09:06:46.339218+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:06:46.339218+0000","last_undegraded":"2024-05-09T09:06:46.339218+0000","last_fullsized":"2024-05-09T09:06:46.339218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338052+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:06:46.338052+0000","last_peered":"2024-05-09T09:06:46.338052+0000","last_clean":"2024-05-09T09:06:46.338052+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:06:46.338052+0000","last_undegraded":"2024-05-09T09:06:46.338052+0000","last_fullsized":"2024-05-09T09:06:46.338052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338385+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:06:46.338385+0000","last_peered":"2024-05-09T09:06:46.338385+0000","last_clean":"2024-05-09T09:06:46.338385+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:06:46.338385+0000","last_undegraded":"2024-05-09T09:06:46.338385+0000","last_fullsized":"2024-05-09T09:06:46.338385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502898+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:06:46.502898+0000","last_peered":"2024-05-09T09:06:46.502898+0000","last_clean":"2024-05-09T09:06:46.502898+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:06:46.502898+0000","last_undegraded":"2024-05-09T09:06:46.502898+0000","last_fullsized":"2024-05-09T09:06:46.502898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159833+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:06:46.159833+0000","last_peered":"2024-05-09T09:06:46.159833+0000","last_clean":"2024-05-09T09:06:46.159833+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:06:46.159833+0000","last_undegraded":"2024-05-09T09:06:46.159833+0000","last_fullsized":"2024-05-09T09:06:46.159833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156493+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:06:46.156493+0000","last_peered":"2024-05-09T09:06:46.156493+0000","last_clean":"2024-05-09T09:06:46.156493+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:06:46.156493+0000","last_undegraded":"2024-05-09T09:06:46.156493+0000","last_fullsized":"2024-05-09T09:06:46.156493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339370+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:06:46.339370+0000","last_peered":"2024-05-09T09:06:46.339370+0000","last_clean":"2024-05-09T09:06:46.339370+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:06:46.339370+0000","last_undegraded":"2024-05-09T09:06:46.339370+0000","last_fullsized":"2024-05-09T09:06:46.339370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.226013+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:07:06.226013+0000","last_peered":"2024-05-09T09:07:06.226013+0000","last_clean":"2024-05-09T09:07:06.226013+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:07:06.226013+0000","last_undegraded":"2024-05-09T09:07:06.226013+0000","last_fullsized":"2024-05-09T09:07:06.226013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339115+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:06:46.339115+0000","last_peered":"2024-05-09T09:06:46.339115+0000","last_clean":"2024-05-09T09:06:46.339115+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:06:46.339115+0000","last_undegraded":"2024-05-09T09:06:46.339115+0000","last_fullsized":"2024-05-09T09:06:46.339115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156591+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:06:46.156591+0000","last_peered":"2024-05-09T09:06:46.156591+0000","last_clean":"2024-05-09T09:06:46.156591+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:06:46.156591+0000","last_undegraded":"2024-05-09T09:06:46.156591+0000","last_fullsized":"2024-05-09T09:06:46.156591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225020+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:07:06.225020+0000","last_peered":"2024-05-09T09:07:06.225020+0000","last_clean":"2024-05-09T09:07:06.225020+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:07:06.225020+0000","last_undegraded":"2024-05-09T09:07:06.225020+0000","last_fullsized":"2024-05-09T09:07:06.225020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502875+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:06:46.502875+0000","last_peered":"2024-05-09T09:06:46.502875+0000","last_clean":"2024-05-09T09:06:46.502875+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:06:46.502875+0000","last_undegraded":"2024-05-09T09:06:46.502875+0000","last_fullsized":"2024-05-09T09:06:46.502875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339641+0000","last_change":"2024-05-09T09:06:45.186842+0000","last_active":"2024-05-09T09:06:46.339641+0000","last_peered":"2024-05-09T09:06:46.339641+0000","last_clean":"2024-05-09T09:06:46.339641+0000","last_became_active":"2024-05-09T09:06:45.184745+0000","last_became_peered":"2024-05-09T09:06:45.184745+0000","last_unstale":"2024-05-09T09:06:46.339641+0000","last_undegraded":"2024-05-09T09:06:46.339641+0000","last_fullsized":"2024-05-09T09:06:46.339641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:56.933012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160294+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:06:46.160294+0000","last_peered":"2024-05-09T09:06:46.160294+0000","last_clean":"2024-05-09T09:06:46.160294+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:06:46.160294+0000","last_undegraded":"2024-05-09T09:06:46.160294+0000","last_fullsized":"2024-05-09T09:06:46.160294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338841+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:06:46.338841+0000","last_peered":"2024-05-09T09:06:46.338841+0000","last_clean":"2024-05-09T09:06:46.338841+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:06:46.338841+0000","last_undegraded":"2024-05-09T09:06:46.338841+0000","last_fullsized":"2024-05-09T09:06:46.338841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156854+0000","last_change":"2024-05-09T09:06:45.182604+0000","last_active":"2024-05-09T09:06:46.156854+0000","last_peered":"2024-05-09T09:06:46.156854+0000","last_clean":"2024-05-09T09:06:46.156854+0000","last_became_active":"2024-05-09T09:06:45.181320+0000","last_became_peered":"2024-05-09T09:06:45.181320+0000","last_unstale":"2024-05-09T09:06:46.156854+0000","last_undegraded":"2024-05-09T09:06:46.156854+0000","last_fullsized":"2024-05-09T09:06:46.156854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:48.099742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225582+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:07:06.225582+0000","last_peered":"2024-05-09T09:07:06.225582+0000","last_clean":"2024-05-09T09:07:06.225582+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:07:06.225582+0000","last_undegraded":"2024-05-09T09:07:06.225582+0000","last_fullsized":"2024-05-09T09:07:06.225582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339080+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:06:46.339080+0000","last_peered":"2024-05-09T09:06:46.339080+0000","last_clean":"2024-05-09T09:06:46.339080+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:06:46.339080+0000","last_undegraded":"2024-05-09T09:06:46.339080+0000","last_fullsized":"2024-05-09T09:06:46.339080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160636+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:06:46.160636+0000","last_peered":"2024-05-09T09:06:46.160636+0000","last_clean":"2024-05-09T09:06:46.160636+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:06:46.160636+0000","last_undegraded":"2024-05-09T09:06:46.160636+0000","last_fullsized":"2024-05-09T09:06:46.160636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.236122+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:07:07.236122+0000","last_peered":"2024-05-09T09:07:07.236122+0000","last_clean":"2024-05-09T09:07:07.236122+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:07:07.236122+0000","last_undegraded":"2024-05-09T09:07:07.236122+0000","last_fullsized":"2024-05-09T09:07:07.236122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338800+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:06:46.338800+0000","last_peered":"2024-05-09T09:06:46.338800+0000","last_clean":"2024-05-09T09:06:46.338800+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:06:46.338800+0000","last_undegraded":"2024-05-09T09:06:46.338800+0000","last_fullsized":"2024-05-09T09:06:46.338800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338731+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:06:46.338731+0000","last_peered":"2024-05-09T09:06:46.338731+0000","last_clean":"2024-05-09T09:06:46.338731+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:06:46.338731+0000","last_undegraded":"2024-05-09T09:06:46.338731+0000","last_fullsized":"2024-05-09T09:06:46.338731+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225721+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:07:06.225721+0000","last_peered":"2024-05-09T09:07:06.225721+0000","last_clean":"2024-05-09T09:07:06.225721+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:07:06.225721+0000","last_undegraded":"2024-05-09T09:07:06.225721+0000","last_fullsized":"2024-05-09T09:07:06.225721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157740+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:06:46.157740+0000","last_peered":"2024-05-09T09:06:46.157740+0000","last_clean":"2024-05-09T09:06:46.157740+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:06:46.157740+0000","last_undegraded":"2024-05-09T09:06:46.157740+0000","last_fullsized":"2024-05-09T09:06:46.157740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157141+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:06:46.157141+0000","last_peered":"2024-05-09T09:06:46.157141+0000","last_clean":"2024-05-09T09:06:46.157141+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:06:46.157141+0000","last_undegraded":"2024-05-09T09:06:46.157141+0000","last_fullsized":"2024-05-09T09:06:46.157141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:08.238372+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:07:08.238372+0000","last_peered":"2024-05-09T09:07:08.238372+0000","last_clean":"2024-05-09T09:07:08.238372+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:07:08.238372+0000","last_undegraded":"2024-05-09T09:07:08.238372+0000","last_fullsized":"2024-05-09T09:07:08.238372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157372+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:06:46.157372+0000","last_peered":"2024-05-09T09:06:46.157372+0000","last_clean":"2024-05-09T09:06:46.157372+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:06:46.157372+0000","last_undegraded":"2024-05-09T09:06:46.157372+0000","last_fullsized":"2024-05-09T09:06:46.157372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339266+0000","last_change":"2024-05-09T09:06:45.184784+0000","last_active":"2024-05-09T09:06:46.339266+0000","last_peered":"2024-05-09T09:06:46.339266+0000","last_clean":"2024-05-09T09:06:46.339266+0000","last_became_active":"2024-05-09T09:06:45.184394+0000","last_became_peered":"2024-05-09T09:06:45.184394+0000","last_unstale":"2024-05-09T09:06:46.339266+0000","last_undegraded":"2024-05-09T09:06:46.339266+0000","last_fullsized":"2024-05-09T09:06:46.339266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:02.111601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339484+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:06:46.339484+0000","last_peered":"2024-05-09T09:06:46.339484+0000","last_clean":"2024-05-09T09:06:46.339484+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:06:46.339484+0000","last_undegraded":"2024-05-09T09:06:46.339484+0000","last_fullsized":"2024-05-09T09:06:46.339484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159737+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:06:46.159737+0000","last_peered":"2024-05-09T09:06:46.159737+0000","last_clean":"2024-05-09T09:06:46.159737+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:06:46.159737+0000","last_undegraded":"2024-05-09T09:06:46.159737+0000","last_fullsized":"2024-05-09T09:06:46.159737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159899+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:06:46.159899+0000","last_peered":"2024-05-09T09:06:46.159899+0000","last_clean":"2024-05-09T09:06:46.159899+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:06:46.159899+0000","last_undegraded":"2024-05-09T09:06:46.159899+0000","last_fullsized":"2024-05-09T09:06:46.159899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340041+0000","last_change":"2024-05-09T09:06:40.113621+0000","last_active":"2024-05-09T09:06:46.340041+0000","last_peered":"2024-05-09T09:06:46.340041+0000","last_clean":"2024-05-09T09:06:46.340041+0000","last_became_active":"2024-05-09T09:06:40.113283+0000","last_became_peered":"2024-05-09T09:06:40.113283+0000","last_unstale":"2024-05-09T09:06:46.340041+0000","last_undegraded":"2024-05-09T09:06:46.340041+0000","last_fullsized":"2024-05-09T09:06:46.340041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:27.393494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339085+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:06:46.339085+0000","last_peered":"2024-05-09T09:06:46.339085+0000","last_clean":"2024-05-09T09:06:46.339085+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:06:46.339085+0000","last_undegraded":"2024-05-09T09:06:46.339085+0000","last_fullsized":"2024-05-09T09:06:46.339085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339682+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:06:46.339682+0000","last_peered":"2024-05-09T09:06:46.339682+0000","last_clean":"2024-05-09T09:06:46.339682+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:06:46.339682+0000","last_undegraded":"2024-05-09T09:06:46.339682+0000","last_fullsized":"2024-05-09T09:06:46.339682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156882+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:06:46.156882+0000","last_peered":"2024-05-09T09:06:46.156882+0000","last_clean":"2024-05-09T09:06:46.156882+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:06:46.156882+0000","last_undegraded":"2024-05-09T09:06:46.156882+0000","last_fullsized":"2024-05-09T09:06:46.156882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339315+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:06:46.339315+0000","last_peered":"2024-05-09T09:06:46.339315+0000","last_clean":"2024-05-09T09:06:46.339315+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:06:46.339315+0000","last_undegraded":"2024-05-09T09:06:46.339315+0000","last_fullsized":"2024-05-09T09:06:46.339315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338612+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:06:46.338612+0000","last_peered":"2024-05-09T09:06:46.338612+0000","last_clean":"2024-05-09T09:06:46.338612+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:06:46.338612+0000","last_undegraded":"2024-05-09T09:06:46.338612+0000","last_fullsized":"2024-05-09T09:06:46.338612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157473+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:06:46.157473+0000","last_peered":"2024-05-09T09:06:46.157473+0000","last_clean":"2024-05-09T09:06:46.157473+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:06:46.157473+0000","last_undegraded":"2024-05-09T09:06:46.157473+0000","last_fullsized":"2024-05-09T09:06:46.157473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339975+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:06:46.339975+0000","last_peered":"2024-05-09T09:06:46.339975+0000","last_clean":"2024-05-09T09:06:46.339975+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:06:46.339975+0000","last_undegraded":"2024-05-09T09:06:46.339975+0000","last_fullsized":"2024-05-09T09:06:46.339975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339181+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:06:46.339181+0000","last_peered":"2024-05-09T09:06:46.339181+0000","last_clean":"2024-05-09T09:06:46.339181+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:06:46.339181+0000","last_undegraded":"2024-05-09T09:06:46.339181+0000","last_fullsized":"2024-05-09T09:06:46.339181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159895+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:06:46.159895+0000","last_peered":"2024-05-09T09:06:46.159895+0000","last_clean":"2024-05-09T09:06:46.159895+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:06:46.159895+0000","last_undegraded":"2024-05-09T09:06:46.159895+0000","last_fullsized":"2024-05-09T09:06:46.159895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.337975+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:06:46.337975+0000","last_peered":"2024-05-09T09:06:46.337975+0000","last_clean":"2024-05-09T09:06:46.337975+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:06:46.337975+0000","last_undegraded":"2024-05-09T09:06:46.337975+0000","last_fullsized":"2024-05-09T09:06:46.337975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157948+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:06:46.157948+0000","last_peered":"2024-05-09T09:06:46.157948+0000","last_clean":"2024-05-09T09:06:46.157948+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:06:46.157948+0000","last_undegraded":"2024-05-09T09:06:46.157948+0000","last_fullsized":"2024-05-09T09:06:46.157948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338184+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:06:46.338184+0000","last_peered":"2024-05-09T09:06:46.338184+0000","last_clean":"2024-05-09T09:06:46.338184+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:06:46.338184+0000","last_undegraded":"2024-05-09T09:06:46.338184+0000","last_fullsized":"2024-05-09T09:06:46.338184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:08.235954+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:07:08.235954+0000","last_peered":"2024-05-09T09:07:08.235954+0000","last_clean":"2024-05-09T09:07:08.235954+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:07:08.235954+0000","last_undegraded":"2024-05-09T09:07:08.235954+0000","last_fullsized":"2024-05-09T09:07:08.235954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":346,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157185+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:06:46.157185+0000","last_peered":"2024-05-09T09:06:46.157185+0000","last_clean":"2024-05-09T09:06:46.157185+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:06:46.157185+0000","last_undegraded":"2024-05-09T09:06:46.157185+0000","last_fullsized":"2024-05-09T09:06:46.157185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160485+0000","last_change":"2024-05-09T09:06:45.180565+0000","last_active":"2024-05-09T09:06:46.160485+0000","last_peered":"2024-05-09T09:06:46.160485+0000","last_clean":"2024-05-09T09:06:46.160485+0000","last_became_active":"2024-05-09T09:06:45.180141+0000","last_became_peered":"2024-05-09T09:06:45.180141+0000","last_unstale":"2024-05-09T09:06:46.160485+0000","last_undegraded":"2024-05-09T09:06:46.160485+0000","last_fullsized":"2024-05-09T09:06:46.160485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:58.412376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.337905+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:06:46.337905+0000","last_peered":"2024-05-09T09:06:46.337905+0000","last_clean":"2024-05-09T09:06:46.337905+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:06:46.337905+0000","last_undegraded":"2024-05-09T09:06:46.337905+0000","last_fullsized":"2024-05-09T09:06:46.337905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.235949+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:07:07.235949+0000","last_peered":"2024-05-09T09:07:07.235949+0000","last_clean":"2024-05-09T09:07:07.235949+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:07:07.235949+0000","last_undegraded":"2024-05-09T09:07:07.235949+0000","last_fullsized":"2024-05-09T09:07:07.235949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339071+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:06:46.339071+0000","last_peered":"2024-05-09T09:06:46.339071+0000","last_clean":"2024-05-09T09:06:46.339071+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:06:46.339071+0000","last_undegraded":"2024-05-09T09:06:46.339071+0000","last_fullsized":"2024-05-09T09:06:46.339071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339182+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:06:46.339182+0000","last_peered":"2024-05-09T09:06:46.339182+0000","last_clean":"2024-05-09T09:06:46.339182+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:06:46.339182+0000","last_undegraded":"2024-05-09T09:06:46.339182+0000","last_fullsized":"2024-05-09T09:06:46.339182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338716+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:06:46.338716+0000","last_peered":"2024-05-09T09:06:46.338716+0000","last_clean":"2024-05-09T09:06:46.338716+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:06:46.338716+0000","last_undegraded":"2024-05-09T09:06:46.338716+0000","last_fullsized":"2024-05-09T09:06:46.338716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338326+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:06:46.338326+0000","last_peered":"2024-05-09T09:06:46.338326+0000","last_clean":"2024-05-09T09:06:46.338326+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:06:46.338326+0000","last_undegraded":"2024-05-09T09:06:46.338326+0000","last_fullsized":"2024-05-09T09:06:46.338326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157049+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:06:46.157049+0000","last_peered":"2024-05-09T09:06:46.157049+0000","last_clean":"2024-05-09T09:06:46.157049+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:06:46.157049+0000","last_undegraded":"2024-05-09T09:06:46.157049+0000","last_fullsized":"2024-05-09T09:06:46.157049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338391+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:06:46.338391+0000","last_peered":"2024-05-09T09:06:46.338391+0000","last_clean":"2024-05-09T09:06:46.338391+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:06:46.338391+0000","last_undegraded":"2024-05-09T09:06:46.338391+0000","last_fullsized":"2024-05-09T09:06:46.338391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340207+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:06:46.340207+0000","last_peered":"2024-05-09T09:06:46.340207+0000","last_clean":"2024-05-09T09:06:46.340207+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:06:46.340207+0000","last_undegraded":"2024-05-09T09:06:46.340207+0000","last_fullsized":"2024-05-09T09:06:46.340207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.226643+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:07:06.226643+0000","last_peered":"2024-05-09T09:07:06.226643+0000","last_clean":"2024-05-09T09:07:06.226643+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:07:06.226643+0000","last_undegraded":"2024-05-09T09:07:06.226643+0000","last_fullsized":"2024-05-09T09:07:06.226643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502980+0000","last_change":"2024-05-09T09:06:41.134756+0000","last_active":"2024-05-09T09:06:46.502980+0000","last_peered":"2024-05-09T09:06:46.502980+0000","last_clean":"2024-05-09T09:06:46.502980+0000","last_became_active":"2024-05-09T09:06:41.134420+0000","last_became_peered":"2024-05-09T09:06:41.134420+0000","last_unstale":"2024-05-09T09:06:46.502980+0000","last_undegraded":"2024-05-09T09:06:46.502980+0000","last_fullsized":"2024-05-09T09:06:46.502980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:29:15.995691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339333+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:06:46.339333+0000","last_peered":"2024-05-09T09:06:46.339333+0000","last_clean":"2024-05-09T09:06:46.339333+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:06:46.339333+0000","last_undegraded":"2024-05-09T09:06:46.339333+0000","last_fullsized":"2024-05-09T09:06:46.339333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338351+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:06:46.338351+0000","last_peered":"2024-05-09T09:06:46.338351+0000","last_clean":"2024-05-09T09:06:46.338351+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:06:46.338351+0000","last_undegraded":"2024-05-09T09:06:46.338351+0000","last_fullsized":"2024-05-09T09:06:46.338351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338594+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:06:46.338594+0000","last_peered":"2024-05-09T09:06:46.338594+0000","last_clean":"2024-05-09T09:06:46.338594+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:06:46.338594+0000","last_undegraded":"2024-05-09T09:06:46.338594+0000","last_fullsized":"2024-05-09T09:06:46.338594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157353+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:06:46.157353+0000","last_peered":"2024-05-09T09:06:46.157353+0000","last_clean":"2024-05-09T09:06:46.157353+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:06:46.157353+0000","last_undegraded":"2024-05-09T09:06:46.157353+0000","last_fullsized":"2024-05-09T09:06:46.157353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.227063+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:07:06.227063+0000","last_peered":"2024-05-09T09:07:06.227063+0000","last_clean":"2024-05-09T09:07:06.227063+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:07:06.227063+0000","last_undegraded":"2024-05-09T09:07:06.227063+0000","last_fullsized":"2024-05-09T09:07:06.227063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338431+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:06:46.338431+0000","last_peered":"2024-05-09T09:06:46.338431+0000","last_clean":"2024-05-09T09:06:46.338431+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:06:46.338431+0000","last_undegraded":"2024-05-09T09:06:46.338431+0000","last_fullsized":"2024-05-09T09:06:46.338431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339996+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:06:46.339996+0000","last_peered":"2024-05-09T09:06:46.339996+0000","last_clean":"2024-05-09T09:06:46.339996+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:06:46.339996+0000","last_undegraded":"2024-05-09T09:06:46.339996+0000","last_fullsized":"2024-05-09T09:06:46.339996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338769+0000","last_change":"2024-05-09T09:06:40.113416+0000","last_active":"2024-05-09T09:06:46.338769+0000","last_peered":"2024-05-09T09:06:46.338769+0000","last_clean":"2024-05-09T09:06:46.338769+0000","last_became_active":"2024-05-09T09:06:40.112949+0000","last_became_peered":"2024-05-09T09:06:40.112949+0000","last_unstale":"2024-05-09T09:06:46.338769+0000","last_undegraded":"2024-05-09T09:06:46.338769+0000","last_fullsized":"2024-05-09T09:06:46.338769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:47.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338945+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:06:46.338945+0000","last_peered":"2024-05-09T09:06:46.338945+0000","last_clean":"2024-05-09T09:06:46.338945+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:06:46.338945+0000","last_undegraded":"2024-05-09T09:06:46.338945+0000","last_fullsized":"2024-05-09T09:06:46.338945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339304+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:06:46.339304+0000","last_peered":"2024-05-09T09:06:46.339304+0000","last_clean":"2024-05-09T09:06:46.339304+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:06:46.339304+0000","last_undegraded":"2024-05-09T09:06:46.339304+0000","last_fullsized":"2024-05-09T09:06:46.339304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340180+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:06:46.340180+0000","last_peered":"2024-05-09T09:06:46.340180+0000","last_clean":"2024-05-09T09:06:46.340180+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:06:46.340180+0000","last_undegraded":"2024-05-09T09:06:46.340180+0000","last_fullsized":"2024-05-09T09:06:46.340180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157481+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:06:46.157481+0000","last_peered":"2024-05-09T09:06:46.157481+0000","last_clean":"2024-05-09T09:06:46.157481+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:06:46.157481+0000","last_undegraded":"2024-05-09T09:06:46.157481+0000","last_fullsized":"2024-05-09T09:06:46.157481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338346+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:06:46.338346+0000","last_peered":"2024-05-09T09:06:46.338346+0000","last_clean":"2024-05-09T09:06:46.338346+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:06:46.338346+0000","last_undegraded":"2024-05-09T09:06:46.338346+0000","last_fullsized":"2024-05-09T09:06:46.338346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.236682+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:07:07.236682+0000","last_peered":"2024-05-09T09:07:07.236682+0000","last_clean":"2024-05-09T09:07:07.236682+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:07:07.236682+0000","last_undegraded":"2024-05-09T09:07:07.236682+0000","last_fullsized":"2024-05-09T09:07:07.236682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.158108+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:06:46.158108+0000","last_peered":"2024-05-09T09:06:46.158108+0000","last_clean":"2024-05-09T09:06:46.158108+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:06:46.158108+0000","last_undegraded":"2024-05-09T09:06:46.158108+0000","last_fullsized":"2024-05-09T09:06:46.158108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502560+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:06:46.502560+0000","last_peered":"2024-05-09T09:06:46.502560+0000","last_clean":"2024-05-09T09:06:46.502560+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:06:46.502560+0000","last_undegraded":"2024-05-09T09:06:46.502560+0000","last_fullsized":"2024-05-09T09:06:46.502560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160373+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:06:46.160373+0000","last_peered":"2024-05-09T09:06:46.160373+0000","last_clean":"2024-05-09T09:06:46.160373+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:06:46.160373+0000","last_undegraded":"2024-05-09T09:06:46.160373+0000","last_fullsized":"2024-05-09T09:06:46.160373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157715+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:06:46.157715+0000","last_peered":"2024-05-09T09:06:46.157715+0000","last_clean":"2024-05-09T09:06:46.157715+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:06:46.157715+0000","last_undegraded":"2024-05-09T09:06:46.157715+0000","last_fullsized":"2024-05-09T09:06:46.157715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159687+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:06:46.159687+0000","last_peered":"2024-05-09T09:06:46.159687+0000","last_clean":"2024-05-09T09:06:46.159687+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:06:46.159687+0000","last_undegraded":"2024-05-09T09:06:46.159687+0000","last_fullsized":"2024-05-09T09:06:46.159687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339113+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:06:46.339113+0000","last_peered":"2024-05-09T09:06:46.339113+0000","last_clean":"2024-05-09T09:06:46.339113+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:06:46.339113+0000","last_undegraded":"2024-05-09T09:06:46.339113+0000","last_fullsized":"2024-05-09T09:06:46.339113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.158050+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:06:46.158050+0000","last_peered":"2024-05-09T09:06:46.158050+0000","last_clean":"2024-05-09T09:06:46.158050+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:06:46.158050+0000","last_undegraded":"2024-05-09T09:06:46.158050+0000","last_fullsized":"2024-05-09T09:06:46.158050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338720+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:06:46.338720+0000","last_peered":"2024-05-09T09:06:46.338720+0000","last_clean":"2024-05-09T09:06:46.338720+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:06:46.338720+0000","last_undegraded":"2024-05-09T09:06:46.338720+0000","last_fullsized":"2024-05-09T09:06:46.338720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339413+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:06:46.339413+0000","last_peered":"2024-05-09T09:06:46.339413+0000","last_clean":"2024-05-09T09:06:46.339413+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:06:46.339413+0000","last_undegraded":"2024-05-09T09:06:46.339413+0000","last_fullsized":"2024-05-09T09:06:46.339413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338642+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:06:46.338642+0000","last_peered":"2024-05-09T09:06:46.338642+0000","last_clean":"2024-05-09T09:06:46.338642+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:06:46.338642+0000","last_undegraded":"2024-05-09T09:06:46.338642+0000","last_fullsized":"2024-05-09T09:06:46.338642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502628+0000","last_change":"2024-05-09T09:06:41.124927+0000","last_active":"2024-05-09T09:06:46.502628+0000","last_peered":"2024-05-09T09:06:46.502628+0000","last_clean":"2024-05-09T09:06:46.502628+0000","last_became_active":"2024-05-09T09:06:41.124566+0000","last_became_peered":"2024-05-09T09:06:41.124566+0000","last_unstale":"2024-05-09T09:06:46.502628+0000","last_undegraded":"2024-05-09T09:06:46.502628+0000","last_fullsized":"2024-05-09T09:06:46.502628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.792308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338816+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:06:46.338816+0000","last_peered":"2024-05-09T09:06:46.338816+0000","last_clean":"2024-05-09T09:06:46.338816+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:06:46.338816+0000","last_undegraded":"2024-05-09T09:06:46.338816+0000","last_fullsized":"2024-05-09T09:06:46.338816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339196+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:06:46.339196+0000","last_peered":"2024-05-09T09:06:46.339196+0000","last_clean":"2024-05-09T09:06:46.339196+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:06:46.339196+0000","last_undegraded":"2024-05-09T09:06:46.339196+0000","last_fullsized":"2024-05-09T09:06:46.339196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339938+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:06:46.339938+0000","last_peered":"2024-05-09T09:06:46.339938+0000","last_clean":"2024-05-09T09:06:46.339938+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:06:46.339938+0000","last_undegraded":"2024-05-09T09:06:46.339938+0000","last_fullsized":"2024-05-09T09:06:46.339938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338579+0000","last_change":"2024-05-09T09:06:41.131374+0000","last_active":"2024-05-09T09:06:46.338579+0000","last_peered":"2024-05-09T09:06:46.338579+0000","last_clean":"2024-05-09T09:06:46.338579+0000","last_became_active":"2024-05-09T09:06:41.131052+0000","last_became_peered":"2024-05-09T09:06:41.131052+0000","last_unstale":"2024-05-09T09:06:46.338579+0000","last_undegraded":"2024-05-09T09:06:46.338579+0000","last_fullsized":"2024-05-09T09:06:46.338579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:22.200079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159798+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:06:46.159798+0000","last_peered":"2024-05-09T09:06:46.159798+0000","last_clean":"2024-05-09T09:06:46.159798+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:06:46.159798+0000","last_undegraded":"2024-05-09T09:06:46.159798+0000","last_fullsized":"2024-05-09T09:06:46.159798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339422+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:06:46.339422+0000","last_peered":"2024-05-09T09:06:46.339422+0000","last_clean":"2024-05-09T09:06:46.339422+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:06:46.339422+0000","last_undegraded":"2024-05-09T09:06:46.339422+0000","last_fullsized":"2024-05-09T09:06:46.339422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338647+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:06:46.338647+0000","last_peered":"2024-05-09T09:06:46.338647+0000","last_clean":"2024-05-09T09:06:46.338647+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:06:46.338647+0000","last_undegraded":"2024-05-09T09:06:46.338647+0000","last_fullsized":"2024-05-09T09:06:46.338647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338823+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:06:46.338823+0000","last_peered":"2024-05-09T09:06:46.338823+0000","last_clean":"2024-05-09T09:06:46.338823+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:06:46.338823+0000","last_undegraded":"2024-05-09T09:06:46.338823+0000","last_fullsized":"2024-05-09T09:06:46.338823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338796+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:06:46.338796+0000","last_peered":"2024-05-09T09:06:46.338796+0000","last_clean":"2024-05-09T09:06:46.338796+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:06:46.338796+0000","last_undegraded":"2024-05-09T09:06:46.338796+0000","last_fullsized":"2024-05-09T09:06:46.338796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7228,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":94,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":20508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1076054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":999424,"data_stored":480870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1072426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":484108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":485458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":480550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":480910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1613824,"data_stored":1072110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:07:11.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 -- 172.21.15.64:0/3119569518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198104c40 msgr2=0x7ff198107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3119569518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198104c40 0x7ff198107030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff18802a410 comp rx=0 tx=0).stop 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 -- 172.21.15.64:0/3119569518 shutdown_connections 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3119569518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff198107570 0x7ff1981099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3119569518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198104c40 0x7ff198107030 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3119569518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff198102310 0x7ff198104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 -- 172.21.15.64:0/3119569518 >> 172.21.15.64:0/3119569518 conn(0x7ff1980fbd20 msgr2=0x7ff1980fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 -- 172.21.15.64:0/3119569518 shutdown_connections 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.194+0000 7ff19f51d640 1 -- 172.21.15.64:0/3119569518 wait complete. 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 Processor -- start 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 -- start start 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 0x7ff198101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff198104c40 0x7ff1981002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff198107570 0x7ff198100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff19810d3c0 con 0x7ff198102310 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff19810d240 con 0x7ff198107570 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19f51d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff19810d540 con 0x7ff198104c40 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19d2b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 0x7ff198101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19d2b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 0x7ff198101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37846/0 (socket says 172.21.15.64:37846) 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19d2b9640 1 -- 172.21.15.64:0/3202711040 learned_addr learned my addr 172.21.15.64:0/3202711040 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19cab8640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff198104c40 0x7ff1981002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19daba640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff198107570 0x7ff198100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19d2b9640 1 -- 172.21.15.64:0/3202711040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff198104c40 msgr2=0x7ff1981002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.195+0000 7ff19d2b9640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff198104c40 0x7ff1981002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff19d2b9640 1 -- 172.21.15.64:0/3202711040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff198107570 msgr2=0x7ff198100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff19d2b9640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff198107570 0x7ff198100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff19d2b9640 1 -- 172.21.15.64:0/3202711040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981010c0 con 0x7ff198102310 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff19d2b9640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 0x7ff198101c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff19400d950 tx=0x7ff19400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194017030 con 0x7ff198102310 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff194004020 con 0x7ff198102310 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981a1c80 con 0x7ff198102310 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1940047a0 con 0x7ff198102310 2024-05-09T09:07:11.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.196+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1981a21a0 con 0x7ff198102310 2024-05-09T09:07:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.197+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1980fd810 con 0x7ff198102310 2024-05-09T09:07:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.198+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff1940041d0 con 0x7ff198102310 2024-05-09T09:07:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.199+0000 7ff18e7fc640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff1840a2c20 0x7ff1840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.199+0000 7ff19cab8640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff1840a2c20 0x7ff1840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.199+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff1940cee40 con 0x7ff198102310 2024-05-09T09:07:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.199+0000 7ff19cab8640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff1840a2c20 0x7ff1840a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff188004770 tx=0x7ff188000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:11.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.203+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff19409ba00 con 0x7ff198102310 2024-05-09T09:07:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.388+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff198061c70 con 0x7ff1840a2c20 2024-05-09T09:07:11.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.397+0000 7ff18e7fc640 1 -- 172.21.15.64:0/3202711040 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340314 (secure 0 0 0) 0x7ff198061c70 con 0x7ff1840a2c20 2024-05-09T09:07:11.400 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:11.402 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:07:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff1840a2c20 msgr2=0x7ff1840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff1840a2c20 0x7ff1840a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff188004770 tx=0x7ff188000bc0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 msgr2=0x7ff198101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:11.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 0x7ff198101c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff19400d950 tx=0x7ff19400de10 comp rx=0 tx=0).stop 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 shutdown_connections 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff1840a2c20 0x7ff1840a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff198107570 0x7ff198100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff198104c40 0x7ff1981002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 --2- 172.21.15.64:0/3202711040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff198102310 0x7ff198101c30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 >> 172.21.15.64:0/3202711040 conn(0x7ff1980fbd20 msgr2=0x7ff198102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 shutdown_connections 2024-05-09T09:07:11.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:11.401+0000 7ff19f51d640 1 -- 172.21.15.64:0/3202711040 wait complete. 2024-05-09T09:07:11.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:11 smithi064 ceph-mon[30149]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:11.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1977513569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:07:11.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3219190589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:11.411 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-05-09T09:07:11.102324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597730,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":139,"num_read_kb":121,"num_write":306,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":246,"ondisk_log_size":246,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391872,"kb_used_data":13600,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549952,"statfs":{"total":1151940427776,"available":1151539150848,"internally_reserved":0,"allocated":13926400,"data_stored":7564754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2722,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":76,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004254"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339280+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:06:46.339280+0000","last_peered":"2024-05-09T09:06:46.339280+0000","last_clean":"2024-05-09T09:06:46.339280+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:06:46.339280+0000","last_undegraded":"2024-05-09T09:06:46.339280+0000","last_fullsized":"2024-05-09T09:06:46.339280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502876+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:06:46.502876+0000","last_peered":"2024-05-09T09:06:46.502876+0000","last_clean":"2024-05-09T09:06:46.502876+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:06:46.502876+0000","last_undegraded":"2024-05-09T09:06:46.502876+0000","last_fullsized":"2024-05-09T09:06:46.502876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502762+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:06:46.502762+0000","last_peered":"2024-05-09T09:06:46.502762+0000","last_clean":"2024-05-09T09:06:46.502762+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:06:46.502762+0000","last_undegraded":"2024-05-09T09:06:46.502762+0000","last_fullsized":"2024-05-09T09:06:46.502762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340076+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:06:46.340076+0000","last_peered":"2024-05-09T09:06:46.340076+0000","last_clean":"2024-05-09T09:06:46.340076+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:06:46.340076+0000","last_undegraded":"2024-05-09T09:06:46.340076+0000","last_fullsized":"2024-05-09T09:06:46.340076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339799+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:06:46.339799+0000","last_peered":"2024-05-09T09:06:46.339799+0000","last_clean":"2024-05-09T09:06:46.339799+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:06:46.339799+0000","last_undegraded":"2024-05-09T09:06:46.339799+0000","last_fullsized":"2024-05-09T09:06:46.339799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339742+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:06:46.339742+0000","last_peered":"2024-05-09T09:06:46.339742+0000","last_clean":"2024-05-09T09:06:46.339742+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:06:46.339742+0000","last_undegraded":"2024-05-09T09:06:46.339742+0000","last_fullsized":"2024-05-09T09:06:46.339742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157288+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:06:46.157288+0000","last_peered":"2024-05-09T09:06:46.157288+0000","last_clean":"2024-05-09T09:06:46.157288+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:06:46.157288+0000","last_undegraded":"2024-05-09T09:06:46.157288+0000","last_fullsized":"2024-05-09T09:06:46.157288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338094+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:06:46.338094+0000","last_peered":"2024-05-09T09:06:46.338094+0000","last_clean":"2024-05-09T09:06:46.338094+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:06:46.338094+0000","last_undegraded":"2024-05-09T09:06:46.338094+0000","last_fullsized":"2024-05-09T09:06:46.338094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339450+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:06:46.339450+0000","last_peered":"2024-05-09T09:06:46.339450+0000","last_clean":"2024-05-09T09:06:46.339450+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:06:46.339450+0000","last_undegraded":"2024-05-09T09:06:46.339450+0000","last_fullsized":"2024-05-09T09:06:46.339450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340239+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:06:46.340239+0000","last_peered":"2024-05-09T09:06:46.340239+0000","last_clean":"2024-05-09T09:06:46.340239+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:06:46.340239+0000","last_undegraded":"2024-05-09T09:06:46.340239+0000","last_fullsized":"2024-05-09T09:06:46.340239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.158034+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:06:46.158034+0000","last_peered":"2024-05-09T09:06:46.158034+0000","last_clean":"2024-05-09T09:06:46.158034+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:06:46.158034+0000","last_undegraded":"2024-05-09T09:06:46.158034+0000","last_fullsized":"2024-05-09T09:06:46.158034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340060+0000","last_change":"2024-05-09T09:06:41.137455+0000","last_active":"2024-05-09T09:06:46.340060+0000","last_peered":"2024-05-09T09:06:46.340060+0000","last_clean":"2024-05-09T09:06:46.340060+0000","last_became_active":"2024-05-09T09:06:41.136139+0000","last_became_peered":"2024-05-09T09:06:41.136139+0000","last_unstale":"2024-05-09T09:06:46.340060+0000","last_undegraded":"2024-05-09T09:06:46.340060+0000","last_fullsized":"2024-05-09T09:06:46.340060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:10.693831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338938+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:06:46.338938+0000","last_peered":"2024-05-09T09:06:46.338938+0000","last_clean":"2024-05-09T09:06:46.338938+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:06:46.338938+0000","last_undegraded":"2024-05-09T09:06:46.338938+0000","last_fullsized":"2024-05-09T09:06:46.338938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502463+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:06:46.502463+0000","last_peered":"2024-05-09T09:06:46.502463+0000","last_clean":"2024-05-09T09:06:46.502463+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:06:46.502463+0000","last_undegraded":"2024-05-09T09:06:46.502463+0000","last_fullsized":"2024-05-09T09:06:46.502463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502512+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:06:46.502512+0000","last_peered":"2024-05-09T09:06:46.502512+0000","last_clean":"2024-05-09T09:06:46.502512+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:06:46.502512+0000","last_undegraded":"2024-05-09T09:06:46.502512+0000","last_fullsized":"2024-05-09T09:06:46.502512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338292+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:06:46.338292+0000","last_peered":"2024-05-09T09:06:46.338292+0000","last_clean":"2024-05-09T09:06:46.338292+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:06:46.338292+0000","last_undegraded":"2024-05-09T09:06:46.338292+0000","last_fullsized":"2024-05-09T09:06:46.338292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339235+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:06:46.339235+0000","last_peered":"2024-05-09T09:06:46.339235+0000","last_clean":"2024-05-09T09:06:46.339235+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:06:46.339235+0000","last_undegraded":"2024-05-09T09:06:46.339235+0000","last_fullsized":"2024-05-09T09:06:46.339235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157064+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:06:46.157064+0000","last_peered":"2024-05-09T09:06:46.157064+0000","last_clean":"2024-05-09T09:06:46.157064+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:06:46.157064+0000","last_undegraded":"2024-05-09T09:06:46.157064+0000","last_fullsized":"2024-05-09T09:06:46.157064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339066+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:06:46.339066+0000","last_peered":"2024-05-09T09:06:46.339066+0000","last_clean":"2024-05-09T09:06:46.339066+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:06:46.339066+0000","last_undegraded":"2024-05-09T09:06:46.339066+0000","last_fullsized":"2024-05-09T09:06:46.339066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339206+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:06:46.339206+0000","last_peered":"2024-05-09T09:06:46.339206+0000","last_clean":"2024-05-09T09:06:46.339206+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:06:46.339206+0000","last_undegraded":"2024-05-09T09:06:46.339206+0000","last_fullsized":"2024-05-09T09:06:46.339206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.238893+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:07:07.238893+0000","last_peered":"2024-05-09T09:07:07.238893+0000","last_clean":"2024-05-09T09:07:07.238893+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:07:07.238893+0000","last_undegraded":"2024-05-09T09:07:07.238893+0000","last_fullsized":"2024-05-09T09:07:07.238893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338179+0000","last_change":"2024-05-09T09:06:41.137679+0000","last_active":"2024-05-09T09:06:46.338179+0000","last_peered":"2024-05-09T09:06:46.338179+0000","last_clean":"2024-05-09T09:06:46.338179+0000","last_became_active":"2024-05-09T09:06:41.137353+0000","last_became_peered":"2024-05-09T09:06:41.137353+0000","last_unstale":"2024-05-09T09:06:46.338179+0000","last_undegraded":"2024-05-09T09:06:46.338179+0000","last_fullsized":"2024-05-09T09:06:46.338179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:36.080021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338646+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:06:46.338646+0000","last_peered":"2024-05-09T09:06:46.338646+0000","last_clean":"2024-05-09T09:06:46.338646+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:06:46.338646+0000","last_undegraded":"2024-05-09T09:06:46.338646+0000","last_fullsized":"2024-05-09T09:06:46.338646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338583+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:06:46.338583+0000","last_peered":"2024-05-09T09:06:46.338583+0000","last_clean":"2024-05-09T09:06:46.338583+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:06:46.338583+0000","last_undegraded":"2024-05-09T09:06:46.338583+0000","last_fullsized":"2024-05-09T09:06:46.338583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502915+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:06:46.502915+0000","last_peered":"2024-05-09T09:06:46.502915+0000","last_clean":"2024-05-09T09:06:46.502915+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:06:46.502915+0000","last_undegraded":"2024-05-09T09:06:46.502915+0000","last_fullsized":"2024-05-09T09:06:46.502915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339294+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:06:46.339294+0000","last_peered":"2024-05-09T09:06:46.339294+0000","last_clean":"2024-05-09T09:06:46.339294+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:06:46.339294+0000","last_undegraded":"2024-05-09T09:06:46.339294+0000","last_fullsized":"2024-05-09T09:06:46.339294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.235765+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:07:07.235765+0000","last_peered":"2024-05-09T09:07:07.235765+0000","last_clean":"2024-05-09T09:07:07.235765+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:07:07.235765+0000","last_undegraded":"2024-05-09T09:07:07.235765+0000","last_fullsized":"2024-05-09T09:07:07.235765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338732+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:06:46.338732+0000","last_peered":"2024-05-09T09:06:46.338732+0000","last_clean":"2024-05-09T09:06:46.338732+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:06:46.338732+0000","last_undegraded":"2024-05-09T09:06:46.338732+0000","last_fullsized":"2024-05-09T09:06:46.338732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339401+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:06:46.339401+0000","last_peered":"2024-05-09T09:06:46.339401+0000","last_clean":"2024-05-09T09:06:46.339401+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:06:46.339401+0000","last_undegraded":"2024-05-09T09:06:46.339401+0000","last_fullsized":"2024-05-09T09:06:46.339401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339157+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:06:46.339157+0000","last_peered":"2024-05-09T09:06:46.339157+0000","last_clean":"2024-05-09T09:06:46.339157+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:06:46.339157+0000","last_undegraded":"2024-05-09T09:06:46.339157+0000","last_fullsized":"2024-05-09T09:06:46.339157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160309+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:06:46.160309+0000","last_peered":"2024-05-09T09:06:46.160309+0000","last_clean":"2024-05-09T09:06:46.160309+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:06:46.160309+0000","last_undegraded":"2024-05-09T09:06:46.160309+0000","last_fullsized":"2024-05-09T09:06:46.160309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338983+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:06:46.338983+0000","last_peered":"2024-05-09T09:06:46.338983+0000","last_clean":"2024-05-09T09:06:46.338983+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:06:46.338983+0000","last_undegraded":"2024-05-09T09:06:46.338983+0000","last_fullsized":"2024-05-09T09:06:46.338983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.226235+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:07:06.226235+0000","last_peered":"2024-05-09T09:07:06.226235+0000","last_clean":"2024-05-09T09:07:06.226235+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:07:06.226235+0000","last_undegraded":"2024-05-09T09:07:06.226235+0000","last_fullsized":"2024-05-09T09:07:06.226235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339646+0000","last_change":"2024-05-09T09:06:41.131157+0000","last_active":"2024-05-09T09:06:46.339646+0000","last_peered":"2024-05-09T09:06:46.339646+0000","last_clean":"2024-05-09T09:06:46.339646+0000","last_became_active":"2024-05-09T09:06:41.130708+0000","last_became_peered":"2024-05-09T09:06:41.130708+0000","last_unstale":"2024-05-09T09:06:46.339646+0000","last_undegraded":"2024-05-09T09:06:46.339646+0000","last_fullsized":"2024-05-09T09:06:46.339646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:09.418840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339865+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:06:46.339865+0000","last_peered":"2024-05-09T09:06:46.339865+0000","last_clean":"2024-05-09T09:06:46.339865+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:06:46.339865+0000","last_undegraded":"2024-05-09T09:06:46.339865+0000","last_fullsized":"2024-05-09T09:06:46.339865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.235658+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:07:07.235658+0000","last_peered":"2024-05-09T09:07:07.235658+0000","last_clean":"2024-05-09T09:07:07.235658+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:07:07.235658+0000","last_undegraded":"2024-05-09T09:07:07.235658+0000","last_fullsized":"2024-05-09T09:07:07.235658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160249+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:06:46.160249+0000","last_peered":"2024-05-09T09:06:46.160249+0000","last_clean":"2024-05-09T09:06:46.160249+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:06:46.160249+0000","last_undegraded":"2024-05-09T09:06:46.160249+0000","last_fullsized":"2024-05-09T09:06:46.160249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339218+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:06:46.339218+0000","last_peered":"2024-05-09T09:06:46.339218+0000","last_clean":"2024-05-09T09:06:46.339218+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:06:46.339218+0000","last_undegraded":"2024-05-09T09:06:46.339218+0000","last_fullsized":"2024-05-09T09:06:46.339218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338052+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:06:46.338052+0000","last_peered":"2024-05-09T09:06:46.338052+0000","last_clean":"2024-05-09T09:06:46.338052+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:06:46.338052+0000","last_undegraded":"2024-05-09T09:06:46.338052+0000","last_fullsized":"2024-05-09T09:06:46.338052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338385+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:06:46.338385+0000","last_peered":"2024-05-09T09:06:46.338385+0000","last_clean":"2024-05-09T09:06:46.338385+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:06:46.338385+0000","last_undegraded":"2024-05-09T09:06:46.338385+0000","last_fullsized":"2024-05-09T09:06:46.338385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502898+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:06:46.502898+0000","last_peered":"2024-05-09T09:06:46.502898+0000","last_clean":"2024-05-09T09:06:46.502898+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:06:46.502898+0000","last_undegraded":"2024-05-09T09:06:46.502898+0000","last_fullsized":"2024-05-09T09:06:46.502898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159833+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:06:46.159833+0000","last_peered":"2024-05-09T09:06:46.159833+0000","last_clean":"2024-05-09T09:06:46.159833+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:06:46.159833+0000","last_undegraded":"2024-05-09T09:06:46.159833+0000","last_fullsized":"2024-05-09T09:06:46.159833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156493+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:06:46.156493+0000","last_peered":"2024-05-09T09:06:46.156493+0000","last_clean":"2024-05-09T09:06:46.156493+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:06:46.156493+0000","last_undegraded":"2024-05-09T09:06:46.156493+0000","last_fullsized":"2024-05-09T09:06:46.156493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339370+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:06:46.339370+0000","last_peered":"2024-05-09T09:06:46.339370+0000","last_clean":"2024-05-09T09:06:46.339370+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:06:46.339370+0000","last_undegraded":"2024-05-09T09:06:46.339370+0000","last_fullsized":"2024-05-09T09:06:46.339370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.226013+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:07:06.226013+0000","last_peered":"2024-05-09T09:07:06.226013+0000","last_clean":"2024-05-09T09:07:06.226013+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:07:06.226013+0000","last_undegraded":"2024-05-09T09:07:06.226013+0000","last_fullsized":"2024-05-09T09:07:06.226013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339115+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:06:46.339115+0000","last_peered":"2024-05-09T09:06:46.339115+0000","last_clean":"2024-05-09T09:06:46.339115+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:06:46.339115+0000","last_undegraded":"2024-05-09T09:06:46.339115+0000","last_fullsized":"2024-05-09T09:06:46.339115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156591+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:06:46.156591+0000","last_peered":"2024-05-09T09:06:46.156591+0000","last_clean":"2024-05-09T09:06:46.156591+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:06:46.156591+0000","last_undegraded":"2024-05-09T09:06:46.156591+0000","last_fullsized":"2024-05-09T09:06:46.156591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225020+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:07:06.225020+0000","last_peered":"2024-05-09T09:07:06.225020+0000","last_clean":"2024-05-09T09:07:06.225020+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:07:06.225020+0000","last_undegraded":"2024-05-09T09:07:06.225020+0000","last_fullsized":"2024-05-09T09:07:06.225020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502875+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:06:46.502875+0000","last_peered":"2024-05-09T09:06:46.502875+0000","last_clean":"2024-05-09T09:06:46.502875+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:06:46.502875+0000","last_undegraded":"2024-05-09T09:06:46.502875+0000","last_fullsized":"2024-05-09T09:06:46.502875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339641+0000","last_change":"2024-05-09T09:06:45.186842+0000","last_active":"2024-05-09T09:06:46.339641+0000","last_peered":"2024-05-09T09:06:46.339641+0000","last_clean":"2024-05-09T09:06:46.339641+0000","last_became_active":"2024-05-09T09:06:45.184745+0000","last_became_peered":"2024-05-09T09:06:45.184745+0000","last_unstale":"2024-05-09T09:06:46.339641+0000","last_undegraded":"2024-05-09T09:06:46.339641+0000","last_fullsized":"2024-05-09T09:06:46.339641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:56.933012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160294+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:06:46.160294+0000","last_peered":"2024-05-09T09:06:46.160294+0000","last_clean":"2024-05-09T09:06:46.160294+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:06:46.160294+0000","last_undegraded":"2024-05-09T09:06:46.160294+0000","last_fullsized":"2024-05-09T09:06:46.160294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338841+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:06:46.338841+0000","last_peered":"2024-05-09T09:06:46.338841+0000","last_clean":"2024-05-09T09:06:46.338841+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:06:46.338841+0000","last_undegraded":"2024-05-09T09:06:46.338841+0000","last_fullsized":"2024-05-09T09:06:46.338841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156854+0000","last_change":"2024-05-09T09:06:45.182604+0000","last_active":"2024-05-09T09:06:46.156854+0000","last_peered":"2024-05-09T09:06:46.156854+0000","last_clean":"2024-05-09T09:06:46.156854+0000","last_became_active":"2024-05-09T09:06:45.181320+0000","last_became_peered":"2024-05-09T09:06:45.181320+0000","last_unstale":"2024-05-09T09:06:46.156854+0000","last_undegraded":"2024-05-09T09:06:46.156854+0000","last_fullsized":"2024-05-09T09:06:46.156854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:48.099742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225582+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:07:06.225582+0000","last_peered":"2024-05-09T09:07:06.225582+0000","last_clean":"2024-05-09T09:07:06.225582+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:07:06.225582+0000","last_undegraded":"2024-05-09T09:07:06.225582+0000","last_fullsized":"2024-05-09T09:07:06.225582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339080+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:06:46.339080+0000","last_peered":"2024-05-09T09:06:46.339080+0000","last_clean":"2024-05-09T09:06:46.339080+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:06:46.339080+0000","last_undegraded":"2024-05-09T09:06:46.339080+0000","last_fullsized":"2024-05-09T09:06:46.339080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160636+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:06:46.160636+0000","last_peered":"2024-05-09T09:06:46.160636+0000","last_clean":"2024-05-09T09:06:46.160636+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:06:46.160636+0000","last_undegraded":"2024-05-09T09:06:46.160636+0000","last_fullsized":"2024-05-09T09:06:46.160636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.236122+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:07:07.236122+0000","last_peered":"2024-05-09T09:07:07.236122+0000","last_clean":"2024-05-09T09:07:07.236122+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:07:07.236122+0000","last_undegraded":"2024-05-09T09:07:07.236122+0000","last_fullsized":"2024-05-09T09:07:07.236122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338800+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:06:46.338800+0000","last_peered":"2024-05-09T09:06:46.338800+0000","last_clean":"2024-05-09T09:06:46.338800+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:06:46.338800+0000","last_undegraded":"2024-05-09T09:06:46.338800+0000","last_fullsized":"2024-05-09T09:06:46.338800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338731+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:06:46.338731+0000","last_peered":"2024-05-09T09:06:46.338731+0000","last_clean":"2024-05-09T09:06:46.338731+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:06:46.338731+0000","last_undegraded":"2024-05-09T09:06:46.338731+0000","last_fullsized":"2024-05-09T09:06:46.338731+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225721+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:07:06.225721+0000","last_peered":"2024-05-09T09:07:06.225721+0000","last_clean":"2024-05-09T09:07:06.225721+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:07:06.225721+0000","last_undegraded":"2024-05-09T09:07:06.225721+0000","last_fullsized":"2024-05-09T09:07:06.225721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157740+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:06:46.157740+0000","last_peered":"2024-05-09T09:06:46.157740+0000","last_clean":"2024-05-09T09:06:46.157740+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:06:46.157740+0000","last_undegraded":"2024-05-09T09:06:46.157740+0000","last_fullsized":"2024-05-09T09:06:46.157740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157141+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:06:46.157141+0000","last_peered":"2024-05-09T09:06:46.157141+0000","last_clean":"2024-05-09T09:06:46.157141+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:06:46.157141+0000","last_undegraded":"2024-05-09T09:06:46.157141+0000","last_fullsized":"2024-05-09T09:06:46.157141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:08.238372+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:07:08.238372+0000","last_peered":"2024-05-09T09:07:08.238372+0000","last_clean":"2024-05-09T09:07:08.238372+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:07:08.238372+0000","last_undegraded":"2024-05-09T09:07:08.238372+0000","last_fullsized":"2024-05-09T09:07:08.238372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157372+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:06:46.157372+0000","last_peered":"2024-05-09T09:06:46.157372+0000","last_clean":"2024-05-09T09:06:46.157372+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:06:46.157372+0000","last_undegraded":"2024-05-09T09:06:46.157372+0000","last_fullsized":"2024-05-09T09:06:46.157372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339266+0000","last_change":"2024-05-09T09:06:45.184784+0000","last_active":"2024-05-09T09:06:46.339266+0000","last_peered":"2024-05-09T09:06:46.339266+0000","last_clean":"2024-05-09T09:06:46.339266+0000","last_became_active":"2024-05-09T09:06:45.184394+0000","last_became_peered":"2024-05-09T09:06:45.184394+0000","last_unstale":"2024-05-09T09:06:46.339266+0000","last_undegraded":"2024-05-09T09:06:46.339266+0000","last_fullsized":"2024-05-09T09:06:46.339266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:02.111601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.237499+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:07:07.237499+0000","last_peered":"2024-05-09T09:07:07.237499+0000","last_clean":"2024-05-09T09:07:07.237499+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:07:07.237499+0000","last_undegraded":"2024-05-09T09:07:07.237499+0000","last_fullsized":"2024-05-09T09:07:07.237499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159737+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:06:46.159737+0000","last_peered":"2024-05-09T09:06:46.159737+0000","last_clean":"2024-05-09T09:06:46.159737+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:06:46.159737+0000","last_undegraded":"2024-05-09T09:06:46.159737+0000","last_fullsized":"2024-05-09T09:06:46.159737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159899+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:06:46.159899+0000","last_peered":"2024-05-09T09:06:46.159899+0000","last_clean":"2024-05-09T09:06:46.159899+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:06:46.159899+0000","last_undegraded":"2024-05-09T09:06:46.159899+0000","last_fullsized":"2024-05-09T09:06:46.159899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:08.232152+0000","last_change":"2024-05-09T09:06:40.113621+0000","last_active":"2024-05-09T09:07:08.232152+0000","last_peered":"2024-05-09T09:07:08.232152+0000","last_clean":"2024-05-09T09:07:08.232152+0000","last_became_active":"2024-05-09T09:06:40.113283+0000","last_became_peered":"2024-05-09T09:06:40.113283+0000","last_unstale":"2024-05-09T09:07:08.232152+0000","last_undegraded":"2024-05-09T09:07:08.232152+0000","last_fullsized":"2024-05-09T09:07:08.232152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:27.393494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339085+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:06:46.339085+0000","last_peered":"2024-05-09T09:06:46.339085+0000","last_clean":"2024-05-09T09:06:46.339085+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:06:46.339085+0000","last_undegraded":"2024-05-09T09:06:46.339085+0000","last_fullsized":"2024-05-09T09:06:46.339085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339682+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:06:46.339682+0000","last_peered":"2024-05-09T09:06:46.339682+0000","last_clean":"2024-05-09T09:06:46.339682+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:06:46.339682+0000","last_undegraded":"2024-05-09T09:06:46.339682+0000","last_fullsized":"2024-05-09T09:06:46.339682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.156882+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:06:46.156882+0000","last_peered":"2024-05-09T09:06:46.156882+0000","last_clean":"2024-05-09T09:06:46.156882+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:06:46.156882+0000","last_undegraded":"2024-05-09T09:06:46.156882+0000","last_fullsized":"2024-05-09T09:06:46.156882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339315+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:06:46.339315+0000","last_peered":"2024-05-09T09:06:46.339315+0000","last_clean":"2024-05-09T09:06:46.339315+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:06:46.339315+0000","last_undegraded":"2024-05-09T09:06:46.339315+0000","last_fullsized":"2024-05-09T09:06:46.339315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338612+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:06:46.338612+0000","last_peered":"2024-05-09T09:06:46.338612+0000","last_clean":"2024-05-09T09:06:46.338612+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:06:46.338612+0000","last_undegraded":"2024-05-09T09:06:46.338612+0000","last_fullsized":"2024-05-09T09:06:46.338612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157473+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:06:46.157473+0000","last_peered":"2024-05-09T09:06:46.157473+0000","last_clean":"2024-05-09T09:06:46.157473+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:06:46.157473+0000","last_undegraded":"2024-05-09T09:06:46.157473+0000","last_fullsized":"2024-05-09T09:06:46.157473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339975+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:06:46.339975+0000","last_peered":"2024-05-09T09:06:46.339975+0000","last_clean":"2024-05-09T09:06:46.339975+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:06:46.339975+0000","last_undegraded":"2024-05-09T09:06:46.339975+0000","last_fullsized":"2024-05-09T09:06:46.339975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339181+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:06:46.339181+0000","last_peered":"2024-05-09T09:06:46.339181+0000","last_clean":"2024-05-09T09:06:46.339181+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:06:46.339181+0000","last_undegraded":"2024-05-09T09:06:46.339181+0000","last_fullsized":"2024-05-09T09:06:46.339181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159895+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:06:46.159895+0000","last_peered":"2024-05-09T09:06:46.159895+0000","last_clean":"2024-05-09T09:06:46.159895+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:06:46.159895+0000","last_undegraded":"2024-05-09T09:06:46.159895+0000","last_fullsized":"2024-05-09T09:06:46.159895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.337975+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:06:46.337975+0000","last_peered":"2024-05-09T09:06:46.337975+0000","last_clean":"2024-05-09T09:06:46.337975+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:06:46.337975+0000","last_undegraded":"2024-05-09T09:06:46.337975+0000","last_fullsized":"2024-05-09T09:06:46.337975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157948+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:06:46.157948+0000","last_peered":"2024-05-09T09:06:46.157948+0000","last_clean":"2024-05-09T09:06:46.157948+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:06:46.157948+0000","last_undegraded":"2024-05-09T09:06:46.157948+0000","last_fullsized":"2024-05-09T09:06:46.157948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338184+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:06:46.338184+0000","last_peered":"2024-05-09T09:06:46.338184+0000","last_clean":"2024-05-09T09:06:46.338184+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:06:46.338184+0000","last_undegraded":"2024-05-09T09:06:46.338184+0000","last_fullsized":"2024-05-09T09:06:46.338184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:08.235954+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:07:08.235954+0000","last_peered":"2024-05-09T09:07:08.235954+0000","last_clean":"2024-05-09T09:07:08.235954+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:07:08.235954+0000","last_undegraded":"2024-05-09T09:07:08.235954+0000","last_fullsized":"2024-05-09T09:07:08.235954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":346,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157185+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:06:46.157185+0000","last_peered":"2024-05-09T09:06:46.157185+0000","last_clean":"2024-05-09T09:06:46.157185+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:06:46.157185+0000","last_undegraded":"2024-05-09T09:06:46.157185+0000","last_fullsized":"2024-05-09T09:06:46.157185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160485+0000","last_change":"2024-05-09T09:06:45.180565+0000","last_active":"2024-05-09T09:06:46.160485+0000","last_peered":"2024-05-09T09:06:46.160485+0000","last_clean":"2024-05-09T09:06:46.160485+0000","last_became_active":"2024-05-09T09:06:45.180141+0000","last_became_peered":"2024-05-09T09:06:45.180141+0000","last_unstale":"2024-05-09T09:06:46.160485+0000","last_undegraded":"2024-05-09T09:06:46.160485+0000","last_fullsized":"2024-05-09T09:06:46.160485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:58.412376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.337905+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:06:46.337905+0000","last_peered":"2024-05-09T09:06:46.337905+0000","last_clean":"2024-05-09T09:06:46.337905+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:06:46.337905+0000","last_undegraded":"2024-05-09T09:06:46.337905+0000","last_fullsized":"2024-05-09T09:06:46.337905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.235949+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:07:07.235949+0000","last_peered":"2024-05-09T09:07:07.235949+0000","last_clean":"2024-05-09T09:07:07.235949+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:07:07.235949+0000","last_undegraded":"2024-05-09T09:07:07.235949+0000","last_fullsized":"2024-05-09T09:07:07.235949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339071+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:06:46.339071+0000","last_peered":"2024-05-09T09:06:46.339071+0000","last_clean":"2024-05-09T09:06:46.339071+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:06:46.339071+0000","last_undegraded":"2024-05-09T09:06:46.339071+0000","last_fullsized":"2024-05-09T09:06:46.339071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.223672+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:07:06.223672+0000","last_peered":"2024-05-09T09:07:06.223672+0000","last_clean":"2024-05-09T09:07:06.223672+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:07:06.223672+0000","last_undegraded":"2024-05-09T09:07:06.223672+0000","last_fullsized":"2024-05-09T09:07:06.223672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338716+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:06:46.338716+0000","last_peered":"2024-05-09T09:06:46.338716+0000","last_clean":"2024-05-09T09:06:46.338716+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:06:46.338716+0000","last_undegraded":"2024-05-09T09:06:46.338716+0000","last_fullsized":"2024-05-09T09:06:46.338716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338326+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:06:46.338326+0000","last_peered":"2024-05-09T09:06:46.338326+0000","last_clean":"2024-05-09T09:06:46.338326+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:06:46.338326+0000","last_undegraded":"2024-05-09T09:06:46.338326+0000","last_fullsized":"2024-05-09T09:06:46.338326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:09.133330+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:07:09.133330+0000","last_peered":"2024-05-09T09:07:09.133330+0000","last_clean":"2024-05-09T09:07:09.133330+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:07:09.133330+0000","last_undegraded":"2024-05-09T09:07:09.133330+0000","last_fullsized":"2024-05-09T09:07:09.133330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338391+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:06:46.338391+0000","last_peered":"2024-05-09T09:06:46.338391+0000","last_clean":"2024-05-09T09:06:46.338391+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:06:46.338391+0000","last_undegraded":"2024-05-09T09:06:46.338391+0000","last_fullsized":"2024-05-09T09:06:46.338391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.340207+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:06:46.340207+0000","last_peered":"2024-05-09T09:06:46.340207+0000","last_clean":"2024-05-09T09:06:46.340207+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:06:46.340207+0000","last_undegraded":"2024-05-09T09:06:46.340207+0000","last_fullsized":"2024-05-09T09:06:46.340207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.226643+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:07:06.226643+0000","last_peered":"2024-05-09T09:07:06.226643+0000","last_clean":"2024-05-09T09:07:06.226643+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:07:06.226643+0000","last_undegraded":"2024-05-09T09:07:06.226643+0000","last_fullsized":"2024-05-09T09:07:06.226643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502980+0000","last_change":"2024-05-09T09:06:41.134756+0000","last_active":"2024-05-09T09:06:46.502980+0000","last_peered":"2024-05-09T09:06:46.502980+0000","last_clean":"2024-05-09T09:06:46.502980+0000","last_became_active":"2024-05-09T09:06:41.134420+0000","last_became_peered":"2024-05-09T09:06:41.134420+0000","last_unstale":"2024-05-09T09:06:46.502980+0000","last_undegraded":"2024-05-09T09:06:46.502980+0000","last_fullsized":"2024-05-09T09:06:46.502980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:29:15.995691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339333+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:06:46.339333+0000","last_peered":"2024-05-09T09:06:46.339333+0000","last_clean":"2024-05-09T09:06:46.339333+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:06:46.339333+0000","last_undegraded":"2024-05-09T09:06:46.339333+0000","last_fullsized":"2024-05-09T09:06:46.339333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.224744+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:07:06.224744+0000","last_peered":"2024-05-09T09:07:06.224744+0000","last_clean":"2024-05-09T09:07:06.224744+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:07:06.224744+0000","last_undegraded":"2024-05-09T09:07:06.224744+0000","last_fullsized":"2024-05-09T09:07:06.224744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338594+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:06:46.338594+0000","last_peered":"2024-05-09T09:06:46.338594+0000","last_clean":"2024-05-09T09:06:46.338594+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:06:46.338594+0000","last_undegraded":"2024-05-09T09:06:46.338594+0000","last_fullsized":"2024-05-09T09:06:46.338594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157353+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:06:46.157353+0000","last_peered":"2024-05-09T09:06:46.157353+0000","last_clean":"2024-05-09T09:06:46.157353+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:06:46.157353+0000","last_undegraded":"2024-05-09T09:06:46.157353+0000","last_fullsized":"2024-05-09T09:06:46.157353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.227063+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:07:06.227063+0000","last_peered":"2024-05-09T09:07:06.227063+0000","last_clean":"2024-05-09T09:07:06.227063+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:07:06.227063+0000","last_undegraded":"2024-05-09T09:07:06.227063+0000","last_fullsized":"2024-05-09T09:07:06.227063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338431+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:06:46.338431+0000","last_peered":"2024-05-09T09:06:46.338431+0000","last_clean":"2024-05-09T09:06:46.338431+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:06:46.338431+0000","last_undegraded":"2024-05-09T09:06:46.338431+0000","last_fullsized":"2024-05-09T09:06:46.338431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339996+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:06:46.339996+0000","last_peered":"2024-05-09T09:06:46.339996+0000","last_clean":"2024-05-09T09:06:46.339996+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:06:46.339996+0000","last_undegraded":"2024-05-09T09:06:46.339996+0000","last_fullsized":"2024-05-09T09:06:46.339996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338769+0000","last_change":"2024-05-09T09:06:40.113416+0000","last_active":"2024-05-09T09:06:46.338769+0000","last_peered":"2024-05-09T09:06:46.338769+0000","last_clean":"2024-05-09T09:06:46.338769+0000","last_became_active":"2024-05-09T09:06:40.112949+0000","last_became_peered":"2024-05-09T09:06:40.112949+0000","last_unstale":"2024-05-09T09:06:46.338769+0000","last_undegraded":"2024-05-09T09:06:46.338769+0000","last_fullsized":"2024-05-09T09:06:46.338769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:47.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338945+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:06:46.338945+0000","last_peered":"2024-05-09T09:06:46.338945+0000","last_clean":"2024-05-09T09:06:46.338945+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:06:46.338945+0000","last_undegraded":"2024-05-09T09:06:46.338945+0000","last_fullsized":"2024-05-09T09:06:46.338945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339304+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:06:46.339304+0000","last_peered":"2024-05-09T09:06:46.339304+0000","last_clean":"2024-05-09T09:06:46.339304+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:06:46.339304+0000","last_undegraded":"2024-05-09T09:06:46.339304+0000","last_fullsized":"2024-05-09T09:06:46.339304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:06.225729+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:07:06.225729+0000","last_peered":"2024-05-09T09:07:06.225729+0000","last_clean":"2024-05-09T09:07:06.225729+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:07:06.225729+0000","last_undegraded":"2024-05-09T09:07:06.225729+0000","last_fullsized":"2024-05-09T09:07:06.225729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157481+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:06:46.157481+0000","last_peered":"2024-05-09T09:06:46.157481+0000","last_clean":"2024-05-09T09:06:46.157481+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:06:46.157481+0000","last_undegraded":"2024-05-09T09:06:46.157481+0000","last_fullsized":"2024-05-09T09:06:46.157481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338346+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:06:46.338346+0000","last_peered":"2024-05-09T09:06:46.338346+0000","last_clean":"2024-05-09T09:06:46.338346+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:06:46.338346+0000","last_undegraded":"2024-05-09T09:06:46.338346+0000","last_fullsized":"2024-05-09T09:06:46.338346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:07:07.236682+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:07:07.236682+0000","last_peered":"2024-05-09T09:07:07.236682+0000","last_clean":"2024-05-09T09:07:07.236682+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:07:07.236682+0000","last_undegraded":"2024-05-09T09:07:07.236682+0000","last_fullsized":"2024-05-09T09:07:07.236682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.158108+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:06:46.158108+0000","last_peered":"2024-05-09T09:06:46.158108+0000","last_clean":"2024-05-09T09:06:46.158108+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:06:46.158108+0000","last_undegraded":"2024-05-09T09:06:46.158108+0000","last_fullsized":"2024-05-09T09:06:46.158108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502560+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:06:46.502560+0000","last_peered":"2024-05-09T09:06:46.502560+0000","last_clean":"2024-05-09T09:06:46.502560+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:06:46.502560+0000","last_undegraded":"2024-05-09T09:06:46.502560+0000","last_fullsized":"2024-05-09T09:06:46.502560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.160373+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:06:46.160373+0000","last_peered":"2024-05-09T09:06:46.160373+0000","last_clean":"2024-05-09T09:06:46.160373+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:06:46.160373+0000","last_undegraded":"2024-05-09T09:06:46.160373+0000","last_fullsized":"2024-05-09T09:06:46.160373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.157715+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:06:46.157715+0000","last_peered":"2024-05-09T09:06:46.157715+0000","last_clean":"2024-05-09T09:06:46.157715+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:06:46.157715+0000","last_undegraded":"2024-05-09T09:06:46.157715+0000","last_fullsized":"2024-05-09T09:06:46.157715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159687+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:06:46.159687+0000","last_peered":"2024-05-09T09:06:46.159687+0000","last_clean":"2024-05-09T09:06:46.159687+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:06:46.159687+0000","last_undegraded":"2024-05-09T09:06:46.159687+0000","last_fullsized":"2024-05-09T09:06:46.159687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339113+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:06:46.339113+0000","last_peered":"2024-05-09T09:06:46.339113+0000","last_clean":"2024-05-09T09:06:46.339113+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:06:46.339113+0000","last_undegraded":"2024-05-09T09:06:46.339113+0000","last_fullsized":"2024-05-09T09:06:46.339113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.158050+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:06:46.158050+0000","last_peered":"2024-05-09T09:06:46.158050+0000","last_clean":"2024-05-09T09:06:46.158050+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:06:46.158050+0000","last_undegraded":"2024-05-09T09:06:46.158050+0000","last_fullsized":"2024-05-09T09:06:46.158050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338720+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:06:46.338720+0000","last_peered":"2024-05-09T09:06:46.338720+0000","last_clean":"2024-05-09T09:06:46.338720+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:06:46.338720+0000","last_undegraded":"2024-05-09T09:06:46.338720+0000","last_fullsized":"2024-05-09T09:06:46.338720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339413+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:06:46.339413+0000","last_peered":"2024-05-09T09:06:46.339413+0000","last_clean":"2024-05-09T09:06:46.339413+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:06:46.339413+0000","last_undegraded":"2024-05-09T09:06:46.339413+0000","last_fullsized":"2024-05-09T09:06:46.339413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338642+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:06:46.338642+0000","last_peered":"2024-05-09T09:06:46.338642+0000","last_clean":"2024-05-09T09:06:46.338642+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:06:46.338642+0000","last_undegraded":"2024-05-09T09:06:46.338642+0000","last_fullsized":"2024-05-09T09:06:46.338642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.502628+0000","last_change":"2024-05-09T09:06:41.124927+0000","last_active":"2024-05-09T09:06:46.502628+0000","last_peered":"2024-05-09T09:06:46.502628+0000","last_clean":"2024-05-09T09:06:46.502628+0000","last_became_active":"2024-05-09T09:06:41.124566+0000","last_became_peered":"2024-05-09T09:06:41.124566+0000","last_unstale":"2024-05-09T09:06:46.502628+0000","last_undegraded":"2024-05-09T09:06:46.502628+0000","last_fullsized":"2024-05-09T09:06:46.502628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.792308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338816+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:06:46.338816+0000","last_peered":"2024-05-09T09:06:46.338816+0000","last_clean":"2024-05-09T09:06:46.338816+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:06:46.338816+0000","last_undegraded":"2024-05-09T09:06:46.338816+0000","last_fullsized":"2024-05-09T09:06:46.338816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339196+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:06:46.339196+0000","last_peered":"2024-05-09T09:06:46.339196+0000","last_clean":"2024-05-09T09:06:46.339196+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:06:46.339196+0000","last_undegraded":"2024-05-09T09:06:46.339196+0000","last_fullsized":"2024-05-09T09:06:46.339196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339938+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:06:46.339938+0000","last_peered":"2024-05-09T09:06:46.339938+0000","last_clean":"2024-05-09T09:06:46.339938+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:06:46.339938+0000","last_undegraded":"2024-05-09T09:06:46.339938+0000","last_fullsized":"2024-05-09T09:06:46.339938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338579+0000","last_change":"2024-05-09T09:06:41.131374+0000","last_active":"2024-05-09T09:06:46.338579+0000","last_peered":"2024-05-09T09:06:46.338579+0000","last_clean":"2024-05-09T09:06:46.338579+0000","last_became_active":"2024-05-09T09:06:41.131052+0000","last_became_peered":"2024-05-09T09:06:41.131052+0000","last_unstale":"2024-05-09T09:06:46.338579+0000","last_undegraded":"2024-05-09T09:06:46.338579+0000","last_fullsized":"2024-05-09T09:06:46.338579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:22.200079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.159798+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:06:46.159798+0000","last_peered":"2024-05-09T09:06:46.159798+0000","last_clean":"2024-05-09T09:06:46.159798+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:06:46.159798+0000","last_undegraded":"2024-05-09T09:06:46.159798+0000","last_fullsized":"2024-05-09T09:06:46.159798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.339422+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:06:46.339422+0000","last_peered":"2024-05-09T09:06:46.339422+0000","last_clean":"2024-05-09T09:06:46.339422+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:06:46.339422+0000","last_undegraded":"2024-05-09T09:06:46.339422+0000","last_fullsized":"2024-05-09T09:06:46.339422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338647+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:06:46.338647+0000","last_peered":"2024-05-09T09:06:46.338647+0000","last_clean":"2024-05-09T09:06:46.338647+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:06:46.338647+0000","last_undegraded":"2024-05-09T09:06:46.338647+0000","last_fullsized":"2024-05-09T09:06:46.338647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338823+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:06:46.338823+0000","last_peered":"2024-05-09T09:06:46.338823+0000","last_clean":"2024-05-09T09:06:46.338823+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:06:46.338823+0000","last_undegraded":"2024-05-09T09:06:46.338823+0000","last_fullsized":"2024-05-09T09:06:46.338823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-09T09:06:46.338796+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:06:46.338796+0000","last_peered":"2024-05-09T09:06:46.338796+0000","last_clean":"2024-05-09T09:06:46.338796+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:06:46.338796+0000","last_undegraded":"2024-05-09T09:06:46.338796+0000","last_fullsized":"2024-05-09T09:06:46.338796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7362,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":125,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":41565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1076054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":489276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037795,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":481004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1072426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":484108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":481744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":485458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":480550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":480910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1613824,"data_stored":1072110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:07:11.416 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T09:07:11.417 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T09:07:11.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T09:07:11.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:11 smithi139 ceph-mon[33599]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:11.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:11 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1977513569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:07:11.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:11 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3219190589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:11.702 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:11.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:11 smithi179 ceph-mon[34086]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:07:11.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:11 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1977513569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:07:11.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:11 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3219190589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:12.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:12 smithi064 ceph-mon[30149]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:07:12.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:12 smithi064 ceph-mon[30149]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-09T09:07:12.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:12 smithi064 ceph-mon[30149]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:07:12.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:12 smithi139 ceph-mon[33599]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:07:12.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:12 smithi139 ceph-mon[33599]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-09T09:07:12.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:12 smithi139 ceph-mon[33599]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:07:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:12 smithi179 ceph-mon[34086]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:07:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:12 smithi179 ceph-mon[34086]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-09T09:07:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:12 smithi179 ceph-mon[34086]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2697044763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac1024a0 msgr2=0x7fb0ac102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2697044763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac102900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00099c0 tx=0x7fb0a0009cd0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2697044763 shutdown_connections 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2697044763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac102e40 0x7fb0ac1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2697044763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1012a0 0x7fb0ac1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2697044763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac102900 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.588+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2697044763 >> 172.21.15.64:0/2697044763 conn(0x7fb0ac0fc840 msgr2=0x7fb0ac0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.589+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2697044763 shutdown_connections 2024-05-09T09:07:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.589+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2697044763 wait complete. 2024-05-09T09:07:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.589+0000 7fb0b0e70700 1 Processor -- start 2024-05-09T09:07:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.589+0000 7fb0b0e70700 1 -- start start 2024-05-09T09:07:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0b0e70700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac1012a0 0x7fb0ac198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0b0e70700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0b0e70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 0x7fb0ac19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0b0e70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac10ce00 con 0x7fb0ac102e40 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0b0e70700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac10cc80 con 0x7fb0ac1012a0 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0b0e70700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac10cf80 con 0x7fb0ac1024a0 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 0x7fb0ac19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 0x7fb0ac19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37866/0 (socket says 172.21.15.64:37866) 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 -- 172.21.15.64:0/2359270086 learned_addr learned my addr 172.21.15.64:0/2359270086 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53392/0 (socket says 172.21.15.64:53392) 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aa59c700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac1012a0 0x7fb0ac198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 -- 172.21.15.64:0/2359270086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1024a0 msgr2=0x7fb0ac198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 -- 172.21.15.64:0/2359270086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac1012a0 msgr2=0x7fb0ac198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac1012a0 0x7fb0ac198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 -- 172.21.15.64:0/2359270086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac1a0160 con 0x7fb0ac102e40 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aa59c700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac1012a0 0x7fb0ac198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.590+0000 7fb0aad9d700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 0x7fb0ac19fac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400add0 tx=0x7fb0a400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.591+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4017020 con 0x7fb0ac102e40 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.591+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a400ae00 con 0x7fb0ac102e40 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.591+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac1a0450 con 0x7fb0ac102e40 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.591+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4005350 con 0x7fb0ac102e40 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.591+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac1a0a10 con 0x7fb0ac102e40 2024-05-09T09:07:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.592+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac050890 con 0x7fb0ac102e40 2024-05-09T09:07:13.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.593+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0a400c940 con 0x7fb0ac102e40 2024-05-09T09:07:13.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.594+0000 7fb09b7fe700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0940a2d60 0x7fb0940a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:13.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.594+0000 7fb0aa59c700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0940a2d60 0x7fb0940a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:13.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.594+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb0a40cf1c0 con 0x7fb0ac102e40 2024-05-09T09:07:13.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.594+0000 7fb0aa59c700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0940a2d60 0x7fb0940a5220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac102300 tx=0x7fb09c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:13.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.597+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb0a4021020 con 0x7fb0ac102e40 2024-05-09T09:07:13.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.883+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb0ac1a0ed0 con 0x7fb0ac102e40 2024-05-09T09:07:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.886+0000 7fb09b7fe700 1 -- 172.21.15.64:0/2359270086 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb0ac1a0ed0 con 0x7fb0ac102e40 2024-05-09T09:07:13.885 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-05-09T09:07:13.885 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBBkjxmRIjCNBAAhnPdR1qL6UXe+myMd1S/HA== 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0940a2d60 msgr2=0x7fb0940a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0940a2d60 0x7fb0940a5220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac102300 tx=0x7fb09c008040 comp rx=0 tx=0).stop 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 msgr2=0x7fb0ac19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 0x7fb0ac19fac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400add0 tx=0x7fb0a400c330 comp rx=0 tx=0).stop 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 shutdown_connections 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac102e40 0x7fb0ac19fac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac1012a0 0x7fb0ac198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0940a2d60 0x7fb0940a5220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 --2- 172.21.15.64:0/2359270086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ac1024a0 0x7fb0ac198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:13.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 >> 172.21.15.64:0/2359270086 conn(0x7fb0ac0fc840 msgr2=0x7fb0ac0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:13.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 shutdown_connections 2024-05-09T09:07:13.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:13.888+0000 7fb0b0e70700 1 -- 172.21.15.64:0/2359270086 wait complete. 2024-05-09T09:07:14.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:14 smithi139 ceph-mon[33599]: pgmap v309: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-05-09T09:07:14.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:14 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2359270086' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T09:07:14.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:14 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2359270086' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T09:07:14.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:14 smithi064 ceph-mon[30149]: pgmap v309: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-05-09T09:07:14.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:14 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2359270086' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T09:07:14.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:14 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2359270086' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T09:07:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:14 smithi179 ceph-mon[34086]: pgmap v309: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-05-09T09:07:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:14 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2359270086' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T09:07:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:14 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2359270086' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T09:07:14.741 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:07:14.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T09:07:14.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T09:07:14.785 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T09:07:14.785 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T09:07:14.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T09:07:15.091 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:16.394 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:16 smithi064 ceph-mon[30149]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T09:07:16.531 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:16 smithi179 ceph-mon[34086]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T09:07:16.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:16 smithi139 ceph-mon[33599]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T09:07:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 -- 172.21.15.64:0/1926536277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0806aa50 msgr2=0x7fae0806aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1926536277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0806aa50 0x7fae0806aed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009990 tx=0x7fadfc009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 -- 172.21.15.64:0/1926536277 shutdown_connections 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1926536277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806b410 0x7fae0810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1926536277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae08108430 0x7fae08108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1926536277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0806aa50 0x7fae0806aed0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.973+0000 7fae0da3b700 1 -- 172.21.15.64:0/1926536277 >> 172.21.15.64:0/1926536277 conn(0x7fae08077090 msgr2=0x7fae080774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.974+0000 7fae0da3b700 1 -- 172.21.15.64:0/1926536277 shutdown_connections 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.974+0000 7fae0da3b700 1 -- 172.21.15.64:0/1926536277 wait complete. 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.974+0000 7fae0da3b700 1 Processor -- start 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.974+0000 7fae0da3b700 1 -- start start 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae0da3b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806aa50 0x7fae08101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae0da3b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae0806b410 0x7fae08101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae0da3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae08108430 0x7fae081020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae0da3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fae08112580 con 0x7fae08108430 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae0da3b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae08112400 con 0x7fae0806aa50 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae0da3b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fae08112700 con 0x7fae0806b410 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae08108430 0x7fae081020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae06ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806aa50 0x7fae08101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae067fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae0806b410 0x7fae08101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae06ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806aa50 0x7fae08101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48468/0 (socket says 172.21.15.64:48468) 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae06ffd700 1 -- 172.21.15.64:0/1604066264 learned_addr learned my addr 172.21.15.64:0/1604066264 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae077fe700 1 -- 172.21.15.64:0/1604066264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae0806b410 msgr2=0x7fae08101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae077fe700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae0806b410 0x7fae08101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae077fe700 1 -- 172.21.15.64:0/1604066264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806aa50 msgr2=0x7fae08101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae077fe700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806aa50 0x7fae08101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.975+0000 7fae077fe700 1 -- 172.21.15.64:0/1604066264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae081a6f60 con 0x7fae08108430 2024-05-09T09:07:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.976+0000 7fae077fe700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae08108430 0x7fae081020c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae0000be20 tx=0x7fae00009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.976+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00019070 con 0x7fae08108430 2024-05-09T09:07:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.976+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae0000cb90 con 0x7fae08108430 2024-05-09T09:07:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.976+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae081a7250 con 0x7fae08108430 2024-05-09T09:07:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.976+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000125a0 con 0x7fae08108430 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.976+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae081a7710 con 0x7fae08108430 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.977+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae08103a80 con 0x7fae08108430 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.978+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae0001c070 con 0x7fae08108430 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.979+0000 7fae0ca39700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fadf00a2cb0 0x7fadf00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.979+0000 7fae06ffd700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fadf00a2cb0 0x7fadf00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.979+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fae000ce9f0 con 0x7fae08108430 2024-05-09T09:07:16.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.979+0000 7fae06ffd700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fadf00a2cb0 0x7fadf00a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005950 tx=0x7fadf800a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:16.983+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae0009b6c0 con 0x7fae08108430 2024-05-09T09:07:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.249+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fae08102b70 con 0x7fae08108430 2024-05-09T09:07:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.255+0000 7fae0ca39700 1 -- 172.21.15.64:0/1604066264 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7fae0009afc0 con 0x7fae08108430 2024-05-09T09:07:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fadf00a2cb0 msgr2=0x7fadf00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fadf00a2cb0 0x7fadf00a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005950 tx=0x7fadf800a400 comp rx=0 tx=0).stop 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae08108430 msgr2=0x7fae081020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae08108430 0x7fae081020c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae0000be20 tx=0x7fae00009ed0 comp rx=0 tx=0).stop 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 shutdown_connections 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae08108430 0x7fae081020c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0806aa50 0x7fae08101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fadf00a2cb0 0x7fadf00a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 --2- 172.21.15.64:0/1604066264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae0806b410 0x7fae08101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:17.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 >> 172.21.15.64:0/1604066264 conn(0x7fae08077090 msgr2=0x7fae08100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 shutdown_connections 2024-05-09T09:07:17.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:17.258+0000 7fae0da3b700 1 -- 172.21.15.64:0/1604066264 wait complete. 2024-05-09T09:07:17.562 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:17 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1604066264' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T09:07:17.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:17 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1604066264' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T09:07:17.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:17 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1604066264' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T09:07:18.011 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2084062196},{"type":"v1","addr":"172.21.15.64:6801","nonce":2084062196}]},"active_addr":"172.21.15.64:6801/2084062196","active_change":"2024-05-09T08:59:35.011264+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":608771884}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2225685543}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1191000631}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":31296508}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2491828807}]}]} 2024-05-09T09:07:18.015 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T09:07:18.016 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T09:07:18.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T09:07:18.271 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:18.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:18 smithi139 ceph-mon[33599]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T09:07:18.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:18 smithi064 ceph-mon[30149]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T09:07:18.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:18 smithi179 ceph-mon[34086]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T09:07:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2585560443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec01014b0 msgr2=0x7f4ec0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2585560443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec01014b0 0x7f4ec0101890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099c0 tx=0x7f4eb4009cd0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2585560443 shutdown_connections 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2585560443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ec0106540 0x7f4ec010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2585560443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec01014b0 0x7f4ec0101890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2585560443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ec0101e60 0x7f4ec0105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.095+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2585560443 >> 172.21.15.64:0/2585560443 conn(0x7f4ec00781d0 msgr2=0x7f4ec00785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2585560443 shutdown_connections 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2585560443 wait complete. 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 Processor -- start 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 -- start start 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ec01014b0 0x7f4ec019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 0x7f4ec019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.096+0000 7f4ec78c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ec0106540 0x7f4ec0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec78c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0111420 con 0x7f4ec0101e60 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec78c3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01112a0 con 0x7f4ec01014b0 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec78c3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01115a0 con 0x7f4ec0106540 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 0x7f4ec019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 0x7f4ec019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36984/0 (socket says 172.21.15.64:36984) 2024-05-09T09:07:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 -- 172.21.15.64:0/2836679610 learned_addr learned my addr 172.21.15.64:0/2836679610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec565f700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ec01014b0 0x7f4ec019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec5e60700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ec0106540 0x7f4ec0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 -- 172.21.15.64:0/2836679610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ec0106540 msgr2=0x7f4ec0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ec0106540 0x7f4ec0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 -- 172.21.15.64:0/2836679610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ec01014b0 msgr2=0x7f4ec019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ec01014b0 0x7f4ec019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 -- 172.21.15.64:0/2836679610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec0198fe0 con 0x7f4ec0101e60 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec4e5e700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 0x7f4ec019ebc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00d980 tx=0x7f4ebc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc017070 con 0x7f4ec0101e60 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01992d0 con 0x7f4ec0101e60 2024-05-09T09:07:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ebc00fd40 con 0x7f4ec0101e60 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00e3b0 con 0x7f4ec0101e60 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.097+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01a54e0 con 0x7f4ec0101e60 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.098+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0102940 con 0x7f4ec0101e60 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.100+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4ebc00f890 con 0x7f4ec0101e60 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.101+0000 7f4eb27fc700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4eac0a2cb0 0x7f4eac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.101+0000 7f4ec565f700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4eac0a2cb0 0x7f4eac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.101+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4ebc0cee80 con 0x7f4ec0101e60 2024-05-09T09:07:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.101+0000 7f4ec565f700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4eac0a2cb0 0x7f4eac0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4000c00 tx=0x7f4eb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.106+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ebc09ba40 con 0x7f4ec0101e60 2024-05-09T09:07:20.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.317+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ec0069010 con 0x7f4ec0101e60 2024-05-09T09:07:20.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.318+0000 7f4eb27fc700 1 -- 172.21.15.64:0/2836679610 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7f4ec0069010 con 0x7f4ec0101e60 2024-05-09T09:07:20.317 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:20.319 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:07:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4eac0a2cb0 msgr2=0x7f4eac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:20.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4eac0a2cb0 0x7f4eac0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4000c00 tx=0x7f4eb4005fb0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 msgr2=0x7f4ec019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 0x7f4ec019ebc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00d980 tx=0x7f4ebc00dc90 comp rx=0 tx=0).stop 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 shutdown_connections 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4eac0a2cb0 0x7f4eac0a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ec0106540 0x7f4ec0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ec01014b0 0x7f4ec019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 --2- 172.21.15.64:0/2836679610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec0101e60 0x7f4ec019ebc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.320+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 >> 172.21.15.64:0/2836679610 conn(0x7f4ec00781d0 msgr2=0x7f4ec0105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.321+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 shutdown_connections 2024-05-09T09:07:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:20.321+0000 7f4ec78c3700 1 -- 172.21.15.64:0/2836679610 wait complete. 2024-05-09T09:07:20.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:20 smithi064 ceph-mon[30149]: pgmap v312: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-05-09T09:07:20.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:20 smithi139 ceph-mon[33599]: pgmap v312: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-05-09T09:07:20.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:20 smithi179 ceph-mon[34086]: pgmap v312: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-05-09T09:07:21.156 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T09:07:21.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T09:07:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:21 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2836679610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:21.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:21 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2836679610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:21.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:21 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2836679610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:22.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:22 smithi139 ceph-mon[33599]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-05-09T09:07:22.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:22 smithi064 ceph-mon[30149]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-05-09T09:07:22.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:22 smithi179 ceph-mon[34086]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 -- 172.21.15.64:0/2069442069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 msgr2=0x7effa010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 --2- 172.21.15.64:0/2069442069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa010f250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b0d0 tx=0x7eff9800b3e0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 -- 172.21.15.64:0/2069442069 shutdown_connections 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 --2- 172.21.15.64:0/2069442069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa010f250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 --2- 172.21.15.64:0/2069442069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006acf0 0x7effa006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 --2- 172.21.15.64:0/2069442069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa006b6a0 0x7effa0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 -- 172.21.15.64:0/2069442069 >> 172.21.15.64:0/2069442069 conn(0x7effa0077290 msgr2=0x7effa00776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 -- 172.21.15.64:0/2069442069 shutdown_connections 2024-05-09T09:07:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.253+0000 7effa6514700 1 -- 172.21.15.64:0/2069442069 wait complete. 2024-05-09T09:07:23.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 Processor -- start 2024-05-09T09:07:23.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 -- start start 2024-05-09T09:07:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa006acf0 0x7effa0101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006b6a0 0x7effa0101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa0101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7effa01125c0 con 0x7effa01076d0 2024-05-09T09:07:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7effa0112440 con 0x7effa006b6a0 2024-05-09T09:07:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa6514700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7effa0112740 con 0x7effa006acf0 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa4ab1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa0101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa4ab1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa0101da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37004/0 (socket says 172.21.15.64:37004) 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7effa4ab1700 1 -- 172.21.15.64:0/1572444808 learned_addr learned my addr 172.21.15.64:0/1572444808 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.254+0000 7eff9ffff700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa006acf0 0x7effa0101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7eff9f7fe700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006b6a0 0x7effa0101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa4ab1700 1 -- 172.21.15.64:0/1572444808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa006acf0 msgr2=0x7effa0101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa4ab1700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa006acf0 0x7effa0101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa4ab1700 1 -- 172.21.15.64:0/1572444808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006b6a0 msgr2=0x7effa0101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa4ab1700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006b6a0 0x7effa0101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa4ab1700 1 -- 172.21.15.64:0/1572444808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01a6fa0 con 0x7effa01076d0 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7eff9f7fe700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006b6a0 0x7effa0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa4ab1700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa0101da0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b820 tx=0x7eff980079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98021070 con 0x7effa01076d0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01a7230 con 0x7effa01076d0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff9801d900 con 0x7effa01076d0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9801b990 con 0x7effa01076d0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.255+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01a76e0 con 0x7effa01076d0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.256+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa0103a80 con 0x7effa01076d0 2024-05-09T09:07:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.257+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff98024070 con 0x7effa01076d0 2024-05-09T09:07:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.258+0000 7eff9d7fa700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7eff8c0a2cb0 0x7eff8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.258+0000 7eff9ffff700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7eff8c0a2cb0 0x7eff8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.259+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7eff980d04f0 con 0x7effa01076d0 2024-05-09T09:07:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.259+0000 7eff9ffff700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7eff8c0a2cb0 0x7eff8c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff90005950 tx=0x7eff9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.263+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff9809d140 con 0x7effa01076d0 2024-05-09T09:07:23.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.469+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effa002db80 con 0x7effa01076d0 2024-05-09T09:07:23.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.471+0000 7eff9d7fa700 1 -- 172.21.15.64:0/1572444808 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24126 (secure 0 0 0) 0x7eff9809ca40 con 0x7effa01076d0 2024-05-09T09:07:23.470 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:07:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:06:45.536183+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:07:23.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7eff8c0a2cb0 msgr2=0x7eff8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:23.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7eff8c0a2cb0 0x7eff8c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff90005950 tx=0x7eff9000b410 comp rx=0 tx=0).stop 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 msgr2=0x7effa0101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa0101da0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b820 tx=0x7eff980079a0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 shutdown_connections 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa01076d0 0x7effa0101da0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7eff8c0a2cb0 0x7eff8c0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa006acf0 0x7effa0101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 --2- 172.21.15.64:0/1572444808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa006b6a0 0x7effa0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.473+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 >> 172.21.15.64:0/1572444808 conn(0x7effa0077290 msgr2=0x7effa00ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.474+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 shutdown_connections 2024-05-09T09:07:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:23.474+0000 7effa6514700 1 -- 172.21.15.64:0/1572444808 wait complete. 2024-05-09T09:07:24.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T09:07:24.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T09:07:24.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T09:07:24.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T09:07:24.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T09:07:24.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T09:07:24.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T09:07:24.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T09:07:24.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T09:07:24.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T09:07:24.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T09:07:24.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T09:07:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:24 smithi139 ceph-mon[33599]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-09T09:07:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:24 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1572444808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:24.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:24 smithi064 ceph-mon[30149]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-09T09:07:24.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:24 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1572444808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:24.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:24 smithi179 ceph-mon[34086]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-09T09:07:24.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:24 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1572444808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:07:24.969 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.129 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.129 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.130 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.134 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.137 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:26.459 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:26 smithi064 ceph-mon[30149]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-09T09:07:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:26 smithi139 ceph-mon[33599]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-09T09:07:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:26 smithi179 ceph-mon[34086]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-09T09:07:28.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:28 smithi064 ceph-mon[30149]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-09T09:07:28.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:28 smithi139 ceph-mon[33599]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-09T09:07:28.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:28 smithi179 ceph-mon[34086]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-09T09:07:30.544 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:30 smithi064 ceph-mon[30149]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-09T09:07:30.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:30 smithi139 ceph-mon[33599]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-09T09:07:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:30 smithi179 ceph-mon[34086]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-09T09:07:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3629085645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0107290 msgr2=0x7fe7f0107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3629085645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0107290 0x7fe7f0107670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009990 tx=0x7fe7e4009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3629085645 shutdown_connections 2024-05-09T09:07:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3629085645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7f0101be0 0x7fe7f010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3629085645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0107290 0x7fe7f0107670 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3629085645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f0101220 0x7fe7f01016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3629085645 >> 172.21.15.64:0/3629085645 conn(0x7fe7f00781d0 msgr2=0x7fe7f00785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3629085645 shutdown_connections 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3629085645 wait complete. 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.306+0000 7fe7f6af3700 1 Processor -- start 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 -- start start 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f0101220 0x7fe7f019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0101be0 0x7fe7f019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7f0107290 0x7fe7f01976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01113b0 con 0x7fe7f0101be0 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0111230 con 0x7fe7f0107290 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f6af3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0111530 con 0x7fe7f0101220 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0101be0 0x7fe7f019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0101be0 0x7fe7f019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33374/0 (socket says 172.21.15.64:33374) 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 -- 172.21.15.64:0/3285570620 learned_addr learned my addr 172.21.15.64:0/3285570620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f488f700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f0101220 0x7fe7f019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7f5090700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7f0107290 0x7fe7f01976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 -- 172.21.15.64:0/3285570620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f0101220 msgr2=0x7fe7f019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f0101220 0x7fe7f019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 -- 172.21.15.64:0/3285570620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7f0107290 msgr2=0x7fe7f01976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7f0107290 0x7fe7f01976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.307+0000 7fe7effff700 1 -- 172.21.15.64:0/3285570620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0197fb0 con 0x7fe7f0101be0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.308+0000 7fe7effff700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0101be0 0x7fe7f019db40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00fb60 tx=0x7fe7dc00fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.308+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc009b40 con 0x7fe7f0101be0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.308+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc009ce0 con 0x7fe7f0101be0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.308+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f0198240 con 0x7fe7f0101be0 2024-05-09T09:07:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.308+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0186d0 con 0x7fe7f0101be0 2024-05-09T09:07:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.308+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f006a1d0 con 0x7fe7f0101be0 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.309+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7f006a780 con 0x7fe7f0101be0 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.310+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe7dc01a450 con 0x7fe7f0101be0 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7edffb700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7e00a2cb0 0x7fe7e00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7f488f700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7e00a2cb0 0x7fe7e00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe7dc0d17c0 con 0x7fe7f0101be0 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7edffb700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] conn(0x7fe7e00ad970 0x7fe7e00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 --> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7dc012ec0 con 0x7fe7e00ad970 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc0d1c10 con 0x7fe7f0101be0 2024-05-09T09:07:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7f5090700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] conn(0x7fe7e00ad970 0x7fe7e00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.311+0000 7fe7f488f700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7e00a2cb0 0x7fe7e00a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400b5c0 tx=0x7fe7e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.312+0000 7fe7f5090700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] conn(0x7fe7e00ad970 0x7fe7e00afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.312+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== osd.9 v2:172.21.15.64:6826/2394827163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7dc012ec0 con 0x7fe7e00ad970 2024-05-09T09:07:32.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.329+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 --> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7f006ac00 con 0x7fe7e00ad970 2024-05-09T09:07:32.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.329+0000 7fe7edffb700 1 -- 172.21.15.64:0/3285570620 <== osd.9 v2:172.21.15.64:6826/2394827163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe7f006ac00 con 0x7fe7e00ad970 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] conn(0x7fe7e00ad970 msgr2=0x7fe7e00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] conn(0x7fe7e00ad970 0x7fe7e00afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7e00a2cb0 msgr2=0x7fe7e00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7e00a2cb0 0x7fe7e00a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400b5c0 tx=0x7fe7e4005960 comp rx=0 tx=0).stop 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0101be0 msgr2=0x7fe7f019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 --2- 172.21.15.64:0/3285570620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7f0101be0 0x7fe7f019db40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00fb60 tx=0x7fe7dc00fe70 comp rx=0 tx=0).stop 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f5090700 1 -- 172.21.15.64:0/3285570620 reap_dead start 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 shutdown_connections 2024-05-09T09:07:32.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 >> 172.21.15.64:0/3285570620 conn(0x7fe7f00781d0 msgr2=0x7fe7f0100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:32.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 shutdown_connections 2024-05-09T09:07:32.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:32.330+0000 7fe7f6af3700 1 -- 172.21.15.64:0/3285570620 wait complete. 2024-05-09T09:07:32.510 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:32 smithi064 ceph-mon[30149]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-09T09:07:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:32 smithi139 ceph-mon[33599]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-09T09:07:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:32 smithi179 ceph-mon[34086]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 -- 172.21.15.64:0/2398681980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c107280 msgr2=0x7f299c107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 --2- 172.21.15.64:0/2398681980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c107280 0x7f299c107660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f298c009990 tx=0x7f298c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 -- 172.21.15.64:0/2398681980 shutdown_connections 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 --2- 172.21.15.64:0/2398681980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299c101bd0 0x7f299c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 --2- 172.21.15.64:0/2398681980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c107280 0x7f299c107660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 --2- 172.21.15.64:0/2398681980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f299c101210 0x7f299c101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 -- 172.21.15.64:0/2398681980 >> 172.21.15.64:0/2398681980 conn(0x7f299c078190 msgr2=0x7f299c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 -- 172.21.15.64:0/2398681980 shutdown_connections 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.071+0000 7f29a0c47700 1 -- 172.21.15.64:0/2398681980 wait complete. 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 Processor -- start 2024-05-09T09:07:34.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 -- start start 2024-05-09T09:07:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f299c101210 0x7f299c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101bd0 0x7f299c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299c107280 0x7f299c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1124f0 con 0x7f299c101bd0 2024-05-09T09:07:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f299c112370 con 0x7f299c107280 2024-05-09T09:07:34.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f29a0c47700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f299c112670 con 0x7f299c101210 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101bd0 0x7f299c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f2999d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101bd0 0x7f299c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33394/0 (socket says 172.21.15.64:33394) 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f2999d9b700 1 -- 172.21.15.64:0/1734497454 learned_addr learned my addr 172.21.15.64:0/1734497454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.072+0000 7f299a59c700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f299c101210 0x7f299c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f299ad9d700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299c107280 0x7f299c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f2999d9b700 1 -- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f299c101210 msgr2=0x7f299c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f2999d9b700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f299c101210 0x7f299c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f2999d9b700 1 -- 172.21.15.64:0/1734497454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299c107280 msgr2=0x7f299c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f2999d9b700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299c107280 0x7f299c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f2999d9b700 1 -- 172.21.15.64:0/1734497454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c198ff0 con 0x7f299c101bd0 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f299a59c700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f299c101210 0x7f299c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f2999d9b700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101bd0 0x7f299c19ebd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f299000d980 tx=0x7f299000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990017070 con 0x7f299c101bd0 2024-05-09T09:07:34.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2990005020 con 0x7f299c101bd0 2024-05-09T09:07:34.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c1992e0 con 0x7f299c101bd0 2024-05-09T09:07:34.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299000fa90 con 0x7f299c101bd0 2024-05-09T09:07:34.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.073+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1a54f0 con 0x7f299c101bd0 2024-05-09T09:07:34.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.074+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f299c062ab0 con 0x7f299c101bd0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.076+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f29900051d0 con 0x7f299c101bd0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.076+0000 7f298b7fe700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f29840a2d30 0x7f29840a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.076+0000 7f299a59c700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f29840a2d30 0x7f29840a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.076+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f29900cf780 con 0x7f299c101bd0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.077+0000 7f298b7fe700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] conn(0x7f29840ad9f0 0x7f29840afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.077+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 --> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f299001de70 con 0x7f29840ad9f0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.077+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f29900cfbd0 con 0x7f299c101bd0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.077+0000 7f299a59c700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f29840a2d30 0x7f29840a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f298c00b5c0 tx=0x7f298c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.077+0000 7f299ad9d700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] conn(0x7f29840ad9f0 0x7f29840afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:34.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.077+0000 7f299ad9d700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] conn(0x7f29840ad9f0 0x7f29840afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:34.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.078+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== osd.5 v2:172.21.15.179:6808/2037999484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f299001de70 con 0x7f29840ad9f0 2024-05-09T09:07:34.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.095+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 --> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f299c062f30 con 0x7f29840ad9f0 2024-05-09T09:07:34.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.095+0000 7f298b7fe700 1 -- 172.21.15.64:0/1734497454 <== osd.5 v2:172.21.15.179:6808/2037999484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f299c062f30 con 0x7f29840ad9f0 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] conn(0x7f29840ad9f0 msgr2=0x7f29840afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] conn(0x7f29840ad9f0 0x7f29840afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f29840a2d30 msgr2=0x7f29840a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f29840a2d30 0x7f29840a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f298c00b5c0 tx=0x7f298c005960 comp rx=0 tx=0).stop 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101bd0 msgr2=0x7f299c19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 --2- 172.21.15.64:0/1734497454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101bd0 0x7f299c19ebd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f299000d980 tx=0x7f299000dc90 comp rx=0 tx=0).stop 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f299ad9d700 1 -- 172.21.15.64:0/1734497454 reap_dead start 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 shutdown_connections 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 >> 172.21.15.64:0/1734497454 conn(0x7f299c078190 msgr2=0x7f299c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 shutdown_connections 2024-05-09T09:07:34.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:34.096+0000 7f29a0c47700 1 -- 172.21.15.64:0/1734497454 wait complete. 2024-05-09T09:07:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:34 smithi139 ceph-mon[33599]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T09:07:34.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:34 smithi064 ceph-mon[30149]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T09:07:34.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:34 smithi179 ceph-mon[34086]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T09:07:35.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T09:07:35.388 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-09T09:07:35.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-09T09:07:35.397 INFO:teuthology.orchestra.run.smithi064.stdout:167503724600 2024-05-09T09:07:35.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T09:07:35.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-09T09:07:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-09T09:07:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T09:07:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-09T09:07:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-09T09:07:35.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-09T09:07:36.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-09T09:07:36.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:36 smithi064 ceph-mon[30149]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T09:07:36.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-09T09:07:36.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-09T09:07:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:36 smithi139 ceph-mon[33599]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-09T09:07:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-09T09:07:36.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-09T09:07:36.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:36 smithi179 ceph-mon[34086]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T09:07:37.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:37 smithi064 ceph-mon[30149]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T09:07:37.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:37 smithi139 ceph-mon[33599]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T09:07:37.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:37 smithi179 ceph-mon[34086]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T09:07:38.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:38 smithi139 ceph-mon[33599]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-09T09:07:38.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:38 smithi064 ceph-mon[30149]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-09T09:07:38.681 INFO:teuthology.orchestra.run.smithi064.stdout:292057776163 2024-05-09T09:07:38.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T09:07:38.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:38 smithi179 ceph-mon[34086]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-09T09:07:39.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 -- 172.21.15.64:0/10735153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06aa90 msgr2=0x7fb9ac06aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 --2- 172.21.15.64:0/10735153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06aa90 0x7fb9ac06aef0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009990 tx=0x7fb99c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 -- 172.21.15.64:0/10735153 shutdown_connections 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 --2- 172.21.15.64:0/10735153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac06b430 0x7fb9ac10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 --2- 172.21.15.64:0/10735153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ac1083a0 0x7fb9ac108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 --2- 172.21.15.64:0/10735153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06aa90 0x7fb9ac06aef0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 -- 172.21.15.64:0/10735153 >> 172.21.15.64:0/10735153 conn(0x7fb9ac077290 msgr2=0x7fb9ac0776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 -- 172.21.15.64:0/10735153 shutdown_connections 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.274+0000 7fb9b440f700 1 -- 172.21.15.64:0/10735153 wait complete. 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 Processor -- start 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 -- start start 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ac06aa90 0x7fb9ac101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06b430 0x7fb9ac101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac1083a0 0x7fb9ac101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac1124d0 con 0x7fb9ac06b430 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac112350 con 0x7fb9ac06aa90 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b440f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac112650 con 0x7fb9ac1083a0 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b19aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06b430 0x7fb9ac101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b19aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06b430 0x7fb9ac101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59534/0 (socket says 172.21.15.64:59534) 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b19aa700 1 -- 172.21.15.64:0/4119966258 learned_addr learned my addr 172.21.15.64:0/4119966258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.275+0000 7fb9b21ab700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ac06aa90 0x7fb9ac101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b29ac700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac1083a0 0x7fb9ac101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b19aa700 1 -- 172.21.15.64:0/4119966258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac1083a0 msgr2=0x7fb9ac101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b19aa700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac1083a0 0x7fb9ac101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b19aa700 1 -- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ac06aa90 msgr2=0x7fb9ac101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b19aa700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ac06aa90 0x7fb9ac101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b19aa700 1 -- 172.21.15.64:0/4119966258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac1a6f10 con 0x7fb9ac06b430 2024-05-09T09:07:39.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b21ab700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ac06aa90 0x7fb9ac101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:39.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b19aa700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06b430 0x7fb9ac101860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0057c0 tx=0x7fb99c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:39.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c026070 con 0x7fb9ac06b430 2024-05-09T09:07:39.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb99c00fbb0 con 0x7fb9ac06b430 2024-05-09T09:07:39.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac1a71a0 con 0x7fb9ac06b430 2024-05-09T09:07:39.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0173f0 con 0x7fb9ac06b430 2024-05-09T09:07:39.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.276+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac1a7680 con 0x7fb9ac06b430 2024-05-09T09:07:39.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.277+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9ac062a30 con 0x7fb9ac06b430 2024-05-09T09:07:39.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.278+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb99c029070 con 0x7fb9ac06b430 2024-05-09T09:07:39.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9a37fe700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9980a2cb0 0x7fb9980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9b21ab700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9980a2cb0 0x7fb9980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb99c0d04d0 con 0x7fb9ac06b430 2024-05-09T09:07:39.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9a37fe700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] conn(0x7fb9980ade10 0x7fb9980b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 --> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb99c01ae50 con 0x7fb9980ade10 2024-05-09T09:07:39.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb99c0d0920 con 0x7fb9ac06b430 2024-05-09T09:07:39.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9b21ab700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9980a2cb0 0x7fb9980a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009c90 tx=0x7fb9a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:39.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.279+0000 7fb9b29ac700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] conn(0x7fb9980ade10 0x7fb9980b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.280+0000 7fb9b29ac700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] conn(0x7fb9980ade10 0x7fb9980b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:39.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.281+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== osd.7 v2:172.21.15.139:6816/3275587574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb99c01ae50 con 0x7fb9980ade10 2024-05-09T09:07:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.297+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 --> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9ac062eb0 con 0x7fb9980ade10 2024-05-09T09:07:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9a37fe700 1 -- 172.21.15.64:0/4119966258 <== osd.7 v2:172.21.15.139:6816/3275587574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9ac062eb0 con 0x7fb9980ade10 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] conn(0x7fb9980ade10 msgr2=0x7fb9980b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] conn(0x7fb9980ade10 0x7fb9980b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9980a2cb0 msgr2=0x7fb9980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9980a2cb0 0x7fb9980a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009c90 tx=0x7fb9a8009260 comp rx=0 tx=0).stop 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06b430 msgr2=0x7fb9ac101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 --2- 172.21.15.64:0/4119966258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac06b430 0x7fb9ac101860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0057c0 tx=0x7fb99c00b730 comp rx=0 tx=0).stop 2024-05-09T09:07:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b29ac700 1 -- 172.21.15.64:0/4119966258 reap_dead start 2024-05-09T09:07:39.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 shutdown_connections 2024-05-09T09:07:39.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 >> 172.21.15.64:0/4119966258 conn(0x7fb9ac077290 msgr2=0x7fb9ac100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:39.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 shutdown_connections 2024-05-09T09:07:39.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.298+0000 7fb9b440f700 1 -- 172.21.15.64:0/4119966258 wait complete. 2024-05-09T09:07:39.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 -- 172.21.15.64:0/3515290310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 msgr2=0x7fed881016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/3515290310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 0x7fed881016a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fed70009990 tx=0x7fed70009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 -- 172.21.15.64:0/3515290310 shutdown_connections 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/3515290310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88101be0 0x7fed8810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/3515290310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88107290 0x7fed88107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/3515290310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 0x7fed881016a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 -- 172.21.15.64:0/3515290310 >> 172.21.15.64:0/3515290310 conn(0x7fed880781d0 msgr2=0x7fed880785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 -- 172.21.15.64:0/3515290310 shutdown_connections 2024-05-09T09:07:39.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.477+0000 7fed8e5b5700 1 -- 172.21.15.64:0/3515290310 wait complete. 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.478+0000 7fed8e5b5700 1 Processor -- start 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.478+0000 7fed8e5b5700 1 -- start start 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8e5b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 0x7fed881998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8e5b5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88101be0 0x7fed88199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8e5b5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107290 0x7fed881a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8e5b5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed881113e0 con 0x7fed88101220 2024-05-09T09:07:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8e5b5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed88111260 con 0x7fed88101be0 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8e5b5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed88111560 con 0x7fed88107290 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88101be0 0x7fed88199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88101be0 0x7fed88199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47450/0 (socket says 172.21.15.64:47450) 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 -- 172.21.15.64:0/15872018 learned_addr learned my addr 172.21.15.64:0/15872018 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8cb52700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107290 0x7fed881a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed87fff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 0x7fed881998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 -- 172.21.15.64:0/15872018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107290 msgr2=0x7fed881a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107290 0x7fed881a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 -- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 msgr2=0x7fed881998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 0x7fed881998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed7ffff700 1 -- 172.21.15.64:0/15872018 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed881a4ea0 con 0x7fed88101be0 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed8cb52700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107290 0x7fed881a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.479+0000 7fed87fff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88101220 0x7fed881998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.480+0000 7fed7ffff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88101be0 0x7fed88199e20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed70000c00 tx=0x7fed70004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:39.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.480+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70027070 con 0x7fed88101be0 2024-05-09T09:07:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.480+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed881a5070 con 0x7fed88101be0 2024-05-09T09:07:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.480+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed881a5520 con 0x7fed88101be0 2024-05-09T09:07:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.480+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed70022470 con 0x7fed88101be0 2024-05-09T09:07:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.480+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70017650 con 0x7fed88101be0 2024-05-09T09:07:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.481+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed68001030 con 0x7fed88101be0 2024-05-09T09:07:39.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.482+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fed7002a070 con 0x7fed88101be0 2024-05-09T09:07:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.483+0000 7fed85ffb700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed740a2d30 0x7fed740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.483+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fed700d09a0 con 0x7fed88101be0 2024-05-09T09:07:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.483+0000 7fed87fff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed740a2d30 0x7fed740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.483+0000 7fed85ffb700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] conn(0x7fed740ade90 0x7fed740b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.484+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 --> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed7000ce90 con 0x7fed740ade90 2024-05-09T09:07:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.484+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fed700d0df0 con 0x7fed88101be0 2024-05-09T09:07:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.484+0000 7fed87fff700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed740a2d30 0x7fed740a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed78009c90 tx=0x7fed78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.484+0000 7fed8cb52700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] conn(0x7fed740ade90 0x7fed740b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.484+0000 7fed8cb52700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] conn(0x7fed740ade90 0x7fed740b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:39.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.485+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== osd.1 v2:172.21.15.139:6800/1721854761 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed7000ce90 con 0x7fed740ade90 2024-05-09T09:07:39.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.508+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 --> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed68002e60 con 0x7fed740ade90 2024-05-09T09:07:39.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.508+0000 7fed85ffb700 1 -- 172.21.15.64:0/15872018 <== osd.1 v2:172.21.15.139:6800/1721854761 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed68002e60 con 0x7fed740ade90 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] conn(0x7fed740ade90 msgr2=0x7fed740b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] conn(0x7fed740ade90 0x7fed740b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed740a2d30 msgr2=0x7fed740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed740a2d30 0x7fed740a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed78009c90 tx=0x7fed78009260 comp rx=0 tx=0).stop 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88101be0 msgr2=0x7fed88199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 --2- 172.21.15.64:0/15872018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88101be0 0x7fed88199e20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed70000c00 tx=0x7fed70004790 comp rx=0 tx=0).stop 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8cb52700 1 -- 172.21.15.64:0/15872018 reap_dead start 2024-05-09T09:07:39.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 shutdown_connections 2024-05-09T09:07:39.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 >> 172.21.15.64:0/15872018 conn(0x7fed880781d0 msgr2=0x7fed88100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:39.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 shutdown_connections 2024-05-09T09:07:39.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:39.509+0000 7fed8e5b5700 1 -- 172.21.15.64:0/15872018 wait complete. 2024-05-09T09:07:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:40 smithi064 ceph-mon[30149]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:40.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:07:40.502 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:40 smithi179 ceph-mon[34086]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:40.502 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:07:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:40 smithi139 ceph-mon[33599]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 -- 172.21.15.64:0/2349947018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102600 msgr2=0x7fed1c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 --2- 172.21.15.64:0/2349947018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102600 0x7fed1c1029e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009990 tx=0x7fed0c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 -- 172.21.15.64:0/2349947018 shutdown_connections 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 --2- 172.21.15.64:0/2349947018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed1c107680 0x7fed1c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 --2- 172.21.15.64:0/2349947018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102600 0x7fed1c1029e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 --2- 172.21.15.64:0/2349947018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed1c102fb0 0x7fed1c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 -- 172.21.15.64:0/2349947018 >> 172.21.15.64:0/2349947018 conn(0x7fed1c079490 msgr2=0x7fed1c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 -- 172.21.15.64:0/2349947018 shutdown_connections 2024-05-09T09:07:40.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 -- 172.21.15.64:0/2349947018 wait complete. 2024-05-09T09:07:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.810+0000 7fed21425700 1 Processor -- start 2024-05-09T09:07:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 -- start start 2024-05-09T09:07:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed1c102600 0x7fed1c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102fb0 0x7fed1c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:40.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed1c107680 0x7fed1c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1124f0 con 0x7fed1c102fb0 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c112370 con 0x7fed1c102600 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed21425700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c112670 con 0x7fed1c107680 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102fb0 0x7fed1c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102fb0 0x7fed1c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59560/0 (socket says 172.21.15.64:59560) 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 -- 172.21.15.64:0/3979966956 learned_addr learned my addr 172.21.15.64:0/3979966956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1affd700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed1c102600 0x7fed1c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1b7fe700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed1c107680 0x7fed1c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 -- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed1c107680 msgr2=0x7fed1c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed1c107680 0x7fed1c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 -- 172.21.15.64:0/3979966956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed1c102600 msgr2=0x7fed1c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed1c102600 0x7fed1c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1a7fc700 1 -- 172.21.15.64:0/3979966956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c198ff0 con 0x7fed1c102fb0 2024-05-09T09:07:40.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.811+0000 7fed1affd700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed1c102600 0x7fed1c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.812+0000 7fed1a7fc700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102fb0 0x7fed1c19ebd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e970 tx=0x7fed1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.812+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10004a30 con 0x7fed1c102fb0 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.812+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed1000cc90 con 0x7fed1c102fb0 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.812+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c1992e0 con 0x7fed1c102fb0 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.812+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed100055e0 con 0x7fed1c102fb0 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.812+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c1a54f0 con 0x7fed1c102fb0 2024-05-09T09:07:40.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.813+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed1c062ab0 con 0x7fed1c102fb0 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.814+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fed1000c7e0 con 0x7fed1c102fb0 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed03fff700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed040a2d30 0x7fed040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed1affd700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed040a2d30 0x7fed040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fed100cfab0 con 0x7fed1c102fb0 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed03fff700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] conn(0x7fed040ade90 0x7fed040b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 --> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed040b0a00 con 0x7fed040ade90 2024-05-09T09:07:40.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fed100cff00 con 0x7fed1c102fb0 2024-05-09T09:07:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed1affd700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed040a2d30 0x7fed040a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0098e0 tx=0x7fed0c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.815+0000 7fed1b7fe700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] conn(0x7fed040ade90 0x7fed040b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.816+0000 7fed1b7fe700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] conn(0x7fed040ade90 0x7fed040b0330 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.817+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== osd.11 v2:172.21.15.179:6824/880358389 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed040b0a00 con 0x7fed040ade90 2024-05-09T09:07:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 --> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed1c062eb0 con 0x7fed040ade90 2024-05-09T09:07:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed03fff700 1 -- 172.21.15.64:0/3979966956 <== osd.11 v2:172.21.15.179:6824/880358389 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed1c062eb0 con 0x7fed040ade90 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] conn(0x7fed040ade90 msgr2=0x7fed040b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] conn(0x7fed040ade90 0x7fed040b0330 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed040a2d30 msgr2=0x7fed040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed040a2d30 0x7fed040a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0098e0 tx=0x7fed0c005960 comp rx=0 tx=0).stop 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102fb0 msgr2=0x7fed1c19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 --2- 172.21.15.64:0/3979966956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed1c102fb0 0x7fed1c19ebd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e970 tx=0x7fed1000ec80 comp rx=0 tx=0).stop 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed1b7fe700 1 -- 172.21.15.64:0/3979966956 reap_dead start 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.834+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 shutdown_connections 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.835+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 >> 172.21.15.64:0/3979966956 conn(0x7fed1c079490 msgr2=0x7fed1c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.835+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 shutdown_connections 2024-05-09T09:07:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:40.835+0000 7fed21425700 1 -- 172.21.15.64:0/3979966956 wait complete. 2024-05-09T09:07:42.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 -- 172.21.15.64:0/2629302783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101be0 msgr2=0x7fbe1410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:42.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/2629302783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101be0 0x7fbe1410e0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009f90 tx=0x7fbe0c00b330 comp rx=0 tx=0).stop 2024-05-09T09:07:42.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 -- 172.21.15.64:0/2629302783 shutdown_connections 2024-05-09T09:07:42.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/2629302783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101be0 0x7fbe1410e0d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/2629302783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe14107290 0x7fbe14107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/2629302783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe14101220 0x7fbe141016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 -- 172.21.15.64:0/2629302783 >> 172.21.15.64:0/2629302783 conn(0x7fbe140781d0 msgr2=0x7fbe140785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 -- 172.21.15.64:0/2629302783 shutdown_connections 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.098+0000 7fbe1b18f700 1 -- 172.21.15.64:0/2629302783 wait complete. 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 Processor -- start 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 -- start start 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101220 0x7fbe141a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe14101be0 0x7fbe141a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe14107290 0x7fbe1410ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe141102a0 con 0x7fbe14101220 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110120 con 0x7fbe14101be0 2024-05-09T09:07:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1b18f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110420 con 0x7fbe14107290 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe18f2b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101220 0x7fbe141a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe18f2b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101220 0x7fbe141a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59588/0 (socket says 172.21.15.64:59588) 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe18f2b700 1 -- 172.21.15.64:0/4264477098 learned_addr learned my addr 172.21.15.64:0/4264477098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe1972c700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe14107290 0x7fbe1410ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.099+0000 7fbe13fff700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe14101be0 0x7fbe141a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe18f2b700 1 -- 172.21.15.64:0/4264477098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe14107290 msgr2=0x7fbe1410ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe18f2b700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe14107290 0x7fbe1410ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe18f2b700 1 -- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe14101be0 msgr2=0x7fbe141a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe18f2b700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe14101be0 0x7fbe141a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe18f2b700 1 -- 172.21.15.64:0/4264477098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1410d560 con 0x7fbe14101220 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe1972c700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe14107290 0x7fbe1410ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe18f2b700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101220 0x7fbe141a4d20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e9a0 tx=0x7fbe0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800ca10 con 0x7fbe14101220 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1410d850 con 0x7fbe14101220 2024-05-09T09:07:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe0800cbb0 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141abcc0 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.100+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08018620 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.101+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe140629f0 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.103+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe08010430 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.103+0000 7fbe11ffb700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe040a2c90 0x7fbe040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbe080cfab0 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe13fff700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe040a2c90 0x7fbe040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe11ffb700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] conn(0x7fbe040addf0 0x7fbe040b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 --> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe040b0960 con 0x7fbe040addf0 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbe080cff00 con 0x7fbe14101220 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe1972c700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] conn(0x7fbe040addf0 0x7fbe040b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.104+0000 7fbe13fff700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe040a2c90 0x7fbe040a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009750 tx=0x7fbe00009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.105+0000 7fbe1972c700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] conn(0x7fbe040addf0 0x7fbe040b0290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.105+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== osd.4 v2:172.21.15.139:6808/2100308120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe040b0960 con 0x7fbe040addf0 2024-05-09T09:07:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.122+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 --> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe14062df0 con 0x7fbe040addf0 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe11ffb700 1 -- 172.21.15.64:0/4264477098 <== osd.4 v2:172.21.15.139:6808/2100308120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe14062df0 con 0x7fbe040addf0 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] conn(0x7fbe040addf0 msgr2=0x7fbe040b0290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] conn(0x7fbe040addf0 0x7fbe040b0290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe040a2c90 msgr2=0x7fbe040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe040a2c90 0x7fbe040a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009750 tx=0x7fbe00009310 comp rx=0 tx=0).stop 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101220 msgr2=0x7fbe141a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 --2- 172.21.15.64:0/4264477098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe14101220 0x7fbe141a4d20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e9a0 tx=0x7fbe0800ecb0 comp rx=0 tx=0).stop 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1972c700 1 -- 172.21.15.64:0/4264477098 reap_dead start 2024-05-09T09:07:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 shutdown_connections 2024-05-09T09:07:42.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 >> 172.21.15.64:0/4264477098 conn(0x7fbe140781d0 msgr2=0x7fbe14102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:42.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.123+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 shutdown_connections 2024-05-09T09:07:42.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.124+0000 7fbe1b18f700 1 -- 172.21.15.64:0/4264477098 wait complete. 2024-05-09T09:07:42.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:42 smithi064 ceph-mon[30149]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:42.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.241 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:42 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:42 smithi179 ceph-mon[34086]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:42 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:42 smithi139 ceph-mon[33599]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:42.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:42 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:07:43.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 -- 172.21.15.64:0/4136069086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e41014b0 msgr2=0x7f61e4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:43.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 --2- 172.21.15.64:0/4136069086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e41014b0 0x7f61e4101890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f61d40099c0 tx=0x7f61d4009cd0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 -- 172.21.15.64:0/4136069086 shutdown_connections 2024-05-09T09:07:43.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 --2- 172.21.15.64:0/4136069086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61e4106540 0x7f61e410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 --2- 172.21.15.64:0/4136069086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e41014b0 0x7f61e4101890 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 --2- 172.21.15.64:0/4136069086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e4101e60 0x7f61e4105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 -- 172.21.15.64:0/4136069086 >> 172.21.15.64:0/4136069086 conn(0x7f61e40781d0 msgr2=0x7f61e40785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 -- 172.21.15.64:0/4136069086 shutdown_connections 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.207+0000 7f61eaa2b700 1 -- 172.21.15.64:0/4136069086 wait complete. 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 Processor -- start 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 -- start start 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61e41014b0 0x7f61e419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e4101e60 0x7f61e419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e4106540 0x7f61e4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4111470 con 0x7f61e4106540 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41112f0 con 0x7f61e41014b0 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61eaa2b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41115f0 con 0x7f61e4101e60 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e4106540 0x7f61e4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e4106540 0x7f61e4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59612/0 (socket says 172.21.15.64:59612) 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 -- 172.21.15.64:0/554612230 learned_addr learned my addr 172.21.15.64:0/554612230 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61dbfff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e4101e60 0x7f61e419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e3fff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61e41014b0 0x7f61e419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 -- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e4101e60 msgr2=0x7f61e419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e4101e60 0x7f61e419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 -- 172.21.15.64:0/554612230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61e41014b0 msgr2=0x7f61e419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61e41014b0 0x7f61e419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e8fc8700 1 -- 172.21.15.64:0/554612230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e4199060 con 0x7f61e4106540 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.208+0000 7f61e3fff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61e41014b0 0x7f61e419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61e8fc8700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e4106540 0x7f61e4198900 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b970 tx=0x7f61dc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc019070 con 0x7f61e4106540 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61eaa2b700 1 -- 172.21.15.64:0/554612230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e4199350 con 0x7f61e4106540 2024-05-09T09:07:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61dc00cde0 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc007370 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61eaa2b700 1 -- 172.21.15.64:0/554612230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a55c0 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.209+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61c8001030 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.211+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f61dc01c070 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e1ffb700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61d00a2cb0 0x7f61d00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e3fff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61d00a2cb0 0x7f61d00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f61dc0cf310 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e1ffb700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] conn(0x7f61d00add80 0x7f61d00b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 --> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61d00b08f0 con 0x7f61d00add80 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f61dc0cf760 con 0x7f61e4106540 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61e3fff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61d00a2cb0 0x7f61d00a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d4005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.212+0000 7f61dbfff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] conn(0x7f61d00add80 0x7f61d00b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.213+0000 7f61dbfff700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] conn(0x7f61d00add80 0x7f61d00b0220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.214+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== osd.8 v2:172.21.15.179:6816/365956094 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61d00b08f0 con 0x7f61d00add80 2024-05-09T09:07:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.230+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 --> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61c8002e60 con 0x7f61d00add80 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.230+0000 7f61e1ffb700 1 -- 172.21.15.64:0/554612230 <== osd.8 v2:172.21.15.179:6816/365956094 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f61c8002e60 con 0x7f61d00add80 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] conn(0x7f61d00add80 msgr2=0x7f61d00b0220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] conn(0x7f61d00add80 0x7f61d00b0220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61d00a2cb0 msgr2=0x7f61d00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61d00a2cb0 0x7f61d00a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d4005ab0 comp rx=0 tx=0).stop 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e4106540 msgr2=0x7f61e4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 --2- 172.21.15.64:0/554612230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61e4106540 0x7f61e4198900 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b970 tx=0x7f61dc00bc80 comp rx=0 tx=0).stop 2024-05-09T09:07:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61e8fc8700 1 -- 172.21.15.64:0/554612230 reap_dead start 2024-05-09T09:07:43.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 shutdown_connections 2024-05-09T09:07:43.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 >> 172.21.15.64:0/554612230 conn(0x7f61e40781d0 msgr2=0x7f61e406ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:43.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 shutdown_connections 2024-05-09T09:07:43.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:42.231+0000 7f61daffd700 1 -- 172.21.15.64:0/554612230 wait complete. 2024-05-09T09:07:43.753 INFO:teuthology.orchestra.run.smithi064.stdout:257698037802 2024-05-09T09:07:43.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T09:07:44.150 INFO:teuthology.orchestra.run.smithi064.stdout:347892351002 2024-05-09T09:07:44.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T09:07:44.375 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:44 smithi064 ceph-mon[30149]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:44.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:44 smithi179 ceph-mon[34086]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:44 smithi139 ceph-mon[33599]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:44.850 INFO:teuthology.orchestra.run.smithi064.stdout:146028888126 2024-05-09T09:07:44.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 -- 172.21.15.64:0/3135947390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c107290 msgr2=0x7fec5c107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 --2- 172.21.15.64:0/3135947390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c107290 0x7fec5c107670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec50009990 tx=0x7fec50009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 -- 172.21.15.64:0/3135947390 shutdown_connections 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 --2- 172.21.15.64:0/3135947390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec5c101be0 0x7fec5c10e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 --2- 172.21.15.64:0/3135947390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c107290 0x7fec5c107670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 --2- 172.21.15.64:0/3135947390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec5c101220 0x7fec5c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 -- 172.21.15.64:0/3135947390 >> 172.21.15.64:0/3135947390 conn(0x7fec5c0781d0 msgr2=0x7fec5c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.036+0000 7fec62b7e700 1 -- 172.21.15.64:0/3135947390 shutdown_connections 2024-05-09T09:07:45.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.037+0000 7fec62b7e700 1 -- 172.21.15.64:0/3135947390 wait complete. 2024-05-09T09:07:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.037+0000 7fec62b7e700 1 Processor -- start 2024-05-09T09:07:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.037+0000 7fec62b7e700 1 -- start start 2024-05-09T09:07:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec62b7e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec5c101220 0x7fec5c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec62b7e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c101be0 0x7fec5c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec62b7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec5c107290 0x7fec5c1a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:45.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec62b7e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c111430 con 0x7fec5c101be0 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec62b7e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1112b0 con 0x7fec5c101220 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec62b7e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1115b0 con 0x7fec5c107290 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c101be0 0x7fec5c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c101be0 0x7fec5c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53558/0 (socket says 172.21.15.64:53558) 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec5bfff700 1 -- 172.21.15.64:0/3593907704 learned_addr learned my addr 172.21.15.64:0/3593907704 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6091a700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec5c101220 0x7fec5c19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6111b700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec5c107290 0x7fec5c1a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6091a700 1 -- 172.21.15.64:0/3593907704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec5c107290 msgr2=0x7fec5c1a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6091a700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec5c107290 0x7fec5c1a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6091a700 1 -- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c101be0 msgr2=0x7fec5c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6091a700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c101be0 0x7fec5c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec6091a700 1 -- 172.21.15.64:0/3593907704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c1a4040 con 0x7fec5c101220 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.038+0000 7fec5bfff700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec5c101be0 0x7fec5c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.039+0000 7fec6091a700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec5c101220 0x7fec5c19f2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec50000c00 tx=0x7fec50005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.039+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50003680 con 0x7fec5c101220 2024-05-09T09:07:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.039+0000 7fec62b7e700 1 -- 172.21.15.64:0/3593907704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c1a42d0 con 0x7fec5c101220 2024-05-09T09:07:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.039+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec50004ce0 con 0x7fec5c101220 2024-05-09T09:07:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.039+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50003cc0 con 0x7fec5c101220 2024-05-09T09:07:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.039+0000 7fec62b7e700 1 -- 172.21.15.64:0/3593907704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c1abbb0 con 0x7fec5c101220 2024-05-09T09:07:45.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.040+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec3c001030 con 0x7fec5c101220 2024-05-09T09:07:45.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.043+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fec50005000 con 0x7fec5c101220 2024-05-09T09:07:45.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.043+0000 7fec59ffb700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec4c0a2db0 0x7fec4c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fec500d0b70 con 0x7fec5c101220 2024-05-09T09:07:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec59ffb700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] conn(0x7fec4c0adf10 0x7fec4c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec5bfff700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec4c0a2db0 0x7fec4c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 --> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec50022e50 con 0x7fec4c0adf10 2024-05-09T09:07:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fec500d0f60 con 0x7fec5c101220 2024-05-09T09:07:45.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec6111b700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] conn(0x7fec4c0adf10 0x7fec4c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:45.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.044+0000 7fec5bfff700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec4c0a2db0 0x7fec4c0a5270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06b0c0 tx=0x7fec4800a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:45.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.045+0000 7fec6111b700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] conn(0x7fec4c0adf10 0x7fec4c0b03b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:45.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.046+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== osd.3 v2:172.21.15.64:6810/3665517430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec50022e50 con 0x7fec4c0adf10 2024-05-09T09:07:45.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.068+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 --> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec3c002e60 con 0x7fec4c0adf10 2024-05-09T09:07:45.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec59ffb700 1 -- 172.21.15.64:0/3593907704 <== osd.3 v2:172.21.15.64:6810/3665517430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec3c002e60 con 0x7fec4c0adf10 2024-05-09T09:07:45.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] conn(0x7fec4c0adf10 msgr2=0x7fec4c0b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:45.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec477fe700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] conn(0x7fec4c0adf10 0x7fec4c0b03b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec4c0a2db0 msgr2=0x7fec4c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec477fe700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec4c0a2db0 0x7fec4c0a5270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06b0c0 tx=0x7fec4800a400 comp rx=0 tx=0).stop 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec5c101220 msgr2=0x7fec5c19f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.069+0000 7fec477fe700 1 --2- 172.21.15.64:0/3593907704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec5c101220 0x7fec5c19f2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec50000c00 tx=0x7fec50005f10 comp rx=0 tx=0).stop 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.070+0000 7fec6111b700 1 -- 172.21.15.64:0/3593907704 reap_dead start 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.070+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 shutdown_connections 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.070+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 >> 172.21.15.64:0/3593907704 conn(0x7fec5c0781d0 msgr2=0x7fec5c06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:45.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.070+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 shutdown_connections 2024-05-09T09:07:45.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:45.070+0000 7fec477fe700 1 -- 172.21.15.64:0/3593907704 wait complete. 2024-05-09T09:07:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:46 smithi179 ceph-mon[34086]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:46.537 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:46 smithi064 ceph-mon[30149]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:46.565 INFO:teuthology.orchestra.run.smithi064.stdout:55834574925 2024-05-09T09:07:46.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T09:07:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:46 smithi139 ceph-mon[33599]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.772+0000 7fa783c9f700 1 -- 172.21.15.64:0/2685282532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c107280 msgr2=0x7fa77c107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.772+0000 7fa783c9f700 1 --2- 172.21.15.64:0/2685282532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c107280 0x7fa77c107660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa778009990 tx=0x7fa778009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 -- 172.21.15.64:0/2685282532 shutdown_connections 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 --2- 172.21.15.64:0/2685282532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c101bd0 0x7fa77c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 --2- 172.21.15.64:0/2685282532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c107280 0x7fa77c107660 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 --2- 172.21.15.64:0/2685282532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77c101210 0x7fa77c101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 -- 172.21.15.64:0/2685282532 >> 172.21.15.64:0/2685282532 conn(0x7fa77c078190 msgr2=0x7fa77c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 -- 172.21.15.64:0/2685282532 shutdown_connections 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 -- 172.21.15.64:0/2685282532 wait complete. 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 Processor -- start 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.773+0000 7fa783c9f700 1 -- start start 2024-05-09T09:07:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa783c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c101210 0x7fa77c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa783c9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77c101bd0 0x7fa77c103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa783c9f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c107280 0x7fa77c103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa783c9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c111420 con 0x7fa77c101210 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa783c9f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c1112a0 con 0x7fa77c107280 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa783c9f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c1115a0 con 0x7fa77c101bd0 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c101210 0x7fa77c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c101210 0x7fa77c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53572/0 (socket says 172.21.15.64:53572) 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 -- 172.21.15.64:0/3242139077 learned_addr learned my addr 172.21.15.64:0/3242139077 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa78123a700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77c101bd0 0x7fa77c103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa78223c700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c107280 0x7fa77c103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 -- 172.21.15.64:0/3242139077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77c101bd0 msgr2=0x7fa77c103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77c101bd0 0x7fa77c103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 -- 172.21.15.64:0/3242139077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c107280 msgr2=0x7fa77c103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa77c107280 0x7fa77c103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.774+0000 7fa781a3b700 1 -- 172.21.15.64:0/3242139077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c1a6f80 con 0x7fa77c101210 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.775+0000 7fa781a3b700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c101210 0x7fa77c102ef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa778005910 tx=0x7fa7780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.775+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778027070 con 0x7fa77c101210 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.775+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c1a7210 con 0x7fa77c101210 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.775+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa778022470 con 0x7fa77c101210 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.775+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778017650 con 0x7fa77c101210 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.775+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c1a76c0 con 0x7fa77c101210 2024-05-09T09:07:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.776+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa77c1062d0 con 0x7fa77c101210 2024-05-09T09:07:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.777+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa77802a070 con 0x7fa77c101210 2024-05-09T09:07:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.778+0000 7fa772ffd700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7680a2cb0 0x7fa7680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.778+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa7780d0840 con 0x7fa77c101210 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.778+0000 7fa78123a700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7680a2cb0 0x7fa7680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.778+0000 7fa772ffd700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] conn(0x7fa7680ade10 0x7fa7680b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.778+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 --> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7680b0980 con 0x7fa7680ade10 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.778+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa7780d0c90 con 0x7fa77c101210 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.779+0000 7fa78223c700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] conn(0x7fa7680ade10 0x7fa7680b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.779+0000 7fa78123a700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7680a2cb0 0x7fa7680a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa77c078790 tx=0x7fa76c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.779+0000 7fa78223c700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] conn(0x7fa7680ade10 0x7fa7680b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.780+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== osd.6 v2:172.21.15.64:6818/784727785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7680b0980 con 0x7fa7680ade10 2024-05-09T09:07:46.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.797+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 --> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa77c102930 con 0x7fa7680ade10 2024-05-09T09:07:47.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa772ffd700 1 -- 172.21.15.64:0/3242139077 <== osd.6 v2:172.21.15.64:6818/784727785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa77c102930 con 0x7fa7680ade10 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] conn(0x7fa7680ade10 msgr2=0x7fa7680b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] conn(0x7fa7680ade10 0x7fa7680b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7680a2cb0 msgr2=0x7fa7680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7680a2cb0 0x7fa7680a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa77c078790 tx=0x7fa76c009260 comp rx=0 tx=0).stop 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c101210 msgr2=0x7fa77c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 --2- 172.21.15.64:0/3242139077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa77c101210 0x7fa77c102ef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa778005910 tx=0x7fa7780047b0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa78223c700 1 -- 172.21.15.64:0/3242139077 reap_dead start 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 shutdown_connections 2024-05-09T09:07:47.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 >> 172.21.15.64:0/3242139077 conn(0x7fa77c078190 msgr2=0x7fa77c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:47.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 shutdown_connections 2024-05-09T09:07:47.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:46.798+0000 7fa783c9f700 1 -- 172.21.15.64:0/3242139077 wait complete. 2024-05-09T09:07:47.737 INFO:teuthology.orchestra.run.smithi064.stdout:201863462964 2024-05-09T09:07:47.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T09:07:47.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 -- 172.21.15.64:0/3407183726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06b1e0 msgr2=0x7f246c10e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 --2- 172.21.15.64:0/3407183726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06b1e0 0x7f246c10e880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f245c009990 tx=0x7f245c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 -- 172.21.15.64:0/3407183726 shutdown_connections 2024-05-09T09:07:47.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 --2- 172.21.15.64:0/3407183726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f246c10edc0 0x7f246c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 --2- 172.21.15.64:0/3407183726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f246c06a8c0 0x7f246c06aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 --2- 172.21.15.64:0/3407183726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06b1e0 0x7f246c10e880 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 -- 172.21.15.64:0/3407183726 >> 172.21.15.64:0/3407183726 conn(0x7f246c078190 msgr2=0x7f246c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:47.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 -- 172.21.15.64:0/3407183726 shutdown_connections 2024-05-09T09:07:47.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.850+0000 7f24743e9700 1 -- 172.21.15.64:0/3407183726 wait complete. 2024-05-09T09:07:47.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 Processor -- start 2024-05-09T09:07:47.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 -- start start 2024-05-09T09:07:47.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06a8c0 0x7f246c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:47.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f246c06b1e0 0x7f246c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:47.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f246c10edc0 0x7f246c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:47.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1146d0 con 0x7f246c06a8c0 2024-05-09T09:07:47.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f246c114550 con 0x7f246c10edc0 2024-05-09T09:07:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f24743e9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f246c114850 con 0x7f246c06b1e0 2024-05-09T09:07:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06a8c0 0x7f246c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06a8c0 0x7f246c1a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53576/0 (socket says 172.21.15.64:53576) 2024-05-09T09:07:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 -- 172.21.15.64:0/4283456672 learned_addr learned my addr 172.21.15.64:0/4283456672 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:47.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472986700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f246c10edc0 0x7f246c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2471984700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f246c06b1e0 0x7f246c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 -- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f246c06b1e0 msgr2=0x7f246c1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f246c06b1e0 0x7f246c1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 -- 172.21.15.64:0/4283456672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f246c10edc0 msgr2=0x7f246c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.851+0000 7f2472185700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f246c10edc0 0x7f246c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f2472185700 1 -- 172.21.15.64:0/4283456672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c19b150 con 0x7f246c06a8c0 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f2472986700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f246c10edc0 0x7f246c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f2472185700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06a8c0 0x7f246c1a0830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f246800d950 tx=0x7f246800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:47.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468016070 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f246800fd40 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c19b440 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246800e3b0 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.852+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c1a7700 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.853+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f246c062ab0 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.854+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2468019070 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.855+0000 7f24637fe700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24580a2cb0 0x7f24580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.855+0000 7f2471984700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24580a2cb0 0x7f24580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.855+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24680cf4b0 con 0x7f246c06a8c0 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.855+0000 7f24637fe700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] conn(0x7f24580ade10 0x7f24580b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.855+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 --> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2468014e80 con 0x7f24580ade10 2024-05-09T09:07:47.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.855+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f24680cf900 con 0x7f246c06a8c0 2024-05-09T09:07:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.856+0000 7f2471984700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24580a2cb0 0x7f24580a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f245c00b2e0 tx=0x7f245c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.856+0000 7f2472986700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] conn(0x7f24580ade10 0x7f24580b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.856+0000 7f2472986700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] conn(0x7f24580ade10 0x7f24580b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:47.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.857+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== osd.2 v2:172.21.15.179:6800/3117011354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2468014e80 con 0x7f24580ade10 2024-05-09T09:07:47.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.874+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 --> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f246c062eb0 con 0x7f24580ade10 2024-05-09T09:07:47.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24637fe700 1 -- 172.21.15.64:0/4283456672 <== osd.2 v2:172.21.15.179:6800/3117011354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f246c062eb0 con 0x7f24580ade10 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] conn(0x7f24580ade10 msgr2=0x7f24580b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24743e9700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] conn(0x7f24580ade10 0x7f24580b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24580a2cb0 msgr2=0x7f24580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24743e9700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24580a2cb0 0x7f24580a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f245c00b2e0 tx=0x7f245c005960 comp rx=0 tx=0).stop 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06a8c0 msgr2=0x7f246c1a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f24743e9700 1 --2- 172.21.15.64:0/4283456672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f246c06a8c0 0x7f246c1a0830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f246800d950 tx=0x7f246800dc60 comp rx=0 tx=0).stop 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.875+0000 7f2472986700 1 -- 172.21.15.64:0/4283456672 reap_dead start 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.876+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 shutdown_connections 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.876+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 >> 172.21.15.64:0/4283456672 conn(0x7f246c078190 msgr2=0x7f246c110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.876+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 shutdown_connections 2024-05-09T09:07:47.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:47.876+0000 7f24743e9700 1 -- 172.21.15.64:0/4283456672 wait complete. 2024-05-09T09:07:48.478 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:48 smithi064 ceph-mon[30149]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:48 smithi179 ceph-mon[34086]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 -- 172.21.15.64:0/1298287426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 msgr2=0x7f7124107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 --2- 172.21.15.64:0/1298287426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124107670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7114009990 tx=0x7f7114009ca0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 -- 172.21.15.64:0/1298287426 shutdown_connections 2024-05-09T09:07:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 --2- 172.21.15.64:0/1298287426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7124101be0 0x7f712410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 --2- 172.21.15.64:0/1298287426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124107670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 --2- 172.21.15.64:0/1298287426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124101220 0x7f71241016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 -- 172.21.15.64:0/1298287426 >> 172.21.15.64:0/1298287426 conn(0x7f71240781d0 msgr2=0x7f71240785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 -- 172.21.15.64:0/1298287426 shutdown_connections 2024-05-09T09:07:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.486+0000 7f7129ab5700 1 -- 172.21.15.64:0/1298287426 wait complete. 2024-05-09T09:07:48.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 Processor -- start 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 -- start start 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124101220 0x7f7124102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7124101be0 0x7f7124103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7124111430 con 0x7f7124107290 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71241112b0 con 0x7f7124101be0 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.487+0000 7f7129ab5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f71241115b0 con 0x7f7124101220 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7122ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7124101be0 0x7f7124103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124103980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53602/0 (socket says 172.21.15.64:53602) 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7122ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7124101be0 0x7f7124103440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51852/0 (socket says 172.21.15.64:51852) 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 -- 172.21.15.64:0/576464362 learned_addr learned my addr 172.21.15.64:0/576464362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f71237fe700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124101220 0x7f7124102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 -- 172.21.15.64:0/576464362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124101220 msgr2=0x7f7124102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124101220 0x7f7124102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 -- 172.21.15.64:0/576464362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7124101be0 msgr2=0x7f7124103440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7124101be0 0x7f7124103440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 -- 172.21.15.64:0/576464362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71241a6fa0 con 0x7f7124107290 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7123fff700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124103980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f711c007ee0 tx=0x7f711c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c00dd20 con 0x7f7124107290 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71241a7290 con 0x7f7124107290 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f711c004020 con 0x7f7124107290 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c015660 con 0x7f7124107290 2024-05-09T09:07:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.488+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241a7820 con 0x7f7124107290 2024-05-09T09:07:48.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.489+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71241062e0 con 0x7f7124107290 2024-05-09T09:07:48.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.491+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f711c0041d0 con 0x7f7124107290 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.491+0000 7f7120ff9700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f710c0a2d30 0x7f710c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.491+0000 7f71237fe700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f710c0a2d30 0x7f710c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f711c0cf8c0 con 0x7f7124107290 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f7120ff9700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] conn(0x7f710c0ade90 0x7f710c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 --> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f711c020e50 con 0x7f710c0ade90 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f711c0cfd10 con 0x7f7124107290 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f7122ffd700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] conn(0x7f710c0ade90 0x7f710c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:48.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f71237fe700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f710c0a2d30 0x7f710c0a51f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7114006010 tx=0x7f711400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.492+0000 7f7122ffd700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] conn(0x7f710c0ade90 0x7f710c0b0330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:48.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.493+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== osd.0 v2:172.21.15.64:6802/2595942736 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f711c020e50 con 0x7f710c0ade90 2024-05-09T09:07:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.509+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 --> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7124102940 con 0x7f710c0ade90 2024-05-09T09:07:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7120ff9700 1 -- 172.21.15.64:0/576464362 <== osd.0 v2:172.21.15.64:6802/2595942736 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7124102940 con 0x7f710c0ade90 2024-05-09T09:07:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] conn(0x7f710c0ade90 msgr2=0x7f710c0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] conn(0x7f710c0ade90 0x7f710c0b0330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f710c0a2d30 msgr2=0x7f710c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f710c0a2d30 0x7f710c0a51f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7114006010 tx=0x7f711400b540 comp rx=0 tx=0).stop 2024-05-09T09:07:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 msgr2=0x7f7124103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 --2- 172.21.15.64:0/576464362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7124107290 0x7f7124103980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f711c007ee0 tx=0x7f711c00d3b0 comp rx=0 tx=0).stop 2024-05-09T09:07:48.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7123fff700 1 -- 172.21.15.64:0/576464362 reap_dead start 2024-05-09T09:07:48.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 shutdown_connections 2024-05-09T09:07:48.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 >> 172.21.15.64:0/576464362 conn(0x7f71240781d0 msgr2=0x7f712406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:48.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 shutdown_connections 2024-05-09T09:07:48.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:48.510+0000 7f7129ab5700 1 -- 172.21.15.64:0/576464362 wait complete. 2024-05-09T09:07:48.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:48 smithi139 ceph-mon[33599]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T09:07:49.137 INFO:teuthology.orchestra.run.smithi064.stdout:111669149763 2024-05-09T09:07:49.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 -- 172.21.15.64:0/3563215048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006ad50 msgr2=0x7fe99006b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 --2- 172.21.15.64:0/3563215048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006ad50 0x7fe99006b130 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe9800099c0 tx=0x7fe980009cd0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 -- 172.21.15.64:0/3563215048 shutdown_connections 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 --2- 172.21.15.64:0/3563215048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9901076b0 0x7fe99010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 --2- 172.21.15.64:0/3563215048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006ad50 0x7fe99006b130 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 --2- 172.21.15.64:0/3563215048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99006b700 0x7fe990106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 -- 172.21.15.64:0/3563215048 >> 172.21.15.64:0/3563215048 conn(0x7fe990077130 msgr2=0x7fe990077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:49.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 -- 172.21.15.64:0/3563215048 shutdown_connections 2024-05-09T09:07:49.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.336+0000 7fe994f1f700 1 -- 172.21.15.64:0/3563215048 wait complete. 2024-05-09T09:07:49.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 Processor -- start 2024-05-09T09:07:49.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 -- start start 2024-05-09T09:07:49.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99006ad50 0x7fe99019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:49.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006b700 0x7fe99019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9901076b0 0x7fe99019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901125b0 con 0x7fe99006b700 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe990112430 con 0x7fe9901076b0 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.337+0000 7fe994f1f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe990112730 con 0x7fe99006ad50 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006b700 0x7fe99019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006b700 0x7fe99019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53620/0 (socket says 172.21.15.64:53620) 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 -- 172.21.15.64:0/1751823276 learned_addr learned my addr 172.21.15.64:0/1751823276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98e59c700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99006ad50 0x7fe99019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98ed9d700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9901076b0 0x7fe99019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 -- 172.21.15.64:0/1751823276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99006ad50 msgr2=0x7fe99019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe99006ad50 0x7fe99019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 -- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9901076b0 msgr2=0x7fe99019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9901076b0 0x7fe99019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 -- 172.21.15.64:0/1751823276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019fce0 con 0x7fe99006b700 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe98dd9b700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006b700 0x7fe99019b300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe98400d950 tx=0x7fe98400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984017070 con 0x7fe99006b700 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.338+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe984005020 con 0x7fe99006b700 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.339+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99019ffd0 con 0x7fe99006b700 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.339+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400fa90 con 0x7fe99006b700 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.339+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901a7850 con 0x7fe99006b700 2024-05-09T09:07:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.339+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe990101f20 con 0x7fe99006b700 2024-05-09T09:07:49.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.342+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe98400fc30 con 0x7fe99006b700 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.342+0000 7fe97f7fe700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9780a2cb0 0x7fe9780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.342+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe9840cf290 con 0x7fe99006b700 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.342+0000 7fe98e59c700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9780a2cb0 0x7fe9780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.343+0000 7fe97f7fe700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] conn(0x7fe9780ade10 0x7fe9780b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.343+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 --> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe984026e50 con 0x7fe9780ade10 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.343+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe9840cf6e0 con 0x7fe99006b700 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.343+0000 7fe98ed9d700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] conn(0x7fe9780ade10 0x7fe9780b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.343+0000 7fe98e59c700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9780a2cb0 0x7fe9780a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe980003760 tx=0x7fe980005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:49.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.343+0000 7fe98ed9d700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] conn(0x7fe9780ade10 0x7fe9780b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:49.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.344+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== osd.10 v2:172.21.15.139:6824/4105758039 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe984026e50 con 0x7fe9780ade10 2024-05-09T09:07:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.362+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 --> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe990103ab0 con 0x7fe9780ade10 2024-05-09T09:07:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.362+0000 7fe97f7fe700 1 -- 172.21.15.64:0/1751823276 <== osd.10 v2:172.21.15.139:6824/4105758039 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe990103ab0 con 0x7fe9780ade10 2024-05-09T09:07:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] conn(0x7fe9780ade10 msgr2=0x7fe9780b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] conn(0x7fe9780ade10 0x7fe9780b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9780a2cb0 msgr2=0x7fe9780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9780a2cb0 0x7fe9780a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe980003760 tx=0x7fe980005fb0 comp rx=0 tx=0).stop 2024-05-09T09:07:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006b700 msgr2=0x7fe99019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 --2- 172.21.15.64:0/1751823276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe99006b700 0x7fe99019b300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe98400d950 tx=0x7fe98400dc60 comp rx=0 tx=0).stop 2024-05-09T09:07:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe98ed9d700 1 -- 172.21.15.64:0/1751823276 reap_dead start 2024-05-09T09:07:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 shutdown_connections 2024-05-09T09:07:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 >> 172.21.15.64:0/1751823276 conn(0x7fe990077130 msgr2=0x7fe990108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 shutdown_connections 2024-05-09T09:07:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:49.363+0000 7fe994f1f700 1 -- 172.21.15.64:0/1751823276 wait complete. 2024-05-09T09:07:49.809 INFO:teuthology.orchestra.run.smithi064.stdout:223338299437 2024-05-09T09:07:49.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T09:07:50.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:50 smithi064 ceph-mon[30149]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T09:07:50.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:50 smithi179 ceph-mon[34086]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T09:07:50.486 INFO:teuthology.orchestra.run.smithi064.stdout:313532612641 2024-05-09T09:07:50.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T09:07:50.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:50 smithi139 ceph-mon[33599]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T09:07:51.276 INFO:teuthology.orchestra.run.smithi064.stdout:77309411402 2024-05-09T09:07:51.276 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T09:07:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:34359738451 2024-05-09T09:07:51.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T09:07:51.793 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:51.799 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:51.829 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:51.830 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:51.836 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:52.143 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:52.148 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:52.148 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:07:52.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:52 smithi064 ceph-mon[30149]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T09:07:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:52 smithi179 ceph-mon[34086]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T09:07:52.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:52 smithi139 ceph-mon[33599]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T09:07:54.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:54 smithi064 ceph-mon[30149]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:07:54.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:54 smithi179 ceph-mon[34086]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:07:54.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:54 smithi139 ceph-mon[33599]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:07:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:56 smithi179 ceph-mon[34086]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:07:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:56 smithi139 ceph-mon[33599]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:07:56.648 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:56 smithi064 ceph-mon[30149]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:07:58.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:07:58 smithi179 ceph-mon[34086]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:07:58.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:07:58 smithi139 ceph-mon[33599]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:07:58.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:07:58 smithi064 ceph-mon[30149]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:07:59.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 -- 172.21.15.64:0/2827450159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c106580 msgr2=0x7fb29c10e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 --2- 172.21.15.64:0/2827450159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c106580 0x7fb29c10e150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b0d0 tx=0x7fb29400b3e0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 -- 172.21.15.64:0/2827450159 shutdown_connections 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 --2- 172.21.15.64:0/2827450159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c106580 0x7fb29c10e150 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 --2- 172.21.15.64:0/2827450159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 --2- 172.21.15.64:0/2827450159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb29c101e50 0x7fb29c105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 -- 172.21.15.64:0/2827450159 >> 172.21.15.64:0/2827450159 conn(0x7fb29c0782c0 msgr2=0x7fb29c0786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 -- 172.21.15.64:0/2827450159 shutdown_connections 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 -- 172.21.15.64:0/2827450159 wait complete. 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 Processor -- start 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.068+0000 7fb2a2993700 1 -- start start 2024-05-09T09:07:59.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a2993700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a2993700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c101e50 0x7fb29c103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a2993700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb29c106580 0x7fb29c1039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a2993700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c111440 con 0x7fb29c101e50 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a2993700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1112c0 con 0x7fb29c106580 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a2993700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1115c0 con 0x7fb29c1014a0 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36662/0 (socket says 172.21.15.64:36662) 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb29bfff700 1 -- 172.21.15.64:0/1584460183 learned_addr learned my addr 172.21.15.64:0/1584460183 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb2a0f30700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb29c106580 0x7fb29c1039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.069+0000 7fb29b7fe700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c101e50 0x7fb29c103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29bfff700 1 -- 172.21.15.64:0/1584460183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb29c106580 msgr2=0x7fb29c1039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29bfff700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb29c106580 0x7fb29c1039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29bfff700 1 -- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c101e50 msgr2=0x7fb29c103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:59.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29bfff700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c101e50 0x7fb29c103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29bfff700 1 -- 172.21.15.64:0/1584460183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c1a6ea0 con 0x7fb29c1014a0 2024-05-09T09:07:59.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29b7fe700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c101e50 0x7fb29c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:07:59.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb29bfff700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c102f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00d980 tx=0x7fb28c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c017070 con 0x7fb29c1014a0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c1a7190 con 0x7fb29c1014a0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28c005020 con 0x7fb29c1014a0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c009b50 con 0x7fb29c1014a0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.070+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1a76d0 con 0x7fb29c1014a0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.071+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c069010 con 0x7fb29c1014a0 2024-05-09T09:07:59.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.073+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb28c009cf0 con 0x7fb29c1014a0 2024-05-09T09:07:59.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.073+0000 7fb2997fa700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2840a2cb0 0x7fb2840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:07:59.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.074+0000 7fb29b7fe700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2840a2cb0 0x7fb2840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:07:59.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.074+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb28c0cf760 con 0x7fb29c1014a0 2024-05-09T09:07:59.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.074+0000 7fb29b7fe700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2840a2cb0 0x7fb2840a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb290005fd0 tx=0x7fb290005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:07:59.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.077+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb28c020020 con 0x7fb29c1014a0 2024-05-09T09:07:59.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.290+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb29c02db80 con 0x7fb29c1014a0 2024-05-09T09:07:59.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.291+0000 7fb2997fa700 1 -- 172.21.15.64:0/1584460183 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb28c09c200 con 0x7fb29c1014a0 2024-05-09T09:07:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:167503724605 2024-05-09T09:07:59.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2840a2cb0 msgr2=0x7fb2840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:59.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2840a2cb0 0x7fb2840a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb290005fd0 tx=0x7fb290005d40 comp rx=0 tx=0).stop 2024-05-09T09:07:59.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 msgr2=0x7fb29c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c102f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00d980 tx=0x7fb28c00dc90 comp rx=0 tx=0).stop 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 shutdown_connections 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb29c106580 0x7fb29c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2840a2cb0 0x7fb2840a5170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb29c1014a0 0x7fb29c102f40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 --2- 172.21.15.64:0/1584460183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb29c101e50 0x7fb29c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 >> 172.21.15.64:0/1584460183 conn(0x7fb29c0782c0 msgr2=0x7fb29c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 shutdown_connections 2024-05-09T09:07:59.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:07:59.293+0000 7fb2a2993700 1 -- 172.21.15.64:0/1584460183 wait complete. 2024-05-09T09:08:00.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:00 smithi179 ceph-mon[34086]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T09:08:00.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:00 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1584460183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T09:08:00.529 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:00 smithi064 ceph-mon[30149]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T09:08:00.529 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:00 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1584460183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T09:08:00.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:00 smithi139 ceph-mon[33599]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T09:08:00.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:00 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1584460183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T09:08:00.974 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:02.312 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:02 smithi064 ceph-mon[30149]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:02.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:02 smithi179 ceph-mon[34086]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:02.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:02 smithi139 ceph-mon[33599]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:02.729 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724600 got 167503724605 for osd.5 2024-05-09T09:08:02.729 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:03.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:03 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/872608611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T09:08:03.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:03 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3966147344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T09:08:03.572 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:03 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/872608611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T09:08:03.572 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:03 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3966147344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T09:08:03.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:03 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/872608611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T09:08:03.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:03 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3966147344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T09:08:04.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:04 smithi179 ceph-mon[34086]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-09T09:08:04.544 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:04 smithi064 ceph-mon[30149]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-09T09:08:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:04 smithi139 ceph-mon[33599]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 -- 172.21.15.64:0/1191732428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8074e70 msgr2=0x7f54e810e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 --2- 172.21.15.64:0/1191732428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8074e70 0x7f54e810e810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009990 tx=0x7f54d8009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 -- 172.21.15.64:0/1191732428 shutdown_connections 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 --2- 172.21.15.64:0/1191732428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e810ed50 0x7f54e8111140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 --2- 172.21.15.64:0/1191732428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074550 0x7f54e8074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 --2- 172.21.15.64:0/1191732428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8074e70 0x7f54e810e810 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 -- 172.21.15.64:0/1191732428 >> 172.21.15.64:0/1191732428 conn(0x7f54e80794d0 msgr2=0x7f54e80798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 -- 172.21.15.64:0/1191732428 shutdown_connections 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.320+0000 7f54f029b700 1 -- 172.21.15.64:0/1191732428 wait complete. 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 Processor -- start 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 -- start start 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e8074550 0x7f54e81a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074e70 0x7f54e81a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 0x7f54e819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81147a0 con 0x7f54e810ed50 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8114620 con 0x7f54e8074e70 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54f029b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8114920 con 0x7f54e8074550 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 0x7f54e819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 0x7f54e819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50972/0 (socket says 172.21.15.64:50972) 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ed836700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074e70 0x7f54e81a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 -- 172.21.15.64:0/872608611 learned_addr learned my addr 172.21.15.64:0/872608611 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:04.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee037700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e8074550 0x7f54e81a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:04.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 -- 172.21.15.64:0/872608611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e8074550 msgr2=0x7f54e81a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:04.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e8074550 0x7f54e81a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.650 INFO:teuthology.orchestra.run.smithi064.stdout:347892351006 2024-05-09T09:08:04.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 -- 172.21.15.64:0/872608611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074e70 msgr2=0x7f54e81a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074e70 0x7f54e81a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee838700 1 -- 172.21.15.64:0/872608611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e819b240 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ee037700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e8074550 0x7f54e81a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.321+0000 7f54ed836700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074e70 0x7f54e81a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.322+0000 7f54ee838700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 0x7f54e819aae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54e0007ee0 tx=0x7f54e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.322+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e000ddd0 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.322+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e0004020 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.322+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0015660 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.322+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e819b530 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.322+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a77f0 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.323+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e8069010 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.324+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f54e00041d0 con 0x7f54e810ed50 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.325+0000 7f54df7fe700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54d40a2cb0 0x7f54d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:04.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.325+0000 7f54ee037700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54d40a2cb0 0x7f54d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.325+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f54e00cef20 con 0x7f54e810ed50 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.326+0000 7f54ee037700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54d40a2cb0 0x7f54d40a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005950 tx=0x7f54e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.330+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f54e0010070 con 0x7f54e810ed50 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.539+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f54e802db80 con 0x7f54e810ed50 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.539+0000 7f54df7fe700 1 -- 172.21.15.64:0/872608611 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f54e009b9c0 con 0x7f54e810ed50 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54d40a2cb0 msgr2=0x7f54d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54d40a2cb0 0x7f54d40a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005950 tx=0x7f54e40058e0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 msgr2=0x7f54e819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 0x7f54e819aae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54e0007ee0 tx=0x7f54e000d3b0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 shutdown_connections 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e810ed50 0x7f54e819aae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f54d40a2cb0 0x7f54d40a5170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e8074550 0x7f54e81a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 --2- 172.21.15.64:0/872608611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54e8074e70 0x7f54e81a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:04.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 >> 172.21.15.64:0/872608611 conn(0x7f54e80794d0 msgr2=0x7f54e80ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 shutdown_connections 2024-05-09T09:08:04.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.542+0000 7f54f029b700 1 -- 172.21.15.64:0/872608611 wait complete. 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 -- 172.21.15.64:0/3683422288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101210 msgr2=0x7ff2dc101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/3683422288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101210 0x7ff2dc101690 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009990 tx=0x7ff2d0009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 -- 172.21.15.64:0/3683422288 shutdown_connections 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/3683422288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101bd0 0x7ff2dc10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/3683422288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 0x7ff2dc107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/3683422288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101210 0x7ff2dc101690 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.547+0000 7ff2e32c9700 1 -- 172.21.15.64:0/3683422288 >> 172.21.15.64:0/3683422288 conn(0x7ff2dc078190 msgr2=0x7ff2dc0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.548+0000 7ff2e32c9700 1 -- 172.21.15.64:0/3683422288 shutdown_connections 2024-05-09T09:08:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.548+0000 7ff2e32c9700 1 -- 172.21.15.64:0/3683422288 wait complete. 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.548+0000 7ff2e32c9700 1 Processor -- start 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.548+0000 7ff2e32c9700 1 -- start start 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e32c9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 0x7ff2dc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e32c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101bd0 0x7ff2dc199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e32c9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 0x7ff2dc1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e32c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1113d0 con 0x7ff2dc101bd0 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e32c9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc111250 con 0x7ff2dc107280 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e32c9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc111550 con 0x7ff2dc101210 2024-05-09T09:08:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 0x7ff2dc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 0x7ff2dc1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42874/0 (socket says 172.21.15.64:42874) 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 -- 172.21.15.64:0/795699908 learned_addr learned my addr 172.21.15.64:0/795699908 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e0864700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101bd0 0x7ff2dc199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1866700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 0x7ff2dc1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 -- 172.21.15.64:0/795699908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 msgr2=0x7ff2dc1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 0x7ff2dc1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 -- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101bd0 msgr2=0x7ff2dc199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101bd0 0x7ff2dc199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1065700 1 -- 172.21.15.64:0/795699908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc1a4e70 con 0x7ff2dc101210 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.549+0000 7ff2e1866700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 0x7ff2dc1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.550+0000 7ff2e1065700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 0x7ff2dc1998b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800d980 tx=0x7ff2d800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.550+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8016070 con 0x7ff2dc101210 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.550+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1a5040 con 0x7ff2dc101210 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.550+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d8004830 con 0x7ff2dc101210 2024-05-09T09:08:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.550+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80054b0 con 0x7ff2dc101210 2024-05-09T09:08:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.550+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1a5550 con 0x7ff2dc101210 2024-05-09T09:08:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.551+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc069010 con 0x7ff2dc101210 2024-05-09T09:08:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.553+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff2d80049e0 con 0x7ff2dc101210 2024-05-09T09:08:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.553+0000 7ff2ce7fc700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2cb0 0x7ff2c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.553+0000 7ff2e0864700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2cb0 0x7ff2c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.553+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff2d80cf7c0 con 0x7ff2dc101210 2024-05-09T09:08:05.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.554+0000 7ff2e0864700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2cb0 0x7ff2c80a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000b5c0 tx=0x7ff2d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.557+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2d801f020 con 0x7ff2dc101210 2024-05-09T09:08:05.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.422+0000 7f1fd1705700 1 -- 172.21.15.64:0/1935183524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc102f80 msgr2=0x7f1fcc106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.422+0000 7f1fd1705700 1 --2- 172.21.15.64:0/1935183524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc102f80 0x7f1fcc106fb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009990 tx=0x7f1fc0009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 -- 172.21.15.64:0/1935183524 shutdown_connections 2024-05-09T09:08:05.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 --2- 172.21.15.64:0/1935183524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc107690 0x7f1fcc10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 --2- 172.21.15.64:0/1935183524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fcc1025d0 0x7f1fcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 --2- 172.21.15.64:0/1935183524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc102f80 0x7f1fcc106fb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 -- 172.21.15.64:0/1935183524 >> 172.21.15.64:0/1935183524 conn(0x7f1fcc0794d0 msgr2=0x7f1fcc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 -- 172.21.15.64:0/1935183524 shutdown_connections 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 -- 172.21.15.64:0/1935183524 wait complete. 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 Processor -- start 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.423+0000 7f1fd1705700 1 -- start start 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fd1705700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fcc1025d0 0x7f1fcc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fd1705700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc102f80 0x7f1fcc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fd1705700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 0x7f1fcc1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fd1705700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc112500 con 0x7f1fcc107690 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fd1705700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc112380 con 0x7f1fcc102f80 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fd1705700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc112680 con 0x7f1fcc1025d0 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 0x7f1fcc1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 0x7f1fcc1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50986/0 (socket says 172.21.15.64:50986) 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 -- 172.21.15.64:0/3966147344 learned_addr learned my addr 172.21.15.64:0/3966147344 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcaffd700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fcc1025d0 0x7f1fcc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.650 INFO:teuthology.orchestra.run.smithi064.stdout:146028888130 2024-05-09T09:08:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fca7fc700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc102f80 0x7f1fcc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 -- 172.21.15.64:0/3966147344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fcc1025d0 msgr2=0x7f1fcc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fcc1025d0 0x7f1fcc19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 -- 172.21.15.64:0/3966147344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc102f80 msgr2=0x7f1fcc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc102f80 0x7f1fcc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 -- 172.21.15.64:0/3966147344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc198fa0 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fca7fc700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc102f80 0x7f1fcc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.424+0000 7f1fcb7fe700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 0x7f1fcc1987c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400cca0 tx=0x7f1fc400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.425+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4016020 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.425+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc199290 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.425+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fc400aba0 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.425+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40115e0 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.425+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc1a54a0 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.426+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc103a90 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.427+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fc4019020 con 0x7f1fcc107690 2024-05-09T09:08:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.428+0000 7f1fb3fff700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fb40a2cb0 0x7f1fb40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.428+0000 7f1fcaffd700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fb40a2cb0 0x7f1fb40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.428+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1fc40cf7b0 con 0x7f1fcc107690 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.428+0000 7f1fcaffd700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fb40a2cb0 0x7f1fb40a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc004630 tx=0x7f1fbc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.432+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1fc4020020 con 0x7f1fcc107690 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.641+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1fcc02db80 con 0x7f1fcc107690 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.642+0000 7f1fb3fff700 1 -- 172.21.15.64:0/3966147344 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1fc409c360 con 0x7f1fcc107690 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fb40a2cb0 msgr2=0x7f1fb40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fb40a2cb0 0x7f1fb40a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc004630 tx=0x7f1fbc00b3c0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 msgr2=0x7f1fcc1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 0x7f1fcc1987c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400cca0 tx=0x7f1fc400a060 comp rx=0 tx=0).stop 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 shutdown_connections 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fcc107690 0x7f1fcc1987c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fb40a2cb0 0x7f1fb40a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fcc1025d0 0x7f1fcc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 --2- 172.21.15.64:0/3966147344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fcc102f80 0x7f1fcc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.644+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 >> 172.21.15.64:0/3966147344 conn(0x7f1fcc0794d0 msgr2=0x7f1fcc101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.645+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 shutdown_connections 2024-05-09T09:08:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:02.645+0000 7f1fd1705700 1 -- 172.21.15.64:0/3966147344 wait complete. 2024-05-09T09:08:05.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.772+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff2dc02db80 con 0x7ff2dc101210 2024-05-09T09:08:05.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.773+0000 7ff2ce7fc700 1 -- 172.21.15.64:0/795699908 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff2d80193b0 con 0x7ff2dc101210 2024-05-09T09:08:05.772 INFO:teuthology.orchestra.run.smithi064.stdout:257698037807 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2cb0 msgr2=0x7ff2c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2cb0 0x7ff2c80a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000b5c0 tx=0x7ff2d0005960 comp rx=0 tx=0).stop 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 msgr2=0x7ff2dc1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 0x7ff2dc1998b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800d980 tx=0x7ff2d800dc90 comp rx=0 tx=0).stop 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 shutdown_connections 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2dc107280 0x7ff2dc1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2cb0 0x7ff2c80a5170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc101210 0x7ff2dc1998b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 --2- 172.21.15.64:0/795699908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2dc101bd0 0x7ff2dc199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:05.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 >> 172.21.15.64:0/795699908 conn(0x7ff2dc078190 msgr2=0x7ff2dc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:05.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 shutdown_connections 2024-05-09T09:08:05.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.775+0000 7ff2e32c9700 1 -- 172.21.15.64:0/795699908 wait complete. 2024-05-09T09:08:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:06 smithi179 ceph-mon[34086]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2682852722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T09:08:06.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/795699908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T09:08:06.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:06 smithi064 ceph-mon[30149]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:06.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2682852722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T09:08:06.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/795699908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T09:08:06.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351002 got 347892351006 for osd.11 2024-05-09T09:08:06.512 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:06 smithi139 ceph-mon[33599]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2682852722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T09:08:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/795699908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 -- 172.21.15.64:0/1314379928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541083e0 msgr2=0x7f95541087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 --2- 172.21.15.64:0/1314379928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541083e0 0x7f95541087c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9550009990 tx=0x7f9550009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 -- 172.21.15.64:0/1314379928 shutdown_connections 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 --2- 172.21.15.64:0/1314379928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 0x7f955410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 --2- 172.21.15.64:0/1314379928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541083e0 0x7f95541087c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 --2- 172.21.15.64:0/1314379928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541023a0 0x7f9554102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 -- 172.21.15.64:0/1314379928 >> 172.21.15.64:0/1314379928 conn(0x7f95540793f0 msgr2=0x7f9554079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 -- 172.21.15.64:0/1314379928 shutdown_connections 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.022+0000 7f955c0d6700 1 -- 172.21.15.64:0/1314379928 wait complete. 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 Processor -- start 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 -- start start 2024-05-09T09:08:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 0x7f9554074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 0x7f9554074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541083e0 0x7f95540750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f95541124f0 con 0x7f95541023a0 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9554112370 con 0x7f95541083e0 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955c0d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9554112670 con 0x7f9554102d40 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f9559e72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 0x7f9554074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f9559e72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 0x7f9554074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41062/0 (socket says 172.21.15.64:41062) 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f9559e72700 1 -- 172.21.15.64:0/344145828 learned_addr learned my addr 172.21.15.64:0/344145828 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.023+0000 7f955a673700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541083e0 0x7f95540750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559671700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 0x7f9554074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559e72700 1 -- 172.21.15.64:0/344145828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 msgr2=0x7f9554074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559e72700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 0x7f9554074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559e72700 1 -- 172.21.15.64:0/344145828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541083e0 msgr2=0x7f95540750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559e72700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541083e0 0x7f95540750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559e72700 1 -- 172.21.15.64:0/344145828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95541a6f40 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f955a673700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541083e0 0x7f95540750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559671700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 0x7f9554074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9559e72700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 0x7f9554074620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9550000c00 tx=0x7f9550003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550027070 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955000bdc0 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95541a71d0 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955000f460 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.024+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95541a7620 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.025+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9554069010 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.026+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f955002a070 con 0x7f95541023a0 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.027+0000 7f9546ffd700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95400a2cb0 0x7f95400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:07.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.027+0000 7f9559671700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95400a2cb0 0x7f95400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.027+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f95500d0450 con 0x7f95541023a0 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.028+0000 7f9559671700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95400a2cb0 0x7f95400a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9548005d90 tx=0x7f9548005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.032+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9550013070 con 0x7f95541023a0 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.243+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f955402dd40 con 0x7f95541023a0 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.243+0000 7f9546ffd700 1 -- 172.21.15.64:0/344145828 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f955009cf80 con 0x7f95541023a0 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stdout:55834574930 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95400a2cb0 msgr2=0x7f95400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95400a2cb0 0x7f95400a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9548005d90 tx=0x7f9548005d00 comp rx=0 tx=0).stop 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 msgr2=0x7f9554074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 0x7f9554074620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9550000c00 tx=0x7f9550003b30 comp rx=0 tx=0).stop 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 shutdown_connections 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95541083e0 0x7f95540750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541023a0 0x7f9554074620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95400a2cb0 0x7f95400a5170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 --2- 172.21.15.64:0/344145828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554102d40 0x7f9554074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 >> 172.21.15.64:0/344145828 conn(0x7f95540793f0 msgr2=0x7f9554101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 shutdown_connections 2024-05-09T09:08:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:07.246+0000 7f955c0d6700 1 -- 172.21.15.64:0/344145828 wait complete. 2024-05-09T09:08:08.335 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:08 smithi064 ceph-mon[30149]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:08.335 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:08 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/344145828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T09:08:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:08 smithi179 ceph-mon[34086]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:08.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:08 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/344145828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T09:08:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:08 smithi139 ceph-mon[33599]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:08 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/344145828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T09:08:09.007 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037802 got 257698037807 for osd.8 2024-05-09T09:08:09.007 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:09.643 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888126 got 146028888130 for osd.4 2024-05-09T09:08:09.643 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:10.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 -- 172.21.15.64:0/1162229365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff96406b260 msgr2=0x7ff96410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 --2- 172.21.15.64:0/1162229365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff96406b260 0x7ff96410f1d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00b0a0 tx=0x7ff95c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 -- 172.21.15.64:0/1162229365 shutdown_connections 2024-05-09T09:08:10.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 --2- 172.21.15.64:0/1162229365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff96406b260 0x7ff96410f1d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 --2- 172.21.15.64:0/1162229365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff964108390 0x7ff964108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 --2- 172.21.15.64:0/1162229365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96406a8c0 0x7ff96406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 -- 172.21.15.64:0/1162229365 >> 172.21.15.64:0/1162229365 conn(0x7ff964078190 msgr2=0x7ff9640785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 -- 172.21.15.64:0/1162229365 shutdown_connections 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.430+0000 7ff96a17b700 1 -- 172.21.15.64:0/1162229365 wait complete. 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 Processor -- start 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 -- start start 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff96406a8c0 0x7ff9641a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96406b260 0x7ff9641a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 0x7ff96419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641124c0 con 0x7ff964108390 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112340 con 0x7ff96406a8c0 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff96a17b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112640 con 0x7ff96406b260 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.431+0000 7ff963fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 0x7ff96419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 0x7ff96419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41034/0 (socket says 172.21.15.64:41034) 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 -- 172.21.15.64:0/2682852722 learned_addr learned my addr 172.21.15.64:0/2682852722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff962ffd700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96406b260 0x7ff9641a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stdout:292057776169 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff9637fe700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff96406a8c0 0x7ff9641a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 -- 172.21.15.64:0/2682852722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96406b260 msgr2=0x7ff9641a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96406b260 0x7ff9641a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 -- 172.21.15.64:0/2682852722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff96406a8c0 msgr2=0x7ff9641a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff96406a8c0 0x7ff9641a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 -- 172.21.15.64:0/2682852722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96419b140 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff963fff700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 0x7ff96419a9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00b780 tx=0x7ff95c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c021070 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff95c007e30 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96419b3d0 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004700 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.432+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641a76f0 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.434+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff964069010 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.435+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff95c007980 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.435+0000 7ff960ff9700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9500a2cb0 0x7ff9500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.436+0000 7ff9637fe700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9500a2cb0 0x7ff9500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.436+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff95c00f020 con 0x7ff964108390 2024-05-09T09:08:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.436+0000 7ff9637fe700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9500a2cb0 0x7ff9500a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff96419bb20 tx=0x7ff954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.440+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff95c02a020 con 0x7ff964108390 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.674+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff9641a78c0 con 0x7ff964108390 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.675+0000 7ff960ff9700 1 -- 172.21.15.64:0/2682852722 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff95c003c00 con 0x7ff964108390 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9500a2cb0 msgr2=0x7ff9500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9500a2cb0 0x7ff9500a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff96419bb20 tx=0x7ff954009260 comp rx=0 tx=0).stop 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 msgr2=0x7ff96419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 0x7ff96419a9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00b780 tx=0x7ff95c004d00 comp rx=0 tx=0).stop 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 shutdown_connections 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff964108390 0x7ff96419a9e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff96406a8c0 0x7ff9641a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9500a2cb0 0x7ff9500a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 --2- 172.21.15.64:0/2682852722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff96406b260 0x7ff9641a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 >> 172.21.15.64:0/2682852722 conn(0x7ff964078190 msgr2=0x7ff96410d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 shutdown_connections 2024-05-09T09:08:10.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:05.678+0000 7ff96a17b700 1 -- 172.21.15.64:0/2682852722 wait complete. 2024-05-09T09:08:10.349 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:10 smithi064 ceph-mon[30149]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:10.349 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:10 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2202986926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T09:08:10.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 -- 172.21.15.64:0/2554266853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd81083a0 msgr2=0x7efcd8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 --2- 172.21.15.64:0/2554266853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd81083a0 0x7efcd8108780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009990 tx=0x7efcc8009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 -- 172.21.15.64:0/2554266853 shutdown_connections 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 --2- 172.21.15.64:0/2554266853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcd806b3b0 0x7efcd810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 --2- 172.21.15.64:0/2554266853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd81083a0 0x7efcd8108780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 --2- 172.21.15.64:0/2554266853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd806a9f0 0x7efcd806ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.332+0000 7efcdda22700 1 -- 172.21.15.64:0/2554266853 >> 172.21.15.64:0/2554266853 conn(0x7efcd80781d0 msgr2=0x7efcd80785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.333+0000 7efcdda22700 1 -- 172.21.15.64:0/2554266853 shutdown_connections 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.333+0000 7efcdda22700 1 -- 172.21.15.64:0/2554266853 wait complete. 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.333+0000 7efcdda22700 1 Processor -- start 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.333+0000 7efcdda22700 1 -- start start 2024-05-09T09:08:10.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdda22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcd806a9f0 0x7efcd81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdda22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 0x7efcd81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdda22700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd81083a0 0x7efcd819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdda22700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcd81124d0 con 0x7efcd806b3b0 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdda22700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8112350 con 0x7efcd81083a0 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdda22700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8112650 con 0x7efcd806a9f0 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 0x7efcd81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 0x7efcd81a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41076/0 (socket says 172.21.15.64:41076) 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 -- 172.21.15.64:0/2202986926 learned_addr learned my addr 172.21.15.64:0/2202986926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd77fe700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd81083a0 0x7efcd819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd6ffd700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcd806a9f0 0x7efcd81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 -- 172.21.15.64:0/2202986926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcd806a9f0 msgr2=0x7efcd81a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcd806a9f0 0x7efcd81a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 -- 172.21.15.64:0/2202986926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd81083a0 msgr2=0x7efcd819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd81083a0 0x7efcd819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 -- 172.21.15.64:0/2202986926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd819b170 con 0x7efcd806b3b0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcd67fc700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 0x7efcd81a0d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efccc00c9b0 tx=0x7efccc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc007b00 con 0x7efcd806b3b0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.335+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd819b460 con 0x7efcd806b3b0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.334+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efccc007ca0 con 0x7efcd806b3b0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.335+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc018660 con 0x7efcd806b3b0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.335+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81a7720 con 0x7efcd806b3b0 2024-05-09T09:08:10.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.336+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd8103a70 con 0x7efcd806b3b0 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stdout:111669149768 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.338+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efccc00f450 con 0x7efcd806b3b0 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.338+0000 7efcdca20700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efcc00a2cb0 0x7efcc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.338+0000 7efcd6ffd700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efcc00a2cb0 0x7efcc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.338+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efccc0cf770 con 0x7efcd806b3b0 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.339+0000 7efcd6ffd700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efcc00a2cb0 0x7efcc00a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006010 tx=0x7efcc800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.342+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efccc014030 con 0x7efcd806b3b0 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.549+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efcd81a79b0 con 0x7efcd806b3b0 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.550+0000 7efcdca20700 1 -- 172.21.15.64:0/2202986926 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efccc09c2a0 con 0x7efcd806b3b0 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efcc00a2cb0 msgr2=0x7efcc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efcc00a2cb0 0x7efcc00a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006010 tx=0x7efcc800b540 comp rx=0 tx=0).stop 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 msgr2=0x7efcd81a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 0x7efcd81a0d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efccc00c9b0 tx=0x7efccc00ccc0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 shutdown_connections 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd81083a0 0x7efcd819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efcc00a2cb0 0x7efcc00a5170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efcd806a9f0 0x7efcd81a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 --2- 172.21.15.64:0/2202986926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd806b3b0 0x7efcd81a0d90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:10.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 >> 172.21.15.64:0/2202986926 conn(0x7efcd80781d0 msgr2=0x7efcd810d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:10.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 shutdown_connections 2024-05-09T09:08:10.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:09.552+0000 7efcdda22700 1 -- 172.21.15.64:0/2202986926 wait complete. 2024-05-09T09:08:10.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:10 smithi179 ceph-mon[34086]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:10.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:10 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2202986926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T09:08:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:10 smithi139 ceph-mon[33599]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:10 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2202986926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T09:08:11.153 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:11.186 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776163 got 292057776169 for osd.9 2024-05-09T09:08:11.187 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:12.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:12 smithi179 ceph-mon[34086]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:12.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:12 smithi064 ceph-mon[30149]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:12.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:12 smithi139 ceph-mon[33599]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 -- 172.21.15.64:0/3821863527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b40106a90 msgr2=0x7f1b40106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/3821863527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b40106a90 0x7f1b40106e70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009990 tx=0x7f1b34009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 -- 172.21.15.64:0/3821863527 shutdown_connections 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/3821863527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b401016a0 0x7f1b4010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/3821863527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b40106a90 0x7f1b40106e70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/3821863527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 0x7f1b40101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 -- 172.21.15.64:0/3821863527 >> 172.21.15.64:0/3821863527 conn(0x7f1b40076540 msgr2=0x7f1b40076950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 -- 172.21.15.64:0/3821863527 shutdown_connections 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 -- 172.21.15.64:0/3821863527 wait complete. 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.748+0000 7f1b46fc3700 1 Processor -- start 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 -- start start 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 0x7f1b4019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 0x7f1b4019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40106a90 0x7f1b401987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40110320 con 0x7f1b401016a0 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401101a0 con 0x7f1b40106a90 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b46fc3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401104a0 con 0x7f1b40100ce0 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 0x7f1b4019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b44d5f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 0x7f1b4019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 0x7f1b4019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41100/0 (socket says 172.21.15.64:41100) 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 -- 172.21.15.64:0/1965055907 learned_addr learned my addr 172.21.15.64:0/1965055907 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b45560700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40106a90 0x7f1b401987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 -- 172.21.15.64:0/1965055907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 msgr2=0x7f1b4019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 0x7f1b4019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 -- 172.21.15.64:0/1965055907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40106a90 msgr2=0x7f1b401987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40106a90 0x7f1b401987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b3ffff700 1 -- 172.21.15.64:0/1965055907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b40198fc0 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.749+0000 7f1b44d5f700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 0x7f1b4019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.750+0000 7f1b3ffff700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 0x7f1b4019eba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00ddc0 tx=0x7f1b2c00a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.750+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c01b070 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.750+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b2c00ac90 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.750+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0166a0 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.750+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b401992b0 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.750+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b401024e0 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.752+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b40102120 con 0x7f1b401016a0 2024-05-09T09:08:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.754+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1b2c01e070 con 0x7f1b401016a0 2024-05-09T09:08:12.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.755+0000 7f1b3dffb700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b300a2d30 0x7f1b300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:12.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.755+0000 7f1b44d5f700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b300a2d30 0x7f1b300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:12.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.755+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1b2c0ceed0 con 0x7f1b401016a0 2024-05-09T09:08:12.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.755+0000 7f1b44d5f700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b300a2d30 0x7f1b300a51f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b5c0 tx=0x7f1b34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:12.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.760+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b2c09ba80 con 0x7f1b401016a0 2024-05-09T09:08:12.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.979+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1b4002db80 con 0x7f1b401016a0 2024-05-09T09:08:12.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.979+0000 7f1b3dffb700 1 -- 172.21.15.64:0/1965055907 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1b2c09b380 con 0x7f1b401016a0 2024-05-09T09:08:12.979 INFO:teuthology.orchestra.run.smithi064.stdout:313532612645 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b300a2d30 msgr2=0x7f1b300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b300a2d30 0x7f1b300a51f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b5c0 tx=0x7f1b34005960 comp rx=0 tx=0).stop 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 msgr2=0x7f1b4019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 0x7f1b4019eba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00ddc0 tx=0x7f1b2c00a530 comp rx=0 tx=0).stop 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 shutdown_connections 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40106a90 0x7f1b401987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b300a2d30 0x7f1b300a51f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b40100ce0 0x7f1b4019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 --2- 172.21.15.64:0/1965055907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b401016a0 0x7f1b4019eba0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 >> 172.21.15.64:0/1965055907 conn(0x7f1b40076540 msgr2=0x7f1b400fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 shutdown_connections 2024-05-09T09:08:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:12.982+0000 7f1b46fc3700 1 -- 172.21.15.64:0/1965055907 wait complete. 2024-05-09T09:08:13.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:13 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1965055907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T09:08:13.486 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:13 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1965055907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T09:08:13.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:13 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1965055907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T09:08:14.248 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149763 got 111669149768 for osd.3 2024-05-09T09:08:14.248 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:14.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:14 smithi179 ceph-mon[34086]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:14.530 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:14 smithi064 ceph-mon[30149]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:14.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:14 smithi139 ceph-mon[33599]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 -- 172.21.15.64:0/1866457460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24107660 msgr2=0x7f9d2410f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/1866457460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24107660 0x7f9d2410f230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00b0a0 tx=0x7f9d1c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 -- 172.21.15.64:0/1866457460 shutdown_connections 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/1866457460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24107660 0x7f9d2410f230 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/1866457460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d241025d0 0x7f9d241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/1866457460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d24102f80 0x7f9d24106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 -- 172.21.15.64:0/1866457460 >> 172.21.15.64:0/1866457460 conn(0x7f9d24079480 msgr2=0x7f9d24079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 -- 172.21.15.64:0/1866457460 shutdown_connections 2024-05-09T09:08:14.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 -- 172.21.15.64:0/1866457460 wait complete. 2024-05-09T09:08:14.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.641+0000 7f9d2aeac700 1 Processor -- start 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 -- start start 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d241025d0 0x7f9d2419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 0x7f9d2419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d24107660 0x7f9d24198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24112550 con 0x7f9d24102f80 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241123d0 con 0x7f9d24107660 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d2aeac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241126d0 con 0x7f9d241025d0 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 0x7f9d2419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 0x7f9d2419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42640/0 (socket says 172.21.15.64:42640) 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 -- 172.21.15.64:0/3968869400 learned_addr learned my addr 172.21.15.64:0/3968869400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d28c48700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d241025d0 0x7f9d2419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d29449700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d24107660 0x7f9d24198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 -- 172.21.15.64:0/3968869400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d241025d0 msgr2=0x7f9d2419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d241025d0 0x7f9d2419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 -- 172.21.15.64:0/3968869400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d24107660 msgr2=0x7f9d24198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d24107660 0x7f9d24198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 -- 172.21.15.64:0/3968869400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d24199010 con 0x7f9d24102f80 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d29449700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d24107660 0x7f9d24198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.642+0000 7f9d23fff700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 0x7f9d2419eba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000b500 tx=0x7f9d1000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.643+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10015020 con 0x7f9d24102f80 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.643+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d10011ce0 con 0x7f9d24102f80 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.643+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d24199300 con 0x7f9d24102f80 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.643+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10010d40 con 0x7f9d24102f80 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.643+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d241a54c0 con 0x7f9d24102f80 2024-05-09T09:08:14.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.644+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d24103a60 con 0x7f9d24102f80 2024-05-09T09:08:14.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.645+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9d10018020 con 0x7f9d24102f80 2024-05-09T09:08:14.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.646+0000 7f9d21ffb700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d140a2d30 0x7f9d140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:14.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.646+0000 7f9d28c48700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d140a2d30 0x7f9d140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:14.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.646+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9d100ceb00 con 0x7f9d24102f80 2024-05-09T09:08:14.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.646+0000 7f9d28c48700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d140a2d30 0x7f9d140a51f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800ac50 tx=0x7f9d1800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:14.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.650+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9d1001f020 con 0x7f9d24102f80 2024-05-09T09:08:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.861+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9d2402db80 con 0x7f9d24102f80 2024-05-09T09:08:14.861 INFO:teuthology.orchestra.run.smithi064.stdout:223338299443 2024-05-09T09:08:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.862+0000 7f9d21ffb700 1 -- 172.21.15.64:0/3968869400 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d1009b6b0 con 0x7f9d24102f80 2024-05-09T09:08:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.864+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d140a2d30 msgr2=0x7f9d140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.864+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d140a2d30 0x7f9d140a51f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800ac50 tx=0x7f9d1800a380 comp rx=0 tx=0).stop 2024-05-09T09:08:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.864+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 msgr2=0x7f9d2419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.864+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 0x7f9d2419eba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000b500 tx=0x7f9d1000b810 comp rx=0 tx=0).stop 2024-05-09T09:08:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 shutdown_connections 2024-05-09T09:08:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d24107660 0x7f9d24198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d140a2d30 0x7f9d140a51f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d241025d0 0x7f9d2419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 --2- 172.21.15.64:0/3968869400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24102f80 0x7f9d2419eba0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 >> 172.21.15.64:0/3968869400 conn(0x7f9d24079480 msgr2=0x7f9d24100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 shutdown_connections 2024-05-09T09:08:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:14.865+0000 7f9d2aeac700 1 -- 172.21.15.64:0/3968869400 wait complete. 2024-05-09T09:08:15.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 -- 172.21.15.64:0/1796214235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444100d50 msgr2=0x7f24441011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 --2- 172.21.15.64:0/1796214235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444100d50 0x7f24441011d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2434009990 tx=0x7f2434009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 -- 172.21.15.64:0/1796214235 shutdown_connections 2024-05-09T09:08:15.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 --2- 172.21.15.64:0/1796214235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444101710 0x7f244410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 --2- 172.21.15.64:0/1796214235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2444108500 0x7f24441088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 --2- 172.21.15.64:0/1796214235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444100d50 0x7f24441011d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 -- 172.21.15.64:0/1796214235 >> 172.21.15.64:0/1796214235 conn(0x7f2444076490 msgr2=0x7f24440768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 -- 172.21.15.64:0/1796214235 shutdown_connections 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 -- 172.21.15.64:0/1796214235 wait complete. 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.337+0000 7f244c2ac700 1 Processor -- start 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 -- start start 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2444100d50 0x7f2444101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 0x7f2444102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444108500 0x7f24441029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24441124b0 con 0x7f2444101710 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2444112330 con 0x7f2444100d50 2024-05-09T09:08:15.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244c2ac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2444112630 con 0x7f2444108500 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 0x7f2444102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 0x7f2444102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42662/0 (socket says 172.21.15.64:42662) 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 -- 172.21.15.64:0/408033960 learned_addr learned my addr 172.21.15.64:0/408033960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244a048700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2444100d50 0x7f2444101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244a849700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444108500 0x7f24441029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 -- 172.21.15.64:0/408033960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444108500 msgr2=0x7f24441029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444108500 0x7f24441029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 -- 172.21.15.64:0/408033960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2444100d50 msgr2=0x7f2444101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2444100d50 0x7f2444101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f2449847700 1 -- 172.21.15.64:0/408033960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24441a6f70 con 0x7f2444101710 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.338+0000 7f244a849700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444108500 0x7f24441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.339+0000 7f2449847700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 0x7f2444102470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24340057c0 tx=0x7f24340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:15.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.339+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434026070 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.339+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2434021410 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.339+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24441a7200 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.339+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434017650 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.339+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24441a7730 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.340+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2444069010 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.341+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2434029070 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.342+0000 7f243b7fe700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24300a2d30 0x7f24300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.342+0000 7f244a048700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24300a2d30 0x7f24300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.342+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24340d0540 con 0x7f2444101710 2024-05-09T09:08:15.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.342+0000 7f244a048700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24300a2d30 0x7f24300a51f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2440009be0 tx=0x7f2440009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.345+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2434030080 con 0x7f2444101710 2024-05-09T09:08:15.437 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:15 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3968869400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T09:08:15.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 -- 172.21.15.64:0/1518626927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101220 msgr2=0x7f8c801016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 --2- 172.21.15.64:0/1518626927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101220 0x7f8c801016a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009990 tx=0x7f8c74009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 -- 172.21.15.64:0/1518626927 shutdown_connections 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 --2- 172.21.15.64:0/1518626927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101be0 0x7f8c8010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 --2- 172.21.15.64:0/1518626927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c80107290 0x7f8c80107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 --2- 172.21.15.64:0/1518626927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101220 0x7f8c801016a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.543+0000 7f8c85eef700 1 -- 172.21.15.64:0/1518626927 >> 172.21.15.64:0/1518626927 conn(0x7f8c800781d0 msgr2=0x7f8c800785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.544+0000 7f8c85eef700 1 -- 172.21.15.64:0/1518626927 shutdown_connections 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.544+0000 7f8c85eef700 1 -- 172.21.15.64:0/1518626927 wait complete. 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.544+0000 7f8c85eef700 1 Processor -- start 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.544+0000 7f8c85eef700 1 -- start start 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.544+0000 7f8c85eef700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101220 0x7f8c801a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c85eef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 0x7f8c801a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c85eef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c80107290 0x7f8c8019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c85eef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80111400 con 0x7f8c80101be0 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c85eef700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80111280 con 0x7f8c80101220 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c85eef700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80111580 con 0x7f8c80107290 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 0x7f8c801a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 0x7f8c801a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42680/0 (socket says 172.21.15.64:42680) 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 -- 172.21.15.64:0/3681394459 learned_addr learned my addr 172.21.15.64:0/3681394459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7ffff700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c80107290 0x7f8c8019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7f7fe700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101220 0x7f8c801a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 -- 172.21.15.64:0/3681394459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c80107290 msgr2=0x7f8c8019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c80107290 0x7f8c8019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 -- 172.21.15.64:0/3681394459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101220 msgr2=0x7f8c801a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101220 0x7f8c801a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 -- 172.21.15.64:0/3681394459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8019b250 con 0x7f8c80101be0 2024-05-09T09:08:15.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7f7fe700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101220 0x7f8c801a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7effd700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 0x7f8c801a0e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74027070 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c74003f10 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.545+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8019b4e0 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.546+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74022cd0 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.546+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801a7800 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.547+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c80069010 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.548+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8c7402a070 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.549+0000 7f8c7cff9700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8c680a2d30 0x7f8c680a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.549+0000 7f8c7f7fe700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8c680a2d30 0x7f8c680a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.549+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8c740d0ad0 con 0x7f8c80101be0 2024-05-09T09:08:15.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.549+0000 7f8c7f7fe700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8c680a2d30 0x7f8c680a51f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bb90 tx=0x7f8c70005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:15.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.556+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8c74031020 con 0x7f8c80101be0 2024-05-09T09:08:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.562+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f244402db80 con 0x7f2444101710 2024-05-09T09:08:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.563+0000 7f243b7fe700 1 -- 172.21.15.64:0/408033960 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2434013070 con 0x7f2444101710 2024-05-09T09:08:15.562 INFO:teuthology.orchestra.run.smithi064.stdout:201863462970 2024-05-09T09:08:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24300a2d30 msgr2=0x7f24300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24300a2d30 0x7f24300a51f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2440009be0 tx=0x7f2440009260 comp rx=0 tx=0).stop 2024-05-09T09:08:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 msgr2=0x7f2444102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 0x7f2444102470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24340057c0 tx=0x7f24340057f0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 shutdown_connections 2024-05-09T09:08:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f24300a2d30 0x7f24300a51f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2444108500 0x7f24441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2444100d50 0x7f2444101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 --2- 172.21.15.64:0/408033960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2444101710 0x7f2444102470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 >> 172.21.15.64:0/408033960 conn(0x7f2444076490 msgr2=0x7f24440ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 shutdown_connections 2024-05-09T09:08:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.565+0000 7f244c2ac700 1 -- 172.21.15.64:0/408033960 wait complete. 2024-05-09T09:08:15.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:15 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3968869400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T09:08:15.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:15 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3968869400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T09:08:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.768+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8c801a79d0 con 0x7f8c80101be0 2024-05-09T09:08:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.769+0000 7f8c7cff9700 1 -- 172.21.15.64:0/3681394459 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c74013070 con 0x7f8c80101be0 2024-05-09T09:08:15.768 INFO:teuthology.orchestra.run.smithi064.stdout:34359738456 2024-05-09T09:08:15.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8c680a2d30 msgr2=0x7f8c680a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8c680a2d30 0x7f8c680a51f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bb90 tx=0x7f8c70005d80 comp rx=0 tx=0).stop 2024-05-09T09:08:15.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 msgr2=0x7f8c801a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:15.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 0x7f8c801a0e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c740057c0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 shutdown_connections 2024-05-09T09:08:15.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8c680a2d30 0x7f8c680a51f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c80107290 0x7f8c8019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80101220 0x7f8c801a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 --2- 172.21.15.64:0/3681394459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c80101be0 0x7f8c801a0e70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:15.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 >> 172.21.15.64:0/3681394459 conn(0x7f8c800781d0 msgr2=0x7f8c800ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:15.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 shutdown_connections 2024-05-09T09:08:15.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:15.771+0000 7f8c85eef700 1 -- 172.21.15.64:0/3681394459 wait complete. 2024-05-09T09:08:16.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612641 got 313532612645 for osd.10 2024-05-09T09:08:16.401 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:16.488 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:16 smithi064 ceph-mon[30149]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:16.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:16 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/408033960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T09:08:16.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:16 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3681394459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T09:08:16.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:16 smithi139 ceph-mon[33599]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:16.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:16 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/408033960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T09:08:16.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:16 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3681394459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T09:08:16.630 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:16 smithi179 ceph-mon[34086]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:16.630 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:16 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/408033960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T09:08:16.630 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:16 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3681394459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T09:08:17.189 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574930 for osd.1 2024-05-09T09:08:17.190 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:17.607 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299437 got 223338299443 for osd.7 2024-05-09T09:08:17.607 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:18.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462964 got 201863462970 for osd.6 2024-05-09T09:08:18.132 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:18.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:18 smithi064 ceph-mon[30149]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:18.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 -- 172.21.15.64:0/2773778783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06a8c0 msgr2=0x7fc3dc06ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:18.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 --2- 172.21.15.64:0/2773778783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06a8c0 0x7fc3dc06ad20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009990 tx=0x7fc3d0009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 -- 172.21.15.64:0/2773778783 shutdown_connections 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 --2- 172.21.15.64:0/2773778783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc06b260 0x7fc3dc10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 --2- 172.21.15.64:0/2773778783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3dc108390 0x7fc3dc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 --2- 172.21.15.64:0/2773778783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06a8c0 0x7fc3dc06ad20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 -- 172.21.15.64:0/2773778783 >> 172.21.15.64:0/2773778783 conn(0x7fc3dc078190 msgr2=0x7fc3dc0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 -- 172.21.15.64:0/2773778783 shutdown_connections 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.377+0000 7fc3e247d700 1 -- 172.21.15.64:0/2773778783 wait complete. 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.378+0000 7fc3e247d700 1 Processor -- start 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.378+0000 7fc3e247d700 1 -- start start 2024-05-09T09:08:18.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.378+0000 7fc3e247d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc06a8c0 0x7fc3dc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:18.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.378+0000 7fc3e247d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 0x7fc3dc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:18.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.378+0000 7fc3e247d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3dc108390 0x7fc3dc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:18.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3e247d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1124c0 con 0x7fc3dc06b260 2024-05-09T09:08:18.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3e247d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc112340 con 0x7fc3dc108390 2024-05-09T09:08:18.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3e247d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc112640 con 0x7fc3dc06a8c0 2024-05-09T09:08:18.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 0x7fc3dc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 0x7fc3dc19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42698/0 (socket says 172.21.15.64:42698) 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 -- 172.21.15.64:0/4159196936 learned_addr learned my addr 172.21.15.64:0/4159196936 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3e0a1a700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3dc108390 0x7fc3dc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3dbfff700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc06a8c0 0x7fc3dc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 -- 172.21.15.64:0/4159196936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc06a8c0 msgr2=0x7fc3dc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc06a8c0 0x7fc3dc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 -- 172.21.15.64:0/4159196936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3dc108390 msgr2=0x7fc3dc198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:18.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3dc108390 0x7fc3dc198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 -- 172.21.15.64:0/4159196936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc199040 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3db7fe700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 0x7fc3dc19ebd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0003ab0 tx=0x7fc3d0003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.379+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0027070 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.380+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d000bdc0 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.380+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0022bf0 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.380+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc1992d0 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.380+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a54f0 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.381+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc103a40 con 0x7fc3dc06b260 2024-05-09T09:08:18.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.382+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3d002a070 con 0x7fc3dc06b260 2024-05-09T09:08:18.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.383+0000 7fc3d97fa700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3c40a2cb0 0x7fc3c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:18.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.383+0000 7fc3dbfff700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3c40a2cb0 0x7fc3c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:18.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.383+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc3d00d0300 con 0x7fc3dc06b260 2024-05-09T09:08:18.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.383+0000 7fc3dbfff700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3c40a2cb0 0x7fc3c40a5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00abd0 tx=0x7fc3cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:18.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.391+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3d005e740 con 0x7fc3dc06b260 2024-05-09T09:08:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:18 smithi139 ceph-mon[33599]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:18.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.604+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc3dc02db80 con 0x7fc3dc06b260 2024-05-09T09:08:18.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.604+0000 7fc3d97fa700 1 -- 172.21.15.64:0/4159196936 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3d0013070 con 0x7fc3dc06b260 2024-05-09T09:08:18.604 INFO:teuthology.orchestra.run.smithi064.stdout:77309411407 2024-05-09T09:08:18.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3c40a2cb0 msgr2=0x7fc3c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:18.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3c40a2cb0 0x7fc3c40a5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00abd0 tx=0x7fc3cc00a380 comp rx=0 tx=0).stop 2024-05-09T09:08:18.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 msgr2=0x7fc3dc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:18.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 0x7fc3dc19ebd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0003ab0 tx=0x7fc3d0003ae0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 shutdown_connections 2024-05-09T09:08:18.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3dc108390 0x7fc3dc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3c40a2cb0 0x7fc3c40a5170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc06a8c0 0x7fc3dc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 --2- 172.21.15.64:0/4159196936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc06b260 0x7fc3dc19ebd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 >> 172.21.15.64:0/4159196936 conn(0x7fc3dc078190 msgr2=0x7fc3dc0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.606+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 shutdown_connections 2024-05-09T09:08:18.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:18.607+0000 7fc3e247d700 1 -- 172.21.15.64:0/4159196936 wait complete. 2024-05-09T09:08:18.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:18 smithi179 ceph-mon[34086]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:18.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738456 for osd.0 2024-05-09T09:08:18.797 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:19.484 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:19 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4159196936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T09:08:19.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411407 for osd.2 2024-05-09T09:08:19.512 DEBUG:teuthology.parallel:result is None 2024-05-09T09:08:19.512 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T09:08:19.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-09T09:08:19.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:19 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4159196936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T09:08:19.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:19 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4159196936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T09:08:19.788 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:20.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:20 smithi064 ceph-mon[30149]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:20.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:20.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:20 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:20 smithi179 ceph-mon[34086]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:20 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:20 smithi139 ceph-mon[33599]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:20 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:21.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 -- 172.21.15.64:0/44420153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada8108390 msgr2=0x7fada8108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:21.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 --2- 172.21.15.64:0/44420153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada8108390 0x7fada8108770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad9800b0a0 tx=0x7fad9800b3b0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 -- 172.21.15.64:0/44420153 shutdown_connections 2024-05-09T09:08:21.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 --2- 172.21.15.64:0/44420153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada806b260 0x7fada810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 --2- 172.21.15.64:0/44420153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada8108390 0x7fada8108770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 --2- 172.21.15.64:0/44420153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada806a8c0 0x7fada806ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 -- 172.21.15.64:0/44420153 >> 172.21.15.64:0/44420153 conn(0x7fada8078190 msgr2=0x7fada80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:21.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 -- 172.21.15.64:0/44420153 shutdown_connections 2024-05-09T09:08:21.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.594+0000 7fadad118700 1 -- 172.21.15.64:0/44420153 wait complete. 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 Processor -- start 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 -- start start 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 0x7fada81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada806b260 0x7fada8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada8108390 0x7fada8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fada81124a0 con 0x7fada806a8c0 2024-05-09T09:08:21.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fada8112320 con 0x7fada806b260 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fadad118700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fada8112620 con 0x7fada8108390 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fada6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 0x7fada81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fada6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 0x7fada81012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42712/0 (socket says 172.21.15.64:42712) 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fada6d9d700 1 -- 172.21.15.64:0/1590199967 learned_addr learned my addr 172.21.15.64:0/1590199967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.595+0000 7fada759e700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada8108390 0x7fada8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fad9e59c700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada806b260 0x7fada8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada6d9d700 1 -- 172.21.15.64:0/1590199967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada8108390 msgr2=0x7fada8101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada6d9d700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada8108390 0x7fada8101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada6d9d700 1 -- 172.21.15.64:0/1590199967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada806b260 msgr2=0x7fada8101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada6d9d700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada806b260 0x7fada8101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada6d9d700 1 -- 172.21.15.64:0/1590199967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada81a6fc0 con 0x7fada806a8c0 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada6d9d700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 0x7fada81012e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad9800b070 tx=0x7fad98004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98021070 con 0x7fada806a8c0 2024-05-09T09:08:21.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada81a7250 con 0x7fada806a8c0 2024-05-09T09:08:21.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad98007e30 con 0x7fada806a8c0 2024-05-09T09:08:21.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98004780 con 0x7fada806a8c0 2024-05-09T09:08:21.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.596+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada81a7750 con 0x7fada806a8c0 2024-05-09T09:08:21.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.597+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada8069010 con 0x7fada806a8c0 2024-05-09T09:08:21.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.599+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad98007980 con 0x7fada806a8c0 2024-05-09T09:08:21.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.599+0000 7fada4d99700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad940a2cb0 0x7fad940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:21.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.600+0000 7fad9e59c700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad940a2cb0 0x7fad940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:21.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.600+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fad9800f020 con 0x7fada806a8c0 2024-05-09T09:08:21.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.600+0000 7fad9e59c700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad940a2cb0 0x7fad940a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fad90005fd0 tx=0x7fad9000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:21.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.604+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad9802a020 con 0x7fada806a8c0 2024-05-09T09:08:21.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.806+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fada8102810 con 0x7fad940a2cb0 2024-05-09T09:08:21.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.816+0000 7fada4d99700 1 -- 172.21.15.64:0/1590199967 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340365 (secure 0 0 0) 0x7fada8102810 con 0x7fad940a2cb0 2024-05-09T09:08:21.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad940a2cb0 msgr2=0x7fad940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad940a2cb0 0x7fad940a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fad90005fd0 tx=0x7fad9000a560 comp rx=0 tx=0).stop 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 msgr2=0x7fada81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 0x7fada81012e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad9800b070 tx=0x7fad98004d00 comp rx=0 tx=0).stop 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 shutdown_connections 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad940a2cb0 0x7fad940a5170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fada8108390 0x7fada8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada806a8c0 0x7fada81012e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 --2- 172.21.15.64:0/1590199967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada806b260 0x7fada8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 >> 172.21.15.64:0/1590199967 conn(0x7fada8078190 msgr2=0x7fada80fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 shutdown_connections 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:21.819+0000 7fadad118700 1 -- 172.21.15.64:0/1590199967 wait complete. 2024-05-09T09:08:21.823 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:08:21.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:08:21.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:08:21.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:21.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:08:21.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:08:21.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:22.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:08:22.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:08:22.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:08:22.702 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-09T09:08:21.122583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655402,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":570,"num_read_kb":399,"num_write":340,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392524,"kb_used_data":14180,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549300,"statfs":{"total":1151940427776,"available":1151538483200,"internally_reserved":0,"allocated":14520320,"data_stored":7988624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003490"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318971+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:07:36.318971+0000","last_peered":"2024-05-09T09:07:36.318971+0000","last_clean":"2024-05-09T09:07:36.318971+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:07:36.318971+0000","last_undegraded":"2024-05-09T09:07:36.318971+0000","last_fullsized":"2024-05-09T09:07:36.318971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317309+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:07:36.317309+0000","last_peered":"2024-05-09T09:07:36.317309+0000","last_clean":"2024-05-09T09:07:36.317309+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:07:36.317309+0000","last_undegraded":"2024-05-09T09:07:36.317309+0000","last_fullsized":"2024-05-09T09:07:36.317309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316386+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:07:36.316386+0000","last_peered":"2024-05-09T09:07:36.316386+0000","last_clean":"2024-05-09T09:07:36.316386+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:07:36.316386+0000","last_undegraded":"2024-05-09T09:07:36.316386+0000","last_fullsized":"2024-05-09T09:07:36.316386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315544+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:07:36.315544+0000","last_peered":"2024-05-09T09:07:36.315544+0000","last_clean":"2024-05-09T09:07:36.315544+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:07:36.315544+0000","last_undegraded":"2024-05-09T09:07:36.315544+0000","last_fullsized":"2024-05-09T09:07:36.315544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315448+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:07:36.315448+0000","last_peered":"2024-05-09T09:07:36.315448+0000","last_clean":"2024-05-09T09:07:36.315448+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:07:36.315448+0000","last_undegraded":"2024-05-09T09:07:36.315448+0000","last_fullsized":"2024-05-09T09:07:36.315448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316822+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:07:36.316822+0000","last_peered":"2024-05-09T09:07:36.316822+0000","last_clean":"2024-05-09T09:07:36.316822+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:07:36.316822+0000","last_undegraded":"2024-05-09T09:07:36.316822+0000","last_fullsized":"2024-05-09T09:07:36.316822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613318+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:07:36.613318+0000","last_peered":"2024-05-09T09:07:36.613318+0000","last_clean":"2024-05-09T09:07:36.613318+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:07:36.613318+0000","last_undegraded":"2024-05-09T09:07:36.613318+0000","last_fullsized":"2024-05-09T09:07:36.613318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320784+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:07:36.320784+0000","last_peered":"2024-05-09T09:07:36.320784+0000","last_clean":"2024-05-09T09:07:36.320784+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:07:36.320784+0000","last_undegraded":"2024-05-09T09:07:36.320784+0000","last_fullsized":"2024-05-09T09:07:36.320784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435257+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:07:36.435257+0000","last_peered":"2024-05-09T09:07:36.435257+0000","last_clean":"2024-05-09T09:07:36.435257+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:07:36.435257+0000","last_undegraded":"2024-05-09T09:07:36.435257+0000","last_fullsized":"2024-05-09T09:07:36.435257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317130+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:07:36.317130+0000","last_peered":"2024-05-09T09:07:36.317130+0000","last_clean":"2024-05-09T09:07:36.317130+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:07:36.317130+0000","last_undegraded":"2024-05-09T09:07:36.317130+0000","last_fullsized":"2024-05-09T09:07:36.317130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613834+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:07:36.613834+0000","last_peered":"2024-05-09T09:07:36.613834+0000","last_clean":"2024-05-09T09:07:36.613834+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:07:36.613834+0000","last_undegraded":"2024-05-09T09:07:36.613834+0000","last_fullsized":"2024-05-09T09:07:36.613834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317979+0000","last_change":"2024-05-09T09:07:36.317979+0000","last_active":"2024-05-09T09:07:36.317979+0000","last_peered":"2024-05-09T09:07:36.317979+0000","last_clean":"2024-05-09T09:07:36.317979+0000","last_became_active":"2024-05-09T09:07:36.317572+0000","last_became_peered":"2024-05-09T09:07:36.317572+0000","last_unstale":"2024-05-09T09:07:36.317979+0000","last_undegraded":"2024-05-09T09:07:36.317979+0000","last_fullsized":"2024-05-09T09:07:36.317979+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:45.499023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316654+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:07:36.316654+0000","last_peered":"2024-05-09T09:07:36.316654+0000","last_clean":"2024-05-09T09:07:36.316654+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:07:36.316654+0000","last_undegraded":"2024-05-09T09:07:36.316654+0000","last_fullsized":"2024-05-09T09:07:36.316654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316248+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:07:36.316248+0000","last_peered":"2024-05-09T09:07:36.316248+0000","last_clean":"2024-05-09T09:07:36.316248+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:07:36.316248+0000","last_undegraded":"2024-05-09T09:07:36.316248+0000","last_fullsized":"2024-05-09T09:07:36.316248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316330+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:07:36.316330+0000","last_peered":"2024-05-09T09:07:36.316330+0000","last_clean":"2024-05-09T09:07:36.316330+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:07:36.316330+0000","last_undegraded":"2024-05-09T09:07:36.316330+0000","last_fullsized":"2024-05-09T09:07:36.316330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434738+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:07:36.434738+0000","last_peered":"2024-05-09T09:07:36.434738+0000","last_clean":"2024-05-09T09:07:36.434738+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:07:36.434738+0000","last_undegraded":"2024-05-09T09:07:36.434738+0000","last_fullsized":"2024-05-09T09:07:36.434738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318971+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:07:36.318971+0000","last_peered":"2024-05-09T09:07:36.318971+0000","last_clean":"2024-05-09T09:07:36.318971+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:07:36.318971+0000","last_undegraded":"2024-05-09T09:07:36.318971+0000","last_fullsized":"2024-05-09T09:07:36.318971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314247+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:07:36.314247+0000","last_peered":"2024-05-09T09:07:36.314247+0000","last_clean":"2024-05-09T09:07:36.314247+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:07:36.314247+0000","last_undegraded":"2024-05-09T09:07:36.314247+0000","last_fullsized":"2024-05-09T09:07:36.314247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323149+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:07:36.323149+0000","last_peered":"2024-05-09T09:07:36.323149+0000","last_clean":"2024-05-09T09:07:36.323149+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:07:36.323149+0000","last_undegraded":"2024-05-09T09:07:36.323149+0000","last_fullsized":"2024-05-09T09:07:36.323149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323847+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:07:36.323847+0000","last_peered":"2024-05-09T09:07:36.323847+0000","last_clean":"2024-05-09T09:07:36.323847+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:07:36.323847+0000","last_undegraded":"2024-05-09T09:07:36.323847+0000","last_fullsized":"2024-05-09T09:07:36.323847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314269+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:07:36.314269+0000","last_peered":"2024-05-09T09:07:36.314269+0000","last_clean":"2024-05-09T09:07:36.314269+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:07:36.314269+0000","last_undegraded":"2024-05-09T09:07:36.314269+0000","last_fullsized":"2024-05-09T09:07:36.314269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320464+0000","last_change":"2024-05-09T09:06:41.137679+0000","last_active":"2024-05-09T09:07:36.320464+0000","last_peered":"2024-05-09T09:07:36.320464+0000","last_clean":"2024-05-09T09:07:36.320464+0000","last_became_active":"2024-05-09T09:06:41.137353+0000","last_became_peered":"2024-05-09T09:06:41.137353+0000","last_unstale":"2024-05-09T09:07:36.320464+0000","last_undegraded":"2024-05-09T09:07:36.320464+0000","last_fullsized":"2024-05-09T09:07:36.320464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:36.080021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314517+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:07:36.314517+0000","last_peered":"2024-05-09T09:07:36.314517+0000","last_clean":"2024-05-09T09:07:36.314517+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:07:36.314517+0000","last_undegraded":"2024-05-09T09:07:36.314517+0000","last_fullsized":"2024-05-09T09:07:36.314517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321638+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:07:36.321638+0000","last_peered":"2024-05-09T09:07:36.321638+0000","last_clean":"2024-05-09T09:07:36.321638+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:07:36.321638+0000","last_undegraded":"2024-05-09T09:07:36.321638+0000","last_fullsized":"2024-05-09T09:07:36.321638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317330+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:07:36.317330+0000","last_peered":"2024-05-09T09:07:36.317330+0000","last_clean":"2024-05-09T09:07:36.317330+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:07:36.317330+0000","last_undegraded":"2024-05-09T09:07:36.317330+0000","last_fullsized":"2024-05-09T09:07:36.317330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314905+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:07:36.314905+0000","last_peered":"2024-05-09T09:07:36.314905+0000","last_clean":"2024-05-09T09:07:36.314905+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:07:36.314905+0000","last_undegraded":"2024-05-09T09:07:36.314905+0000","last_fullsized":"2024-05-09T09:07:36.314905+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315959+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:07:36.315959+0000","last_peered":"2024-05-09T09:07:36.315959+0000","last_clean":"2024-05-09T09:07:36.315959+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:07:36.315959+0000","last_undegraded":"2024-05-09T09:07:36.315959+0000","last_fullsized":"2024-05-09T09:07:36.315959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435121+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:07:36.435121+0000","last_peered":"2024-05-09T09:07:36.435121+0000","last_clean":"2024-05-09T09:07:36.435121+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:07:36.435121+0000","last_undegraded":"2024-05-09T09:07:36.435121+0000","last_fullsized":"2024-05-09T09:07:36.435121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314645+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:07:36.314645+0000","last_peered":"2024-05-09T09:07:36.314645+0000","last_clean":"2024-05-09T09:07:36.314645+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:07:36.314645+0000","last_undegraded":"2024-05-09T09:07:36.314645+0000","last_fullsized":"2024-05-09T09:07:36.314645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316278+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:07:36.316278+0000","last_peered":"2024-05-09T09:07:36.316278+0000","last_clean":"2024-05-09T09:07:36.316278+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:07:36.316278+0000","last_undegraded":"2024-05-09T09:07:36.316278+0000","last_fullsized":"2024-05-09T09:07:36.316278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323676+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:07:36.323676+0000","last_peered":"2024-05-09T09:07:36.323676+0000","last_clean":"2024-05-09T09:07:36.323676+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:07:36.323676+0000","last_undegraded":"2024-05-09T09:07:36.323676+0000","last_fullsized":"2024-05-09T09:07:36.323676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322976+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:07:36.322976+0000","last_peered":"2024-05-09T09:07:36.322976+0000","last_clean":"2024-05-09T09:07:36.322976+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:07:36.322976+0000","last_undegraded":"2024-05-09T09:07:36.322976+0000","last_fullsized":"2024-05-09T09:07:36.322976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:16.337607+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:08:16.337607+0000","last_peered":"2024-05-09T09:08:16.337607+0000","last_clean":"2024-05-09T09:08:16.337607+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:08:16.337607+0000","last_undegraded":"2024-05-09T09:08:16.337607+0000","last_fullsized":"2024-05-09T09:08:16.337607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.615226+0000","last_change":"2024-05-09T09:07:36.615226+0000","last_active":"2024-05-09T09:07:36.615226+0000","last_peered":"2024-05-09T09:07:36.615226+0000","last_clean":"2024-05-09T09:07:36.615226+0000","last_became_active":"2024-05-09T09:07:36.614762+0000","last_became_peered":"2024-05-09T09:07:36.614762+0000","last_unstale":"2024-05-09T09:07:36.615226+0000","last_undegraded":"2024-05-09T09:07:36.615226+0000","last_fullsized":"2024-05-09T09:07:36.615226+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:29.045379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322498+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:07:36.322498+0000","last_peered":"2024-05-09T09:07:36.322498+0000","last_clean":"2024-05-09T09:07:36.322498+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:07:36.322498+0000","last_undegraded":"2024-05-09T09:07:36.322498+0000","last_fullsized":"2024-05-09T09:07:36.322498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316836+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:07:36.316836+0000","last_peered":"2024-05-09T09:07:36.316836+0000","last_clean":"2024-05-09T09:07:36.316836+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:07:36.316836+0000","last_undegraded":"2024-05-09T09:07:36.316836+0000","last_fullsized":"2024-05-09T09:07:36.316836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323259+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:07:36.323259+0000","last_peered":"2024-05-09T09:07:36.323259+0000","last_clean":"2024-05-09T09:07:36.323259+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:07:36.323259+0000","last_undegraded":"2024-05-09T09:07:36.323259+0000","last_fullsized":"2024-05-09T09:07:36.323259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318835+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:07:36.318835+0000","last_peered":"2024-05-09T09:07:36.318835+0000","last_clean":"2024-05-09T09:07:36.318835+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:07:36.318835+0000","last_undegraded":"2024-05-09T09:07:36.318835+0000","last_fullsized":"2024-05-09T09:07:36.318835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316166+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:07:36.316166+0000","last_peered":"2024-05-09T09:07:36.316166+0000","last_clean":"2024-05-09T09:07:36.316166+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:07:36.316166+0000","last_undegraded":"2024-05-09T09:07:36.316166+0000","last_fullsized":"2024-05-09T09:07:36.316166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320899+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:07:36.320899+0000","last_peered":"2024-05-09T09:07:36.320899+0000","last_clean":"2024-05-09T09:07:36.320899+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:07:36.320899+0000","last_undegraded":"2024-05-09T09:07:36.320899+0000","last_fullsized":"2024-05-09T09:07:36.320899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316250+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:07:36.316250+0000","last_peered":"2024-05-09T09:07:36.316250+0000","last_clean":"2024-05-09T09:07:36.316250+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:07:36.316250+0000","last_undegraded":"2024-05-09T09:07:36.316250+0000","last_fullsized":"2024-05-09T09:07:36.316250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321720+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:07:36.321720+0000","last_peered":"2024-05-09T09:07:36.321720+0000","last_clean":"2024-05-09T09:07:36.321720+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:07:36.321720+0000","last_undegraded":"2024-05-09T09:07:36.321720+0000","last_fullsized":"2024-05-09T09:07:36.321720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.313670+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:07:36.313670+0000","last_peered":"2024-05-09T09:07:36.313670+0000","last_clean":"2024-05-09T09:07:36.313670+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:07:36.313670+0000","last_undegraded":"2024-05-09T09:07:36.313670+0000","last_fullsized":"2024-05-09T09:07:36.313670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320638+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:07:36.320638+0000","last_peered":"2024-05-09T09:07:36.320638+0000","last_clean":"2024-05-09T09:07:36.320638+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:07:36.320638+0000","last_undegraded":"2024-05-09T09:07:36.320638+0000","last_fullsized":"2024-05-09T09:07:36.320638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:16.336927+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:08:16.336927+0000","last_peered":"2024-05-09T09:08:16.336927+0000","last_clean":"2024-05-09T09:08:16.336927+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:08:16.336927+0000","last_undegraded":"2024-05-09T09:08:16.336927+0000","last_fullsized":"2024-05-09T09:08:16.336927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435065+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:07:36.435065+0000","last_peered":"2024-05-09T09:07:36.435065+0000","last_clean":"2024-05-09T09:07:36.435065+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:07:36.435065+0000","last_undegraded":"2024-05-09T09:07:36.435065+0000","last_fullsized":"2024-05-09T09:07:36.435065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.313514+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:07:36.313514+0000","last_peered":"2024-05-09T09:07:36.313514+0000","last_clean":"2024-05-09T09:07:36.313514+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:07:36.313514+0000","last_undegraded":"2024-05-09T09:07:36.313514+0000","last_fullsized":"2024-05-09T09:07:36.313514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613769+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:07:36.613769+0000","last_peered":"2024-05-09T09:07:36.613769+0000","last_clean":"2024-05-09T09:07:36.613769+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:07:36.613769+0000","last_undegraded":"2024-05-09T09:07:36.613769+0000","last_fullsized":"2024-05-09T09:07:36.613769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317329+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:07:36.317329+0000","last_peered":"2024-05-09T09:07:36.317329+0000","last_clean":"2024-05-09T09:07:36.317329+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:07:36.317329+0000","last_undegraded":"2024-05-09T09:07:36.317329+0000","last_fullsized":"2024-05-09T09:07:36.317329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.436587+0000","last_change":"2024-05-09T09:07:36.436587+0000","last_active":"2024-05-09T09:07:36.436587+0000","last_peered":"2024-05-09T09:07:36.436587+0000","last_clean":"2024-05-09T09:07:36.436587+0000","last_became_active":"2024-05-09T09:07:36.436083+0000","last_became_peered":"2024-05-09T09:07:36.436083+0000","last_unstale":"2024-05-09T09:07:36.436587+0000","last_undegraded":"2024-05-09T09:07:36.436587+0000","last_fullsized":"2024-05-09T09:07:36.436587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:02:34.904307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323557+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:07:36.323557+0000","last_peered":"2024-05-09T09:07:36.323557+0000","last_clean":"2024-05-09T09:07:36.323557+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:07:36.323557+0000","last_undegraded":"2024-05-09T09:07:36.323557+0000","last_fullsized":"2024-05-09T09:07:36.323557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435284+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:07:36.435284+0000","last_peered":"2024-05-09T09:07:36.435284+0000","last_clean":"2024-05-09T09:07:36.435284+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:07:36.435284+0000","last_undegraded":"2024-05-09T09:07:36.435284+0000","last_fullsized":"2024-05-09T09:07:36.435284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.325270+0000","last_change":"2024-05-09T09:07:36.324821+0000","last_active":"2024-05-09T09:07:36.325270+0000","last_peered":"2024-05-09T09:07:36.325270+0000","last_clean":"2024-05-09T09:07:36.325270+0000","last_became_active":"2024-05-09T09:07:36.324478+0000","last_became_peered":"2024-05-09T09:07:36.324478+0000","last_unstale":"2024-05-09T09:07:36.325270+0000","last_undegraded":"2024-05-09T09:07:36.325270+0000","last_fullsized":"2024-05-09T09:07:36.325270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:25.476000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":206,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:17.355574+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:08:17.355574+0000","last_peered":"2024-05-09T09:08:17.355574+0000","last_clean":"2024-05-09T09:08:17.355574+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:08:17.355574+0000","last_undegraded":"2024-05-09T09:08:17.355574+0000","last_fullsized":"2024-05-09T09:08:17.355574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61190,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":147,"num_read_kb":134,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315606+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:07:36.315606+0000","last_peered":"2024-05-09T09:07:36.315606+0000","last_clean":"2024-05-09T09:07:36.315606+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:07:36.315606+0000","last_undegraded":"2024-05-09T09:07:36.315606+0000","last_fullsized":"2024-05-09T09:07:36.315606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.324086+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:07:36.324086+0000","last_peered":"2024-05-09T09:07:36.324086+0000","last_clean":"2024-05-09T09:07:36.324086+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:07:36.324086+0000","last_undegraded":"2024-05-09T09:07:36.324086+0000","last_fullsized":"2024-05-09T09:07:36.324086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322308+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:07:36.322308+0000","last_peered":"2024-05-09T09:07:36.322308+0000","last_clean":"2024-05-09T09:07:36.322308+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:07:36.322308+0000","last_undegraded":"2024-05-09T09:07:36.322308+0000","last_fullsized":"2024-05-09T09:07:36.322308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316458+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:07:36.316458+0000","last_peered":"2024-05-09T09:07:36.316458+0000","last_clean":"2024-05-09T09:07:36.316458+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:07:36.316458+0000","last_undegraded":"2024-05-09T09:07:36.316458+0000","last_fullsized":"2024-05-09T09:07:36.316458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318689+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:07:36.318689+0000","last_peered":"2024-05-09T09:07:36.318689+0000","last_clean":"2024-05-09T09:07:36.318689+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:07:36.318689+0000","last_undegraded":"2024-05-09T09:07:36.318689+0000","last_fullsized":"2024-05-09T09:07:36.318689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322949+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:07:36.322949+0000","last_peered":"2024-05-09T09:07:36.322949+0000","last_clean":"2024-05-09T09:07:36.322949+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:07:36.322949+0000","last_undegraded":"2024-05-09T09:07:36.322949+0000","last_fullsized":"2024-05-09T09:07:36.322949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613599+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:07:36.613599+0000","last_peered":"2024-05-09T09:07:36.613599+0000","last_clean":"2024-05-09T09:07:36.613599+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:07:36.613599+0000","last_undegraded":"2024-05-09T09:07:36.613599+0000","last_fullsized":"2024-05-09T09:07:36.613599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314113+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:07:36.314113+0000","last_peered":"2024-05-09T09:07:36.314113+0000","last_clean":"2024-05-09T09:07:36.314113+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:07:36.314113+0000","last_undegraded":"2024-05-09T09:07:36.314113+0000","last_fullsized":"2024-05-09T09:07:36.314113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:17.338542+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:08:17.338542+0000","last_peered":"2024-05-09T09:08:17.338542+0000","last_clean":"2024-05-09T09:08:17.338542+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:08:17.338542+0000","last_undegraded":"2024-05-09T09:08:17.338542+0000","last_fullsized":"2024-05-09T09:08:17.338542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613401+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:07:36.613401+0000","last_peered":"2024-05-09T09:07:36.613401+0000","last_clean":"2024-05-09T09:07:36.613401+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:07:36.613401+0000","last_undegraded":"2024-05-09T09:07:36.613401+0000","last_fullsized":"2024-05-09T09:07:36.613401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323604+0000","last_change":"2024-05-09T09:07:36.323604+0000","last_active":"2024-05-09T09:07:36.323604+0000","last_peered":"2024-05-09T09:07:36.323604+0000","last_clean":"2024-05-09T09:07:36.323604+0000","last_became_active":"2024-05-09T09:07:36.323254+0000","last_became_peered":"2024-05-09T09:07:36.323254+0000","last_unstale":"2024-05-09T09:07:36.323604+0000","last_undegraded":"2024-05-09T09:07:36.323604+0000","last_fullsized":"2024-05-09T09:07:36.323604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:31:18.746847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315125+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:07:36.315125+0000","last_peered":"2024-05-09T09:07:36.315125+0000","last_clean":"2024-05-09T09:07:36.315125+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:07:36.315125+0000","last_undegraded":"2024-05-09T09:07:36.315125+0000","last_fullsized":"2024-05-09T09:07:36.315125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321357+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:07:36.321357+0000","last_peered":"2024-05-09T09:07:36.321357+0000","last_clean":"2024-05-09T09:07:36.321357+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:07:36.321357+0000","last_undegraded":"2024-05-09T09:07:36.321357+0000","last_fullsized":"2024-05-09T09:07:36.321357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321561+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:07:36.321561+0000","last_peered":"2024-05-09T09:07:36.321561+0000","last_clean":"2024-05-09T09:07:36.321561+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:07:36.321561+0000","last_undegraded":"2024-05-09T09:07:36.321561+0000","last_fullsized":"2024-05-09T09:07:36.321561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.331734+0000","last_change":"2024-05-09T09:07:36.331318+0000","last_active":"2024-05-09T09:07:36.331734+0000","last_peered":"2024-05-09T09:07:36.331734+0000","last_clean":"2024-05-09T09:07:36.331734+0000","last_became_active":"2024-05-09T09:07:36.323269+0000","last_became_peered":"2024-05-09T09:07:36.323269+0000","last_unstale":"2024-05-09T09:07:36.331734+0000","last_undegraded":"2024-05-09T09:07:36.331734+0000","last_fullsized":"2024-05-09T09:07:36.331734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:25:53.779933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.319495+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:07:36.319495+0000","last_peered":"2024-05-09T09:07:36.319495+0000","last_clean":"2024-05-09T09:07:36.319495+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:07:36.319495+0000","last_undegraded":"2024-05-09T09:07:36.319495+0000","last_fullsized":"2024-05-09T09:07:36.319495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315897+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:07:36.315897+0000","last_peered":"2024-05-09T09:07:36.315897+0000","last_clean":"2024-05-09T09:07:36.315897+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:07:36.315897+0000","last_undegraded":"2024-05-09T09:07:36.315897+0000","last_fullsized":"2024-05-09T09:07:36.315897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.313866+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:07:36.313866+0000","last_peered":"2024-05-09T09:07:36.313866+0000","last_clean":"2024-05-09T09:07:36.313866+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:07:36.313866+0000","last_undegraded":"2024-05-09T09:07:36.313866+0000","last_fullsized":"2024-05-09T09:07:36.313866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316459+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:07:36.316459+0000","last_peered":"2024-05-09T09:07:36.316459+0000","last_clean":"2024-05-09T09:07:36.316459+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:07:36.316459+0000","last_undegraded":"2024-05-09T09:07:36.316459+0000","last_fullsized":"2024-05-09T09:07:36.316459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317147+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:07:36.317147+0000","last_peered":"2024-05-09T09:07:36.317147+0000","last_clean":"2024-05-09T09:07:36.317147+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:07:36.317147+0000","last_undegraded":"2024-05-09T09:07:36.317147+0000","last_fullsized":"2024-05-09T09:07:36.317147+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613401+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:07:36.613401+0000","last_peered":"2024-05-09T09:07:36.613401+0000","last_clean":"2024-05-09T09:07:36.613401+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:07:36.613401+0000","last_undegraded":"2024-05-09T09:07:36.613401+0000","last_fullsized":"2024-05-09T09:07:36.613401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316793+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:07:36.316793+0000","last_peered":"2024-05-09T09:07:36.316793+0000","last_clean":"2024-05-09T09:07:36.316793+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:07:36.316793+0000","last_undegraded":"2024-05-09T09:07:36.316793+0000","last_fullsized":"2024-05-09T09:07:36.316793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316929+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:07:36.316929+0000","last_peered":"2024-05-09T09:07:36.316929+0000","last_clean":"2024-05-09T09:07:36.316929+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:07:36.316929+0000","last_undegraded":"2024-05-09T09:07:36.316929+0000","last_fullsized":"2024-05-09T09:07:36.316929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321954+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:07:36.321954+0000","last_peered":"2024-05-09T09:07:36.321954+0000","last_clean":"2024-05-09T09:07:36.321954+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:07:36.321954+0000","last_undegraded":"2024-05-09T09:07:36.321954+0000","last_fullsized":"2024-05-09T09:07:36.321954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:15.335158+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:08:15.335158+0000","last_peered":"2024-05-09T09:08:15.335158+0000","last_clean":"2024-05-09T09:08:15.335158+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:08:15.335158+0000","last_undegraded":"2024-05-09T09:08:15.335158+0000","last_fullsized":"2024-05-09T09:08:15.335158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613567+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:07:36.613567+0000","last_peered":"2024-05-09T09:07:36.613567+0000","last_clean":"2024-05-09T09:07:36.613567+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:07:36.613567+0000","last_undegraded":"2024-05-09T09:07:36.613567+0000","last_fullsized":"2024-05-09T09:07:36.613567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320275+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:07:36.320275+0000","last_peered":"2024-05-09T09:07:36.320275+0000","last_clean":"2024-05-09T09:07:36.320275+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:07:36.320275+0000","last_undegraded":"2024-05-09T09:07:36.320275+0000","last_fullsized":"2024-05-09T09:07:36.320275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315054+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:07:36.315054+0000","last_peered":"2024-05-09T09:07:36.315054+0000","last_clean":"2024-05-09T09:07:36.315054+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:07:36.315054+0000","last_undegraded":"2024-05-09T09:07:36.315054+0000","last_fullsized":"2024-05-09T09:07:36.315054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613351+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:07:36.613351+0000","last_peered":"2024-05-09T09:07:36.613351+0000","last_clean":"2024-05-09T09:07:36.613351+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:07:36.613351+0000","last_undegraded":"2024-05-09T09:07:36.613351+0000","last_fullsized":"2024-05-09T09:07:36.613351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322233+0000","last_change":"2024-05-09T09:07:36.319419+0000","last_active":"2024-05-09T09:07:36.322233+0000","last_peered":"2024-05-09T09:07:36.322233+0000","last_clean":"2024-05-09T09:07:36.322233+0000","last_became_active":"2024-05-09T09:07:36.318825+0000","last_became_peered":"2024-05-09T09:07:36.318825+0000","last_unstale":"2024-05-09T09:07:36.322233+0000","last_undegraded":"2024-05-09T09:07:36.322233+0000","last_fullsized":"2024-05-09T09:07:36.322233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:21:10.289893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315919+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:07:36.315919+0000","last_peered":"2024-05-09T09:07:36.315919+0000","last_clean":"2024-05-09T09:07:36.315919+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:07:36.315919+0000","last_undegraded":"2024-05-09T09:07:36.315919+0000","last_fullsized":"2024-05-09T09:07:36.315919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613269+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:07:36.613269+0000","last_peered":"2024-05-09T09:07:36.613269+0000","last_clean":"2024-05-09T09:07:36.613269+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:07:36.613269+0000","last_undegraded":"2024-05-09T09:07:36.613269+0000","last_fullsized":"2024-05-09T09:07:36.613269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318515+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:07:36.318515+0000","last_peered":"2024-05-09T09:07:36.318515+0000","last_clean":"2024-05-09T09:07:36.318515+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:07:36.318515+0000","last_undegraded":"2024-05-09T09:07:36.318515+0000","last_fullsized":"2024-05-09T09:07:36.318515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315753+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:07:36.315753+0000","last_peered":"2024-05-09T09:07:36.315753+0000","last_clean":"2024-05-09T09:07:36.315753+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:07:36.315753+0000","last_undegraded":"2024-05-09T09:07:36.315753+0000","last_fullsized":"2024-05-09T09:07:36.315753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317307+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:07:36.317307+0000","last_peered":"2024-05-09T09:07:36.317307+0000","last_clean":"2024-05-09T09:07:36.317307+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:07:36.317307+0000","last_undegraded":"2024-05-09T09:07:36.317307+0000","last_fullsized":"2024-05-09T09:07:36.317307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320830+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:07:36.320830+0000","last_peered":"2024-05-09T09:07:36.320830+0000","last_clean":"2024-05-09T09:07:36.320830+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:07:36.320830+0000","last_undegraded":"2024-05-09T09:07:36.320830+0000","last_fullsized":"2024-05-09T09:07:36.320830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314280+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:07:36.314280+0000","last_peered":"2024-05-09T09:07:36.314280+0000","last_clean":"2024-05-09T09:07:36.314280+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:07:36.314280+0000","last_undegraded":"2024-05-09T09:07:36.314280+0000","last_fullsized":"2024-05-09T09:07:36.314280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321136+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:07:36.321136+0000","last_peered":"2024-05-09T09:07:36.321136+0000","last_clean":"2024-05-09T09:07:36.321136+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:07:36.321136+0000","last_undegraded":"2024-05-09T09:07:36.321136+0000","last_fullsized":"2024-05-09T09:07:36.321136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321815+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:07:36.321815+0000","last_peered":"2024-05-09T09:07:36.321815+0000","last_clean":"2024-05-09T09:07:36.321815+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:07:36.321815+0000","last_undegraded":"2024-05-09T09:07:36.321815+0000","last_fullsized":"2024-05-09T09:07:36.321815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315740+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:07:36.315740+0000","last_peered":"2024-05-09T09:07:36.315740+0000","last_clean":"2024-05-09T09:07:36.315740+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:07:36.315740+0000","last_undegraded":"2024-05-09T09:07:36.315740+0000","last_fullsized":"2024-05-09T09:07:36.315740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322492+0000","last_change":"2024-05-09T09:07:36.322492+0000","last_active":"2024-05-09T09:07:36.322492+0000","last_peered":"2024-05-09T09:07:36.322492+0000","last_clean":"2024-05-09T09:07:36.322492+0000","last_became_active":"2024-05-09T09:07:36.322098+0000","last_became_peered":"2024-05-09T09:07:36.322098+0000","last_unstale":"2024-05-09T09:07:36.322492+0000","last_undegraded":"2024-05-09T09:07:36.322492+0000","last_fullsized":"2024-05-09T09:07:36.322492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:55.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315198+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:07:36.315198+0000","last_peered":"2024-05-09T09:07:36.315198+0000","last_clean":"2024-05-09T09:07:36.315198+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:07:36.315198+0000","last_undegraded":"2024-05-09T09:07:36.315198+0000","last_fullsized":"2024-05-09T09:07:36.315198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434745+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:07:36.434745+0000","last_peered":"2024-05-09T09:07:36.434745+0000","last_clean":"2024-05-09T09:07:36.434745+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:07:36.434745+0000","last_undegraded":"2024-05-09T09:07:36.434745+0000","last_fullsized":"2024-05-09T09:07:36.434745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315133+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:07:36.315133+0000","last_peered":"2024-05-09T09:07:36.315133+0000","last_clean":"2024-05-09T09:07:36.315133+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:07:36.315133+0000","last_undegraded":"2024-05-09T09:07:36.315133+0000","last_fullsized":"2024-05-09T09:07:36.315133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314234+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:07:36.314234+0000","last_peered":"2024-05-09T09:07:36.314234+0000","last_clean":"2024-05-09T09:07:36.314234+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:07:36.314234+0000","last_undegraded":"2024-05-09T09:07:36.314234+0000","last_fullsized":"2024-05-09T09:07:36.314234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320748+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:07:36.320748+0000","last_peered":"2024-05-09T09:07:36.320748+0000","last_clean":"2024-05-09T09:07:36.320748+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:07:36.320748+0000","last_undegraded":"2024-05-09T09:07:36.320748+0000","last_fullsized":"2024-05-09T09:07:36.320748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318276+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:07:36.318276+0000","last_peered":"2024-05-09T09:07:36.318276+0000","last_clean":"2024-05-09T09:07:36.318276+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:07:36.318276+0000","last_undegraded":"2024-05-09T09:07:36.318276+0000","last_fullsized":"2024-05-09T09:07:36.318276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315220+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:07:36.315220+0000","last_peered":"2024-05-09T09:07:36.315220+0000","last_clean":"2024-05-09T09:07:36.315220+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:07:36.315220+0000","last_undegraded":"2024-05-09T09:07:36.315220+0000","last_fullsized":"2024-05-09T09:07:36.315220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.327910+0000","last_change":"2024-05-09T09:07:36.327470+0000","last_active":"2024-05-09T09:07:36.327910+0000","last_peered":"2024-05-09T09:07:36.327910+0000","last_clean":"2024-05-09T09:07:36.327910+0000","last_became_active":"2024-05-09T09:07:36.321516+0000","last_became_peered":"2024-05-09T09:07:36.321516+0000","last_unstale":"2024-05-09T09:07:36.327910+0000","last_undegraded":"2024-05-09T09:07:36.327910+0000","last_fullsized":"2024-05-09T09:07:36.327910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:30.824518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321220+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:07:36.321220+0000","last_peered":"2024-05-09T09:07:36.321220+0000","last_clean":"2024-05-09T09:07:36.321220+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:07:36.321220+0000","last_undegraded":"2024-05-09T09:07:36.321220+0000","last_fullsized":"2024-05-09T09:07:36.321220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315395+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:07:36.315395+0000","last_peered":"2024-05-09T09:07:36.315395+0000","last_clean":"2024-05-09T09:07:36.315395+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:07:36.315395+0000","last_undegraded":"2024-05-09T09:07:36.315395+0000","last_fullsized":"2024-05-09T09:07:36.315395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315260+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:07:36.315260+0000","last_peered":"2024-05-09T09:07:36.315260+0000","last_clean":"2024-05-09T09:07:36.315260+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:07:36.315260+0000","last_undegraded":"2024-05-09T09:07:36.315260+0000","last_fullsized":"2024-05-09T09:07:36.315260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613369+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:07:36.613369+0000","last_peered":"2024-05-09T09:07:36.613369+0000","last_clean":"2024-05-09T09:07:36.613369+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:07:36.613369+0000","last_undegraded":"2024-05-09T09:07:36.613369+0000","last_fullsized":"2024-05-09T09:07:36.613369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434715+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:07:36.434715+0000","last_peered":"2024-05-09T09:07:36.434715+0000","last_clean":"2024-05-09T09:07:36.434715+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:07:36.434715+0000","last_undegraded":"2024-05-09T09:07:36.434715+0000","last_fullsized":"2024-05-09T09:07:36.434715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322742+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:07:36.322742+0000","last_peered":"2024-05-09T09:07:36.322742+0000","last_clean":"2024-05-09T09:07:36.322742+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:07:36.322742+0000","last_undegraded":"2024-05-09T09:07:36.322742+0000","last_fullsized":"2024-05-09T09:07:36.322742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613855+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:07:36.613855+0000","last_peered":"2024-05-09T09:07:36.613855+0000","last_clean":"2024-05-09T09:07:36.613855+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:07:36.613855+0000","last_undegraded":"2024-05-09T09:07:36.613855+0000","last_fullsized":"2024-05-09T09:07:36.613855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316991+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:07:36.316991+0000","last_peered":"2024-05-09T09:07:36.316991+0000","last_clean":"2024-05-09T09:07:36.316991+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:07:36.316991+0000","last_undegraded":"2024-05-09T09:07:36.316991+0000","last_fullsized":"2024-05-09T09:07:36.316991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323802+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:07:36.323802+0000","last_peered":"2024-05-09T09:07:36.323802+0000","last_clean":"2024-05-09T09:07:36.323802+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:07:36.323802+0000","last_undegraded":"2024-05-09T09:07:36.323802+0000","last_fullsized":"2024-05-09T09:07:36.323802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613706+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:07:36.613706+0000","last_peered":"2024-05-09T09:07:36.613706+0000","last_clean":"2024-05-09T09:07:36.613706+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:07:36.613706+0000","last_undegraded":"2024-05-09T09:07:36.613706+0000","last_fullsized":"2024-05-09T09:07:36.613706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321107+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:07:36.321107+0000","last_peered":"2024-05-09T09:07:36.321107+0000","last_clean":"2024-05-09T09:07:36.321107+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:07:36.321107+0000","last_undegraded":"2024-05-09T09:07:36.321107+0000","last_fullsized":"2024-05-09T09:07:36.321107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316904+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:07:36.316904+0000","last_peered":"2024-05-09T09:07:36.316904+0000","last_clean":"2024-05-09T09:07:36.316904+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:07:36.316904+0000","last_undegraded":"2024-05-09T09:07:36.316904+0000","last_fullsized":"2024-05-09T09:07:36.316904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613934+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:07:36.613934+0000","last_peered":"2024-05-09T09:07:36.613934+0000","last_clean":"2024-05-09T09:07:36.613934+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:07:36.613934+0000","last_undegraded":"2024-05-09T09:07:36.613934+0000","last_fullsized":"2024-05-09T09:07:36.613934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318373+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:07:36.318373+0000","last_peered":"2024-05-09T09:07:36.318373+0000","last_clean":"2024-05-09T09:07:36.318373+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:07:36.318373+0000","last_undegraded":"2024-05-09T09:07:36.318373+0000","last_fullsized":"2024-05-09T09:07:36.318373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322230+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:07:36.322230+0000","last_peered":"2024-05-09T09:07:36.322230+0000","last_clean":"2024-05-09T09:07:36.322230+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:07:36.322230+0000","last_undegraded":"2024-05-09T09:07:36.322230+0000","last_fullsized":"2024-05-09T09:07:36.322230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321842+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:07:36.321842+0000","last_peered":"2024-05-09T09:07:36.321842+0000","last_clean":"2024-05-09T09:07:36.321842+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:07:36.321842+0000","last_undegraded":"2024-05-09T09:07:36.321842+0000","last_fullsized":"2024-05-09T09:07:36.321842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321453+0000","last_change":"2024-05-09T09:07:36.321453+0000","last_active":"2024-05-09T09:07:36.321453+0000","last_peered":"2024-05-09T09:07:36.321453+0000","last_clean":"2024-05-09T09:07:36.321453+0000","last_became_active":"2024-05-09T09:07:36.321074+0000","last_became_peered":"2024-05-09T09:07:36.321074+0000","last_unstale":"2024-05-09T09:07:36.321453+0000","last_undegraded":"2024-05-09T09:07:36.321453+0000","last_fullsized":"2024-05-09T09:07:36.321453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:26.690978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317413+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:07:36.317413+0000","last_peered":"2024-05-09T09:07:36.317413+0000","last_clean":"2024-05-09T09:07:36.317413+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:07:36.317413+0000","last_undegraded":"2024-05-09T09:07:36.317413+0000","last_fullsized":"2024-05-09T09:07:36.317413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.319356+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:07:36.319356+0000","last_peered":"2024-05-09T09:07:36.319356+0000","last_clean":"2024-05-09T09:07:36.319356+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:07:36.319356+0000","last_undegraded":"2024-05-09T09:07:36.319356+0000","last_fullsized":"2024-05-09T09:07:36.319356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323728+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:07:36.323728+0000","last_peered":"2024-05-09T09:07:36.323728+0000","last_clean":"2024-05-09T09:07:36.323728+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:07:36.323728+0000","last_undegraded":"2024-05-09T09:07:36.323728+0000","last_fullsized":"2024-05-09T09:07:36.323728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321222+0000","last_change":"2024-05-09T09:06:41.131374+0000","last_active":"2024-05-09T09:07:36.321222+0000","last_peered":"2024-05-09T09:07:36.321222+0000","last_clean":"2024-05-09T09:07:36.321222+0000","last_became_active":"2024-05-09T09:06:41.131052+0000","last_became_peered":"2024-05-09T09:06:41.131052+0000","last_unstale":"2024-05-09T09:07:36.321222+0000","last_undegraded":"2024-05-09T09:07:36.321222+0000","last_fullsized":"2024-05-09T09:07:36.321222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:22.200079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321835+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:07:36.321835+0000","last_peered":"2024-05-09T09:07:36.321835+0000","last_clean":"2024-05-09T09:07:36.321835+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:07:36.321835+0000","last_undegraded":"2024-05-09T09:07:36.321835+0000","last_fullsized":"2024-05-09T09:07:36.321835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320941+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:07:36.320941+0000","last_peered":"2024-05-09T09:07:36.320941+0000","last_clean":"2024-05-09T09:07:36.320941+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:07:36.320941+0000","last_undegraded":"2024-05-09T09:07:36.320941+0000","last_fullsized":"2024-05-09T09:07:36.320941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434948+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:07:36.434948+0000","last_peered":"2024-05-09T09:07:36.434948+0000","last_clean":"2024-05-09T09:07:36.434948+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:07:36.434948+0000","last_undegraded":"2024-05-09T09:07:36.434948+0000","last_fullsized":"2024-05-09T09:07:36.434948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316474+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:07:36.316474+0000","last_peered":"2024-05-09T09:07:36.316474+0000","last_clean":"2024-05-09T09:07:36.316474+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:07:36.316474+0000","last_undegraded":"2024-05-09T09:07:36.316474+0000","last_fullsized":"2024-05-09T09:07:36.316474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318828+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:07:36.318828+0000","last_peered":"2024-05-09T09:07:36.318828+0000","last_clean":"2024-05-09T09:07:36.318828+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:07:36.318828+0000","last_undegraded":"2024-05-09T09:07:36.318828+0000","last_fullsized":"2024-05-09T09:07:36.318828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64800,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":444,"num_read_kb":290,"num_write":157,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":503896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1152332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":508206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":504008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1099548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":499286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":560386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":500674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":561736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":499480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":503936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1650688,"data_stored":1095136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:08:22.708 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-09T09:08:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:22 smithi179 ceph-mon[34086]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:22 smithi179 ceph-mon[34086]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:08:23.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:22 smithi139 ceph-mon[33599]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:23.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:22 smithi139 ceph-mon[33599]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:08:23.089 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:22 smithi064 ceph-mon[30149]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:23.089 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:22 smithi064 ceph-mon[30149]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:08:24.739 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:24 smithi064 ceph-mon[30149]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 -- 172.21.15.64:0/3662803245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f401025d0 msgr2=0x7f4f401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3662803245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f401025d0 0x7f4f401029b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009990 tx=0x7f4f30009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 -- 172.21.15.64:0/3662803245 shutdown_connections 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3662803245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 0x7f4f4010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3662803245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f401025d0 0x7f4f401029b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3662803245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f40102f80 0x7f4f40106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.745+0000 7f4f451a6700 1 -- 172.21.15.64:0/3662803245 >> 172.21.15.64:0/3662803245 conn(0x7f4f40079480 msgr2=0x7f4f40079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.746+0000 7f4f451a6700 1 -- 172.21.15.64:0/3662803245 shutdown_connections 2024-05-09T09:08:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.746+0000 7f4f451a6700 1 -- 172.21.15.64:0/3662803245 wait complete. 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.746+0000 7f4f451a6700 1 Processor -- start 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.746+0000 7f4f451a6700 1 -- start start 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f451a6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f401025d0 0x7f4f4019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f451a6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 0x7f4f4019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f451a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 0x7f4f40198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f451a6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401124d0 con 0x7f4f40102f80 2024-05-09T09:08:24.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f451a6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112350 con 0x7f4f401025d0 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f451a6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112650 con 0x7f4f40107660 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 0x7f4f4019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 0x7f4f4019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54324/0 (socket says 172.21.15.64:54324) 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 -- 172.21.15.64:0/3358723660 learned_addr learned my addr 172.21.15.64:0/3358723660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3ed9d700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f401025d0 0x7f4f4019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3f59e700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 0x7f4f40198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 -- 172.21.15.64:0/3358723660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 msgr2=0x7f4f40198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 0x7f4f40198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 -- 172.21.15.64:0/3358723660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f401025d0 msgr2=0x7f4f4019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f401025d0 0x7f4f4019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 -- 172.21.15.64:0/3358723660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40198ff0 con 0x7f4f40102f80 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3f59e700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 0x7f4f40198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.747+0000 7f4f3e59c700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 0x7f4f4019eb80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c9e0 tx=0x7f4f3400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:24.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.748+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34007aa0 con 0x7f4f40102f80 2024-05-09T09:08:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.748+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f34007c40 con 0x7f4f40102f80 2024-05-09T09:08:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.748+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34018660 con 0x7f4f40102f80 2024-05-09T09:08:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.748+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f401992e0 con 0x7f4f40102f80 2024-05-09T09:08:24.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.748+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a55c0 con 0x7f4f40102f80 2024-05-09T09:08:24.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.749+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40103a60 con 0x7f4f40102f80 2024-05-09T09:08:24.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.750+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4f3400f450 con 0x7f4f40102f80 2024-05-09T09:08:24.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.751+0000 7f4f27fff700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4f280a2cb0 0x7f4f280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:24.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.751+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4f340cf4f0 con 0x7f4f40102f80 2024-05-09T09:08:24.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.751+0000 7f4f3ed9d700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4f280a2cb0 0x7f4f280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:24.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.751+0000 7f4f3ed9d700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4f280a2cb0 0x7f4f280a5170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000b5c0 tx=0x7f4f30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:24.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.755+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f34014030 con 0x7f4f40102f80 2024-05-09T09:08:24.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.956+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f40199c40 con 0x7f4f280a2cb0 2024-05-09T09:08:24.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.965+0000 7f4f27fff700 1 -- 172.21.15.64:0/3358723660 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340365 (secure 0 0 0) 0x7f4f40199c40 con 0x7f4f280a2cb0 2024-05-09T09:08:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4f280a2cb0 msgr2=0x7f4f280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4f280a2cb0 0x7f4f280a5170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000b5c0 tx=0x7f4f30005960 comp rx=0 tx=0).stop 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 msgr2=0x7f4f4019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 0x7f4f4019eb80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c9e0 tx=0x7f4f3400ccf0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 shutdown_connections 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4f280a2cb0 0x7f4f280a5170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f40107660 0x7f4f40198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f401025d0 0x7f4f4019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 --2- 172.21.15.64:0/3358723660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f40102f80 0x7f4f4019eb80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 >> 172.21.15.64:0/3358723660 conn(0x7f4f40079480 msgr2=0x7f4f40100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 shutdown_connections 2024-05-09T09:08:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:24.969+0000 7f4f451a6700 1 -- 172.21.15.64:0/3358723660 wait complete. 2024-05-09T09:08:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:08:24.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:24 smithi179 ceph-mon[34086]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:25.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:24 smithi139 ceph-mon[33599]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:25.740 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-09T09:08:23.123180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655402,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":595,"num_read_kb":414,"num_write":340,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392524,"kb_used_data":14180,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549300,"statfs":{"total":1151940427776,"available":1151538483200,"internally_reserved":0,"allocated":14520320,"data_stored":7988624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318971+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:07:36.318971+0000","last_peered":"2024-05-09T09:07:36.318971+0000","last_clean":"2024-05-09T09:07:36.318971+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:07:36.318971+0000","last_undegraded":"2024-05-09T09:07:36.318971+0000","last_fullsized":"2024-05-09T09:07:36.318971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317309+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:07:36.317309+0000","last_peered":"2024-05-09T09:07:36.317309+0000","last_clean":"2024-05-09T09:07:36.317309+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:07:36.317309+0000","last_undegraded":"2024-05-09T09:07:36.317309+0000","last_fullsized":"2024-05-09T09:07:36.317309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316386+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:07:36.316386+0000","last_peered":"2024-05-09T09:07:36.316386+0000","last_clean":"2024-05-09T09:07:36.316386+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:07:36.316386+0000","last_undegraded":"2024-05-09T09:07:36.316386+0000","last_fullsized":"2024-05-09T09:07:36.316386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315544+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:07:36.315544+0000","last_peered":"2024-05-09T09:07:36.315544+0000","last_clean":"2024-05-09T09:07:36.315544+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:07:36.315544+0000","last_undegraded":"2024-05-09T09:07:36.315544+0000","last_fullsized":"2024-05-09T09:07:36.315544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315448+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:07:36.315448+0000","last_peered":"2024-05-09T09:07:36.315448+0000","last_clean":"2024-05-09T09:07:36.315448+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:07:36.315448+0000","last_undegraded":"2024-05-09T09:07:36.315448+0000","last_fullsized":"2024-05-09T09:07:36.315448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316822+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:07:36.316822+0000","last_peered":"2024-05-09T09:07:36.316822+0000","last_clean":"2024-05-09T09:07:36.316822+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:07:36.316822+0000","last_undegraded":"2024-05-09T09:07:36.316822+0000","last_fullsized":"2024-05-09T09:07:36.316822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613318+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:07:36.613318+0000","last_peered":"2024-05-09T09:07:36.613318+0000","last_clean":"2024-05-09T09:07:36.613318+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:07:36.613318+0000","last_undegraded":"2024-05-09T09:07:36.613318+0000","last_fullsized":"2024-05-09T09:07:36.613318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320784+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:07:36.320784+0000","last_peered":"2024-05-09T09:07:36.320784+0000","last_clean":"2024-05-09T09:07:36.320784+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:07:36.320784+0000","last_undegraded":"2024-05-09T09:07:36.320784+0000","last_fullsized":"2024-05-09T09:07:36.320784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435257+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:07:36.435257+0000","last_peered":"2024-05-09T09:07:36.435257+0000","last_clean":"2024-05-09T09:07:36.435257+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:07:36.435257+0000","last_undegraded":"2024-05-09T09:07:36.435257+0000","last_fullsized":"2024-05-09T09:07:36.435257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317130+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:07:36.317130+0000","last_peered":"2024-05-09T09:07:36.317130+0000","last_clean":"2024-05-09T09:07:36.317130+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:07:36.317130+0000","last_undegraded":"2024-05-09T09:07:36.317130+0000","last_fullsized":"2024-05-09T09:07:36.317130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613834+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:07:36.613834+0000","last_peered":"2024-05-09T09:07:36.613834+0000","last_clean":"2024-05-09T09:07:36.613834+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:07:36.613834+0000","last_undegraded":"2024-05-09T09:07:36.613834+0000","last_fullsized":"2024-05-09T09:07:36.613834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317979+0000","last_change":"2024-05-09T09:07:36.317979+0000","last_active":"2024-05-09T09:07:36.317979+0000","last_peered":"2024-05-09T09:07:36.317979+0000","last_clean":"2024-05-09T09:07:36.317979+0000","last_became_active":"2024-05-09T09:07:36.317572+0000","last_became_peered":"2024-05-09T09:07:36.317572+0000","last_unstale":"2024-05-09T09:07:36.317979+0000","last_undegraded":"2024-05-09T09:07:36.317979+0000","last_fullsized":"2024-05-09T09:07:36.317979+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:45.499023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316654+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:07:36.316654+0000","last_peered":"2024-05-09T09:07:36.316654+0000","last_clean":"2024-05-09T09:07:36.316654+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:07:36.316654+0000","last_undegraded":"2024-05-09T09:07:36.316654+0000","last_fullsized":"2024-05-09T09:07:36.316654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316248+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:07:36.316248+0000","last_peered":"2024-05-09T09:07:36.316248+0000","last_clean":"2024-05-09T09:07:36.316248+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:07:36.316248+0000","last_undegraded":"2024-05-09T09:07:36.316248+0000","last_fullsized":"2024-05-09T09:07:36.316248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316330+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:07:36.316330+0000","last_peered":"2024-05-09T09:07:36.316330+0000","last_clean":"2024-05-09T09:07:36.316330+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:07:36.316330+0000","last_undegraded":"2024-05-09T09:07:36.316330+0000","last_fullsized":"2024-05-09T09:07:36.316330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434738+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:07:36.434738+0000","last_peered":"2024-05-09T09:07:36.434738+0000","last_clean":"2024-05-09T09:07:36.434738+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:07:36.434738+0000","last_undegraded":"2024-05-09T09:07:36.434738+0000","last_fullsized":"2024-05-09T09:07:36.434738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318971+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:07:36.318971+0000","last_peered":"2024-05-09T09:07:36.318971+0000","last_clean":"2024-05-09T09:07:36.318971+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:07:36.318971+0000","last_undegraded":"2024-05-09T09:07:36.318971+0000","last_fullsized":"2024-05-09T09:07:36.318971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314247+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:07:36.314247+0000","last_peered":"2024-05-09T09:07:36.314247+0000","last_clean":"2024-05-09T09:07:36.314247+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:07:36.314247+0000","last_undegraded":"2024-05-09T09:07:36.314247+0000","last_fullsized":"2024-05-09T09:07:36.314247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323149+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:07:36.323149+0000","last_peered":"2024-05-09T09:07:36.323149+0000","last_clean":"2024-05-09T09:07:36.323149+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:07:36.323149+0000","last_undegraded":"2024-05-09T09:07:36.323149+0000","last_fullsized":"2024-05-09T09:07:36.323149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323847+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:07:36.323847+0000","last_peered":"2024-05-09T09:07:36.323847+0000","last_clean":"2024-05-09T09:07:36.323847+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:07:36.323847+0000","last_undegraded":"2024-05-09T09:07:36.323847+0000","last_fullsized":"2024-05-09T09:07:36.323847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314269+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:07:36.314269+0000","last_peered":"2024-05-09T09:07:36.314269+0000","last_clean":"2024-05-09T09:07:36.314269+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:07:36.314269+0000","last_undegraded":"2024-05-09T09:07:36.314269+0000","last_fullsized":"2024-05-09T09:07:36.314269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320464+0000","last_change":"2024-05-09T09:06:41.137679+0000","last_active":"2024-05-09T09:07:36.320464+0000","last_peered":"2024-05-09T09:07:36.320464+0000","last_clean":"2024-05-09T09:07:36.320464+0000","last_became_active":"2024-05-09T09:06:41.137353+0000","last_became_peered":"2024-05-09T09:06:41.137353+0000","last_unstale":"2024-05-09T09:07:36.320464+0000","last_undegraded":"2024-05-09T09:07:36.320464+0000","last_fullsized":"2024-05-09T09:07:36.320464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:36.080021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314517+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:07:36.314517+0000","last_peered":"2024-05-09T09:07:36.314517+0000","last_clean":"2024-05-09T09:07:36.314517+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:07:36.314517+0000","last_undegraded":"2024-05-09T09:07:36.314517+0000","last_fullsized":"2024-05-09T09:07:36.314517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321638+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:07:36.321638+0000","last_peered":"2024-05-09T09:07:36.321638+0000","last_clean":"2024-05-09T09:07:36.321638+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:07:36.321638+0000","last_undegraded":"2024-05-09T09:07:36.321638+0000","last_fullsized":"2024-05-09T09:07:36.321638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317330+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:07:36.317330+0000","last_peered":"2024-05-09T09:07:36.317330+0000","last_clean":"2024-05-09T09:07:36.317330+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:07:36.317330+0000","last_undegraded":"2024-05-09T09:07:36.317330+0000","last_fullsized":"2024-05-09T09:07:36.317330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314905+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:07:36.314905+0000","last_peered":"2024-05-09T09:07:36.314905+0000","last_clean":"2024-05-09T09:07:36.314905+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:07:36.314905+0000","last_undegraded":"2024-05-09T09:07:36.314905+0000","last_fullsized":"2024-05-09T09:07:36.314905+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315959+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:07:36.315959+0000","last_peered":"2024-05-09T09:07:36.315959+0000","last_clean":"2024-05-09T09:07:36.315959+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:07:36.315959+0000","last_undegraded":"2024-05-09T09:07:36.315959+0000","last_fullsized":"2024-05-09T09:07:36.315959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435121+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:07:36.435121+0000","last_peered":"2024-05-09T09:07:36.435121+0000","last_clean":"2024-05-09T09:07:36.435121+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:07:36.435121+0000","last_undegraded":"2024-05-09T09:07:36.435121+0000","last_fullsized":"2024-05-09T09:07:36.435121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314645+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:07:36.314645+0000","last_peered":"2024-05-09T09:07:36.314645+0000","last_clean":"2024-05-09T09:07:36.314645+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:07:36.314645+0000","last_undegraded":"2024-05-09T09:07:36.314645+0000","last_fullsized":"2024-05-09T09:07:36.314645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316278+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:07:36.316278+0000","last_peered":"2024-05-09T09:07:36.316278+0000","last_clean":"2024-05-09T09:07:36.316278+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:07:36.316278+0000","last_undegraded":"2024-05-09T09:07:36.316278+0000","last_fullsized":"2024-05-09T09:07:36.316278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323676+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:07:36.323676+0000","last_peered":"2024-05-09T09:07:36.323676+0000","last_clean":"2024-05-09T09:07:36.323676+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:07:36.323676+0000","last_undegraded":"2024-05-09T09:07:36.323676+0000","last_fullsized":"2024-05-09T09:07:36.323676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322976+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:07:36.322976+0000","last_peered":"2024-05-09T09:07:36.322976+0000","last_clean":"2024-05-09T09:07:36.322976+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:07:36.322976+0000","last_undegraded":"2024-05-09T09:07:36.322976+0000","last_fullsized":"2024-05-09T09:07:36.322976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:21.344606+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:08:21.344606+0000","last_peered":"2024-05-09T09:08:21.344606+0000","last_clean":"2024-05-09T09:08:21.344606+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:08:21.344606+0000","last_undegraded":"2024-05-09T09:08:21.344606+0000","last_fullsized":"2024-05-09T09:08:21.344606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.615226+0000","last_change":"2024-05-09T09:07:36.615226+0000","last_active":"2024-05-09T09:07:36.615226+0000","last_peered":"2024-05-09T09:07:36.615226+0000","last_clean":"2024-05-09T09:07:36.615226+0000","last_became_active":"2024-05-09T09:07:36.614762+0000","last_became_peered":"2024-05-09T09:07:36.614762+0000","last_unstale":"2024-05-09T09:07:36.615226+0000","last_undegraded":"2024-05-09T09:07:36.615226+0000","last_fullsized":"2024-05-09T09:07:36.615226+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:29.045379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322498+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:07:36.322498+0000","last_peered":"2024-05-09T09:07:36.322498+0000","last_clean":"2024-05-09T09:07:36.322498+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:07:36.322498+0000","last_undegraded":"2024-05-09T09:07:36.322498+0000","last_fullsized":"2024-05-09T09:07:36.322498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316836+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:07:36.316836+0000","last_peered":"2024-05-09T09:07:36.316836+0000","last_clean":"2024-05-09T09:07:36.316836+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:07:36.316836+0000","last_undegraded":"2024-05-09T09:07:36.316836+0000","last_fullsized":"2024-05-09T09:07:36.316836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323259+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:07:36.323259+0000","last_peered":"2024-05-09T09:07:36.323259+0000","last_clean":"2024-05-09T09:07:36.323259+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:07:36.323259+0000","last_undegraded":"2024-05-09T09:07:36.323259+0000","last_fullsized":"2024-05-09T09:07:36.323259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318835+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:07:36.318835+0000","last_peered":"2024-05-09T09:07:36.318835+0000","last_clean":"2024-05-09T09:07:36.318835+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:07:36.318835+0000","last_undegraded":"2024-05-09T09:07:36.318835+0000","last_fullsized":"2024-05-09T09:07:36.318835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316166+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:07:36.316166+0000","last_peered":"2024-05-09T09:07:36.316166+0000","last_clean":"2024-05-09T09:07:36.316166+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:07:36.316166+0000","last_undegraded":"2024-05-09T09:07:36.316166+0000","last_fullsized":"2024-05-09T09:07:36.316166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320899+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:07:36.320899+0000","last_peered":"2024-05-09T09:07:36.320899+0000","last_clean":"2024-05-09T09:07:36.320899+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:07:36.320899+0000","last_undegraded":"2024-05-09T09:07:36.320899+0000","last_fullsized":"2024-05-09T09:07:36.320899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316250+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:07:36.316250+0000","last_peered":"2024-05-09T09:07:36.316250+0000","last_clean":"2024-05-09T09:07:36.316250+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:07:36.316250+0000","last_undegraded":"2024-05-09T09:07:36.316250+0000","last_fullsized":"2024-05-09T09:07:36.316250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321720+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:07:36.321720+0000","last_peered":"2024-05-09T09:07:36.321720+0000","last_clean":"2024-05-09T09:07:36.321720+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:07:36.321720+0000","last_undegraded":"2024-05-09T09:07:36.321720+0000","last_fullsized":"2024-05-09T09:07:36.321720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.313670+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:07:36.313670+0000","last_peered":"2024-05-09T09:07:36.313670+0000","last_clean":"2024-05-09T09:07:36.313670+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:07:36.313670+0000","last_undegraded":"2024-05-09T09:07:36.313670+0000","last_fullsized":"2024-05-09T09:07:36.313670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320638+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:07:36.320638+0000","last_peered":"2024-05-09T09:07:36.320638+0000","last_clean":"2024-05-09T09:07:36.320638+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:07:36.320638+0000","last_undegraded":"2024-05-09T09:07:36.320638+0000","last_fullsized":"2024-05-09T09:07:36.320638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:21.343858+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:08:21.343858+0000","last_peered":"2024-05-09T09:08:21.343858+0000","last_clean":"2024-05-09T09:08:21.343858+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:08:21.343858+0000","last_undegraded":"2024-05-09T09:08:21.343858+0000","last_fullsized":"2024-05-09T09:08:21.343858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435065+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:07:36.435065+0000","last_peered":"2024-05-09T09:07:36.435065+0000","last_clean":"2024-05-09T09:07:36.435065+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:07:36.435065+0000","last_undegraded":"2024-05-09T09:07:36.435065+0000","last_fullsized":"2024-05-09T09:07:36.435065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.313514+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:07:36.313514+0000","last_peered":"2024-05-09T09:07:36.313514+0000","last_clean":"2024-05-09T09:07:36.313514+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:07:36.313514+0000","last_undegraded":"2024-05-09T09:07:36.313514+0000","last_fullsized":"2024-05-09T09:07:36.313514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613769+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:07:36.613769+0000","last_peered":"2024-05-09T09:07:36.613769+0000","last_clean":"2024-05-09T09:07:36.613769+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:07:36.613769+0000","last_undegraded":"2024-05-09T09:07:36.613769+0000","last_fullsized":"2024-05-09T09:07:36.613769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317329+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:07:36.317329+0000","last_peered":"2024-05-09T09:07:36.317329+0000","last_clean":"2024-05-09T09:07:36.317329+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:07:36.317329+0000","last_undegraded":"2024-05-09T09:07:36.317329+0000","last_fullsized":"2024-05-09T09:07:36.317329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.436587+0000","last_change":"2024-05-09T09:07:36.436587+0000","last_active":"2024-05-09T09:07:36.436587+0000","last_peered":"2024-05-09T09:07:36.436587+0000","last_clean":"2024-05-09T09:07:36.436587+0000","last_became_active":"2024-05-09T09:07:36.436083+0000","last_became_peered":"2024-05-09T09:07:36.436083+0000","last_unstale":"2024-05-09T09:07:36.436587+0000","last_undegraded":"2024-05-09T09:07:36.436587+0000","last_fullsized":"2024-05-09T09:07:36.436587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:02:34.904307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323557+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:07:36.323557+0000","last_peered":"2024-05-09T09:07:36.323557+0000","last_clean":"2024-05-09T09:07:36.323557+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:07:36.323557+0000","last_undegraded":"2024-05-09T09:07:36.323557+0000","last_fullsized":"2024-05-09T09:07:36.323557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.435284+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:07:36.435284+0000","last_peered":"2024-05-09T09:07:36.435284+0000","last_clean":"2024-05-09T09:07:36.435284+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:07:36.435284+0000","last_undegraded":"2024-05-09T09:07:36.435284+0000","last_fullsized":"2024-05-09T09:07:36.435284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.325270+0000","last_change":"2024-05-09T09:07:36.324821+0000","last_active":"2024-05-09T09:07:36.325270+0000","last_peered":"2024-05-09T09:07:36.325270+0000","last_clean":"2024-05-09T09:07:36.325270+0000","last_became_active":"2024-05-09T09:07:36.324478+0000","last_became_peered":"2024-05-09T09:07:36.324478+0000","last_unstale":"2024-05-09T09:07:36.325270+0000","last_undegraded":"2024-05-09T09:07:36.325270+0000","last_fullsized":"2024-05-09T09:07:36.325270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:25.476000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:22.363386+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:08:22.363386+0000","last_peered":"2024-05-09T09:08:22.363386+0000","last_clean":"2024-05-09T09:08:22.363386+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:08:22.363386+0000","last_undegraded":"2024-05-09T09:08:22.363386+0000","last_fullsized":"2024-05-09T09:08:22.363386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61190,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":157,"num_read_kb":139,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315606+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:07:36.315606+0000","last_peered":"2024-05-09T09:07:36.315606+0000","last_clean":"2024-05-09T09:07:36.315606+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:07:36.315606+0000","last_undegraded":"2024-05-09T09:07:36.315606+0000","last_fullsized":"2024-05-09T09:07:36.315606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.324086+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:07:36.324086+0000","last_peered":"2024-05-09T09:07:36.324086+0000","last_clean":"2024-05-09T09:07:36.324086+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:07:36.324086+0000","last_undegraded":"2024-05-09T09:07:36.324086+0000","last_fullsized":"2024-05-09T09:07:36.324086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322308+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:07:36.322308+0000","last_peered":"2024-05-09T09:07:36.322308+0000","last_clean":"2024-05-09T09:07:36.322308+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:07:36.322308+0000","last_undegraded":"2024-05-09T09:07:36.322308+0000","last_fullsized":"2024-05-09T09:07:36.322308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316458+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:07:36.316458+0000","last_peered":"2024-05-09T09:07:36.316458+0000","last_clean":"2024-05-09T09:07:36.316458+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:07:36.316458+0000","last_undegraded":"2024-05-09T09:07:36.316458+0000","last_fullsized":"2024-05-09T09:07:36.316458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318689+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:07:36.318689+0000","last_peered":"2024-05-09T09:07:36.318689+0000","last_clean":"2024-05-09T09:07:36.318689+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:07:36.318689+0000","last_undegraded":"2024-05-09T09:07:36.318689+0000","last_fullsized":"2024-05-09T09:07:36.318689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322949+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:07:36.322949+0000","last_peered":"2024-05-09T09:07:36.322949+0000","last_clean":"2024-05-09T09:07:36.322949+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:07:36.322949+0000","last_undegraded":"2024-05-09T09:07:36.322949+0000","last_fullsized":"2024-05-09T09:07:36.322949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613599+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:07:36.613599+0000","last_peered":"2024-05-09T09:07:36.613599+0000","last_clean":"2024-05-09T09:07:36.613599+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:07:36.613599+0000","last_undegraded":"2024-05-09T09:07:36.613599+0000","last_fullsized":"2024-05-09T09:07:36.613599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314113+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:07:36.314113+0000","last_peered":"2024-05-09T09:07:36.314113+0000","last_clean":"2024-05-09T09:07:36.314113+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:07:36.314113+0000","last_undegraded":"2024-05-09T09:07:36.314113+0000","last_fullsized":"2024-05-09T09:07:36.314113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:17.338542+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:08:17.338542+0000","last_peered":"2024-05-09T09:08:17.338542+0000","last_clean":"2024-05-09T09:08:17.338542+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:08:17.338542+0000","last_undegraded":"2024-05-09T09:08:17.338542+0000","last_fullsized":"2024-05-09T09:08:17.338542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613401+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:07:36.613401+0000","last_peered":"2024-05-09T09:07:36.613401+0000","last_clean":"2024-05-09T09:07:36.613401+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:07:36.613401+0000","last_undegraded":"2024-05-09T09:07:36.613401+0000","last_fullsized":"2024-05-09T09:07:36.613401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323604+0000","last_change":"2024-05-09T09:07:36.323604+0000","last_active":"2024-05-09T09:07:36.323604+0000","last_peered":"2024-05-09T09:07:36.323604+0000","last_clean":"2024-05-09T09:07:36.323604+0000","last_became_active":"2024-05-09T09:07:36.323254+0000","last_became_peered":"2024-05-09T09:07:36.323254+0000","last_unstale":"2024-05-09T09:07:36.323604+0000","last_undegraded":"2024-05-09T09:07:36.323604+0000","last_fullsized":"2024-05-09T09:07:36.323604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:31:18.746847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315125+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:07:36.315125+0000","last_peered":"2024-05-09T09:07:36.315125+0000","last_clean":"2024-05-09T09:07:36.315125+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:07:36.315125+0000","last_undegraded":"2024-05-09T09:07:36.315125+0000","last_fullsized":"2024-05-09T09:07:36.315125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321357+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:07:36.321357+0000","last_peered":"2024-05-09T09:07:36.321357+0000","last_clean":"2024-05-09T09:07:36.321357+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:07:36.321357+0000","last_undegraded":"2024-05-09T09:07:36.321357+0000","last_fullsized":"2024-05-09T09:07:36.321357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321561+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:07:36.321561+0000","last_peered":"2024-05-09T09:07:36.321561+0000","last_clean":"2024-05-09T09:07:36.321561+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:07:36.321561+0000","last_undegraded":"2024-05-09T09:07:36.321561+0000","last_fullsized":"2024-05-09T09:07:36.321561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.331734+0000","last_change":"2024-05-09T09:07:36.331318+0000","last_active":"2024-05-09T09:07:36.331734+0000","last_peered":"2024-05-09T09:07:36.331734+0000","last_clean":"2024-05-09T09:07:36.331734+0000","last_became_active":"2024-05-09T09:07:36.323269+0000","last_became_peered":"2024-05-09T09:07:36.323269+0000","last_unstale":"2024-05-09T09:07:36.331734+0000","last_undegraded":"2024-05-09T09:07:36.331734+0000","last_fullsized":"2024-05-09T09:07:36.331734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:25:53.779933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.319495+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:07:36.319495+0000","last_peered":"2024-05-09T09:07:36.319495+0000","last_clean":"2024-05-09T09:07:36.319495+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:07:36.319495+0000","last_undegraded":"2024-05-09T09:07:36.319495+0000","last_fullsized":"2024-05-09T09:07:36.319495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315897+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:07:36.315897+0000","last_peered":"2024-05-09T09:07:36.315897+0000","last_clean":"2024-05-09T09:07:36.315897+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:07:36.315897+0000","last_undegraded":"2024-05-09T09:07:36.315897+0000","last_fullsized":"2024-05-09T09:07:36.315897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.313866+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:07:36.313866+0000","last_peered":"2024-05-09T09:07:36.313866+0000","last_clean":"2024-05-09T09:07:36.313866+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:07:36.313866+0000","last_undegraded":"2024-05-09T09:07:36.313866+0000","last_fullsized":"2024-05-09T09:07:36.313866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316459+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:07:36.316459+0000","last_peered":"2024-05-09T09:07:36.316459+0000","last_clean":"2024-05-09T09:07:36.316459+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:07:36.316459+0000","last_undegraded":"2024-05-09T09:07:36.316459+0000","last_fullsized":"2024-05-09T09:07:36.316459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317147+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:07:36.317147+0000","last_peered":"2024-05-09T09:07:36.317147+0000","last_clean":"2024-05-09T09:07:36.317147+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:07:36.317147+0000","last_undegraded":"2024-05-09T09:07:36.317147+0000","last_fullsized":"2024-05-09T09:07:36.317147+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613401+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:07:36.613401+0000","last_peered":"2024-05-09T09:07:36.613401+0000","last_clean":"2024-05-09T09:07:36.613401+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:07:36.613401+0000","last_undegraded":"2024-05-09T09:07:36.613401+0000","last_fullsized":"2024-05-09T09:07:36.613401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316793+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:07:36.316793+0000","last_peered":"2024-05-09T09:07:36.316793+0000","last_clean":"2024-05-09T09:07:36.316793+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:07:36.316793+0000","last_undegraded":"2024-05-09T09:07:36.316793+0000","last_fullsized":"2024-05-09T09:07:36.316793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316929+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:07:36.316929+0000","last_peered":"2024-05-09T09:07:36.316929+0000","last_clean":"2024-05-09T09:07:36.316929+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:07:36.316929+0000","last_undegraded":"2024-05-09T09:07:36.316929+0000","last_fullsized":"2024-05-09T09:07:36.316929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321954+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:07:36.321954+0000","last_peered":"2024-05-09T09:07:36.321954+0000","last_clean":"2024-05-09T09:07:36.321954+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:07:36.321954+0000","last_undegraded":"2024-05-09T09:07:36.321954+0000","last_fullsized":"2024-05-09T09:07:36.321954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:08:20.341960+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:08:20.341960+0000","last_peered":"2024-05-09T09:08:20.341960+0000","last_clean":"2024-05-09T09:08:20.341960+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:08:20.341960+0000","last_undegraded":"2024-05-09T09:08:20.341960+0000","last_fullsized":"2024-05-09T09:08:20.341960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613567+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:07:36.613567+0000","last_peered":"2024-05-09T09:07:36.613567+0000","last_clean":"2024-05-09T09:07:36.613567+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:07:36.613567+0000","last_undegraded":"2024-05-09T09:07:36.613567+0000","last_fullsized":"2024-05-09T09:07:36.613567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320275+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:07:36.320275+0000","last_peered":"2024-05-09T09:07:36.320275+0000","last_clean":"2024-05-09T09:07:36.320275+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:07:36.320275+0000","last_undegraded":"2024-05-09T09:07:36.320275+0000","last_fullsized":"2024-05-09T09:07:36.320275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315054+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:07:36.315054+0000","last_peered":"2024-05-09T09:07:36.315054+0000","last_clean":"2024-05-09T09:07:36.315054+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:07:36.315054+0000","last_undegraded":"2024-05-09T09:07:36.315054+0000","last_fullsized":"2024-05-09T09:07:36.315054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613351+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:07:36.613351+0000","last_peered":"2024-05-09T09:07:36.613351+0000","last_clean":"2024-05-09T09:07:36.613351+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:07:36.613351+0000","last_undegraded":"2024-05-09T09:07:36.613351+0000","last_fullsized":"2024-05-09T09:07:36.613351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322233+0000","last_change":"2024-05-09T09:07:36.319419+0000","last_active":"2024-05-09T09:07:36.322233+0000","last_peered":"2024-05-09T09:07:36.322233+0000","last_clean":"2024-05-09T09:07:36.322233+0000","last_became_active":"2024-05-09T09:07:36.318825+0000","last_became_peered":"2024-05-09T09:07:36.318825+0000","last_unstale":"2024-05-09T09:07:36.322233+0000","last_undegraded":"2024-05-09T09:07:36.322233+0000","last_fullsized":"2024-05-09T09:07:36.322233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:21:10.289893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315919+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:07:36.315919+0000","last_peered":"2024-05-09T09:07:36.315919+0000","last_clean":"2024-05-09T09:07:36.315919+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:07:36.315919+0000","last_undegraded":"2024-05-09T09:07:36.315919+0000","last_fullsized":"2024-05-09T09:07:36.315919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613269+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:07:36.613269+0000","last_peered":"2024-05-09T09:07:36.613269+0000","last_clean":"2024-05-09T09:07:36.613269+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:07:36.613269+0000","last_undegraded":"2024-05-09T09:07:36.613269+0000","last_fullsized":"2024-05-09T09:07:36.613269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318515+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:07:36.318515+0000","last_peered":"2024-05-09T09:07:36.318515+0000","last_clean":"2024-05-09T09:07:36.318515+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:07:36.318515+0000","last_undegraded":"2024-05-09T09:07:36.318515+0000","last_fullsized":"2024-05-09T09:07:36.318515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315753+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:07:36.315753+0000","last_peered":"2024-05-09T09:07:36.315753+0000","last_clean":"2024-05-09T09:07:36.315753+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:07:36.315753+0000","last_undegraded":"2024-05-09T09:07:36.315753+0000","last_fullsized":"2024-05-09T09:07:36.315753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317307+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:07:36.317307+0000","last_peered":"2024-05-09T09:07:36.317307+0000","last_clean":"2024-05-09T09:07:36.317307+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:07:36.317307+0000","last_undegraded":"2024-05-09T09:07:36.317307+0000","last_fullsized":"2024-05-09T09:07:36.317307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320830+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:07:36.320830+0000","last_peered":"2024-05-09T09:07:36.320830+0000","last_clean":"2024-05-09T09:07:36.320830+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:07:36.320830+0000","last_undegraded":"2024-05-09T09:07:36.320830+0000","last_fullsized":"2024-05-09T09:07:36.320830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314280+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:07:36.314280+0000","last_peered":"2024-05-09T09:07:36.314280+0000","last_clean":"2024-05-09T09:07:36.314280+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:07:36.314280+0000","last_undegraded":"2024-05-09T09:07:36.314280+0000","last_fullsized":"2024-05-09T09:07:36.314280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321136+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:07:36.321136+0000","last_peered":"2024-05-09T09:07:36.321136+0000","last_clean":"2024-05-09T09:07:36.321136+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:07:36.321136+0000","last_undegraded":"2024-05-09T09:07:36.321136+0000","last_fullsized":"2024-05-09T09:07:36.321136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321815+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:07:36.321815+0000","last_peered":"2024-05-09T09:07:36.321815+0000","last_clean":"2024-05-09T09:07:36.321815+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:07:36.321815+0000","last_undegraded":"2024-05-09T09:07:36.321815+0000","last_fullsized":"2024-05-09T09:07:36.321815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315740+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:07:36.315740+0000","last_peered":"2024-05-09T09:07:36.315740+0000","last_clean":"2024-05-09T09:07:36.315740+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:07:36.315740+0000","last_undegraded":"2024-05-09T09:07:36.315740+0000","last_fullsized":"2024-05-09T09:07:36.315740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322492+0000","last_change":"2024-05-09T09:07:36.322492+0000","last_active":"2024-05-09T09:07:36.322492+0000","last_peered":"2024-05-09T09:07:36.322492+0000","last_clean":"2024-05-09T09:07:36.322492+0000","last_became_active":"2024-05-09T09:07:36.322098+0000","last_became_peered":"2024-05-09T09:07:36.322098+0000","last_unstale":"2024-05-09T09:07:36.322492+0000","last_undegraded":"2024-05-09T09:07:36.322492+0000","last_fullsized":"2024-05-09T09:07:36.322492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:55.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315198+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:07:36.315198+0000","last_peered":"2024-05-09T09:07:36.315198+0000","last_clean":"2024-05-09T09:07:36.315198+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:07:36.315198+0000","last_undegraded":"2024-05-09T09:07:36.315198+0000","last_fullsized":"2024-05-09T09:07:36.315198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434745+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:07:36.434745+0000","last_peered":"2024-05-09T09:07:36.434745+0000","last_clean":"2024-05-09T09:07:36.434745+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:07:36.434745+0000","last_undegraded":"2024-05-09T09:07:36.434745+0000","last_fullsized":"2024-05-09T09:07:36.434745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315133+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:07:36.315133+0000","last_peered":"2024-05-09T09:07:36.315133+0000","last_clean":"2024-05-09T09:07:36.315133+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:07:36.315133+0000","last_undegraded":"2024-05-09T09:07:36.315133+0000","last_fullsized":"2024-05-09T09:07:36.315133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.314234+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:07:36.314234+0000","last_peered":"2024-05-09T09:07:36.314234+0000","last_clean":"2024-05-09T09:07:36.314234+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:07:36.314234+0000","last_undegraded":"2024-05-09T09:07:36.314234+0000","last_fullsized":"2024-05-09T09:07:36.314234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320748+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:07:36.320748+0000","last_peered":"2024-05-09T09:07:36.320748+0000","last_clean":"2024-05-09T09:07:36.320748+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:07:36.320748+0000","last_undegraded":"2024-05-09T09:07:36.320748+0000","last_fullsized":"2024-05-09T09:07:36.320748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318276+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:07:36.318276+0000","last_peered":"2024-05-09T09:07:36.318276+0000","last_clean":"2024-05-09T09:07:36.318276+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:07:36.318276+0000","last_undegraded":"2024-05-09T09:07:36.318276+0000","last_fullsized":"2024-05-09T09:07:36.318276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315220+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:07:36.315220+0000","last_peered":"2024-05-09T09:07:36.315220+0000","last_clean":"2024-05-09T09:07:36.315220+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:07:36.315220+0000","last_undegraded":"2024-05-09T09:07:36.315220+0000","last_fullsized":"2024-05-09T09:07:36.315220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.327910+0000","last_change":"2024-05-09T09:07:36.327470+0000","last_active":"2024-05-09T09:07:36.327910+0000","last_peered":"2024-05-09T09:07:36.327910+0000","last_clean":"2024-05-09T09:07:36.327910+0000","last_became_active":"2024-05-09T09:07:36.321516+0000","last_became_peered":"2024-05-09T09:07:36.321516+0000","last_unstale":"2024-05-09T09:07:36.327910+0000","last_undegraded":"2024-05-09T09:07:36.327910+0000","last_fullsized":"2024-05-09T09:07:36.327910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:30.824518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321220+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:07:36.321220+0000","last_peered":"2024-05-09T09:07:36.321220+0000","last_clean":"2024-05-09T09:07:36.321220+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:07:36.321220+0000","last_undegraded":"2024-05-09T09:07:36.321220+0000","last_fullsized":"2024-05-09T09:07:36.321220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315395+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:07:36.315395+0000","last_peered":"2024-05-09T09:07:36.315395+0000","last_clean":"2024-05-09T09:07:36.315395+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:07:36.315395+0000","last_undegraded":"2024-05-09T09:07:36.315395+0000","last_fullsized":"2024-05-09T09:07:36.315395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.315260+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:07:36.315260+0000","last_peered":"2024-05-09T09:07:36.315260+0000","last_clean":"2024-05-09T09:07:36.315260+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:07:36.315260+0000","last_undegraded":"2024-05-09T09:07:36.315260+0000","last_fullsized":"2024-05-09T09:07:36.315260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613369+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:07:36.613369+0000","last_peered":"2024-05-09T09:07:36.613369+0000","last_clean":"2024-05-09T09:07:36.613369+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:07:36.613369+0000","last_undegraded":"2024-05-09T09:07:36.613369+0000","last_fullsized":"2024-05-09T09:07:36.613369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434715+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:07:36.434715+0000","last_peered":"2024-05-09T09:07:36.434715+0000","last_clean":"2024-05-09T09:07:36.434715+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:07:36.434715+0000","last_undegraded":"2024-05-09T09:07:36.434715+0000","last_fullsized":"2024-05-09T09:07:36.434715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322742+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:07:36.322742+0000","last_peered":"2024-05-09T09:07:36.322742+0000","last_clean":"2024-05-09T09:07:36.322742+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:07:36.322742+0000","last_undegraded":"2024-05-09T09:07:36.322742+0000","last_fullsized":"2024-05-09T09:07:36.322742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613855+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:07:36.613855+0000","last_peered":"2024-05-09T09:07:36.613855+0000","last_clean":"2024-05-09T09:07:36.613855+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:07:36.613855+0000","last_undegraded":"2024-05-09T09:07:36.613855+0000","last_fullsized":"2024-05-09T09:07:36.613855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316991+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:07:36.316991+0000","last_peered":"2024-05-09T09:07:36.316991+0000","last_clean":"2024-05-09T09:07:36.316991+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:07:36.316991+0000","last_undegraded":"2024-05-09T09:07:36.316991+0000","last_fullsized":"2024-05-09T09:07:36.316991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323802+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:07:36.323802+0000","last_peered":"2024-05-09T09:07:36.323802+0000","last_clean":"2024-05-09T09:07:36.323802+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:07:36.323802+0000","last_undegraded":"2024-05-09T09:07:36.323802+0000","last_fullsized":"2024-05-09T09:07:36.323802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613706+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:07:36.613706+0000","last_peered":"2024-05-09T09:07:36.613706+0000","last_clean":"2024-05-09T09:07:36.613706+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:07:36.613706+0000","last_undegraded":"2024-05-09T09:07:36.613706+0000","last_fullsized":"2024-05-09T09:07:36.613706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321107+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:07:36.321107+0000","last_peered":"2024-05-09T09:07:36.321107+0000","last_clean":"2024-05-09T09:07:36.321107+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:07:36.321107+0000","last_undegraded":"2024-05-09T09:07:36.321107+0000","last_fullsized":"2024-05-09T09:07:36.321107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316904+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:07:36.316904+0000","last_peered":"2024-05-09T09:07:36.316904+0000","last_clean":"2024-05-09T09:07:36.316904+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:07:36.316904+0000","last_undegraded":"2024-05-09T09:07:36.316904+0000","last_fullsized":"2024-05-09T09:07:36.316904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.613934+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:07:36.613934+0000","last_peered":"2024-05-09T09:07:36.613934+0000","last_clean":"2024-05-09T09:07:36.613934+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:07:36.613934+0000","last_undegraded":"2024-05-09T09:07:36.613934+0000","last_fullsized":"2024-05-09T09:07:36.613934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318373+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:07:36.318373+0000","last_peered":"2024-05-09T09:07:36.318373+0000","last_clean":"2024-05-09T09:07:36.318373+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:07:36.318373+0000","last_undegraded":"2024-05-09T09:07:36.318373+0000","last_fullsized":"2024-05-09T09:07:36.318373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.322230+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:07:36.322230+0000","last_peered":"2024-05-09T09:07:36.322230+0000","last_clean":"2024-05-09T09:07:36.322230+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:07:36.322230+0000","last_undegraded":"2024-05-09T09:07:36.322230+0000","last_fullsized":"2024-05-09T09:07:36.322230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321842+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:07:36.321842+0000","last_peered":"2024-05-09T09:07:36.321842+0000","last_clean":"2024-05-09T09:07:36.321842+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:07:36.321842+0000","last_undegraded":"2024-05-09T09:07:36.321842+0000","last_fullsized":"2024-05-09T09:07:36.321842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321453+0000","last_change":"2024-05-09T09:07:36.321453+0000","last_active":"2024-05-09T09:07:36.321453+0000","last_peered":"2024-05-09T09:07:36.321453+0000","last_clean":"2024-05-09T09:07:36.321453+0000","last_became_active":"2024-05-09T09:07:36.321074+0000","last_became_peered":"2024-05-09T09:07:36.321074+0000","last_unstale":"2024-05-09T09:07:36.321453+0000","last_undegraded":"2024-05-09T09:07:36.321453+0000","last_fullsized":"2024-05-09T09:07:36.321453+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:26.690978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.317413+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:07:36.317413+0000","last_peered":"2024-05-09T09:07:36.317413+0000","last_clean":"2024-05-09T09:07:36.317413+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:07:36.317413+0000","last_undegraded":"2024-05-09T09:07:36.317413+0000","last_fullsized":"2024-05-09T09:07:36.317413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.319356+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:07:36.319356+0000","last_peered":"2024-05-09T09:07:36.319356+0000","last_clean":"2024-05-09T09:07:36.319356+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:07:36.319356+0000","last_undegraded":"2024-05-09T09:07:36.319356+0000","last_fullsized":"2024-05-09T09:07:36.319356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.323728+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:07:36.323728+0000","last_peered":"2024-05-09T09:07:36.323728+0000","last_clean":"2024-05-09T09:07:36.323728+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:07:36.323728+0000","last_undegraded":"2024-05-09T09:07:36.323728+0000","last_fullsized":"2024-05-09T09:07:36.323728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321222+0000","last_change":"2024-05-09T09:06:41.131374+0000","last_active":"2024-05-09T09:07:36.321222+0000","last_peered":"2024-05-09T09:07:36.321222+0000","last_clean":"2024-05-09T09:07:36.321222+0000","last_became_active":"2024-05-09T09:06:41.131052+0000","last_became_peered":"2024-05-09T09:06:41.131052+0000","last_unstale":"2024-05-09T09:07:36.321222+0000","last_undegraded":"2024-05-09T09:07:36.321222+0000","last_fullsized":"2024-05-09T09:07:36.321222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:22.200079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.321835+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:07:36.321835+0000","last_peered":"2024-05-09T09:07:36.321835+0000","last_clean":"2024-05-09T09:07:36.321835+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:07:36.321835+0000","last_undegraded":"2024-05-09T09:07:36.321835+0000","last_fullsized":"2024-05-09T09:07:36.321835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.320941+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:07:36.320941+0000","last_peered":"2024-05-09T09:07:36.320941+0000","last_clean":"2024-05-09T09:07:36.320941+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:07:36.320941+0000","last_undegraded":"2024-05-09T09:07:36.320941+0000","last_fullsized":"2024-05-09T09:07:36.320941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.434948+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:07:36.434948+0000","last_peered":"2024-05-09T09:07:36.434948+0000","last_clean":"2024-05-09T09:07:36.434948+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:07:36.434948+0000","last_undegraded":"2024-05-09T09:07:36.434948+0000","last_fullsized":"2024-05-09T09:07:36.434948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.316474+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:07:36.316474+0000","last_peered":"2024-05-09T09:07:36.316474+0000","last_clean":"2024-05-09T09:07:36.316474+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:07:36.316474+0000","last_undegraded":"2024-05-09T09:07:36.316474+0000","last_fullsized":"2024-05-09T09:07:36.316474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-09T09:07:36.318828+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:07:36.318828+0000","last_peered":"2024-05-09T09:07:36.318828+0000","last_clean":"2024-05-09T09:07:36.318828+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:07:36.318828+0000","last_undegraded":"2024-05-09T09:07:36.318828+0000","last_fullsized":"2024-05-09T09:07:36.318828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64800,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":469,"num_read_kb":305,"num_write":157,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":503896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1152332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":508206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":504008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1099548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":499286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":560386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":500674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":561736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":499480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":503936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1650688,"data_stored":1095136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:08:25.745 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T09:08:25.745 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T09:08:25.745 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T09:08:25.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-09T09:08:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:26.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:26 smithi064 ceph-mon[30149]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:08:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:26 smithi064 ceph-mon[30149]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:26.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:26 smithi179 ceph-mon[34086]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:08:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:26 smithi179 ceph-mon[34086]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:26 smithi139 ceph-mon[33599]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:08:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:26 smithi139 ceph-mon[33599]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:27.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 -- 172.21.15.64:0/1769162068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81014a0 msgr2=0x7f16e8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 --2- 172.21.15.64:0/1769162068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81014a0 0x7f16e8101880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f16d80099c0 tx=0x7f16d8009cd0 comp rx=0 tx=0).stop 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 -- 172.21.15.64:0/1769162068 shutdown_connections 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 --2- 172.21.15.64:0/1769162068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 0x7f16e810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 --2- 172.21.15.64:0/1769162068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81014a0 0x7f16e8101880 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 --2- 172.21.15.64:0/1769162068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e8101e50 0x7f16e8105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 -- 172.21.15.64:0/1769162068 >> 172.21.15.64:0/1769162068 conn(0x7f16e80782c0 msgr2=0x7f16e80786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 -- 172.21.15.64:0/1769162068 shutdown_connections 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.971+0000 7f16ee904700 1 -- 172.21.15.64:0/1769162068 wait complete. 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 Processor -- start 2024-05-09T09:08:27.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 -- start start 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e81014a0 0x7f16e810e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 0x7f16e810ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 0x7f16e810f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8112570 con 0x7f16e8101e50 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81123f0 con 0x7f16e8106580 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ee904700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81126f0 con 0x7f16e81014a0 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 0x7f16e810ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 0x7f16e810ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54340/0 (socket says 172.21.15.64:54340) 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 -- 172.21.15.64:0/1097904688 learned_addr learned my addr 172.21.15.64:0/1097904688 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16e7fff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e81014a0 0x7f16e810e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ecea1700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 0x7f16e810f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:27.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 -- 172.21.15.64:0/1097904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e81014a0 msgr2=0x7f16e810e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e81014a0 0x7f16e810e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 -- 172.21.15.64:0/1097904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 msgr2=0x7f16e810f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 0x7f16e810f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16dffff700 1 -- 172.21.15.64:0/1097904688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e81a6fd0 con 0x7f16e8101e50 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16e7fff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e81014a0 0x7f16e810e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.972+0000 7f16ecea1700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 0x7f16e810f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.973+0000 7f16dffff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 0x7f16e810ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d950 tx=0x7f16d000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.973+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0017070 con 0x7f16e8101e50 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.973+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d000de30 con 0x7f16e8101e50 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.973+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0005440 con 0x7f16e8101e50 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.973+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81a72c0 con 0x7f16e8101e50 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.973+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a77b0 con 0x7f16e8101e50 2024-05-09T09:08:27.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.974+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e8069010 con 0x7f16e8101e50 2024-05-09T09:08:27.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.975+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16d0004750 con 0x7f16e8101e50 2024-05-09T09:08:27.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.976+0000 7f16e5ffb700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16d40a2d30 0x7f16d40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:27.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.976+0000 7f16e7fff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16d40a2d30 0x7f16d40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:27.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.976+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f16d00cede0 con 0x7f16e8101e50 2024-05-09T09:08:27.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.976+0000 7f16e7fff700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16d40a2d30 0x7f16d40a51f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:27.980+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16d009b880 con 0x7f16e8101e50 2024-05-09T09:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.232+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f16e802dca0 con 0x7f16e8101e50 2024-05-09T09:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.233+0000 7f16e5ffb700 1 -- 172.21.15.64:0/1097904688 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f16d009b180 con 0x7f16e8101e50 2024-05-09T09:08:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T09:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16d40a2d30 msgr2=0x7f16d40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16d40a2d30 0x7f16d40a51f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d8005fd0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 msgr2=0x7f16e810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 0x7f16e810ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d950 tx=0x7f16d000dd10 comp rx=0 tx=0).stop 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 shutdown_connections 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e8106580 0x7f16e810f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16d40a2d30 0x7f16d40a51f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16e81014a0 0x7f16e810e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 --2- 172.21.15.64:0/1097904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e8101e50 0x7f16e810ee70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 >> 172.21.15.64:0/1097904688 conn(0x7f16e80782c0 msgr2=0x7f16e80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 shutdown_connections 2024-05-09T09:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.235+0000 7f16ee904700 1 -- 172.21.15.64:0/1097904688 wait complete. 2024-05-09T09:08:28.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:28 smithi064 ceph-mon[30149]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:28.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:28 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1097904688' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T09:08:28.884 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T09:08:28.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:08:28.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:28 smithi179 ceph-mon[34086]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:28.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:28 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1097904688' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T09:08:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 -- 172.21.15.64:0/964500076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4107630 msgr2=0x7f6cb4109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 --2- 172.21.15.64:0/964500076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4107630 0x7f6cb4109a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009ca0 tx=0x7f6cac02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 -- 172.21.15.64:0/964500076 shutdown_connections 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 --2- 172.21.15.64:0/964500076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4107630 0x7f6cb4109a20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 --2- 172.21.15.64:0/964500076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 0x7f6cb41070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 --2- 172.21.15.64:0/964500076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb41023d0 0x7f6cb41047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 -- 172.21.15.64:0/964500076 >> 172.21.15.64:0/964500076 conn(0x7f6cb40fbd20 msgr2=0x7f6cb40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 -- 172.21.15.64:0/964500076 shutdown_connections 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 -- 172.21.15.64:0/964500076 wait complete. 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 Processor -- start 2024-05-09T09:08:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 -- start start 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 0x7f6cb4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.996+0000 7f6cbad90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 0x7f6cb4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cbad90640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb4107630 0x7f6cb419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cbad90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410d3f0 con 0x7f6cb41023d0 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cbad90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410d270 con 0x7f6cb4104d00 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cbad90640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb410d570 con 0x7f6cb4107630 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 0x7f6cb4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 0x7f6cb4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54364/0 (socket says 172.21.15.64:54364) 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 -- 172.21.15.64:0/893993307 learned_addr learned my addr 172.21.15.64:0/893993307 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb3fff640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 0x7f6cb4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb932d640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb4107630 0x7f6cb419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 -- 172.21.15.64:0/893993307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb4107630 msgr2=0x7f6cb419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb4107630 0x7f6cb419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 -- 172.21.15.64:0/893993307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 msgr2=0x7f6cb4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 0x7f6cb4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 -- 172.21.15.64:0/893993307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb419f710 con 0x7f6cb41023d0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb3fff640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 0x7f6cb4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb932d640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb4107630 0x7f6cb419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb8b2c640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 0x7f6cb4197af0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e9f0 tx=0x7f6ca400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400cd80 con 0x7f6cb41023d0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca40054d0 con 0x7f6cb41023d0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb419fa00 con 0x7f6cb41023d0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.997+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4010490 con 0x7f6cb41023d0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.998+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb419ff00 con 0x7f6cb41023d0 2024-05-09T09:08:29.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:28.998+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb40fd810 con 0x7f6cb41023d0 2024-05-09T09:08:29.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.000+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6ca4005020 con 0x7f6cb41023d0 2024-05-09T09:08:29.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.000+0000 7f6cb1ffb640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c9c0a2c20 0x7f6c9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.000+0000 7f6cb3fff640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c9c0a2c20 0x7f6c9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.000+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6ca40cfa10 con 0x7f6cb41023d0 2024-05-09T09:08:29.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.001+0000 7f6cb3fff640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c9c0a2c20 0x7f6c9c0a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005d00 tx=0x7f6ca8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.005+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6ca409c4b0 con 0x7f6cb41023d0 2024-05-09T09:08:29.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:28 smithi139 ceph-mon[33599]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:29.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:28 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1097904688' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T09:08:29.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.195+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cb4062430 con 0x7f6cb41023d0 2024-05-09T09:08:29.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.197+0000 7f6cb1ffb640 1 -- 172.21.15.64:0/893993307 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24653 (secure 0 0 0) 0x7f6ca409bdb0 con 0x7f6cb41023d0 2024-05-09T09:08:29.196 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:29.196 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":92,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:07:36.304544+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:08:29.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c9c0a2c20 msgr2=0x7f6c9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c9c0a2c20 0x7f6c9c0a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005d00 tx=0x7f6ca8009550 comp rx=0 tx=0).stop 2024-05-09T09:08:29.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 msgr2=0x7f6cb4197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 0x7f6cb4197af0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e9f0 tx=0x7f6ca400edf0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 shutdown_connections 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c9c0a2c20 0x7f6c9c0a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cb4107630 0x7f6cb419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cb4104d00 0x7f6cb4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 --2- 172.21.15.64:0/893993307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb41023d0 0x7f6cb4197af0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 >> 172.21.15.64:0/893993307 conn(0x7f6cb40fbd20 msgr2=0x7f6cb4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 shutdown_connections 2024-05-09T09:08:29.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.199+0000 7f6cbad90640 1 -- 172.21.15.64:0/893993307 wait complete. 2024-05-09T09:08:29.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:08:29.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 -- 172.21.15.64:0/2744199298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 msgr2=0x7f48c4104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 --2- 172.21.15.64:0/2744199298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4104760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009900 tx=0x7f48b402a410 comp rx=0 tx=0).stop 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 -- 172.21.15.64:0/2744199298 shutdown_connections 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 --2- 172.21.15.64:0/2744199298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 0x7f48c41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 --2- 172.21.15.64:0/2744199298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c4104ca0 0x7f48c4107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 --2- 172.21.15.64:0/2744199298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4104760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 -- 172.21.15.64:0/2744199298 >> 172.21.15.64:0/2744199298 conn(0x7f48c40fbd00 msgr2=0x7f48c40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 -- 172.21.15.64:0/2744199298 shutdown_connections 2024-05-09T09:08:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 -- 172.21.15.64:0/2744199298 wait complete. 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.320+0000 7f48c903f640 1 Processor -- start 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 -- start start 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c4104ca0 0x7f48c4100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 0x7f48c4100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48c410d3e0 con 0x7f48c4102370 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48c410d260 con 0x7f48c41075d0 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c903f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48c410d560 con 0x7f48c4104ca0 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54386/0 (socket says 172.21.15.64:54386) 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 -- 172.21.15.64:0/2526773886 learned_addr learned my addr 172.21.15.64:0/2526773886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c259c640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c4104ca0 0x7f48c4100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c359e640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 0x7f48c4100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 -- 172.21.15.64:0/2526773886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c4104ca0 msgr2=0x7f48c4100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c4104ca0 0x7f48c4100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 -- 172.21.15.64:0/2526773886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 msgr2=0x7f48c4100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 0x7f48c4100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 -- 172.21.15.64:0/2526773886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c41010f0 con 0x7f48c4102370 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c359e640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 0x7f48c4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.321+0000 7f48c2d9d640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4101ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005940 tx=0x7f48b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.322+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b403f030 con 0x7f48c4102370 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.322+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c41a1c10 con 0x7f48c4102370 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.322+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48b4003cb0 con 0x7f48c4102370 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.322+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4035070 con 0x7f48c4102370 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.322+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c41a2160 con 0x7f48c4102370 2024-05-09T09:08:29.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.323+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c40fd7f0 con 0x7f48c4102370 2024-05-09T09:08:29.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.324+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f48b4042030 con 0x7f48c4102370 2024-05-09T09:08:29.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.324+0000 7f48bbfff640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48b00a2c20 0x7f48b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.325+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f48b40efb40 con 0x7f48c4102370 2024-05-09T09:08:29.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.325+0000 7f48c259c640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48b00a2c20 0x7f48b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.325+0000 7f48c259c640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48b00a2c20 0x7f48b00a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48c4101a50 tx=0x7f48ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.329+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48b40bc5e0 con 0x7f48c4102370 2024-05-09T09:08:29.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.512+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48c402d410 con 0x7f48c4102370 2024-05-09T09:08:29.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.513+0000 7f48bbfff640 1 -- 172.21.15.64:0/2526773886 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f48b40bbee0 con 0x7f48c4102370 2024-05-09T09:08:29.512 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.514+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48b00a2c20 msgr2=0x7f48b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.514+0000 7f48c903f640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48b00a2c20 0x7f48b00a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48c4101a50 tx=0x7f48ac005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.514+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 msgr2=0x7f48c4101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.514+0000 7f48c903f640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4101ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005940 tx=0x7f48b40042f0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 shutdown_connections 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48b00a2c20 0x7f48b00a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48c41075d0 0x7f48c4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c4104ca0 0x7f48c4100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 --2- 172.21.15.64:0/2526773886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c4102370 0x7f48c4101ce0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 >> 172.21.15.64:0/2526773886 conn(0x7f48c40fbd00 msgr2=0x7f48c4102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 shutdown_connections 2024-05-09T09:08:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.515+0000 7f48c903f640 1 -- 172.21.15.64:0/2526773886 wait complete. 2024-05-09T09:08:29.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 -- 172.21.15.64:0/1872495840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c101980 msgr2=0x7f8e5c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 --2- 172.21.15.64:0/1872495840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c101980 0x7f8e5c101e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009900 tx=0x7f8e5002a410 comp rx=0 tx=0).stop 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 -- 172.21.15.64:0/1872495840 shutdown_connections 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 --2- 172.21.15.64:0/1872495840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e5c102340 0x7f8e5c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 --2- 172.21.15.64:0/1872495840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c101980 0x7f8e5c101e00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 --2- 172.21.15.64:0/1872495840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e5c100790 0x7f8e5c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.634+0000 7f8e6139e640 1 -- 172.21.15.64:0/1872495840 >> 172.21.15.64:0/1872495840 conn(0x7f8e5c0fbd20 msgr2=0x7f8e5c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 -- 172.21.15.64:0/1872495840 shutdown_connections 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 -- 172.21.15.64:0/1872495840 wait complete. 2024-05-09T09:08:29.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 Processor -- start 2024-05-09T09:08:29.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 -- start start 2024-05-09T09:08:29.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e5c100790 0x7f8e5c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e5c101980 0x7f8e5c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 0x7f8e5c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c10c2c0 con 0x7f8e5c102340 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c10c140 con 0x7f8e5c101980 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.635+0000 7f8e6139e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c10c440 con 0x7f8e5c100790 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 0x7f8e5c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e5c100790 0x7f8e5c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 0x7f8e5c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54408/0 (socket says 172.21.15.64:54408) 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 -- 172.21.15.64:0/339258517 learned_addr learned my addr 172.21.15.64:0/339258517 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5a7fc640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e5c101980 0x7f8e5c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 -- 172.21.15.64:0/339258517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e5c100790 msgr2=0x7f8e5c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e5c100790 0x7f8e5c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 -- 172.21.15.64:0/339258517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e5c101980 msgr2=0x7f8e5c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e5c101980 0x7f8e5c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 -- 172.21.15.64:0/339258517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c19f6e0 con 0x7f8e5c102340 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e5b7fe640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 0x7f8e5c19f070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400cce0 tx=0x7f8e5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400f450 con 0x7f8e5c102340 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e5400faa0 con 0x7f8e5c102340 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c19f9d0 con 0x7f8e5c102340 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54018b70 con 0x7f8e5c102340 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.636+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c19fed0 con 0x7f8e5c102340 2024-05-09T09:08:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.637+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5c1a0160 con 0x7f8e5c102340 2024-05-09T09:08:29.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.639+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8e54018d10 con 0x7f8e5c102340 2024-05-09T09:08:29.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.639+0000 7f8e43fff640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e440a2c20 0x7f8e440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.639+0000 7f8e5affd640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e440a2c20 0x7f8e440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.639+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8e540cf950 con 0x7f8e5c102340 2024-05-09T09:08:29.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.640+0000 7f8e5affd640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e440a2c20 0x7f8e440a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c1017e0 tx=0x7f8e4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.643+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8e54014030 con 0x7f8e5c102340 2024-05-09T09:08:29.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.828+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e5c1a0330 con 0x7f8e5c102340 2024-05-09T09:08:29.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.828+0000 7f8e43fff640 1 -- 172.21.15.64:0/339258517 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f8e5409c3f0 con 0x7f8e5c102340 2024-05-09T09:08:29.828 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e440a2c20 msgr2=0x7f8e440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e440a2c20 0x7f8e440a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c1017e0 tx=0x7f8e4c005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 msgr2=0x7f8e5c19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 0x7f8e5c19f070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400cce0 tx=0x7f8e5400a620 comp rx=0 tx=0).stop 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 shutdown_connections 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e440a2c20 0x7f8e440a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e5c102340 0x7f8e5c19f070 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e5c101980 0x7f8e5c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 --2- 172.21.15.64:0/339258517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e5c100790 0x7f8e5c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.830+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 >> 172.21.15.64:0/339258517 conn(0x7f8e5c0fbd20 msgr2=0x7f8e5c069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.831+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 shutdown_connections 2024-05-09T09:08:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.831+0000 7f8e6139e640 1 -- 172.21.15.64:0/339258517 wait complete. 2024-05-09T09:08:29.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:29 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/893993307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:29.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:29 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2526773886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:29.841 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:08:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 -- 172.21.15.64:0/558956594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c102310 msgr2=0x7fe77c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 --2- 172.21.15.64:0/558956594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c102310 0x7fe77c104700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe764009900 tx=0x7fe76402a410 comp rx=0 tx=0).stop 2024-05-09T09:08:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 -- 172.21.15.64:0/558956594 shutdown_connections 2024-05-09T09:08:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 --2- 172.21.15.64:0/558956594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77c107570 0x7fe77c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 --2- 172.21.15.64:0/558956594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe77c104c40 0x7fe77c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 --2- 172.21.15.64:0/558956594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c102310 0x7fe77c104700 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 -- 172.21.15.64:0/558956594 >> 172.21.15.64:0/558956594 conn(0x7fe77c0fbd20 msgr2=0x7fe77c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 -- 172.21.15.64:0/558956594 shutdown_connections 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.949+0000 7fe78254f640 1 -- 172.21.15.64:0/558956594 wait complete. 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 Processor -- start 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 -- start start 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77c102310 0x7fe77c101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 0x7fe77c1002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe77c107570 0x7fe77c100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c10d3c0 con 0x7fe77c104c40 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c10d240 con 0x7fe77c102310 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe78254f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c10d540 con 0x7fe77c107570 2024-05-09T09:08:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe77b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 0x7fe77c1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe77b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 0x7fe77c1002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54434/0 (socket says 172.21.15.64:54434) 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe77b7fe640 1 -- 172.21.15.64:0/3273584924 learned_addr learned my addr 172.21.15.64:0/3273584924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.950+0000 7fe780aec640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe77c107570 0x7fe77c100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77bfff640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77c102310 0x7fe77c101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77b7fe640 1 -- 172.21.15.64:0/3273584924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe77c107570 msgr2=0x7fe77c100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77b7fe640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe77c107570 0x7fe77c100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77b7fe640 1 -- 172.21.15.64:0/3273584924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77c102310 msgr2=0x7fe77c101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77b7fe640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77c102310 0x7fe77c101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77b7fe640 1 -- 172.21.15.64:0/3273584924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c101090 con 0x7fe77c104c40 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe77b7fe640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 0x7fe77c1002c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00d770 tx=0x7fe76c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c017070 con 0x7fe77c104c40 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c009c30 con 0x7fe77c104c40 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c1a1c20 con 0x7fe77c104c40 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00f620 con 0x7fe77c104c40 2024-05-09T09:08:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.951+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c1a2170 con 0x7fe77c104c40 2024-05-09T09:08:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.952+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77c0fd810 con 0x7fe77c104c40 2024-05-09T09:08:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.953+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe76c01a070 con 0x7fe77c104c40 2024-05-09T09:08:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.954+0000 7fe7797fa640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7680a2c20 0x7fe7680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.954+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe76c0cf060 con 0x7fe77c104c40 2024-05-09T09:08:29.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.954+0000 7fe77bfff640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7680a2c20 0x7fe7680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:29.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.955+0000 7fe77bfff640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7680a2c20 0x7fe7680a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe764000c00 tx=0x7fe7640314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:29.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:29.958+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe76c009de0 con 0x7fe77c104c40 2024-05-09T09:08:29.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:29 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/893993307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:29 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2526773886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:29 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/893993307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:30.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:29 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2526773886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.143+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe77c02d3c0 con 0x7fe77c104c40 2024-05-09T09:08:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.144+0000 7fe7797fa640 1 -- 172.21.15.64:0/3273584924 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fe76c09bb00 con 0x7fe77c104c40 2024-05-09T09:08:30.143 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7680a2c20 msgr2=0x7fe7680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7680a2c20 0x7fe7680a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe764000c00 tx=0x7fe7640314a0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 msgr2=0x7fe77c1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 0x7fe77c1002c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00d770 tx=0x7fe76c00dc30 comp rx=0 tx=0).stop 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 shutdown_connections 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7680a2c20 0x7fe7680a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe77c107570 0x7fe77c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c104c40 0x7fe77c1002c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 --2- 172.21.15.64:0/3273584924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77c102310 0x7fe77c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 >> 172.21.15.64:0/3273584924 conn(0x7fe77c0fbd20 msgr2=0x7fe77c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 shutdown_connections 2024-05-09T09:08:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.146+0000 7fe78254f640 1 -- 172.21.15.64:0/3273584924 wait complete. 2024-05-09T09:08:30.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:08:30.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 -- 172.21.15.64:0/4028171330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc100ca0 msgr2=0x7f75dc1010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 --2- 172.21.15.64:0/4028171330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc100ca0 0x7f75dc1010c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 -- 172.21.15.64:0/4028171330 shutdown_connections 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 --2- 172.21.15.64:0/4028171330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75dc102370 0x7f75dc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 --2- 172.21.15.64:0/4028171330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc1019b0 0x7f75dc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 --2- 172.21.15.64:0/4028171330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc100ca0 0x7f75dc1010c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 -- 172.21.15.64:0/4028171330 >> 172.21.15.64:0/4028171330 conn(0x7f75dc0fb470 msgr2=0x7f75dc0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 -- 172.21.15.64:0/4028171330 shutdown_connections 2024-05-09T09:08:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.267+0000 7f75e08b3640 1 -- 172.21.15.64:0/4028171330 wait complete. 2024-05-09T09:08:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.268+0000 7f75e08b3640 1 Processor -- start 2024-05-09T09:08:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.268+0000 7f75e08b3640 1 -- start start 2024-05-09T09:08:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.268+0000 7f75e08b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75dc100ca0 0x7f75dc196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.268+0000 7f75e08b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 0x7f75dc196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.268+0000 7f75e08b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc102370 0x7f75dc19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.268+0000 7f75e08b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10c2c0 con 0x7f75dc1019b0 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75e08b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10c140 con 0x7f75dc102370 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75e08b3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10c440 con 0x7f75dc100ca0 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 0x7f75dc196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 0x7f75dc196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54458/0 (socket says 172.21.15.64:54458) 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 -- 172.21.15.64:0/2666034324 learned_addr learned my addr 172.21.15.64:0/2666034324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75dad9d640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc102370 0x7f75dc19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75da59c640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75dc100ca0 0x7f75dc196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 -- 172.21.15.64:0/2666034324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75dc100ca0 msgr2=0x7f75dc196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75dc100ca0 0x7f75dc196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 -- 172.21.15.64:0/2666034324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc102370 msgr2=0x7f75dc19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc102370 0x7f75dc19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 -- 172.21.15.64:0/2666034324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc19e600 con 0x7f75dc1019b0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75d9d9b640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 0x7f75dc196f70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d950 tx=0x7f75d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0017070 con 0x7f75dc1019b0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc0ff850 con 0x7f75dc1019b0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75d0009c30 con 0x7f75dc1019b0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d000f620 con 0x7f75dc1019b0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.269+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc0ffdf0 con 0x7f75dc1019b0 2024-05-09T09:08:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.271+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc068aa0 con 0x7f75dc1019b0 2024-05-09T09:08:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.272+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f75d00109a0 con 0x7f75dc1019b0 2024-05-09T09:08:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.272+0000 7f75cb7fe640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75c40a2c20 0x7f75c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.273+0000 7f75da59c640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75c40a2c20 0x7f75c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.273+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f75d00cf150 con 0x7f75dc1019b0 2024-05-09T09:08:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.273+0000 7f75da59c640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75c40a2c20 0x7f75c40a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.276+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75d0009de0 con 0x7f75dc1019b0 2024-05-09T09:08:30.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.460+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f75dc107690 con 0x7f75dc1019b0 2024-05-09T09:08:30.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.460+0000 7f75cb7fe640 1 -- 172.21.15.64:0/2666034324 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f75d009bbf0 con 0x7f75dc1019b0 2024-05-09T09:08:30.459 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:30.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75c40a2c20 msgr2=0x7f75c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75c40a2c20 0x7f75c40a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc0314a0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 msgr2=0x7f75dc196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 0x7f75dc196f70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d950 tx=0x7f75d000de10 comp rx=0 tx=0).stop 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 shutdown_connections 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75c40a2c20 0x7f75c40a50e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc102370 0x7f75dc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75dc1019b0 0x7f75dc196f70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 --2- 172.21.15.64:0/2666034324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75dc100ca0 0x7f75dc196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 >> 172.21.15.64:0/2666034324 conn(0x7f75dc0fb470 msgr2=0x7f75dc104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 shutdown_connections 2024-05-09T09:08:30.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.463+0000 7f75e08b3640 1 -- 172.21.15.64:0/2666034324 wait complete. 2024-05-09T09:08:30.473 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:08:30.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 -- 172.21.15.64:0/3347501561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 msgr2=0x7f1d60101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 --2- 172.21.15.64:0/3347501561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60101e00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009ca0 tx=0x7f1d5002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 -- 172.21.15.64:0/3347501561 shutdown_connections 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 --2- 172.21.15.64:0/3347501561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60102340 0x7f1d60108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 --2- 172.21.15.64:0/3347501561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60101e00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 --2- 172.21.15.64:0/3347501561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d60100790 0x7f1d60100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 -- 172.21.15.64:0/3347501561 >> 172.21.15.64:0/3347501561 conn(0x7f1d600fbd20 msgr2=0x7f1d600fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 -- 172.21.15.64:0/3347501561 shutdown_connections 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 -- 172.21.15.64:0/3347501561 wait complete. 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 Processor -- start 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 -- start start 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.579+0000 7f1d66eba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60100790 0x7f1d601048b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d66eba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d66eba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d60102340 0x7f1d60103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d66eba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010c2c0 con 0x7f1d60101980 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d66eba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010c140 con 0x7f1d60100790 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d66eba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010c440 con 0x7f1d60102340 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54466/0 (socket says 172.21.15.64:54466) 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 -- 172.21.15.64:0/2204294162 learned_addr learned my addr 172.21.15.64:0/2204294162 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d64c56640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60100790 0x7f1d601048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d65457640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d60102340 0x7f1d60103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 -- 172.21.15.64:0/2204294162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d60102340 msgr2=0x7f1d60103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d60102340 0x7f1d60103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 -- 172.21.15.64:0/2204294162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60100790 msgr2=0x7f1d601048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60100790 0x7f1d601048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 -- 172.21.15.64:0/2204294162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d60103d40 con 0x7f1d60101980 2024-05-09T09:08:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5ffff640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60102f40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009dc0 tx=0x7f1d50003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:30.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50032ec0 con 0x7f1d60101980 2024-05-09T09:08:30.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d500044f0 con 0x7f1d60101980 2024-05-09T09:08:30.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601a1be0 con 0x7f1d60101980 2024-05-09T09:08:30.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.580+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50030cb0 con 0x7f1d60101980 2024-05-09T09:08:30.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.581+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a2100 con 0x7f1d60101980 2024-05-09T09:08:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.581+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d601a22d0 con 0x7f1d60101980 2024-05-09T09:08:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.583+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d500046a0 con 0x7f1d60101980 2024-05-09T09:08:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.584+0000 7f1d5dffb640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1d4c0a2dc0 0x7f1d4c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.584+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1d500f0b00 con 0x7f1d60101980 2024-05-09T09:08:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.584+0000 7f1d64c56640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1d4c0a2dc0 0x7f1d4c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.585+0000 7f1d64c56640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1d4c0a2dc0 0x7f1d4c0a5280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d601017e0 tx=0x7f1d48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:30.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.588+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d500bd6b0 con 0x7f1d60101980 2024-05-09T09:08:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.811+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d601a2660 con 0x7f1d60101980 2024-05-09T09:08:30.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.812+0000 7f1d5dffb640 1 -- 172.21.15.64:0/2204294162 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f1d500bcfb0 con 0x7f1d60101980 2024-05-09T09:08:30.811 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:30.811 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:07:08.230747+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:08:30.811 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1d4c0a2dc0 msgr2=0x7f1d4c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1d4c0a2dc0 0x7f1d4c0a5280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d601017e0 tx=0x7f1d48005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 msgr2=0x7f1d60102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60102f40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009dc0 tx=0x7f1d50003ba0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 shutdown_connections 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1d4c0a2dc0 0x7f1d4c0a5280 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d60102340 0x7f1d60103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d60101980 0x7f1d60102f40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 --2- 172.21.15.64:0/2204294162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60100790 0x7f1d601048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 >> 172.21.15.64:0/2204294162 conn(0x7f1d600fbd20 msgr2=0x7f1d60069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:30.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 shutdown_connections 2024-05-09T09:08:30.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.814+0000 7f1d66eba640 1 -- 172.21.15.64:0/2204294162 wait complete. 2024-05-09T09:08:30.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:30 smithi064 ceph-mon[30149]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:30.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:30 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/339258517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:30 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3273584924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:30 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2666034324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 -- 172.21.15.64:0/2284218628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4100770 msgr2=0x7f42a4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 --2- 172.21.15.64:0/2284218628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4100770 0x7f42a4100b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f429c009ca0 tx=0x7f429c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 -- 172.21.15.64:0/2284218628 shutdown_connections 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 --2- 172.21.15.64:0/2284218628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42a4102320 0x7f42a4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 --2- 172.21.15.64:0/2284218628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a4101960 0x7f42a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 --2- 172.21.15.64:0/2284218628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4100770 0x7f42a4100b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.929+0000 7f42a92ca640 1 -- 172.21.15.64:0/2284218628 >> 172.21.15.64:0/2284218628 conn(0x7f42a40fbce0 msgr2=0x7f42a40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 -- 172.21.15.64:0/2284218628 shutdown_connections 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 -- 172.21.15.64:0/2284218628 wait complete. 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 Processor -- start 2024-05-09T09:08:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 -- start start 2024-05-09T09:08:30.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42a4100770 0x7f42a4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a4101960 0x7f42a419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 0x7f42a419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f42a410c270 con 0x7f42a4102320 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42a410c0f0 con 0x7f42a4100770 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.930+0000 7f42a92ca640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42a410c3f0 con 0x7f42a4101960 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 0x7f42a419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 0x7f42a419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54490/0 (socket says 172.21.15.64:54490) 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 -- 172.21.15.64:0/181676034 learned_addr learned my addr 172.21.15.64:0/181676034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a27fc640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a4101960 0x7f42a419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a2ffd640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42a4100770 0x7f42a4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 -- 172.21.15.64:0/181676034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a4101960 msgr2=0x7f42a419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a4101960 0x7f42a419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 -- 172.21.15.64:0/181676034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42a4100770 msgr2=0x7f42a4069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42a4100770 0x7f42a4069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 -- 172.21.15.64:0/181676034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a419f720 con 0x7f42a4102320 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a37fe640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 0x7f42a419ee60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f429400d750 tx=0x7f429400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:30.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294017070 con 0x7f42a4102320 2024-05-09T09:08:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42940054d0 con 0x7f42a4102320 2024-05-09T09:08:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a419fa10 con 0x7f42a4102320 2024-05-09T09:08:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294004c60 con 0x7f42a4102320 2024-05-09T09:08:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.931+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a419ff10 con 0x7f42a4102320 2024-05-09T09:08:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.932+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a4107b50 con 0x7f42a4102320 2024-05-09T09:08:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.933+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4294004e00 con 0x7f42a4102320 2024-05-09T09:08:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.934+0000 7f429bfff640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42900a2c20 0x7f42900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.934+0000 7f42a2ffd640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42900a2c20 0x7f42900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.934+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f42940ceeb0 con 0x7f42a4102320 2024-05-09T09:08:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.935+0000 7f42a2ffd640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42900a2c20 0x7f42900a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f429c000f80 tx=0x7f429c031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:30.939+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f429409b300 con 0x7f42a4102320 2024-05-09T09:08:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:30 smithi179 ceph-mon[34086]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:30 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/339258517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:30 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3273584924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:30 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2666034324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:31.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:30 smithi139 ceph-mon[33599]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:31.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:30 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/339258517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:31.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:30 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3273584924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:30 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2666034324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:31.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.128+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42a402d410 con 0x7f42a4102320 2024-05-09T09:08:31.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.129+0000 7f429bfff640 1 -- 172.21.15.64:0/181676034 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24653 (secure 0 0 0) 0x7f429409f600 con 0x7f42a4102320 2024-05-09T09:08:31.129 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:31.129 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":92,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:07:36.304544+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.131+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42900a2c20 msgr2=0x7f42900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.131+0000 7f42a92ca640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42900a2c20 0x7f42900a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f429c000f80 tx=0x7f429c031000 comp rx=0 tx=0).stop 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.131+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 msgr2=0x7f42a419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.131+0000 7f42a92ca640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 0x7f42a419ee60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f429400d750 tx=0x7f429400dc10 comp rx=0 tx=0).stop 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 shutdown_connections 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42900a2c20 0x7f42900a50e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a4102320 0x7f42a419ee60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a4101960 0x7f42a419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 --2- 172.21.15.64:0/181676034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42a4100770 0x7f42a4069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 >> 172.21.15.64:0/181676034 conn(0x7f42a40fbce0 msgr2=0x7f42a4104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 shutdown_connections 2024-05-09T09:08:31.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.132+0000 7f42a92ca640 1 -- 172.21.15.64:0/181676034 wait complete. 2024-05-09T09:08:31.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:08:31.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:08:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 -- 172.21.15.64:0/886766440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 msgr2=0x7fd1ec1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 --2- 172.21.15.64:0/886766440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec1099c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009ca0 tx=0x7fd1e402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 -- 172.21.15.64:0/886766440 shutdown_connections 2024-05-09T09:08:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 --2- 172.21.15.64:0/886766440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec1099c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 --2- 172.21.15.64:0/886766440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1ec104c40 0x7fd1ec107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 --2- 172.21.15.64:0/886766440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec102310 0x7fd1ec104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 -- 172.21.15.64:0/886766440 >> 172.21.15.64:0/886766440 conn(0x7fd1ec0fbd20 msgr2=0x7fd1ec0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 -- 172.21.15.64:0/886766440 shutdown_connections 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.252+0000 7fd1f184d640 1 -- 172.21.15.64:0/886766440 wait complete. 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 Processor -- start 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 -- start start 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1ec102310 0x7fd1ec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec104c40 0x7fd1ec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec10d3e0 con 0x7fd1ec107570 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec10d260 con 0x7fd1ec102310 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1f184d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec10d560 con 0x7fd1ec104c40 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54518/0 (socket says 172.21.15.64:54518) 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 -- 172.21.15.64:0/1734081674 learned_addr learned my addr 172.21.15.64:0/1734081674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eaffd640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1ec102310 0x7fd1ec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1e3fff640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec104c40 0x7fd1ec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 -- 172.21.15.64:0/1734081674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec104c40 msgr2=0x7fd1ec197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec104c40 0x7fd1ec197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 -- 172.21.15.64:0/1734081674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1ec102310 msgr2=0x7fd1ec197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1ec102310 0x7fd1ec197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.253+0000 7fd1eb7fe640 1 -- 172.21.15.64:0/1734081674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec19f6d0 con 0x7fd1ec107570 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.254+0000 7fd1eb7fe640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec19f060 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009dc0 tx=0x7fd1e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.254+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4032ec0 con 0x7fd1ec107570 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.254+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec19f960 con 0x7fd1ec107570 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.254+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e40044f0 con 0x7fd1ec107570 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.254+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4030d40 con 0x7fd1ec107570 2024-05-09T09:08:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.254+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec19feb0 con 0x7fd1ec107570 2024-05-09T09:08:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.255+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ec0fd810 con 0x7fd1ec107570 2024-05-09T09:08:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.256+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd1e4004040 con 0x7fd1ec107570 2024-05-09T09:08:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.257+0000 7fd1f084b640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2dc0 0x7fd1d80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.257+0000 7fd1eaffd640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2dc0 0x7fd1d80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.257+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd1e40f0a00 con 0x7fd1ec107570 2024-05-09T09:08:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.257+0000 7fd1eaffd640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2dc0 0x7fd1d80a5280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc005fd0 tx=0x7fd1dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.261+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd1e40f2e20 con 0x7fd1ec107570 2024-05-09T09:08:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.446+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1ec02d3e0 con 0x7fd1ec107570 2024-05-09T09:08:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.446+0000 7fd1f084b640 1 -- 172.21.15.64:0/1734081674 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fd1e40bd5b0 con 0x7fd1ec107570 2024-05-09T09:08:31.446 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2dc0 msgr2=0x7fd1d80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2dc0 0x7fd1d80a5280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc005fd0 tx=0x7fd1dc00b390 comp rx=0 tx=0).stop 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 msgr2=0x7fd1ec19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec19f060 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009dc0 tx=0x7fd1e4003ba0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 shutdown_connections 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2dc0 0x7fd1d80a5280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ec107570 0x7fd1ec19f060 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec104c40 0x7fd1ec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 --2- 172.21.15.64:0/1734081674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1ec102310 0x7fd1ec197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 >> 172.21.15.64:0/1734081674 conn(0x7fd1ec0fbd20 msgr2=0x7fd1ec102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 shutdown_connections 2024-05-09T09:08:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.449+0000 7fd1f184d640 1 -- 172.21.15.64:0/1734081674 wait complete. 2024-05-09T09:08:31.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/1243429549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 msgr2=0x7f4c28107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/1243429549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28107030 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009900 tx=0x7f4c1c02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/1243429549 shutdown_connections 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/1243429549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28107570 0x7f4c281099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/1243429549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28107030 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/1243429549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28102310 0x7f4c28104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/1243429549 >> 172.21.15.64:0/1243429549 conn(0x7f4c280fbce0 msgr2=0x7f4c280fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/1243429549 shutdown_connections 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/1243429549 wait complete. 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.568+0000 7f4c2dc8c640 1 Processor -- start 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 -- start start 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28102310 0x7f4c28197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28107570 0x7f4c2819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810d390 con 0x7f4c28104c40 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810d210 con 0x7f4c28107570 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c2dc8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810d510 con 0x7f4c28102310 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54538/0 (socket says 172.21.15.64:54538) 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 -- 172.21.15.64:0/3623473452 learned_addr learned my addr 172.21.15.64:0/3623473452 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c27fff640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28107570 0x7f4c2819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c277fe640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28102310 0x7f4c28197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 -- 172.21.15.64:0/3623473452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28102310 msgr2=0x7f4c28197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28102310 0x7f4c28197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 -- 172.21.15.64:0/3623473452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28107570 msgr2=0x7f4c2819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28107570 0x7f4c2819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 -- 172.21.15.64:0/3623473452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2819f720 con 0x7f4c28104c40 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c277fe640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28102310 0x7f4c28197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c27fff640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28107570 0x7f4c2819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.569+0000 7f4c26ffd640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28198040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0314e0 tx=0x7f4c1c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.570+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c03f030 con 0x7f4c28104c40 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.570+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c1c003cb0 con 0x7f4c28104c40 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.570+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c035070 con 0x7f4c28104c40 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.570+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2819f9b0 con 0x7f4c28104c40 2024-05-09T09:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.570+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2819fe50 con 0x7f4c28104c40 2024-05-09T09:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.571+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c28068680 con 0x7f4c28104c40 2024-05-09T09:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.572+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4c1c042030 con 0x7f4c28104c40 2024-05-09T09:08:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.572+0000 7f4c24ff9640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4c100a2c20 0x7f4c100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.573+0000 7f4c277fe640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4c100a2c20 0x7f4c100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.573+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4c1c0efc00 con 0x7f4c28104c40 2024-05-09T09:08:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.573+0000 7f4c277fe640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4c100a2c20 0x7f4c100a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005fd0 tx=0x7f4c18009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:31.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.577+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c28068680 con 0x7f4c28104c40 2024-05-09T09:08:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.761+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4c2802d240 con 0x7f4c28104c40 2024-05-09T09:08:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.761+0000 7f4c24ff9640 1 -- 172.21.15.64:0/3623473452 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f4c1c0bc6a0 con 0x7f4c28104c40 2024-05-09T09:08:31.760 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:31.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4c100a2c20 msgr2=0x7f4c100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4c100a2c20 0x7f4c100a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005fd0 tx=0x7f4c18009510 comp rx=0 tx=0).stop 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 msgr2=0x7f4c28198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28198040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0314e0 tx=0x7f4c1c0042e0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 shutdown_connections 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4c100a2c20 0x7f4c100a50e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28107570 0x7f4c2819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.763+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c28104c40 0x7f4c28198040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.764+0000 7f4c2dc8c640 1 --2- 172.21.15.64:0/3623473452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c28102310 0x7f4c28197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.764+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 >> 172.21.15.64:0/3623473452 conn(0x7f4c280fbce0 msgr2=0x7f4c28102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.764+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 shutdown_connections 2024-05-09T09:08:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.764+0000 7f4c2dc8c640 1 -- 172.21.15.64:0/3623473452 wait complete. 2024-05-09T09:08:31.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:31 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2204294162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:31.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:31 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/181676034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:31.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:31 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1734081674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:31.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.882+0000 7f3528a8c640 1 -- 172.21.15.64:0/3576978869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 msgr2=0x7f3524107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.882+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3576978869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f3524107030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3518009900 tx=0x7f351802a410 comp rx=0 tx=0).stop 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 -- 172.21.15.64:0/3576978869 shutdown_connections 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3576978869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524107570 0x7f35241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3576978869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f3524107030 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3576978869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524102310 0x7f3524104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 -- 172.21.15.64:0/3576978869 >> 172.21.15.64:0/3576978869 conn(0x7f35240fbd20 msgr2=0x7f35240fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 -- 172.21.15.64:0/3576978869 shutdown_connections 2024-05-09T09:08:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 -- 172.21.15.64:0/3576978869 wait complete. 2024-05-09T09:08:31.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 Processor -- start 2024-05-09T09:08:31.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.883+0000 7f3528a8c640 1 -- start start 2024-05-09T09:08:31.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524102310 0x7f3524197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f35241980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524107570 0x7f352419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f352410d3d0 con 0x7f3524104c40 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f352410d250 con 0x7f3524102310 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f352410d550 con 0x7f3524107570 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f35241980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f35241980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54552/0 (socket says 172.21.15.64:54552) 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 -- 172.21.15.64:0/3138527427 learned_addr learned my addr 172.21.15.64:0/3138527427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3522d9d640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524107570 0x7f352419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f352259c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524102310 0x7f3524197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 -- 172.21.15.64:0/3138527427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524107570 msgr2=0x7f352419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524107570 0x7f352419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 -- 172.21.15.64:0/3138527427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524102310 msgr2=0x7f3524197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524102310 0x7f3524197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:31.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 -- 172.21.15.64:0/3138527427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352419f740 con 0x7f3524104c40 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3521d9b640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f35241980b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35180314e0 tx=0x7f35180042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351803f030 con 0x7f3524104c40 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352419f9d0 con 0x7f3524104c40 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3518003cb0 con 0x7f3524104c40 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518035070 con 0x7f3524104c40 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.884+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352419ff10 con 0x7f3524104c40 2024-05-09T09:08:31.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.885+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35240fd810 con 0x7f3524104c40 2024-05-09T09:08:31.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.886+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3518042030 con 0x7f3524104c40 2024-05-09T09:08:31.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.887+0000 7f35137fe640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f350c0a2c20 0x7f350c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:31.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.887+0000 7f352259c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f350c0a2c20 0x7f350c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:31.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.887+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f35180efb70 con 0x7f3524104c40 2024-05-09T09:08:31.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.888+0000 7f352259c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f350c0a2c20 0x7f350c0a50e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3514005fd0 tx=0x7f3514005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:31.892+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35180bc610 con 0x7f3524104c40 2024-05-09T09:08:31.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:31 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2204294162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:31.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:31 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/181676034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:31.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:31 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1734081674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:32.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:31 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2204294162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:32.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:31 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/181676034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:32.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:31 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1734081674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.077+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f352402d370 con 0x7f3524104c40 2024-05-09T09:08:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.078+0000 7f35137fe640 1 -- 172.21.15.64:0/3138527427 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f35180bbf10 con 0x7f3524104c40 2024-05-09T09:08:32.077 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f350c0a2c20 msgr2=0x7f350c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f350c0a2c20 0x7f350c0a50e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3514005fd0 tx=0x7f3514005850 comp rx=0 tx=0).stop 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 msgr2=0x7f35241980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f35241980b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35180314e0 tx=0x7f35180042e0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 shutdown_connections 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f350c0a2c20 0x7f350c0a50e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524107570 0x7f352419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524104c40 0x7f35241980b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 --2- 172.21.15.64:0/3138527427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524102310 0x7f3524197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 >> 172.21.15.64:0/3138527427 conn(0x7f35240fbd20 msgr2=0x7f3524102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:32.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 shutdown_connections 2024-05-09T09:08:32.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.080+0000 7f3528a8c640 1 -- 172.21.15.64:0/3138527427 wait complete. 2024-05-09T09:08:32.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 -- 172.21.15.64:0/3493407162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 msgr2=0x7fc6fc109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 --2- 172.21.15.64:0/3493407162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc109cd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009900 tx=0x7fc6f402a410 comp rx=0 tx=0).stop 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 -- 172.21.15.64:0/3493407162 shutdown_connections 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 --2- 172.21.15.64:0/3493407162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc109cd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 --2- 172.21.15.64:0/3493407162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6fc102a80 0x7fc6fc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 --2- 172.21.15.64:0/3493407162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6fc101890 0x7fc6fc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 -- 172.21.15.64:0/3493407162 >> 172.21.15.64:0/3493407162 conn(0x7fc6fc0fce00 msgr2=0x7fc6fc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 -- 172.21.15.64:0/3493407162 shutdown_connections 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.200+0000 7fc702f6a640 1 -- 172.21.15.64:0/3493407162 wait complete. 2024-05-09T09:08:32.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 Processor -- start 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 -- start start 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6fc101890 0x7fc6fc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6fc102a80 0x7fc6fc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc10d3c0 con 0x7fc6fc103440 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc10d240 con 0x7fc6fc102a80 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc702f6a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc10d540 con 0x7fc6fc101890 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc19f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54558/0 (socket says 172.21.15.64:54558) 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 -- 172.21.15.64:0/4154587739 learned_addr learned my addr 172.21.15.64:0/4154587739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc6fbfff640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6fc102a80 0x7fc6fc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc700d06640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6fc101890 0x7fc6fc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 -- 172.21.15.64:0/4154587739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6fc101890 msgr2=0x7fc6fc197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6fc101890 0x7fc6fc197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 -- 172.21.15.64:0/4154587739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6fc102a80 msgr2=0x7fc6fc197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6fc102a80 0x7fc6fc197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.201+0000 7fc701507640 1 -- 172.21.15.64:0/4154587739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc19f6a0 con 0x7fc6fc103440 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.202+0000 7fc701507640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc19f030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005940 tx=0x7fc6f4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.202+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f403f030 con 0x7fc6fc103440 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.202+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6f4003aa0 con 0x7fc6fc103440 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.202+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc19f930 con 0x7fc6fc103440 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.202+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4035070 con 0x7fc6fc103440 2024-05-09T09:08:32.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.202+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc19fe20 con 0x7fc6fc103440 2024-05-09T09:08:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.203+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6fc108c80 con 0x7fc6fc103440 2024-05-09T09:08:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.204+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc6f4042030 con 0x7fc6fc103440 2024-05-09T09:08:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.205+0000 7fc6f9ffb640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6e80a2c20 0x7fc6e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.205+0000 7fc700d06640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6e80a2c20 0x7fc6e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.205+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc6f40efbb0 con 0x7fc6fc103440 2024-05-09T09:08:32.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.206+0000 7fc700d06640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6e80a2c20 0x7fc6e80a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec005db0 tx=0x7fc6ec005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:32.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.210+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6f40bc650 con 0x7fc6fc103440 2024-05-09T09:08:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.394+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc6fc062430 con 0x7fc6fc103440 2024-05-09T09:08:32.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.395+0000 7fc6f9ffb640 1 -- 172.21.15.64:0/4154587739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fc6f40bbf50 con 0x7fc6fc103440 2024-05-09T09:08:32.394 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6e80a2c20 msgr2=0x7fc6e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6e80a2c20 0x7fc6e80a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec005db0 tx=0x7fc6ec005d40 comp rx=0 tx=0).stop 2024-05-09T09:08:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 msgr2=0x7fc6fc19f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc19f030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005940 tx=0x7fc6f4031a30 comp rx=0 tx=0).stop 2024-05-09T09:08:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 shutdown_connections 2024-05-09T09:08:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6e80a2c20 0x7fc6e80a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6fc103440 0x7fc6fc19f030 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6fc102a80 0x7fc6fc197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 --2- 172.21.15.64:0/4154587739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6fc101890 0x7fc6fc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 >> 172.21.15.64:0/4154587739 conn(0x7fc6fc0fce00 msgr2=0x7fc6fc105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 shutdown_connections 2024-05-09T09:08:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.397+0000 7fc702f6a640 1 -- 172.21.15.64:0/4154587739 wait complete. 2024-05-09T09:08:32.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:08:32.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 -- 172.21.15.64:0/556831877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8101970 msgr2=0x7f07f8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 --2- 172.21.15.64:0/556831877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8101970 0x7f07f8101df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009900 tx=0x7f07ec02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:32.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 -- 172.21.15.64:0/556831877 shutdown_connections 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 --2- 172.21.15.64:0/556831877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8102330 0x7f07f8108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 --2- 172.21.15.64:0/556831877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8101970 0x7f07f8101df0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 --2- 172.21.15.64:0/556831877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f8100780 0x7f07f8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 -- 172.21.15.64:0/556831877 >> 172.21.15.64:0/556831877 conn(0x7f07f80fbd10 msgr2=0x7f07f80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 -- 172.21.15.64:0/556831877 shutdown_connections 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.514+0000 7f07fe815640 1 -- 172.21.15.64:0/556831877 wait complete. 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 Processor -- start 2024-05-09T09:08:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 -- start start 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8100780 0x7f07f8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f8101970 0x7f07f8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 0x7f07f819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07f810d430 con 0x7f07f8102330 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f810d2b0 con 0x7f07f8100780 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fe815640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07f810d5b0 con 0x7f07f8101970 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 0x7f07f819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 0x7f07f819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54568/0 (socket says 172.21.15.64:54568) 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 -- 172.21.15.64:0/2685590668 learned_addr learned my addr 172.21.15.64:0/2685590668 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07f77fe640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f8101970 0x7f07f8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07f7fff640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8100780 0x7f07f8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 -- 172.21.15.64:0/2685590668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f8101970 msgr2=0x7f07f8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f8101970 0x7f07f8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 -- 172.21.15.64:0/2685590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8100780 msgr2=0x7f07f8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8100780 0x7f07f8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07fcdb2640 1 -- 172.21.15.64:0/2685590668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f819f6b0 con 0x7f07f8102330 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.515+0000 7f07f7fff640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8100780 0x7f07f8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.516+0000 7f07fcdb2640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 0x7f07f819f040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f07f000eb10 tx=0x7f07f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.516+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004b20 con 0x7f07f8102330 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.516+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07f0004cc0 con 0x7f07f8102330 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.516+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f819f9a0 con 0x7f07f8102330 2024-05-09T09:08:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.516+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0010690 con 0x7f07f8102330 2024-05-09T09:08:32.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.516+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f819fea0 con 0x7f07f8102330 2024-05-09T09:08:32.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.517+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f8107aa0 con 0x7f07f8102330 2024-05-09T09:08:32.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.518+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f07f0005000 con 0x7f07f8102330 2024-05-09T09:08:32.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.519+0000 7f07f57fa640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07e00a2c20 0x7f07e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.519+0000 7f07f7fff640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07e00a2c20 0x7f07e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.519+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f07f00cfe30 con 0x7f07f8102330 2024-05-09T09:08:32.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.519+0000 7f07f7fff640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07e00a2c20 0x7f07e00a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07e80058c0 tx=0x7f07e8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.522+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07f0053740 con 0x7f07f8102330 2024-05-09T09:08:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.747+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07f802d410 con 0x7f07f8102330 2024-05-09T09:08:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.748+0000 7f07f57fa640 1 -- 172.21.15.64:0/2685590668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f07f009c8d0 con 0x7f07f8102330 2024-05-09T09:08:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:32.747 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:07:08.230747+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:08:32.747 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07e00a2c20 msgr2=0x7f07e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07e00a2c20 0x7f07e00a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07e80058c0 tx=0x7f07e8005d50 comp rx=0 tx=0).stop 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 msgr2=0x7f07f819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 0x7f07f819f040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f07f000eb10 tx=0x7f07f000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 shutdown_connections 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07e00a2c20 0x7f07e00a50e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f8102330 0x7f07f819f040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f8101970 0x7f07f8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 --2- 172.21.15.64:0/2685590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f8100780 0x7f07f8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 >> 172.21.15.64:0/2685590668 conn(0x7f07f80fbd10 msgr2=0x7f07f8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 shutdown_connections 2024-05-09T09:08:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.750+0000 7f07fe815640 1 -- 172.21.15.64:0/2685590668 wait complete. 2024-05-09T09:08:32.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 -- 172.21.15.64:0/1724547875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4102320 msgr2=0x7fc4c4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/1724547875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4102320 0x7fc4c4108bb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009ca0 tx=0x7fc4bc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 -- 172.21.15.64:0/1724547875 shutdown_connections 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/1724547875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4102320 0x7fc4c4108bb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/1724547875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c4101960 0x7fc4c4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/1724547875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4100770 0x7fc4c4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 -- 172.21.15.64:0/1724547875 >> 172.21.15.64:0/1724547875 conn(0x7fc4c40fbce0 msgr2=0x7fc4c40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 -- 172.21.15.64:0/1724547875 shutdown_connections 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.866+0000 7fc4c9c38640 1 -- 172.21.15.64:0/1724547875 wait complete. 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 Processor -- start 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 -- start start 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 0x7fc4c4104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4101960 0x7fc4c4102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c4102320 0x7fc4c4103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c410c280 con 0x7fc4c4100770 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c410c100 con 0x7fc4c4102320 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c9c38640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c410c400 con 0x7fc4c4101960 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 0x7fc4c4104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 0x7fc4c4104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54594/0 (socket says 172.21.15.64:54594) 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.867+0000 7fc4c37fe640 1 -- 172.21.15.64:0/3046154757 learned_addr learned my addr 172.21.15.64:0/3046154757 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c2ffd640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4101960 0x7fc4c4102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c3fff640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c4102320 0x7fc4c4103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c37fe640 1 -- 172.21.15.64:0/3046154757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4101960 msgr2=0x7fc4c4102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c37fe640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4101960 0x7fc4c4102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c37fe640 1 -- 172.21.15.64:0/3046154757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c4102320 msgr2=0x7fc4c4103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c37fe640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c4102320 0x7fc4c4103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c37fe640 1 -- 172.21.15.64:0/3046154757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4103aa0 con 0x7fc4c4100770 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c37fe640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 0x7fc4c4104690 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4c4100770 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c41a1b90 con 0x7fc4c4100770 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b4009c30 con 0x7fc4c4100770 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4003c90 con 0x7fc4c4100770 2024-05-09T09:08:32.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.868+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c41a2130 con 0x7fc4c4100770 2024-05-09T09:08:32.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.869+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c41a2300 con 0x7fc4c4100770 2024-05-09T09:08:32.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.870+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc4b4003e30 con 0x7fc4c4100770 2024-05-09T09:08:32.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.871+0000 7fc4c0ff9640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4ac0a2c20 0x7fc4ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.871+0000 7fc4c2ffd640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4ac0a2c20 0x7fc4ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.872+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc4b40ceef0 con 0x7fc4c4100770 2024-05-09T09:08:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.872+0000 7fc4c2ffd640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4ac0a2c20 0x7fc4ac0a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4104400 tx=0x7fc4b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:32.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:32.876+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc4b409b990 con 0x7fc4c4100770 2024-05-09T09:08:32.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:32 smithi179 ceph-mon[34086]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:32 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3623473452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:32 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3138527427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:32 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4154587739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.066+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4c41a25f0 con 0x7fc4c4100770 2024-05-09T09:08:33.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.067+0000 7fc4c0ff9640 1 -- 172.21.15.64:0/3046154757 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24653 (secure 0 0 0) 0x7fc4b4009de0 con 0x7fc4c4100770 2024-05-09T09:08:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:33.067 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":92,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:07:36.304544+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4ac0a2c20 msgr2=0x7fc4ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4ac0a2c20 0x7fc4ac0a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4104400 tx=0x7fc4b800b410 comp rx=0 tx=0).stop 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 msgr2=0x7fc4c4104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 0x7fc4c4104690 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).stop 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 shutdown_connections 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4ac0a2c20 0x7fc4ac0a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c4102320 0x7fc4c4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4101960 0x7fc4c4102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 --2- 172.21.15.64:0/3046154757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c4100770 0x7fc4c4104690 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 >> 172.21.15.64:0/3046154757 conn(0x7fc4c40fbce0 msgr2=0x7fc4c40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 shutdown_connections 2024-05-09T09:08:33.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.070+0000 7fc4c9c38640 1 -- 172.21.15.64:0/3046154757 wait complete. 2024-05-09T09:08:33.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:32 smithi139 ceph-mon[33599]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:33.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:32 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3623473452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:32 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3138527427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:32 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4154587739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.079 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:32 smithi064 ceph-mon[30149]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:33.079 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:32 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3623473452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.079 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:32 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3138527427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.079 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:32 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4154587739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:33.081 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 -- 172.21.15.64:0/4092006086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102320 msgr2=0x7f9b04108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- 172.21.15.64:0/4092006086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102320 0x7f9b04108bb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009ca0 tx=0x7f9afc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 -- 172.21.15.64:0/4092006086 shutdown_connections 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- 172.21.15.64:0/4092006086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102320 0x7f9b04108bb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- 172.21.15.64:0/4092006086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b04101960 0x7f9b04101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- 172.21.15.64:0/4092006086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b04100770 0x7f9b04100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 -- 172.21.15.64:0/4092006086 >> 172.21.15.64:0/4092006086 conn(0x7f9b040fbce0 msgr2=0x7f9b040fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 -- 172.21.15.64:0/4092006086 shutdown_connections 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 -- 172.21.15.64:0/4092006086 wait complete. 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 Processor -- start 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 -- start start 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b04100770 0x7f9b041968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 0x7f9b04196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.191+0000 7f9b0b812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b04102320 0x7f9b0419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b0b812640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0410c280 con 0x7f9b04101960 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b0b812640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0410c100 con 0x7f9b04102320 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b0b812640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0410c400 con 0x7f9b04100770 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 0x7f9b04196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 0x7f9b04196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54620/0 (socket says 172.21.15.64:54620) 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 -- 172.21.15.64:0/1603804336 learned_addr learned my addr 172.21.15.64:0/1603804336 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b095ae640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b04100770 0x7f9b041968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b09daf640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b04102320 0x7f9b0419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 -- 172.21.15.64:0/1603804336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b04100770 msgr2=0x7f9b041968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b04100770 0x7f9b041968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 -- 172.21.15.64:0/1603804336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b04102320 msgr2=0x7f9b0419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b04102320 0x7f9b0419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 -- 172.21.15.64:0/1603804336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0419e4c0 con 0x7f9b04101960 2024-05-09T09:08:33.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b08dad640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 0x7f9b04196de0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c890 tx=0x7f9af400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004220 con 0x7f9b04101960 2024-05-09T09:08:33.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9af40043c0 con 0x7f9b04101960 2024-05-09T09:08:33.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0419e7b0 con 0x7f9b04101960 2024-05-09T09:08:33.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004a00 con 0x7f9b04101960 2024-05-09T09:08:33.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.192+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0419ed00 con 0x7f9b04101960 2024-05-09T09:08:33.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.193+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b04107a30 con 0x7f9b04101960 2024-05-09T09:08:33.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.194+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9af4004ba0 con 0x7f9b04101960 2024-05-09T09:08:33.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.196+0000 7f9afa7fc640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9af00a2c20 0x7f9af00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.196+0000 7f9b095ae640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9af00a2c20 0x7f9af00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.196+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9af40cf1f0 con 0x7f9b04101960 2024-05-09T09:08:33.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.196+0000 7f9b095ae640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9af00a2c20 0x7f9af00a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b041017c0 tx=0x7f9b00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.200+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9af4014030 con 0x7f9b04101960 2024-05-09T09:08:33.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.423+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b0402d1c0 con 0x7f9b04101960 2024-05-09T09:08:33.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.424+0000 7f9afa7fc640 1 -- 172.21.15.64:0/1603804336 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f9af409bc90 con 0x7f9b04101960 2024-05-09T09:08:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:33.423 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:07:08.230747+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:08:33.423 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9af00a2c20 msgr2=0x7f9af00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9af00a2c20 0x7f9af00a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b041017c0 tx=0x7f9b00005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 msgr2=0x7f9b04196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 0x7f9b04196de0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c890 tx=0x7f9af400cd50 comp rx=0 tx=0).stop 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 shutdown_connections 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9af00a2c20 0x7f9af00a50e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b04102320 0x7f9b0419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04101960 0x7f9b04196de0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 --2- 172.21.15.64:0/1603804336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b04100770 0x7f9b041968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 >> 172.21.15.64:0/1603804336 conn(0x7f9b040fbce0 msgr2=0x7f9b04104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 shutdown_connections 2024-05-09T09:08:33.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.426+0000 7f9b0b812640 1 -- 172.21.15.64:0/1603804336 wait complete. 2024-05-09T09:08:33.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T09:06:40.458278+0000', 'modified': '2024-05-09T09:07:08.230747+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24503, 'mds_1': 24496, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6835/3760508086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3760508086}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 3760508086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/2410946724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2410946724}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2410946724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/502414567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 502414567}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 502414567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.179:6837/2858892786', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2858892786}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2858892786}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/1577800782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 1577800782}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 1577800782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6839/1823707244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 1823707244}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 1823707244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24487, 24503, 24496]} 2024-05-09T09:08:33.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T09:08:33.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 -- 172.21.15.64:0/2368727285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978104ca0 msgr2=0x7f6978107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 --2- 172.21.15.64:0/2368727285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978104ca0 0x7f6978107090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f696c009900 tx=0x7f696c02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 -- 172.21.15.64:0/2368727285 shutdown_connections 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 --2- 172.21.15.64:0/2368727285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69781075d0 0x7f69781099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 --2- 172.21.15.64:0/2368727285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978104ca0 0x7f6978107090 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 --2- 172.21.15.64:0/2368727285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978102370 0x7f6978104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 -- 172.21.15.64:0/2368727285 >> 172.21.15.64:0/2368727285 conn(0x7f69780fbd20 msgr2=0x7f69780fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 -- 172.21.15.64:0/2368727285 shutdown_connections 2024-05-09T09:08:33.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 -- 172.21.15.64:0/2368727285 wait complete. 2024-05-09T09:08:33.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 Processor -- start 2024-05-09T09:08:33.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.544+0000 7f697ea15640 1 -- start start 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 0x7f6978101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978104ca0 0x7f6978100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69781075d0 0x7f6978100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f697810d390 con 0x7f6978102370 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f697810d210 con 0x7f6978104ca0 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f697810d510 con 0x7f69781075d0 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 0x7f6978101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 0x7f6978101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54640/0 (socket says 172.21.15.64:54640) 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 -- 172.21.15.64:0/2282630715 learned_addr learned my addr 172.21.15.64:0/2282630715 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697cfb2640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69781075d0 0x7f6978100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f69777fe640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978104ca0 0x7f6978100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 -- 172.21.15.64:0/2282630715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69781075d0 msgr2=0x7f6978100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69781075d0 0x7f6978100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 -- 172.21.15.64:0/2282630715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978104ca0 msgr2=0x7f6978100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978104ca0 0x7f6978100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 -- 172.21.15.64:0/2282630715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69781010f0 con 0x7f6978102370 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f69777fe640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978104ca0 0x7f6978100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f6977fff640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 0x7f6978101ce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f696800d950 tx=0x7f696800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f69757fa640 1 -- 172.21.15.64:0/2282630715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968017070 con 0x7f6978102370 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f69757fa640 1 -- 172.21.15.64:0/2282630715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6968009c30 con 0x7f6978102370 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69781a1c70 con 0x7f6978102370 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f69757fa640 1 -- 172.21.15.64:0/2282630715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696800f620 con 0x7f6978102370 2024-05-09T09:08:33.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.545+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69781a21c0 con 0x7f6978102370 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f69757fa640 1 -- 172.21.15.64:0/2282630715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f69680109a0 con 0x7f6978102370 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 Processor -- start 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 -- start start 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x560273422ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x560273427db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560273428420 0x56027342a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5602733864d0 con 0x5602734228c0 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56027332e3e0 con 0x560273428420 2024-05-09T09:08:33.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6981803740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560273382490 con 0x5602734232b0 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f697cfb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x560273422ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f697cfb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x560273422ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54652/0 (socket says 172.21.15.64:54652) 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f697cfb2640 1 -- 172.21.15.64:0/1294561465 learned_addr learned my addr 172.21.15.64:0/1294561465 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f69777fe640 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560273428420 0x56027342a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f69757fa640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69600a2c20 0x7f69600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.548+0000 7f6977fff640 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x560273427db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 -- 172.21.15.64:0/1294561465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 msgr2=0x560273427db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x560273427db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 -- 172.21.15.64:0/1294561465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560273428420 msgr2=0x56027342a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560273428420 0x56027342a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 -- 172.21.15.64:0/1294561465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027309a2d0 con 0x5602734228c0 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f69777fe640 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560273428420 0x56027342a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69600a2c20 0x7f69600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f69757fa640 1 -- 172.21.15.64:0/2282630715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69680cf230 con 0x7f6978102370 2024-05-09T09:08:33.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f697cfb2640 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x560273422ce0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f697000f480 tx=0x7f697000f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bcaafb0808899d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f695e7fc640 1 -- 172.21.15.64:0/1294561465 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697001c400 con 0x5602734228c0 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f695e7fc640 1 -- 172.21.15.64:0/1294561465 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697001ca50 con 0x5602734228c0 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.549+0000 7f695e7fc640 1 -- 172.21.15.64:0/1294561465 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697001b6c0 con 0x5602734228c0 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 -- 172.21.15.64:0/1294561465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 msgr2=0x560273422ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f697cfb2640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69600a2c20 0x7f69600a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6970020000 tx=0x7f697001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x560273422ce0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f697000f480 tx=0x7f697000f940 comp rx=0 tx=0).stop 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 -- 172.21.15.64:0/1294561465 shutdown_connections 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560273428420 0x56027342a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x560273427db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 --2- 172.21.15.64:0/1294561465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x560273422ce0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 -- 172.21.15.64:0/1294561465 >> 172.21.15.64:0/1294561465 conn(0x56027341df30 msgr2=0x56027341e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 -- 172.21.15.64:0/1294561465 shutdown_connections 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 -- 172.21.15.64:0/1294561465 wait complete. 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 -- start start 2024-05-09T09:08:33.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.550+0000 7f6981803740 1 Processor -- start 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x56027342cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734232b0 0x56027342d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x5602734341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56027342b3d0 con 0x5602734228c0 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5602733418e0 con 0x5602734232b0 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56027342b550 con 0x560273428420 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 10 client.0 _ll_register_callbacks cb 0x56027330e9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x56027342cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x56027342cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54666/0 (socket says 172.21.15.64:54666) 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 -- 172.21.15.64:0/3900993719 learned_addr learned my addr 172.21.15.64:0/3900993719 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f69777fe640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734232b0 0x56027342d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f697cfb2640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x5602734341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 -- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 msgr2=0x5602734341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x5602734341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 -- 172.21.15.64:0/3900993719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734232b0 msgr2=0x56027342d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734232b0 0x56027342d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 -- 172.21.15.64:0/3900993719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602731646b0 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6977fff640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x56027342cca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f696809b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f12c3c7559d78c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f69577fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f69577fe640 1 -- 172.21.15.64:0/3900993719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968020070 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f69577fe640 1 -- 172.21.15.64:0/3900993719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6968097c40 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697001ca50 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.551+0000 7f69577fe640 1 -- 172.21.15.64:0/3900993719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f696809bcd0 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 10 client.15228 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560273437250 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f69577fe640 1 -- 172.21.15.64:0/3900993719 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697001ca50 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f69577fe640 1 -- 172.21.15.64:0/3900993719 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6968084b60 con 0x5602734228c0 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 10 client.15228 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 10 client.15228 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 20 client.15228 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:33.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734398a0 0x56027343ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 4 client.15228 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560273393c00 con 0x5602734398a0 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.552+0000 7f69777fe640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734398a0 0x56027343ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.553+0000 7f69577fe640 10 client.15228 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.553+0000 7f69777fe640 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734398a0 0x56027343ad10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52661ac211365d32 server_cookie=c59ecc326267f7a6 in_seq=0 out_seq=0 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f69577fe640 1 -- 172.21.15.64:0/3900993719 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560273393c00 con 0x5602734398a0 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f69577fe640 10 client.15228 handle_command_reply: tid=0 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 client.15228 shutdown 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 msgr2=0x56027342cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x56027342cca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f696809b210 comp rx=0 tx=0).stop 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 shutdown_connections 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734398a0 0x56027343ad10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x5602734341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734232b0 0x56027342d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 --2- 172.21.15.64:0/3900993719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734228c0 0x56027342cca0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f69577fe640 0 client.15228 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:33.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 >> 172.21.15.64:0/3900993719 conn(0x56027341df30 msgr2=0x56027341d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 shutdown_connections 2024-05-09T09:08:33.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 1 -- 172.21.15.64:0/3900993719 wait complete. 2024-05-09T09:08:33.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.554+0000 7f6981803740 20 client.15228 trim_cache size 0 max 16384 2024-05-09T09:08:33.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 Processor -- start 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 -- start start 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x56027342cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 0x5602734341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x56027342d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560273439050 con 0x5602734232b0 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560273439720 con 0x5602734228c0 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f6981803740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560273439df0 con 0x560273428420 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.572+0000 7f69777fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 0x5602734341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 0x5602734341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54668/0 (socket says 172.21.15.64:54668) 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 -- 172.21.15.64:0/2893350692 learned_addr learned my addr 172.21.15.64:0/2893350692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f697cfb2640 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x56027342d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 -- 172.21.15.64:0/2893350692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 msgr2=0x56027342d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x56027342d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 -- 172.21.15.64:0/2893350692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 msgr2=0x56027342cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f6977fff640 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x56027342cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x56027342cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 -- 172.21.15.64:0/2893350692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027341e980 con 0x5602734232b0 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f6977fff640 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x56027342cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f69777fe640 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 0x5602734341a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f696c0314e0 tx=0x7f696c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5329093409d12c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f695dffb640 1 -- 172.21.15.64:0/2893350692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c048070 con 0x5602734232b0 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f695dffb640 1 -- 172.21.15.64:0/2893350692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c0041b0 con 0x5602734232b0 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.573+0000 7f695dffb640 1 -- 172.21.15.64:0/2893350692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c035040 con 0x5602734232b0 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 -- 172.21.15.64:0/2893350692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 msgr2=0x5602734341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 0x5602734341a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f696c0314e0 tx=0x7f696c005b50 comp rx=0 tx=0).stop 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 -- 172.21.15.64:0/2893350692 shutdown_connections 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560273428420 0x56027342d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602734232b0 0x5602734341a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 --2- 172.21.15.64:0/2893350692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x56027342cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 -- 172.21.15.64:0/2893350692 >> 172.21.15.64:0/2893350692 conn(0x56027341df30 msgr2=0x560273424640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 -- 172.21.15.64:0/2893350692 shutdown_connections 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 -- 172.21.15.64:0/2893350692 wait complete. 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 -- start start 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.574+0000 7f6981803740 1 Processor -- start 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x560273433b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x5602734340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 0x560273430840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56027343ab90 con 0x560273428420 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56027343a4c0 con 0x5602734228c0 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56027343b260 con 0x5602734232b0 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 10 client.0 _ll_register_callbacks cb 0x7f696809b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 0x560273430840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 0x560273430840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54676/0 (socket says 172.21.15.64:54676) 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 -- 172.21.15.64:0/3259456081 learned_addr learned my addr 172.21.15.64:0/3259456081 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6977fff640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x560273433b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f69777fe640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x5602734340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 -- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 msgr2=0x5602734340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x5602734340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 -- 172.21.15.64:0/3259456081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 msgr2=0x560273433b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x560273433b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 -- 172.21.15.64:0/3259456081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560273431070 con 0x560273428420 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f697cfb2640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 0x560273430840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f697000fe40 tx=0x7f697001b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f745ad8a2248f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f69577fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f69577fe640 1 -- 172.21.15.64:0/3259456081 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970035020 con 0x560273428420 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f69577fe640 1 -- 172.21.15.64:0/3259456081 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69700049e0 con 0x560273428420 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.575+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696c0041b0 con 0x560273428420 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f69577fe640 1 -- 172.21.15.64:0/3259456081 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6970040050 con 0x560273428420 2024-05-09T09:08:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 10 client.15234 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5602734a05e0 con 0x560273428420 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f69577fe640 1 -- 172.21.15.64:0/3259456081 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c0041b0 con 0x560273428420 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f69577fe640 1 -- 172.21.15.64:0/3259456081 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6970005050 con 0x560273428420 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 10 client.15234 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 10 client.15234 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 20 client.15234 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734a5080 0x5602734a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 4 client.15234 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "status"}) -- 0x56027338df50 con 0x5602734a5080 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.576+0000 7f6977fff640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734a5080 0x5602734a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f69577fe640 10 client.15234 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6977fff640 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734a5080 0x5602734a3d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f75d197024f8eb server_cookie=2573bbac8c7e2f14 in_seq=0 out_seq=0 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f69577fe640 1 -- 172.21.15.64:0/3259456081 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56027338df50 con 0x5602734a5080 2024-05-09T09:08:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f69577fe640 10 client.15234 handle_command_reply: tid=0 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 client.15234 shutdown 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 msgr2=0x560273430840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 0x560273430840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f697000fe40 tx=0x7f697001b650 comp rx=0 tx=0).stop 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 shutdown_connections 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5602734a5080 0x5602734a3d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560273428420 0x560273430840 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602734232b0 0x5602734340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f69577fe640 0 client.15234 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 --2- 172.21.15.64:0/3259456081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5602734228c0 0x560273433b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 >> 172.21.15.64:0/3259456081 conn(0x56027341df30 msgr2=0x56027342cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 shutdown_connections 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 1 -- 172.21.15.64:0/3259456081 wait complete. 2024-05-09T09:08:33.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.577+0000 7f6981803740 20 client.15234 trim_cache size 0 max 16384 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24487, 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 86.350514834999998, 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 92, 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 133.03157345400001 2024-05-09T09:08:33.585 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:08:33.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69600a2c20 msgr2=0x7f69600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69600a2c20 0x7f69600a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6970020000 tx=0x7f697001d000 comp rx=0 tx=0).stop 2024-05-09T09:08:33.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 msgr2=0x7f6978101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 0x7f6978101ce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f696800d950 tx=0x7f696800de10 comp rx=0 tx=0).stop 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 shutdown_connections 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f69600a2c20 0x7f69600a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69781075d0 0x7f6978100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978104ca0 0x7f6978100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 --2- 172.21.15.64:0/2282630715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6978102370 0x7f6978101ce0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 >> 172.21.15.64:0/2282630715 conn(0x7f69780fbd20 msgr2=0x7f6978102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 shutdown_connections 2024-05-09T09:08:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.578+0000 7f697ea15640 1 -- 172.21.15.64:0/2282630715 wait complete. 2024-05-09T09:08:33.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-09T09:08:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 -- 172.21.15.64:0/91233195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 msgr2=0x7f068c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 --2- 172.21.15.64:0/91233195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c108bb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0684009900 tx=0x7f068402a410 comp rx=0 tx=0).stop 2024-05-09T09:08:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 -- 172.21.15.64:0/91233195 shutdown_connections 2024-05-09T09:08:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 --2- 172.21.15.64:0/91233195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c108bb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 --2- 172.21.15.64:0/91233195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c101960 0x7f068c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 --2- 172.21.15.64:0/91233195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c100770 0x7f068c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 -- 172.21.15.64:0/91233195 >> 172.21.15.64:0/91233195 conn(0x7f068c0fbce0 msgr2=0x7f068c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 -- 172.21.15.64:0/91233195 shutdown_connections 2024-05-09T09:08:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.696+0000 7f06910ee640 1 -- 172.21.15.64:0/91233195 wait complete. 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 Processor -- start 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 -- start start 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c100770 0x7f068c10b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c101960 0x7f068c109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10d3c0 con 0x7f068c102320 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10d240 con 0x7f068c101960 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f06910ee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10d540 con 0x7f068c100770 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f068b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f068b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54692/0 (socket says 172.21.15.64:54692) 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f068b59e640 1 -- 172.21.15.64:0/4181691778 learned_addr learned my addr 172.21.15.64:0/4181691778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.697+0000 7f068ad9d640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c100770 0x7f068c10b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068a59c640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c101960 0x7f068c109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068b59e640 1 -- 172.21.15.64:0/4181691778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c100770 msgr2=0x7f068c10b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068b59e640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c100770 0x7f068c10b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068b59e640 1 -- 172.21.15.64:0/4181691778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c101960 msgr2=0x7f068c109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068b59e640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c101960 0x7f068c109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068b59e640 1 -- 172.21.15.64:0/4181691778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c10a500 con 0x7f068c102320 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f068b59e640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c109c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0684005940 tx=0x7f06840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f0673fff640 1 -- 172.21.15.64:0/4181691778 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068403f030 con 0x7f068c102320 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c1a1b60 con 0x7f068c102320 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f0673fff640 1 -- 172.21.15.64:0/4181691778 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06840318f0 con 0x7f068c102320 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f0673fff640 1 -- 172.21.15.64:0/4181691778 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684035070 con 0x7f068c102320 2024-05-09T09:08:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.698+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1a20b0 con 0x7f068c102320 2024-05-09T09:08:33.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.700+0000 7f0673fff640 1 -- 172.21.15.64:0/4181691778 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0684042030 con 0x7f068c102320 2024-05-09T09:08:33.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 Processor -- start 2024-05-09T09:08:33.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 -- start start 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 0x55b97f10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f1108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b97f06c4d0 con 0x55b97f1092b0 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b97f0143e0 con 0x55b97f1088c0 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0693dfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b97f068490 con 0x55b97f10e420 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 0x55b97f10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 0x55b97f10ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54702/0 (socket says 172.21.15.64:54702) 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 -- 172.21.15.64:0/417523692 learned_addr learned my addr 172.21.15.64:0/417523692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068a59c640 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f1108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0673fff640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06740a2c20 0x7f06740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068a59c640 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068b59e640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06740a2c20 0x7f06740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 -- 172.21.15.64:0/417523692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 msgr2=0x55b97f1108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f1108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 -- 172.21.15.64:0/417523692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 msgr2=0x55b97f108ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f108ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f0673fff640 1 -- 172.21.15.64:0/4181691778 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f06840ef9e0 con 0x7f068c102320 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068ad9d640 1 -- 172.21.15.64:0/417523692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b97ed802d0 con 0x55b97f1092b0 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.701+0000 7f068a59c640 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f068ad9d640 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 0x55b97f10ddb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f067c011470 tx=0x7f067c011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21983dfe9de3091 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f06717fa640 1 -- 172.21.15.64:0/417523692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c003970 con 0x55b97f1092b0 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f068b59e640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06740a2c20 0x7f06740a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06840bbe80 tx=0x7f06840b13d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f06717fa640 1 -- 172.21.15.64:0/417523692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f067c004020 con 0x55b97f1092b0 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f06717fa640 1 -- 172.21.15.64:0/417523692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c00f930 con 0x55b97f1092b0 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 -- 172.21.15.64:0/417523692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 msgr2=0x55b97f10ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 0x55b97f10ddb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f067c011470 tx=0x7f067c011930 comp rx=0 tx=0).stop 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 -- 172.21.15.64:0/417523692 shutdown_connections 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f1108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1092b0 0x55b97f10ddb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 --2- 172.21.15.64:0/417523692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.702+0000 7f0693dfb740 1 -- 172.21.15.64:0/417523692 >> 172.21.15.64:0/417523692 conn(0x55b97f103f30 msgr2=0x55b97f104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 -- 172.21.15.64:0/417523692 shutdown_connections 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 -- 172.21.15.64:0/417523692 wait complete. 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 -- start start 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 Processor -- start 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 0x55b97f112d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1092b0 0x55b97f1132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f11a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b97f1113d0 con 0x55b97f1088c0 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b97f0278e0 con 0x55b97f1092b0 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b97f111550 con 0x55b97f10e420 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f0693dfb740 10 client.0 _ll_register_callbacks cb 0x55b97eff49c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f068a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 0x55b97f112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f068a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 0x55b97f112d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54708/0 (socket says 172.21.15.64:54708) 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f068a59c640 1 -- 172.21.15.64:0/2757557734 learned_addr learned my addr 172.21.15.64:0/2757557734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.703+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1092b0 0x55b97f1132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f068a59c640 1 -- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 msgr2=0x55b97f11a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f068a59c640 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f11a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f068a59c640 1 -- 172.21.15.64:0/2757557734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1092b0 msgr2=0x55b97f1132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f068a59c640 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1092b0 0x55b97f1132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f068a59c640 1 -- 172.21.15.64:0/2757557734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b97ee4a6b0 con 0x55b97f1088c0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f068a59c640 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 0x55b97f112d60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0680009d60 tx=0x7f0680010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8781cb5c969774c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f066eff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f066eff5640 1 -- 172.21.15.64:0/2757557734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068000fb90 con 0x55b97f1088c0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f066eff5640 1 -- 172.21.15.64:0/2757557734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0680009040 con 0x55b97f1088c0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067c004020 con 0x55b97f1088c0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f066eff5640 1 -- 172.21.15.64:0/2757557734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f068000fd30 con 0x55b97f1088c0 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f0693dfb740 10 client.15246 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:33.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b97f11d370 con 0x55b97f1088c0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f066eff5640 1 -- 172.21.15.64:0/2757557734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f067c004020 con 0x55b97f1088c0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f066eff5640 1 -- 172.21.15.64:0/2757557734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0680003c00 con 0x55b97f1088c0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.704+0000 7f0693dfb740 10 client.15246 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f0693dfb740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f0693dfb740 20 client.15246 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f11f9c0 0x55b97f120e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f0693dfb740 4 client.15246 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b97f079c00 con 0x55b97f11f9c0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f11f9c0 0x55b97f120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f066eff5640 10 client.15246 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.705+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f11f9c0 0x55b97f120e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de59e41bc5b9b8a9 server_cookie=f2eb27bc8d55d12 in_seq=0 out_seq=0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f066eff5640 1 -- 172.21.15.64:0/2757557734 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b97f079c00 con 0x55b97f11f9c0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f066eff5640 10 client.15246 handle_command_reply: tid=0 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 client.15246 shutdown 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 msgr2=0x55b97f112d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 0x55b97f112d60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0680009d60 tx=0x7f0680010f70 comp rx=0 tx=0).stop 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 shutdown_connections 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f10e420 0x55b97f11a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1092b0 0x55b97f1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f11f9c0 0x55b97f120e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2757557734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f1088c0 0x55b97f112d60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 >> 172.21.15.64:0/2757557734 conn(0x55b97f103f30 msgr2=0x55b97f103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f066eff5640 0 client.15246 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 shutdown_connections 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 1 -- 172.21.15.64:0/2757557734 wait complete. 2024-05-09T09:08:33.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.706+0000 7f0693dfb740 20 client.15246 trim_cache size 0 max 16384 2024-05-09T09:08:33.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 Processor -- start 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 -- start start 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f110810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f11a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b97f11f170 con 0x55b97f10e420 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b97f11f840 con 0x55b97f1088c0 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f0693dfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b97f11ff10 con 0x55b97f1092b0 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f068a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f068a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54720/0 (socket says 172.21.15.64:54720) 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.724+0000 7f068a59c640 1 -- 172.21.15.64:0/3348496288 learned_addr learned my addr 172.21.15.64:0/3348496288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068ad9d640 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f11a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068a59c640 1 -- 172.21.15.64:0/3348496288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 msgr2=0x55b97f11a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068a59c640 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f11a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068a59c640 1 -- 172.21.15.64:0/3348496288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 msgr2=0x55b97f110810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068a59c640 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f110810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068a59c640 1 -- 172.21.15.64:0/3348496288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b97f103a70 con 0x55b97f10e420 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f068a59c640 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f106fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f06800099e0 tx=0x7f0680009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4682492d0d1c7cee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f0670ff9640 1 -- 172.21.15.64:0/3348496288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680032070 con 0x55b97f10e420 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f0670ff9640 1 -- 172.21.15.64:0/3348496288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06800035f0 con 0x55b97f10e420 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.725+0000 7f0670ff9640 1 -- 172.21.15.64:0/3348496288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680004350 con 0x55b97f10e420 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- 172.21.15.64:0/3348496288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 msgr2=0x55b97f106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f106fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f06800099e0 tx=0x7f0680009bf0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- 172.21.15.64:0/3348496288 shutdown_connections 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f106fc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f11a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- 172.21.15.64:0/3348496288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- 172.21.15.64:0/3348496288 >> 172.21.15.64:0/3348496288 conn(0x55b97f103f30 msgr2=0x55b97f10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- 172.21.15.64:0/3348496288 shutdown_connections 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- 172.21.15.64:0/3348496288 wait complete. 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- start start 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 Processor -- start 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f115940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f115e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f118ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b97f120cb0 con 0x55b97f10e420 2024-05-09T09:08:33.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b97f1205e0 con 0x55b97f1088c0 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b97f121380 con 0x55b97f1092b0 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f0693dfb740 10 client.0 _ll_register_callbacks cb 0x7f0680010850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f068a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f118ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.726+0000 7f068a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f118ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54734/0 (socket says 172.21.15.64:54734) 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 -- 172.21.15.64:0/2109309450 learned_addr learned my addr 172.21.15.64:0/2109309450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f115940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 -- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 msgr2=0x55b97f115e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f115e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f115e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 -- 172.21.15.64:0/2109309450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 msgr2=0x55b97f115940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f115940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 -- 172.21.15.64:0/2109309450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b97f119490 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f115e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f068a59c640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f118ea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f068001e4e0 tx=0x7f0680005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a5795650ee1459 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f066eff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f066eff5640 1 -- 172.21.15.64:0/2109309450 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680032030 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f066eff5640 1 -- 172.21.15.64:0/2109309450 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f06800037a0 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06800035f0 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f066eff5640 1 -- 172.21.15.64:0/2109309450 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0680042050 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f0693dfb740 10 client.15252 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b97f185970 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.727+0000 7f066eff5640 1 -- 172.21.15.64:0/2109309450 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06800035f0 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f066eff5640 1 -- 172.21.15.64:0/2109309450 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f068002d900 con 0x55b97f10e420 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 10 client.15252 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:33.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 20 client.15252 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f18a370 0x55b97f189080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 4 client.15252 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "status"}) -- 0x55b97f073f50 con 0x55b97f18a370 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f18a370 0x55b97f189080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f066eff5640 10 client.15252 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f068ad9d640 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f18a370 0x55b97f189080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b1207d54d3a4ab3 server_cookie=90f940a4f4237ef2 in_seq=0 out_seq=0 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f066eff5640 1 -- 172.21.15.64:0/2109309450 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b97f073f50 con 0x55b97f18a370 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f066eff5640 10 client.15252 handle_command_reply: tid=0 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.728+0000 7f0693dfb740 1 client.15252 shutdown 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 msgr2=0x55b97f118ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f118ea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f068001e4e0 tx=0x7f0680005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:33.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 shutdown_connections 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b97f18a370 0x55b97f189080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b97f10e420 0x55b97f118ea0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b97f1092b0 0x55b97f115e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 --2- 172.21.15.64:0/2109309450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b97f1088c0 0x55b97f115940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f066eff5640 0 client.15252 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 >> 172.21.15.64:0/2109309450 conn(0x55b97f103f30 msgr2=0x55b97f106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 shutdown_connections 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 1 -- 172.21.15.64:0/2109309450 wait complete. 2024-05-09T09:08:33.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f0693dfb740 20 client.15252 trim_cache size 0 max 16384 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24496, 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 87.515770515, 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 92, 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:08:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 127.49692564199999 2024-05-09T09:08:33.738 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:08:33.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06740a2c20 msgr2=0x7f06740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06740a2c20 0x7f06740a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06840bbe80 tx=0x7f06840b13d0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 msgr2=0x7f068c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c109c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0684005940 tx=0x7f06840042f0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 shutdown_connections 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06740a2c20 0x7f06740a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c102320 0x7f068c109c70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068c101960 0x7f068c109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 --2- 172.21.15.64:0/4181691778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f068c100770 0x7f068c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.729+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 >> 172.21.15.64:0/4181691778 conn(0x7f068c0fbce0 msgr2=0x7f068c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.730+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 shutdown_connections 2024-05-09T09:08:33.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.730+0000 7f06910ee640 1 -- 172.21.15.64:0/4181691778 wait complete. 2024-05-09T09:08:33.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T09:08:33.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 -- 172.21.15.64:0/1764801584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870107610 msgr2=0x7f5870109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 --2- 172.21.15.64:0/1764801584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870107610 0x7f5870109a00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 -- 172.21.15.64:0/1764801584 shutdown_connections 2024-05-09T09:08:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 --2- 172.21.15.64:0/1764801584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870107610 0x7f5870109a00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 --2- 172.21.15.64:0/1764801584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5870104ce0 0x7f58701070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 --2- 172.21.15.64:0/1764801584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58701023b0 0x7f58701047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 -- 172.21.15.64:0/1764801584 >> 172.21.15.64:0/1764801584 conn(0x7f58700fbd20 msgr2=0x7f58700fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 -- 172.21.15.64:0/1764801584 shutdown_connections 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.848+0000 7f5876910640 1 -- 172.21.15.64:0/1764801584 wait complete. 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 Processor -- start 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 -- start start 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58701023b0 0x7f5870197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 0x7f5870197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5870107610 0x7f587019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f587010d3e0 con 0x7f5870104ce0 2024-05-09T09:08:33.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f587010d260 con 0x7f5870107610 2024-05-09T09:08:33.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5876910640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f587010d560 con 0x7f58701023b0 2024-05-09T09:08:33.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 0x7f5870197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 0x7f5870197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54764/0 (socket says 172.21.15.64:54764) 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f586f7fe640 1 -- 172.21.15.64:0/476331274 learned_addr learned my addr 172.21.15.64:0/476331274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f5874ead640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5870107610 0x7f587019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.849+0000 7f586ffff640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58701023b0 0x7f5870197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586f7fe640 1 -- 172.21.15.64:0/476331274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58701023b0 msgr2=0x7f5870197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586f7fe640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58701023b0 0x7f5870197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586f7fe640 1 -- 172.21.15.64:0/476331274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5870107610 msgr2=0x7f587019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586f7fe640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5870107610 0x7f587019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586f7fe640 1 -- 172.21.15.64:0/476331274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587019f6b0 con 0x7f5870104ce0 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586ffff640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58701023b0 0x7f5870197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586f7fe640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 0x7f5870197fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f585800d740 tx=0x7f585800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586d7fa640 1 -- 172.21.15.64:0/476331274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858017070 con 0x7f5870104ce0 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587019f9a0 con 0x7f5870104ce0 2024-05-09T09:08:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586d7fa640 1 -- 172.21.15.64:0/476331274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f585800de50 con 0x7f5870104ce0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f586d7fa640 1 -- 172.21.15.64:0/476331274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858004d60 con 0x7f5870104ce0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.850+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587019fea0 con 0x7f5870104ce0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.852+0000 7f586d7fa640 1 -- 172.21.15.64:0/476331274 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5858004f00 con 0x7f5870104ce0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 Processor -- start 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 -- start start 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd212080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd2120d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd2120fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bd211674d0 con 0x55bd21207cd0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bd2110f3e0 con 0x55bd212086c0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f587961e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bd21163490 con 0x55bd2120d830 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f5874ead640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd212080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f586d7fa640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f585c0a2c20 0x7f585c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f5874ead640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd212080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54772/0 (socket says 172.21.15.64:54772) 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f5874ead640 1 -- 172.21.15.64:0/4001748215 learned_addr learned my addr 172.21.15.64:0/4001748215 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f5874ead640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f585c0a2c20 0x7f585c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f586ffff640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd2120d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f586d7fa640 1 -- 172.21.15.64:0/476331274 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f58580cef10 con 0x7f5870104ce0 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.853+0000 7f586f7fe640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd2120fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 -- 172.21.15.64:0/4001748215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 msgr2=0x55bd2120fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd2120fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 -- 172.21.15.64:0/4001748215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 msgr2=0x55bd2120d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd2120d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 -- 172.21.15.64:0/4001748215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd20e7b2d0 con 0x55bd21207cd0 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f586ffff640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd2120d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f586f7fe640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd2120fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd212080f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5868009430 tx=0x7f5868031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede7d322e611b749 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f58667fc640 1 -- 172.21.15.64:0/4001748215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868031450 con 0x55bd21207cd0 2024-05-09T09:08:33.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f58667fc640 1 -- 172.21.15.64:0/4001748215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5868036070 con 0x55bd21207cd0 2024-05-09T09:08:33.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f58667fc640 1 -- 172.21.15.64:0/4001748215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868030b40 con 0x55bd21207cd0 2024-05-09T09:08:33.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.854+0000 7f5874ead640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f585c0a2c20 0x7f585c0a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f586804e000 tx=0x7f586804c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 -- 172.21.15.64:0/4001748215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 msgr2=0x55bd212080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd212080f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5868009430 tx=0x7f5868031c40 comp rx=0 tx=0).stop 2024-05-09T09:08:33.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 -- 172.21.15.64:0/4001748215 shutdown_connections 2024-05-09T09:08:33.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd2120fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd2120d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 --2- 172.21.15.64:0/4001748215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd212080f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 -- 172.21.15.64:0/4001748215 >> 172.21.15.64:0/4001748215 conn(0x55bd211fef30 msgr2=0x55bd211ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 -- 172.21.15.64:0/4001748215 shutdown_connections 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 -- 172.21.15.64:0/4001748215 wait complete. 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 -- start start 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.855+0000 7f587961e740 1 Processor -- start 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21211f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd212086c0 0x55bd21212450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 0x55bd21219490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bd21210660 con 0x55bd2120d830 2024-05-09T09:08:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bd211228e0 con 0x55bd21207cd0 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bd212107e0 con 0x55bd212086c0 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f587961e740 10 client.0 _ll_register_callbacks cb 0x55bd210ef9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 0x55bd21219490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 0x55bd21219490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54782/0 (socket says 172.21.15.64:54782) 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 -- 172.21.15.64:0/678718251 learned_addr learned my addr 172.21.15.64:0/678718251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f586f7fe640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd212086c0 0x55bd21212450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f586ffff640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21211f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 -- 172.21.15.64:0/678718251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd212086c0 msgr2=0x55bd21212450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd212086c0 0x55bd21212450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 -- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 msgr2=0x55bd21211f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21211f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 -- 172.21.15.64:0/678718251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd20f456b0 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f586ffff640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21211f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f5874ead640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 0x55bd21219490 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5868031970 tx=0x7f5868030fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf7e84ef18d05f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.856+0000 7f584f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f584f7fe640 1 -- 172.21.15.64:0/678718251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868059040 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f584f7fe640 1 -- 172.21.15.64:0/678718251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5868030ac0 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5868036070 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f584f7fe640 1 -- 172.21.15.64:0/678718251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f586806a050 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bd21210660 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f584f7fe640 1 -- 172.21.15.64:0/678718251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5868036070 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f584f7fe640 1 -- 172.21.15.64:0/678718251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5868065030 con 0x55bd2120d830 2024-05-09T09:08:33.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 20 client.15264 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd2121e7f0 0x55bd2121fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 4 client.15264 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.857+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd21174c00 con 0x55bd2121e7f0 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.858+0000 7f586ffff640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd2121e7f0 0x55bd2121fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.858+0000 7f586ffff640 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd2121e7f0 0x55bd2121fc60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d836349a3caa5da server_cookie=83b05dc7d0fa84ca in_seq=0 out_seq=0 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.858+0000 7f584f7fe640 10 client.15264 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f584f7fe640 1 -- 172.21.15.64:0/678718251 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd21174c00 con 0x55bd2121e7f0 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f584f7fe640 10 client.15264 handle_command_reply: tid=0 2024-05-09T09:08:33.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 client.15264 shutdown 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 msgr2=0x55bd21219490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 0x55bd21219490 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5868031970 tx=0x7f5868030fb0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 shutdown_connections 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd2121e7f0 0x55bd2121fc60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd2120d830 0x55bd21219490 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd212086c0 0x55bd21212450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f584f7fe640 0 client.15264 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 --2- 172.21.15.64:0/678718251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21211f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 >> 172.21.15.64:0/678718251 conn(0x55bd211fef30 msgr2=0x55bd211fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 shutdown_connections 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 1 -- 172.21.15.64:0/678718251 wait complete. 2024-05-09T09:08:33.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.859+0000 7f587961e740 20 client.15264 trim_cache size 0 max 16384 2024-05-09T09:08:33.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 Processor -- start 2024-05-09T09:08:33.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 -- start start 2024-05-09T09:08:33.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd2120fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd21219450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd211f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bd2121dfa0 con 0x55bd21207cd0 2024-05-09T09:08:33.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bd2121e670 con 0x55bd212086c0 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f587961e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bd2121ed40 con 0x55bd2120d830 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f586ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd2120fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f586ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd2120fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54784/0 (socket says 172.21.15.64:54784) 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.877+0000 7f586ffff640 1 -- 172.21.15.64:0/366041202 learned_addr learned my addr 172.21.15.64:0/366041202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f586ffff640 1 -- 172.21.15.64:0/366041202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 msgr2=0x55bd211f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f586ffff640 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd211f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f586ffff640 1 -- 172.21.15.64:0/366041202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 msgr2=0x55bd21219450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f586ffff640 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd21219450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f586ffff640 1 -- 172.21.15.64:0/366041202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd21212db0 con 0x55bd21207cd0 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f586ffff640 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd2120fc20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55bd21208ac0 tx=0x7f586000a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f1c25922565496 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f5865ffb640 1 -- 172.21.15.64:0/366041202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860013dd0 con 0x55bd21207cd0 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f5865ffb640 1 -- 172.21.15.64:0/366041202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f586001b3b0 con 0x55bd21207cd0 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f5865ffb640 1 -- 172.21.15.64:0/366041202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860005440 con 0x55bd21207cd0 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 -- 172.21.15.64:0/366041202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 msgr2=0x55bd2120fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd2120fc20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55bd21208ac0 tx=0x7f586000a3a0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 -- 172.21.15.64:0/366041202 shutdown_connections 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd211f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd212086c0 0x55bd21219450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 --2- 172.21.15.64:0/366041202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd21207cd0 0x55bd2120fc20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.878+0000 7f587961e740 1 -- 172.21.15.64:0/366041202 >> 172.21.15.64:0/366041202 conn(0x55bd211fef30 msgr2=0x55bd211fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 -- 172.21.15.64:0/366041202 shutdown_connections 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 -- 172.21.15.64:0/366041202 wait complete. 2024-05-09T09:08:33.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 -- start start 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 Processor -- start 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21214820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 0x55bd21214d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd21217d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bd2121fae0 con 0x55bd212086c0 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bd2121f410 con 0x55bd21207cd0 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bd212201b0 con 0x55bd2120d830 2024-05-09T09:08:33.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f587961e740 10 client.0 _ll_register_callbacks cb 0x7f5868056040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:33.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 0x55bd21214d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 0x55bd21214d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54792/0 (socket says 172.21.15.64:54792) 2024-05-09T09:08:33.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f586f7fe640 1 -- 172.21.15.64:0/3855576536 learned_addr learned my addr 172.21.15.64:0/3855576536 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:33.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.879+0000 7f5874ead640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd21217d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586ffff640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21214820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586f7fe640 1 -- 172.21.15.64:0/3855576536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 msgr2=0x55bd21217d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586f7fe640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd21217d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586f7fe640 1 -- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 msgr2=0x55bd21214820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586f7fe640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21214820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586f7fe640 1 -- 172.21.15.64:0/3855576536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd21218370 con 0x55bd212086c0 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f5874ead640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd21217d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f586f7fe640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 0x55bd21214d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5858009f60 tx=0x7f5858009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6b324aafb9af08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:33.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f584f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f584f7fe640 1 -- 172.21.15.64:0/3855576536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580b2020 con 0x55bd212086c0 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f584f7fe640 1 -- 172.21.15.64:0/3855576536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f58580ccb70 con 0x55bd212086c0 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586001b3b0 con 0x55bd212086c0 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f584f7fe640 1 -- 172.21.15.64:0/3855576536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5858020040 con 0x55bd212086c0 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f587961e740 10 client.15270 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bd212196e0 con 0x55bd212086c0 2024-05-09T09:08:33.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f584f7fe640 1 -- 172.21.15.64:0/3855576536 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f586001b3b0 con 0x55bd212086c0 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f584f7fe640 1 -- 172.21.15.64:0/3855576536 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f58580bb0b0 con 0x55bd212086c0 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.880+0000 7f587961e740 10 client.15270 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f587961e740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f587961e740 20 client.15270 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f587961e740 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd21289060 0x55bd21287d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f587961e740 4 client.15270 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:08:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "status"}) -- 0x55bd2116ef50 con 0x55bd21289060 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f586ffff640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd21289060 0x55bd21287d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f586ffff640 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd21289060 0x55bd21287d70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faefe3ff4b0779e4 server_cookie=6f22236317fb9ddb in_seq=0 out_seq=0 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f584f7fe640 10 client.15270 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f584f7fe640 1 -- 172.21.15.64:0/3855576536 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55bd2116ef50 con 0x55bd21289060 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f584f7fe640 10 client.15270 handle_command_reply: tid=0 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.881+0000 7f587961e740 1 client.15270 shutdown 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 msgr2=0x55bd21214d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 0x55bd21214d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5858009f60 tx=0x7f5858009e10 comp rx=0 tx=0).stop 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 shutdown_connections 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bd21289060 0x55bd21287d70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bd2120d830 0x55bd21217d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd212086c0 0x55bd21214d60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f584f7fe640 0 client.15270 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 --2- 172.21.15.64:0/3855576536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bd21207cd0 0x55bd21214820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 >> 172.21.15.64:0/3855576536 conn(0x55bd211fef30 msgr2=0x55bd21211d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 shutdown_connections 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 1 -- 172.21.15.64:0/3855576536 wait complete. 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f587961e740 20 client.15270 trim_cache size 0 max 16384 2024-05-09T09:08:33.903 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24503, 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 112.760266274, 2024-05-09T09:08:33.904 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-05-09T09:08:33.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 92, 2024-05-09T09:08:33.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:08:33.905 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 118.811709127 2024-05-09T09:08:33.905 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f585c0a2c20 msgr2=0x7f585c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f585c0a2c20 0x7f585c0a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f586804e000 tx=0x7f586804c000 comp rx=0 tx=0).stop 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 msgr2=0x7f5870197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 0x7f5870197fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f585800d740 tx=0x7f585800dc00 comp rx=0 tx=0).stop 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 shutdown_connections 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f585c0a2c20 0x7f585c0a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5870107610 0x7f587019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870104ce0 0x7f5870197fd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 --2- 172.21.15.64:0/476331274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58701023b0 0x7f5870197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:33.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 >> 172.21.15.64:0/476331274 conn(0x7f58700fbd20 msgr2=0x7f5870102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:33.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 shutdown_connections 2024-05-09T09:08:33.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:33.882+0000 7f5876910640 1 -- 172.21.15.64:0/476331274 wait complete. 2024-05-09T09:08:33.908 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T09:08:33.908 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T09:08:34.019 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:08:34.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T09:08:34.020 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:33 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2685590668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.021 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:33 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3046154757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.021 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:33 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1603804336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:33 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2685590668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.043 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:33 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3046154757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.043 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:33 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1603804336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:33 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2685590668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:33 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3046154757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:33 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1603804336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:34.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:34 smithi179 ceph-mon[34086]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:35.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:34 smithi139 ceph-mon[33599]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:34 smithi064 ceph-mon[30149]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:35.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-09T09:08:35.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-09T09:08:35.993 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-09T09:08:35.993 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-09T09:08:36.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-09T09:08:36.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-09T09:08:36.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 -- 172.21.15.64:0/3997032267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 msgr2=0x7fc2901051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:36.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 --2- 172.21.15.64:0/3997032267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc2901051f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009990 tx=0x7fc28c009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 -- 172.21.15.64:0/3997032267 shutdown_connections 2024-05-09T09:08:36.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 --2- 172.21.15.64:0/3997032267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc290108060 0x7fc29010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 --2- 172.21.15.64:0/3997032267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc2901051f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 --2- 172.21.15.64:0/3997032267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc290105730 0x7fc290107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 -- 172.21.15.64:0/3997032267 >> 172.21.15.64:0/3997032267 conn(0x7fc2900fc810 msgr2=0x7fc2900fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 -- 172.21.15.64:0/3997032267 shutdown_connections 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.102+0000 7fc29768a700 1 -- 172.21.15.64:0/3997032267 wait complete. 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 Processor -- start 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 -- start start 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc290102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc290105730 0x7fc290100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc290108060 0x7fc2901012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:36.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010ddf0 con 0x7fc290102e00 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010dc70 con 0x7fc290105730 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc29768a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010df70 con 0x7fc290108060 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc290102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc290102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44354/0 (socket says 172.21.15.64:44354) 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 -- 172.21.15.64:0/3953074854 learned_addr learned my addr 172.21.15.64:0/3953074854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295c27700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc290108060 0x7fc2901012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc294c25700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc290105730 0x7fc290100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 -- 172.21.15.64:0/3953074854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc290108060 msgr2=0x7fc2901012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc290108060 0x7fc2901012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 -- 172.21.15.64:0/3953074854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc290105730 msgr2=0x7fc290100db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc290105730 0x7fc290100db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.103+0000 7fc295426700 1 -- 172.21.15.64:0/3953074854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc290101ba0 con 0x7fc290102e00 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.104+0000 7fc295426700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc290102760 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00b5c0 tx=0x7fc28c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:36.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.104+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c003ca0 con 0x7fc290102e00 2024-05-09T09:08:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.104+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc28c004830 con 0x7fc290102e00 2024-05-09T09:08:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.104+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c005480 con 0x7fc290102e00 2024-05-09T09:08:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.104+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2901a2580 con 0x7fc290102e00 2024-05-09T09:08:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.104+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2901a2970 con 0x7fc290102e00 2024-05-09T09:08:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.105+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc290069010 con 0x7fc290102e00 2024-05-09T09:08:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.106+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc28c029060 con 0x7fc290102e00 2024-05-09T09:08:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.107+0000 7fc2867fc700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc27c0a2c20 0x7fc27c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.107+0000 7fc294c25700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc27c0a2c20 0x7fc27c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.107+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc28c0d0570 con 0x7fc290102e00 2024-05-09T09:08:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.108+0000 7fc294c25700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc27c0a2c20 0x7fc27c0a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc280005950 tx=0x7fc2800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:36.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.111+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc28c013070 con 0x7fc290102e00 2024-05-09T09:08:36.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.337+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc290062760 con 0x7fc27c0a2c20 2024-05-09T09:08:36.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.338+0000 7fc2867fc700 1 -- 172.21.15.64:0/3953074854 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc290062760 con 0x7fc27c0a2c20 2024-05-09T09:08:36.337 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2024-05-09T09:08:36.337 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2024-05-09T09:08:36.337 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2024-05-09T09:08:36.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc27c0a2c20 msgr2=0x7fc27c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:36.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc27c0a2c20 0x7fc27c0a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc280005950 tx=0x7fc2800058e0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 msgr2=0x7fc290102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc290102760 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00b5c0 tx=0x7fc28c0047b0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 shutdown_connections 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc27c0a2c20 0x7fc27c0a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc290108060 0x7fc2901012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc290102e00 0x7fc290102760 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 --2- 172.21.15.64:0/3953074854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc290105730 0x7fc290100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 >> 172.21.15.64:0/3953074854 conn(0x7fc2900fc810 msgr2=0x7fc290103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 shutdown_connections 2024-05-09T09:08:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:36.341+0000 7fc29768a700 1 -- 172.21.15.64:0/3953074854 wait complete. 2024-05-09T09:08:36.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:36 smithi179 ceph-mon[34086]: pgmap v352: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-09T09:08:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-09T09:08:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:36 smithi179 ceph-mon[34086]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T09:08:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:36 smithi179 ceph-mon[34086]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T09:08:37.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:36 smithi139 ceph-mon[33599]: pgmap v352: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:37.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-09T09:08:37.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-09T09:08:37.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:36 smithi139 ceph-mon[33599]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T09:08:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:36 smithi139 ceph-mon[33599]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T09:08:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:36 smithi064 ceph-mon[30149]: pgmap v352: 129 pgs: 129 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-09T09:08:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-09T09:08:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:36 smithi064 ceph-mon[30149]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T09:08:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:36 smithi064 ceph-mon[30149]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T09:08:37.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T09:08:37.483 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:37.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:37 smithi064 ceph-mon[30149]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:37.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:37 smithi064 ceph-mon[30149]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T09:08:37.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:37 smithi179 ceph-mon[34086]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:37.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:37 smithi179 ceph-mon[34086]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T09:08:38.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:37 smithi139 ceph-mon[33599]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:38.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:37 smithi139 ceph-mon[33599]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T09:08:38.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:38 smithi064 ceph-mon[30149]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-09T09:08:38.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:38 smithi064 ceph-mon[30149]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-09T09:08:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:38 smithi179 ceph-mon[34086]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-09T09:08:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:38 smithi179 ceph-mon[34086]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-09T09:08:39.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:38 smithi139 ceph-mon[33599]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-09T09:08:39.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:38 smithi139 ceph-mon[33599]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-09T09:08:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 -- 172.21.15.64:0/1565558612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78102e30 msgr2=0x7fab78105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 --2- 172.21.15.64:0/1565558612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78102e30 0x7fab78105220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fab68009990 tx=0x7fab68009ca0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 -- 172.21.15.64:0/1565558612 shutdown_connections 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 --2- 172.21.15.64:0/1565558612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 0x7fab7810a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 --2- 172.21.15.64:0/1565558612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78102e30 0x7fab78105220 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 --2- 172.21.15.64:0/1565558612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78105760 0x7fab78107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 -- 172.21.15.64:0/1565558612 >> 172.21.15.64:0/1565558612 conn(0x7fab780fc840 msgr2=0x7fab780fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 -- 172.21.15.64:0/1565558612 shutdown_connections 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.329+0000 7fab7cde5700 1 -- 172.21.15.64:0/1565558612 wait complete. 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 Processor -- start 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 -- start start 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78102e30 0x7fab78198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 0x7fab78198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 0x7fab7819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810dea0 con 0x7fab78105760 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810dd20 con 0x7fab78102e30 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab7cde5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810e020 con 0x7fab78108090 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab75d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 0x7fab78198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab75d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 0x7fab78198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44368/0 (socket says 172.21.15.64:44368) 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.330+0000 7fab75d9b700 1 -- 172.21.15.64:0/4132313382 learned_addr learned my addr 172.21.15.64:0/4132313382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab76d9d700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 0x7fab7819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab7659c700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78102e30 0x7fab78198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab75d9b700 1 -- 172.21.15.64:0/4132313382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 msgr2=0x7fab7819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab75d9b700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 0x7fab7819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab75d9b700 1 -- 172.21.15.64:0/4132313382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78102e30 msgr2=0x7fab78198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab75d9b700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78102e30 0x7fab78198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab75d9b700 1 -- 172.21.15.64:0/4132313382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab781a0180 con 0x7fab78105760 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab76d9d700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 0x7fab7819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab7659c700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78102e30 0x7fab78198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab75d9b700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 0x7fab78198a60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b560 tx=0x7fab6c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c015020 con 0x7fab78105760 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab781a0470 con 0x7fab78105760 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab6c011ce0 con 0x7fab78105760 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c010da0 con 0x7fab78105760 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.331+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab781a0a00 con 0x7fab78105760 2024-05-09T09:08:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.332+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab78069010 con 0x7fab78105760 2024-05-09T09:08:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.334+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab6c018020 con 0x7fab78105760 2024-05-09T09:08:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.334+0000 7fab677fe700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fab600a2c20 0x7fab600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.334+0000 7fab7659c700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fab600a2c20 0x7fab600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.334+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab6c0cec10 con 0x7fab78105760 2024-05-09T09:08:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.335+0000 7fab7659c700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fab600a2c20 0x7fab600a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fab6800b5c0 tx=0x7fab68003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:39.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.339+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab6c01f020 con 0x7fab78105760 2024-05-09T09:08:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.556+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fab78062760 con 0x7fab600a2c20 2024-05-09T09:08:39.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.572+0000 7fab677fe700 1 -- 172.21.15.64:0/4132313382 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fab78062760 con 0x7fab600a2c20 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.a smithi064 running (2m) 19s ago 2m 12.9M - 19.0.0-2515-g96805dcf d7acac7d5657 5f814df7527c 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.b smithi139 running (2m) 58s ago 2m 25.7M - 19.0.0-2515-g96805dcf d7acac7d5657 7b9d727b4326 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.c smithi179 running (2m) 58s ago 2m 16.7M - 19.0.0-2515-g96805dcf d7acac7d5657 0fccc11519eb 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.d smithi064 running (2m) 19s ago 2m 14.0M - 19.0.0-2515-g96805dcf d7acac7d5657 d52b83eb665b 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.e smithi139 running (2m) 58s ago 2m 18.2M - 19.0.0-2515-g96805dcf d7acac7d5657 816b3bfd0e59 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.f smithi179 running (2m) 58s ago 2m 17.1M - 19.0.0-2515-g96805dcf d7acac7d5657 9ce10c83ce2b 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.g smithi064 running (2m) 19s ago 2m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 05109ada3bef 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.h smithi139 running (2m) 58s ago 2m 16.2M - 19.0.0-2515-g96805dcf d7acac7d5657 83a4eba0ae3d 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.i smithi179 running (2m) 58s ago 2m 19.2M - 19.0.0-2515-g96805dcf d7acac7d5657 d5f5d76ae01a 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.j smithi064 running (2m) 19s ago 2m 14.0M - 19.0.0-2515-g96805dcf d7acac7d5657 60e87e37b152 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.k smithi139 running (2m) 58s ago 2m 14.1M - 19.0.0-2515-g96805dcf d7acac7d5657 6290d5fab291 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mds.l smithi179 running (2m) 58s ago 2m 13.9M - 19.0.0-2515-g96805dcf d7acac7d5657 7b3cac009369 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.x smithi064 *:9283,8765 running (9m) 19s ago 9m 515M - 19.0.0-2515-g96805dcf d7acac7d5657 ef41d033accc 2024-05-09T09:08:39.571 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.y smithi139 *:8443,8765 running (7m) 58s ago 7m 454M - 19.0.0-2515-g96805dcf d7acac7d5657 92f936f5be87 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.z smithi179 *:8443,8765 running (7m) 58s ago 7m 452M - 19.0.0-2515-g96805dcf d7acac7d5657 5170f2b17a64 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:mon.a smithi064 running (10m) 19s ago 10m 48.9M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8a68b30aba11 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:mon.b smithi139 running (8m) 58s ago 8m 40.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 0a9efa793d19 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:mon.c smithi179 running (8m) 58s ago 8m 40.1M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 915ca2680a39 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (7m) 19s ago 7m 45.8M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 5240c23a9f55 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi139 running (7m) 58s ago 7m 53.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 e23eb75103fe 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi179 running (6m) 58s ago 6m 54.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 5a7f4e139547 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi064 running (6m) 19s ago 6m 54.8M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 e6dca3b6e2dc 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi139 running (5m) 58s ago 5m 46.0M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 74427bc88562 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi179 running (5m) 58s ago 5m 52.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 29232c80157f 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 smithi064 running (5m) 19s ago 5m 43.2M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 e6d27b07303d 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 smithi139 running (4m) 58s ago 4m 44.1M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 db2fb58726df 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 smithi179 running (4m) 58s ago 4m 51.9M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 3766cdb5154c 2024-05-09T09:08:39.572 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 smithi064 running (3m) 19s ago 3m 51.0M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 f6c8d63bf25b 2024-05-09T09:08:39.573 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 smithi139 running (3m) 58s ago 3m 43.4M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 22a8c6220257 2024-05-09T09:08:39.573 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 smithi179 running (2m) 58s ago 2m 49.8M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 9d87f069b0ad 2024-05-09T09:08:39.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fab600a2c20 msgr2=0x7fab600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:39.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fab600a2c20 0x7fab600a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fab6800b5c0 tx=0x7fab68003680 comp rx=0 tx=0).stop 2024-05-09T09:08:39.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 msgr2=0x7fab78198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:39.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 0x7fab78198a60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b560 tx=0x7fab6c00b920 comp rx=0 tx=0).stop 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 shutdown_connections 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fab600a2c20 0x7fab600a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab78108090 0x7fab7819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab78102e30 0x7fab78198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 --2- 172.21.15.64:0/4132313382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab78105760 0x7fab78198a60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 >> 172.21.15.64:0/4132313382 conn(0x7fab780fc840 msgr2=0x7fab78103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 shutdown_connections 2024-05-09T09:08:39.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:39.574+0000 7fab7cde5700 1 -- 172.21.15.64:0/4132313382 wait complete. 2024-05-09T09:08:40.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T09:08:40.536 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:40 smithi179 ceph-mon[34086]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-05-09T09:08:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:40 smithi179 ceph-mon[34086]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:40.979 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:40 smithi064 ceph-mon[30149]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-05-09T09:08:40.979 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:40 smithi064 ceph-mon[30149]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:41.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:40 smithi139 ceph-mon[33599]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-05-09T09:08:41.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:40 smithi139 ceph-mon[33599]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:41.817 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:41 smithi064 ceph-mon[30149]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-05-09T09:08:41.817 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:41 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:08:42.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:41 smithi139 ceph-mon[33599]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-05-09T09:08:42.275 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:41 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:08:42.276 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:41 smithi179 ceph-mon[34086]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-05-09T09:08:42.276 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:41 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:08:42.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1406249217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824108110 msgr2=0x7fa82410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:42.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1406249217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824108110 0x7fa82410a500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b0a0 tx=0x7fa81c00b3b0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1406249217 shutdown_connections 2024-05-09T09:08:42.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1406249217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa824108110 0x7fa82410a500 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1406249217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824102eb0 0x7fa8241052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1406249217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8241057e0 0x7fa824107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1406249217 >> 172.21.15.64:0/1406249217 conn(0x7fa8240fc840 msgr2=0x7fa8240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1406249217 shutdown_connections 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.380+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1406249217 wait complete. 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 Processor -- start 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 -- start start 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824102eb0 0x7fa8241961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 0x7fa824196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824108110 0x7fa82419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410de40 con 0x7fa8241057e0 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410dcc0 con 0x7fa824108110 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa82ae2a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410dfc0 con 0x7fa824102eb0 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa823fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 0x7fa824196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa823fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 0x7fa824196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44386/0 (socket says 172.21.15.64:44386) 2024-05-09T09:08:42.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa823fff700 1 -- 172.21.15.64:0/1927045094 learned_addr learned my addr 172.21.15.64:0/1927045094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa8293c7700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824108110 0x7fa82419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.381+0000 7fa828bc6700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824102eb0 0x7fa8241961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa823fff700 1 -- 172.21.15.64:0/1927045094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824102eb0 msgr2=0x7fa8241961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa823fff700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824102eb0 0x7fa8241961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa823fff700 1 -- 172.21.15.64:0/1927045094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824108110 msgr2=0x7fa82419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa823fff700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824108110 0x7fa82419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa823fff700 1 -- 172.21.15.64:0/1927045094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419deb0 con 0x7fa8241057e0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa823fff700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 0x7fa824196730 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c9b0 tx=0x7fa81800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818007b00 con 0x7fa8241057e0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa818007ca0 con 0x7fa8241057e0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419e1a0 con 0x7fa8241057e0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818018660 con 0x7fa8241057e0 2024-05-09T09:08:42.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.382+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82419e6e0 con 0x7fa8241057e0 2024-05-09T09:08:42.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.383+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa824069010 con 0x7fa8241057e0 2024-05-09T09:08:42.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.385+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa81800f450 con 0x7fa8241057e0 2024-05-09T09:08:42.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.385+0000 7fa821ffb700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa80c0a2c20 0x7fa80c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:42.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.385+0000 7fa828bc6700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa80c0a2c20 0x7fa80c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:42.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.385+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa8180cf340 con 0x7fa8241057e0 2024-05-09T09:08:42.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.386+0000 7fa828bc6700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa80c0a2c20 0x7fa80c0a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa814005950 tx=0x7fa8140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:42.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.390+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa818014030 con 0x7fa8241057e0 2024-05-09T09:08:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.606+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa824109750 con 0x7fa80c0a2c20 2024-05-09T09:08:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.612+0000 7fa821ffb700 1 -- 172.21.15.64:0/1927045094 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa824109750 con 0x7fa80c0a2c20 2024-05-09T09:08:42.612 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T09:08:42.612 INFO:teuthology.orchestra.run.smithi064.stdout:mds.all 12/12 61s ago 2m smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi139=b;smithi139=e;smithi139=h;smithi139=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-09T09:08:42.612 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 3/3 61s ago 8m smithi064=x;smithi139=y;smithi179=z;count:3 2024-05-09T09:08:42.613 INFO:teuthology.orchestra.run.smithi064.stdout:mon 3/3 61s ago 8m smithi064:172.21.15.64=a;smithi139:172.21.15.139=b;smithi179:172.21.15.179=c;count:3 2024-05-09T09:08:42.613 INFO:teuthology.orchestra.run.smithi064.stdout:osd 12 61s ago - 2024-05-09T09:08:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.614+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa80c0a2c20 msgr2=0x7fa80c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.614+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa80c0a2c20 0x7fa80c0a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa814005950 tx=0x7fa8140058e0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.614+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 msgr2=0x7fa824196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.614+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 0x7fa824196730 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c9b0 tx=0x7fa81800ccc0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 shutdown_connections 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa824108110 0x7fa82419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa80c0a2c20 0x7fa80c0a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa824102eb0 0x7fa8241961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 --2- 172.21.15.64:0/1927045094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8241057e0 0x7fa824196730 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 >> 172.21.15.64:0/1927045094 conn(0x7fa8240fc840 msgr2=0x7fa824102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 shutdown_connections 2024-05-09T09:08:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:42.615+0000 7fa82ae2a700 1 -- 172.21.15.64:0/1927045094 wait complete. 2024-05-09T09:08:43.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:42 smithi139 ceph-mon[33599]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s 2024-05-09T09:08:43.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:42 smithi064 ceph-mon[30149]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s 2024-05-09T09:08:43.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:42 smithi179 ceph-mon[34086]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s 2024-05-09T09:08:43.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T09:08:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:43.932 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:43 smithi064 ceph-mon[30149]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:44.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:43 smithi139 ceph-mon[33599]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:44.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:43 smithi179 ceph-mon[34086]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:45.043 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:44 smithi064 ceph-mon[30149]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s 2024-05-09T09:08:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:44 smithi139 ceph-mon[33599]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s 2024-05-09T09:08:45.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:44 smithi179 ceph-mon[34086]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s 2024-05-09T09:08:45.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 -- 172.21.15.64:0/2845702286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4102660 msgr2=0x7f45d4108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:45.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 --2- 172.21.15.64:0/2845702286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4102660 0x7f45d4108f10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00b0d0 tx=0x7f45cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 -- 172.21.15.64:0/2845702286 shutdown_connections 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 --2- 172.21.15.64:0/2845702286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4102660 0x7f45d4108f10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 --2- 172.21.15.64:0/2845702286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45d4100ac0 0x7f45d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 --2- 172.21.15.64:0/2845702286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d4101cc0 0x7f45d4102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 -- 172.21.15.64:0/2845702286 >> 172.21.15.64:0/2845702286 conn(0x7f45d40fc060 msgr2=0x7f45d40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 -- 172.21.15.64:0/2845702286 shutdown_connections 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 -- 172.21.15.64:0/2845702286 wait complete. 2024-05-09T09:08:45.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.304+0000 7f45d9285700 1 Processor -- start 2024-05-09T09:08:45.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 -- start start 2024-05-09T09:08:45.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45d4100ac0 0x7f45d4104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:45.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 0x7f45d41030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:45.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d4102660 0x7f45d41035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:45.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45d410bce0 con 0x7f45d4101cc0 2024-05-09T09:08:45.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45d410bb60 con 0x7f45d4100ac0 2024-05-09T09:08:45.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d9285700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f45d410be60 con 0x7f45d4102660 2024-05-09T09:08:45.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 0x7f45d41030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:45.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 0x7f45d41030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35192/0 (socket says 172.21.15.64:35192) 2024-05-09T09:08:45.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 -- 172.21.15.64:0/751818490 learned_addr learned my addr 172.21.15.64:0/751818490 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:45.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d2ffd700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45d4100ac0 0x7f45d4104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:45.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d37fe700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d4102660 0x7f45d41035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:45.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 -- 172.21.15.64:0/751818490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d4102660 msgr2=0x7f45d41035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:45.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d4102660 0x7f45d41035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 -- 172.21.15.64:0/751818490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45d4100ac0 msgr2=0x7f45d4104a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:45.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45d4100ac0 0x7f45d4104a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.305+0000 7f45d27fc700 1 -- 172.21.15.64:0/751818490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d4103e90 con 0x7f45d4101cc0 2024-05-09T09:08:45.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.306+0000 7f45d27fc700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 0x7f45d41030a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f45c800ddf0 tx=0x7f45c800a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:45.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.306+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800de20 con 0x7f45d4101cc0 2024-05-09T09:08:45.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.306+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d41a2660 con 0x7f45d4101cc0 2024-05-09T09:08:45.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.306+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45c8004ce0 con 0x7f45d4101cc0 2024-05-09T09:08:45.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.306+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800e450 con 0x7f45d4101cc0 2024-05-09T09:08:45.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.306+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d41a2b70 con 0x7f45d4101cc0 2024-05-09T09:08:45.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.307+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d4069010 con 0x7f45d4101cc0 2024-05-09T09:08:45.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.308+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f45c8004750 con 0x7f45d4101cc0 2024-05-09T09:08:45.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.309+0000 7f45bbfff700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f45bc0a2c20 0x7f45bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:45.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.309+0000 7f45d2ffd700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f45bc0a2c20 0x7f45bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:45.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.309+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f45c80cf780 con 0x7f45d4101cc0 2024-05-09T09:08:45.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.310+0000 7f45d2ffd700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f45bc0a2c20 0x7f45bc0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f45d4101b20 tx=0x7f45c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:45.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.313+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45c809c2f0 con 0x7f45d4101cc0 2024-05-09T09:08:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.525+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f45d4108820 con 0x7f45bc0a2c20 2024-05-09T09:08:45.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.527+0000 7f45bbfff700 1 -- 172.21.15.64:0/751818490 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f45d4108820 con 0x7f45bc0a2c20 2024-05-09T09:08:45.527 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2024-05-09T09:08:45.527 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2024-05-09T09:08:45.527 INFO:teuthology.orchestra.run.smithi064.stdout:smithi139 172.21.15.139 2024-05-09T09:08:45.527 INFO:teuthology.orchestra.run.smithi064.stdout:smithi179 172.21.15.179 2024-05-09T09:08:45.527 INFO:teuthology.orchestra.run.smithi064.stdout:3 hosts in cluster 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f45bc0a2c20 msgr2=0x7f45bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f45bc0a2c20 0x7f45bc0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f45d4101b20 tx=0x7f45c400b410 comp rx=0 tx=0).stop 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 msgr2=0x7f45d41030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 0x7f45d41030a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f45c800ddf0 tx=0x7f45c800a530 comp rx=0 tx=0).stop 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 shutdown_connections 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f45bc0a2c20 0x7f45bc0a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d4102660 0x7f45d41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45d4100ac0 0x7f45d4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 --2- 172.21.15.64:0/751818490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d4101cc0 0x7f45d41030a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 >> 172.21.15.64:0/751818490 conn(0x7f45d40fc060 msgr2=0x7f45d4107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 shutdown_connections 2024-05-09T09:08:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:45.530+0000 7f45d9285700 1 -- 172.21.15.64:0/751818490 wait complete. 2024-05-09T09:08:46.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T09:08:46.560 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:46.948 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:46 smithi064 ceph-mon[30149]: pgmap v359: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 325 B/s wr, 6 op/s 2024-05-09T09:08:46.948 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:46 smithi064 ceph-mon[30149]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:47.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:46 smithi139 ceph-mon[33599]: pgmap v359: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 325 B/s wr, 6 op/s 2024-05-09T09:08:47.076 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:46 smithi139 ceph-mon[33599]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:47.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:46 smithi179 ceph-mon[34086]: pgmap v359: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 325 B/s wr, 6 op/s 2024-05-09T09:08:47.233 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:46 smithi179 ceph-mon[34086]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:48.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 -- 172.21.15.64:0/1576206775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08107ff0 msgr2=0x7fad0810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:48.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 --2- 172.21.15.64:0/1576206775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08107ff0 0x7fad0810a400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fad0000b0a0 tx=0x7fad0000b3b0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 -- 172.21.15.64:0/1576206775 shutdown_connections 2024-05-09T09:08:48.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 --2- 172.21.15.64:0/1576206775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08107ff0 0x7fad0810a400 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 --2- 172.21.15.64:0/1576206775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad08102d90 0x7fad08105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 --2- 172.21.15.64:0/1576206775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad081056c0 0x7fad08107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 -- 172.21.15.64:0/1576206775 >> 172.21.15.64:0/1576206775 conn(0x7fad080fc740 msgr2=0x7fad080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 -- 172.21.15.64:0/1576206775 shutdown_connections 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 -- 172.21.15.64:0/1576206775 wait complete. 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.301+0000 7fad0eda1700 1 Processor -- start 2024-05-09T09:08:48.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 -- start start 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 0x7fad0819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad081056c0 0x7fad0819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad08107ff0 0x7fad081a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810dd10 con 0x7fad08102d90 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810db90 con 0x7fad081056c0 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0eda1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810de90 con 0x7fad08107ff0 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 0x7fad0819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 0x7fad0819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35218/0 (socket says 172.21.15.64:35218) 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 -- 172.21.15.64:0/3872027439 learned_addr learned my addr 172.21.15.64:0/3872027439 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0d59e700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad081056c0 0x7fad0819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:48.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0e5a0700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad08107ff0 0x7fad081a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 -- 172.21.15.64:0/3872027439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad08107ff0 msgr2=0x7fad081a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad08107ff0 0x7fad081a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 -- 172.21.15.64:0/3872027439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad081056c0 msgr2=0x7fad0819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad081056c0 0x7fad0819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.302+0000 7fad0dd9f700 1 -- 172.21.15.64:0/3872027439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad081a2190 con 0x7fad08102d90 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.303+0000 7fad0dd9f700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 0x7fad0819a530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fad0400d980 tx=0x7fad0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.303+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04016070 con 0x7fad08102d90 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.303+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad04009d80 con 0x7fad08102d90 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.303+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081a2480 con 0x7fad08102d90 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.303+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0400f3c0 con 0x7fad08102d90 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.303+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a29c0 con 0x7fad08102d90 2024-05-09T09:08:48.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.304+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad08069010 con 0x7fad08102d90 2024-05-09T09:08:48.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.305+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad04019070 con 0x7fad08102d90 2024-05-09T09:08:48.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.306+0000 7facfeffd700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facf40a2c20 0x7facf40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:48.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.306+0000 7fad0d59e700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facf40a2c20 0x7facf40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:48.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.306+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad040cee50 con 0x7fad08102d90 2024-05-09T09:08:48.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.307+0000 7fad0d59e700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facf40a2c20 0x7facf40a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7facf8005950 tx=0x7facf800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:48.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.309+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad0409ba20 con 0x7fad08102d90 2024-05-09T09:08:48.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.522+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fad08062760 con 0x7facf40a2c20 2024-05-09T09:08:48.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.524+0000 7facfeffd700 1 -- 172.21.15.64:0/3872027439 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fad08062760 con 0x7facf40a2c20 2024-05-09T09:08:48.523 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T09:08:48.523 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T09:08:48.523 INFO:teuthology.orchestra.run.smithi064.stdout:smithi139 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T09:08:48.523 INFO:teuthology.orchestra.run.smithi064.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-09T09:08:48.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facf40a2c20 msgr2=0x7facf40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:48.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facf40a2c20 0x7facf40a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7facf8005950 tx=0x7facf800b410 comp rx=0 tx=0).stop 2024-05-09T09:08:48.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 msgr2=0x7fad0819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 0x7fad0819a530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fad0400d980 tx=0x7fad0400dc90 comp rx=0 tx=0).stop 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 shutdown_connections 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facf40a2c20 0x7facf40a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad08107ff0 0x7fad081a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad08102d90 0x7fad0819a530 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 --2- 172.21.15.64:0/3872027439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad081056c0 0x7fad0819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 >> 172.21.15.64:0/3872027439 conn(0x7fad080fc740 msgr2=0x7fad08102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 shutdown_connections 2024-05-09T09:08:48.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:48.526+0000 7fad0eda1700 1 -- 172.21.15.64:0/3872027439 wait complete. 2024-05-09T09:08:49.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:48 smithi139 ceph-mon[33599]: pgmap v360: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 294 B/s wr, 6 op/s 2024-05-09T09:08:49.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:48 smithi064 ceph-mon[30149]: pgmap v360: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 294 B/s wr, 6 op/s 2024-05-09T09:08:49.162 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T09:08:49.234 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:08:49.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T09:08:49.235 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:48 smithi179 ceph-mon[34086]: pgmap v360: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 294 B/s wr, 6 op/s 2024-05-09T09:08:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:49.880 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:49 smithi064 ceph-mon[30149]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:50.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:49 smithi139 ceph-mon[33599]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:50.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:49 smithi179 ceph-mon[34086]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T09:08:50.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:50 smithi064 ceph-mon[30149]: pgmap v361: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T09:08:51.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:50 smithi139 ceph-mon[33599]: pgmap v361: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T09:08:51.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:50 smithi179 ceph-mon[34086]: pgmap v361: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T09:08:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 -- 172.21.15.64:0/2609310500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c103580 msgr2=0x7f5f8c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/2609310500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c103580 0x7f5f8c1039e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f780099c0 tx=0x7f5f78009cd0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 -- 172.21.15.64:0/2609310500 shutdown_connections 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/2609310500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103f20 0x7f5f8c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/2609310500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f8c102380 0x7f5f8c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/2609310500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c103580 0x7f5f8c1039e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 -- 172.21.15.64:0/2609310500 >> 172.21.15.64:0/2609310500 conn(0x7f5f8c0fd920 msgr2=0x7f5f8c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 -- 172.21.15.64:0/2609310500 shutdown_connections 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 -- 172.21.15.64:0/2609310500 wait complete. 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 Processor -- start 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.208+0000 7f5f92e2c700 1 -- start start 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 0x7f5f8c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103580 0x7f5f8c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f8c103f20 0x7f5f8c19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10ddf0 con 0x7f5f8c102380 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10dc70 con 0x7f5f8c103580 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10df70 con 0x7f5f8c103f20 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 0x7f5f8c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 0x7f5f8c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35230/0 (socket says 172.21.15.64:35230) 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 -- 172.21.15.64:0/1032743654 learned_addr learned my addr 172.21.15.64:0/1032743654 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f913c9700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f8c103f20 0x7f5f8c19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f8bfff700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103580 0x7f5f8c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 -- 172.21.15.64:0/1032743654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f8c103f20 msgr2=0x7f5f8c19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f8c103f20 0x7f5f8c19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 -- 172.21.15.64:0/1032743654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103580 msgr2=0x7f5f8c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103580 0x7f5f8c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 -- 172.21.15.64:0/1032743654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c1a0060 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f8bfff700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103580 0x7f5f8c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f90bc8700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 0x7f5f8c198400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c9c0 tx=0x7f5f8000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80007ac0 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f80007c60 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c1a0350 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.209+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80018660 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.210+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c1a0890 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.210+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c069010 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.211+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f8000f450 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.212+0000 7f5f89ffb700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f7c0a2c20 0x7f5f7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.212+0000 7f5f8bfff700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f7c0a2c20 0x7f5f7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.212+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5f800cf370 con 0x7f5f8c102380 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.213+0000 7f5f8bfff700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f7c0a2c20 0x7f5f7c0a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800b580 tx=0x7f5f78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.217+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5f80014030 con 0x7f5f8c102380 2024-05-09T09:08:51.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.469+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5f8c062f30 con 0x7f5f8c102380 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.470+0000 7f5f89ffb700 1 -- 172.21.15.64:0/1032743654 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3056 (secure 0 0 0) 0x7f5f8009b990 con 0x7f5f8c102380 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:e24 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-05-09T09:08:51.469 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 24 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-05-09T09:06:40.458278+0000 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-05-09T09:07:08.230747+0000 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 300 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 3 2024-05-09T09:08:51.470 INFO:teuthology.orchestra.run.smithi064.stdout:in 0,1,2 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24503,1=24496,2=24487} 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3,4] 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:qdb_cluster leader: 24503 members: 24487,24503,24496 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b{0:24503} state up:active seq 3 addr [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.f{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.179:6834/502414567,v1:172.21.15.179:6835/502414567] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.i{1:24496} state up:active seq 11 addr [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.471 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.h{1:24464} state up:standby-replay seq 1 addr [v2:172.21.15.139:6836/2410946724,v1:172.21.15.139:6837/2410946724] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c{2:24487} state up:active seq 13 addr [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.e{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.139:6834/3760508086,v1:172.21.15.139:6835/3760508086] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.64:6834/201840839,v1:172.21.15.64:6835/201840839] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.64:6836/4065753229,v1:172.21.15.64:6837/4065753229] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.64:6838/437565752,v1:172.21.15.64:6839/437565752] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a{-1:14643} state up:standby seq 1 addr [v2:172.21.15.64:6840/4047794572,v1:172.21.15.64:6841/4047794572] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.139:6832/3830149624,v1:172.21.15.139:6833/3830149624] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.472 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.l{-1:24424} state up:standby seq 1 addr [v2:172.21.15.179:6832/3794352368,v1:172.21.15.179:6833/3794352368] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T09:08:51.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f7c0a2c20 msgr2=0x7f5f7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:51.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f7c0a2c20 0x7f5f7c0a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800b580 tx=0x7f5f78000bc0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 msgr2=0x7f5f8c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:51.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 0x7f5f8c198400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c9c0 tx=0x7f5f8000ccd0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 shutdown_connections 2024-05-09T09:08:51.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f7c0a2c20 0x7f5f7c0a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f8c103f20 0x7f5f8c19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f8c102380 0x7f5f8c198400 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 --2- 172.21.15.64:0/1032743654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f8c103580 0x7f5f8c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 >> 172.21.15.64:0/1032743654 conn(0x7f5f8c0fd920 msgr2=0x7f5f8c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 shutdown_connections 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:51.472+0000 7f5f92e2c700 1 -- 172.21.15.64:0/1032743654 wait complete. 2024-05-09T09:08:51.474 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:08:52.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:51 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1032743654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T09:08:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:51 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1032743654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T09:08:52.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T09:08:52.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:51 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1032743654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T09:08:52.503 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:08:52.880 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:52 smithi064 ceph-mon[30149]: pgmap v362: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:52 smithi139 ceph-mon[33599]: pgmap v362: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:52 smithi179 ceph-mon[34086]: pgmap v362: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 -- 172.21.15.64:0/1670067459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4104090 msgr2=0x7f07f410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/1670067459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4104090 0x7f07f410a940 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00b0d0 tx=0x7f07ec00b3e0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 -- 172.21.15.64:0/1670067459 shutdown_connections 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/1670067459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4104090 0x7f07f410a940 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/1670067459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4100670 0x7f07f4100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/1670067459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f41036f0 0x7f07f4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 -- 172.21.15.64:0/1670067459 >> 172.21.15.64:0/1670067459 conn(0x7f07f40fbf90 msgr2=0x7f07f40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 -- 172.21.15.64:0/1670067459 shutdown_connections 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.257+0000 7f07f8a7e700 1 -- 172.21.15.64:0/1670067459 wait complete. 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 Processor -- start 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 -- start start 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 0x7f07f4103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41036f0 0x7f07f41016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4104090 0x7f07f4101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410de10 con 0x7f07f4100670 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410dc90 con 0x7f07f41036f0 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f8a7e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410df90 con 0x7f07f4104090 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 0x7f07f4103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 0x7f07f4103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35258/0 (socket says 172.21.15.64:35258) 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 -- 172.21.15.64:0/4008623833 learned_addr learned my addr 172.21.15.64:0/4008623833 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f2d9d700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4104090 0x7f07f4101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f1d9b700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41036f0 0x7f07f41016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 -- 172.21.15.64:0/4008623833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4104090 msgr2=0x7f07f4101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4104090 0x7f07f4101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 -- 172.21.15.64:0/4008623833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41036f0 msgr2=0x7f07f41016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41036f0 0x7f07f41016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.258+0000 7f07f259c700 1 -- 172.21.15.64:0/4008623833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f4102490 con 0x7f07f4100670 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.259+0000 7f07f259c700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 0x7f07f4103050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c990 tx=0x7f07e400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.259+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4007940 con 0x7f07f4100670 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.259+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07e400f450 con 0x7f07f4100670 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.259+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4018660 con 0x7f07f4100670 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.259+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f41a24f0 con 0x7f07f4100670 2024-05-09T09:08:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.259+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f41a2a00 con 0x7f07f4100670 2024-05-09T09:08:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.260+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f4069010 con 0x7f07f4100670 2024-05-09T09:08:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.261+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f07e400f600 con 0x7f07f4100670 2024-05-09T09:08:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.262+0000 7f07e37fe700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2c20 0x7f07dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.262+0000 7f07f1d9b700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2c20 0x7f07dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.262+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f07e40cf4d0 con 0x7f07f4100670 2024-05-09T09:08:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.263+0000 7f07f1d9b700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2c20 0x7f07dc0a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f07e8005950 tx=0x7f07e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:54.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.266+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07e4014030 con 0x7f07f4100670 2024-05-09T09:08:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.471+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f07f4062f30 con 0x7f07f4100670 2024-05-09T09:08:54.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.472+0000 7f07e37fe700 1 -- 172.21.15.64:0/4008623833 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5652 (secure 0 0 0) 0x7f07e409c040 con 0x7f07f4100670 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 94 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-05-09T08:58:31.503287+0000 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-05-09T09:08:36.699384+0000 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:crush_version 27 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:full_ratio 0.95 2024-05-09T09:08:54.475 INFO:teuthology.orchestra.run.smithi064.stdout:backfillfull_ratio 0.9 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:nearfull_ratio 0.85 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:require_min_compat_client luminous 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:min_compat_client luminous 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:require_osd_release squid 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:stretch_mode_enabled false 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:max_osd 12 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6802/2595942736,v1:172.21.15.64:6803/2595942736] [v2:172.21.15.64:6804/2595942736,v1:172.21.15.64:6805/2595942736] exists,up f17d19a2-cc4a-48e7-a9ab-39baba537bd3 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6800/1721854761,v1:172.21.15.139:6801/1721854761] [v2:172.21.15.139:6802/1721854761,v1:172.21.15.139:6803/1721854761] exists,up fa7e401d-8122-43c1-9c90-c5ec8f78e1c6 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/3117011354,v1:172.21.15.179:6801/3117011354] [v2:172.21.15.179:6802/3117011354,v1:172.21.15.179:6803/3117011354] exists,up 2b8bc7a9-9231-4f53-ae83-151cc4898360 2024-05-09T09:08:54.476 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6810/3665517430,v1:172.21.15.64:6811/3665517430] [v2:172.21.15.64:6812/3665517430,v1:172.21.15.64:6813/3665517430] exists,up 3cd6b939-e38b-417b-86af-bf815725ee86 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 up in weight 1 up_from 34 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6808/2100308120,v1:172.21.15.139:6809/2100308120] [v2:172.21.15.139:6810/2100308120,v1:172.21.15.139:6811/2100308120] exists,up 01787529-c26f-4e8e-aca7-3254a587b036 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/2037999484,v1:172.21.15.179:6809/2037999484] [v2:172.21.15.179:6810/2037999484,v1:172.21.15.179:6811/2037999484] exists,up 6f02944f-d0d9-423c-8747-06be792aa16d 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 up in weight 1 up_from 47 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6818/784727785,v1:172.21.15.64:6819/784727785] [v2:172.21.15.64:6820/784727785,v1:172.21.15.64:6821/784727785] exists,up 4df6bd66-57d2-4efd-927e-75def737bed2 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6816/3275587574,v1:172.21.15.139:6817/3275587574] [v2:172.21.15.139:6818/3275587574,v1:172.21.15.139:6819/3275587574] exists,up 4e0802fb-8046-46e9-9a0a-923e400299d1 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 up in weight 1 up_from 60 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/365956094,v1:172.21.15.179:6817/365956094] [v2:172.21.15.179:6818/365956094,v1:172.21.15.179:6819/365956094] exists,up 35bee3f0-b62d-4743-a492-5b91e6d58126 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6826/2394827163,v1:172.21.15.64:6827/2394827163] [v2:172.21.15.64:6828/2394827163,v1:172.21.15.64:6829/2394827163] exists,up b9f9fbad-4a3a-4dd6-91c1-b856e1b53018 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 up in weight 1 up_from 73 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6824/4105758039,v1:172.21.15.139:6825/4105758039] [v2:172.21.15.139:6826/4105758039,v1:172.21.15.139:6827/4105758039] exists,up 38196d80-7f0b-4dd2-af5f-5c7341bcd2e8 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/880358389,v1:172.21.15.179:6825/880358389] [v2:172.21.15.179:6826/880358389,v1:172.21.15.179:6827/880358389] exists,up 5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.a [2,11] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.f [8,11] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.6 [4,6] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1a [5,7] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1e [3,9] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.23 [4,8] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.27 [3,9] 2024-05-09T09:08:54.477 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.34 [4,9] 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.6 [5,8] 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.12 [9,10] 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.13 [9,8] 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.16 [9,8] 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1152515804 expires 2024-05-10T08:59:35.011155+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/615130791 expires 2024-05-10T08:59:35.011155+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3043598500 expires 2024-05-10T08:59:35.011155+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/2011681658 expires 2024-05-10T08:59:08.628153+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/615130791 expires 2024-05-10T08:59:35.011155+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3326248924 expires 2024-05-10T08:59:08.628153+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1070396506 expires 2024-05-10T08:59:08.628153+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/2526964267 expires 2024-05-10T08:59:35.011155+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/421426620 expires 2024-05-10T08:59:08.628153+0000 2024-05-09T09:08:54.478 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/421426620 expires 2024-05-10T08:59:08.628153+0000 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2c20 msgr2=0x7f07dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2c20 0x7f07dc0a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f07e8005950 tx=0x7f07e800b410 comp rx=0 tx=0).stop 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 msgr2=0x7f07f4103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 0x7f07f4103050 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c990 tx=0x7f07e400cca0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 shutdown_connections 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f07dc0a2c20 0x7f07dc0a50e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07f4104090 0x7f07f4101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07f4100670 0x7f07f4103050 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 --2- 172.21.15.64:0/4008623833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41036f0 0x7f07f41016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 >> 172.21.15.64:0/4008623833 conn(0x7f07f40fbf90 msgr2=0x7f07f4106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:54.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 shutdown_connections 2024-05-09T09:08:54.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:54.474+0000 7f07f8a7e700 1 -- 172.21.15.64:0/4008623833 wait complete. 2024-05-09T09:08:55.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:54 smithi139 ceph-mon[33599]: pgmap v363: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:55.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:54 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4008623833' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T09:08:55.099 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:54 smithi064 ceph-mon[30149]: pgmap v363: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:55.099 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:54 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4008623833' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T09:08:55.129 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T09:08:55.140 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:08:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:54 smithi179 ceph-mon[34086]: pgmap v363: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:08:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:54 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4008623833' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T09:08:55.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 -- 172.21.15.64:0/618462470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 msgr2=0x7ff7c4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 --2- 172.21.15.64:0/618462470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c4108bb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009980 tx=0x7ff7ac02c620 comp rx=0 tx=0).stop 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 -- 172.21.15.64:0/618462470 shutdown_connections 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 --2- 172.21.15.64:0/618462470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c4108bb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 --2- 172.21.15.64:0/618462470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7c4101960 0x7ff7c4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 --2- 172.21.15.64:0/618462470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7c4100770 0x7ff7c4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 -- 172.21.15.64:0/618462470 >> 172.21.15.64:0/618462470 conn(0x7ff7c40fbce0 msgr2=0x7ff7c40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 -- 172.21.15.64:0/618462470 shutdown_connections 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 -- 172.21.15.64:0/618462470 wait complete. 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 Processor -- start 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 -- start start 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.252+0000 7ff7cb112640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7c4100770 0x7ff7c4199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7cb112640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7c4101960 0x7ff7c419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7cb112640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c41a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7cb112640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c410b1a0 con 0x7ff7c4102320 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7cb112640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c410b020 con 0x7ff7c4101960 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7cb112640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c410b320 con 0x7ff7c4100770 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c41a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c41a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32830/0 (socket says 172.21.15.64:32830) 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 -- 172.21.15.64:0/759850267 learned_addr learned my addr 172.21.15.64:0/759850267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c8eae640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7c4100770 0x7ff7c4199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c3fff640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7c4101960 0x7ff7c419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 -- 172.21.15.64:0/759850267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7c4100770 msgr2=0x7ff7c4199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7c4100770 0x7ff7c4199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 -- 172.21.15.64:0/759850267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7c4101960 msgr2=0x7ff7c419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7c4101960 0x7ff7c419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 -- 172.21.15.64:0/759850267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c41a1860 con 0x7ff7c4102320 2024-05-09T09:08:55.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c96af640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c41a11c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac005940 tx=0x7ff7ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:55.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac037050 con 0x7ff7c4102320 2024-05-09T09:08:55.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7ac0338f0 con 0x7ff7c4102320 2024-05-09T09:08:55.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c41a1af0 con 0x7ff7c4102320 2024-05-09T09:08:55.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.253+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac03cc60 con 0x7ff7c4102320 2024-05-09T09:08:55.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.254+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c41a1fe0 con 0x7ff7c4102320 2024-05-09T09:08:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.255+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c4107b10 con 0x7ff7c4102320 2024-05-09T09:08:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.256+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7ac003a00 con 0x7ff7c4102320 2024-05-09T09:08:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.256+0000 7ff7c1ffb640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7b00a2b90 0x7ff7b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.256+0000 7ff7c8eae640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7b00a2b90 0x7ff7b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.256+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff7ac0f00f0 con 0x7ff7c4102320 2024-05-09T09:08:55.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.257+0000 7ff7c8eae640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7b00a2b90 0x7ff7b00a5050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0058c0 tx=0x7ff7bc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:55.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.261+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7ac041020 con 0x7ff7c4102320 2024-05-09T09:08:55.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.448+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7c4062430 con 0x7ff7c4102320 2024-05-09T09:08:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.450+0000 7ff7c1ffb640 1 -- 172.21.15.64:0/759850267 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7ff7ac0bccc0 con 0x7ff7c4102320 2024-05-09T09:08:55.449 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:55.449 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7b00a2b90 msgr2=0x7ff7b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7b00a2b90 0x7ff7b00a5050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0058c0 tx=0x7ff7bc00a2d0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 msgr2=0x7ff7c41a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c41a11c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac005940 tx=0x7ff7ac0042f0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 shutdown_connections 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7b00a2b90 0x7ff7b00a5050 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7c4102320 0x7ff7c41a11c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7c4101960 0x7ff7c419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 --2- 172.21.15.64:0/759850267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7c4100770 0x7ff7c4199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 >> 172.21.15.64:0/759850267 conn(0x7ff7c40fbce0 msgr2=0x7ff7c4104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 shutdown_connections 2024-05-09T09:08:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.452+0000 7ff7cb112640 1 -- 172.21.15.64:0/759850267 wait complete. 2024-05-09T09:08:55.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:08:55.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.572+0000 7efd33cda640 1 -- 172.21.15.64:0/1883477078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c102350 msgr2=0x7efd2c104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.572+0000 7efd33cda640 1 --2- 172.21.15.64:0/1883477078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c102350 0x7efd2c104740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efd24009900 tx=0x7efd2402a410 comp rx=0 tx=0).stop 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- 172.21.15.64:0/1883477078 shutdown_connections 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 --2- 172.21.15.64:0/1883477078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd2c1075b0 0x7efd2c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 --2- 172.21.15.64:0/1883477078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c104c80 0x7efd2c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 --2- 172.21.15.64:0/1883477078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c102350 0x7efd2c104740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- 172.21.15.64:0/1883477078 >> 172.21.15.64:0/1883477078 conn(0x7efd2c0fbce0 msgr2=0x7efd2c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- 172.21.15.64:0/1883477078 shutdown_connections 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- 172.21.15.64:0/1883477078 wait complete. 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 Processor -- start 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- start start 2024-05-09T09:08:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c102350 0x7efd2c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd2c104c80 0x7efd2c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 0x7efd2c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c10d390 con 0x7efd2c1075b0 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c10d210 con 0x7efd2c104c80 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.573+0000 7efd33cda640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c10d510 con 0x7efd2c102350 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 0x7efd2c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 0x7efd2c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32842/0 (socket says 172.21.15.64:32842) 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 -- 172.21.15.64:0/2367243731 learned_addr learned my addr 172.21.15.64:0/2367243731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd31275640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd2c104c80 0x7efd2c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd31a76640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c102350 0x7efd2c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 -- 172.21.15.64:0/2367243731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c102350 msgr2=0x7efd2c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c102350 0x7efd2c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 -- 172.21.15.64:0/2367243731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd2c104c80 msgr2=0x7efd2c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd2c104c80 0x7efd2c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 -- 172.21.15.64:0/2367243731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c19f6e0 con 0x7efd2c1075b0 2024-05-09T09:08:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd32277640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 0x7efd2c19f070 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00cce0 tx=0x7efd1c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c00f450 con 0x7efd2c1075b0 2024-05-09T09:08:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd1c00faa0 con 0x7efd2c1075b0 2024-05-09T09:08:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c19f9d0 con 0x7efd2c1075b0 2024-05-09T09:08:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c018b70 con 0x7efd2c1075b0 2024-05-09T09:08:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.574+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c19ff20 con 0x7efd2c1075b0 2024-05-09T09:08:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.575+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c0fd7d0 con 0x7efd2c1075b0 2024-05-09T09:08:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.576+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd1c018d10 con 0x7efd2c1075b0 2024-05-09T09:08:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.577+0000 7efd2affd640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd140a2c10 0x7efd140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.577+0000 7efd31a76640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd140a2c10 0x7efd140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.577+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd1c0cf820 con 0x7efd2c1075b0 2024-05-09T09:08:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.578+0000 7efd31a76640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd140a2c10 0x7efd140a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efd24000c00 tx=0x7efd240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:55.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.581+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd1c014030 con 0x7efd2c1075b0 2024-05-09T09:08:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.764+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd2c02d1c0 con 0x7efd2c1075b0 2024-05-09T09:08:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.764+0000 7efd2affd640 1 -- 172.21.15.64:0/2367243731 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7efd1c09c500 con 0x7efd2c1075b0 2024-05-09T09:08:55.763 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:08:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd140a2c10 msgr2=0x7efd140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd140a2c10 0x7efd140a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efd24000c00 tx=0x7efd240314a0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 msgr2=0x7efd2c19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 0x7efd2c19f070 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00cce0 tx=0x7efd1c00a620 comp rx=0 tx=0).stop 2024-05-09T09:08:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 shutdown_connections 2024-05-09T09:08:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd140a2c10 0x7efd140a50d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd2c1075b0 0x7efd2c19f070 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd2c104c80 0x7efd2c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 --2- 172.21.15.64:0/2367243731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c102350 0x7efd2c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 >> 172.21.15.64:0/2367243731 conn(0x7efd2c0fbce0 msgr2=0x7efd2c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 shutdown_connections 2024-05-09T09:08:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.766+0000 7efd33cda640 1 -- 172.21.15.64:0/2367243731 wait complete. 2024-05-09T09:08:55.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:08:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 -- 172.21.15.64:0/508488422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 msgr2=0x7f9acc100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 --2- 172.21.15.64:0/508488422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc100bb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009ca0 tx=0x7f9ac002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 -- 172.21.15.64:0/508488422 shutdown_connections 2024-05-09T09:08:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 --2- 172.21.15.64:0/508488422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9acc102340 0x7f9acc108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 --2- 172.21.15.64:0/508488422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc101980 0x7f9acc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 --2- 172.21.15.64:0/508488422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc100bb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 -- 172.21.15.64:0/508488422 >> 172.21.15.64:0/508488422 conn(0x7f9acc0fbd00 msgr2=0x7f9acc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 -- 172.21.15.64:0/508488422 shutdown_connections 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 -- 172.21.15.64:0/508488422 wait complete. 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 Processor -- start 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.886+0000 7f9ad4734640 1 -- start start 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9acc101980 0x7f9acc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc102340 0x7f9acc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10c2b0 con 0x7f9acc100790 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10c130 con 0x7f9acc101980 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10c430 con 0x7f9acc102340 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32854/0 (socket says 172.21.15.64:32854) 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 -- 172.21.15.64:0/2155805240 learned_addr learned my addr 172.21.15.64:0/2155805240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad1ccf640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9acc101980 0x7f9acc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad2cd1640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc102340 0x7f9acc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 -- 172.21.15.64:0/2155805240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc102340 msgr2=0x7f9acc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc102340 0x7f9acc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 -- 172.21.15.64:0/2155805240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9acc101980 msgr2=0x7f9acc1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9acc101980 0x7f9acc1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 -- 172.21.15.64:0/2155805240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc19f780 con 0x7f9acc100790 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad2cd1640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc102340 0x7f9acc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad24d0640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc197b60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0003ba0 tx=0x7f9ac0003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0004040 con 0x7f9acc100790 2024-05-09T09:08:55.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc19fa10 con 0x7f9acc100790 2024-05-09T09:08:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ac0004690 con 0x7f9acc100790 2024-05-09T09:08:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.887+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0030cd0 con 0x7f9acc100790 2024-05-09T09:08:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.888+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc19ffb0 con 0x7f9acc100790 2024-05-09T09:08:55.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.889+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc1a01e0 con 0x7f9acc100790 2024-05-09T09:08:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.890+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ac00041e0 con 0x7f9acc100790 2024-05-09T09:08:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.890+0000 7f9abf7fe640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9ab80a2b90 0x7f9ab80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.891+0000 7f9ad1ccf640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9ab80a2b90 0x7f9ab80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.891+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9ac00f0950 con 0x7f9acc100790 2024-05-09T09:08:55.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.891+0000 7f9ad1ccf640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9ab80a2b90 0x7f9ab80a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9acc199070 tx=0x7f9ac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:55.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:55.895+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ac00bd520 con 0x7f9acc100790 2024-05-09T09:08:56.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:55 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/759850267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:56.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:55 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2367243731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:56.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.078+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9acc1a03b0 con 0x7f9acc100790 2024-05-09T09:08:56.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.078+0000 7f9abf7fe640 1 -- 172.21.15.64:0/2155805240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f9ac00bce20 con 0x7f9acc100790 2024-05-09T09:08:56.077 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9ab80a2b90 msgr2=0x7f9ab80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9ab80a2b90 0x7f9ab80a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9acc199070 tx=0x7f9ac8005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 msgr2=0x7f9acc197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc197b60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0003ba0 tx=0x7f9ac0003bd0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 shutdown_connections 2024-05-09T09:08:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9ab80a2b90 0x7f9ab80a5050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9acc102340 0x7f9acc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9acc101980 0x7f9acc1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 --2- 172.21.15.64:0/2155805240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc100790 0x7f9acc197b60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 >> 172.21.15.64:0/2155805240 conn(0x7f9acc0fbd00 msgr2=0x7f9acc069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:56.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 shutdown_connections 2024-05-09T09:08:56.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.080+0000 7f9ad4734640 1 -- 172.21.15.64:0/2155805240 wait complete. 2024-05-09T09:08:56.089 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/759850267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:56.089 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2367243731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:56.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.201+0000 7ff9052b3640 1 -- 172.21.15.64:0/2153900922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 msgr2=0x7ff900100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.201+0000 7ff9052b3640 1 --2- 172.21.15.64:0/2153900922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900100b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f002a410 comp rx=0 tx=0).stop 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- 172.21.15.64:0/2153900922 shutdown_connections 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 --2- 172.21.15.64:0/2153900922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff900102320 0x7ff900108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 --2- 172.21.15.64:0/2153900922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 0x7ff900101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 --2- 172.21.15.64:0/2153900922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900100b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- 172.21.15.64:0/2153900922 >> 172.21.15.64:0/2153900922 conn(0x7ff9000fbce0 msgr2=0x7ff9000fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- 172.21.15.64:0/2153900922 shutdown_connections 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- 172.21.15.64:0/2153900922 wait complete. 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 Processor -- start 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- start start 2024-05-09T09:08:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 0x7ff900198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff900102320 0x7ff90019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff90010c2c0 con 0x7ff900100770 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff90010c140 con 0x7ff900102320 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff9052b3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff90010c440 con 0x7ff900101960 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff8feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.202+0000 7ff8feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32872/0 (socket says 172.21.15.64:32872) 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 -- 172.21.15.64:0/3886750633 learned_addr learned my addr 172.21.15.64:0/3886750633 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8f7fff640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 0x7ff900198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8ff7fe640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff900102320 0x7ff90019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 -- 172.21.15.64:0/3886750633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 msgr2=0x7ff900198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 0x7ff900198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 -- 172.21.15.64:0/3886750633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff900102320 msgr2=0x7ff90019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff900102320 0x7ff90019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 -- 172.21.15.64:0/3886750633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90019f710 con 0x7ff900100770 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8f7fff640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 0x7ff900198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8feffd640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900197af0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00314e0 tx=0x7ff8f0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f003f070 con 0x7ff900100770 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8f0031e40 con 0x7ff900100770 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90019f9a0 con 0x7ff900100770 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0035030 con 0x7ff900100770 2024-05-09T09:08:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.203+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff90019fe90 con 0x7ff900100770 2024-05-09T09:08:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.204+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9001a0060 con 0x7ff900100770 2024-05-09T09:08:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.206+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff8f0042070 con 0x7ff900100770 2024-05-09T09:08:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.206+0000 7ff8fcff9640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8ec0a2b90 0x7ff8ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.206+0000 7ff8f7fff640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8ec0a2b90 0x7ff8ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.206+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff8f00f02d0 con 0x7ff900100770 2024-05-09T09:08:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.207+0000 7ff8f7fff640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8ec0a2b90 0x7ff8ec0a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005d00 tx=0x7ff8e8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.210+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8f00bcea0 con 0x7ff900100770 2024-05-09T09:08:56.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:55 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/759850267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:56.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:55 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2367243731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:56.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.394+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9001a03f0 con 0x7ff900100770 2024-05-09T09:08:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.394+0000 7ff8fcff9640 1 -- 172.21.15.64:0/3886750633 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ff8f00bc7a0 con 0x7ff900100770 2024-05-09T09:08:56.393 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:08:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8ec0a2b90 msgr2=0x7ff8ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8ec0a2b90 0x7ff8ec0a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005d00 tx=0x7ff8e8005c70 comp rx=0 tx=0).stop 2024-05-09T09:08:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 msgr2=0x7ff900197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900197af0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00314e0 tx=0x7ff8f0003c80 comp rx=0 tx=0).stop 2024-05-09T09:08:56.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 shutdown_connections 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8ec0a2b90 0x7ff8ec0a5050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff900102320 0x7ff90019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff900101960 0x7ff900198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 --2- 172.21.15.64:0/3886750633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff900100770 0x7ff900197af0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 >> 172.21.15.64:0/3886750633 conn(0x7ff9000fbce0 msgr2=0x7ff9000689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 shutdown_connections 2024-05-09T09:08:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.396+0000 7ff9052b3640 1 -- 172.21.15.64:0/3886750633 wait complete. 2024-05-09T09:08:56.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 -- 172.21.15.64:0/3304210094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 msgr2=0x7f4dbc104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 --2- 172.21.15.64:0/3304210094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc104700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802a410 comp rx=0 tx=0).stop 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 -- 172.21.15.64:0/3304210094 shutdown_connections 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 --2- 172.21.15.64:0/3304210094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dbc107570 0x7f4dbc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 --2- 172.21.15.64:0/3304210094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc104c40 0x7f4dbc107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 --2- 172.21.15.64:0/3304210094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc104700 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 -- 172.21.15.64:0/3304210094 >> 172.21.15.64:0/3304210094 conn(0x7f4dbc0fbd20 msgr2=0x7f4dbc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.515+0000 7f4dc4342640 1 -- 172.21.15.64:0/3304210094 shutdown_connections 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 -- 172.21.15.64:0/3304210094 wait complete. 2024-05-09T09:08:56.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 Processor -- start 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 -- start start 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc104c40 0x7f4dbc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dbc107570 0x7f4dbc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10d3c0 con 0x7f4dbc102310 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10d240 con 0x7f4dbc107570 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc4342640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10d540 con 0x7f4dbc104c40 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc20de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc20de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32884/0 (socket says 172.21.15.64:32884) 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.516+0000 7f4dc20de640 1 -- 172.21.15.64:0/1357442558 learned_addr learned my addr 172.21.15.64:0/1357442558 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc18dd640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc104c40 0x7f4dbc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc20de640 1 -- 172.21.15.64:0/1357442558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc104c40 msgr2=0x7f4dbc197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc20de640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc104c40 0x7f4dbc197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc20de640 1 -- 172.21.15.64:0/1357442558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dbc107570 msgr2=0x7f4dbc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc20de640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dbc107570 0x7f4dbc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc20de640 1 -- 172.21.15.64:0/1357442558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc19f650 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc20de640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc197a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005940 tx=0x7f4db8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db803f030 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4db8003cb0 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc19f8e0 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8035070 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.517+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc19fe20 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.518+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc0fd810 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.519+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4db8042030 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.520+0000 7f4db37fe640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4da80a2c10 0x7f4da80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.520+0000 7f4dc18dd640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4da80a2c10 0x7f4da80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.520+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4db80eff00 con 0x7f4dbc102310 2024-05-09T09:08:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.520+0000 7f4dc18dd640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4da80a2c10 0x7f4da80a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005e10 tx=0x7f4dac005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.524+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4db80bca70 con 0x7f4dbc102310 2024-05-09T09:08:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.708+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4dbc0624d0 con 0x7f4dbc102310 2024-05-09T09:08:56.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.708+0000 7f4db37fe640 1 -- 172.21.15.64:0/1357442558 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f4db80bc370 con 0x7f4dbc102310 2024-05-09T09:08:56.707 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4da80a2c10 msgr2=0x7f4da80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4da80a2c10 0x7f4da80a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005e10 tx=0x7f4dac005da0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 msgr2=0x7f4dbc197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc197a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005940 tx=0x7f4db8004430 comp rx=0 tx=0).stop 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 shutdown_connections 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4da80a2c10 0x7f4da80a50d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dbc107570 0x7f4dbc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc104c40 0x7f4dbc197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 --2- 172.21.15.64:0/1357442558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dbc102310 0x7f4dbc197a30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 >> 172.21.15.64:0/1357442558 conn(0x7f4dbc0fbd20 msgr2=0x7f4dbc102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 shutdown_connections 2024-05-09T09:08:56.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.710+0000 7f4dc4342640 1 -- 172.21.15.64:0/1357442558 wait complete. 2024-05-09T09:08:56.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:08:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 -- 172.21.15.64:0/1527712818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c1023b0 msgr2=0x7f732c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 --2- 172.21.15.64:0/1527712818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c1023b0 0x7f732c1047a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f731c009900 tx=0x7f731c02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 -- 172.21.15.64:0/1527712818 shutdown_connections 2024-05-09T09:08:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 --2- 172.21.15.64:0/1527712818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c107610 0x7f732c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 --2- 172.21.15.64:0/1527712818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f732c104ce0 0x7f732c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 --2- 172.21.15.64:0/1527712818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c1023b0 0x7f732c1047a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 -- 172.21.15.64:0/1527712818 >> 172.21.15.64:0/1527712818 conn(0x7f732c0fbd20 msgr2=0x7f732c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 -- 172.21.15.64:0/1527712818 shutdown_connections 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.827+0000 7f73318bd640 1 -- 172.21.15.64:0/1527712818 wait complete. 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 Processor -- start 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 -- start start 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f732c1023b0 0x7f732c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c104ce0 0x7f732c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 0x7f732c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10d430 con 0x7f732c107610 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10d2b0 con 0x7f732c104ce0 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f73318bd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10d5b0 con 0x7f732c1023b0 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f732b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 0x7f732c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f732b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 0x7f732c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32906/0 (socket says 172.21.15.64:32906) 2024-05-09T09:08:56.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f732b7fe640 1 -- 172.21.15.64:0/3339683921 learned_addr learned my addr 172.21.15.64:0/3339683921 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f732affd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f732c1023b0 0x7f732c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f7323fff640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c104ce0 0x7f732c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f732b7fe640 1 -- 172.21.15.64:0/3339683921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f732c1023b0 msgr2=0x7f732c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.828+0000 7f732b7fe640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f732c1023b0 0x7f732c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f732b7fe640 1 -- 172.21.15.64:0/3339683921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c104ce0 msgr2=0x7f732c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f732b7fe640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c104ce0 0x7f732c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f732b7fe640 1 -- 172.21.15.64:0/3339683921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c19f780 con 0x7f732c107610 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f732b7fe640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 0x7f732c19f110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f732400b970 tx=0x7f732400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324019070 con 0x7f732c107610 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c19fa70 con 0x7f732c107610 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7324014440 con 0x7f732c107610 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73240075d0 con 0x7f732c107610 2024-05-09T09:08:56.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.829+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c19fff0 con 0x7f732c107610 2024-05-09T09:08:56.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.830+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c0fd810 con 0x7f732c107610 2024-05-09T09:08:56.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.831+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f732401c070 con 0x7f732c107610 2024-05-09T09:08:56.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.832+0000 7f73308bb640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73180a2b90 0x7f73180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:56.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.832+0000 7f732affd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73180a2b90 0x7f73180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:56.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.832+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f73240cee40 con 0x7f732c107610 2024-05-09T09:08:56.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.832+0000 7f732affd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73180a2b90 0x7f73180a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f731c006010 tx=0x7f731c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:56.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:56.835+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f732409b3c0 con 0x7f732c107610 2024-05-09T09:08:57.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.060+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f732c02d1c0 con 0x7f732c107610 2024-05-09T09:08:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.061+0000 7f73308bb640 1 -- 172.21.15.64:0/3339683921 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f732409f6c0 con 0x7f732c107610 2024-05-09T09:08:57.060 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:57.060 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:07:08.230747+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:08:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:08:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73180a2b90 msgr2=0x7f73180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73180a2b90 0x7f73180a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f731c006010 tx=0x7f731c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 msgr2=0x7f732c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 0x7f732c19f110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f732400b970 tx=0x7f732400be30 comp rx=0 tx=0).stop 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 shutdown_connections 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73180a2b90 0x7f73180a5050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f732c107610 0x7f732c19f110 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c104ce0 0x7f732c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 --2- 172.21.15.64:0/3339683921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f732c1023b0 0x7f732c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 >> 172.21.15.64:0/3339683921 conn(0x7f732c0fbd20 msgr2=0x7f732c102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 shutdown_connections 2024-05-09T09:08:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.063+0000 7f73318bd640 1 -- 172.21.15.64:0/3339683921 wait complete. 2024-05-09T09:08:57.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:56 smithi064 ceph-mon[30149]: pgmap v364: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2155805240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3886750633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:56 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1357442558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:56 smithi139 ceph-mon[33599]: pgmap v364: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2155805240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3886750633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:56 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1357442558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:08:57.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 -- 172.21.15.64:0/61583078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278102340 msgr2=0x7ff278108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 --2- 172.21.15.64:0/61583078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278102340 0x7ff278108bd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff270009ca0 tx=0x7ff27002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 -- 172.21.15.64:0/61583078 shutdown_connections 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 --2- 172.21.15.64:0/61583078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278102340 0x7ff278108bd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 --2- 172.21.15.64:0/61583078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff278101980 0x7ff278101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 --2- 172.21.15.64:0/61583078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff278100790 0x7ff278100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 -- 172.21.15.64:0/61583078 >> 172.21.15.64:0/61583078 conn(0x7ff2780fbd20 msgr2=0x7ff2780fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 -- 172.21.15.64:0/61583078 shutdown_connections 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.182+0000 7ff27ce93640 1 -- 172.21.15.64:0/61583078 wait complete. 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 Processor -- start 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 -- start start 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 0x7ff27806aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff278101980 0x7ff27819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff278102340 0x7ff27819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810c2a0 con 0x7ff278100790 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810c120 con 0x7ff278102340 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27ce93640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810c420 con 0x7ff278101980 2024-05-09T09:08:57.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 0x7ff27806aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 0x7ff27806aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32924/0 (socket says 172.21.15.64:32924) 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff27659c640 1 -- 172.21.15.64:0/1585587132 learned_addr learned my addr 172.21.15.64:0/1585587132 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.183+0000 7ff276d9d640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff278102340 0x7ff27819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff275d9b640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff278101980 0x7ff27819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27659c640 1 -- 172.21.15.64:0/1585587132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff278101980 msgr2=0x7ff27819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27659c640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff278101980 0x7ff27819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27659c640 1 -- 172.21.15.64:0/1585587132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff278102340 msgr2=0x7ff27819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27659c640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff278102340 0x7ff27819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27659c640 1 -- 172.21.15.64:0/1585587132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27819f6c0 con 0x7ff278100790 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27659c640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 0x7ff27806aa00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff26800d770 tx=0x7ff26800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268017070 con 0x7ff278100790 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff268009c30 con 0x7ff278100790 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268003d00 con 0x7ff278100790 2024-05-09T09:08:57.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27819f950 con 0x7ff278100790 2024-05-09T09:08:57.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.184+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27819fea0 con 0x7ff278100790 2024-05-09T09:08:57.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.185+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff278107ba0 con 0x7ff278100790 2024-05-09T09:08:57.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.186+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff268003ea0 con 0x7ff278100790 2024-05-09T09:08:57.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.187+0000 7ff2677fe640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2600a2b90 0x7ff2600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.187+0000 7ff275d9b640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2600a2b90 0x7ff2600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.187+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2680cf1c0 con 0x7ff278100790 2024-05-09T09:08:57.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.188+0000 7ff275d9b640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2600a2b90 0x7ff2600a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005fd0 tx=0x7ff26c009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:57.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.191+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff26809bd90 con 0x7ff278100790 2024-05-09T09:08:57.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:56 smithi179 ceph-mon[34086]: pgmap v364: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:08:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2155805240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3886750633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:56 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1357442558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:57.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.376+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff27802d470 con 0x7ff278100790 2024-05-09T09:08:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.378+0000 7ff2677fe640 1 -- 172.21.15.64:0/1585587132 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7ff268009de0 con 0x7ff278100790 2024-05-09T09:08:57.377 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:57.377 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:08:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2600a2b90 msgr2=0x7ff2600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2600a2b90 0x7ff2600a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005fd0 tx=0x7ff26c009470 comp rx=0 tx=0).stop 2024-05-09T09:08:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 msgr2=0x7ff27806aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 0x7ff27806aa00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff26800d770 tx=0x7ff26800dc30 comp rx=0 tx=0).stop 2024-05-09T09:08:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 shutdown_connections 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2600a2b90 0x7ff2600a5050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff278102340 0x7ff27819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff278101980 0x7ff27819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 --2- 172.21.15.64:0/1585587132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff278100790 0x7ff27806aa00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 >> 172.21.15.64:0/1585587132 conn(0x7ff2780fbd20 msgr2=0x7ff278104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 shutdown_connections 2024-05-09T09:08:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.380+0000 7ff27ce93640 1 -- 172.21.15.64:0/1585587132 wait complete. 2024-05-09T09:08:57.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:08:57.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:08:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.500+0000 7f789b664640 1 -- 172.21.15.64:0/1960439158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 msgr2=0x7f7894101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.500+0000 7f789b664640 1 --2- 172.21.15.64:0/1960439158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f7894101880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7884009900 tx=0x7f788402a410 comp rx=0 tx=0).stop 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- 172.21.15.64:0/1960439158 shutdown_connections 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 --2- 172.21.15.64:0/1960439158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f7894101880 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 --2- 172.21.15.64:0/1960439158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7894100a60 0x7f7894100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 --2- 172.21.15.64:0/1960439158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7894100100 0x7f7894100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- 172.21.15.64:0/1960439158 >> 172.21.15.64:0/1960439158 conn(0x7f78940fbd20 msgr2=0x7f78940fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- 172.21.15.64:0/1960439158 shutdown_connections 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- 172.21.15.64:0/1960439158 wait complete. 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 Processor -- start 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- start start 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7894100100 0x7f7894199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7894100a60 0x7f789419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f78941a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f789410f600 con 0x7f7894101400 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f789410f480 con 0x7f7894100100 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.501+0000 7f789b664640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f789410f780 con 0x7f7894100a60 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f78941a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f78941a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32942/0 (socket says 172.21.15.64:32942) 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 -- 172.21.15.64:0/719648396 learned_addr learned my addr 172.21.15.64:0/719648396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899400640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7894100100 0x7f7894199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7898bff640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7894100a60 0x7f789419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 -- 172.21.15.64:0/719648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7894100a60 msgr2=0x7f789419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7894100a60 0x7f789419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 -- 172.21.15.64:0/719648396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7894100100 msgr2=0x7f7894199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7894100100 0x7f7894199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 -- 172.21.15.64:0/719648396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78941a18c0 con 0x7f7894101400 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f7899c01640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f78941a1220 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7884005ab0 tx=0x7f7884003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788403f070 con 0x7f7894101400 2024-05-09T09:08:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7884031c30 con 0x7f7894101400 2024-05-09T09:08:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884035030 con 0x7f7894101400 2024-05-09T09:08:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78941a1b50 con 0x7f7894101400 2024-05-09T09:08:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.502+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78941a2090 con 0x7f7894101400 2024-05-09T09:08:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.503+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789410ae50 con 0x7f7894101400 2024-05-09T09:08:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.505+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7884042070 con 0x7f7894101400 2024-05-09T09:08:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.505+0000 7f788a7fc640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78800a2c10 0x7f78800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.506+0000 7f7899400640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78800a2c10 0x7f78800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.506+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f78840efb50 con 0x7f7894101400 2024-05-09T09:08:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.506+0000 7f7899400640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78800a2c10 0x7f78800a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f789419ab50 tx=0x7f7890009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:57.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.510+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7884031de0 con 0x7f7894101400 2024-05-09T09:08:57.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.696+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78940624d0 con 0x7f7894101400 2024-05-09T09:08:57.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.696+0000 7f788a7fc640 1 -- 172.21.15.64:0/719648396 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f78840bc0d0 con 0x7f7894101400 2024-05-09T09:08:57.695 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:08:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78800a2c10 msgr2=0x7f78800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78800a2c10 0x7f78800a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f789419ab50 tx=0x7f7890009500 comp rx=0 tx=0).stop 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 msgr2=0x7f78941a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f78941a1220 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7884005ab0 tx=0x7f7884003b90 comp rx=0 tx=0).stop 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 shutdown_connections 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78800a2c10 0x7f78800a50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894101400 0x7f78941a1220 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7894100a60 0x7f789419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 --2- 172.21.15.64:0/719648396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7894100100 0x7f7894199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 >> 172.21.15.64:0/719648396 conn(0x7f78940fbd20 msgr2=0x7f78940fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 shutdown_connections 2024-05-09T09:08:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.698+0000 7f789b664640 1 -- 172.21.15.64:0/719648396 wait complete. 2024-05-09T09:08:57.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:08:57.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 -- 172.21.15.64:0/141950818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 msgr2=0x7faedc108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 --2- 172.21.15.64:0/141950818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc108bb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faed4009ca0 tx=0x7faed402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 -- 172.21.15.64:0/141950818 shutdown_connections 2024-05-09T09:08:57.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 --2- 172.21.15.64:0/141950818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc108bb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 --2- 172.21.15.64:0/141950818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc101960 0x7faedc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 --2- 172.21.15.64:0/141950818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc100770 0x7faedc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.818+0000 7faee0b1b640 1 -- 172.21.15.64:0/141950818 >> 172.21.15.64:0/141950818 conn(0x7faedc0fbce0 msgr2=0x7faedc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 -- 172.21.15.64:0/141950818 shutdown_connections 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 -- 172.21.15.64:0/141950818 wait complete. 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 Processor -- start 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 -- start start 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc100770 0x7faedc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc101960 0x7faedc19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faedc10c2f0 con 0x7faedc102320 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faedc10c170 con 0x7faedc101960 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.819+0000 7faee0b1b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faedc10c470 con 0x7faedc100770 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faedad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc100770 0x7faedc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faedad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32960/0 (socket says 172.21.15.64:32960) 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faedad9d640 1 -- 172.21.15.64:0/31536456 learned_addr learned my addr 172.21.15.64:0/31536456 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faed3fff640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc101960 0x7faedc19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 -- 172.21.15.64:0/31536456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc101960 msgr2=0x7faedc19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc101960 0x7faedc19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 -- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 msgr2=0x7faedc19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 -- 172.21.15.64:0/31536456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc19f740 con 0x7faedc100770 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faedad9d640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faed3fff640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc101960 0x7faedc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:08:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.820+0000 7faeda59c640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc100770 0x7faedc069530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faecc00e9c0 tx=0x7faecc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:57.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.824+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ccc0 con 0x7faedc100770 2024-05-09T09:08:57.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.824+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faecc0035f0 con 0x7faedc100770 2024-05-09T09:08:57.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.824+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc19f9d0 con 0x7faedc100770 2024-05-09T09:08:57.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.824+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ce60 con 0x7faedc100770 2024-05-09T09:08:57.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.824+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc19ff70 con 0x7faedc100770 2024-05-09T09:08:57.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.825+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc107b10 con 0x7faedc100770 2024-05-09T09:08:57.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.827+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faecc0037a0 con 0x7faedc100770 2024-05-09T09:08:57.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.827+0000 7faed37fe640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faec80a2c10 0x7faec80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:57.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.827+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faecc0cf750 con 0x7faedc100770 2024-05-09T09:08:57.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.827+0000 7faed3fff640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faec80a2c10 0x7faec80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:57.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.828+0000 7faed3fff640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faec80a2c10 0x7faec80a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faedc0692a0 tx=0x7faec4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:57.831+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faecc09c320 con 0x7faedc100770 2024-05-09T09:08:58.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.014+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faedc02d410 con 0x7faedc100770 2024-05-09T09:08:58.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.015+0000 7faed37fe640 1 -- 172.21.15.64:0/31536456 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7faecc09bc20 con 0x7faedc100770 2024-05-09T09:08:58.014 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faec80a2c10 msgr2=0x7faec80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faec80a2c10 0x7faec80a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faedc0692a0 tx=0x7faec4005f90 comp rx=0 tx=0).stop 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc100770 msgr2=0x7faedc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc100770 0x7faedc069530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faecc00e9c0 tx=0x7faecc00ee80 comp rx=0 tx=0).stop 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 shutdown_connections 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faec80a2c10 0x7faec80a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faedc102320 0x7faedc19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faedc101960 0x7faedc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 --2- 172.21.15.64:0/31536456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faedc100770 0x7faedc069530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 >> 172.21.15.64:0/31536456 conn(0x7faedc0fbce0 msgr2=0x7faedc104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:58.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 shutdown_connections 2024-05-09T09:08:58.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.017+0000 7faee0b1b640 1 -- 172.21.15.64:0/31536456 wait complete. 2024-05-09T09:08:58.026 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3339683921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:58.026 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1585587132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:58.027 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/719648396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:58.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:08:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3339683921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1585587132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/719648396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2336969218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4104c30 msgr2=0x7f3ea4107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2336969218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4104c30 0x7f3ea4107020 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009900 tx=0x7f3e8c02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2336969218 shutdown_connections 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2336969218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 0x7f3ea41099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2336969218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4104c30 0x7f3ea4107020 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2336969218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea4102300 0x7f3ea41046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2336969218 >> 172.21.15.64:0/2336969218 conn(0x7f3ea40fbcf0 msgr2=0x7f3ea40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2336969218 shutdown_connections 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2336969218 wait complete. 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.142+0000 7f3eaa44d640 1 Processor -- start 2024-05-09T09:08:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 -- start start 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 0x7f3ea4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea4104c30 0x7f3ea4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 0x7f3ea419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410d3a0 con 0x7f3ea4102300 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410d220 con 0x7f3ea4104c30 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3eaa44d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410d520 con 0x7f3ea4107560 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 0x7f3ea4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 0x7f3ea4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32982/0 (socket says 172.21.15.64:32982) 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2991730148 learned_addr learned my addr 172.21.15.64:0/2991730148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2991730148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 msgr2=0x7f3ea419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea89ea640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 0x7f3ea419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3e9bfff640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea4104c30 0x7f3ea4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 0x7f3ea419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2991730148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea4104c30 msgr2=0x7f3ea4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea4104c30 0x7f3ea4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 -- 172.21.15.64:0/2991730148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea419f730 con 0x7f3ea4102300 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea89ea640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 0x7f3ea419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea3fff640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 0x7f3ea4197b10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400d950 tx=0x7f3e9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.143+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94017070 con 0x7f3ea4102300 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.144+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e94009c30 con 0x7f3ea4102300 2024-05-09T09:08:58.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.144+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400f620 con 0x7f3ea4102300 2024-05-09T09:08:58.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.144+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea419fa20 con 0x7f3ea4102300 2024-05-09T09:08:58.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.144+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea419ffc0 con 0x7f3ea4102300 2024-05-09T09:08:58.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.145+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea40fd7e0 con 0x7f3ea4102300 2024-05-09T09:08:58.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.146+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e940109a0 con 0x7f3ea4102300 2024-05-09T09:08:58.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.147+0000 7f3ea1ffb640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e900a2c10 0x7f3e900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.147+0000 7f3e9bfff640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e900a2c10 0x7f3e900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.147+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e940cf050 con 0x7f3ea4102300 2024-05-09T09:08:58.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.147+0000 7f3e9bfff640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e900a2c10 0x7f3e900a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c02a930 tx=0x7f3e8c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:58.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.151+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e94009de0 con 0x7f3ea4102300 2024-05-09T09:08:58.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3339683921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1585587132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:58.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/719648396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:08:58.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.334+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ea4062460 con 0x7f3ea4102300 2024-05-09T09:08:58.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.335+0000 7f3ea1ffb640 1 -- 172.21.15.64:0/2991730148 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f3e9409bc20 con 0x7f3ea4102300 2024-05-09T09:08:58.334 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:08:58.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e900a2c10 msgr2=0x7f3e900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e900a2c10 0x7f3e900a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c02a930 tx=0x7f3e8c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 msgr2=0x7f3ea4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 0x7f3ea4197b10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400d950 tx=0x7f3e9400de10 comp rx=0 tx=0).stop 2024-05-09T09:08:58.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 shutdown_connections 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e900a2c10 0x7f3e900a50d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ea4107560 0x7f3ea419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea4104c30 0x7f3ea4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 --2- 172.21.15.64:0/2991730148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ea4102300 0x7f3ea4197b10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 >> 172.21.15.64:0/2991730148 conn(0x7f3ea40fbcf0 msgr2=0x7f3ea4102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 shutdown_connections 2024-05-09T09:08:58.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.337+0000 7f3eaa44d640 1 -- 172.21.15.64:0/2991730148 wait complete. 2024-05-09T09:08:58.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 -- 172.21.15.64:0/265016273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488102320 msgr2=0x7f4488108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 --2- 172.21.15.64:0/265016273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488102320 0x7f4488108bb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4480009ca0 tx=0x7f448002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 -- 172.21.15.64:0/265016273 shutdown_connections 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 --2- 172.21.15.64:0/265016273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488102320 0x7f4488108bb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 --2- 172.21.15.64:0/265016273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4488101960 0x7f4488101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 --2- 172.21.15.64:0/265016273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488100770 0x7f4488100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 -- 172.21.15.64:0/265016273 >> 172.21.15.64:0/265016273 conn(0x7f44880fbce0 msgr2=0x7f44880fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 -- 172.21.15.64:0/265016273 shutdown_connections 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.456+0000 7f448e377640 1 -- 172.21.15.64:0/265016273 wait complete. 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 Processor -- start 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 -- start start 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4488100770 0x7f4488197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 0x7f4488197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488102320 0x7f448819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f448810c280 con 0x7f4488101960 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f448810c100 con 0x7f4488100770 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f448e377640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f448810c400 con 0x7f4488102320 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f44877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 0x7f4488197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.457+0000 7f44877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 0x7f4488197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33008/0 (socket says 172.21.15.64:33008) 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 -- 172.21.15.64:0/661010249 learned_addr learned my addr 172.21.15.64:0/661010249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f448c914640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488102320 0x7f448819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f4487fff640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4488100770 0x7f4488197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 -- 172.21.15.64:0/661010249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488102320 msgr2=0x7f448819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488102320 0x7f448819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 -- 172.21.15.64:0/661010249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4488100770 msgr2=0x7f4488197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4488100770 0x7f4488197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 -- 172.21.15.64:0/661010249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448819f620 con 0x7f4488101960 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f448c914640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488102320 0x7f448819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44877fe640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 0x7f4488197f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f447c00d950 tx=0x7f447c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c017030 con 0x7f4488101960 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f447c004020 con 0x7f4488101960 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448819f910 con 0x7f4488101960 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c0047a0 con 0x7f4488101960 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.458+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448819fe60 con 0x7f4488101960 2024-05-09T09:08:58.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.459+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44881a0150 con 0x7f4488101960 2024-05-09T09:08:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.460+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f447c0041d0 con 0x7f4488101960 2024-05-09T09:08:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.461+0000 7f44857fa640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f44700a2b90 0x7f44700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.462+0000 7f4487fff640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f44700a2b90 0x7f44700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.462+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f447c0cef10 con 0x7f4488101960 2024-05-09T09:08:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.462+0000 7f4487fff640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f44700a2b90 0x7f44700a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4478005fd0 tx=0x7f4478005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:58.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.466+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f447c09bae0 con 0x7f4488101960 2024-05-09T09:08:58.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.648+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f44881a0320 con 0x7f4488101960 2024-05-09T09:08:58.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.649+0000 7f44857fa640 1 -- 172.21.15.64:0/661010249 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f447c09b3e0 con 0x7f4488101960 2024-05-09T09:08:58.648 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f44700a2b90 msgr2=0x7f44700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f44700a2b90 0x7f44700a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4478005fd0 tx=0x7f4478005ce0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 msgr2=0x7f4488197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 0x7f4488197f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f447c00d950 tx=0x7f447c00de10 comp rx=0 tx=0).stop 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 shutdown_connections 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f44700a2b90 0x7f44700a5050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4488102320 0x7f448819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488101960 0x7f4488197f90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 --2- 172.21.15.64:0/661010249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4488100770 0x7f4488197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 >> 172.21.15.64:0/661010249 conn(0x7f44880fbce0 msgr2=0x7f44880fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:58.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 shutdown_connections 2024-05-09T09:08:58.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.651+0000 7f448e377640 1 -- 172.21.15.64:0/661010249 wait complete. 2024-05-09T09:08:58.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:08:58.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 -- 172.21.15.64:0/677506151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 msgr2=0x7fee80101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 --2- 172.21.15.64:0/677506151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee80101e00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fee68009900 tx=0x7fee6802a410 comp rx=0 tx=0).stop 2024-05-09T09:08:58.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 -- 172.21.15.64:0/677506151 shutdown_connections 2024-05-09T09:08:58.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 --2- 172.21.15.64:0/677506151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee80102340 0x7fee80108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 --2- 172.21.15.64:0/677506151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee80101e00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 --2- 172.21.15.64:0/677506151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee80100790 0x7fee80100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.767+0000 7fee8542c640 1 -- 172.21.15.64:0/677506151 >> 172.21.15.64:0/677506151 conn(0x7fee800fbd20 msgr2=0x7fee800fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 -- 172.21.15.64:0/677506151 shutdown_connections 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 -- 172.21.15.64:0/677506151 wait complete. 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 Processor -- start 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 -- start start 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee80100790 0x7fee8006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee8019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee80102340 0x7fee8019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010c2c0 con 0x7fee80101980 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010c140 con 0x7fee80102340 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee8542c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010c440 con 0x7fee80100790 2024-05-09T09:08:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee8019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee8019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33032/0 (socket says 172.21.15.64:33032) 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee77fff640 1 -- 172.21.15.64:0/2156634487 learned_addr learned my addr 172.21.15.64:0/2156634487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.768+0000 7fee7effd640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee80100790 0x7fee8006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee7f7fe640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee80102340 0x7fee8019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee77fff640 1 -- 172.21.15.64:0/2156634487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee80100790 msgr2=0x7fee8006aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee77fff640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee80100790 0x7fee8006aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee77fff640 1 -- 172.21.15.64:0/2156634487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee80102340 msgr2=0x7fee8019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee77fff640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee80102340 0x7fee8019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee77fff640 1 -- 172.21.15.64:0/2156634487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8019f720 con 0x7fee80101980 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee77fff640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee8019c7f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee68005940 tx=0x7fee680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6803f030 con 0x7fee80101980 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee68003cb0 con 0x7fee80101980 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8019f950 con 0x7fee80101980 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68035070 con 0x7fee80101980 2024-05-09T09:08:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.769+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8019fe90 con 0x7fee80101980 2024-05-09T09:08:58.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.770+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee80107b30 con 0x7fee80101980 2024-05-09T09:08:58.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.772+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee68042030 con 0x7fee80101980 2024-05-09T09:08:58.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.773+0000 7fee7cff9640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fee6c0a2c10 0x7fee6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:58.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.773+0000 7fee7effd640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fee6c0a2c10 0x7fee6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.773+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fee680ef920 con 0x7fee80101980 2024-05-09T09:08:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.773+0000 7fee7effd640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fee6c0a2c10 0x7fee6c0a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee70005fd0 tx=0x7fee7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:58.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:58.777+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee680bc4f0 con 0x7fee80101980 2024-05-09T09:08:59.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.005+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee8002d1c0 con 0x7fee80101980 2024-05-09T09:08:59.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.005+0000 7fee7cff9640 1 -- 172.21.15.64:0/2156634487 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7fee680bbdf0 con 0x7fee80101980 2024-05-09T09:08:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:59.004 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:07:08.230747+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:08:59.005 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-05-09T09:08:59.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fee6c0a2c10 msgr2=0x7fee6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fee6c0a2c10 0x7fee6c0a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee70005fd0 tx=0x7fee7000b410 comp rx=0 tx=0).stop 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 msgr2=0x7fee8019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee8019c7f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee68005940 tx=0x7fee680042f0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 shutdown_connections 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fee6c0a2c10 0x7fee6c0a50d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee80102340 0x7fee8019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee80101980 0x7fee8019c7f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 --2- 172.21.15.64:0/2156634487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee80100790 0x7fee8006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.007+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 >> 172.21.15.64:0/2156634487 conn(0x7fee800fbd20 msgr2=0x7fee80104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.008+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 shutdown_connections 2024-05-09T09:08:59.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.008+0000 7fee8542c640 1 -- 172.21.15.64:0/2156634487 wait complete. 2024-05-09T09:08:59.017 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:58 smithi064 ceph-mon[30149]: pgmap v365: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:59.017 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:58 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/31536456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.017 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:58 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2991730148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.017 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:58 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/661010249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:08:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:58 smithi139 ceph-mon[33599]: pgmap v365: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:58 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/31536456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:58 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2991730148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:58 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/661010249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 -- 172.21.15.64:0/1962479785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954107570 msgr2=0x7f79541099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 --2- 172.21.15.64:0/1962479785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954107570 0x7f79541099c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f794c009900 tx=0x7f794c02a410 comp rx=0 tx=0).stop 2024-05-09T09:08:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 -- 172.21.15.64:0/1962479785 shutdown_connections 2024-05-09T09:08:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 --2- 172.21.15.64:0/1962479785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954107570 0x7f79541099c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 --2- 172.21.15.64:0/1962479785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7954104c40 0x7f7954107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 --2- 172.21.15.64:0/1962479785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7954102310 0x7f7954104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 -- 172.21.15.64:0/1962479785 >> 172.21.15.64:0/1962479785 conn(0x7f79540fbd20 msgr2=0x7f79540fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 -- 172.21.15.64:0/1962479785 shutdown_connections 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.123+0000 7f795c6bd640 1 -- 172.21.15.64:0/1962479785 wait complete. 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 Processor -- start 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 -- start start 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7954102310 0x7f7954197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 0x7f7954197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7954107570 0x7f795419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f795410d3c0 con 0x7f7954104c40 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f795410d240 con 0x7f7954107570 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795c6bd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f795410d540 con 0x7f7954102310 2024-05-09T09:08:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f7959c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 0x7f7954197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f7959c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 0x7f7954197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33058/0 (socket says 172.21.15.64:33058) 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f7959c58640 1 -- 172.21.15.64:0/4154319989 learned_addr learned my addr 172.21.15.64:0/4154319989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795a459640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7954102310 0x7f7954197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.124+0000 7f795ac5a640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7954107570 0x7f795419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f7959c58640 1 -- 172.21.15.64:0/4154319989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7954102310 msgr2=0x7f7954197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f7959c58640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7954102310 0x7f7954197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f7959c58640 1 -- 172.21.15.64:0/4154319989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7954107570 msgr2=0x7f795419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f7959c58640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7954107570 0x7f795419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f7959c58640 1 -- 172.21.15.64:0/4154319989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795419f680 con 0x7f7954104c40 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f7959c58640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 0x7f7954197fa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f794800d770 tx=0x7f794800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948017070 con 0x7f7954104c40 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7948009c30 con 0x7f7954104c40 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795419f970 con 0x7f7954104c40 2024-05-09T09:08:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794800f620 con 0x7f7954104c40 2024-05-09T09:08:59.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.125+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795419fec0 con 0x7f7954104c40 2024-05-09T09:08:59.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.126+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79540fd810 con 0x7f7954104c40 2024-05-09T09:08:59.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.127+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f794801a070 con 0x7f7954104c40 2024-05-09T09:08:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.128+0000 7f79537fe640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f793c0a2b90 0x7f793c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.128+0000 7f795a459640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f793c0a2b90 0x7f793c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.128+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f79480cef00 con 0x7f7954104c40 2024-05-09T09:08:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.128+0000 7f795a459640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f793c0a2b90 0x7f793c0a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7944005fd0 tx=0x7f7944009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.132+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7948009de0 con 0x7f7954104c40 2024-05-09T09:08:59.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:58 smithi179 ceph-mon[34086]: pgmap v365: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:08:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:58 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/31536456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:58 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2991730148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:58 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/661010249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:08:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.317+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7954062430 con 0x7f7954104c40 2024-05-09T09:08:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.319+0000 7f79537fe640 1 -- 172.21.15.64:0/4154319989 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f794809bad0 con 0x7f7954104c40 2024-05-09T09:08:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:08:59.318 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:08:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.320+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f793c0a2b90 msgr2=0x7f793c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.320+0000 7f795c6bd640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f793c0a2b90 0x7f793c0a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7944005fd0 tx=0x7f7944009680 comp rx=0 tx=0).stop 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.320+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 msgr2=0x7f7954197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 0x7f7954197fa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f794800d770 tx=0x7f794800dc30 comp rx=0 tx=0).stop 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 shutdown_connections 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f793c0a2b90 0x7f793c0a5050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7954107570 0x7f795419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7954104c40 0x7f7954197fa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 --2- 172.21.15.64:0/4154319989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7954102310 0x7f7954197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 >> 172.21.15.64:0/4154319989 conn(0x7f79540fbd20 msgr2=0x7f7954102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 shutdown_connections 2024-05-09T09:08:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.321+0000 7f795c6bd640 1 -- 172.21.15.64:0/4154319989 wait complete. 2024-05-09T09:08:59.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T09:06:40.458278+0000', 'modified': '2024-05-09T09:07:08.230747+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24503, 'mds_1': 24496, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6835/3760508086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3760508086}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 3760508086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/2410946724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 2410946724}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2410946724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/502414567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 502414567}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 502414567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.179:6837/2858892786', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2858892786}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2858892786}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/1577800782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 1577800782}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 1577800782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6839/1823707244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 1823707244}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 1823707244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24487, 24503, 24496]} 2024-05-09T09:08:59.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T09:08:59.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T09:08:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 -- 172.21.15.64:0/2153279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 msgr2=0x7fedfc101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- 172.21.15.64:0/2153279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc101860 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ca0 tx=0x7fedf402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 -- 172.21.15.64:0/2153279280 shutdown_connections 2024-05-09T09:08:59.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- 172.21.15.64:0/2153279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc101860 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- 172.21.15.64:0/2153279280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc100a40 0x7fedfc100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- 172.21.15.64:0/2153279280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedfc1000e0 0x7fedfc100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 -- 172.21.15.64:0/2153279280 >> 172.21.15.64:0/2153279280 conn(0x7fedfc0fbce0 msgr2=0x7fedfc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 -- 172.21.15.64:0/2153279280 shutdown_connections 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 -- 172.21.15.64:0/2153279280 wait complete. 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 Processor -- start 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 -- start start 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc1000e0 0x7fedfc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedfc100a40 0x7fedfc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.439+0000 7fee032bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee032bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc10f570 con 0x7fedfc1013e0 2024-05-09T09:08:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee032bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc10f3f0 con 0x7fedfc100a40 2024-05-09T09:08:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee032bd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc10f6f0 con 0x7fedfc1000e0 2024-05-09T09:08:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc1a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33088/0 (socket says 172.21.15.64:33088) 2024-05-09T09:08:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 -- 172.21.15.64:0/3975103722 learned_addr learned my addr 172.21.15.64:0/3975103722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee00858640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedfc100a40 0x7fedfc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee01059640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc1000e0 0x7fedfc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 -- 172.21.15.64:0/3975103722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc1000e0 msgr2=0x7fedfc199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc1000e0 0x7fedfc199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 -- 172.21.15.64:0/3975103722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedfc100a40 msgr2=0x7fedfc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedfc100a40 0x7fedfc19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 -- 172.21.15.64:0/3975103722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc1a1800 con 0x7fedfc1013e0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee0185a640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc1a1190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009dc0 tx=0x7fedf4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fedf27fc640 1 -- 172.21.15.64:0/3975103722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4032ec0 con 0x7fedfc1013e0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc1a1a90 con 0x7fedfc1013e0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fedf27fc640 1 -- 172.21.15.64:0/3975103722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf40044f0 con 0x7fedfc1013e0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fedf27fc640 1 -- 172.21.15.64:0/3975103722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4030d40 con 0x7fedfc1013e0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.440+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1a2030 con 0x7fedfc1013e0 2024-05-09T09:08:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fedf27fc640 1 -- 172.21.15.64:0/3975103722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fedf4004040 con 0x7fedfc1013e0 2024-05-09T09:08:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 Processor -- start 2024-05-09T09:08:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 -- start start 2024-05-09T09:08:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 0x565220106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x56522010bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56522010c420 0x56522010e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56522006a4d0 con 0x5652201068c0 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5652200123e0 con 0x5652201072b0 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee05fd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x565220066490 con 0x56522010c420 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee00858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 0x565220106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.443+0000 7fee00858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 0x565220106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33094/0 (socket says 172.21.15.64:33094) 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 -- 172.21.15.64:0/2691399101 learned_addr learned my addr 172.21.15.64:0/2691399101 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee01059640 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x56522010bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fedf27fc640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fede80a2d30 0x7fede80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 -- 172.21.15.64:0/2691399101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56522010c420 msgr2=0x56522010e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56522010c420 0x56522010e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 -- 172.21.15.64:0/2691399101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 msgr2=0x56522010bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x56522010bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 -- 172.21.15.64:0/2691399101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521fd7e2d0 con 0x5652201068c0 2024-05-09T09:08:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee01059640 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x56522010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee0185a640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fede80a2d30 0x7fede80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fedf27fc640 1 -- 172.21.15.64:0/3975103722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fedf40f0b10 con 0x7fedfc1013e0 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fee00858640 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 0x565220106ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedec00bd40 tx=0x7fedec0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f68ceb6b640a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fede77fe640 1 -- 172.21.15.64:0/2691399101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec01b470 con 0x5652201068c0 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fede77fe640 1 -- 172.21.15.64:0/2691399101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec016070 con 0x5652201068c0 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.444+0000 7fede77fe640 1 -- 172.21.15.64:0/2691399101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec010740 con 0x5652201068c0 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 -- 172.21.15.64:0/2691399101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 msgr2=0x565220106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee0185a640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fede80a2d30 0x7fede80a51f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedf40900d0 tx=0x7fedf4090100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 0x565220106ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedec00bd40 tx=0x7fedec0127c0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 -- 172.21.15.64:0/2691399101 shutdown_connections 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56522010c420 0x56522010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x56522010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 --2- 172.21.15.64:0/2691399101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201068c0 0x565220106ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 -- 172.21.15.64:0/2691399101 >> 172.21.15.64:0/2691399101 conn(0x565220101f30 msgr2=0x565220102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 -- 172.21.15.64:0/2691399101 shutdown_connections 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 -- 172.21.15.64:0/2691399101 wait complete. 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 -- start start 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.445+0000 7fee05fd3740 1 Processor -- start 2024-05-09T09:08:59.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x565220110d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 0x565220111280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56522010c420 0x565220118280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56522010f3d0 con 0x5652201072b0 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5652200258e0 con 0x56522010c420 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56522010f550 con 0x5652201068c0 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee05fd3740 10 client.0 _ll_register_callbacks cb 0x56521fff29c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 0x565220111280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 0x565220111280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33106/0 (socket says 172.21.15.64:33106) 2024-05-09T09:08:59.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 -- 172.21.15.64:0/3290906898 learned_addr learned my addr 172.21.15.64:0/3290906898 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee00858640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56522010c420 0x565220118280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee00858640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x565220110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 -- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 msgr2=0x565220110d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x565220110d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 -- 172.21.15.64:0/3290906898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56522010c420 msgr2=0x565220118280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56522010c420 0x565220118280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee01059640 1 -- 172.21.15.64:0/3290906898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521fe486b0 con 0x5652201072b0 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.446+0000 7fee00858640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56522010c420 0x565220118280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee01059640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 0x565220111280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fedf8012800 tx=0x7fedf8012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc661aff359255f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fede4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fede4ff9640 1 -- 172.21.15.64:0/3290906898 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8010dc0 con 0x5652201072b0 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fede4ff9640 1 -- 172.21.15.64:0/3290906898 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fedf80047a0 con 0x5652201072b0 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedec016070 con 0x5652201072b0 2024-05-09T09:08:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fede4ff9640 1 -- 172.21.15.64:0/3290906898 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fedf8025050 con 0x5652201072b0 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56522011b350 con 0x5652201072b0 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fede4ff9640 1 -- 172.21.15.64:0/3290906898 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec016070 con 0x5652201072b0 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fede4ff9640 1 -- 172.21.15.64:0/3290906898 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fedf800e5e0 con 0x5652201072b0 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 10 client.15402 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 20 client.15402 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56522011da30 0x56522011eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 4 client.15402 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.447+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565220077c00 con 0x56522011da30 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.448+0000 7fee00858640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56522011da30 0x56522011eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.448+0000 7fede4ff9640 10 client.15402 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.448+0000 7fee00858640 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56522011da30 0x56522011eea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6466a4510bc2a1a5 server_cookie=9617dc4a18a1f0e4 in_seq=0 out_seq=0 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fede4ff9640 1 -- 172.21.15.64:0/3290906898 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565220077c00 con 0x56522011da30 2024-05-09T09:08:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fede4ff9640 10 client.15402 handle_command_reply: tid=0 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 client.15402 shutdown 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 msgr2=0x565220111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 0x565220111280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fedf8012800 tx=0x7fedf8012cc0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 shutdown_connections 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56522011da30 0x56522011eea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56522010c420 0x565220118280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652201072b0 0x565220111280 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 --2- 172.21.15.64:0/3290906898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x565220110d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fede4ff9640 0 client.15402 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 >> 172.21.15.64:0/3290906898 conn(0x565220101f30 msgr2=0x565220101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 shutdown_connections 2024-05-09T09:08:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.449+0000 7fee05fd3740 1 -- 172.21.15.64:0/3290906898 wait complete. 2024-05-09T09:08:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.450+0000 7fee05fd3740 20 client.15402 trim_cache size 0 max 16384 2024-05-09T09:08:59.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 Processor -- start 2024-05-09T09:08:59.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 -- start start 2024-05-09T09:08:59.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201068c0 0x56522010e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201072b0 0x565220118240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x5652200fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56522011d1e0 con 0x56522010c420 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56522011d8b0 con 0x5652201068c0 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee05fd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56522011df80 con 0x5652201072b0 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x5652200fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x5652200fcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33110/0 (socket says 172.21.15.64:33110) 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 -- 172.21.15.64:0/4197742263 learned_addr learned my addr 172.21.15.64:0/4197742263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 -- 172.21.15.64:0/4197742263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201072b0 msgr2=0x565220118240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201072b0 0x565220118240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 -- 172.21.15.64:0/4197742263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201068c0 msgr2=0x56522010e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201068c0 0x56522010e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 -- 172.21.15.64:0/4197742263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56522011dd60 con 0x56522010c420 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fee00858640 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x5652200fcd10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c010 tx=0x7fedec01cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea10509281f60413 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.468+0000 7fede6ffd640 1 -- 172.21.15.64:0/4197742263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec000e60 con 0x56522010c420 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fede6ffd640 1 -- 172.21.15.64:0/4197742263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec016070 con 0x56522010c420 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fede6ffd640 1 -- 172.21.15.64:0/4197742263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec004d40 con 0x56522010c420 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 -- 172.21.15.64:0/4197742263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 msgr2=0x5652200fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x5652200fcd10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c010 tx=0x7fedec01cfc0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 -- 172.21.15.64:0/4197742263 shutdown_connections 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x5652200fcd10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201072b0 0x565220118240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 --2- 172.21.15.64:0/4197742263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201068c0 0x56522010e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 -- 172.21.15.64:0/4197742263 >> 172.21.15.64:0/4197742263 conn(0x565220101f30 msgr2=0x565220102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 -- 172.21.15.64:0/4197742263 shutdown_connections 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 -- 172.21.15.64:0/4197742263 wait complete. 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 -- start start 2024-05-09T09:08:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.469+0000 7fee05fd3740 1 Processor -- start 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x5652201139d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x565220113f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x565220116ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56522011ed20 con 0x56522010c420 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56522011e650 con 0x5652201072b0 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56522011f3f0 con 0x5652201068c0 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee05fd3740 10 client.0 _ll_register_callbacks cb 0x7fedf800a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x565220116ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x565220116ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33116/0 (socket says 172.21.15.64:33116) 2024-05-09T09:08:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 -- 172.21.15.64:0/1700255318 learned_addr learned my addr 172.21.15.64:0/1700255318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee01059640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x565220113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x5652201139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 -- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 msgr2=0x5652201139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x5652201139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 -- 172.21.15.64:0/1700255318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 msgr2=0x565220113f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x565220113f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 -- 172.21.15.64:0/1700255318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652201174b0 con 0x56522010c420 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee01059640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x565220113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fee00858640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x565220116ec0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fedec012cc0 tx=0x7fedec00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78308133902476ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.470+0000 7fede4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fede4ff9640 1 -- 172.21.15.64:0/1700255318 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec0121d0 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fede4ff9640 1 -- 172.21.15.64:0/1700255318 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fedec004cd0 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedec016070 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fede4ff9640 1 -- 172.21.15.64:0/1700255318 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fedec040050 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 10 client.15408 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565220183a40 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fede4ff9640 1 -- 172.21.15.64:0/1700255318 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec016070 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fede4ff9640 1 -- 172.21.15.64:0/1700255318 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fedec02a660 con 0x56522010c420 2024-05-09T09:08:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 20 client.15408 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x565220187df0 0x565220186b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 4 client.15408 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.471+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "status"}) -- 0x565220071f50 con 0x565220187df0 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee01059640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x565220187df0 0x565220186b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fede4ff9640 10 client.15408 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee01059640 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x565220187df0 0x565220186b00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82d11522ce431533 server_cookie=3f0d95b62d8a6e3b in_seq=0 out_seq=0 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fede4ff9640 1 -- 172.21.15.64:0/1700255318 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x565220071f50 con 0x565220187df0 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fede4ff9640 10 client.15408 handle_command_reply: tid=0 2024-05-09T09:08:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 client.15408 shutdown 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 msgr2=0x565220116ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x565220116ec0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fedec012cc0 tx=0x7fedec00b510 comp rx=0 tx=0).stop 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 shutdown_connections 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x565220187df0 0x565220186b00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56522010c420 0x565220116ec0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652201072b0 0x565220113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fede4ff9640 0 client.15408 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.472+0000 7fee05fd3740 1 --2- 172.21.15.64:0/1700255318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652201068c0 0x5652201139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 >> 172.21.15.64:0/1700255318 conn(0x565220101f30 msgr2=0x565220107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 shutdown_connections 2024-05-09T09:08:59.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee05fd3740 1 -- 172.21.15.64:0/1700255318 wait complete. 2024-05-09T09:08:59.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee05fd3740 20 client.15408 trim_cache size 0 max 16384 2024-05-09T09:08:59.487 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:08:59.487 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:08:59.487 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-05-09T09:08:59.487 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24487, 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 112.24595789, 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 94, 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 158.92701618300001 2024-05-09T09:08:59.488 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fede80a2d30 msgr2=0x7fede80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fede80a2d30 0x7fede80a51f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedf40900d0 tx=0x7fedf4090100 comp rx=0 tx=0).stop 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 msgr2=0x7fedfc1a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc1a1190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009dc0 tx=0x7fedf4003ba0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 shutdown_connections 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fede80a2d30 0x7fede80a51f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedfc1013e0 0x7fedfc1a1190 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedfc100a40 0x7fedfc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 --2- 172.21.15.64:0/3975103722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc1000e0 0x7fedfc199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 >> 172.21.15.64:0/3975103722 conn(0x7fedfc0fbce0 msgr2=0x7fedfc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 shutdown_connections 2024-05-09T09:08:59.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.473+0000 7fee032bd640 1 -- 172.21.15.64:0/3975103722 wait complete. 2024-05-09T09:08:59.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-09T09:08:59.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 -- 172.21.15.64:0/1522912479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 msgr2=0x7f6668108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1522912479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f6668108bd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6660009ca0 tx=0x7f666002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 -- 172.21.15.64:0/1522912479 shutdown_connections 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1522912479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f6668108bd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1522912479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 0x7f6668101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1522912479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6668100790 0x7f6668100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 -- 172.21.15.64:0/1522912479 >> 172.21.15.64:0/1522912479 conn(0x7f66680fbd20 msgr2=0x7f66680fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 -- 172.21.15.64:0/1522912479 shutdown_connections 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 -- 172.21.15.64:0/1522912479 wait complete. 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.598+0000 7f666ecb3640 1 Processor -- start 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 -- start start 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6668100790 0x7f66681979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 0x7f6668197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f666819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f666810c2a0 con 0x7f6668102340 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f666810c120 con 0x7f6668101980 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ecb3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f666810c420 con 0x7f6668100790 2024-05-09T09:08:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f666819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f666819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33134/0 (socket says 172.21.15.64:33134) 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 -- 172.21.15.64:0/1222313067 learned_addr learned my addr 172.21.15.64:0/1222313067 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f6667fff640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 0x7f6668197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666ca4f640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6668100790 0x7f66681979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 -- 172.21.15.64:0/1222313067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6668100790 msgr2=0x7f66681979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6668100790 0x7f66681979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 -- 172.21.15.64:0/1222313067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 msgr2=0x7f6668197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 0x7f6668197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 -- 172.21.15.64:0/1222313067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666819f5f0 con 0x7f6668102340 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f6667fff640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 0x7f6668197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f666d250640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f666819ef80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6660009dc0 tx=0x7f6660003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.599+0000 7f6665ffb640 1 -- 172.21.15.64:0/1222313067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660032ec0 con 0x7f6668102340 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.600+0000 7f6665ffb640 1 -- 172.21.15.64:0/1222313067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66600044f0 con 0x7f6668102340 2024-05-09T09:08:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.600+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666819f880 con 0x7f6668102340 2024-05-09T09:08:59.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.600+0000 7f6665ffb640 1 -- 172.21.15.64:0/1222313067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660030d40 con 0x7f6668102340 2024-05-09T09:08:59.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.600+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666819fdd0 con 0x7f6668102340 2024-05-09T09:08:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.602+0000 7f6665ffb640 1 -- 172.21.15.64:0/1222313067 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6660004040 con 0x7f6668102340 2024-05-09T09:08:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.602+0000 7f66719c1740 1 Processor -- start 2024-05-09T09:08:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.602+0000 7f66719c1740 1 -- start start 2024-05-09T09:08:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.602+0000 7f66719c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf907ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.602+0000 7f66719c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf90cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f66719c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf90f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f66719c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf86b4d0 con 0x55dbaf90d420 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f66719c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf8133e0 con 0x55dbaf9078c0 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f66719c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf867490 con 0x55dbaf9082b0 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6665ffb640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66580a2d30 0x7f66580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf90f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf90f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33140/0 (socket says 172.21.15.64:33140) 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 -- 172.21.15.64:0/38546050 learned_addr learned my addr 172.21.15.64:0/38546050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f666ca4f640 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf90cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf907ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f666d250640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66580a2d30 0x7f66580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6665ffb640 1 -- 172.21.15.64:0/1222313067 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66600f0a90 con 0x7f6668102340 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 -- 172.21.15.64:0/38546050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 msgr2=0x55dbaf90cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf90cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 -- 172.21.15.64:0/38546050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 msgr2=0x55dbaf907ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf907ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 -- 172.21.15.64:0/38546050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbaf57f2d0 con 0x55dbaf90d420 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6667fff640 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf90f8b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6654005bc0 tx=0x7f665400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1b675939d220b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f666d250640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66580a2d30 0x7f66580a51f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66600bcef0 tx=0x7f66600b2440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6652ffd640 1 -- 172.21.15.64:0/38546050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654004230 con 0x55dbaf90d420 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6652ffd640 1 -- 172.21.15.64:0/38546050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66540043d0 con 0x55dbaf90d420 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.603+0000 7f6652ffd640 1 -- 172.21.15.64:0/38546050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654004c00 con 0x55dbaf90d420 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 -- 172.21.15.64:0/38546050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 msgr2=0x55dbaf90f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf90f8b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6654005bc0 tx=0x7f665400b720 comp rx=0 tx=0).stop 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 -- 172.21.15.64:0/38546050 shutdown_connections 2024-05-09T09:08:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf90f8b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf90cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 --2- 172.21.15.64:0/38546050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf907ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 -- 172.21.15.64:0/38546050 >> 172.21.15.64:0/38546050 conn(0x55dbaf902f30 msgr2=0x55dbaf903330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 -- 172.21.15.64:0/38546050 shutdown_connections 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 -- 172.21.15.64:0/38546050 wait complete. 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.604+0000 7f66719c1740 1 -- start start 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 Processor -- start 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf911d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 0x55dbaf9122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf9192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf9103d0 con 0x55dbaf9082b0 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf8268e0 con 0x55dbaf9078c0 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf910550 con 0x55dbaf90d420 2024-05-09T09:08:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f66719c1740 10 client.0 _ll_register_callbacks cb 0x55dbaf7f39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 0x55dbaf9122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 0x55dbaf9122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33156/0 (socket says 172.21.15.64:33156) 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 -- 172.21.15.64:0/3815533435 learned_addr learned my addr 172.21.15.64:0/3815533435 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f6667fff640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf9192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f6667fff640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf911d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 -- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 msgr2=0x55dbaf9192a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf9192a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 -- 172.21.15.64:0/3815533435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 msgr2=0x55dbaf911d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf911d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f666ca4f640 1 -- 172.21.15.64:0/3815533435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbaf6496b0 con 0x55dbaf9082b0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.605+0000 7f6667fff640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf911d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f666ca4f640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 0x55dbaf9122a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f665c011790 tx=0x7f665c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9f76c75de43419 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f6647fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f6647fff640 1 -- 172.21.15.64:0/3815533435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c00fe70 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f6647fff640 1 -- 172.21.15.64:0/3815533435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f665c005020 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66540043d0 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f6647fff640 1 -- 172.21.15.64:0/3815533435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f665c024050 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dbaf91c370 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f6647fff640 1 -- 172.21.15.64:0/3815533435 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66540043d0 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f6647fff640 1 -- 172.21.15.64:0/3815533435 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f665c018030 con 0x55dbaf9082b0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 10 client.15420 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 20 client.15420 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf91ea60 0x55dbaf91fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 4 client.15420 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.606+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbaf878c00 con 0x55dbaf91ea60 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.607+0000 7f6667fff640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf91ea60 0x55dbaf91fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.607+0000 7f6667fff640 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf91ea60 0x55dbaf91fed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba3a5f4cb3a6bf79 server_cookie=b0b77d6d7a3c0c9c in_seq=0 out_seq=0 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.607+0000 7f6647fff640 10 client.15420 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f6647fff640 1 -- 172.21.15.64:0/3815533435 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbaf878c00 con 0x55dbaf91ea60 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f6647fff640 10 client.15420 handle_command_reply: tid=0 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 client.15420 shutdown 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 msgr2=0x55dbaf9122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 0x55dbaf9122a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f665c011790 tx=0x7f665c011c50 comp rx=0 tx=0).stop 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 shutdown_connections 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf9192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf91ea60 0x55dbaf91fed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9082b0 0x55dbaf9122a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 --2- 172.21.15.64:0/3815533435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf911d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f6647fff640 0 client.15420 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 >> 172.21.15.64:0/3815533435 conn(0x55dbaf902f30 msgr2=0x55dbaf902000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 shutdown_connections 2024-05-09T09:08:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 1 -- 172.21.15.64:0/3815533435 wait complete. 2024-05-09T09:08:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.608+0000 7f66719c1740 20 client.15420 trim_cache size 0 max 16384 2024-05-09T09:08:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 Processor -- start 2024-05-09T09:08:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 -- start start 2024-05-09T09:08:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 0x55dbaf909670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9082b0 0x55dbaf909bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf919260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf91e210 con 0x55dbaf9078c0 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf91e8e0 con 0x55dbaf9082b0 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.626+0000 7f66719c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf91efb0 con 0x55dbaf90d420 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 0x55dbaf909670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 0x55dbaf909670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33168/0 (socket says 172.21.15.64:33168) 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 -- 172.21.15.64:0/450670102 learned_addr learned my addr 172.21.15.64:0/450670102 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f666ca4f640 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9082b0 0x55dbaf909bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 -- 172.21.15.64:0/450670102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 msgr2=0x55dbaf919260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf919260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 -- 172.21.15.64:0/450670102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9082b0 msgr2=0x55dbaf909bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9082b0 0x55dbaf909bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 -- 172.21.15.64:0/450670102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbaf910710 con 0x55dbaf9078c0 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f6667fff640 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 0x55dbaf909670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6654013640 tx=0x7f665400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3722f254d259d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f66527fc640 1 -- 172.21.15.64:0/450670102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654005310 con 0x55dbaf9078c0 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f66527fc640 1 -- 172.21.15.64:0/450670102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6654011d10 con 0x55dbaf9078c0 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.627+0000 7f66527fc640 1 -- 172.21.15.64:0/450670102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654003ae0 con 0x55dbaf9078c0 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 -- 172.21.15.64:0/450670102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 msgr2=0x55dbaf909670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 0x55dbaf909670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6654013640 tx=0x7f665400b720 comp rx=0 tx=0).stop 2024-05-09T09:08:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 -- 172.21.15.64:0/450670102 shutdown_connections 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf90d420 0x55dbaf919260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9082b0 0x55dbaf909bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 --2- 172.21.15.64:0/450670102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf9078c0 0x55dbaf909670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 -- 172.21.15.64:0/450670102 >> 172.21.15.64:0/450670102 conn(0x55dbaf902f30 msgr2=0x55dbaf903310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 -- 172.21.15.64:0/450670102 shutdown_connections 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 -- 172.21.15.64:0/450670102 wait complete. 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 -- start start 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.628+0000 7f66719c1740 1 Processor -- start 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf9148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf914de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf917e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf91fd50 con 0x55dbaf90d420 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf91f680 con 0x55dbaf9078c0 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dbaf920420 con 0x55dbaf9082b0 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 10 client.0 _ll_register_callbacks cb 0x7f665c00c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf917e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf917e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33174/0 (socket says 172.21.15.64:33174) 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 -- 172.21.15.64:0/3870859261 learned_addr learned my addr 172.21.15.64:0/3870859261 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf9148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f666ca4f640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf914de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 -- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 msgr2=0x55dbaf914de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf914de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 -- 172.21.15.64:0/3870859261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 msgr2=0x55dbaf9148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf9148a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 -- 172.21.15.64:0/3870859261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbaf9183f0 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6667fff640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf917e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66540117b0 tx=0x7f665400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ad64d0d4b67eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6647fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6647fff640 1 -- 172.21.15.64:0/3870859261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665403f060 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f6647fff640 1 -- 172.21.15.64:0/3870859261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6654003860 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.629+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6654011d10 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f6647fff640 1 -- 172.21.15.64:0/3870859261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f66540053e0 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 10 client.15426 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dbaf984a10 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f6647fff640 1 -- 172.21.15.64:0/3870859261 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6654011d10 con 0x55dbaf90d420 2024-05-09T09:08:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f6647fff640 1 -- 172.21.15.64:0/3870859261 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f665401d070 con 0x55dbaf90d420 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 10 client.15426 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 20 client.15426 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf988d20 0x55dbaf987a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 4 client.15426 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "status"}) -- 0x55dbaf872f50 con 0x55dbaf988d20 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.630+0000 7f666ca4f640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf988d20 0x55dbaf987a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f666ca4f640 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf988d20 0x55dbaf987a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6171fe35000a4685 server_cookie=6eb9745c6828b20f in_seq=0 out_seq=0 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f6647fff640 10 client.15426 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f6647fff640 1 -- 172.21.15.64:0/3870859261 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55dbaf872f50 con 0x55dbaf988d20 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f6647fff640 10 client.15426 handle_command_reply: tid=0 2024-05-09T09:08:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 client.15426 shutdown 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 msgr2=0x55dbaf917e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf917e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66540117b0 tx=0x7f665400b8b0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 shutdown_connections 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbaf988d20 0x55dbaf987a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbaf90d420 0x55dbaf917e00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f6647fff640 0 client.15426 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbaf9082b0 0x55dbaf914de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 --2- 172.21.15.64:0/3870859261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbaf9078c0 0x55dbaf9148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.631+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 >> 172.21.15.64:0/3870859261 conn(0x55dbaf902f30 msgr2=0x55dbaf909670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 shutdown_connections 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f66719c1740 1 -- 172.21.15.64:0/3870859261 wait complete. 2024-05-09T09:08:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f66719c1740 20 client.15426 trim_cache size 0 max 16384 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24496, 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:08:59.641 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 113.418527787, 2024-05-09T09:08:59.642 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-05-09T09:08:59.642 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 94, 2024-05-09T09:08:59.642 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:08:59.642 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 153.399706665 2024-05-09T09:08:59.642 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:08:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66580a2d30 msgr2=0x7f66580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66580a2d30 0x7f66580a51f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66600bcef0 tx=0x7f66600b2440 comp rx=0 tx=0).stop 2024-05-09T09:08:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 msgr2=0x7f666819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f666819ef80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6660009dc0 tx=0x7f6660003ba0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 shutdown_connections 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66580a2d30 0x7f66580a51f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6668102340 0x7f666819ef80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6668101980 0x7f6668197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 --2- 172.21.15.64:0/1222313067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6668100790 0x7f66681979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 >> 172.21.15.64:0/1222313067 conn(0x7f66680fbd20 msgr2=0x7f66680fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 shutdown_connections 2024-05-09T09:08:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.632+0000 7f666ecb3640 1 -- 172.21.15.64:0/1222313067 wait complete. 2024-05-09T09:08:59.644 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 -- 172.21.15.64:0/3570421584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c100790 msgr2=0x7f473c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 --2- 172.21.15.64:0/3570421584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c100790 0x7f473c100bb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4730009900 tx=0x7f473002a410 comp rx=0 tx=0).stop 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 -- 172.21.15.64:0/3570421584 shutdown_connections 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 --2- 172.21.15.64:0/3570421584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102340 0x7f473c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 --2- 172.21.15.64:0/3570421584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f473c101980 0x7f473c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 --2- 172.21.15.64:0/3570421584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c100790 0x7f473c100bb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 -- 172.21.15.64:0/3570421584 >> 172.21.15.64:0/3570421584 conn(0x7f473c0fbd20 msgr2=0x7f473c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 -- 172.21.15.64:0/3570421584 shutdown_connections 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.751+0000 7f474467d640 1 -- 172.21.15.64:0/3570421584 wait complete. 2024-05-09T09:08:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 Processor -- start 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 -- start start 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f473c100790 0x7f473c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 0x7f473c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102340 0x7f473c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10d3a0 con 0x7f473c101980 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10d220 con 0x7f473c100790 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f474467d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10d520 con 0x7f473c102340 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 0x7f473c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 0x7f473c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33196/0 (socket says 172.21.15.64:33196) 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 -- 172.21.15.64:0/704373794 learned_addr learned my addr 172.21.15.64:0/704373794 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4742c1a640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102340 0x7f473c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4742419640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f473c100790 0x7f473c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 -- 172.21.15.64:0/704373794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102340 msgr2=0x7f473c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102340 0x7f473c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 -- 172.21.15.64:0/704373794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f473c100790 msgr2=0x7f473c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f473c100790 0x7f473c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.752+0000 7f4741c18640 1 -- 172.21.15.64:0/704373794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c19f750 con 0x7f473c101980 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.753+0000 7f4741c18640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 0x7f473c1980c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f473800db80 tx=0x7f473800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.753+0000 7f472f7fe640 1 -- 172.21.15.64:0/704373794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738017070 con 0x7f473c101980 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.753+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c19fa40 con 0x7f473c101980 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.753+0000 7f472f7fe640 1 -- 172.21.15.64:0/704373794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4738003b70 con 0x7f473c101980 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.753+0000 7f472f7fe640 1 -- 172.21.15.64:0/704373794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738010480 con 0x7f473c101980 2024-05-09T09:08:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.753+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c19ffe0 con 0x7f473c101980 2024-05-09T09:08:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.755+0000 7f472f7fe640 1 -- 172.21.15.64:0/704373794 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4738004020 con 0x7f473c101980 2024-05-09T09:08:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.755+0000 7f474738f740 1 Processor -- start 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.755+0000 7f474738f740 1 -- start start 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f474738f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 0x55ad246e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f474738f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f474738f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246ea420 0x55ad246ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f474738f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad246484d0 con 0x55ad246e52b0 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f474738f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad245f03e0 con 0x55ad246ea420 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f474738f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad24644490 con 0x55ad246e48c0 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742419640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 0x55ad246e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742419640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33212/0 (socket says 172.21.15.64:33212) 2024-05-09T09:08:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 0x55ad246e4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54184/0 (socket says 172.21.15.64:54184) 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742419640 1 -- 172.21.15.64:0/1437855405 learned_addr learned my addr 172.21.15.64:0/1437855405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4741c18640 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246ea420 0x55ad246ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f472f7fe640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47280a2b90 0x7f47280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 -- 172.21.15.64:0/1437855405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246ea420 msgr2=0x55ad246ec8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246ea420 0x55ad246ec8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 -- 172.21.15.64:0/1437855405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 msgr2=0x55ad246e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 -- 172.21.15.64:0/1437855405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad2435c2d0 con 0x55ad246e48c0 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742419640 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f472f7fe640 1 -- 172.21.15.64:0/704373794 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47380cf4c0 con 0x7f473c101980 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.756+0000 7f4742c1a640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47280a2b90 0x7f47280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f4742c1a640 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 0x55ad246e4ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47340057b0 tx=0x7f473400bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a53d05c8002716 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f472cff9640 1 -- 172.21.15.64:0/1437855405 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734024070 con 0x55ad246e48c0 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f4742c1a640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47280a2b90 0x7f47280a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4734021000 tx=0x7f473401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f472cff9640 1 -- 172.21.15.64:0/1437855405 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f473401e930 con 0x55ad246e48c0 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f472cff9640 1 -- 172.21.15.64:0/1437855405 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340116f0 con 0x55ad246e48c0 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f474738f740 1 -- 172.21.15.64:0/1437855405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 msgr2=0x55ad246e4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.757+0000 7f474738f740 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 0x55ad246e4ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47340057b0 tx=0x7f473400bb10 comp rx=0 tx=0).stop 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- 172.21.15.64:0/1437855405 shutdown_connections 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246ea420 0x55ad246ec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 --2- 172.21.15.64:0/1437855405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e48c0 0x55ad246e4ce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- 172.21.15.64:0/1437855405 >> 172.21.15.64:0/1437855405 conn(0x55ad246dff30 msgr2=0x55ad246e0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- 172.21.15.64:0/1437855405 shutdown_connections 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- 172.21.15.64:0/1437855405 wait complete. 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- start start 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 Processor -- start 2024-05-09T09:08:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e52b0 0x55ad246ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 0x55ad246f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad246ed3d0 con 0x55ad246ea420 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad246038e0 con 0x55ad246e48c0 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad246ed550 con 0x55ad246e52b0 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f474738f740 10 client.0 _ll_register_callbacks cb 0x55ad245d09c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f4742c1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 0x55ad246f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f4742c1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 0x55ad246f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33228/0 (socket says 172.21.15.64:33228) 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f4742c1a640 1 -- 172.21.15.64:0/591970492 learned_addr learned my addr 172.21.15.64:0/591970492 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f4741c18640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e52b0 0x55ad246ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.758+0000 7f4742419640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4742c1a640 1 -- 172.21.15.64:0/591970492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e52b0 msgr2=0x55ad246ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4742c1a640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e52b0 0x55ad246ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4742c1a640 1 -- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 msgr2=0x55ad246eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4742c1a640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4742c1a640 1 -- 172.21.15.64:0/591970492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad244266b0 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4742c1a640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 0x55ad246f62c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f47340132d0 tx=0x7f473400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbb95df55ed80ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4725ffb640 1 -- 172.21.15.64:0/591970492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734024040 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4725ffb640 1 -- 172.21.15.64:0/591970492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f473401e480 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473401e930 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4725ffb640 1 -- 172.21.15.64:0/591970492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4734039050 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f474738f740 10 client.15438 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ad246f9310 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.759+0000 7f4725ffb640 1 -- 172.21.15.64:0/591970492 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f473401e930 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f4725ffb640 1 -- 172.21.15.64:0/591970492 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f47340049d0 con 0x55ad246ea420 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f474738f740 10 client.15438 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:59.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f474738f740 10 client.15438 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f474738f740 20 client.15438 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f474738f740 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad246fb650 0x55ad246fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f474738f740 4 client.15438 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad24655c00 con 0x55ad246fb650 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f4742419640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad246fb650 0x55ad246fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f4725ffb640 10 client.15438 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.760+0000 7f4742419640 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad246fb650 0x55ad246fcac0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e3bd8501cff8280 server_cookie=66acbbab91f8b3a3 in_seq=0 out_seq=0 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f4725ffb640 1 -- 172.21.15.64:0/591970492 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad24655c00 con 0x55ad246fb650 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f4725ffb640 10 client.15438 handle_command_reply: tid=0 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 client.15438 shutdown 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 msgr2=0x55ad246f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 0x55ad246f62c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f47340132d0 tx=0x7f473400bf60 comp rx=0 tx=0).stop 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 shutdown_connections 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad246fb650 0x55ad246fcac0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246ea420 0x55ad246f62c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f4725ffb640 0 client.15438 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246e52b0 0x55ad246ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 --2- 172.21.15.64:0/591970492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 >> 172.21.15.64:0/591970492 conn(0x55ad246dff30 msgr2=0x55ad246df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.761+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 shutdown_connections 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.762+0000 7f474738f740 1 -- 172.21.15.64:0/591970492 wait complete. 2024-05-09T09:08:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.762+0000 7f474738f740 20 client.15438 trim_cache size 0 max 16384 2024-05-09T09:08:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.779+0000 7f474738f740 1 Processor -- start 2024-05-09T09:08:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 -- start start 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad246fae00 con 0x55ad246e52b0 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad246fb4d0 con 0x55ad246e48c0 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f474738f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad246fbba0 con 0x55ad246ea420 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246ed980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33236/0 (socket says 172.21.15.64:33236) 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 -- 172.21.15.64:0/7586006 learned_addr learned my addr 172.21.15.64:0/7586006 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4742c1a640 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 -- 172.21.15.64:0/7586006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 msgr2=0x55ad246dad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246dad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 -- 172.21.15.64:0/7586006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 msgr2=0x55ad246e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 -- 172.21.15.64:0/7586006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad246ed710 con 0x55ad246e52b0 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4742c1a640 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246dad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4741c18640 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246ed980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4738097dd0 tx=0x7f473806a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbccb9fe0c9909b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4727fff640 1 -- 172.21.15.64:0/7586006 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380cf050 con 0x55ad246e52b0 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4727fff640 1 -- 172.21.15.64:0/7586006 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f473809b970 con 0x55ad246e52b0 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.780+0000 7f4727fff640 1 -- 172.21.15.64:0/7586006 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738003d20 con 0x55ad246e52b0 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- 172.21.15.64:0/7586006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 msgr2=0x55ad246ed980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246ed980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4738097dd0 tx=0x7f473806a8f0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- 172.21.15.64:0/7586006 shutdown_connections 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246dad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246ed980 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- 172.21.15.64:0/7586006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- 172.21.15.64:0/7586006 >> 172.21.15.64:0/7586006 conn(0x55ad246dff30 msgr2=0x55ad246e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- 172.21.15.64:0/7586006 shutdown_connections 2024-05-09T09:08:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- 172.21.15.64:0/7586006 wait complete. 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- start start 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 Processor -- start 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246f4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.781+0000 7f474738f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad246fc940 con 0x55ad246e52b0 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f474738f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad246fc270 con 0x55ad246e48c0 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f474738f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad246fd010 con 0x55ad246ea420 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f474738f740 10 client.0 _ll_register_callbacks cb 0x7f4734022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33238/0 (socket says 172.21.15.64:33238) 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 -- 172.21.15.64:0/2712290565 learned_addr learned my addr 172.21.15.64:0/2712290565 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4742c1a640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4742419640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 -- 172.21.15.64:0/2712290565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 msgr2=0x55ad246f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 -- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 msgr2=0x55ad246e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 -- 172.21.15.64:0/2712290565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad246f4ab0 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4741c18640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e8910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4738009b10 tx=0x7f47380ccf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176babf3de1dea35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4725ffb640 1 -- 172.21.15.64:0/2712290565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380a45e0 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4725ffb640 1 -- 172.21.15.64:0/2712290565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47380976c0 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473809b970 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f4725ffb640 1 -- 172.21.15.64:0/2712290565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f47380cf030 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f474738f740 10 client.15444 fetch_fsmap learned FSMap version 24 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.782+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ad24761600 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f4725ffb640 1 -- 172.21.15.64:0/2712290565 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f473809b970 con 0x55ad246e52b0 2024-05-09T09:08:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f4725ffb640 1 -- 172.21.15.64:0/2712290565 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4738023060 con 0x55ad246e52b0 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f474738f740 10 client.15444 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f474738f740 10 client.15444 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f474738f740 20 client.15444 populate_metadata read hostname 'smithi064' 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f474738f740 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad24765c60 0x55ad24762770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f474738f740 4 client.15444 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "status"}) -- 0x55ad2464ff50 con 0x55ad24765c60 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f4742419640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad24765c60 0x55ad24762770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f4725ffb640 10 client.15444 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.783+0000 7f4742419640 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad24765c60 0x55ad24762770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478ac0fdaeded1c2 server_cookie=8d7ae0d746955fd0 in_seq=0 out_seq=0 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f4725ffb640 1 -- 172.21.15.64:0/2712290565 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ad2464ff50 con 0x55ad24765c60 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f4725ffb640 10 client.15444 handle_command_reply: tid=0 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 client.15444 shutdown 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 msgr2=0x55ad246e8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e8910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4738009b10 tx=0x7f47380ccf40 comp rx=0 tx=0).stop 2024-05-09T09:08:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 shutdown_connections 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad24765c60 0x55ad24762770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad246ea420 0x55ad246f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f4725ffb640 0 client.15444 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad246e52b0 0x55ad246e8910 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 --2- 172.21.15.64:0/2712290565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad246e48c0 0x55ad246e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 >> 172.21.15.64:0/2712290565 conn(0x55ad246dff30 msgr2=0x55ad246e9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 shutdown_connections 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 1 -- 172.21.15.64:0/2712290565 wait complete. 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474738f740 20 client.15444 trim_cache size 0 max 16384 2024-05-09T09:08:59.790 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "2877b902-0de2-11ef-bc98-c7b262605968", 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24503, 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 138.66252346100001, 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 94, 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 90, 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 144.713977333 2024-05-09T09:08:59.791 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:08:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47280a2b90 msgr2=0x7f47280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47280a2b90 0x7f47280a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4734021000 tx=0x7f473401e000 comp rx=0 tx=0).stop 2024-05-09T09:08:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 msgr2=0x7f473c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:08:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 0x7f473c1980c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f473800db80 tx=0x7f473800b630 comp rx=0 tx=0).stop 2024-05-09T09:08:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 shutdown_connections 2024-05-09T09:08:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47280a2b90 0x7f47280a5050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102340 0x7f473c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f473c101980 0x7f473c1980c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.784+0000 7f474467d640 1 --2- 172.21.15.64:0/704373794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f473c100790 0x7f473c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:08:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.785+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 >> 172.21.15.64:0/704373794 conn(0x7f473c0fbd20 msgr2=0x7f473c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:08:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.785+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 shutdown_connections 2024-05-09T09:08:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:08:59.785+0000 7f474467d640 1 -- 172.21.15.64:0/704373794 wait complete. 2024-05-09T09:08:59.794 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T09:08:59.805 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T09:08:59.805 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T09:08:59.806 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T09:08:59.806 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-09T09:08:59.822 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T09:08:59.822 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-09T09:08:59.823 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-09T09:08:59.823 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T09:08:59.823 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-09T09:08:59.823 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-09T09:08:59.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:08:59.823 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-09T09:08:59.839 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T09:08:59.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:08:59.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T09:08:59.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:59 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2156634487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:08:59.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:08:59 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4154319989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:08:59.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:08:59.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-05-09T09:09:00.015 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T09:09:00.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-09T09:09:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:59 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2156634487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:00.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:08:59 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4154319989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:00.267 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:09:00.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:59 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2156634487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:00.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:08:59 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4154319989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:01.126 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:00 smithi064 ceph-mon[30149]: pgmap v366: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:09:01.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:00 smithi179 ceph-mon[34086]: pgmap v366: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:09:01.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:00 smithi139 ceph-mon[33599]: pgmap v366: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 -- 172.21.15.64:0/693066651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406b6a0 msgr2=0x7ffb94106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/693066651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406b6a0 0x7ffb94106fb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009990 tx=0x7ffb88009ca0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 -- 172.21.15.64:0/693066651 shutdown_connections 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/693066651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb94107690 0x7ffb9410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/693066651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb9406acf0 0x7ffb9406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/693066651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406b6a0 0x7ffb94106fb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 -- 172.21.15.64:0/693066651 >> 172.21.15.64:0/693066651 conn(0x7ffb94077270 msgr2=0x7ffb94077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 -- 172.21.15.64:0/693066651 shutdown_connections 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.057+0000 7ffb9b02b700 1 -- 172.21.15.64:0/693066651 wait complete. 2024-05-09T09:09:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 Processor -- start 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 -- start start 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 0x7ffb94101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb9406b6a0 0x7ffb941018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb94107690 0x7ffb94101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941124e0 con 0x7ffb9406acf0 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94112360 con 0x7ffb94107690 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb9b02b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94112660 con 0x7ffb9406b6a0 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb98dc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 0x7ffb94101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb98dc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 0x7ffb94101380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33256/0 (socket says 172.21.15.64:33256) 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.058+0000 7ffb98dc7700 1 -- 172.21.15.64:0/1235736025 learned_addr learned my addr 172.21.15.64:0/1235736025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb93fff700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb9406b6a0 0x7ffb941018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb995c8700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb94107690 0x7ffb94101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb98dc7700 1 -- 172.21.15.64:0/1235736025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb9406b6a0 msgr2=0x7ffb941018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:02.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb98dc7700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb9406b6a0 0x7ffb941018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb98dc7700 1 -- 172.21.15.64:0/1235736025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb94107690 msgr2=0x7ffb94101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb98dc7700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb94107690 0x7ffb94101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb98dc7700 1 -- 172.21.15.64:0/1235736025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb941a6f10 con 0x7ffb9406acf0 2024-05-09T09:09:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb98dc7700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 0x7ffb94101380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e9c0 tx=0x7ffb8400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:02.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400cbd0 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb8400cd70 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb941a7200 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.059+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84010430 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.060+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb941a7740 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.061+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb94069010 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.062+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb84004750 con 0x7ffb9406acf0 2024-05-09T09:09:02.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.062+0000 7ffb91ffb700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb7c0a2ca0 0x7ffb7c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.063+0000 7ffb93fff700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb7c0a2ca0 0x7ffb7c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.063+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffb840cf770 con 0x7ffb9406acf0 2024-05-09T09:09:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.063+0000 7ffb93fff700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb7c0a2ca0 0x7ffb7c0a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffb94108ad0 tx=0x7ffb8801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:02.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.067+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb8409c340 con 0x7ffb9406acf0 2024-05-09T09:09:02.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.320+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffb9402db80 con 0x7ffb9406acf0 2024-05-09T09:09:02.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.320+0000 7ffb91ffb700 1 -- 172.21.15.64:0/1235736025 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7ffb8409bc40 con 0x7ffb9406acf0 2024-05-09T09:09:02.320 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb7c0a2ca0 msgr2=0x7ffb7c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb7c0a2ca0 0x7ffb7c0a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffb94108ad0 tx=0x7ffb8801a040 comp rx=0 tx=0).stop 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 msgr2=0x7ffb94101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 0x7ffb94101380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e9c0 tx=0x7ffb8400ed80 comp rx=0 tx=0).stop 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 shutdown_connections 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb94107690 0x7ffb94101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9406acf0 0x7ffb94101380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb7c0a2ca0 0x7ffb7c0a5160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 --2- 172.21.15.64:0/1235736025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb9406b6a0 0x7ffb941018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 >> 172.21.15.64:0/1235736025 conn(0x7ffb94077270 msgr2=0x7ffb940ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 shutdown_connections 2024-05-09T09:09:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:02.323+0000 7ffb9b02b700 1 -- 172.21.15.64:0/1235736025 wait complete. 2024-05-09T09:09:03.019 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:02 smithi064 ceph-mon[30149]: pgmap v367: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:09:03.019 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:02 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1235736025' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T09:09:03.049 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-09T09:09:03.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:03.049 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-05-09T09:09:03.067 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T09:09:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T09:09:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T09:09:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:09:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-05-09T09:09:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42121sec preferred_lft 42121sec 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-05-09T09:09:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-05-09T09:09:03.069 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T09:09:03.069 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:09:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:09:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:09:03.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:03.215 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T09:09:03.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:02 smithi179 ceph-mon[34086]: pgmap v367: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:09:03.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:02 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1235736025' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:09:03.239 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-05-09T09:09:03.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:03.241 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-05-09T09:09:03.298 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-05-09T09:09:03.298 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T09:09:03.298 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T09:09:03.298 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T09:09:03.299 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T09:09:03.299 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:09:03.299 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:09:03.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T09:09:03.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T09:09:03.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T09:09:03.299 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:09:03.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:02 smithi139 ceph-mon[33599]: pgmap v367: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:09:03.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:02 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1235736025' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T09:09:03.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:09:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:09:03.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:03.429 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-09T09:09:03.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:03.446 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-05-09T09:09:03.502 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:09:03.503 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:09:03.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-09T09:09:03.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-09T09:09:03.503 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:09:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:09:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:09:03.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-09T09:09:03.624 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:09:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:09:03.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:09:03.742 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T09:09:03.742 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-05-09T09:09:03.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-05-09T09:09:03.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T09:09:03.742 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-05-09T09:09:03.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T09:09:03.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:09:03.822 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-09T09:09:03.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:09:03.822 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /tmp/cephfs 2024-05-09T09:09:03.838 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-09T09:09:03.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T09:09:03.838 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /tmp/cephfs 2024-05-09T09:09:03.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-05-09T09:09:03.961 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:09:04.019 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-05-09T09:09:04.019 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-05-09T09:09:04.019 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-05-09T09:09:04.019 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-05-09T09:09:04.019 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:09:04.020 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:09:04.021 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/772ca47141e89a435b1d4995ca7ca0255eeb3e7e3688439228116871ac4d9c3b/merged 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5fd57eec47dd9362180c164c9948523587125bcef039d506fd3669295e466963/merged 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:09:04.022 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/cc30f61bab2b46c2740ffa4e83d1465ed8121ce8a19bdf5854792c73278a4b99/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/682bf516c81da3d3ea12bc411010a33978c3943b58d077694c76780c78065648/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/66eeb9b9f88ac7c0171f72bc86ef7dd1bd98829e92f8b1170ec7e931d97a22bf/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/39d3a895bf51eac157ab546c2811481aed20bb20e2d1f7f2b06d93af2db82269/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/efb2205fd00fd12c2bb8b9f0f71d4d65e45a88d143b0e0a79ab4a9c2dd4f6a0e/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7b5fd69a89f54593dcb5250ec9a0353112a929e54486fbf644a57c5b2bc1057d/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a69532b418b1328df84cbcdd5bca8a0a218491254d483d92be609490f02b12ee/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/aca1353679f4b3104cf0788d46b2eb0aa5c85c46745ddf8949a56baadf7ec1d6/merged 2024-05-09T09:09:04.023 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-05-09T09:09:04.024 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-09T09:09:04.024 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-09T09:09:04.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:04.024 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-05-09T09:09:04.075 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T09:09:04.075 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-09T09:09:04.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-05-09T09:09:04.145 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-05-09T09:09:04.202 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-05-09T09:09:04.203 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/772ca47141e89a435b1d4995ca7ca0255eeb3e7e3688439228116871ac4d9c3b/merged 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5fd57eec47dd9362180c164c9948523587125bcef039d506fd3669295e466963/merged 2024-05-09T09:09:04.204 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/cc30f61bab2b46c2740ffa4e83d1465ed8121ce8a19bdf5854792c73278a4b99/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/682bf516c81da3d3ea12bc411010a33978c3943b58d077694c76780c78065648/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/66eeb9b9f88ac7c0171f72bc86ef7dd1bd98829e92f8b1170ec7e931d97a22bf/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/39d3a895bf51eac157ab546c2811481aed20bb20e2d1f7f2b06d93af2db82269/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/efb2205fd00fd12c2bb8b9f0f71d4d65e45a88d143b0e0a79ab4a9c2dd4f6a0e/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7b5fd69a89f54593dcb5250ec9a0353112a929e54486fbf644a57c5b2bc1057d/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a69532b418b1328df84cbcdd5bca8a0a218491254d483d92be609490f02b12ee/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/aca1353679f4b3104cf0788d46b2eb0aa5c85c46745ddf8949a56baadf7ec1d6/merged 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-09T09:09:04.205 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-09T09:09:04.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:04.206 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-05-09T09:09:04.207 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-05-09T09:09:04.207+0000 7fa24400c480 -1 init, newargv = 0x562e8d7884c0 newargc=17 2024-05-09T09:09:04.207 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[82154]: starting ceph client 2024-05-09T09:09:04.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[82154]: starting fuse 2024-05-09T09:09:04.223 INFO:teuthology.orchestra.run.smithi064.stdout:114 2024-05-09T09:09:04.223 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T09:09:04.223 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-05-09T09:09:04.223 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-05-09T09:09:04.223 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-05-09T09:09:04.223 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-05-09T09:09:04.223 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-05-09T09:09:04.223 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/tmp/cephfs" 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T09:09:04.224 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-05-09T09:09:04.225 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-05-09T09:09:04.345 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.82154.asok 2024-05-09T09:09:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T09:09:04.356 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.82154.asok 2024-05-09T09:09:04.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:04.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.82154.asok status 2024-05-09T09:09:04.494 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/tmp/cephfs", 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "82154", 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85" 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-05-09T09:09:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 15456, 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 15456 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2662783838 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:09:04.496 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2662783838 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.15456 192.168.0.1:0/2662783838", 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2662783838", 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 5, 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 24, 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 94, 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-05-09T09:09:04.497 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:09:04.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:04.503 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-05-09T09:09:04.504 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /tmp/cephfs 2024-05-09T09:09:04.520 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-05-09T09:09:04.521 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-09T09:09:04.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:04.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /tmp/cephfs 2024-05-09T09:09:04.590 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T09:09:04.604 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T09:09:04.604 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:09:04.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T09:09:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/437351001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf81023f0 msgr2=0x7f2cf8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/437351001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf81023f0 0x7f2cf8108c80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009ca0 tx=0x7f2cf002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/437351001 shutdown_connections 2024-05-09T09:09:04.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/437351001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf81023f0 0x7f2cf8108c80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/437351001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8101a30 0x7f2cf8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/437351001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf8100840 0x7f2cf8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/437351001 >> 172.21.15.64:0/437351001 conn(0x7f2cf80fbdb0 msgr2=0x7f2cf80fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/437351001 shutdown_connections 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.737+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/437351001 wait complete. 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 Processor -- start 2024-05-09T09:09:04.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 -- start start 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 0x7f2cf8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8101a30 0x7f2cf8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf81023f0 0x7f2cf819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf810d420 con 0x7f2cf8100840 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf810d2a0 con 0x7f2cf81023f0 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cfd0f5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf810d5a0 con 0x7f2cf8101a30 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 0x7f2cf8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:04.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 0x7f2cf8197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38822/0 (socket says 172.21.15.64:38822) 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 -- 172.21.15.64:0/1002600162 learned_addr learned my addr 172.21.15.64:0/1002600162 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf659c640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8101a30 0x7f2cf8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 -- 172.21.15.64:0/1002600162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8101a30 msgr2=0x7f2cf8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8101a30 0x7f2cf8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 -- 172.21.15.64:0/1002600162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf81023f0 msgr2=0x7f2cf819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf81023f0 0x7f2cf819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.738+0000 7f2cf6d9d640 1 -- 172.21.15.64:0/1002600162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf819f5a0 con 0x7f2cf8100840 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.739+0000 7f2cf6d9d640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 0x7f2cf8197980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c970 tx=0x7f2ce800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.739+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8007c10 con 0x7f2cf8100840 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.739+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ce8007db0 con 0x7f2cf8100840 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.739+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf819f890 con 0x7f2cf8100840 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.739+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800f870 con 0x7f2cf8100840 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.739+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf819fde0 con 0x7f2cf8100840 2024-05-09T09:09:04.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.740+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf8107a90 con 0x7f2cf8100840 2024-05-09T09:09:04.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.741+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ce800fa10 con 0x7f2cf8100840 2024-05-09T09:09:04.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.742+0000 7f2ceffff640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ce40a2c30 0x7f2ce40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:04.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.742+0000 7f2cf659c640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ce40a2c30 0x7f2ce40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:04.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.742+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ce80cf770 con 0x7f2cf8100840 2024-05-09T09:09:04.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.742+0000 7f2cf659c640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ce40a2c30 0x7f2ce40a50f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8198e90 tx=0x7f2ce000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:04.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.746+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ce809c3d0 con 0x7f2cf8100840 2024-05-09T09:09:04.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:04.975+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f2cf802d1c0 con 0x7f2cf8100840 2024-05-09T09:09:05.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:04 smithi064 ceph-mon[30149]: pgmap v368: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:09:05.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:04 smithi179 ceph-mon[34086]: pgmap v368: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:09:05.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:04 smithi139 ceph-mon[33599]: pgmap v368: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:09:05.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.846+0000 7f2ceffff640 1 -- 172.21.15.64:0/1002600162 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f2ce809bcd0 con 0x7f2cf8100840 2024-05-09T09:09:05.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.850+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ce40a2c30 msgr2=0x7f2ce40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.850+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ce40a2c30 0x7f2ce40a50f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8198e90 tx=0x7f2ce000b390 comp rx=0 tx=0).stop 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.850+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 msgr2=0x7f2cf8197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.850+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 0x7f2cf8197980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c970 tx=0x7f2ce800ce30 comp rx=0 tx=0).stop 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 shutdown_connections 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2ce40a2c30 0x7f2ce40a50f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf81023f0 0x7f2cf819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8101a30 0x7f2cf8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 --2- 172.21.15.64:0/1002600162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cf8100840 0x7f2cf8197980 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 >> 172.21.15.64:0/1002600162 conn(0x7f2cf80fbdb0 msgr2=0x7f2cf8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 shutdown_connections 2024-05-09T09:09:05.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:05.851+0000 7f2cfd0f5640 1 -- 172.21.15.64:0/1002600162 wait complete. 2024-05-09T09:09:05.863 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T09:09:05.874 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T09:09:05.886 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:09:06.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.000+0000 7f4daa389640 1 -- 172.21.15.64:0/2650608162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da4101db0 msgr2=0x7f4da41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.000+0000 7f4daa389640 1 --2- 172.21.15.64:0/2650608162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da4101db0 0x7f4da41021d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009900 tx=0x7f4d9402a410 comp rx=0 tx=0).stop 2024-05-09T09:09:06.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 -- 172.21.15.64:0/2650608162 shutdown_connections 2024-05-09T09:09:06.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 --2- 172.21.15.64:0/2650608162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da41014a0 0x7f4da4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 --2- 172.21.15.64:0/2650608162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da4100ae0 0x7f4da4100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 --2- 172.21.15.64:0/2650608162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da4101db0 0x7f4da41021d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 -- 172.21.15.64:0/2650608162 >> 172.21.15.64:0/2650608162 conn(0x7f4da40fb510 msgr2=0x7f4da40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 -- 172.21.15.64:0/2650608162 shutdown_connections 2024-05-09T09:09:06.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 -- 172.21.15.64:0/2650608162 wait complete. 2024-05-09T09:09:06.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 Processor -- start 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.001+0000 7f4daa389640 1 -- start start 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4daa389640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da4100ae0 0x7f4da4104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4daa389640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 0x7f4da4102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4daa389640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da4101db0 0x7f4da4103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4daa389640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4da410c350 con 0x7f4da41014a0 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4daa389640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4da410c1d0 con 0x7f4da4100ae0 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4daa389640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4da410c4d0 con 0x7f4da4101db0 2024-05-09T09:09:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 0x7f4da4102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 0x7f4da4102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38840/0 (socket says 172.21.15.64:38840) 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 -- 172.21.15.64:0/3660904724 learned_addr learned my addr 172.21.15.64:0/3660904724 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da8926640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da4101db0 0x7f4da4103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da3fff640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da4100ae0 0x7f4da4104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 -- 172.21.15.64:0/3660904724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da4101db0 msgr2=0x7f4da4103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da4101db0 0x7f4da4103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 -- 172.21.15.64:0/3660904724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da4100ae0 msgr2=0x7f4da4104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da4100ae0 0x7f4da4104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 -- 172.21.15.64:0/3660904724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da4103ca0 con 0x7f4da41014a0 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.002+0000 7f4da37fe640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 0x7f4da4102ed0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d950 tx=0x7f4d9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.003+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98017030 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.003+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da41a1c40 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.003+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d98004020 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.003+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d980047a0 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.003+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da41a2190 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.004+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da41a23c0 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.005+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d980041d0 con 0x7f4da41014a0 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.005+0000 7f4da17fa640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d8c0a2b90 0x7f4d8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.006+0000 7f4da3fff640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d8c0a2b90 0x7f4d8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.006+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4d980cee70 con 0x7f4da41014a0 2024-05-09T09:09:06.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.006+0000 7f4da3fff640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d8c0a2b90 0x7f4d8c0a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4d94004770 tx=0x7f4d94000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.010+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d9809ba40 con 0x7f4da41014a0 2024-05-09T09:09:06.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:05 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1002600162' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T09:09:06.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.197+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4da41a2750 con 0x7f4da41014a0 2024-05-09T09:09:06.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.199+0000 7f4da17fa640 1 -- 172.21.15.64:0/3660904724 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f4d980810c0 con 0x7f4da41014a0 2024-05-09T09:09:06.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:06.198 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d8c0a2b90 msgr2=0x7f4d8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d8c0a2b90 0x7f4d8c0a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4d94004770 tx=0x7f4d94000bc0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 msgr2=0x7f4da4102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 0x7f4da4102ed0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d950 tx=0x7f4d9800de10 comp rx=0 tx=0).stop 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 shutdown_connections 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d8c0a2b90 0x7f4d8c0a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da4101db0 0x7f4da4103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da41014a0 0x7f4da4102ed0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 --2- 172.21.15.64:0/3660904724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da4100ae0 0x7f4da4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 >> 172.21.15.64:0/3660904724 conn(0x7f4da40fb510 msgr2=0x7f4da40fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 shutdown_connections 2024-05-09T09:09:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.201+0000 7f4daa389640 1 -- 172.21.15.64:0/3660904724 wait complete. 2024-05-09T09:09:06.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:09:06.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:09:06.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:05 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1002600162' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T09:09:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 -- 172.21.15.64:0/825486907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc102340 msgr2=0x7f43cc108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 --2- 172.21.15.64:0/825486907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc102340 0x7f43cc108bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009ca0 tx=0x7f43c402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 -- 172.21.15.64:0/825486907 shutdown_connections 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 --2- 172.21.15.64:0/825486907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc102340 0x7f43cc108bd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 --2- 172.21.15.64:0/825486907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc101980 0x7f43cc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 --2- 172.21.15.64:0/825486907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43cc100790 0x7f43cc100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 -- 172.21.15.64:0/825486907 >> 172.21.15.64:0/825486907 conn(0x7f43cc0fbd20 msgr2=0x7f43cc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 -- 172.21.15.64:0/825486907 shutdown_connections 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.321+0000 7f43d1672640 1 -- 172.21.15.64:0/825486907 wait complete. 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 Processor -- start 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 -- start start 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc100790 0x7f43cc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 0x7f43cc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43cc102340 0x7f43cc19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc10c2a0 con 0x7f43cc101980 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc10c120 con 0x7f43cc102340 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43d1672640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc10c420 con 0x7f43cc100790 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 0x7f43cc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 0x7f43cc197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38856/0 (socket says 172.21.15.64:38856) 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 -- 172.21.15.64:0/1665294560 learned_addr learned my addr 172.21.15.64:0/1665294560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43cb7fe640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43cc102340 0x7f43cc19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43caffd640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc100790 0x7f43cc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 -- 172.21.15.64:0/1665294560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc100790 msgr2=0x7f43cc197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc100790 0x7f43cc197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 -- 172.21.15.64:0/1665294560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43cc102340 msgr2=0x7f43cc19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43cc102340 0x7f43cc19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43ca7fc640 1 -- 172.21.15.64:0/1665294560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc19f6a0 con 0x7f43cc101980 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.322+0000 7f43caffd640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc100790 0x7f43cc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.323+0000 7f43ca7fc640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 0x7f43cc197fc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f43c000d950 tx=0x7f43c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.323+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0017030 con 0x7f43cc101980 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.323+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43c0004020 con 0x7f43cc101980 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.323+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc19f990 con 0x7f43cc101980 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.323+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00047a0 con 0x7f43cc101980 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.323+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc19fee0 con 0x7f43cc101980 2024-05-09T09:09:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.324+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc1a0170 con 0x7f43cc101980 2024-05-09T09:09:06.326 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:05 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1002600162' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T09:09:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.325+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f43c00041d0 con 0x7f43cc101980 2024-05-09T09:09:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.326+0000 7f43b3fff640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43b40a2b90 0x7f43b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.326+0000 7f43caffd640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43b40a2b90 0x7f43b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.326+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43c00cef70 con 0x7f43cc101980 2024-05-09T09:09:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.326+0000 7f43caffd640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43b40a2b90 0x7f43b40a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f43cc1017e0 tx=0x7f43bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.330+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43c009bae0 con 0x7f43cc101980 2024-05-09T09:09:06.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.517+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43cc1a0340 con 0x7f43cc101980 2024-05-09T09:09:06.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.517+0000 7f43b3fff640 1 -- 172.21.15.64:0/1665294560 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f43c009b3e0 con 0x7f43cc101980 2024-05-09T09:09:06.517 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:09:06.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43b40a2b90 msgr2=0x7f43b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43b40a2b90 0x7f43b40a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f43cc1017e0 tx=0x7f43bc005f90 comp rx=0 tx=0).stop 2024-05-09T09:09:06.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 msgr2=0x7f43cc197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 0x7f43cc197fc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f43c000d950 tx=0x7f43c000de10 comp rx=0 tx=0).stop 2024-05-09T09:09:06.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 shutdown_connections 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43b40a2b90 0x7f43b40a5050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43cc102340 0x7f43cc19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43cc101980 0x7f43cc197fc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 --2- 172.21.15.64:0/1665294560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43cc100790 0x7f43cc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 >> 172.21.15.64:0/1665294560 conn(0x7f43cc0fbd20 msgr2=0x7f43cc069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 shutdown_connections 2024-05-09T09:09:06.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.520+0000 7f43d1672640 1 -- 172.21.15.64:0/1665294560 wait complete. 2024-05-09T09:09:06.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:09:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 -- 172.21.15.64:0/1377908496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650101890 msgr2=0x7fc650101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 --2- 172.21.15.64:0/1377908496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650101890 0x7fc650101cb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc640009900 tx=0x7fc64002a410 comp rx=0 tx=0).stop 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 -- 172.21.15.64:0/1377908496 shutdown_connections 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 --2- 172.21.15.64:0/1377908496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc650103440 0x7fc650109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 --2- 172.21.15.64:0/1377908496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc650102a80 0x7fc650102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 --2- 172.21.15.64:0/1377908496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650101890 0x7fc650101cb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.639+0000 7fc654d70640 1 -- 172.21.15.64:0/1377908496 >> 172.21.15.64:0/1377908496 conn(0x7fc6500fce00 msgr2=0x7fc6500ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 -- 172.21.15.64:0/1377908496 shutdown_connections 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 -- 172.21.15.64:0/1377908496 wait complete. 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 Processor -- start 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 -- start start 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc650101890 0x7fc650197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 0x7fc650197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc650103440 0x7fc65019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc65010d390 con 0x7fc650102a80 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc65010d210 con 0x7fc650101890 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.640+0000 7fc654d70640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc65010d510 con 0x7fc650103440 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 0x7fc650197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 0x7fc650197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38868/0 (socket says 172.21.15.64:38868) 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 -- 172.21.15.64:0/3262672841 learned_addr learned my addr 172.21.15.64:0/3262672841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64ed9d640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc650103440 0x7fc65019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64e59c640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc650101890 0x7fc650197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 -- 172.21.15.64:0/3262672841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc650103440 msgr2=0x7fc65019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc650103440 0x7fc65019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 -- 172.21.15.64:0/3262672841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc650101890 msgr2=0x7fc650197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc650101890 0x7fc650197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 -- 172.21.15.64:0/3262672841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65019f6b0 con 0x7fc650102a80 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc64dd9b640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 0x7fc650197fd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e9f0 tx=0x7fc64400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64400cd80 con 0x7fc650102a80 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6440108e0 con 0x7fc650102a80 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644018920 con 0x7fc650102a80 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65019f9a0 con 0x7fc650102a80 2024-05-09T09:09:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.641+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65019fef0 con 0x7fc650102a80 2024-05-09T09:09:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.642+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650108c50 con 0x7fc650102a80 2024-05-09T09:09:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.644+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc644010a90 con 0x7fc650102a80 2024-05-09T09:09:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.644+0000 7fc63f7fe640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6380a2b90 0x7fc6380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.644+0000 7fc64e59c640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6380a2b90 0x7fc6380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.644+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc6440cf750 con 0x7fc650102a80 2024-05-09T09:09:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.645+0000 7fc64e59c640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6380a2b90 0x7fc6380a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc640000c00 tx=0x7fc6400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.649+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc64409c320 con 0x7fc650102a80 2024-05-09T09:09:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.832+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc650062430 con 0x7fc650102a80 2024-05-09T09:09:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.833+0000 7fc63f7fe640 1 -- 172.21.15.64:0/3262672841 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fc644086020 con 0x7fc650102a80 2024-05-09T09:09:06.832 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:09:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6380a2b90 msgr2=0x7fc6380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6380a2b90 0x7fc6380a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc640000c00 tx=0x7fc6400314a0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 msgr2=0x7fc650197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 0x7fc650197fd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e9f0 tx=0x7fc64400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 shutdown_connections 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6380a2b90 0x7fc6380a5050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc650103440 0x7fc65019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc650102a80 0x7fc650197fd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 --2- 172.21.15.64:0/3262672841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc650101890 0x7fc650197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 >> 172.21.15.64:0/3262672841 conn(0x7fc6500fce00 msgr2=0x7fc650105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 shutdown_connections 2024-05-09T09:09:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.835+0000 7fc654d70640 1 -- 172.21.15.64:0/3262672841 wait complete. 2024-05-09T09:09:06.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:09:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 -- 172.21.15.64:0/1469800418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 msgr2=0x7f4f08100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 --2- 172.21.15.64:0/1469800418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08100b90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009ca0 tx=0x7f4f0002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 -- 172.21.15.64:0/1469800418 shutdown_connections 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 --2- 172.21.15.64:0/1469800418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08102320 0x7f4f08108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 --2- 172.21.15.64:0/1469800418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08101960 0x7f4f08101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 --2- 172.21.15.64:0/1469800418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08100b90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 -- 172.21.15.64:0/1469800418 >> 172.21.15.64:0/1469800418 conn(0x7f4f080fbce0 msgr2=0x7f4f080fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 -- 172.21.15.64:0/1469800418 shutdown_connections 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 -- 172.21.15.64:0/1469800418 wait complete. 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 Processor -- start 2024-05-09T09:09:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.954+0000 7f4f10241640 1 -- start start 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08101960 0x7f4f0819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08102320 0x7f4f0819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0810c270 con 0x7f4f08100770 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0810c0f0 con 0x7f4f08102320 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0810c3f0 con 0x7f4f08101960 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38894/0 (socket says 172.21.15.64:38894) 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 -- 172.21.15.64:0/1347026093 learned_addr learned my addr 172.21.15.64:0/1347026093 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0d7dc640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08101960 0x7f4f0819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0e7de640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08102320 0x7f4f0819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 -- 172.21.15.64:0/1347026093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08101960 msgr2=0x7f4f0819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08101960 0x7f4f0819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 -- 172.21.15.64:0/1347026093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08102320 msgr2=0x7f4f0819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08102320 0x7f4f0819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 -- 172.21.15.64:0/1347026093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0819f7d0 con 0x7f4f08100770 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0e7de640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08102320 0x7f4f0819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0d7dc640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08101960 0x7f4f0819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:06.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f0dfdd640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08069530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4f00003ba0 tx=0x7f4f00003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00004040 con 0x7f4f08100770 2024-05-09T09:09:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0819fa00 con 0x7f4f08100770 2024-05-09T09:09:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f00004690 con 0x7f4f08100770 2024-05-09T09:09:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.955+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00030cd0 con 0x7f4f08100770 2024-05-09T09:09:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.956+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0819ff50 con 0x7f4f08100770 2024-05-09T09:09:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.956+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08107bf0 con 0x7f4f08100770 2024-05-09T09:09:06.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.958+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f000041e0 con 0x7f4f08100770 2024-05-09T09:09:06.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.959+0000 7f4f06ffd640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4ef00a2b90 0x7f4ef00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:06.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.959+0000 7f4f0d7dc640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4ef00a2b90 0x7f4ef00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:06.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.959+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4f000f0ba0 con 0x7f4f08100770 2024-05-09T09:09:06.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.960+0000 7f4f0d7dc640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4ef00a2b90 0x7f4ef00a5050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f080692a0 tx=0x7f4efc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:06.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:06.962+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f000bd770 con 0x7f4f08100770 2024-05-09T09:09:07.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.145+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f0802d410 con 0x7f4f08100770 2024-05-09T09:09:07.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.146+0000 7f4f06ffd640 1 -- 172.21.15.64:0/1347026093 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f4f000bd070 con 0x7f4f08100770 2024-05-09T09:09:07.145 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4ef00a2b90 msgr2=0x7f4ef00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4ef00a2b90 0x7f4ef00a5050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f080692a0 tx=0x7f4efc005f90 comp rx=0 tx=0).stop 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 msgr2=0x7f4f08069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08069530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4f00003ba0 tx=0x7f4f00003bd0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 shutdown_connections 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4ef00a2b90 0x7f4ef00a5050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f08102320 0x7f4f0819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f08101960 0x7f4f0819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 --2- 172.21.15.64:0/1347026093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f08100770 0x7f4f08069530 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 >> 172.21.15.64:0/1347026093 conn(0x7f4f080fbce0 msgr2=0x7f4f08104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:07.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 shutdown_connections 2024-05-09T09:09:07.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.148+0000 7f4f10241640 1 -- 172.21.15.64:0/1347026093 wait complete. 2024-05-09T09:09:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:06 smithi064 ceph-mon[30149]: pgmap v369: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:09:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1002600162' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T09:09:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:06 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3660904724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1665294560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:06 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3262672841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:07.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:09:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:06 smithi179 ceph-mon[34086]: pgmap v369: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:09:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1002600162' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T09:09:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:06 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3660904724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1665294560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:06 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3262672841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 -- 172.21.15.64:0/424766297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 msgr2=0x7f430c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 --2- 172.21.15.64:0/424766297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c108bd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4304009ca0 tx=0x7f430402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 -- 172.21.15.64:0/424766297 shutdown_connections 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 --2- 172.21.15.64:0/424766297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c108bd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 --2- 172.21.15.64:0/424766297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c101980 0x7f430c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 --2- 172.21.15.64:0/424766297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f430c100790 0x7f430c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 -- 172.21.15.64:0/424766297 >> 172.21.15.64:0/424766297 conn(0x7f430c0fbd20 msgr2=0x7f430c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 -- 172.21.15.64:0/424766297 shutdown_connections 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.267+0000 7f4313c15640 1 -- 172.21.15.64:0/424766297 wait complete. 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 Processor -- start 2024-05-09T09:09:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 -- start start 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c100790 0x7f430c1046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f430c101980 0x7f430c102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c1032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f430c10c2a0 con 0x7f430c102340 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f430c10c120 con 0x7f430c101980 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f4313c15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f430c10c420 con 0x7f430c100790 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f43121b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c1032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f43121b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c1032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38920/0 (socket says 172.21.15.64:38920) 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f43121b2640 1 -- 172.21.15.64:0/265862060 learned_addr learned my addr 172.21.15.64:0/265862060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.268+0000 7f43111b0640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f430c101980 0x7f430c102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43119b1640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c100790 0x7f430c1046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43121b2640 1 -- 172.21.15.64:0/265862060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c100790 msgr2=0x7f430c1046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43121b2640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c100790 0x7f430c1046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43121b2640 1 -- 172.21.15.64:0/265862060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f430c101980 msgr2=0x7f430c102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43121b2640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f430c101980 0x7f430c102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43121b2640 1 -- 172.21.15.64:0/265862060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c103b60 con 0x7f430c102340 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43119b1640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c100790 0x7f430c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f43121b2640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c1032a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4304009dc0 tx=0x7f4304003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304032ec0 con 0x7f430c102340 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43040044f0 con 0x7f430c102340 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c1a1ad0 con 0x7f430c102340 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304030d40 con 0x7f430c102340 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.269+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c1a1ff0 con 0x7f430c102340 2024-05-09T09:09:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.270+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f430c068aa0 con 0x7f430c102340 2024-05-09T09:09:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.271+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4304004040 con 0x7f430c102340 2024-05-09T09:09:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.272+0000 7f4302ffd640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42f80a2d30 0x7f42f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.272+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43040f0980 con 0x7f430c102340 2024-05-09T09:09:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.272+0000 7f43119b1640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42f80a2d30 0x7f42f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.272+0000 7f43119b1640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42f80a2d30 0x7f42f80a51f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4308005d00 tx=0x7f4308005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:07.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.276+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43040bd550 con 0x7f430c102340 2024-05-09T09:09:07.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:06 smithi139 ceph-mon[33599]: pgmap v369: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:09:07.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1002600162' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T09:09:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:06 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3660904724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1665294560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:06 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3262672841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:07.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.459+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f430c1a2990 con 0x7f430c102340 2024-05-09T09:09:07.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.460+0000 7f4302ffd640 1 -- 172.21.15.64:0/265862060 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f43040bce50 con 0x7f430c102340 2024-05-09T09:09:07.459 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:09:07.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.461+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42f80a2d30 msgr2=0x7f42f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.461+0000 7f4313c15640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42f80a2d30 0x7f42f80a51f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4308005d00 tx=0x7f4308005c90 comp rx=0 tx=0).stop 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.461+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 msgr2=0x7f430c1032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.461+0000 7f4313c15640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c1032a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4304009dc0 tx=0x7f4304003ba0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 shutdown_connections 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42f80a2d30 0x7f42f80a51f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c102340 0x7f430c1032a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f430c101980 0x7f430c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 --2- 172.21.15.64:0/265862060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f430c100790 0x7f430c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 >> 172.21.15.64:0/265862060 conn(0x7f430c0fbd20 msgr2=0x7f430c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 shutdown_connections 2024-05-09T09:09:07.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.462+0000 7f4313c15640 1 -- 172.21.15.64:0/265862060 wait complete. 2024-05-09T09:09:07.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 -- 172.21.15.64:0/3314848439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 msgr2=0x7fc838107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 --2- 172.21.15.64:0/3314848439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838107070 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009900 tx=0x7fc82c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 -- 172.21.15.64:0/3314848439 shutdown_connections 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 --2- 172.21.15.64:0/3314848439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8381075b0 0x7fc8381099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 --2- 172.21.15.64:0/3314848439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838107070 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 --2- 172.21.15.64:0/3314848439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc838102350 0x7fc838104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.577+0000 7fc840363640 1 -- 172.21.15.64:0/3314848439 >> 172.21.15.64:0/3314848439 conn(0x7fc8380fbce0 msgr2=0x7fc8380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 -- 172.21.15.64:0/3314848439 shutdown_connections 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 -- 172.21.15.64:0/3314848439 wait complete. 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 Processor -- start 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 -- start start 2024-05-09T09:09:07.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc838102350 0x7fc838197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8381075b0 0x7fc83819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc83810d3f0 con 0x7fc838104c80 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc83810d270 con 0x7fc838102350 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc840363640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc83810d570 con 0x7fc8381075b0 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc83d8fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc83d8fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38928/0 (socket says 172.21.15.64:38928) 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.578+0000 7fc83d8fe640 1 -- 172.21.15.64:0/2624750562 learned_addr learned my addr 172.21.15.64:0/2624750562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83e0ff640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc838102350 0x7fc838197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83e900640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8381075b0 0x7fc83819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83d8fe640 1 -- 172.21.15.64:0/2624750562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8381075b0 msgr2=0x7fc83819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83d8fe640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8381075b0 0x7fc83819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83d8fe640 1 -- 172.21.15.64:0/2624750562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc838102350 msgr2=0x7fc838197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83d8fe640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc838102350 0x7fc838197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83d8fe640 1 -- 172.21.15.64:0/2624750562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83819f710 con 0x7fc838104c80 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83e900640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8381075b0 0x7fc83819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83e0ff640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc838102350 0x7fc838197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc83d8fe640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838198030 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009230 tx=0x7fc82c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c03f030 con 0x7fc838104c80 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82c003cb0 con 0x7fc838104c80 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83819f9a0 con 0x7fc838104c80 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c035070 con 0x7fc838104c80 2024-05-09T09:09:07.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.579+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83819ff10 con 0x7fc838104c80 2024-05-09T09:09:07.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.580+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8380fd7d0 con 0x7fc838104c80 2024-05-09T09:09:07.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.581+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc82c042030 con 0x7fc838104c80 2024-05-09T09:09:07.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.582+0000 7fc8377fe640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8200a2b90 0x7fc8200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.582+0000 7fc83e0ff640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8200a2b90 0x7fc8200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.582+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc82c0efb10 con 0x7fc838104c80 2024-05-09T09:09:07.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.583+0000 7fc83e0ff640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8200a2b90 0x7fc8200a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc83000baa0 tx=0x7fc830005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:07.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.587+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc82c0bc6e0 con 0x7fc838104c80 2024-05-09T09:09:07.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.771+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc838062430 con 0x7fc838104c80 2024-05-09T09:09:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.773+0000 7fc8377fe640 1 -- 172.21.15.64:0/2624750562 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7fc82c0bbfe0 con 0x7fc838104c80 2024-05-09T09:09:07.772 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:07.772 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.774+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8200a2b90 msgr2=0x7fc8200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.774+0000 7fc840363640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8200a2b90 0x7fc8200a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc83000baa0 tx=0x7fc830005d90 comp rx=0 tx=0).stop 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 msgr2=0x7fc838198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838198030 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009230 tx=0x7fc82c004340 comp rx=0 tx=0).stop 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 shutdown_connections 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8200a2b90 0x7fc8200a5050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8381075b0 0x7fc83819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc838104c80 0x7fc838198030 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 --2- 172.21.15.64:0/2624750562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc838102350 0x7fc838197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 >> 172.21.15.64:0/2624750562 conn(0x7fc8380fbce0 msgr2=0x7fc838102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:07.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 shutdown_connections 2024-05-09T09:09:07.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.775+0000 7fc840363640 1 -- 172.21.15.64:0/2624750562 wait complete. 2024-05-09T09:09:07.785 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:09:07.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.894+0000 7fb1df9f9640 1 -- 172.21.15.64:0/2530453346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 msgr2=0x7fb1d8100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.894+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/2530453346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8100b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009900 tx=0x7fb1d402a410 comp rx=0 tx=0).stop 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- 172.21.15.64:0/2530453346 shutdown_connections 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/2530453346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8102320 0x7fb1d8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/2530453346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8101960 0x7fb1d8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/2530453346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8100b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- 172.21.15.64:0/2530453346 >> 172.21.15.64:0/2530453346 conn(0x7fb1d80fbce0 msgr2=0x7fb1d80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- 172.21.15.64:0/2530453346 shutdown_connections 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- 172.21.15.64:0/2530453346 wait complete. 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 Processor -- start 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- start start 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8101960 0x7fb1d8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8102320 0x7fb1d819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d810d3c0 con 0x7fb1d8100770 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d810d240 con 0x7fb1d8101960 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.895+0000 7fb1df9f9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d810d540 con 0x7fb1d8102320 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38958/0 (socket says 172.21.15.64:38958) 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 -- 172.21.15.64:0/3252562926 learned_addr learned my addr 172.21.15.64:0/3252562926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:07.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dcf94640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8101960 0x7fb1d8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1ddf96640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8102320 0x7fb1d819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 -- 172.21.15.64:0/3252562926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8102320 msgr2=0x7fb1d819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8102320 0x7fb1d819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 -- 172.21.15.64:0/3252562926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8101960 msgr2=0x7fb1d8198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8101960 0x7fb1d8198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 -- 172.21.15.64:0/3252562926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d819f770 con 0x7fb1d8100770 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1ddf96640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8102320 0x7fb1d819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dcf94640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8101960 0x7fb1d8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1dd795640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8197b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40314e0 tx=0x7fb1d40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d403f030 con 0x7fb1d8100770 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1d4003cb0 con 0x7fb1d8100770 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d819fa00 con 0x7fb1d8100770 2024-05-09T09:09:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4035070 con 0x7fb1d8100770 2024-05-09T09:09:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.896+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d819fef0 con 0x7fb1d8100770 2024-05-09T09:09:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.897+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d8107c50 con 0x7fb1d8100770 2024-05-09T09:09:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.898+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1d4042030 con 0x7fb1d8100770 2024-05-09T09:09:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.899+0000 7fb1ce7fc640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1c40a2b90 0x7fb1c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.900+0000 7fb1dcf94640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1c40a2b90 0x7fb1c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.900+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb1d40efec0 con 0x7fb1d8100770 2024-05-09T09:09:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.900+0000 7fb1dcf94640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1c40a2b90 0x7fb1c40a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80058c0 tx=0x7fb1c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:07.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:07.903+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1d40bca90 con 0x7fb1d8100770 2024-05-09T09:09:08.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.092+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1d802d1c0 con 0x7fb1d8100770 2024-05-09T09:09:08.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.092+0000 7fb1ce7fc640 1 -- 172.21.15.64:0/3252562926 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fb1d40bc390 con 0x7fb1d8100770 2024-05-09T09:09:08.091 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:09:08.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1c40a2b90 msgr2=0x7fb1c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1c40a2b90 0x7fb1c40a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80058c0 tx=0x7fb1c8005850 comp rx=0 tx=0).stop 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 msgr2=0x7fb1d8197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8197b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40314e0 tx=0x7fb1d40042e0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 shutdown_connections 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1c40a2b90 0x7fb1c40a5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1d8102320 0x7fb1d819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d8101960 0x7fb1d8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 --2- 172.21.15.64:0/3252562926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d8100770 0x7fb1d8197b50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.094+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 >> 172.21.15.64:0/3252562926 conn(0x7fb1d80fbce0 msgr2=0x7fb1d8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.095+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 shutdown_connections 2024-05-09T09:09:08.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.095+0000 7fb1df9f9640 1 -- 172.21.15.64:0/3252562926 wait complete. 2024-05-09T09:09:08.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1347026093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:08.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/265862060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:08.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2624750562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:08.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 -- 172.21.15.64:0/3924354301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc103440 msgr2=0x7f32dc109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 --2- 172.21.15.64:0/3924354301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc103440 0x7f32dc109cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009ca0 tx=0x7f32cc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 -- 172.21.15.64:0/3924354301 shutdown_connections 2024-05-09T09:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 --2- 172.21.15.64:0/3924354301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc103440 0x7f32dc109cd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 --2- 172.21.15.64:0/3924354301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32dc102a80 0x7f32dc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 --2- 172.21.15.64:0/3924354301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32dc101890 0x7f32dc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 -- 172.21.15.64:0/3924354301 >> 172.21.15.64:0/3924354301 conn(0x7f32dc0fcde0 msgr2=0x7f32dc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 -- 172.21.15.64:0/3924354301 shutdown_connections 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.215+0000 7f32e3721640 1 -- 172.21.15.64:0/3924354301 wait complete. 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 Processor -- start 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 -- start start 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32dc101890 0x7f32dc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 0x7f32dc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32dc103440 0x7f32dc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc10d3a0 con 0x7f32dc102a80 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc10d220 con 0x7f32dc101890 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e3721640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc10d520 con 0x7f32dc103440 2024-05-09T09:09:08.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 0x7f32dc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 0x7f32dc197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38976/0 (socket says 172.21.15.64:38976) 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 -- 172.21.15.64:0/2527862161 learned_addr learned my addr 172.21.15.64:0/2527862161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e14bd640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32dc101890 0x7f32dc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 -- 172.21.15.64:0/2527862161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32dc103440 msgr2=0x7f32dc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32dc103440 0x7f32dc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 -- 172.21.15.64:0/2527862161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32dc101890 msgr2=0x7f32dc197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32dc101890 0x7f32dc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.216+0000 7f32e0cbc640 1 -- 172.21.15.64:0/2527862161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc19f680 con 0x7f32dc102a80 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.217+0000 7f32e0cbc640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 0x7f32dc197fa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d950 tx=0x7f32d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.217+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0017030 con 0x7f32dc102a80 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.217+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d0004020 con 0x7f32dc102a80 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.217+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc19f970 con 0x7f32dc102a80 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.217+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d00047a0 con 0x7f32dc102a80 2024-05-09T09:09:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.217+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc19ff10 con 0x7f32dc102a80 2024-05-09T09:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.218+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc108bb0 con 0x7f32dc102a80 2024-05-09T09:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.219+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32d00041d0 con 0x7f32dc102a80 2024-05-09T09:09:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.220+0000 7f32da7fc640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f32c40a2c10 0x7f32c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.220+0000 7f32e14bd640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f32c40a2c10 0x7f32c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.220+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f32d00cee70 con 0x7f32dc102a80 2024-05-09T09:09:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.220+0000 7f32e14bd640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f32c40a2c10 0x7f32c40a50d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32d4005fd0 tx=0x7f32d4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:08.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.224+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32d009ba40 con 0x7f32dc102a80 2024-05-09T09:09:08.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1347026093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:08.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/265862060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2624750562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:08.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1347026093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:08.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/265862060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:08.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2624750562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.407+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f32dc02d1c0 con 0x7f32dc102a80 2024-05-09T09:09:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.408+0000 7f32da7fc640 1 -- 172.21.15.64:0/2527862161 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f32d009b340 con 0x7f32dc102a80 2024-05-09T09:09:08.407 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f32c40a2c10 msgr2=0x7f32c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f32c40a2c10 0x7f32c40a50d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32d4005fd0 tx=0x7f32d4005850 comp rx=0 tx=0).stop 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 msgr2=0x7f32dc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 0x7f32dc197fa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d950 tx=0x7f32d000de10 comp rx=0 tx=0).stop 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 shutdown_connections 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f32c40a2c10 0x7f32c40a50d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32dc103440 0x7f32dc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32dc102a80 0x7f32dc197fa0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 --2- 172.21.15.64:0/2527862161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32dc101890 0x7f32dc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 >> 172.21.15.64:0/2527862161 conn(0x7f32dc0fcde0 msgr2=0x7f32dc105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 shutdown_connections 2024-05-09T09:09:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.410+0000 7f32e3721640 1 -- 172.21.15.64:0/2527862161 wait complete. 2024-05-09T09:09:08.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:09:08.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 -- 172.21.15.64:0/2157174657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8100790 msgr2=0x7f3ca8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 --2- 172.21.15.64:0/2157174657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8100790 0x7f3ca8100bb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c9802a410 comp rx=0 tx=0).stop 2024-05-09T09:09:08.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 -- 172.21.15.64:0/2157174657 shutdown_connections 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 --2- 172.21.15.64:0/2157174657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca8102340 0x7f3ca8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 --2- 172.21.15.64:0/2157174657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8101980 0x7f3ca8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 --2- 172.21.15.64:0/2157174657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8100790 0x7f3ca8100bb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 -- 172.21.15.64:0/2157174657 >> 172.21.15.64:0/2157174657 conn(0x7f3ca80fbd20 msgr2=0x7f3ca80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 -- 172.21.15.64:0/2157174657 shutdown_connections 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.530+0000 7f3cad729640 1 -- 172.21.15.64:0/2157174657 wait complete. 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 Processor -- start 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 -- start start 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca8100790 0x7f3ca81046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 0x7f3ca8102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8102340 0x7f3ca81032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca810c330 con 0x7f3ca8101980 2024-05-09T09:09:08.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca810c1b0 con 0x7f3ca8100790 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3cad729640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca810c4b0 con 0x7f3ca8102340 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3ca67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 0x7f3ca8102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3ca67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 0x7f3ca8102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38994/0 (socket says 172.21.15.64:38994) 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3ca67fc640 1 -- 172.21.15.64:0/380946568 learned_addr learned my addr 172.21.15.64:0/380946568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3ca77fe640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8102340 0x7f3ca81032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.531+0000 7f3ca6ffd640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca8100790 0x7f3ca81046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca67fc640 1 -- 172.21.15.64:0/380946568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8102340 msgr2=0x7f3ca81032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca67fc640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8102340 0x7f3ca81032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca67fc640 1 -- 172.21.15.64:0/380946568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca8100790 msgr2=0x7f3ca81046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca67fc640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca8100790 0x7f3ca81046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca67fc640 1 -- 172.21.15.64:0/380946568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca8103b60 con 0x7f3ca8101980 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca77fe640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8102340 0x7f3ca81032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:08.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3ca67fc640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 0x7f3ca8102d60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00d770 tx=0x7f3c9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c017070 con 0x7f3ca8101980 2024-05-09T09:09:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca81a1d30 con 0x7f3ca8101980 2024-05-09T09:09:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c9c0054d0 con 0x7f3ca8101980 2024-05-09T09:09:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c004cf0 con 0x7f3ca8101980 2024-05-09T09:09:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.532+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81a2220 con 0x7f3ca8101980 2024-05-09T09:09:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.533+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca81a2450 con 0x7f3ca8101980 2024-05-09T09:09:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.534+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c9c004e90 con 0x7f3ca8101980 2024-05-09T09:09:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.535+0000 7f3c8ffff640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c900a2b90 0x7f3c900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.535+0000 7f3ca6ffd640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c900a2b90 0x7f3c900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.535+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3c9c0ceef0 con 0x7f3ca8101980 2024-05-09T09:09:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.535+0000 7f3ca6ffd640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c900a2b90 0x7f3c900a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c980314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:08.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.539+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3c9c09bac0 con 0x7f3ca8101980 2024-05-09T09:09:08.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.724+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ca81a27e0 con 0x7f3ca8101980 2024-05-09T09:09:08.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.725+0000 7f3c8ffff640 1 -- 172.21.15.64:0/380946568 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f3c9c09b3c0 con 0x7f3ca8101980 2024-05-09T09:09:08.724 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.726+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c900a2b90 msgr2=0x7f3c900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.726+0000 7f3cad729640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c900a2b90 0x7f3c900a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c980314a0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.726+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 msgr2=0x7f3ca8102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.726+0000 7f3cad729640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 0x7f3ca8102d60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00d770 tx=0x7f3c9c00dc30 comp rx=0 tx=0).stop 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 shutdown_connections 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c900a2b90 0x7f3c900a5050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8102340 0x7f3ca81032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ca8101980 0x7f3ca8102d60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 --2- 172.21.15.64:0/380946568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca8100790 0x7f3ca81046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 >> 172.21.15.64:0/380946568 conn(0x7f3ca80fbd20 msgr2=0x7f3ca8069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 shutdown_connections 2024-05-09T09:09:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.727+0000 7f3cad729640 1 -- 172.21.15.64:0/380946568 wait complete. 2024-05-09T09:09:08.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:09:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 -- 172.21.15.64:0/1431079704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 msgr2=0x7fb108101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/1431079704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108101de0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009900 tx=0x7fb0fc02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 -- 172.21.15.64:0/1431079704 shutdown_connections 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/1431079704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb108102320 0x7fb108108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/1431079704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108101de0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/1431079704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb108100770 0x7fb108100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 -- 172.21.15.64:0/1431079704 >> 172.21.15.64:0/1431079704 conn(0x7fb1080fbce0 msgr2=0x7fb1080fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 -- 172.21.15.64:0/1431079704 shutdown_connections 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.845+0000 7fb10dbb7640 1 -- 172.21.15.64:0/1431079704 wait complete. 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 Processor -- start 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 -- start start 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb108100770 0x7fb108104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb108102320 0x7fb108103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb10810c2a0 con 0x7fb108101960 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb10810c120 con 0x7fb108100770 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb10dbb7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb10810c420 con 0x7fb108102320 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39014/0 (socket says 172.21.15.64:39014) 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb106ffd640 1 -- 172.21.15.64:0/2827123871 learned_addr learned my addr 172.21.15.64:0/2827123871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.846+0000 7fb107fff640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb108102320 0x7fb108103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb1077fe640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb108100770 0x7fb108104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb106ffd640 1 -- 172.21.15.64:0/2827123871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb108102320 msgr2=0x7fb108103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb106ffd640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb108102320 0x7fb108103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb106ffd640 1 -- 172.21.15.64:0/2827123871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb108100770 msgr2=0x7fb108104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb106ffd640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb108100770 0x7fb108104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb106ffd640 1 -- 172.21.15.64:0/2827123871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb108103b10 con 0x7fb108101960 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb106ffd640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108102d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc005940 tx=0x7fb0fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc03f030 con 0x7fb108101960 2024-05-09T09:09:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0fc0318f0 con 0x7fb108101960 2024-05-09T09:09:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1081a1be0 con 0x7fb108101960 2024-05-09T09:09:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc035070 con 0x7fb108101960 2024-05-09T09:09:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.847+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1081a2130 con 0x7fb108101960 2024-05-09T09:09:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.848+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1081a2300 con 0x7fb108101960 2024-05-09T09:09:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.849+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0fc042030 con 0x7fb108101960 2024-05-09T09:09:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.850+0000 7fb104ff9640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0f00a2b90 0x7fb0f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.850+0000 7fb1077fe640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0f00a2b90 0x7fb0f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.850+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0fc0ef860 con 0x7fb108101960 2024-05-09T09:09:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.851+0000 7fb1077fe640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0f00a2b90 0x7fb0f00a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1081017c0 tx=0x7fb0f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:08.854+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb0fc0bc430 con 0x7fb108101960 2024-05-09T09:09:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.038+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb1081a24d0 con 0x7fb108101960 2024-05-09T09:09:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.039+0000 7fb104ff9640 1 -- 172.21.15.64:0/2827123871 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fb0fc0bbd30 con 0x7fb108101960 2024-05-09T09:09:09.038 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:09:09.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.040+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0f00a2b90 msgr2=0x7fb0f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.040+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0f00a2b90 0x7fb0f00a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1081017c0 tx=0x7fb0f8005f90 comp rx=0 tx=0).stop 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.040+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 msgr2=0x7fb108102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.040+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108102d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc005940 tx=0x7fb0fc0042f0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 shutdown_connections 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0f00a2b90 0x7fb0f00a5050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb108102320 0x7fb108103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108101960 0x7fb108102d40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 --2- 172.21.15.64:0/2827123871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb108100770 0x7fb108104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 >> 172.21.15.64:0/2827123871 conn(0x7fb1080fbce0 msgr2=0x7fb1080689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 shutdown_connections 2024-05-09T09:09:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.041+0000 7fb10dbb7640 1 -- 172.21.15.64:0/2827123871 wait complete. 2024-05-09T09:09:09.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:08 smithi064 ceph-mon[30149]: pgmap v370: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:09:09.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:08 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3252562926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:08 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2527862161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:08 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/380946568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.051 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T09:09:09.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 -- 172.21.15.64:0/4208360238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 msgr2=0x7f74d4108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 --2- 172.21.15.64:0/4208360238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d4108bd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009ca0 tx=0x7f74cc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 -- 172.21.15.64:0/4208360238 shutdown_connections 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 --2- 172.21.15.64:0/4208360238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d4108bd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 --2- 172.21.15.64:0/4208360238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4101980 0x7f74d4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 --2- 172.21.15.64:0/4208360238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d4100790 0x7f74d4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 -- 172.21.15.64:0/4208360238 >> 172.21.15.64:0/4208360238 conn(0x7f74d40fbd20 msgr2=0x7f74d40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 -- 172.21.15.64:0/4208360238 shutdown_connections 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.157+0000 7f74dbce1640 1 -- 172.21.15.64:0/4208360238 wait complete. 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 Processor -- start 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 -- start start 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4100790 0x7f74d4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d4101980 0x7f74d4196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f74d410c340 con 0x7f74d4102340 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f74d410c1c0 con 0x7f74d4100790 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74dbce1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74d410c4c0 con 0x7f74d4101980 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74da27e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74da27e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39028/0 (socket says 172.21.15.64:39028) 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74da27e640 1 -- 172.21.15.64:0/1029703056 learned_addr learned my addr 172.21.15.64:0/1029703056 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74d9a7d640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4100790 0x7f74d4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.158+0000 7f74d927c640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d4101980 0x7f74d4196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74da27e640 1 -- 172.21.15.64:0/1029703056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d4101980 msgr2=0x7f74d4196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74da27e640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d4101980 0x7f74d4196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74da27e640 1 -- 172.21.15.64:0/1029703056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4100790 msgr2=0x7f74d4196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74da27e640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4100790 0x7f74d4196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74da27e640 1 -- 172.21.15.64:0/1029703056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d419e630 con 0x7f74d4102340 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74d9a7d640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4100790 0x7f74d4196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74da27e640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d419dfc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009dc0 tx=0x7f74cc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc032ec0 con 0x7f74d4102340 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74cc0044f0 con 0x7f74d4102340 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d419e8c0 con 0x7f74d4102340 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc030d40 con 0x7f74d4102340 2024-05-09T09:09:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.159+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d419ee10 con 0x7f74d4102340 2024-05-09T09:09:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.160+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d4107b30 con 0x7f74d4102340 2024-05-09T09:09:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.161+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f74cc004040 con 0x7f74d4102340 2024-05-09T09:09:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.162+0000 7f74caffd640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74c00a2d30 0x7f74c00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.162+0000 7f74d9a7d640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74c00a2d30 0x7f74c00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.162+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f74cc0f0b30 con 0x7f74d4102340 2024-05-09T09:09:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.162+0000 7f74d9a7d640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74c00a2d30 0x7f74c00a51f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058c0 tx=0x7f74d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.166+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f74cc0bd700 con 0x7f74d4102340 2024-05-09T09:09:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:08 smithi179 ceph-mon[34086]: pgmap v370: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:09:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:08 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3252562926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:08 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2527862161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:08 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/380946568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:08 smithi139 ceph-mon[33599]: pgmap v370: 129 pgs: 129 active+clean; 640 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:09:09.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:08 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3252562926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:08 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2527862161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:08 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/380946568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.389+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74d4062430 con 0x7f74d4102340 2024-05-09T09:09:09.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.390+0000 7f74caffd640 1 -- 172.21.15.64:0/1029703056 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7f74cc0dd0c0 con 0x7f74d4102340 2024-05-09T09:09:09.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:09.389 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:09:04.976828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:09:09.389 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-05-09T09:09:09.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74c00a2d30 msgr2=0x7f74c00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74c00a2d30 0x7f74c00a51f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058c0 tx=0x7f74d0005850 comp rx=0 tx=0).stop 2024-05-09T09:09:09.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 msgr2=0x7f74d419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d419dfc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009dc0 tx=0x7f74cc003ba0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 shutdown_connections 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f74c00a2d30 0x7f74c00a51f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74d4102340 0x7f74d419dfc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d4101980 0x7f74d4196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 --2- 172.21.15.64:0/1029703056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74d4100790 0x7f74d4196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 >> 172.21.15.64:0/1029703056 conn(0x7f74d40fbd20 msgr2=0x7f74d4104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 shutdown_connections 2024-05-09T09:09:09.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.392+0000 7f74dbce1640 1 -- 172.21.15.64:0/1029703056 wait complete. 2024-05-09T09:09:09.402 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T09:09:09.403 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T09:09:09.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 -- 172.21.15.64:0/2995430975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 msgr2=0x7fe358100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/2995430975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358100b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe348009900 tx=0x7fe34802a410 comp rx=0 tx=0).stop 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 -- 172.21.15.64:0/2995430975 shutdown_connections 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/2995430975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 0x7fe358108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/2995430975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 0x7fe358101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/2995430975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358100b90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 -- 172.21.15.64:0/2995430975 >> 172.21.15.64:0/2995430975 conn(0x7fe3580fbce0 msgr2=0x7fe3580fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 -- 172.21.15.64:0/2995430975 shutdown_connections 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 -- 172.21.15.64:0/2995430975 wait complete. 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 Processor -- start 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 -- start start 2024-05-09T09:09:09.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.509+0000 7fe35d8a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 0x7fe35819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 0x7fe35819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe35810c310 con 0x7fe358100770 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe35810c190 con 0x7fe358102320 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe35810c490 con 0x7fe358101960 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39052/0 (socket says 172.21.15.64:39052) 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 -- 172.21.15.64:0/905817837 learned_addr learned my addr 172.21.15.64:0/905817837 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe34ffff640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 0x7fe35819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe3577fe640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 0x7fe35819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 -- 172.21.15.64:0/905817837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 msgr2=0x7fe35819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 0x7fe35819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 -- 172.21.15.64:0/905817837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 msgr2=0x7fe35819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 0x7fe35819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 -- 172.21.15.64:0/905817837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35819f810 con 0x7fe358100770 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe34ffff640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 0x7fe35819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe3577fe640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 0x7fe35819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe356ffd640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358069530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe3480314e0 tx=0x7fe3480042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34803f030 con 0x7fe358100770 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35819fa40 con 0x7fe358100770 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe348003cb0 con 0x7fe358100770 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348035070 con 0x7fe358100770 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.510+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35819ff80 con 0x7fe358100770 2024-05-09T09:09:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.511+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe358107b80 con 0x7fe358100770 2024-05-09T09:09:09.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.512+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe348042030 con 0x7fe358100770 2024-05-09T09:09:09.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.513+0000 7fe35c8a5640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3440a2b90 0x7fe3440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.513+0000 7fe34ffff640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3440a2b90 0x7fe3440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.513+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe3480efac0 con 0x7fe358100770 2024-05-09T09:09:09.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.514+0000 7fe34ffff640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3440a2b90 0x7fe3440a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe3400058c0 tx=0x7fe340005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:09.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.518+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe3480bc690 con 0x7fe358100770 2024-05-09T09:09:09.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.700+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe358061c70 con 0x7fe3440a2b90 2024-05-09T09:09:09.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.710+0000 7fe35c8a5640 1 -- 172.21.15.64:0/905817837 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7fe358061c70 con 0x7fe3440a2b90 2024-05-09T09:09:09.713 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:09.715 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3440a2b90 msgr2=0x7fe3440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3440a2b90 0x7fe3440a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe3400058c0 tx=0x7fe340005850 comp rx=0 tx=0).stop 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 msgr2=0x7fe358069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358069530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe3480314e0 tx=0x7fe3480042e0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 shutdown_connections 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3440a2b90 0x7fe3440a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe358102320 0x7fe35819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe358101960 0x7fe35819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 --2- 172.21.15.64:0/905817837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe358100770 0x7fe358069530 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 >> 172.21.15.64:0/905817837 conn(0x7fe3580fbce0 msgr2=0x7fe358104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 shutdown_connections 2024-05-09T09:09:09.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.714+0000 7fe35d8a7640 1 -- 172.21.15.64:0/905817837 wait complete. 2024-05-09T09:09:09.724 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-05-09T09:09:09.135663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661396,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":875,"num_read_kb":558,"num_write":356,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392896,"kb_used_data":14580,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548928,"statfs":{"total":1151940427776,"available":1151538102272,"internally_reserved":0,"allocated":14929920,"data_stored":8227007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5745,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003023"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672739+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:08:37.672739+0000","last_peered":"2024-05-09T09:08:37.672739+0000","last_clean":"2024-05-09T09:08:37.672739+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:08:37.672739+0000","last_undegraded":"2024-05-09T09:08:37.672739+0000","last_fullsized":"2024-05-09T09:08:37.672739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939357+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:08:37.939357+0000","last_peered":"2024-05-09T09:08:37.939357+0000","last_clean":"2024-05-09T09:08:37.939357+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:08:37.939357+0000","last_undegraded":"2024-05-09T09:08:37.939357+0000","last_fullsized":"2024-05-09T09:08:37.939357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939211+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:08:37.939211+0000","last_peered":"2024-05-09T09:08:37.939211+0000","last_clean":"2024-05-09T09:08:37.939211+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:08:37.939211+0000","last_undegraded":"2024-05-09T09:08:37.939211+0000","last_fullsized":"2024-05-09T09:08:37.939211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704349+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:08:36.704349+0000","last_peered":"2024-05-09T09:08:36.704349+0000","last_clean":"2024-05-09T09:08:36.704349+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:08:36.704349+0000","last_undegraded":"2024-05-09T09:08:36.704349+0000","last_fullsized":"2024-05-09T09:08:36.704349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579167+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:08:37.579167+0000","last_peered":"2024-05-09T09:08:37.579167+0000","last_clean":"2024-05-09T09:08:37.579167+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:08:37.579167+0000","last_undegraded":"2024-05-09T09:08:37.579167+0000","last_fullsized":"2024-05-09T09:08:37.579167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846040+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:08:36.846040+0000","last_peered":"2024-05-09T09:08:36.846040+0000","last_clean":"2024-05-09T09:08:36.846040+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:08:36.846040+0000","last_undegraded":"2024-05-09T09:08:36.846040+0000","last_fullsized":"2024-05-09T09:08:36.846040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708968+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:08:37.708968+0000","last_peered":"2024-05-09T09:08:37.708968+0000","last_clean":"2024-05-09T09:08:37.708968+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:08:37.708968+0000","last_undegraded":"2024-05-09T09:08:37.708968+0000","last_fullsized":"2024-05-09T09:08:37.708968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706275+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:08:36.706275+0000","last_peered":"2024-05-09T09:08:36.706275+0000","last_clean":"2024-05-09T09:08:36.706275+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:08:36.706275+0000","last_undegraded":"2024-05-09T09:08:36.706275+0000","last_fullsized":"2024-05-09T09:08:36.706275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124251+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:08:37.124251+0000","last_peered":"2024-05-09T09:08:37.124251+0000","last_clean":"2024-05-09T09:08:37.124251+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:08:37.124251+0000","last_undegraded":"2024-05-09T09:08:37.124251+0000","last_fullsized":"2024-05-09T09:08:37.124251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846387+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:08:36.846387+0000","last_peered":"2024-05-09T09:08:36.846387+0000","last_clean":"2024-05-09T09:08:36.846387+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:08:36.846387+0000","last_undegraded":"2024-05-09T09:08:36.846387+0000","last_fullsized":"2024-05-09T09:08:36.846387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709714+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:08:37.709714+0000","last_peered":"2024-05-09T09:08:37.709714+0000","last_clean":"2024-05-09T09:08:37.709714+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:08:37.709714+0000","last_undegraded":"2024-05-09T09:08:37.709714+0000","last_fullsized":"2024-05-09T09:08:37.709714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706384+0000","last_change":"2024-05-09T09:07:36.317979+0000","last_active":"2024-05-09T09:08:36.706384+0000","last_peered":"2024-05-09T09:08:36.706384+0000","last_clean":"2024-05-09T09:08:36.706384+0000","last_became_active":"2024-05-09T09:07:36.317572+0000","last_became_peered":"2024-05-09T09:07:36.317572+0000","last_unstale":"2024-05-09T09:08:36.706384+0000","last_undegraded":"2024-05-09T09:08:36.706384+0000","last_fullsized":"2024-05-09T09:08:36.706384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:45.499023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919161+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:08:36.919161+0000","last_peered":"2024-05-09T09:08:36.919161+0000","last_clean":"2024-05-09T09:08:36.919161+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:08:36.919161+0000","last_undegraded":"2024-05-09T09:08:36.919161+0000","last_fullsized":"2024-05-09T09:08:36.919161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938799+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:08:37.938799+0000","last_peered":"2024-05-09T09:08:37.938799+0000","last_clean":"2024-05-09T09:08:37.938799+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:08:37.938799+0000","last_undegraded":"2024-05-09T09:08:37.938799+0000","last_fullsized":"2024-05-09T09:08:37.938799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938909+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:08:37.938909+0000","last_peered":"2024-05-09T09:08:37.938909+0000","last_clean":"2024-05-09T09:08:37.938909+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:08:37.938909+0000","last_undegraded":"2024-05-09T09:08:37.938909+0000","last_fullsized":"2024-05-09T09:08:37.938909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123822+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:08:37.123822+0000","last_peered":"2024-05-09T09:08:37.123822+0000","last_clean":"2024-05-09T09:08:37.123822+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:08:37.123822+0000","last_undegraded":"2024-05-09T09:08:37.123822+0000","last_fullsized":"2024-05-09T09:08:37.123822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672730+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:08:37.672730+0000","last_peered":"2024-05-09T09:08:37.672730+0000","last_clean":"2024-05-09T09:08:37.672730+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:08:37.672730+0000","last_undegraded":"2024-05-09T09:08:37.672730+0000","last_fullsized":"2024-05-09T09:08:37.672730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706528+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:08:36.706528+0000","last_peered":"2024-05-09T09:08:36.706528+0000","last_clean":"2024-05-09T09:08:36.706528+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:08:36.706528+0000","last_undegraded":"2024-05-09T09:08:36.706528+0000","last_fullsized":"2024-05-09T09:08:36.706528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704353+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:08:36.704353+0000","last_peered":"2024-05-09T09:08:36.704353+0000","last_clean":"2024-05-09T09:08:36.704353+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:08:36.704353+0000","last_undegraded":"2024-05-09T09:08:36.704353+0000","last_fullsized":"2024-05-09T09:08:36.704353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703943+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:08:36.703943+0000","last_peered":"2024-05-09T09:08:36.703943+0000","last_clean":"2024-05-09T09:08:36.703943+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:08:36.703943+0000","last_undegraded":"2024-05-09T09:08:36.703943+0000","last_fullsized":"2024-05-09T09:08:36.703943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706328+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:08:36.706328+0000","last_peered":"2024-05-09T09:08:36.706328+0000","last_clean":"2024-05-09T09:08:36.706328+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:08:36.706328+0000","last_undegraded":"2024-05-09T09:08:36.706328+0000","last_fullsized":"2024-05-09T09:08:36.706328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.710323+0000","last_change":"2024-05-09T09:08:36.710323+0000","last_active":"2024-05-09T09:08:36.710323+0000","last_peered":"2024-05-09T09:08:36.710323+0000","last_clean":"2024-05-09T09:08:36.710323+0000","last_became_active":"2024-05-09T09:08:36.709965+0000","last_became_peered":"2024-05-09T09:08:36.709965+0000","last_unstale":"2024-05-09T09:08:36.710323+0000","last_undegraded":"2024-05-09T09:08:36.710323+0000","last_fullsized":"2024-05-09T09:08:36.710323+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:01.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703719+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:08:36.703719+0000","last_peered":"2024-05-09T09:08:36.703719+0000","last_clean":"2024-05-09T09:08:36.703719+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:08:36.703719+0000","last_undegraded":"2024-05-09T09:08:36.703719+0000","last_fullsized":"2024-05-09T09:08:36.703719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707362+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:08:36.707362+0000","last_peered":"2024-05-09T09:08:36.707362+0000","last_clean":"2024-05-09T09:08:36.707362+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:08:36.707362+0000","last_undegraded":"2024-05-09T09:08:36.707362+0000","last_fullsized":"2024-05-09T09:08:36.707362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939378+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:08:37.939378+0000","last_peered":"2024-05-09T09:08:37.939378+0000","last_clean":"2024-05-09T09:08:37.939378+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:08:37.939378+0000","last_undegraded":"2024-05-09T09:08:37.939378+0000","last_fullsized":"2024-05-09T09:08:37.939378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703834+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:08:36.703834+0000","last_peered":"2024-05-09T09:08:36.703834+0000","last_clean":"2024-05-09T09:08:36.703834+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:08:36.703834+0000","last_undegraded":"2024-05-09T09:08:36.703834+0000","last_fullsized":"2024-05-09T09:08:36.703834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579531+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:08:37.579531+0000","last_peered":"2024-05-09T09:08:37.579531+0000","last_clean":"2024-05-09T09:08:37.579531+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:08:37.579531+0000","last_undegraded":"2024-05-09T09:08:37.579531+0000","last_fullsized":"2024-05-09T09:08:37.579531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123893+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:08:37.123893+0000","last_peered":"2024-05-09T09:08:37.123893+0000","last_clean":"2024-05-09T09:08:37.123893+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:08:37.123893+0000","last_undegraded":"2024-05-09T09:08:37.123893+0000","last_fullsized":"2024-05-09T09:08:37.123893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703725+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:08:36.703725+0000","last_peered":"2024-05-09T09:08:36.703725+0000","last_clean":"2024-05-09T09:08:36.703725+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:08:36.703725+0000","last_undegraded":"2024-05-09T09:08:36.703725+0000","last_fullsized":"2024-05-09T09:08:36.703725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579279+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:08:37.579279+0000","last_peered":"2024-05-09T09:08:37.579279+0000","last_clean":"2024-05-09T09:08:37.579279+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:08:37.579279+0000","last_undegraded":"2024-05-09T09:08:37.579279+0000","last_fullsized":"2024-05-09T09:08:37.579279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708974+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:08:36.708974+0000","last_peered":"2024-05-09T09:08:36.708974+0000","last_clean":"2024-05-09T09:08:36.708974+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:08:36.708974+0000","last_undegraded":"2024-05-09T09:08:36.708974+0000","last_fullsized":"2024-05-09T09:08:36.708974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704172+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:08:36.704172+0000","last_peered":"2024-05-09T09:08:36.704172+0000","last_clean":"2024-05-09T09:08:36.704172+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:08:36.704172+0000","last_undegraded":"2024-05-09T09:08:36.704172+0000","last_fullsized":"2024-05-09T09:08:36.704172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:06.410475+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:09:06.410475+0000","last_peered":"2024-05-09T09:09:06.410475+0000","last_clean":"2024-05-09T09:09:06.410475+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:09:06.410475+0000","last_undegraded":"2024-05-09T09:09:06.410475+0000","last_fullsized":"2024-05-09T09:09:06.410475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1099,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579170+0000","last_change":"2024-05-09T09:07:36.615226+0000","last_active":"2024-05-09T09:08:37.579170+0000","last_peered":"2024-05-09T09:08:37.579170+0000","last_clean":"2024-05-09T09:08:37.579170+0000","last_became_active":"2024-05-09T09:07:36.614762+0000","last_became_peered":"2024-05-09T09:07:36.614762+0000","last_unstale":"2024-05-09T09:08:37.579170+0000","last_undegraded":"2024-05-09T09:08:37.579170+0000","last_fullsized":"2024-05-09T09:08:37.579170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:29.045379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704747+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:08:36.704747+0000","last_peered":"2024-05-09T09:08:36.704747+0000","last_clean":"2024-05-09T09:08:36.704747+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:08:36.704747+0000","last_undegraded":"2024-05-09T09:08:36.704747+0000","last_fullsized":"2024-05-09T09:08:36.704747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.845978+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:08:36.845978+0000","last_peered":"2024-05-09T09:08:36.845978+0000","last_clean":"2024-05-09T09:08:36.845978+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:08:36.845978+0000","last_undegraded":"2024-05-09T09:08:36.845978+0000","last_fullsized":"2024-05-09T09:08:36.845978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709278+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:08:36.709278+0000","last_peered":"2024-05-09T09:08:36.709278+0000","last_clean":"2024-05-09T09:08:36.709278+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:08:36.709278+0000","last_undegraded":"2024-05-09T09:08:36.709278+0000","last_fullsized":"2024-05-09T09:08:36.709278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672121+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:08:37.672121+0000","last_peered":"2024-05-09T09:08:37.672121+0000","last_clean":"2024-05-09T09:08:37.672121+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:08:37.672121+0000","last_undegraded":"2024-05-09T09:08:37.672121+0000","last_fullsized":"2024-05-09T09:08:37.672121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.918958+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:08:36.918958+0000","last_peered":"2024-05-09T09:08:36.918958+0000","last_clean":"2024-05-09T09:08:36.918958+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:08:36.918958+0000","last_undegraded":"2024-05-09T09:08:36.918958+0000","last_fullsized":"2024-05-09T09:08:36.918958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705457+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:08:36.705457+0000","last_peered":"2024-05-09T09:08:36.705457+0000","last_clean":"2024-05-09T09:08:36.705457+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:08:36.705457+0000","last_undegraded":"2024-05-09T09:08:36.705457+0000","last_fullsized":"2024-05-09T09:08:36.705457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938610+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:08:37.938610+0000","last_peered":"2024-05-09T09:08:37.938610+0000","last_clean":"2024-05-09T09:08:37.938610+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:08:37.938610+0000","last_undegraded":"2024-05-09T09:08:37.938610+0000","last_fullsized":"2024-05-09T09:08:37.938610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708907+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:08:36.708907+0000","last_peered":"2024-05-09T09:08:36.708907+0000","last_clean":"2024-05-09T09:08:36.708907+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:08:36.708907+0000","last_undegraded":"2024-05-09T09:08:36.708907+0000","last_fullsized":"2024-05-09T09:08:36.708907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705674+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:08:36.705674+0000","last_peered":"2024-05-09T09:08:36.705674+0000","last_clean":"2024-05-09T09:08:36.705674+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:08:36.705674+0000","last_undegraded":"2024-05-09T09:08:36.705674+0000","last_fullsized":"2024-05-09T09:08:36.705674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706934+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:08:36.706934+0000","last_peered":"2024-05-09T09:08:36.706934+0000","last_clean":"2024-05-09T09:08:36.706934+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:08:36.706934+0000","last_undegraded":"2024-05-09T09:08:36.706934+0000","last_fullsized":"2024-05-09T09:08:36.706934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.411229+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:09:07.411229+0000","last_peered":"2024-05-09T09:09:07.411229+0000","last_clean":"2024-05-09T09:09:07.411229+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:09:07.411229+0000","last_undegraded":"2024-05-09T09:09:07.411229+0000","last_fullsized":"2024-05-09T09:09:07.411229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124177+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:08:37.124177+0000","last_peered":"2024-05-09T09:08:37.124177+0000","last_clean":"2024-05-09T09:08:37.124177+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:08:37.124177+0000","last_undegraded":"2024-05-09T09:08:37.124177+0000","last_fullsized":"2024-05-09T09:08:37.124177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705636+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:08:36.705636+0000","last_peered":"2024-05-09T09:08:36.705636+0000","last_clean":"2024-05-09T09:08:36.705636+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:08:36.705636+0000","last_undegraded":"2024-05-09T09:08:36.705636+0000","last_fullsized":"2024-05-09T09:08:36.705636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709600+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:08:37.709600+0000","last_peered":"2024-05-09T09:08:37.709600+0000","last_clean":"2024-05-09T09:08:37.709600+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:08:37.709600+0000","last_undegraded":"2024-05-09T09:08:37.709600+0000","last_fullsized":"2024-05-09T09:08:37.709600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939354+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:08:37.939354+0000","last_peered":"2024-05-09T09:08:37.939354+0000","last_clean":"2024-05-09T09:08:37.939354+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:08:37.939354+0000","last_undegraded":"2024-05-09T09:08:37.939354+0000","last_fullsized":"2024-05-09T09:08:37.939354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704180+0000","last_change":"2024-05-09T09:07:36.436587+0000","last_active":"2024-05-09T09:08:36.704180+0000","last_peered":"2024-05-09T09:08:36.704180+0000","last_clean":"2024-05-09T09:08:36.704180+0000","last_became_active":"2024-05-09T09:07:36.436083+0000","last_became_peered":"2024-05-09T09:07:36.436083+0000","last_unstale":"2024-05-09T09:08:36.704180+0000","last_undegraded":"2024-05-09T09:08:36.704180+0000","last_fullsized":"2024-05-09T09:08:36.704180+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:02:34.904307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709115+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:08:36.709115+0000","last_peered":"2024-05-09T09:08:36.709115+0000","last_clean":"2024-05-09T09:08:36.709115+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:08:36.709115+0000","last_undegraded":"2024-05-09T09:08:36.709115+0000","last_fullsized":"2024-05-09T09:08:36.709115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124078+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:08:37.124078+0000","last_peered":"2024-05-09T09:08:37.124078+0000","last_clean":"2024-05-09T09:08:37.124078+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:08:37.124078+0000","last_undegraded":"2024-05-09T09:08:37.124078+0000","last_fullsized":"2024-05-09T09:08:37.124078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.845799+0000","last_change":"2024-05-09T09:07:36.324821+0000","last_active":"2024-05-09T09:08:36.845799+0000","last_peered":"2024-05-09T09:08:36.845799+0000","last_clean":"2024-05-09T09:08:36.845799+0000","last_became_active":"2024-05-09T09:07:36.324478+0000","last_became_peered":"2024-05-09T09:07:36.324478+0000","last_unstale":"2024-05-09T09:08:36.845799+0000","last_undegraded":"2024-05-09T09:08:36.845799+0000","last_fullsized":"2024-05-09T09:08:36.845799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:25.476000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":320,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.437201+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:09:07.437201+0000","last_peered":"2024-05-09T09:09:07.437201+0000","last_clean":"2024-05-09T09:09:07.437201+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:09:07.437201+0000","last_undegraded":"2024-05-09T09:09:07.437201+0000","last_fullsized":"2024-05-09T09:09:07.437201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66230,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579364+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:08:37.579364+0000","last_peered":"2024-05-09T09:08:37.579364+0000","last_clean":"2024-05-09T09:08:37.579364+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:08:37.579364+0000","last_undegraded":"2024-05-09T09:08:37.579364+0000","last_fullsized":"2024-05-09T09:08:37.579364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709325+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:08:36.709325+0000","last_peered":"2024-05-09T09:08:36.709325+0000","last_clean":"2024-05-09T09:08:36.709325+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:08:36.709325+0000","last_undegraded":"2024-05-09T09:08:36.709325+0000","last_fullsized":"2024-05-09T09:08:36.709325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704391+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:08:36.704391+0000","last_peered":"2024-05-09T09:08:36.704391+0000","last_clean":"2024-05-09T09:08:36.704391+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:08:36.704391+0000","last_undegraded":"2024-05-09T09:08:36.704391+0000","last_fullsized":"2024-05-09T09:08:36.704391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919138+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:08:36.919138+0000","last_peered":"2024-05-09T09:08:36.919138+0000","last_clean":"2024-05-09T09:08:36.919138+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:08:36.919138+0000","last_undegraded":"2024-05-09T09:08:36.919138+0000","last_fullsized":"2024-05-09T09:08:36.919138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672057+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:08:37.672057+0000","last_peered":"2024-05-09T09:08:37.672057+0000","last_clean":"2024-05-09T09:08:37.672057+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:08:37.672057+0000","last_undegraded":"2024-05-09T09:08:37.672057+0000","last_fullsized":"2024-05-09T09:08:37.672057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709159+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:08:36.709159+0000","last_peered":"2024-05-09T09:08:36.709159+0000","last_clean":"2024-05-09T09:08:36.709159+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:08:36.709159+0000","last_undegraded":"2024-05-09T09:08:36.709159+0000","last_fullsized":"2024-05-09T09:08:36.709159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709537+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:08:37.709537+0000","last_peered":"2024-05-09T09:08:37.709537+0000","last_clean":"2024-05-09T09:08:37.709537+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:08:37.709537+0000","last_undegraded":"2024-05-09T09:08:37.709537+0000","last_fullsized":"2024-05-09T09:08:37.709537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706621+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:08:36.706621+0000","last_peered":"2024-05-09T09:08:36.706621+0000","last_clean":"2024-05-09T09:08:36.706621+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:08:36.706621+0000","last_undegraded":"2024-05-09T09:08:36.706621+0000","last_fullsized":"2024-05-09T09:08:36.706621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.411204+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:09:07.411204+0000","last_peered":"2024-05-09T09:09:07.411204+0000","last_clean":"2024-05-09T09:09:07.411204+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:09:07.411204+0000","last_undegraded":"2024-05-09T09:09:07.411204+0000","last_fullsized":"2024-05-09T09:09:07.411204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709029+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:08:37.709029+0000","last_peered":"2024-05-09T09:08:37.709029+0000","last_clean":"2024-05-09T09:08:37.709029+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:08:37.709029+0000","last_undegraded":"2024-05-09T09:08:37.709029+0000","last_fullsized":"2024-05-09T09:08:37.709029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919195+0000","last_change":"2024-05-09T09:07:36.323604+0000","last_active":"2024-05-09T09:08:36.919195+0000","last_peered":"2024-05-09T09:08:36.919195+0000","last_clean":"2024-05-09T09:08:36.919195+0000","last_became_active":"2024-05-09T09:07:36.323254+0000","last_became_peered":"2024-05-09T09:07:36.323254+0000","last_unstale":"2024-05-09T09:08:36.919195+0000","last_undegraded":"2024-05-09T09:08:36.919195+0000","last_fullsized":"2024-05-09T09:08:36.919195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:31:18.746847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578879+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:08:37.578879+0000","last_peered":"2024-05-09T09:08:37.578879+0000","last_clean":"2024-05-09T09:08:37.578879+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:08:37.578879+0000","last_undegraded":"2024-05-09T09:08:37.578879+0000","last_fullsized":"2024-05-09T09:08:37.578879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708634+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:08:36.708634+0000","last_peered":"2024-05-09T09:08:36.708634+0000","last_clean":"2024-05-09T09:08:36.708634+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:08:36.708634+0000","last_undegraded":"2024-05-09T09:08:36.708634+0000","last_fullsized":"2024-05-09T09:08:36.708634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708641+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:08:36.708641+0000","last_peered":"2024-05-09T09:08:36.708641+0000","last_clean":"2024-05-09T09:08:36.708641+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:08:36.708641+0000","last_undegraded":"2024-05-09T09:08:36.708641+0000","last_fullsized":"2024-05-09T09:08:36.708641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672602+0000","last_change":"2024-05-09T09:07:36.331318+0000","last_active":"2024-05-09T09:08:37.672602+0000","last_peered":"2024-05-09T09:08:37.672602+0000","last_clean":"2024-05-09T09:08:37.672602+0000","last_became_active":"2024-05-09T09:07:36.323269+0000","last_became_peered":"2024-05-09T09:07:36.323269+0000","last_unstale":"2024-05-09T09:08:37.672602+0000","last_undegraded":"2024-05-09T09:08:37.672602+0000","last_fullsized":"2024-05-09T09:08:37.672602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:25:53.779933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672622+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:08:37.672622+0000","last_peered":"2024-05-09T09:08:37.672622+0000","last_clean":"2024-05-09T09:08:37.672622+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:08:37.672622+0000","last_undegraded":"2024-05-09T09:08:37.672622+0000","last_fullsized":"2024-05-09T09:08:37.672622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578766+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:08:37.578766+0000","last_peered":"2024-05-09T09:08:37.578766+0000","last_clean":"2024-05-09T09:08:37.578766+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:08:37.578766+0000","last_undegraded":"2024-05-09T09:08:37.578766+0000","last_fullsized":"2024-05-09T09:08:37.578766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706539+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:08:36.706539+0000","last_peered":"2024-05-09T09:08:36.706539+0000","last_clean":"2024-05-09T09:08:36.706539+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:08:36.706539+0000","last_undegraded":"2024-05-09T09:08:36.706539+0000","last_fullsized":"2024-05-09T09:08:36.706539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846220+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:08:36.846220+0000","last_peered":"2024-05-09T09:08:36.846220+0000","last_clean":"2024-05-09T09:08:36.846220+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:08:36.846220+0000","last_undegraded":"2024-05-09T09:08:36.846220+0000","last_fullsized":"2024-05-09T09:08:36.846220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919024+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:08:36.919024+0000","last_peered":"2024-05-09T09:08:36.919024+0000","last_clean":"2024-05-09T09:08:36.919024+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:08:36.919024+0000","last_undegraded":"2024-05-09T09:08:36.919024+0000","last_fullsized":"2024-05-09T09:08:36.919024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709369+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:08:37.709369+0000","last_peered":"2024-05-09T09:08:37.709369+0000","last_clean":"2024-05-09T09:08:37.709369+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:08:37.709369+0000","last_undegraded":"2024-05-09T09:08:37.709369+0000","last_fullsized":"2024-05-09T09:08:37.709369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846078+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:08:36.846078+0000","last_peered":"2024-05-09T09:08:36.846078+0000","last_clean":"2024-05-09T09:08:36.846078+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:08:36.846078+0000","last_undegraded":"2024-05-09T09:08:36.846078+0000","last_fullsized":"2024-05-09T09:08:36.846078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919150+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:08:36.919150+0000","last_peered":"2024-05-09T09:08:36.919150+0000","last_clean":"2024-05-09T09:08:36.919150+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:08:36.919150+0000","last_undegraded":"2024-05-09T09:08:36.919150+0000","last_fullsized":"2024-05-09T09:08:36.919150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709052+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:08:36.709052+0000","last_peered":"2024-05-09T09:08:36.709052+0000","last_clean":"2024-05-09T09:08:36.709052+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:08:36.709052+0000","last_undegraded":"2024-05-09T09:08:36.709052+0000","last_fullsized":"2024-05-09T09:08:36.709052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:05.407967+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:09:05.407967+0000","last_peered":"2024-05-09T09:09:05.407967+0000","last_clean":"2024-05-09T09:09:05.407967+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:09:05.407967+0000","last_undegraded":"2024-05-09T09:09:05.407967+0000","last_fullsized":"2024-05-09T09:09:05.407967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708633+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:08:37.708633+0000","last_peered":"2024-05-09T09:08:37.708633+0000","last_clean":"2024-05-09T09:08:37.708633+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:08:37.708633+0000","last_undegraded":"2024-05-09T09:08:37.708633+0000","last_fullsized":"2024-05-09T09:08:37.708633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706507+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:08:36.706507+0000","last_peered":"2024-05-09T09:08:36.706507+0000","last_clean":"2024-05-09T09:08:36.706507+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:08:36.706507+0000","last_undegraded":"2024-05-09T09:08:36.706507+0000","last_fullsized":"2024-05-09T09:08:36.706507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703980+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:08:36.703980+0000","last_peered":"2024-05-09T09:08:36.703980+0000","last_clean":"2024-05-09T09:08:36.703980+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:08:36.703980+0000","last_undegraded":"2024-05-09T09:08:36.703980+0000","last_fullsized":"2024-05-09T09:08:36.703980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":355,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708443+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:08:37.708443+0000","last_peered":"2024-05-09T09:08:37.708443+0000","last_clean":"2024-05-09T09:08:37.708443+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:08:37.708443+0000","last_undegraded":"2024-05-09T09:08:37.708443+0000","last_fullsized":"2024-05-09T09:08:37.708443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579449+0000","last_change":"2024-05-09T09:07:36.319419+0000","last_active":"2024-05-09T09:08:37.579449+0000","last_peered":"2024-05-09T09:08:37.579449+0000","last_clean":"2024-05-09T09:08:37.579449+0000","last_became_active":"2024-05-09T09:07:36.318825+0000","last_became_peered":"2024-05-09T09:07:36.318825+0000","last_unstale":"2024-05-09T09:08:37.579449+0000","last_undegraded":"2024-05-09T09:08:37.579449+0000","last_fullsized":"2024-05-09T09:08:37.579449+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:21:10.289893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.918805+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:08:36.918805+0000","last_peered":"2024-05-09T09:08:36.918805+0000","last_clean":"2024-05-09T09:08:36.918805+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:08:36.918805+0000","last_undegraded":"2024-05-09T09:08:36.918805+0000","last_fullsized":"2024-05-09T09:08:36.918805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709055+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:08:37.709055+0000","last_peered":"2024-05-09T09:08:37.709055+0000","last_clean":"2024-05-09T09:08:37.709055+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:08:37.709055+0000","last_undegraded":"2024-05-09T09:08:37.709055+0000","last_fullsized":"2024-05-09T09:08:37.709055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672046+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:08:37.672046+0000","last_peered":"2024-05-09T09:08:37.672046+0000","last_clean":"2024-05-09T09:08:37.672046+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:08:37.672046+0000","last_undegraded":"2024-05-09T09:08:37.672046+0000","last_fullsized":"2024-05-09T09:08:37.672046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579460+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:08:37.579460+0000","last_peered":"2024-05-09T09:08:37.579460+0000","last_clean":"2024-05-09T09:08:37.579460+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:08:37.579460+0000","last_undegraded":"2024-05-09T09:08:37.579460+0000","last_fullsized":"2024-05-09T09:08:37.579460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919182+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:08:36.919182+0000","last_peered":"2024-05-09T09:08:36.919182+0000","last_clean":"2024-05-09T09:08:36.919182+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:08:36.919182+0000","last_undegraded":"2024-05-09T09:08:36.919182+0000","last_fullsized":"2024-05-09T09:08:36.919182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705339+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:08:36.705339+0000","last_peered":"2024-05-09T09:08:36.705339+0000","last_clean":"2024-05-09T09:08:36.705339+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:08:36.705339+0000","last_undegraded":"2024-05-09T09:08:36.705339+0000","last_fullsized":"2024-05-09T09:08:36.705339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:50.803291+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:08:50.803291+0000","last_peered":"2024-05-09T09:08:50.803291+0000","last_clean":"2024-05-09T09:08:50.803291+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:08:50.803291+0000","last_undegraded":"2024-05-09T09:08:50.803291+0000","last_fullsized":"2024-05-09T09:08:50.803291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707099+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:08:36.707099+0000","last_peered":"2024-05-09T09:08:36.707099+0000","last_clean":"2024-05-09T09:08:36.707099+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:08:36.707099+0000","last_undegraded":"2024-05-09T09:08:36.707099+0000","last_fullsized":"2024-05-09T09:08:36.707099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703979+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:08:36.703979+0000","last_peered":"2024-05-09T09:08:36.703979+0000","last_clean":"2024-05-09T09:08:36.703979+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:08:36.703979+0000","last_undegraded":"2024-05-09T09:08:36.703979+0000","last_fullsized":"2024-05-09T09:08:36.703979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704427+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:08:36.704427+0000","last_peered":"2024-05-09T09:08:36.704427+0000","last_clean":"2024-05-09T09:08:36.704427+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:08:36.704427+0000","last_undegraded":"2024-05-09T09:08:36.704427+0000","last_fullsized":"2024-05-09T09:08:36.704427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939395+0000","last_change":"2024-05-09T09:07:36.322492+0000","last_active":"2024-05-09T09:08:37.939395+0000","last_peered":"2024-05-09T09:08:37.939395+0000","last_clean":"2024-05-09T09:08:37.939395+0000","last_became_active":"2024-05-09T09:07:36.322098+0000","last_became_peered":"2024-05-09T09:07:36.322098+0000","last_unstale":"2024-05-09T09:08:37.939395+0000","last_undegraded":"2024-05-09T09:08:37.939395+0000","last_fullsized":"2024-05-09T09:08:37.939395+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:55.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704203+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:08:36.704203+0000","last_peered":"2024-05-09T09:08:36.704203+0000","last_clean":"2024-05-09T09:08:36.704203+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:08:36.704203+0000","last_undegraded":"2024-05-09T09:08:36.704203+0000","last_fullsized":"2024-05-09T09:08:36.704203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123773+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:08:37.123773+0000","last_peered":"2024-05-09T09:08:37.123773+0000","last_clean":"2024-05-09T09:08:37.123773+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:08:37.123773+0000","last_undegraded":"2024-05-09T09:08:37.123773+0000","last_fullsized":"2024-05-09T09:08:37.123773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578938+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:08:37.578938+0000","last_peered":"2024-05-09T09:08:37.578938+0000","last_clean":"2024-05-09T09:08:37.578938+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:08:37.578938+0000","last_undegraded":"2024-05-09T09:08:37.578938+0000","last_fullsized":"2024-05-09T09:08:37.578938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706560+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:08:36.706560+0000","last_peered":"2024-05-09T09:08:36.706560+0000","last_clean":"2024-05-09T09:08:36.706560+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:08:36.706560+0000","last_undegraded":"2024-05-09T09:08:36.706560+0000","last_fullsized":"2024-05-09T09:08:36.706560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705432+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:08:36.705432+0000","last_peered":"2024-05-09T09:08:36.705432+0000","last_clean":"2024-05-09T09:08:36.705432+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:08:36.705432+0000","last_undegraded":"2024-05-09T09:08:36.705432+0000","last_fullsized":"2024-05-09T09:08:36.705432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672281+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:08:37.672281+0000","last_peered":"2024-05-09T09:08:37.672281+0000","last_clean":"2024-05-09T09:08:37.672281+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:08:37.672281+0000","last_undegraded":"2024-05-09T09:08:37.672281+0000","last_fullsized":"2024-05-09T09:08:37.672281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704138+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:08:36.704138+0000","last_peered":"2024-05-09T09:08:36.704138+0000","last_clean":"2024-05-09T09:08:36.704138+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:08:36.704138+0000","last_undegraded":"2024-05-09T09:08:36.704138+0000","last_fullsized":"2024-05-09T09:08:36.704138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672311+0000","last_change":"2024-05-09T09:07:36.327470+0000","last_active":"2024-05-09T09:08:37.672311+0000","last_peered":"2024-05-09T09:08:37.672311+0000","last_clean":"2024-05-09T09:08:37.672311+0000","last_became_active":"2024-05-09T09:07:36.321516+0000","last_became_peered":"2024-05-09T09:07:36.321516+0000","last_unstale":"2024-05-09T09:08:37.672311+0000","last_undegraded":"2024-05-09T09:08:37.672311+0000","last_fullsized":"2024-05-09T09:08:37.672311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:30.824518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705774+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:08:36.705774+0000","last_peered":"2024-05-09T09:08:36.705774+0000","last_clean":"2024-05-09T09:08:36.705774+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:08:36.705774+0000","last_undegraded":"2024-05-09T09:08:36.705774+0000","last_fullsized":"2024-05-09T09:08:36.705774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704007+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:08:36.704007+0000","last_peered":"2024-05-09T09:08:36.704007+0000","last_clean":"2024-05-09T09:08:36.704007+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:08:36.704007+0000","last_undegraded":"2024-05-09T09:08:36.704007+0000","last_fullsized":"2024-05-09T09:08:36.704007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579278+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:08:37.579278+0000","last_peered":"2024-05-09T09:08:37.579278+0000","last_clean":"2024-05-09T09:08:37.579278+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:08:37.579278+0000","last_undegraded":"2024-05-09T09:08:37.579278+0000","last_fullsized":"2024-05-09T09:08:37.579278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709254+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:08:37.709254+0000","last_peered":"2024-05-09T09:08:37.709254+0000","last_clean":"2024-05-09T09:08:37.709254+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:08:37.709254+0000","last_undegraded":"2024-05-09T09:08:37.709254+0000","last_fullsized":"2024-05-09T09:08:37.709254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123785+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:08:37.123785+0000","last_peered":"2024-05-09T09:08:37.123785+0000","last_clean":"2024-05-09T09:08:37.123785+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:08:37.123785+0000","last_undegraded":"2024-05-09T09:08:37.123785+0000","last_fullsized":"2024-05-09T09:08:37.123785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704679+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:08:36.704679+0000","last_peered":"2024-05-09T09:08:36.704679+0000","last_clean":"2024-05-09T09:08:36.704679+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:08:36.704679+0000","last_undegraded":"2024-05-09T09:08:36.704679+0000","last_fullsized":"2024-05-09T09:08:36.704679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709737+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:08:37.709737+0000","last_peered":"2024-05-09T09:08:37.709737+0000","last_clean":"2024-05-09T09:08:37.709737+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:08:37.709737+0000","last_undegraded":"2024-05-09T09:08:37.709737+0000","last_fullsized":"2024-05-09T09:08:37.709737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938607+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:08:37.938607+0000","last_peered":"2024-05-09T09:08:37.938607+0000","last_clean":"2024-05-09T09:08:37.938607+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:08:37.938607+0000","last_undegraded":"2024-05-09T09:08:37.938607+0000","last_fullsized":"2024-05-09T09:08:37.938607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709214+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:08:36.709214+0000","last_peered":"2024-05-09T09:08:36.709214+0000","last_clean":"2024-05-09T09:08:36.709214+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:08:36.709214+0000","last_undegraded":"2024-05-09T09:08:36.709214+0000","last_fullsized":"2024-05-09T09:08:36.709214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709491+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:08:37.709491+0000","last_peered":"2024-05-09T09:08:37.709491+0000","last_clean":"2024-05-09T09:08:37.709491+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:08:37.709491+0000","last_undegraded":"2024-05-09T09:08:37.709491+0000","last_fullsized":"2024-05-09T09:08:37.709491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708466+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:08:36.708466+0000","last_peered":"2024-05-09T09:08:36.708466+0000","last_clean":"2024-05-09T09:08:36.708466+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:08:36.708466+0000","last_undegraded":"2024-05-09T09:08:36.708466+0000","last_fullsized":"2024-05-09T09:08:36.708466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919179+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:08:36.919179+0000","last_peered":"2024-05-09T09:08:36.919179+0000","last_clean":"2024-05-09T09:08:36.919179+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:08:36.919179+0000","last_undegraded":"2024-05-09T09:08:36.919179+0000","last_fullsized":"2024-05-09T09:08:36.919179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709710+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:08:37.709710+0000","last_peered":"2024-05-09T09:08:37.709710+0000","last_clean":"2024-05-09T09:08:37.709710+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:08:37.709710+0000","last_undegraded":"2024-05-09T09:08:37.709710+0000","last_fullsized":"2024-05-09T09:08:37.709710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672377+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:08:37.672377+0000","last_peered":"2024-05-09T09:08:37.672377+0000","last_clean":"2024-05-09T09:08:37.672377+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:08:37.672377+0000","last_undegraded":"2024-05-09T09:08:37.672377+0000","last_fullsized":"2024-05-09T09:08:37.672377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704264+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:08:36.704264+0000","last_peered":"2024-05-09T09:08:36.704264+0000","last_clean":"2024-05-09T09:08:36.704264+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:08:36.704264+0000","last_undegraded":"2024-05-09T09:08:36.704264+0000","last_fullsized":"2024-05-09T09:08:36.704264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703961+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:08:36.703961+0000","last_peered":"2024-05-09T09:08:36.703961+0000","last_clean":"2024-05-09T09:08:36.703961+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:08:36.703961+0000","last_undegraded":"2024-05-09T09:08:36.703961+0000","last_fullsized":"2024-05-09T09:08:36.703961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939091+0000","last_change":"2024-05-09T09:07:36.321453+0000","last_active":"2024-05-09T09:08:37.939091+0000","last_peered":"2024-05-09T09:08:37.939091+0000","last_clean":"2024-05-09T09:08:37.939091+0000","last_became_active":"2024-05-09T09:07:36.321074+0000","last_became_peered":"2024-05-09T09:07:36.321074+0000","last_unstale":"2024-05-09T09:08:37.939091+0000","last_undegraded":"2024-05-09T09:08:37.939091+0000","last_fullsized":"2024-05-09T09:08:37.939091+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:26.690978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919162+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:08:36.919162+0000","last_peered":"2024-05-09T09:08:36.919162+0000","last_clean":"2024-05-09T09:08:36.919162+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:08:36.919162+0000","last_undegraded":"2024-05-09T09:08:36.919162+0000","last_fullsized":"2024-05-09T09:08:36.919162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672652+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:08:37.672652+0000","last_peered":"2024-05-09T09:08:37.672652+0000","last_clean":"2024-05-09T09:08:37.672652+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:08:37.672652+0000","last_undegraded":"2024-05-09T09:08:37.672652+0000","last_fullsized":"2024-05-09T09:08:37.672652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704543+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:08:36.704543+0000","last_peered":"2024-05-09T09:08:36.704543+0000","last_clean":"2024-05-09T09:08:36.704543+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:08:36.704543+0000","last_undegraded":"2024-05-09T09:08:36.704543+0000","last_fullsized":"2024-05-09T09:08:36.704543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707710+0000","last_change":"2024-05-09T09:08:36.707710+0000","last_active":"2024-05-09T09:08:36.707710+0000","last_peered":"2024-05-09T09:08:36.707710+0000","last_clean":"2024-05-09T09:08:36.707710+0000","last_became_active":"2024-05-09T09:08:36.707333+0000","last_became_peered":"2024-05-09T09:08:36.707333+0000","last_unstale":"2024-05-09T09:08:36.707710+0000","last_undegraded":"2024-05-09T09:08:36.707710+0000","last_fullsized":"2024-05-09T09:08:36.707710+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:08.495289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709005+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:08:36.709005+0000","last_peered":"2024-05-09T09:08:36.709005+0000","last_clean":"2024-05-09T09:08:36.709005+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:08:36.709005+0000","last_undegraded":"2024-05-09T09:08:36.709005+0000","last_fullsized":"2024-05-09T09:08:36.709005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705731+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:08:36.705731+0000","last_peered":"2024-05-09T09:08:36.705731+0000","last_clean":"2024-05-09T09:08:36.705731+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:08:36.705731+0000","last_undegraded":"2024-05-09T09:08:36.705731+0000","last_fullsized":"2024-05-09T09:08:36.705731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124059+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:08:37.124059+0000","last_peered":"2024-05-09T09:08:37.124059+0000","last_clean":"2024-05-09T09:08:37.124059+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:08:37.124059+0000","last_undegraded":"2024-05-09T09:08:37.124059+0000","last_fullsized":"2024-05-09T09:08:37.124059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919025+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:08:36.919025+0000","last_peered":"2024-05-09T09:08:36.919025+0000","last_clean":"2024-05-09T09:08:36.919025+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:08:36.919025+0000","last_undegraded":"2024-05-09T09:08:36.919025+0000","last_fullsized":"2024-05-09T09:08:36.919025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672494+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:08:37.672494+0000","last_peered":"2024-05-09T09:08:37.672494+0000","last_clean":"2024-05-09T09:08:37.672494+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:08:37.672494+0000","last_undegraded":"2024-05-09T09:08:37.672494+0000","last_fullsized":"2024-05-09T09:08:37.672494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70794,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":749,"num_read_kb":449,"num_write":173,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":208068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":522662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612657,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1176615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":526972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1119268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":518052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":579401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":519440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":586019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":518246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":523179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1683456,"data_stored":1114379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:09:09.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:09:09.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 -- 172.21.15.64:0/2224872591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0101960 msgr2=0x7f3ad0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/2224872591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0101960 0x7f3ad0101de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009900 tx=0x7f3ac402a410 comp rx=0 tx=0).stop 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 -- 172.21.15.64:0/2224872591 shutdown_connections 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/2224872591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad0102320 0x7f3ad0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/2224872591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0101960 0x7f3ad0101de0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/2224872591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 0x7f3ad0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 -- 172.21.15.64:0/2224872591 >> 172.21.15.64:0/2224872591 conn(0x7f3ad00fbce0 msgr2=0x7f3ad00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 -- 172.21.15.64:0/2224872591 shutdown_connections 2024-05-09T09:09:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.871+0000 7f3ad4ada640 1 -- 172.21.15.64:0/2224872591 wait complete. 2024-05-09T09:09:09.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 Processor -- start 2024-05-09T09:09:09.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 -- start start 2024-05-09T09:09:09.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 0x7f3ad010b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad0101960 0x7f3ad0109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 0x7f3ad0109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010d3c0 con 0x7f3ad0102320 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010d240 con 0x7f3ad0100770 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3ad4ada640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010d540 con 0x7f3ad0101960 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3aced9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 0x7f3ad0109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3aced9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 0x7f3ad0109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39066/0 (socket says 172.21.15.64:39066) 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.872+0000 7f3aced9d640 1 -- 172.21.15.64:0/1622673030 learned_addr learned my addr 172.21.15.64:0/1622673030 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3acdd9b640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad0101960 0x7f3ad0109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3ace59c640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 0x7f3ad010b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3aced9d640 1 -- 172.21.15.64:0/1622673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad0101960 msgr2=0x7f3ad0109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3aced9d640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad0101960 0x7f3ad0109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3aced9d640 1 -- 172.21.15.64:0/1622673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 msgr2=0x7f3ad010b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3aced9d640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 0x7f3ad010b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3aced9d640 1 -- 172.21.15.64:0/1622673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad010a500 con 0x7f3ad0102320 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3ace59c640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 0x7f3ad010b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3aced9d640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 0x7f3ad0109c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800eaa0 tx=0x7f3ac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800cdb0 con 0x7f3ad0102320 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ac8005020 con 0x7f3ad0102320 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a1ba0 con 0x7f3ad0102320 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80116d0 con 0x7f3ad0102320 2024-05-09T09:09:09.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.873+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a2110 con 0x7f3ad0102320 2024-05-09T09:09:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.874+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0107b10 con 0x7f3ad0102320 2024-05-09T09:09:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.875+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ac80051d0 con 0x7f3ad0102320 2024-05-09T09:09:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.876+0000 7f3abf7fe640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3ab80a2c10 0x7f3ab80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.876+0000 7f3ace59c640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3ab80a2c10 0x7f3ab80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.877+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ac80cf4e0 con 0x7f3ad0102320 2024-05-09T09:09:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.877+0000 7f3ace59c640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3ab80a2c10 0x7f3ab80a50d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0005d00 tx=0x7f3ac0005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:09.881+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ac8016030 con 0x7f3ad0102320 2024-05-09T09:09:10.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.065+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ad0061c70 con 0x7f3ab80a2c10 2024-05-09T09:09:10.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.074+0000 7f3abf7fe640 1 -- 172.21.15.64:0/1622673030 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7f3ad0061c70 con 0x7f3ab80a2c10 2024-05-09T09:09:10.077 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:10.078 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:09:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3ab80a2c10 msgr2=0x7f3ab80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3ab80a2c10 0x7f3ab80a50d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0005d00 tx=0x7f3ac0005c90 comp rx=0 tx=0).stop 2024-05-09T09:09:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 msgr2=0x7f3ad0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 0x7f3ad0109c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800eaa0 tx=0x7f3ac800ef60 comp rx=0 tx=0).stop 2024-05-09T09:09:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 shutdown_connections 2024-05-09T09:09:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3ab80a2c10 0x7f3ab80a50d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad0102320 0x7f3ad0109c70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad0101960 0x7f3ad0109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.077+0000 7f3ad4ada640 1 --2- 172.21.15.64:0/1622673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ad0100770 0x7f3ad010b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.078+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 >> 172.21.15.64:0/1622673030 conn(0x7f3ad00fbce0 msgr2=0x7f3ad0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.078+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 shutdown_connections 2024-05-09T09:09:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.078+0000 7f3ad4ada640 1 -- 172.21.15.64:0/1622673030 wait complete. 2024-05-09T09:09:10.086 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2827123871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:10.087 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1029703056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:10.088 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-05-09T09:09:09.135663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661396,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":875,"num_read_kb":558,"num_write":356,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392896,"kb_used_data":14580,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548928,"statfs":{"total":1151940427776,"available":1151538102272,"internally_reserved":0,"allocated":14929920,"data_stored":8227007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5745,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003023"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672739+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:08:37.672739+0000","last_peered":"2024-05-09T09:08:37.672739+0000","last_clean":"2024-05-09T09:08:37.672739+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:08:37.672739+0000","last_undegraded":"2024-05-09T09:08:37.672739+0000","last_fullsized":"2024-05-09T09:08:37.672739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939357+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:08:37.939357+0000","last_peered":"2024-05-09T09:08:37.939357+0000","last_clean":"2024-05-09T09:08:37.939357+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:08:37.939357+0000","last_undegraded":"2024-05-09T09:08:37.939357+0000","last_fullsized":"2024-05-09T09:08:37.939357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939211+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:08:37.939211+0000","last_peered":"2024-05-09T09:08:37.939211+0000","last_clean":"2024-05-09T09:08:37.939211+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:08:37.939211+0000","last_undegraded":"2024-05-09T09:08:37.939211+0000","last_fullsized":"2024-05-09T09:08:37.939211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704349+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:08:36.704349+0000","last_peered":"2024-05-09T09:08:36.704349+0000","last_clean":"2024-05-09T09:08:36.704349+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:08:36.704349+0000","last_undegraded":"2024-05-09T09:08:36.704349+0000","last_fullsized":"2024-05-09T09:08:36.704349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579167+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:08:37.579167+0000","last_peered":"2024-05-09T09:08:37.579167+0000","last_clean":"2024-05-09T09:08:37.579167+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:08:37.579167+0000","last_undegraded":"2024-05-09T09:08:37.579167+0000","last_fullsized":"2024-05-09T09:08:37.579167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846040+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:08:36.846040+0000","last_peered":"2024-05-09T09:08:36.846040+0000","last_clean":"2024-05-09T09:08:36.846040+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:08:36.846040+0000","last_undegraded":"2024-05-09T09:08:36.846040+0000","last_fullsized":"2024-05-09T09:08:36.846040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708968+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:08:37.708968+0000","last_peered":"2024-05-09T09:08:37.708968+0000","last_clean":"2024-05-09T09:08:37.708968+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:08:37.708968+0000","last_undegraded":"2024-05-09T09:08:37.708968+0000","last_fullsized":"2024-05-09T09:08:37.708968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706275+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:08:36.706275+0000","last_peered":"2024-05-09T09:08:36.706275+0000","last_clean":"2024-05-09T09:08:36.706275+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:08:36.706275+0000","last_undegraded":"2024-05-09T09:08:36.706275+0000","last_fullsized":"2024-05-09T09:08:36.706275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124251+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:08:37.124251+0000","last_peered":"2024-05-09T09:08:37.124251+0000","last_clean":"2024-05-09T09:08:37.124251+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:08:37.124251+0000","last_undegraded":"2024-05-09T09:08:37.124251+0000","last_fullsized":"2024-05-09T09:08:37.124251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846387+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:08:36.846387+0000","last_peered":"2024-05-09T09:08:36.846387+0000","last_clean":"2024-05-09T09:08:36.846387+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:08:36.846387+0000","last_undegraded":"2024-05-09T09:08:36.846387+0000","last_fullsized":"2024-05-09T09:08:36.846387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709714+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:08:37.709714+0000","last_peered":"2024-05-09T09:08:37.709714+0000","last_clean":"2024-05-09T09:08:37.709714+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:08:37.709714+0000","last_undegraded":"2024-05-09T09:08:37.709714+0000","last_fullsized":"2024-05-09T09:08:37.709714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706384+0000","last_change":"2024-05-09T09:07:36.317979+0000","last_active":"2024-05-09T09:08:36.706384+0000","last_peered":"2024-05-09T09:08:36.706384+0000","last_clean":"2024-05-09T09:08:36.706384+0000","last_became_active":"2024-05-09T09:07:36.317572+0000","last_became_peered":"2024-05-09T09:07:36.317572+0000","last_unstale":"2024-05-09T09:08:36.706384+0000","last_undegraded":"2024-05-09T09:08:36.706384+0000","last_fullsized":"2024-05-09T09:08:36.706384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:45.499023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919161+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:08:36.919161+0000","last_peered":"2024-05-09T09:08:36.919161+0000","last_clean":"2024-05-09T09:08:36.919161+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:08:36.919161+0000","last_undegraded":"2024-05-09T09:08:36.919161+0000","last_fullsized":"2024-05-09T09:08:36.919161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938799+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:08:37.938799+0000","last_peered":"2024-05-09T09:08:37.938799+0000","last_clean":"2024-05-09T09:08:37.938799+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:08:37.938799+0000","last_undegraded":"2024-05-09T09:08:37.938799+0000","last_fullsized":"2024-05-09T09:08:37.938799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938909+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:08:37.938909+0000","last_peered":"2024-05-09T09:08:37.938909+0000","last_clean":"2024-05-09T09:08:37.938909+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:08:37.938909+0000","last_undegraded":"2024-05-09T09:08:37.938909+0000","last_fullsized":"2024-05-09T09:08:37.938909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123822+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:08:37.123822+0000","last_peered":"2024-05-09T09:08:37.123822+0000","last_clean":"2024-05-09T09:08:37.123822+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:08:37.123822+0000","last_undegraded":"2024-05-09T09:08:37.123822+0000","last_fullsized":"2024-05-09T09:08:37.123822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672730+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:08:37.672730+0000","last_peered":"2024-05-09T09:08:37.672730+0000","last_clean":"2024-05-09T09:08:37.672730+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:08:37.672730+0000","last_undegraded":"2024-05-09T09:08:37.672730+0000","last_fullsized":"2024-05-09T09:08:37.672730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706528+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:08:36.706528+0000","last_peered":"2024-05-09T09:08:36.706528+0000","last_clean":"2024-05-09T09:08:36.706528+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:08:36.706528+0000","last_undegraded":"2024-05-09T09:08:36.706528+0000","last_fullsized":"2024-05-09T09:08:36.706528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704353+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:08:36.704353+0000","last_peered":"2024-05-09T09:08:36.704353+0000","last_clean":"2024-05-09T09:08:36.704353+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:08:36.704353+0000","last_undegraded":"2024-05-09T09:08:36.704353+0000","last_fullsized":"2024-05-09T09:08:36.704353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703943+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:08:36.703943+0000","last_peered":"2024-05-09T09:08:36.703943+0000","last_clean":"2024-05-09T09:08:36.703943+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:08:36.703943+0000","last_undegraded":"2024-05-09T09:08:36.703943+0000","last_fullsized":"2024-05-09T09:08:36.703943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706328+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:08:36.706328+0000","last_peered":"2024-05-09T09:08:36.706328+0000","last_clean":"2024-05-09T09:08:36.706328+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:08:36.706328+0000","last_undegraded":"2024-05-09T09:08:36.706328+0000","last_fullsized":"2024-05-09T09:08:36.706328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.710323+0000","last_change":"2024-05-09T09:08:36.710323+0000","last_active":"2024-05-09T09:08:36.710323+0000","last_peered":"2024-05-09T09:08:36.710323+0000","last_clean":"2024-05-09T09:08:36.710323+0000","last_became_active":"2024-05-09T09:08:36.709965+0000","last_became_peered":"2024-05-09T09:08:36.709965+0000","last_unstale":"2024-05-09T09:08:36.710323+0000","last_undegraded":"2024-05-09T09:08:36.710323+0000","last_fullsized":"2024-05-09T09:08:36.710323+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:01.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703719+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:08:36.703719+0000","last_peered":"2024-05-09T09:08:36.703719+0000","last_clean":"2024-05-09T09:08:36.703719+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:08:36.703719+0000","last_undegraded":"2024-05-09T09:08:36.703719+0000","last_fullsized":"2024-05-09T09:08:36.703719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707362+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:08:36.707362+0000","last_peered":"2024-05-09T09:08:36.707362+0000","last_clean":"2024-05-09T09:08:36.707362+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:08:36.707362+0000","last_undegraded":"2024-05-09T09:08:36.707362+0000","last_fullsized":"2024-05-09T09:08:36.707362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939378+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:08:37.939378+0000","last_peered":"2024-05-09T09:08:37.939378+0000","last_clean":"2024-05-09T09:08:37.939378+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:08:37.939378+0000","last_undegraded":"2024-05-09T09:08:37.939378+0000","last_fullsized":"2024-05-09T09:08:37.939378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703834+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:08:36.703834+0000","last_peered":"2024-05-09T09:08:36.703834+0000","last_clean":"2024-05-09T09:08:36.703834+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:08:36.703834+0000","last_undegraded":"2024-05-09T09:08:36.703834+0000","last_fullsized":"2024-05-09T09:08:36.703834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579531+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:08:37.579531+0000","last_peered":"2024-05-09T09:08:37.579531+0000","last_clean":"2024-05-09T09:08:37.579531+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:08:37.579531+0000","last_undegraded":"2024-05-09T09:08:37.579531+0000","last_fullsized":"2024-05-09T09:08:37.579531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123893+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:08:37.123893+0000","last_peered":"2024-05-09T09:08:37.123893+0000","last_clean":"2024-05-09T09:08:37.123893+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:08:37.123893+0000","last_undegraded":"2024-05-09T09:08:37.123893+0000","last_fullsized":"2024-05-09T09:08:37.123893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703725+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:08:36.703725+0000","last_peered":"2024-05-09T09:08:36.703725+0000","last_clean":"2024-05-09T09:08:36.703725+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:08:36.703725+0000","last_undegraded":"2024-05-09T09:08:36.703725+0000","last_fullsized":"2024-05-09T09:08:36.703725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579279+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:08:37.579279+0000","last_peered":"2024-05-09T09:08:37.579279+0000","last_clean":"2024-05-09T09:08:37.579279+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:08:37.579279+0000","last_undegraded":"2024-05-09T09:08:37.579279+0000","last_fullsized":"2024-05-09T09:08:37.579279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708974+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:08:36.708974+0000","last_peered":"2024-05-09T09:08:36.708974+0000","last_clean":"2024-05-09T09:08:36.708974+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:08:36.708974+0000","last_undegraded":"2024-05-09T09:08:36.708974+0000","last_fullsized":"2024-05-09T09:08:36.708974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704172+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:08:36.704172+0000","last_peered":"2024-05-09T09:08:36.704172+0000","last_clean":"2024-05-09T09:08:36.704172+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:08:36.704172+0000","last_undegraded":"2024-05-09T09:08:36.704172+0000","last_fullsized":"2024-05-09T09:08:36.704172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:06.410475+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:09:06.410475+0000","last_peered":"2024-05-09T09:09:06.410475+0000","last_clean":"2024-05-09T09:09:06.410475+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:09:06.410475+0000","last_undegraded":"2024-05-09T09:09:06.410475+0000","last_fullsized":"2024-05-09T09:09:06.410475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1099,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579170+0000","last_change":"2024-05-09T09:07:36.615226+0000","last_active":"2024-05-09T09:08:37.579170+0000","last_peered":"2024-05-09T09:08:37.579170+0000","last_clean":"2024-05-09T09:08:37.579170+0000","last_became_active":"2024-05-09T09:07:36.614762+0000","last_became_peered":"2024-05-09T09:07:36.614762+0000","last_unstale":"2024-05-09T09:08:37.579170+0000","last_undegraded":"2024-05-09T09:08:37.579170+0000","last_fullsized":"2024-05-09T09:08:37.579170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:29.045379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704747+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:08:36.704747+0000","last_peered":"2024-05-09T09:08:36.704747+0000","last_clean":"2024-05-09T09:08:36.704747+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:08:36.704747+0000","last_undegraded":"2024-05-09T09:08:36.704747+0000","last_fullsized":"2024-05-09T09:08:36.704747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.845978+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:08:36.845978+0000","last_peered":"2024-05-09T09:08:36.845978+0000","last_clean":"2024-05-09T09:08:36.845978+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:08:36.845978+0000","last_undegraded":"2024-05-09T09:08:36.845978+0000","last_fullsized":"2024-05-09T09:08:36.845978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709278+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:08:36.709278+0000","last_peered":"2024-05-09T09:08:36.709278+0000","last_clean":"2024-05-09T09:08:36.709278+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:08:36.709278+0000","last_undegraded":"2024-05-09T09:08:36.709278+0000","last_fullsized":"2024-05-09T09:08:36.709278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672121+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:08:37.672121+0000","last_peered":"2024-05-09T09:08:37.672121+0000","last_clean":"2024-05-09T09:08:37.672121+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:08:37.672121+0000","last_undegraded":"2024-05-09T09:08:37.672121+0000","last_fullsized":"2024-05-09T09:08:37.672121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.918958+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:08:36.918958+0000","last_peered":"2024-05-09T09:08:36.918958+0000","last_clean":"2024-05-09T09:08:36.918958+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:08:36.918958+0000","last_undegraded":"2024-05-09T09:08:36.918958+0000","last_fullsized":"2024-05-09T09:08:36.918958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705457+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:08:36.705457+0000","last_peered":"2024-05-09T09:08:36.705457+0000","last_clean":"2024-05-09T09:08:36.705457+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:08:36.705457+0000","last_undegraded":"2024-05-09T09:08:36.705457+0000","last_fullsized":"2024-05-09T09:08:36.705457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938610+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:08:37.938610+0000","last_peered":"2024-05-09T09:08:37.938610+0000","last_clean":"2024-05-09T09:08:37.938610+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:08:37.938610+0000","last_undegraded":"2024-05-09T09:08:37.938610+0000","last_fullsized":"2024-05-09T09:08:37.938610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708907+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:08:36.708907+0000","last_peered":"2024-05-09T09:08:36.708907+0000","last_clean":"2024-05-09T09:08:36.708907+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:08:36.708907+0000","last_undegraded":"2024-05-09T09:08:36.708907+0000","last_fullsized":"2024-05-09T09:08:36.708907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705674+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:08:36.705674+0000","last_peered":"2024-05-09T09:08:36.705674+0000","last_clean":"2024-05-09T09:08:36.705674+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:08:36.705674+0000","last_undegraded":"2024-05-09T09:08:36.705674+0000","last_fullsized":"2024-05-09T09:08:36.705674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706934+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:08:36.706934+0000","last_peered":"2024-05-09T09:08:36.706934+0000","last_clean":"2024-05-09T09:08:36.706934+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:08:36.706934+0000","last_undegraded":"2024-05-09T09:08:36.706934+0000","last_fullsized":"2024-05-09T09:08:36.706934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.411229+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:09:07.411229+0000","last_peered":"2024-05-09T09:09:07.411229+0000","last_clean":"2024-05-09T09:09:07.411229+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:09:07.411229+0000","last_undegraded":"2024-05-09T09:09:07.411229+0000","last_fullsized":"2024-05-09T09:09:07.411229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124177+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:08:37.124177+0000","last_peered":"2024-05-09T09:08:37.124177+0000","last_clean":"2024-05-09T09:08:37.124177+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:08:37.124177+0000","last_undegraded":"2024-05-09T09:08:37.124177+0000","last_fullsized":"2024-05-09T09:08:37.124177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705636+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:08:36.705636+0000","last_peered":"2024-05-09T09:08:36.705636+0000","last_clean":"2024-05-09T09:08:36.705636+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:08:36.705636+0000","last_undegraded":"2024-05-09T09:08:36.705636+0000","last_fullsized":"2024-05-09T09:08:36.705636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709600+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:08:37.709600+0000","last_peered":"2024-05-09T09:08:37.709600+0000","last_clean":"2024-05-09T09:08:37.709600+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:08:37.709600+0000","last_undegraded":"2024-05-09T09:08:37.709600+0000","last_fullsized":"2024-05-09T09:08:37.709600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939354+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:08:37.939354+0000","last_peered":"2024-05-09T09:08:37.939354+0000","last_clean":"2024-05-09T09:08:37.939354+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:08:37.939354+0000","last_undegraded":"2024-05-09T09:08:37.939354+0000","last_fullsized":"2024-05-09T09:08:37.939354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704180+0000","last_change":"2024-05-09T09:07:36.436587+0000","last_active":"2024-05-09T09:08:36.704180+0000","last_peered":"2024-05-09T09:08:36.704180+0000","last_clean":"2024-05-09T09:08:36.704180+0000","last_became_active":"2024-05-09T09:07:36.436083+0000","last_became_peered":"2024-05-09T09:07:36.436083+0000","last_unstale":"2024-05-09T09:08:36.704180+0000","last_undegraded":"2024-05-09T09:08:36.704180+0000","last_fullsized":"2024-05-09T09:08:36.704180+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:02:34.904307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709115+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:08:36.709115+0000","last_peered":"2024-05-09T09:08:36.709115+0000","last_clean":"2024-05-09T09:08:36.709115+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:08:36.709115+0000","last_undegraded":"2024-05-09T09:08:36.709115+0000","last_fullsized":"2024-05-09T09:08:36.709115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124078+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:08:37.124078+0000","last_peered":"2024-05-09T09:08:37.124078+0000","last_clean":"2024-05-09T09:08:37.124078+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:08:37.124078+0000","last_undegraded":"2024-05-09T09:08:37.124078+0000","last_fullsized":"2024-05-09T09:08:37.124078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.845799+0000","last_change":"2024-05-09T09:07:36.324821+0000","last_active":"2024-05-09T09:08:36.845799+0000","last_peered":"2024-05-09T09:08:36.845799+0000","last_clean":"2024-05-09T09:08:36.845799+0000","last_became_active":"2024-05-09T09:07:36.324478+0000","last_became_peered":"2024-05-09T09:07:36.324478+0000","last_unstale":"2024-05-09T09:08:36.845799+0000","last_undegraded":"2024-05-09T09:08:36.845799+0000","last_fullsized":"2024-05-09T09:08:36.845799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:25.476000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":320,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.437201+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:09:07.437201+0000","last_peered":"2024-05-09T09:09:07.437201+0000","last_clean":"2024-05-09T09:09:07.437201+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:09:07.437201+0000","last_undegraded":"2024-05-09T09:09:07.437201+0000","last_fullsized":"2024-05-09T09:09:07.437201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66230,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579364+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:08:37.579364+0000","last_peered":"2024-05-09T09:08:37.579364+0000","last_clean":"2024-05-09T09:08:37.579364+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:08:37.579364+0000","last_undegraded":"2024-05-09T09:08:37.579364+0000","last_fullsized":"2024-05-09T09:08:37.579364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709325+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:08:36.709325+0000","last_peered":"2024-05-09T09:08:36.709325+0000","last_clean":"2024-05-09T09:08:36.709325+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:08:36.709325+0000","last_undegraded":"2024-05-09T09:08:36.709325+0000","last_fullsized":"2024-05-09T09:08:36.709325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704391+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:08:36.704391+0000","last_peered":"2024-05-09T09:08:36.704391+0000","last_clean":"2024-05-09T09:08:36.704391+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:08:36.704391+0000","last_undegraded":"2024-05-09T09:08:36.704391+0000","last_fullsized":"2024-05-09T09:08:36.704391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919138+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:08:36.919138+0000","last_peered":"2024-05-09T09:08:36.919138+0000","last_clean":"2024-05-09T09:08:36.919138+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:08:36.919138+0000","last_undegraded":"2024-05-09T09:08:36.919138+0000","last_fullsized":"2024-05-09T09:08:36.919138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672057+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:08:37.672057+0000","last_peered":"2024-05-09T09:08:37.672057+0000","last_clean":"2024-05-09T09:08:37.672057+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:08:37.672057+0000","last_undegraded":"2024-05-09T09:08:37.672057+0000","last_fullsized":"2024-05-09T09:08:37.672057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709159+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:08:36.709159+0000","last_peered":"2024-05-09T09:08:36.709159+0000","last_clean":"2024-05-09T09:08:36.709159+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:08:36.709159+0000","last_undegraded":"2024-05-09T09:08:36.709159+0000","last_fullsized":"2024-05-09T09:08:36.709159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709537+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:08:37.709537+0000","last_peered":"2024-05-09T09:08:37.709537+0000","last_clean":"2024-05-09T09:08:37.709537+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:08:37.709537+0000","last_undegraded":"2024-05-09T09:08:37.709537+0000","last_fullsized":"2024-05-09T09:08:37.709537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706621+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:08:36.706621+0000","last_peered":"2024-05-09T09:08:36.706621+0000","last_clean":"2024-05-09T09:08:36.706621+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:08:36.706621+0000","last_undegraded":"2024-05-09T09:08:36.706621+0000","last_fullsized":"2024-05-09T09:08:36.706621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.411204+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:09:07.411204+0000","last_peered":"2024-05-09T09:09:07.411204+0000","last_clean":"2024-05-09T09:09:07.411204+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:09:07.411204+0000","last_undegraded":"2024-05-09T09:09:07.411204+0000","last_fullsized":"2024-05-09T09:09:07.411204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709029+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:08:37.709029+0000","last_peered":"2024-05-09T09:08:37.709029+0000","last_clean":"2024-05-09T09:08:37.709029+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:08:37.709029+0000","last_undegraded":"2024-05-09T09:08:37.709029+0000","last_fullsized":"2024-05-09T09:08:37.709029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919195+0000","last_change":"2024-05-09T09:07:36.323604+0000","last_active":"2024-05-09T09:08:36.919195+0000","last_peered":"2024-05-09T09:08:36.919195+0000","last_clean":"2024-05-09T09:08:36.919195+0000","last_became_active":"2024-05-09T09:07:36.323254+0000","last_became_peered":"2024-05-09T09:07:36.323254+0000","last_unstale":"2024-05-09T09:08:36.919195+0000","last_undegraded":"2024-05-09T09:08:36.919195+0000","last_fullsized":"2024-05-09T09:08:36.919195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:31:18.746847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578879+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:08:37.578879+0000","last_peered":"2024-05-09T09:08:37.578879+0000","last_clean":"2024-05-09T09:08:37.578879+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:08:37.578879+0000","last_undegraded":"2024-05-09T09:08:37.578879+0000","last_fullsized":"2024-05-09T09:08:37.578879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708634+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:08:36.708634+0000","last_peered":"2024-05-09T09:08:36.708634+0000","last_clean":"2024-05-09T09:08:36.708634+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:08:36.708634+0000","last_undegraded":"2024-05-09T09:08:36.708634+0000","last_fullsized":"2024-05-09T09:08:36.708634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708641+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:08:36.708641+0000","last_peered":"2024-05-09T09:08:36.708641+0000","last_clean":"2024-05-09T09:08:36.708641+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:08:36.708641+0000","last_undegraded":"2024-05-09T09:08:36.708641+0000","last_fullsized":"2024-05-09T09:08:36.708641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672602+0000","last_change":"2024-05-09T09:07:36.331318+0000","last_active":"2024-05-09T09:08:37.672602+0000","last_peered":"2024-05-09T09:08:37.672602+0000","last_clean":"2024-05-09T09:08:37.672602+0000","last_became_active":"2024-05-09T09:07:36.323269+0000","last_became_peered":"2024-05-09T09:07:36.323269+0000","last_unstale":"2024-05-09T09:08:37.672602+0000","last_undegraded":"2024-05-09T09:08:37.672602+0000","last_fullsized":"2024-05-09T09:08:37.672602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:25:53.779933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672622+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:08:37.672622+0000","last_peered":"2024-05-09T09:08:37.672622+0000","last_clean":"2024-05-09T09:08:37.672622+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:08:37.672622+0000","last_undegraded":"2024-05-09T09:08:37.672622+0000","last_fullsized":"2024-05-09T09:08:37.672622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578766+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:08:37.578766+0000","last_peered":"2024-05-09T09:08:37.578766+0000","last_clean":"2024-05-09T09:08:37.578766+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:08:37.578766+0000","last_undegraded":"2024-05-09T09:08:37.578766+0000","last_fullsized":"2024-05-09T09:08:37.578766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706539+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:08:36.706539+0000","last_peered":"2024-05-09T09:08:36.706539+0000","last_clean":"2024-05-09T09:08:36.706539+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:08:36.706539+0000","last_undegraded":"2024-05-09T09:08:36.706539+0000","last_fullsized":"2024-05-09T09:08:36.706539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846220+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:08:36.846220+0000","last_peered":"2024-05-09T09:08:36.846220+0000","last_clean":"2024-05-09T09:08:36.846220+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:08:36.846220+0000","last_undegraded":"2024-05-09T09:08:36.846220+0000","last_fullsized":"2024-05-09T09:08:36.846220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919024+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:08:36.919024+0000","last_peered":"2024-05-09T09:08:36.919024+0000","last_clean":"2024-05-09T09:08:36.919024+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:08:36.919024+0000","last_undegraded":"2024-05-09T09:08:36.919024+0000","last_fullsized":"2024-05-09T09:08:36.919024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709369+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:08:37.709369+0000","last_peered":"2024-05-09T09:08:37.709369+0000","last_clean":"2024-05-09T09:08:37.709369+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:08:37.709369+0000","last_undegraded":"2024-05-09T09:08:37.709369+0000","last_fullsized":"2024-05-09T09:08:37.709369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846078+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:08:36.846078+0000","last_peered":"2024-05-09T09:08:36.846078+0000","last_clean":"2024-05-09T09:08:36.846078+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:08:36.846078+0000","last_undegraded":"2024-05-09T09:08:36.846078+0000","last_fullsized":"2024-05-09T09:08:36.846078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919150+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:08:36.919150+0000","last_peered":"2024-05-09T09:08:36.919150+0000","last_clean":"2024-05-09T09:08:36.919150+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:08:36.919150+0000","last_undegraded":"2024-05-09T09:08:36.919150+0000","last_fullsized":"2024-05-09T09:08:36.919150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709052+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:08:36.709052+0000","last_peered":"2024-05-09T09:08:36.709052+0000","last_clean":"2024-05-09T09:08:36.709052+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:08:36.709052+0000","last_undegraded":"2024-05-09T09:08:36.709052+0000","last_fullsized":"2024-05-09T09:08:36.709052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:05.407967+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:09:05.407967+0000","last_peered":"2024-05-09T09:09:05.407967+0000","last_clean":"2024-05-09T09:09:05.407967+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:09:05.407967+0000","last_undegraded":"2024-05-09T09:09:05.407967+0000","last_fullsized":"2024-05-09T09:09:05.407967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708633+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:08:37.708633+0000","last_peered":"2024-05-09T09:08:37.708633+0000","last_clean":"2024-05-09T09:08:37.708633+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:08:37.708633+0000","last_undegraded":"2024-05-09T09:08:37.708633+0000","last_fullsized":"2024-05-09T09:08:37.708633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706507+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:08:36.706507+0000","last_peered":"2024-05-09T09:08:36.706507+0000","last_clean":"2024-05-09T09:08:36.706507+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:08:36.706507+0000","last_undegraded":"2024-05-09T09:08:36.706507+0000","last_fullsized":"2024-05-09T09:08:36.706507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703980+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:08:36.703980+0000","last_peered":"2024-05-09T09:08:36.703980+0000","last_clean":"2024-05-09T09:08:36.703980+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:08:36.703980+0000","last_undegraded":"2024-05-09T09:08:36.703980+0000","last_fullsized":"2024-05-09T09:08:36.703980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":355,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708443+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:08:37.708443+0000","last_peered":"2024-05-09T09:08:37.708443+0000","last_clean":"2024-05-09T09:08:37.708443+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:08:37.708443+0000","last_undegraded":"2024-05-09T09:08:37.708443+0000","last_fullsized":"2024-05-09T09:08:37.708443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579449+0000","last_change":"2024-05-09T09:07:36.319419+0000","last_active":"2024-05-09T09:08:37.579449+0000","last_peered":"2024-05-09T09:08:37.579449+0000","last_clean":"2024-05-09T09:08:37.579449+0000","last_became_active":"2024-05-09T09:07:36.318825+0000","last_became_peered":"2024-05-09T09:07:36.318825+0000","last_unstale":"2024-05-09T09:08:37.579449+0000","last_undegraded":"2024-05-09T09:08:37.579449+0000","last_fullsized":"2024-05-09T09:08:37.579449+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:21:10.289893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.918805+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:08:36.918805+0000","last_peered":"2024-05-09T09:08:36.918805+0000","last_clean":"2024-05-09T09:08:36.918805+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:08:36.918805+0000","last_undegraded":"2024-05-09T09:08:36.918805+0000","last_fullsized":"2024-05-09T09:08:36.918805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709055+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:08:37.709055+0000","last_peered":"2024-05-09T09:08:37.709055+0000","last_clean":"2024-05-09T09:08:37.709055+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:08:37.709055+0000","last_undegraded":"2024-05-09T09:08:37.709055+0000","last_fullsized":"2024-05-09T09:08:37.709055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672046+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:08:37.672046+0000","last_peered":"2024-05-09T09:08:37.672046+0000","last_clean":"2024-05-09T09:08:37.672046+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:08:37.672046+0000","last_undegraded":"2024-05-09T09:08:37.672046+0000","last_fullsized":"2024-05-09T09:08:37.672046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579460+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:08:37.579460+0000","last_peered":"2024-05-09T09:08:37.579460+0000","last_clean":"2024-05-09T09:08:37.579460+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:08:37.579460+0000","last_undegraded":"2024-05-09T09:08:37.579460+0000","last_fullsized":"2024-05-09T09:08:37.579460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919182+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:08:36.919182+0000","last_peered":"2024-05-09T09:08:36.919182+0000","last_clean":"2024-05-09T09:08:36.919182+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:08:36.919182+0000","last_undegraded":"2024-05-09T09:08:36.919182+0000","last_fullsized":"2024-05-09T09:08:36.919182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705339+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:08:36.705339+0000","last_peered":"2024-05-09T09:08:36.705339+0000","last_clean":"2024-05-09T09:08:36.705339+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:08:36.705339+0000","last_undegraded":"2024-05-09T09:08:36.705339+0000","last_fullsized":"2024-05-09T09:08:36.705339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:50.803291+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:08:50.803291+0000","last_peered":"2024-05-09T09:08:50.803291+0000","last_clean":"2024-05-09T09:08:50.803291+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:08:50.803291+0000","last_undegraded":"2024-05-09T09:08:50.803291+0000","last_fullsized":"2024-05-09T09:08:50.803291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707099+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:08:36.707099+0000","last_peered":"2024-05-09T09:08:36.707099+0000","last_clean":"2024-05-09T09:08:36.707099+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:08:36.707099+0000","last_undegraded":"2024-05-09T09:08:36.707099+0000","last_fullsized":"2024-05-09T09:08:36.707099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703979+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:08:36.703979+0000","last_peered":"2024-05-09T09:08:36.703979+0000","last_clean":"2024-05-09T09:08:36.703979+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:08:36.703979+0000","last_undegraded":"2024-05-09T09:08:36.703979+0000","last_fullsized":"2024-05-09T09:08:36.703979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704427+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:08:36.704427+0000","last_peered":"2024-05-09T09:08:36.704427+0000","last_clean":"2024-05-09T09:08:36.704427+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:08:36.704427+0000","last_undegraded":"2024-05-09T09:08:36.704427+0000","last_fullsized":"2024-05-09T09:08:36.704427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939395+0000","last_change":"2024-05-09T09:07:36.322492+0000","last_active":"2024-05-09T09:08:37.939395+0000","last_peered":"2024-05-09T09:08:37.939395+0000","last_clean":"2024-05-09T09:08:37.939395+0000","last_became_active":"2024-05-09T09:07:36.322098+0000","last_became_peered":"2024-05-09T09:07:36.322098+0000","last_unstale":"2024-05-09T09:08:37.939395+0000","last_undegraded":"2024-05-09T09:08:37.939395+0000","last_fullsized":"2024-05-09T09:08:37.939395+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:55.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704203+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:08:36.704203+0000","last_peered":"2024-05-09T09:08:36.704203+0000","last_clean":"2024-05-09T09:08:36.704203+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:08:36.704203+0000","last_undegraded":"2024-05-09T09:08:36.704203+0000","last_fullsized":"2024-05-09T09:08:36.704203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123773+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:08:37.123773+0000","last_peered":"2024-05-09T09:08:37.123773+0000","last_clean":"2024-05-09T09:08:37.123773+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:08:37.123773+0000","last_undegraded":"2024-05-09T09:08:37.123773+0000","last_fullsized":"2024-05-09T09:08:37.123773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578938+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:08:37.578938+0000","last_peered":"2024-05-09T09:08:37.578938+0000","last_clean":"2024-05-09T09:08:37.578938+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:08:37.578938+0000","last_undegraded":"2024-05-09T09:08:37.578938+0000","last_fullsized":"2024-05-09T09:08:37.578938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706560+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:08:36.706560+0000","last_peered":"2024-05-09T09:08:36.706560+0000","last_clean":"2024-05-09T09:08:36.706560+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:08:36.706560+0000","last_undegraded":"2024-05-09T09:08:36.706560+0000","last_fullsized":"2024-05-09T09:08:36.706560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705432+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:08:36.705432+0000","last_peered":"2024-05-09T09:08:36.705432+0000","last_clean":"2024-05-09T09:08:36.705432+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:08:36.705432+0000","last_undegraded":"2024-05-09T09:08:36.705432+0000","last_fullsized":"2024-05-09T09:08:36.705432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672281+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:08:37.672281+0000","last_peered":"2024-05-09T09:08:37.672281+0000","last_clean":"2024-05-09T09:08:37.672281+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:08:37.672281+0000","last_undegraded":"2024-05-09T09:08:37.672281+0000","last_fullsized":"2024-05-09T09:08:37.672281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704138+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:08:36.704138+0000","last_peered":"2024-05-09T09:08:36.704138+0000","last_clean":"2024-05-09T09:08:36.704138+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:08:36.704138+0000","last_undegraded":"2024-05-09T09:08:36.704138+0000","last_fullsized":"2024-05-09T09:08:36.704138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672311+0000","last_change":"2024-05-09T09:07:36.327470+0000","last_active":"2024-05-09T09:08:37.672311+0000","last_peered":"2024-05-09T09:08:37.672311+0000","last_clean":"2024-05-09T09:08:37.672311+0000","last_became_active":"2024-05-09T09:07:36.321516+0000","last_became_peered":"2024-05-09T09:07:36.321516+0000","last_unstale":"2024-05-09T09:08:37.672311+0000","last_undegraded":"2024-05-09T09:08:37.672311+0000","last_fullsized":"2024-05-09T09:08:37.672311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:30.824518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705774+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:08:36.705774+0000","last_peered":"2024-05-09T09:08:36.705774+0000","last_clean":"2024-05-09T09:08:36.705774+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:08:36.705774+0000","last_undegraded":"2024-05-09T09:08:36.705774+0000","last_fullsized":"2024-05-09T09:08:36.705774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704007+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:08:36.704007+0000","last_peered":"2024-05-09T09:08:36.704007+0000","last_clean":"2024-05-09T09:08:36.704007+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:08:36.704007+0000","last_undegraded":"2024-05-09T09:08:36.704007+0000","last_fullsized":"2024-05-09T09:08:36.704007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579278+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:08:37.579278+0000","last_peered":"2024-05-09T09:08:37.579278+0000","last_clean":"2024-05-09T09:08:37.579278+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:08:37.579278+0000","last_undegraded":"2024-05-09T09:08:37.579278+0000","last_fullsized":"2024-05-09T09:08:37.579278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709254+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:08:37.709254+0000","last_peered":"2024-05-09T09:08:37.709254+0000","last_clean":"2024-05-09T09:08:37.709254+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:08:37.709254+0000","last_undegraded":"2024-05-09T09:08:37.709254+0000","last_fullsized":"2024-05-09T09:08:37.709254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123785+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:08:37.123785+0000","last_peered":"2024-05-09T09:08:37.123785+0000","last_clean":"2024-05-09T09:08:37.123785+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:08:37.123785+0000","last_undegraded":"2024-05-09T09:08:37.123785+0000","last_fullsized":"2024-05-09T09:08:37.123785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704679+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:08:36.704679+0000","last_peered":"2024-05-09T09:08:36.704679+0000","last_clean":"2024-05-09T09:08:36.704679+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:08:36.704679+0000","last_undegraded":"2024-05-09T09:08:36.704679+0000","last_fullsized":"2024-05-09T09:08:36.704679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709737+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:08:37.709737+0000","last_peered":"2024-05-09T09:08:37.709737+0000","last_clean":"2024-05-09T09:08:37.709737+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:08:37.709737+0000","last_undegraded":"2024-05-09T09:08:37.709737+0000","last_fullsized":"2024-05-09T09:08:37.709737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938607+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:08:37.938607+0000","last_peered":"2024-05-09T09:08:37.938607+0000","last_clean":"2024-05-09T09:08:37.938607+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:08:37.938607+0000","last_undegraded":"2024-05-09T09:08:37.938607+0000","last_fullsized":"2024-05-09T09:08:37.938607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709214+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:08:36.709214+0000","last_peered":"2024-05-09T09:08:36.709214+0000","last_clean":"2024-05-09T09:08:36.709214+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:08:36.709214+0000","last_undegraded":"2024-05-09T09:08:36.709214+0000","last_fullsized":"2024-05-09T09:08:36.709214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709491+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:08:37.709491+0000","last_peered":"2024-05-09T09:08:37.709491+0000","last_clean":"2024-05-09T09:08:37.709491+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:08:37.709491+0000","last_undegraded":"2024-05-09T09:08:37.709491+0000","last_fullsized":"2024-05-09T09:08:37.709491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708466+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:08:36.708466+0000","last_peered":"2024-05-09T09:08:36.708466+0000","last_clean":"2024-05-09T09:08:36.708466+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:08:36.708466+0000","last_undegraded":"2024-05-09T09:08:36.708466+0000","last_fullsized":"2024-05-09T09:08:36.708466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919179+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:08:36.919179+0000","last_peered":"2024-05-09T09:08:36.919179+0000","last_clean":"2024-05-09T09:08:36.919179+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:08:36.919179+0000","last_undegraded":"2024-05-09T09:08:36.919179+0000","last_fullsized":"2024-05-09T09:08:36.919179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709710+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:08:37.709710+0000","last_peered":"2024-05-09T09:08:37.709710+0000","last_clean":"2024-05-09T09:08:37.709710+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:08:37.709710+0000","last_undegraded":"2024-05-09T09:08:37.709710+0000","last_fullsized":"2024-05-09T09:08:37.709710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672377+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:08:37.672377+0000","last_peered":"2024-05-09T09:08:37.672377+0000","last_clean":"2024-05-09T09:08:37.672377+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:08:37.672377+0000","last_undegraded":"2024-05-09T09:08:37.672377+0000","last_fullsized":"2024-05-09T09:08:37.672377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704264+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:08:36.704264+0000","last_peered":"2024-05-09T09:08:36.704264+0000","last_clean":"2024-05-09T09:08:36.704264+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:08:36.704264+0000","last_undegraded":"2024-05-09T09:08:36.704264+0000","last_fullsized":"2024-05-09T09:08:36.704264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703961+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:08:36.703961+0000","last_peered":"2024-05-09T09:08:36.703961+0000","last_clean":"2024-05-09T09:08:36.703961+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:08:36.703961+0000","last_undegraded":"2024-05-09T09:08:36.703961+0000","last_fullsized":"2024-05-09T09:08:36.703961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939091+0000","last_change":"2024-05-09T09:07:36.321453+0000","last_active":"2024-05-09T09:08:37.939091+0000","last_peered":"2024-05-09T09:08:37.939091+0000","last_clean":"2024-05-09T09:08:37.939091+0000","last_became_active":"2024-05-09T09:07:36.321074+0000","last_became_peered":"2024-05-09T09:07:36.321074+0000","last_unstale":"2024-05-09T09:08:37.939091+0000","last_undegraded":"2024-05-09T09:08:37.939091+0000","last_fullsized":"2024-05-09T09:08:37.939091+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:26.690978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919162+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:08:36.919162+0000","last_peered":"2024-05-09T09:08:36.919162+0000","last_clean":"2024-05-09T09:08:36.919162+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:08:36.919162+0000","last_undegraded":"2024-05-09T09:08:36.919162+0000","last_fullsized":"2024-05-09T09:08:36.919162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672652+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:08:37.672652+0000","last_peered":"2024-05-09T09:08:37.672652+0000","last_clean":"2024-05-09T09:08:37.672652+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:08:37.672652+0000","last_undegraded":"2024-05-09T09:08:37.672652+0000","last_fullsized":"2024-05-09T09:08:37.672652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704543+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:08:36.704543+0000","last_peered":"2024-05-09T09:08:36.704543+0000","last_clean":"2024-05-09T09:08:36.704543+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:08:36.704543+0000","last_undegraded":"2024-05-09T09:08:36.704543+0000","last_fullsized":"2024-05-09T09:08:36.704543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707710+0000","last_change":"2024-05-09T09:08:36.707710+0000","last_active":"2024-05-09T09:08:36.707710+0000","last_peered":"2024-05-09T09:08:36.707710+0000","last_clean":"2024-05-09T09:08:36.707710+0000","last_became_active":"2024-05-09T09:08:36.707333+0000","last_became_peered":"2024-05-09T09:08:36.707333+0000","last_unstale":"2024-05-09T09:08:36.707710+0000","last_undegraded":"2024-05-09T09:08:36.707710+0000","last_fullsized":"2024-05-09T09:08:36.707710+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:08.495289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709005+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:08:36.709005+0000","last_peered":"2024-05-09T09:08:36.709005+0000","last_clean":"2024-05-09T09:08:36.709005+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:08:36.709005+0000","last_undegraded":"2024-05-09T09:08:36.709005+0000","last_fullsized":"2024-05-09T09:08:36.709005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705731+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:08:36.705731+0000","last_peered":"2024-05-09T09:08:36.705731+0000","last_clean":"2024-05-09T09:08:36.705731+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:08:36.705731+0000","last_undegraded":"2024-05-09T09:08:36.705731+0000","last_fullsized":"2024-05-09T09:08:36.705731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124059+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:08:37.124059+0000","last_peered":"2024-05-09T09:08:37.124059+0000","last_clean":"2024-05-09T09:08:37.124059+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:08:37.124059+0000","last_undegraded":"2024-05-09T09:08:37.124059+0000","last_fullsized":"2024-05-09T09:08:37.124059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919025+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:08:36.919025+0000","last_peered":"2024-05-09T09:08:36.919025+0000","last_clean":"2024-05-09T09:08:36.919025+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:08:36.919025+0000","last_undegraded":"2024-05-09T09:08:36.919025+0000","last_fullsized":"2024-05-09T09:08:36.919025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672494+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:08:37.672494+0000","last_peered":"2024-05-09T09:08:37.672494+0000","last_clean":"2024-05-09T09:08:37.672494+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:08:37.672494+0000","last_undegraded":"2024-05-09T09:08:37.672494+0000","last_fullsized":"2024-05-09T09:08:37.672494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70794,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":749,"num_read_kb":449,"num_write":173,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":208068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":522662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612657,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1176615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":526972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1119268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":518052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":579401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":519440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":586019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":518246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":523179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1683456,"data_stored":1114379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:09:10.093 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T09:09:10.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T09:09:10.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2827123871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1029703056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 -- 172.21.15.64:0/4284151241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8100770 msgr2=0x7fafb8100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 --2- 172.21.15.64:0/4284151241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8100770 0x7fafb8100b90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fafac009900 tx=0x7fafac02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 -- 172.21.15.64:0/4284151241 shutdown_connections 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 --2- 172.21.15.64:0/4284151241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 0x7fafb8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 --2- 172.21.15.64:0/4284151241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8101960 0x7fafb8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 --2- 172.21.15.64:0/4284151241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8100770 0x7fafb8100b90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 -- 172.21.15.64:0/4284151241 >> 172.21.15.64:0/4284151241 conn(0x7fafb80fbce0 msgr2=0x7fafb80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 -- 172.21.15.64:0/4284151241 shutdown_connections 2024-05-09T09:09:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 -- 172.21.15.64:0/4284151241 wait complete. 2024-05-09T09:09:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 Processor -- start 2024-05-09T09:09:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.235+0000 7fafbedab640 1 -- start start 2024-05-09T09:09:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8100770 0x7fafb8104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 0x7fafb8102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 0x7fafb8103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810c2a0 con 0x7fafb8101960 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810c120 con 0x7fafb8100770 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810c420 con 0x7fafb8102320 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 0x7fafb8102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 0x7fafb8102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39086/0 (socket says 172.21.15.64:39086) 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 -- 172.21.15.64:0/1547890277 learned_addr learned my addr 172.21.15.64:0/1547890277 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbd348640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 0x7fafb8103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbcb47640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8100770 0x7fafb8104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 -- 172.21.15.64:0/1547890277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 msgr2=0x7fafb8103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 0x7fafb8103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 -- 172.21.15.64:0/1547890277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8100770 msgr2=0x7fafb8104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8100770 0x7fafb8104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 -- 172.21.15.64:0/1547890277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb8103b10 con 0x7fafb8101960 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbcb47640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8100770 0x7fafb8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbd348640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 0x7fafb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb7fff640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 0x7fafb8102d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d950 tx=0x7fafa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4017030 con 0x7fafb8101960 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa4004020 con 0x7fafb8101960 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.236+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a1c10 con 0x7fafb8101960 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.237+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa40047a0 con 0x7fafb8101960 2024-05-09T09:09:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.237+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a2160 con 0x7fafb8101960 2024-05-09T09:09:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.238+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb81a2390 con 0x7fafb8101960 2024-05-09T09:09:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.239+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafa40041d0 con 0x7fafb8101960 2024-05-09T09:09:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.239+0000 7fafb5ffb640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fafa80a2b90 0x7fafa80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.239+0000 7fafbcb47640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fafa80a2b90 0x7fafa80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.240+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fafa40cf000 con 0x7fafb8101960 2024-05-09T09:09:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.240+0000 7fafbcb47640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fafa80a2b90 0x7fafa80a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fafac006010 tx=0x7fafac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:10.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.244+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fafa409bbd0 con 0x7fafb8101960 2024-05-09T09:09:10.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2827123871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:10.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1029703056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.428+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 --> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafb8069180 con 0x7fafa80a2b90 2024-05-09T09:09:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.437+0000 7fafb5ffb640 1 -- 172.21.15.64:0/1547890277 <== mgr.14152 v2:172.21.15.64:6800/2084062196 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7fafb8069180 con 0x7fafa80a2b90 2024-05-09T09:09:10.440 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:10.442 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fafa80a2b90 msgr2=0x7fafa80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fafa80a2b90 0x7fafa80a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fafac006010 tx=0x7fafac000bc0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 msgr2=0x7fafb8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 0x7fafb8102d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d950 tx=0x7fafa400de10 comp rx=0 tx=0).stop 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 shutdown_connections 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fafa80a2b90 0x7fafa80a5050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafb8102320 0x7fafb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb8101960 0x7fafb8102d40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 --2- 172.21.15.64:0/1547890277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafb8100770 0x7fafb8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 >> 172.21.15.64:0/1547890277 conn(0x7fafb80fbce0 msgr2=0x7fafb8068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 shutdown_connections 2024-05-09T09:09:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.441+0000 7fafbedab640 1 -- 172.21.15.64:0/1547890277 wait complete. 2024-05-09T09:09:10.451 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-05-09T09:09:09.135663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661396,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":875,"num_read_kb":558,"num_write":356,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392896,"kb_used_data":14580,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548928,"statfs":{"total":1151940427776,"available":1151538102272,"internally_reserved":0,"allocated":14929920,"data_stored":8227007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5745,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003023"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672739+0000","last_change":"2024-05-09T09:06:41.123548+0000","last_active":"2024-05-09T09:08:37.672739+0000","last_peered":"2024-05-09T09:08:37.672739+0000","last_clean":"2024-05-09T09:08:37.672739+0000","last_became_active":"2024-05-09T09:06:41.123181+0000","last_became_peered":"2024-05-09T09:06:41.123181+0000","last_unstale":"2024-05-09T09:08:37.672739+0000","last_undegraded":"2024-05-09T09:08:37.672739+0000","last_fullsized":"2024-05-09T09:08:37.672739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:06.427472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939357+0000","last_change":"2024-05-09T09:06:41.129731+0000","last_active":"2024-05-09T09:08:37.939357+0000","last_peered":"2024-05-09T09:08:37.939357+0000","last_clean":"2024-05-09T09:08:37.939357+0000","last_became_active":"2024-05-09T09:06:41.129086+0000","last_became_peered":"2024-05-09T09:06:41.129086+0000","last_unstale":"2024-05-09T09:08:37.939357+0000","last_undegraded":"2024-05-09T09:08:37.939357+0000","last_fullsized":"2024-05-09T09:08:37.939357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:26.707359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939211+0000","last_change":"2024-05-09T09:06:41.135691+0000","last_active":"2024-05-09T09:08:37.939211+0000","last_peered":"2024-05-09T09:08:37.939211+0000","last_clean":"2024-05-09T09:08:37.939211+0000","last_became_active":"2024-05-09T09:06:41.135274+0000","last_became_peered":"2024-05-09T09:06:41.135274+0000","last_unstale":"2024-05-09T09:08:37.939211+0000","last_undegraded":"2024-05-09T09:08:37.939211+0000","last_fullsized":"2024-05-09T09:08:37.939211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:17.558839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704349+0000","last_change":"2024-05-09T09:06:41.138467+0000","last_active":"2024-05-09T09:08:36.704349+0000","last_peered":"2024-05-09T09:08:36.704349+0000","last_clean":"2024-05-09T09:08:36.704349+0000","last_became_active":"2024-05-09T09:06:41.136011+0000","last_became_peered":"2024-05-09T09:06:41.136011+0000","last_unstale":"2024-05-09T09:08:36.704349+0000","last_undegraded":"2024-05-09T09:08:36.704349+0000","last_fullsized":"2024-05-09T09:08:36.704349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:46.746078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579167+0000","last_change":"2024-05-09T09:06:41.130274+0000","last_active":"2024-05-09T09:08:37.579167+0000","last_peered":"2024-05-09T09:08:37.579167+0000","last_clean":"2024-05-09T09:08:37.579167+0000","last_became_active":"2024-05-09T09:06:41.129871+0000","last_became_peered":"2024-05-09T09:06:41.129871+0000","last_unstale":"2024-05-09T09:08:37.579167+0000","last_undegraded":"2024-05-09T09:08:37.579167+0000","last_fullsized":"2024-05-09T09:08:37.579167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:53.468482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846040+0000","last_change":"2024-05-09T09:06:41.126529+0000","last_active":"2024-05-09T09:08:36.846040+0000","last_peered":"2024-05-09T09:08:36.846040+0000","last_clean":"2024-05-09T09:08:36.846040+0000","last_became_active":"2024-05-09T09:06:41.126204+0000","last_became_peered":"2024-05-09T09:06:41.126204+0000","last_unstale":"2024-05-09T09:08:36.846040+0000","last_undegraded":"2024-05-09T09:08:36.846040+0000","last_fullsized":"2024-05-09T09:08:36.846040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:52.115687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708968+0000","last_change":"2024-05-09T09:06:41.132248+0000","last_active":"2024-05-09T09:08:37.708968+0000","last_peered":"2024-05-09T09:08:37.708968+0000","last_clean":"2024-05-09T09:08:37.708968+0000","last_became_active":"2024-05-09T09:06:41.131785+0000","last_became_peered":"2024-05-09T09:06:41.131785+0000","last_unstale":"2024-05-09T09:08:37.708968+0000","last_undegraded":"2024-05-09T09:08:37.708968+0000","last_fullsized":"2024-05-09T09:08:37.708968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:17.909486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706275+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:08:36.706275+0000","last_peered":"2024-05-09T09:08:36.706275+0000","last_clean":"2024-05-09T09:08:36.706275+0000","last_became_active":"2024-05-09T09:06:41.133781+0000","last_became_peered":"2024-05-09T09:06:41.133781+0000","last_unstale":"2024-05-09T09:08:36.706275+0000","last_undegraded":"2024-05-09T09:08:36.706275+0000","last_fullsized":"2024-05-09T09:08:36.706275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:52.734756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124251+0000","last_change":"2024-05-09T09:06:41.143185+0000","last_active":"2024-05-09T09:08:37.124251+0000","last_peered":"2024-05-09T09:08:37.124251+0000","last_clean":"2024-05-09T09:08:37.124251+0000","last_became_active":"2024-05-09T09:06:41.142823+0000","last_became_peered":"2024-05-09T09:06:41.142823+0000","last_unstale":"2024-05-09T09:08:37.124251+0000","last_undegraded":"2024-05-09T09:08:37.124251+0000","last_fullsized":"2024-05-09T09:08:37.124251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:47.551634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846387+0000","last_change":"2024-05-09T09:06:41.125940+0000","last_active":"2024-05-09T09:08:36.846387+0000","last_peered":"2024-05-09T09:08:36.846387+0000","last_clean":"2024-05-09T09:08:36.846387+0000","last_became_active":"2024-05-09T09:06:41.123588+0000","last_became_peered":"2024-05-09T09:06:41.123588+0000","last_unstale":"2024-05-09T09:08:36.846387+0000","last_undegraded":"2024-05-09T09:08:36.846387+0000","last_fullsized":"2024-05-09T09:08:36.846387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:26.016894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709714+0000","last_change":"2024-05-09T09:06:41.145112+0000","last_active":"2024-05-09T09:08:37.709714+0000","last_peered":"2024-05-09T09:08:37.709714+0000","last_clean":"2024-05-09T09:08:37.709714+0000","last_became_active":"2024-05-09T09:06:41.144774+0000","last_became_peered":"2024-05-09T09:06:41.144774+0000","last_unstale":"2024-05-09T09:08:37.709714+0000","last_undegraded":"2024-05-09T09:08:37.709714+0000","last_fullsized":"2024-05-09T09:08:37.709714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:21.608954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706384+0000","last_change":"2024-05-09T09:07:36.317979+0000","last_active":"2024-05-09T09:08:36.706384+0000","last_peered":"2024-05-09T09:08:36.706384+0000","last_clean":"2024-05-09T09:08:36.706384+0000","last_became_active":"2024-05-09T09:07:36.317572+0000","last_became_peered":"2024-05-09T09:07:36.317572+0000","last_unstale":"2024-05-09T09:08:36.706384+0000","last_undegraded":"2024-05-09T09:08:36.706384+0000","last_fullsized":"2024-05-09T09:08:36.706384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:45.499023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919161+0000","last_change":"2024-05-09T09:06:41.131915+0000","last_active":"2024-05-09T09:08:36.919161+0000","last_peered":"2024-05-09T09:08:36.919161+0000","last_clean":"2024-05-09T09:08:36.919161+0000","last_became_active":"2024-05-09T09:06:41.130755+0000","last_became_peered":"2024-05-09T09:06:41.130755+0000","last_unstale":"2024-05-09T09:08:36.919161+0000","last_undegraded":"2024-05-09T09:08:36.919161+0000","last_fullsized":"2024-05-09T09:08:36.919161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:06.525737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938799+0000","last_change":"2024-05-09T09:06:41.136092+0000","last_active":"2024-05-09T09:08:37.938799+0000","last_peered":"2024-05-09T09:08:37.938799+0000","last_clean":"2024-05-09T09:08:37.938799+0000","last_became_active":"2024-05-09T09:06:41.135777+0000","last_became_peered":"2024-05-09T09:06:41.135777+0000","last_unstale":"2024-05-09T09:08:37.938799+0000","last_undegraded":"2024-05-09T09:08:37.938799+0000","last_fullsized":"2024-05-09T09:08:37.938799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.437256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938909+0000","last_change":"2024-05-09T09:06:41.134059+0000","last_active":"2024-05-09T09:08:37.938909+0000","last_peered":"2024-05-09T09:08:37.938909+0000","last_clean":"2024-05-09T09:08:37.938909+0000","last_became_active":"2024-05-09T09:06:41.133691+0000","last_became_peered":"2024-05-09T09:06:41.133691+0000","last_unstale":"2024-05-09T09:08:37.938909+0000","last_undegraded":"2024-05-09T09:08:37.938909+0000","last_fullsized":"2024-05-09T09:08:37.938909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:23.247576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123822+0000","last_change":"2024-05-09T09:06:41.130912+0000","last_active":"2024-05-09T09:08:37.123822+0000","last_peered":"2024-05-09T09:08:37.123822+0000","last_clean":"2024-05-09T09:08:37.123822+0000","last_became_active":"2024-05-09T09:06:41.130351+0000","last_became_peered":"2024-05-09T09:06:41.130351+0000","last_unstale":"2024-05-09T09:08:37.123822+0000","last_undegraded":"2024-05-09T09:08:37.123822+0000","last_fullsized":"2024-05-09T09:08:37.123822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:05.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672730+0000","last_change":"2024-05-09T09:06:41.129072+0000","last_active":"2024-05-09T09:08:37.672730+0000","last_peered":"2024-05-09T09:08:37.672730+0000","last_clean":"2024-05-09T09:08:37.672730+0000","last_became_active":"2024-05-09T09:06:41.128696+0000","last_became_peered":"2024-05-09T09:06:41.128696+0000","last_unstale":"2024-05-09T09:08:37.672730+0000","last_undegraded":"2024-05-09T09:08:37.672730+0000","last_fullsized":"2024-05-09T09:08:37.672730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:58.674169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706528+0000","last_change":"2024-05-09T09:06:41.123255+0000","last_active":"2024-05-09T09:08:36.706528+0000","last_peered":"2024-05-09T09:08:36.706528+0000","last_clean":"2024-05-09T09:08:36.706528+0000","last_became_active":"2024-05-09T09:06:41.122889+0000","last_became_peered":"2024-05-09T09:06:41.122889+0000","last_unstale":"2024-05-09T09:08:36.706528+0000","last_undegraded":"2024-05-09T09:08:36.706528+0000","last_fullsized":"2024-05-09T09:08:36.706528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:12.554506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704353+0000","last_change":"2024-05-09T09:06:41.123624+0000","last_active":"2024-05-09T09:08:36.704353+0000","last_peered":"2024-05-09T09:08:36.704353+0000","last_clean":"2024-05-09T09:08:36.704353+0000","last_became_active":"2024-05-09T09:06:41.123274+0000","last_became_peered":"2024-05-09T09:06:41.123274+0000","last_unstale":"2024-05-09T09:08:36.704353+0000","last_undegraded":"2024-05-09T09:08:36.704353+0000","last_fullsized":"2024-05-09T09:08:36.704353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:15.525861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703943+0000","last_change":"2024-05-09T09:06:45.179529+0000","last_active":"2024-05-09T09:08:36.703943+0000","last_peered":"2024-05-09T09:08:36.703943+0000","last_clean":"2024-05-09T09:08:36.703943+0000","last_became_active":"2024-05-09T09:06:45.179104+0000","last_became_peered":"2024-05-09T09:06:45.179104+0000","last_unstale":"2024-05-09T09:08:36.703943+0000","last_undegraded":"2024-05-09T09:08:36.703943+0000","last_fullsized":"2024-05-09T09:08:36.703943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:14.506062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706328+0000","last_change":"2024-05-09T09:06:40.109816+0000","last_active":"2024-05-09T09:08:36.706328+0000","last_peered":"2024-05-09T09:08:36.706328+0000","last_clean":"2024-05-09T09:08:36.706328+0000","last_became_active":"2024-05-09T09:06:40.109519+0000","last_became_peered":"2024-05-09T09:06:40.109519+0000","last_unstale":"2024-05-09T09:08:36.706328+0000","last_undegraded":"2024-05-09T09:08:36.706328+0000","last_fullsized":"2024-05-09T09:08:36.706328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:10.151798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.710323+0000","last_change":"2024-05-09T09:08:36.710323+0000","last_active":"2024-05-09T09:08:36.710323+0000","last_peered":"2024-05-09T09:08:36.710323+0000","last_clean":"2024-05-09T09:08:36.710323+0000","last_became_active":"2024-05-09T09:08:36.709965+0000","last_became_peered":"2024-05-09T09:08:36.709965+0000","last_unstale":"2024-05-09T09:08:36.710323+0000","last_undegraded":"2024-05-09T09:08:36.710323+0000","last_fullsized":"2024-05-09T09:08:36.710323+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:01.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703719+0000","last_change":"2024-05-09T09:06:45.185334+0000","last_active":"2024-05-09T09:08:36.703719+0000","last_peered":"2024-05-09T09:08:36.703719+0000","last_clean":"2024-05-09T09:08:36.703719+0000","last_became_active":"2024-05-09T09:06:45.184822+0000","last_became_peered":"2024-05-09T09:06:45.184822+0000","last_unstale":"2024-05-09T09:08:36.703719+0000","last_undegraded":"2024-05-09T09:08:36.703719+0000","last_fullsized":"2024-05-09T09:08:36.703719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:06.292543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707362+0000","last_change":"2024-05-09T09:06:40.113230+0000","last_active":"2024-05-09T09:08:36.707362+0000","last_peered":"2024-05-09T09:08:36.707362+0000","last_clean":"2024-05-09T09:08:36.707362+0000","last_became_active":"2024-05-09T09:06:40.112917+0000","last_became_peered":"2024-05-09T09:06:40.112917+0000","last_unstale":"2024-05-09T09:08:36.707362+0000","last_undegraded":"2024-05-09T09:08:36.707362+0000","last_fullsized":"2024-05-09T09:08:36.707362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:41.687427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939378+0000","last_change":"2024-05-09T09:06:41.144297+0000","last_active":"2024-05-09T09:08:37.939378+0000","last_peered":"2024-05-09T09:08:37.939378+0000","last_clean":"2024-05-09T09:08:37.939378+0000","last_became_active":"2024-05-09T09:06:41.143989+0000","last_became_peered":"2024-05-09T09:06:41.143989+0000","last_unstale":"2024-05-09T09:08:37.939378+0000","last_undegraded":"2024-05-09T09:08:37.939378+0000","last_fullsized":"2024-05-09T09:08:37.939378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:06.423537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703834+0000","last_change":"2024-05-09T09:06:45.187305+0000","last_active":"2024-05-09T09:08:36.703834+0000","last_peered":"2024-05-09T09:08:36.703834+0000","last_clean":"2024-05-09T09:08:36.703834+0000","last_became_active":"2024-05-09T09:06:45.184990+0000","last_became_peered":"2024-05-09T09:06:45.184990+0000","last_unstale":"2024-05-09T09:08:36.703834+0000","last_undegraded":"2024-05-09T09:08:36.703834+0000","last_fullsized":"2024-05-09T09:08:36.703834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:17.404299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579531+0000","last_change":"2024-05-09T09:06:40.114089+0000","last_active":"2024-05-09T09:08:37.579531+0000","last_peered":"2024-05-09T09:08:37.579531+0000","last_clean":"2024-05-09T09:08:37.579531+0000","last_became_active":"2024-05-09T09:06:40.113730+0000","last_became_peered":"2024-05-09T09:06:40.113730+0000","last_unstale":"2024-05-09T09:08:37.579531+0000","last_undegraded":"2024-05-09T09:08:37.579531+0000","last_fullsized":"2024-05-09T09:08:37.579531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.550384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123893+0000","last_change":"2024-05-09T09:06:41.140335+0000","last_active":"2024-05-09T09:08:37.123893+0000","last_peered":"2024-05-09T09:08:37.123893+0000","last_clean":"2024-05-09T09:08:37.123893+0000","last_became_active":"2024-05-09T09:06:41.138871+0000","last_became_peered":"2024-05-09T09:06:41.138871+0000","last_unstale":"2024-05-09T09:08:37.123893+0000","last_undegraded":"2024-05-09T09:08:37.123893+0000","last_fullsized":"2024-05-09T09:08:37.123893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703725+0000","last_change":"2024-05-09T09:06:45.180241+0000","last_active":"2024-05-09T09:08:36.703725+0000","last_peered":"2024-05-09T09:08:36.703725+0000","last_clean":"2024-05-09T09:08:36.703725+0000","last_became_active":"2024-05-09T09:06:45.179779+0000","last_became_peered":"2024-05-09T09:06:45.179779+0000","last_unstale":"2024-05-09T09:08:36.703725+0000","last_undegraded":"2024-05-09T09:08:36.703725+0000","last_fullsized":"2024-05-09T09:08:36.703725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:04.466164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579279+0000","last_change":"2024-05-09T09:06:40.113755+0000","last_active":"2024-05-09T09:08:37.579279+0000","last_peered":"2024-05-09T09:08:37.579279+0000","last_clean":"2024-05-09T09:08:37.579279+0000","last_became_active":"2024-05-09T09:06:40.113406+0000","last_became_peered":"2024-05-09T09:06:40.113406+0000","last_unstale":"2024-05-09T09:08:37.579279+0000","last_undegraded":"2024-05-09T09:08:37.579279+0000","last_fullsized":"2024-05-09T09:08:37.579279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:56.142350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708974+0000","last_change":"2024-05-09T09:06:41.146005+0000","last_active":"2024-05-09T09:08:36.708974+0000","last_peered":"2024-05-09T09:08:36.708974+0000","last_clean":"2024-05-09T09:08:36.708974+0000","last_became_active":"2024-05-09T09:06:41.145671+0000","last_became_peered":"2024-05-09T09:06:41.145671+0000","last_unstale":"2024-05-09T09:08:36.708974+0000","last_undegraded":"2024-05-09T09:08:36.708974+0000","last_fullsized":"2024-05-09T09:08:36.708974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.617099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704172+0000","last_change":"2024-05-09T09:06:45.171744+0000","last_active":"2024-05-09T09:08:36.704172+0000","last_peered":"2024-05-09T09:08:36.704172+0000","last_clean":"2024-05-09T09:08:36.704172+0000","last_became_active":"2024-05-09T09:06:45.171294+0000","last_became_peered":"2024-05-09T09:06:45.171294+0000","last_unstale":"2024-05-09T09:08:36.704172+0000","last_undegraded":"2024-05-09T09:08:36.704172+0000","last_fullsized":"2024-05-09T09:08:36.704172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:33.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:06.410475+0000","last_change":"2024-05-09T09:06:40.112531+0000","last_active":"2024-05-09T09:09:06.410475+0000","last_peered":"2024-05-09T09:09:06.410475+0000","last_clean":"2024-05-09T09:09:06.410475+0000","last_became_active":"2024-05-09T09:06:40.112223+0000","last_became_peered":"2024-05-09T09:06:40.112223+0000","last_unstale":"2024-05-09T09:09:06.410475+0000","last_undegraded":"2024-05-09T09:09:06.410475+0000","last_fullsized":"2024-05-09T09:09:06.410475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:16.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1099,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579170+0000","last_change":"2024-05-09T09:07:36.615226+0000","last_active":"2024-05-09T09:08:37.579170+0000","last_peered":"2024-05-09T09:08:37.579170+0000","last_clean":"2024-05-09T09:08:37.579170+0000","last_became_active":"2024-05-09T09:07:36.614762+0000","last_became_peered":"2024-05-09T09:07:36.614762+0000","last_unstale":"2024-05-09T09:08:37.579170+0000","last_undegraded":"2024-05-09T09:08:37.579170+0000","last_fullsized":"2024-05-09T09:08:37.579170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:29.045379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704747+0000","last_change":"2024-05-09T09:06:45.171864+0000","last_active":"2024-05-09T09:08:36.704747+0000","last_peered":"2024-05-09T09:08:36.704747+0000","last_clean":"2024-05-09T09:08:36.704747+0000","last_became_active":"2024-05-09T09:06:45.171426+0000","last_became_peered":"2024-05-09T09:06:45.171426+0000","last_unstale":"2024-05-09T09:08:36.704747+0000","last_undegraded":"2024-05-09T09:08:36.704747+0000","last_fullsized":"2024-05-09T09:08:36.704747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:16:37.968755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.845978+0000","last_change":"2024-05-09T09:06:40.117287+0000","last_active":"2024-05-09T09:08:36.845978+0000","last_peered":"2024-05-09T09:08:36.845978+0000","last_clean":"2024-05-09T09:08:36.845978+0000","last_became_active":"2024-05-09T09:06:40.116925+0000","last_became_peered":"2024-05-09T09:06:40.116925+0000","last_unstale":"2024-05-09T09:08:36.845978+0000","last_undegraded":"2024-05-09T09:08:36.845978+0000","last_fullsized":"2024-05-09T09:08:36.845978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:05.126215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709278+0000","last_change":"2024-05-09T09:06:41.131035+0000","last_active":"2024-05-09T09:08:36.709278+0000","last_peered":"2024-05-09T09:08:36.709278+0000","last_clean":"2024-05-09T09:08:36.709278+0000","last_became_active":"2024-05-09T09:06:41.130658+0000","last_became_peered":"2024-05-09T09:06:41.130658+0000","last_unstale":"2024-05-09T09:08:36.709278+0000","last_undegraded":"2024-05-09T09:08:36.709278+0000","last_fullsized":"2024-05-09T09:08:36.709278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:03.964082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672121+0000","last_change":"2024-05-09T09:06:45.186672+0000","last_active":"2024-05-09T09:08:37.672121+0000","last_peered":"2024-05-09T09:08:37.672121+0000","last_clean":"2024-05-09T09:08:37.672121+0000","last_became_active":"2024-05-09T09:06:45.186211+0000","last_became_peered":"2024-05-09T09:06:45.186211+0000","last_unstale":"2024-05-09T09:08:37.672121+0000","last_undegraded":"2024-05-09T09:08:37.672121+0000","last_fullsized":"2024-05-09T09:08:37.672121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:34.463590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.918958+0000","last_change":"2024-05-09T09:06:40.112483+0000","last_active":"2024-05-09T09:08:36.918958+0000","last_peered":"2024-05-09T09:08:36.918958+0000","last_clean":"2024-05-09T09:08:36.918958+0000","last_became_active":"2024-05-09T09:06:40.112175+0000","last_became_peered":"2024-05-09T09:06:40.112175+0000","last_unstale":"2024-05-09T09:08:36.918958+0000","last_undegraded":"2024-05-09T09:08:36.918958+0000","last_fullsized":"2024-05-09T09:08:36.918958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:58:22.230275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705457+0000","last_change":"2024-05-09T09:06:41.126414+0000","last_active":"2024-05-09T09:08:36.705457+0000","last_peered":"2024-05-09T09:08:36.705457+0000","last_clean":"2024-05-09T09:08:36.705457+0000","last_became_active":"2024-05-09T09:06:41.126083+0000","last_became_peered":"2024-05-09T09:06:41.126083+0000","last_unstale":"2024-05-09T09:08:36.705457+0000","last_undegraded":"2024-05-09T09:08:36.705457+0000","last_fullsized":"2024-05-09T09:08:36.705457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:21.941776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938610+0000","last_change":"2024-05-09T09:06:45.185001+0000","last_active":"2024-05-09T09:08:37.938610+0000","last_peered":"2024-05-09T09:08:37.938610+0000","last_clean":"2024-05-09T09:08:37.938610+0000","last_became_active":"2024-05-09T09:06:45.184617+0000","last_became_peered":"2024-05-09T09:06:45.184617+0000","last_unstale":"2024-05-09T09:08:37.938610+0000","last_undegraded":"2024-05-09T09:08:37.938610+0000","last_fullsized":"2024-05-09T09:08:37.938610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:50.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708907+0000","last_change":"2024-05-09T09:06:40.110578+0000","last_active":"2024-05-09T09:08:36.708907+0000","last_peered":"2024-05-09T09:08:36.708907+0000","last_clean":"2024-05-09T09:08:36.708907+0000","last_became_active":"2024-05-09T09:06:40.110185+0000","last_became_peered":"2024-05-09T09:06:40.110185+0000","last_unstale":"2024-05-09T09:08:36.708907+0000","last_undegraded":"2024-05-09T09:08:36.708907+0000","last_fullsized":"2024-05-09T09:08:36.708907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:17.018554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705674+0000","last_change":"2024-05-09T09:06:41.126433+0000","last_active":"2024-05-09T09:08:36.705674+0000","last_peered":"2024-05-09T09:08:36.705674+0000","last_clean":"2024-05-09T09:08:36.705674+0000","last_became_active":"2024-05-09T09:06:41.126107+0000","last_became_peered":"2024-05-09T09:06:41.126107+0000","last_unstale":"2024-05-09T09:08:36.705674+0000","last_undegraded":"2024-05-09T09:08:36.705674+0000","last_fullsized":"2024-05-09T09:08:36.705674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:04.509149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706934+0000","last_change":"2024-05-09T09:06:45.188104+0000","last_active":"2024-05-09T09:08:36.706934+0000","last_peered":"2024-05-09T09:08:36.706934+0000","last_clean":"2024-05-09T09:08:36.706934+0000","last_became_active":"2024-05-09T09:06:45.187693+0000","last_became_peered":"2024-05-09T09:06:45.187693+0000","last_unstale":"2024-05-09T09:08:36.706934+0000","last_undegraded":"2024-05-09T09:08:36.706934+0000","last_fullsized":"2024-05-09T09:08:36.706934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:45:56.200908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.411229+0000","last_change":"2024-05-09T09:06:40.112245+0000","last_active":"2024-05-09T09:09:07.411229+0000","last_peered":"2024-05-09T09:09:07.411229+0000","last_clean":"2024-05-09T09:09:07.411229+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:09:07.411229+0000","last_undegraded":"2024-05-09T09:09:07.411229+0000","last_fullsized":"2024-05-09T09:09:07.411229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:28.602307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124177+0000","last_change":"2024-05-09T09:06:41.138241+0000","last_active":"2024-05-09T09:08:37.124177+0000","last_peered":"2024-05-09T09:08:37.124177+0000","last_clean":"2024-05-09T09:08:37.124177+0000","last_became_active":"2024-05-09T09:06:41.134448+0000","last_became_peered":"2024-05-09T09:06:41.134448+0000","last_unstale":"2024-05-09T09:08:37.124177+0000","last_undegraded":"2024-05-09T09:08:37.124177+0000","last_fullsized":"2024-05-09T09:08:37.124177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:36.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705636+0000","last_change":"2024-05-09T09:06:45.173557+0000","last_active":"2024-05-09T09:08:36.705636+0000","last_peered":"2024-05-09T09:08:36.705636+0000","last_clean":"2024-05-09T09:08:36.705636+0000","last_became_active":"2024-05-09T09:06:45.172653+0000","last_became_peered":"2024-05-09T09:06:45.172653+0000","last_unstale":"2024-05-09T09:08:36.705636+0000","last_undegraded":"2024-05-09T09:08:36.705636+0000","last_fullsized":"2024-05-09T09:08:36.705636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:10.972568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709600+0000","last_change":"2024-05-09T09:06:40.113073+0000","last_active":"2024-05-09T09:08:37.709600+0000","last_peered":"2024-05-09T09:08:37.709600+0000","last_clean":"2024-05-09T09:08:37.709600+0000","last_became_active":"2024-05-09T09:06:40.112408+0000","last_became_peered":"2024-05-09T09:06:40.112408+0000","last_unstale":"2024-05-09T09:08:37.709600+0000","last_undegraded":"2024-05-09T09:08:37.709600+0000","last_fullsized":"2024-05-09T09:08:37.709600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:00.427554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939354+0000","last_change":"2024-05-09T09:06:41.131693+0000","last_active":"2024-05-09T09:08:37.939354+0000","last_peered":"2024-05-09T09:08:37.939354+0000","last_clean":"2024-05-09T09:08:37.939354+0000","last_became_active":"2024-05-09T09:06:41.131315+0000","last_became_peered":"2024-05-09T09:06:41.131315+0000","last_unstale":"2024-05-09T09:08:37.939354+0000","last_undegraded":"2024-05-09T09:08:37.939354+0000","last_fullsized":"2024-05-09T09:08:37.939354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:47.934470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704180+0000","last_change":"2024-05-09T09:07:36.436587+0000","last_active":"2024-05-09T09:08:36.704180+0000","last_peered":"2024-05-09T09:08:36.704180+0000","last_clean":"2024-05-09T09:08:36.704180+0000","last_became_active":"2024-05-09T09:07:36.436083+0000","last_became_peered":"2024-05-09T09:07:36.436083+0000","last_unstale":"2024-05-09T09:08:36.704180+0000","last_undegraded":"2024-05-09T09:08:36.704180+0000","last_fullsized":"2024-05-09T09:08:36.704180+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:02:34.904307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709115+0000","last_change":"2024-05-09T09:06:40.108860+0000","last_active":"2024-05-09T09:08:36.709115+0000","last_peered":"2024-05-09T09:08:36.709115+0000","last_clean":"2024-05-09T09:08:36.709115+0000","last_became_active":"2024-05-09T09:06:40.108526+0000","last_became_peered":"2024-05-09T09:06:40.108526+0000","last_unstale":"2024-05-09T09:08:36.709115+0000","last_undegraded":"2024-05-09T09:08:36.709115+0000","last_fullsized":"2024-05-09T09:08:36.709115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:17.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124078+0000","last_change":"2024-05-09T09:06:41.141260+0000","last_active":"2024-05-09T09:08:37.124078+0000","last_peered":"2024-05-09T09:08:37.124078+0000","last_clean":"2024-05-09T09:08:37.124078+0000","last_became_active":"2024-05-09T09:06:41.140054+0000","last_became_peered":"2024-05-09T09:06:41.140054+0000","last_unstale":"2024-05-09T09:08:37.124078+0000","last_undegraded":"2024-05-09T09:08:37.124078+0000","last_fullsized":"2024-05-09T09:08:37.124078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:00.700493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.845799+0000","last_change":"2024-05-09T09:07:36.324821+0000","last_active":"2024-05-09T09:08:36.845799+0000","last_peered":"2024-05-09T09:08:36.845799+0000","last_clean":"2024-05-09T09:08:36.845799+0000","last_became_active":"2024-05-09T09:07:36.324478+0000","last_became_peered":"2024-05-09T09:07:36.324478+0000","last_unstale":"2024-05-09T09:08:36.845799+0000","last_undegraded":"2024-05-09T09:08:36.845799+0000","last_fullsized":"2024-05-09T09:08:36.845799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:25.476000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":320,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.437201+0000","last_change":"2024-05-09T09:06:40.107796+0000","last_active":"2024-05-09T09:09:07.437201+0000","last_peered":"2024-05-09T09:09:07.437201+0000","last_clean":"2024-05-09T09:09:07.437201+0000","last_became_active":"2024-05-09T09:06:40.107427+0000","last_became_peered":"2024-05-09T09:06:40.107427+0000","last_unstale":"2024-05-09T09:09:07.437201+0000","last_undegraded":"2024-05-09T09:09:07.437201+0000","last_fullsized":"2024-05-09T09:09:07.437201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:04.221035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66230,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579364+0000","last_change":"2024-05-09T09:06:41.133705+0000","last_active":"2024-05-09T09:08:37.579364+0000","last_peered":"2024-05-09T09:08:37.579364+0000","last_clean":"2024-05-09T09:08:37.579364+0000","last_became_active":"2024-05-09T09:06:41.133338+0000","last_became_peered":"2024-05-09T09:06:41.133338+0000","last_unstale":"2024-05-09T09:08:37.579364+0000","last_undegraded":"2024-05-09T09:08:37.579364+0000","last_fullsized":"2024-05-09T09:08:37.579364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:30.089306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709325+0000","last_change":"2024-05-09T09:06:45.189724+0000","last_active":"2024-05-09T09:08:36.709325+0000","last_peered":"2024-05-09T09:08:36.709325+0000","last_clean":"2024-05-09T09:08:36.709325+0000","last_became_active":"2024-05-09T09:06:45.189397+0000","last_became_peered":"2024-05-09T09:06:45.189397+0000","last_unstale":"2024-05-09T09:08:36.709325+0000","last_undegraded":"2024-05-09T09:08:36.709325+0000","last_fullsized":"2024-05-09T09:08:36.709325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:03.731965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704391+0000","last_change":"2024-05-09T09:06:40.114100+0000","last_active":"2024-05-09T09:08:36.704391+0000","last_peered":"2024-05-09T09:08:36.704391+0000","last_clean":"2024-05-09T09:08:36.704391+0000","last_became_active":"2024-05-09T09:06:40.113715+0000","last_became_peered":"2024-05-09T09:06:40.113715+0000","last_unstale":"2024-05-09T09:08:36.704391+0000","last_undegraded":"2024-05-09T09:08:36.704391+0000","last_fullsized":"2024-05-09T09:08:36.704391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:38.593983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919138+0000","last_change":"2024-05-09T09:06:41.126225+0000","last_active":"2024-05-09T09:08:36.919138+0000","last_peered":"2024-05-09T09:08:36.919138+0000","last_clean":"2024-05-09T09:08:36.919138+0000","last_became_active":"2024-05-09T09:06:41.125918+0000","last_became_peered":"2024-05-09T09:06:41.125918+0000","last_unstale":"2024-05-09T09:08:36.919138+0000","last_undegraded":"2024-05-09T09:08:36.919138+0000","last_fullsized":"2024-05-09T09:08:36.919138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:26.406525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672057+0000","last_change":"2024-05-09T09:06:45.185852+0000","last_active":"2024-05-09T09:08:37.672057+0000","last_peered":"2024-05-09T09:08:37.672057+0000","last_clean":"2024-05-09T09:08:37.672057+0000","last_became_active":"2024-05-09T09:06:45.185425+0000","last_became_peered":"2024-05-09T09:06:45.185425+0000","last_unstale":"2024-05-09T09:08:37.672057+0000","last_undegraded":"2024-05-09T09:08:37.672057+0000","last_fullsized":"2024-05-09T09:08:37.672057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:28.099534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709159+0000","last_change":"2024-05-09T09:06:40.112471+0000","last_active":"2024-05-09T09:08:36.709159+0000","last_peered":"2024-05-09T09:08:36.709159+0000","last_clean":"2024-05-09T09:08:36.709159+0000","last_became_active":"2024-05-09T09:06:40.112131+0000","last_became_peered":"2024-05-09T09:06:40.112131+0000","last_unstale":"2024-05-09T09:08:36.709159+0000","last_undegraded":"2024-05-09T09:08:36.709159+0000","last_fullsized":"2024-05-09T09:08:36.709159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:44.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709537+0000","last_change":"2024-05-09T09:06:41.130674+0000","last_active":"2024-05-09T09:08:37.709537+0000","last_peered":"2024-05-09T09:08:37.709537+0000","last_clean":"2024-05-09T09:08:37.709537+0000","last_became_active":"2024-05-09T09:06:41.130349+0000","last_became_peered":"2024-05-09T09:06:41.130349+0000","last_unstale":"2024-05-09T09:08:37.709537+0000","last_undegraded":"2024-05-09T09:08:37.709537+0000","last_fullsized":"2024-05-09T09:08:37.709537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:01.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706621+0000","last_change":"2024-05-09T09:06:45.180327+0000","last_active":"2024-05-09T09:08:36.706621+0000","last_peered":"2024-05-09T09:08:36.706621+0000","last_clean":"2024-05-09T09:08:36.706621+0000","last_became_active":"2024-05-09T09:06:45.178437+0000","last_became_peered":"2024-05-09T09:06:45.178437+0000","last_unstale":"2024-05-09T09:08:36.706621+0000","last_undegraded":"2024-05-09T09:08:36.706621+0000","last_fullsized":"2024-05-09T09:08:36.706621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:57.281016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:07.411204+0000","last_change":"2024-05-09T09:06:40.113221+0000","last_active":"2024-05-09T09:09:07.411204+0000","last_peered":"2024-05-09T09:09:07.411204+0000","last_clean":"2024-05-09T09:09:07.411204+0000","last_became_active":"2024-05-09T09:06:40.112870+0000","last_became_peered":"2024-05-09T09:06:40.112870+0000","last_unstale":"2024-05-09T09:09:07.411204+0000","last_undegraded":"2024-05-09T09:09:07.411204+0000","last_fullsized":"2024-05-09T09:09:07.411204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:50.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709029+0000","last_change":"2024-05-09T09:06:41.132400+0000","last_active":"2024-05-09T09:08:37.709029+0000","last_peered":"2024-05-09T09:08:37.709029+0000","last_clean":"2024-05-09T09:08:37.709029+0000","last_became_active":"2024-05-09T09:06:41.132071+0000","last_became_peered":"2024-05-09T09:06:41.132071+0000","last_unstale":"2024-05-09T09:08:37.709029+0000","last_undegraded":"2024-05-09T09:08:37.709029+0000","last_fullsized":"2024-05-09T09:08:37.709029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:11.792958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919195+0000","last_change":"2024-05-09T09:07:36.323604+0000","last_active":"2024-05-09T09:08:36.919195+0000","last_peered":"2024-05-09T09:08:36.919195+0000","last_clean":"2024-05-09T09:08:36.919195+0000","last_became_active":"2024-05-09T09:07:36.323254+0000","last_became_peered":"2024-05-09T09:07:36.323254+0000","last_unstale":"2024-05-09T09:08:36.919195+0000","last_undegraded":"2024-05-09T09:08:36.919195+0000","last_fullsized":"2024-05-09T09:08:36.919195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:31:18.746847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578879+0000","last_change":"2024-05-09T09:06:40.117232+0000","last_active":"2024-05-09T09:08:37.578879+0000","last_peered":"2024-05-09T09:08:37.578879+0000","last_clean":"2024-05-09T09:08:37.578879+0000","last_became_active":"2024-05-09T09:06:40.116401+0000","last_became_peered":"2024-05-09T09:06:40.116401+0000","last_unstale":"2024-05-09T09:08:37.578879+0000","last_undegraded":"2024-05-09T09:08:37.578879+0000","last_fullsized":"2024-05-09T09:08:37.578879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:51.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708634+0000","last_change":"2024-05-09T09:06:41.146715+0000","last_active":"2024-05-09T09:08:36.708634+0000","last_peered":"2024-05-09T09:08:36.708634+0000","last_clean":"2024-05-09T09:08:36.708634+0000","last_became_active":"2024-05-09T09:06:41.146359+0000","last_became_peered":"2024-05-09T09:06:41.146359+0000","last_unstale":"2024-05-09T09:08:36.708634+0000","last_undegraded":"2024-05-09T09:08:36.708634+0000","last_fullsized":"2024-05-09T09:08:36.708634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:05.797937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708641+0000","last_change":"2024-05-09T09:06:45.182053+0000","last_active":"2024-05-09T09:08:36.708641+0000","last_peered":"2024-05-09T09:08:36.708641+0000","last_clean":"2024-05-09T09:08:36.708641+0000","last_became_active":"2024-05-09T09:06:45.181680+0000","last_became_peered":"2024-05-09T09:06:45.181680+0000","last_unstale":"2024-05-09T09:08:36.708641+0000","last_undegraded":"2024-05-09T09:08:36.708641+0000","last_fullsized":"2024-05-09T09:08:36.708641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:28.309909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672602+0000","last_change":"2024-05-09T09:07:36.331318+0000","last_active":"2024-05-09T09:08:37.672602+0000","last_peered":"2024-05-09T09:08:37.672602+0000","last_clean":"2024-05-09T09:08:37.672602+0000","last_became_active":"2024-05-09T09:07:36.323269+0000","last_became_peered":"2024-05-09T09:07:36.323269+0000","last_unstale":"2024-05-09T09:08:37.672602+0000","last_undegraded":"2024-05-09T09:08:37.672602+0000","last_fullsized":"2024-05-09T09:08:37.672602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:25:53.779933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672622+0000","last_change":"2024-05-09T09:06:41.131184+0000","last_active":"2024-05-09T09:08:37.672622+0000","last_peered":"2024-05-09T09:08:37.672622+0000","last_clean":"2024-05-09T09:08:37.672622+0000","last_became_active":"2024-05-09T09:06:41.130807+0000","last_became_peered":"2024-05-09T09:06:41.130807+0000","last_unstale":"2024-05-09T09:08:37.672622+0000","last_undegraded":"2024-05-09T09:08:37.672622+0000","last_fullsized":"2024-05-09T09:08:37.672622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:25.888596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578766+0000","last_change":"2024-05-09T09:06:45.185913+0000","last_active":"2024-05-09T09:08:37.578766+0000","last_peered":"2024-05-09T09:08:37.578766+0000","last_clean":"2024-05-09T09:08:37.578766+0000","last_became_active":"2024-05-09T09:06:45.185472+0000","last_became_peered":"2024-05-09T09:06:45.185472+0000","last_unstale":"2024-05-09T09:08:37.578766+0000","last_undegraded":"2024-05-09T09:08:37.578766+0000","last_fullsized":"2024-05-09T09:08:37.578766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:35.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706539+0000","last_change":"2024-05-09T09:06:40.106843+0000","last_active":"2024-05-09T09:08:36.706539+0000","last_peered":"2024-05-09T09:08:36.706539+0000","last_clean":"2024-05-09T09:08:36.706539+0000","last_became_active":"2024-05-09T09:06:40.106516+0000","last_became_peered":"2024-05-09T09:06:40.106516+0000","last_unstale":"2024-05-09T09:08:36.706539+0000","last_undegraded":"2024-05-09T09:08:36.706539+0000","last_fullsized":"2024-05-09T09:08:36.706539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:36.679536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846220+0000","last_change":"2024-05-09T09:06:41.125261+0000","last_active":"2024-05-09T09:08:36.846220+0000","last_peered":"2024-05-09T09:08:36.846220+0000","last_clean":"2024-05-09T09:08:36.846220+0000","last_became_active":"2024-05-09T09:06:41.124717+0000","last_became_peered":"2024-05-09T09:06:41.124717+0000","last_unstale":"2024-05-09T09:08:36.846220+0000","last_undegraded":"2024-05-09T09:08:36.846220+0000","last_fullsized":"2024-05-09T09:08:36.846220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:37.616109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919024+0000","last_change":"2024-05-09T09:06:45.179592+0000","last_active":"2024-05-09T09:08:36.919024+0000","last_peered":"2024-05-09T09:08:36.919024+0000","last_clean":"2024-05-09T09:08:36.919024+0000","last_became_active":"2024-05-09T09:06:45.179212+0000","last_became_peered":"2024-05-09T09:06:45.179212+0000","last_unstale":"2024-05-09T09:08:36.919024+0000","last_undegraded":"2024-05-09T09:08:36.919024+0000","last_fullsized":"2024-05-09T09:08:36.919024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:47:57.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709369+0000","last_change":"2024-05-09T09:06:40.117773+0000","last_active":"2024-05-09T09:08:37.709369+0000","last_peered":"2024-05-09T09:08:37.709369+0000","last_clean":"2024-05-09T09:08:37.709369+0000","last_became_active":"2024-05-09T09:06:40.117436+0000","last_became_peered":"2024-05-09T09:06:40.117436+0000","last_unstale":"2024-05-09T09:08:37.709369+0000","last_undegraded":"2024-05-09T09:08:37.709369+0000","last_fullsized":"2024-05-09T09:08:37.709369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:57:45.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.846078+0000","last_change":"2024-05-09T09:06:41.126578+0000","last_active":"2024-05-09T09:08:36.846078+0000","last_peered":"2024-05-09T09:08:36.846078+0000","last_clean":"2024-05-09T09:08:36.846078+0000","last_became_active":"2024-05-09T09:06:41.126229+0000","last_became_peered":"2024-05-09T09:06:41.126229+0000","last_unstale":"2024-05-09T09:08:36.846078+0000","last_undegraded":"2024-05-09T09:08:36.846078+0000","last_fullsized":"2024-05-09T09:08:36.846078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:12.966753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919150+0000","last_change":"2024-05-09T09:06:45.181417+0000","last_active":"2024-05-09T09:08:36.919150+0000","last_peered":"2024-05-09T09:08:36.919150+0000","last_clean":"2024-05-09T09:08:36.919150+0000","last_became_active":"2024-05-09T09:06:45.180782+0000","last_became_peered":"2024-05-09T09:06:45.180782+0000","last_unstale":"2024-05-09T09:08:36.919150+0000","last_undegraded":"2024-05-09T09:08:36.919150+0000","last_fullsized":"2024-05-09T09:08:36.919150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:57:36.984475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709052+0000","last_change":"2024-05-09T09:06:41.136523+0000","last_active":"2024-05-09T09:08:36.709052+0000","last_peered":"2024-05-09T09:08:36.709052+0000","last_clean":"2024-05-09T09:08:36.709052+0000","last_became_active":"2024-05-09T09:06:41.136184+0000","last_became_peered":"2024-05-09T09:06:41.136184+0000","last_unstale":"2024-05-09T09:08:36.709052+0000","last_undegraded":"2024-05-09T09:08:36.709052+0000","last_fullsized":"2024-05-09T09:08:36.709052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:50.126768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:09:05.407967+0000","last_change":"2024-05-09T09:06:40.111137+0000","last_active":"2024-05-09T09:09:05.407967+0000","last_peered":"2024-05-09T09:09:05.407967+0000","last_clean":"2024-05-09T09:09:05.407967+0000","last_became_active":"2024-05-09T09:06:40.103669+0000","last_became_peered":"2024-05-09T09:06:40.103669+0000","last_unstale":"2024-05-09T09:09:05.407967+0000","last_undegraded":"2024-05-09T09:09:05.407967+0000","last_fullsized":"2024-05-09T09:09:05.407967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708633+0000","last_change":"2024-05-09T09:06:45.179512+0000","last_active":"2024-05-09T09:08:37.708633+0000","last_peered":"2024-05-09T09:08:37.708633+0000","last_clean":"2024-05-09T09:08:37.708633+0000","last_became_active":"2024-05-09T09:06:45.178890+0000","last_became_peered":"2024-05-09T09:06:45.178890+0000","last_unstale":"2024-05-09T09:08:37.708633+0000","last_undegraded":"2024-05-09T09:08:37.708633+0000","last_fullsized":"2024-05-09T09:08:37.708633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:34.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706507+0000","last_change":"2024-05-09T09:06:41.135863+0000","last_active":"2024-05-09T09:08:36.706507+0000","last_peered":"2024-05-09T09:08:36.706507+0000","last_clean":"2024-05-09T09:08:36.706507+0000","last_became_active":"2024-05-09T09:06:41.133876+0000","last_became_peered":"2024-05-09T09:06:41.133876+0000","last_unstale":"2024-05-09T09:08:36.706507+0000","last_undegraded":"2024-05-09T09:08:36.706507+0000","last_fullsized":"2024-05-09T09:08:36.706507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.263395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703980+0000","last_change":"2024-05-09T09:06:40.109774+0000","last_active":"2024-05-09T09:08:36.703980+0000","last_peered":"2024-05-09T09:08:36.703980+0000","last_clean":"2024-05-09T09:08:36.703980+0000","last_became_active":"2024-05-09T09:06:40.107713+0000","last_became_peered":"2024-05-09T09:06:40.107713+0000","last_unstale":"2024-05-09T09:08:36.703980+0000","last_undegraded":"2024-05-09T09:08:36.703980+0000","last_fullsized":"2024-05-09T09:08:36.703980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:35.981960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":355,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.708443+0000","last_change":"2024-05-09T09:05:25.127484+0000","last_active":"2024-05-09T09:08:37.708443+0000","last_peered":"2024-05-09T09:08:37.708443+0000","last_clean":"2024-05-09T09:08:37.708443+0000","last_became_active":"2024-05-09T09:05:25.127044+0000","last_became_peered":"2024-05-09T09:05:25.127044+0000","last_unstale":"2024-05-09T09:08:37.708443+0000","last_undegraded":"2024-05-09T09:08:37.708443+0000","last_fullsized":"2024-05-09T09:08:37.708443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:02:00.076113+0000","last_clean_scrub_stamp":"2024-05-09T09:02:00.076113+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:48.012767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579449+0000","last_change":"2024-05-09T09:07:36.319419+0000","last_active":"2024-05-09T09:08:37.579449+0000","last_peered":"2024-05-09T09:08:37.579449+0000","last_clean":"2024-05-09T09:08:37.579449+0000","last_became_active":"2024-05-09T09:07:36.318825+0000","last_became_peered":"2024-05-09T09:07:36.318825+0000","last_unstale":"2024-05-09T09:08:37.579449+0000","last_undegraded":"2024-05-09T09:08:37.579449+0000","last_fullsized":"2024-05-09T09:08:37.579449+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:21:10.289893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.918805+0000","last_change":"2024-05-09T09:06:41.127097+0000","last_active":"2024-05-09T09:08:36.918805+0000","last_peered":"2024-05-09T09:08:36.918805+0000","last_clean":"2024-05-09T09:08:36.918805+0000","last_became_active":"2024-05-09T09:06:41.126793+0000","last_became_peered":"2024-05-09T09:06:41.126793+0000","last_unstale":"2024-05-09T09:08:36.918805+0000","last_undegraded":"2024-05-09T09:08:36.918805+0000","last_fullsized":"2024-05-09T09:08:36.918805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:59.581438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709055+0000","last_change":"2024-05-09T09:06:40.115613+0000","last_active":"2024-05-09T09:08:37.709055+0000","last_peered":"2024-05-09T09:08:37.709055+0000","last_clean":"2024-05-09T09:08:37.709055+0000","last_became_active":"2024-05-09T09:06:40.113136+0000","last_became_peered":"2024-05-09T09:06:40.113136+0000","last_unstale":"2024-05-09T09:08:37.709055+0000","last_undegraded":"2024-05-09T09:08:37.709055+0000","last_fullsized":"2024-05-09T09:08:37.709055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:32.754315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672046+0000","last_change":"2024-05-09T09:06:45.172093+0000","last_active":"2024-05-09T09:08:37.672046+0000","last_peered":"2024-05-09T09:08:37.672046+0000","last_clean":"2024-05-09T09:08:37.672046+0000","last_became_active":"2024-05-09T09:06:45.171551+0000","last_became_peered":"2024-05-09T09:06:45.171551+0000","last_unstale":"2024-05-09T09:08:37.672046+0000","last_undegraded":"2024-05-09T09:08:37.672046+0000","last_fullsized":"2024-05-09T09:08:37.672046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:33.674211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579460+0000","last_change":"2024-05-09T09:06:40.114834+0000","last_active":"2024-05-09T09:08:37.579460+0000","last_peered":"2024-05-09T09:08:37.579460+0000","last_clean":"2024-05-09T09:08:37.579460+0000","last_became_active":"2024-05-09T09:06:40.114508+0000","last_became_peered":"2024-05-09T09:06:40.114508+0000","last_unstale":"2024-05-09T09:08:37.579460+0000","last_undegraded":"2024-05-09T09:08:37.579460+0000","last_fullsized":"2024-05-09T09:08:37.579460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:07.245872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919182+0000","last_change":"2024-05-09T09:06:41.131591+0000","last_active":"2024-05-09T09:08:36.919182+0000","last_peered":"2024-05-09T09:08:36.919182+0000","last_clean":"2024-05-09T09:08:36.919182+0000","last_became_active":"2024-05-09T09:06:41.131218+0000","last_became_peered":"2024-05-09T09:06:41.131218+0000","last_unstale":"2024-05-09T09:08:36.919182+0000","last_undegraded":"2024-05-09T09:08:36.919182+0000","last_fullsized":"2024-05-09T09:08:36.919182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:56.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705339+0000","last_change":"2024-05-09T09:06:45.177962+0000","last_active":"2024-05-09T09:08:36.705339+0000","last_peered":"2024-05-09T09:08:36.705339+0000","last_clean":"2024-05-09T09:08:36.705339+0000","last_became_active":"2024-05-09T09:06:45.177549+0000","last_became_peered":"2024-05-09T09:06:45.177549+0000","last_unstale":"2024-05-09T09:08:36.705339+0000","last_undegraded":"2024-05-09T09:08:36.705339+0000","last_fullsized":"2024-05-09T09:08:36.705339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:50.803291+0000","last_change":"2024-05-09T09:06:40.113735+0000","last_active":"2024-05-09T09:08:50.803291+0000","last_peered":"2024-05-09T09:08:50.803291+0000","last_clean":"2024-05-09T09:08:50.803291+0000","last_became_active":"2024-05-09T09:06:40.112647+0000","last_became_peered":"2024-05-09T09:06:40.112647+0000","last_unstale":"2024-05-09T09:08:50.803291+0000","last_undegraded":"2024-05-09T09:08:50.803291+0000","last_fullsized":"2024-05-09T09:08:50.803291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:22.629766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707099+0000","last_change":"2024-05-09T09:06:41.130292+0000","last_active":"2024-05-09T09:08:36.707099+0000","last_peered":"2024-05-09T09:08:36.707099+0000","last_clean":"2024-05-09T09:08:36.707099+0000","last_became_active":"2024-05-09T09:06:41.129964+0000","last_became_peered":"2024-05-09T09:06:41.129964+0000","last_unstale":"2024-05-09T09:08:36.707099+0000","last_undegraded":"2024-05-09T09:08:36.707099+0000","last_fullsized":"2024-05-09T09:08:36.707099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:42.125644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703979+0000","last_change":"2024-05-09T09:06:45.173150+0000","last_active":"2024-05-09T09:08:36.703979+0000","last_peered":"2024-05-09T09:08:36.703979+0000","last_clean":"2024-05-09T09:08:36.703979+0000","last_became_active":"2024-05-09T09:06:45.172646+0000","last_became_peered":"2024-05-09T09:06:45.172646+0000","last_unstale":"2024-05-09T09:08:36.703979+0000","last_undegraded":"2024-05-09T09:08:36.703979+0000","last_fullsized":"2024-05-09T09:08:36.703979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:25.584480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704427+0000","last_change":"2024-05-09T09:06:40.106204+0000","last_active":"2024-05-09T09:08:36.704427+0000","last_peered":"2024-05-09T09:08:36.704427+0000","last_clean":"2024-05-09T09:08:36.704427+0000","last_became_active":"2024-05-09T09:06:40.105732+0000","last_became_peered":"2024-05-09T09:06:40.105732+0000","last_unstale":"2024-05-09T09:08:36.704427+0000","last_undegraded":"2024-05-09T09:08:36.704427+0000","last_fullsized":"2024-05-09T09:08:36.704427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:58.369056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939395+0000","last_change":"2024-05-09T09:07:36.322492+0000","last_active":"2024-05-09T09:08:37.939395+0000","last_peered":"2024-05-09T09:08:37.939395+0000","last_clean":"2024-05-09T09:08:37.939395+0000","last_became_active":"2024-05-09T09:07:36.322098+0000","last_became_peered":"2024-05-09T09:07:36.322098+0000","last_unstale":"2024-05-09T09:08:37.939395+0000","last_undegraded":"2024-05-09T09:08:37.939395+0000","last_fullsized":"2024-05-09T09:08:37.939395+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:55.030312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704203+0000","last_change":"2024-05-09T09:06:45.185620+0000","last_active":"2024-05-09T09:08:36.704203+0000","last_peered":"2024-05-09T09:08:36.704203+0000","last_clean":"2024-05-09T09:08:36.704203+0000","last_became_active":"2024-05-09T09:06:45.184817+0000","last_became_peered":"2024-05-09T09:06:45.184817+0000","last_unstale":"2024-05-09T09:08:36.704203+0000","last_undegraded":"2024-05-09T09:08:36.704203+0000","last_fullsized":"2024-05-09T09:08:36.704203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:07.275594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123773+0000","last_change":"2024-05-09T09:06:40.106437+0000","last_active":"2024-05-09T09:08:37.123773+0000","last_peered":"2024-05-09T09:08:37.123773+0000","last_clean":"2024-05-09T09:08:37.123773+0000","last_became_active":"2024-05-09T09:06:40.106052+0000","last_became_peered":"2024-05-09T09:06:40.106052+0000","last_unstale":"2024-05-09T09:08:37.123773+0000","last_undegraded":"2024-05-09T09:08:37.123773+0000","last_fullsized":"2024-05-09T09:08:37.123773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:25.056266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.578938+0000","last_change":"2024-05-09T09:06:41.132091+0000","last_active":"2024-05-09T09:08:37.578938+0000","last_peered":"2024-05-09T09:08:37.578938+0000","last_clean":"2024-05-09T09:08:37.578938+0000","last_became_active":"2024-05-09T09:06:41.131137+0000","last_became_peered":"2024-05-09T09:06:41.131137+0000","last_unstale":"2024-05-09T09:08:37.578938+0000","last_undegraded":"2024-05-09T09:08:37.578938+0000","last_fullsized":"2024-05-09T09:08:37.578938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:45.922380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.706560+0000","last_change":"2024-05-09T09:06:45.180609+0000","last_active":"2024-05-09T09:08:36.706560+0000","last_peered":"2024-05-09T09:08:36.706560+0000","last_clean":"2024-05-09T09:08:36.706560+0000","last_became_active":"2024-05-09T09:06:45.179185+0000","last_became_peered":"2024-05-09T09:06:45.179185+0000","last_unstale":"2024-05-09T09:08:36.706560+0000","last_undegraded":"2024-05-09T09:08:36.706560+0000","last_fullsized":"2024-05-09T09:08:36.706560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:06.780757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705432+0000","last_change":"2024-05-09T09:06:40.107632+0000","last_active":"2024-05-09T09:08:36.705432+0000","last_peered":"2024-05-09T09:08:36.705432+0000","last_clean":"2024-05-09T09:08:36.705432+0000","last_became_active":"2024-05-09T09:06:40.106899+0000","last_became_peered":"2024-05-09T09:06:40.106899+0000","last_unstale":"2024-05-09T09:08:36.705432+0000","last_undegraded":"2024-05-09T09:08:36.705432+0000","last_fullsized":"2024-05-09T09:08:36.705432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:40.741325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672281+0000","last_change":"2024-05-09T09:06:41.126434+0000","last_active":"2024-05-09T09:08:37.672281+0000","last_peered":"2024-05-09T09:08:37.672281+0000","last_clean":"2024-05-09T09:08:37.672281+0000","last_became_active":"2024-05-09T09:06:41.126086+0000","last_became_peered":"2024-05-09T09:06:41.126086+0000","last_unstale":"2024-05-09T09:08:37.672281+0000","last_undegraded":"2024-05-09T09:08:37.672281+0000","last_fullsized":"2024-05-09T09:08:37.672281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:01:15.298329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704138+0000","last_change":"2024-05-09T09:06:45.186347+0000","last_active":"2024-05-09T09:08:36.704138+0000","last_peered":"2024-05-09T09:08:36.704138+0000","last_clean":"2024-05-09T09:08:36.704138+0000","last_became_active":"2024-05-09T09:06:45.185700+0000","last_became_peered":"2024-05-09T09:06:45.185700+0000","last_unstale":"2024-05-09T09:08:36.704138+0000","last_undegraded":"2024-05-09T09:08:36.704138+0000","last_fullsized":"2024-05-09T09:08:36.704138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:34.217100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672311+0000","last_change":"2024-05-09T09:07:36.327470+0000","last_active":"2024-05-09T09:08:37.672311+0000","last_peered":"2024-05-09T09:08:37.672311+0000","last_clean":"2024-05-09T09:08:37.672311+0000","last_became_active":"2024-05-09T09:07:36.321516+0000","last_became_peered":"2024-05-09T09:07:36.321516+0000","last_unstale":"2024-05-09T09:08:37.672311+0000","last_undegraded":"2024-05-09T09:08:37.672311+0000","last_fullsized":"2024-05-09T09:08:37.672311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:30.824518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705774+0000","last_change":"2024-05-09T09:06:41.136533+0000","last_active":"2024-05-09T09:08:36.705774+0000","last_peered":"2024-05-09T09:08:36.705774+0000","last_clean":"2024-05-09T09:08:36.705774+0000","last_became_active":"2024-05-09T09:06:41.136168+0000","last_became_peered":"2024-05-09T09:06:41.136168+0000","last_unstale":"2024-05-09T09:08:36.705774+0000","last_undegraded":"2024-05-09T09:08:36.705774+0000","last_fullsized":"2024-05-09T09:08:36.705774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:45.725871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704007+0000","last_change":"2024-05-09T09:06:45.179899+0000","last_active":"2024-05-09T09:08:36.704007+0000","last_peered":"2024-05-09T09:08:36.704007+0000","last_clean":"2024-05-09T09:08:36.704007+0000","last_became_active":"2024-05-09T09:06:45.179213+0000","last_became_peered":"2024-05-09T09:06:45.179213+0000","last_unstale":"2024-05-09T09:08:36.704007+0000","last_undegraded":"2024-05-09T09:08:36.704007+0000","last_fullsized":"2024-05-09T09:08:36.704007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:48.707549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.579278+0000","last_change":"2024-05-09T09:06:40.115832+0000","last_active":"2024-05-09T09:08:37.579278+0000","last_peered":"2024-05-09T09:08:37.579278+0000","last_clean":"2024-05-09T09:08:37.579278+0000","last_became_active":"2024-05-09T09:06:40.115358+0000","last_became_peered":"2024-05-09T09:06:40.115358+0000","last_unstale":"2024-05-09T09:08:37.579278+0000","last_undegraded":"2024-05-09T09:08:37.579278+0000","last_fullsized":"2024-05-09T09:08:37.579278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:19.238859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709254+0000","last_change":"2024-05-09T09:06:41.129494+0000","last_active":"2024-05-09T09:08:37.709254+0000","last_peered":"2024-05-09T09:08:37.709254+0000","last_clean":"2024-05-09T09:08:37.709254+0000","last_became_active":"2024-05-09T09:06:41.129151+0000","last_became_peered":"2024-05-09T09:06:41.129151+0000","last_unstale":"2024-05-09T09:08:37.709254+0000","last_undegraded":"2024-05-09T09:08:37.709254+0000","last_fullsized":"2024-05-09T09:08:37.709254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:37.839976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.123785+0000","last_change":"2024-05-09T09:06:45.174072+0000","last_active":"2024-05-09T09:08:37.123785+0000","last_peered":"2024-05-09T09:08:37.123785+0000","last_clean":"2024-05-09T09:08:37.123785+0000","last_became_active":"2024-05-09T09:06:45.173208+0000","last_became_peered":"2024-05-09T09:06:45.173208+0000","last_unstale":"2024-05-09T09:08:37.123785+0000","last_undegraded":"2024-05-09T09:08:37.123785+0000","last_fullsized":"2024-05-09T09:08:37.123785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:00:09.814063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704679+0000","last_change":"2024-05-09T09:06:40.118774+0000","last_active":"2024-05-09T09:08:36.704679+0000","last_peered":"2024-05-09T09:08:36.704679+0000","last_clean":"2024-05-09T09:08:36.704679+0000","last_became_active":"2024-05-09T09:06:40.118388+0000","last_became_peered":"2024-05-09T09:06:40.118388+0000","last_unstale":"2024-05-09T09:08:36.704679+0000","last_undegraded":"2024-05-09T09:08:36.704679+0000","last_fullsized":"2024-05-09T09:08:36.704679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:38.312588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709737+0000","last_change":"2024-05-09T09:06:41.133807+0000","last_active":"2024-05-09T09:08:37.709737+0000","last_peered":"2024-05-09T09:08:37.709737+0000","last_clean":"2024-05-09T09:08:37.709737+0000","last_became_active":"2024-05-09T09:06:41.133239+0000","last_became_peered":"2024-05-09T09:06:41.133239+0000","last_unstale":"2024-05-09T09:08:37.709737+0000","last_undegraded":"2024-05-09T09:08:37.709737+0000","last_fullsized":"2024-05-09T09:08:37.709737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:33.486294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.938607+0000","last_change":"2024-05-09T09:06:45.180510+0000","last_active":"2024-05-09T09:08:37.938607+0000","last_peered":"2024-05-09T09:08:37.938607+0000","last_clean":"2024-05-09T09:08:37.938607+0000","last_became_active":"2024-05-09T09:06:45.180038+0000","last_became_peered":"2024-05-09T09:06:45.180038+0000","last_unstale":"2024-05-09T09:08:37.938607+0000","last_undegraded":"2024-05-09T09:08:37.938607+0000","last_fullsized":"2024-05-09T09:08:37.938607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.299251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709214+0000","last_change":"2024-05-09T09:06:40.110437+0000","last_active":"2024-05-09T09:08:36.709214+0000","last_peered":"2024-05-09T09:08:36.709214+0000","last_clean":"2024-05-09T09:08:36.709214+0000","last_became_active":"2024-05-09T09:06:40.110092+0000","last_became_peered":"2024-05-09T09:06:40.110092+0000","last_unstale":"2024-05-09T09:08:36.709214+0000","last_undegraded":"2024-05-09T09:08:36.709214+0000","last_fullsized":"2024-05-09T09:08:36.709214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:04.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709491+0000","last_change":"2024-05-09T09:06:41.132464+0000","last_active":"2024-05-09T09:08:37.709491+0000","last_peered":"2024-05-09T09:08:37.709491+0000","last_clean":"2024-05-09T09:08:37.709491+0000","last_became_active":"2024-05-09T09:06:41.131204+0000","last_became_peered":"2024-05-09T09:06:41.131204+0000","last_unstale":"2024-05-09T09:08:37.709491+0000","last_undegraded":"2024-05-09T09:08:37.709491+0000","last_fullsized":"2024-05-09T09:08:37.709491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:45.745096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.708466+0000","last_change":"2024-05-09T09:06:45.173594+0000","last_active":"2024-05-09T09:08:36.708466+0000","last_peered":"2024-05-09T09:08:36.708466+0000","last_clean":"2024-05-09T09:08:36.708466+0000","last_became_active":"2024-05-09T09:06:45.172899+0000","last_became_peered":"2024-05-09T09:06:45.172899+0000","last_unstale":"2024-05-09T09:08:36.708466+0000","last_undegraded":"2024-05-09T09:08:36.708466+0000","last_fullsized":"2024-05-09T09:08:36.708466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:44.142864+0000","last_clean_scrub_stamp":"2024-05-09T09:06:44.142864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:04.506533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919179+0000","last_change":"2024-05-09T09:06:40.112158+0000","last_active":"2024-05-09T09:08:36.919179+0000","last_peered":"2024-05-09T09:08:36.919179+0000","last_clean":"2024-05-09T09:08:36.919179+0000","last_became_active":"2024-05-09T09:06:40.111824+0000","last_became_peered":"2024-05-09T09:06:40.111824+0000","last_unstale":"2024-05-09T09:08:36.919179+0000","last_undegraded":"2024-05-09T09:08:36.919179+0000","last_fullsized":"2024-05-09T09:08:36.919179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:39.084551+0000","last_clean_scrub_stamp":"2024-05-09T09:06:39.084551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:14.910808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.709710+0000","last_change":"2024-05-09T09:06:41.136516+0000","last_active":"2024-05-09T09:08:37.709710+0000","last_peered":"2024-05-09T09:08:37.709710+0000","last_clean":"2024-05-09T09:08:37.709710+0000","last_became_active":"2024-05-09T09:06:41.136176+0000","last_became_peered":"2024-05-09T09:06:41.136176+0000","last_unstale":"2024-05-09T09:08:37.709710+0000","last_undegraded":"2024-05-09T09:08:37.709710+0000","last_fullsized":"2024-05-09T09:08:37.709710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:57.660333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672377+0000","last_change":"2024-05-09T09:06:41.126594+0000","last_active":"2024-05-09T09:08:37.672377+0000","last_peered":"2024-05-09T09:08:37.672377+0000","last_clean":"2024-05-09T09:08:37.672377+0000","last_became_active":"2024-05-09T09:06:41.126238+0000","last_became_peered":"2024-05-09T09:06:41.126238+0000","last_unstale":"2024-05-09T09:08:37.672377+0000","last_undegraded":"2024-05-09T09:08:37.672377+0000","last_fullsized":"2024-05-09T09:08:37.672377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:09.028426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704264+0000","last_change":"2024-05-09T09:06:41.124241+0000","last_active":"2024-05-09T09:08:36.704264+0000","last_peered":"2024-05-09T09:08:36.704264+0000","last_clean":"2024-05-09T09:08:36.704264+0000","last_became_active":"2024-05-09T09:06:41.123908+0000","last_became_peered":"2024-05-09T09:06:41.123908+0000","last_unstale":"2024-05-09T09:08:36.704264+0000","last_undegraded":"2024-05-09T09:08:36.704264+0000","last_fullsized":"2024-05-09T09:08:36.704264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:16.897470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.703961+0000","last_change":"2024-05-09T09:06:41.123688+0000","last_active":"2024-05-09T09:08:36.703961+0000","last_peered":"2024-05-09T09:08:36.703961+0000","last_clean":"2024-05-09T09:08:36.703961+0000","last_became_active":"2024-05-09T09:06:41.123338+0000","last_became_peered":"2024-05-09T09:06:41.123338+0000","last_unstale":"2024-05-09T09:08:36.703961+0000","last_undegraded":"2024-05-09T09:08:36.703961+0000","last_fullsized":"2024-05-09T09:08:36.703961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:36:59.575342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.939091+0000","last_change":"2024-05-09T09:07:36.321453+0000","last_active":"2024-05-09T09:08:37.939091+0000","last_peered":"2024-05-09T09:08:37.939091+0000","last_clean":"2024-05-09T09:08:37.939091+0000","last_became_active":"2024-05-09T09:07:36.321074+0000","last_became_peered":"2024-05-09T09:07:36.321074+0000","last_unstale":"2024-05-09T09:08:37.939091+0000","last_undegraded":"2024-05-09T09:08:37.939091+0000","last_fullsized":"2024-05-09T09:08:37.939091+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:26.690978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919162+0000","last_change":"2024-05-09T09:06:41.130638+0000","last_active":"2024-05-09T09:08:36.919162+0000","last_peered":"2024-05-09T09:08:36.919162+0000","last_clean":"2024-05-09T09:08:36.919162+0000","last_became_active":"2024-05-09T09:06:41.130217+0000","last_became_peered":"2024-05-09T09:06:41.130217+0000","last_unstale":"2024-05-09T09:08:36.919162+0000","last_undegraded":"2024-05-09T09:08:36.919162+0000","last_fullsized":"2024-05-09T09:08:36.919162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:54:48.265060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672652+0000","last_change":"2024-05-09T09:06:41.135532+0000","last_active":"2024-05-09T09:08:37.672652+0000","last_peered":"2024-05-09T09:08:37.672652+0000","last_clean":"2024-05-09T09:08:37.672652+0000","last_became_active":"2024-05-09T09:06:41.135161+0000","last_became_peered":"2024-05-09T09:06:41.135161+0000","last_unstale":"2024-05-09T09:08:37.672652+0000","last_undegraded":"2024-05-09T09:08:37.672652+0000","last_fullsized":"2024-05-09T09:08:37.672652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:50.395049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.704543+0000","last_change":"2024-05-09T09:06:41.133559+0000","last_active":"2024-05-09T09:08:36.704543+0000","last_peered":"2024-05-09T09:08:36.704543+0000","last_clean":"2024-05-09T09:08:36.704543+0000","last_became_active":"2024-05-09T09:06:41.133135+0000","last_became_peered":"2024-05-09T09:06:41.133135+0000","last_unstale":"2024-05-09T09:08:36.704543+0000","last_undegraded":"2024-05-09T09:08:36.704543+0000","last_fullsized":"2024-05-09T09:08:36.704543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:43:36.459381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.707710+0000","last_change":"2024-05-09T09:08:36.707710+0000","last_active":"2024-05-09T09:08:36.707710+0000","last_peered":"2024-05-09T09:08:36.707710+0000","last_clean":"2024-05-09T09:08:36.707710+0000","last_became_active":"2024-05-09T09:08:36.707333+0000","last_became_peered":"2024-05-09T09:08:36.707333+0000","last_unstale":"2024-05-09T09:08:36.707710+0000","last_undegraded":"2024-05-09T09:08:36.707710+0000","last_fullsized":"2024-05-09T09:08:36.707710+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:08.495289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.709005+0000","last_change":"2024-05-09T09:06:41.141003+0000","last_active":"2024-05-09T09:08:36.709005+0000","last_peered":"2024-05-09T09:08:36.709005+0000","last_clean":"2024-05-09T09:08:36.709005+0000","last_became_active":"2024-05-09T09:06:41.140622+0000","last_became_peered":"2024-05-09T09:06:41.140622+0000","last_unstale":"2024-05-09T09:08:36.709005+0000","last_undegraded":"2024-05-09T09:08:36.709005+0000","last_fullsized":"2024-05-09T09:08:36.709005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:27.193131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.705731+0000","last_change":"2024-05-09T09:06:41.129054+0000","last_active":"2024-05-09T09:08:36.705731+0000","last_peered":"2024-05-09T09:08:36.705731+0000","last_clean":"2024-05-09T09:08:36.705731+0000","last_became_active":"2024-05-09T09:06:41.128728+0000","last_became_peered":"2024-05-09T09:06:41.128728+0000","last_unstale":"2024-05-09T09:08:36.705731+0000","last_undegraded":"2024-05-09T09:08:36.705731+0000","last_fullsized":"2024-05-09T09:08:36.705731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:01:40.396621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.124059+0000","last_change":"2024-05-09T09:06:41.132479+0000","last_active":"2024-05-09T09:08:37.124059+0000","last_peered":"2024-05-09T09:08:37.124059+0000","last_clean":"2024-05-09T09:08:37.124059+0000","last_became_active":"2024-05-09T09:06:41.132163+0000","last_became_peered":"2024-05-09T09:06:41.132163+0000","last_unstale":"2024-05-09T09:08:37.124059+0000","last_undegraded":"2024-05-09T09:08:37.124059+0000","last_fullsized":"2024-05-09T09:08:37.124059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:48.490129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:36.919025+0000","last_change":"2024-05-09T09:06:41.131646+0000","last_active":"2024-05-09T09:08:36.919025+0000","last_peered":"2024-05-09T09:08:36.919025+0000","last_clean":"2024-05-09T09:08:36.919025+0000","last_became_active":"2024-05-09T09:06:41.131276+0000","last_became_peered":"2024-05-09T09:06:41.131276+0000","last_unstale":"2024-05-09T09:08:36.919025+0000","last_undegraded":"2024-05-09T09:08:36.919025+0000","last_fullsized":"2024-05-09T09:08:36.919025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:55.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-05-09T09:08:37.672494+0000","last_change":"2024-05-09T09:06:41.136199+0000","last_active":"2024-05-09T09:08:37.672494+0000","last_peered":"2024-05-09T09:08:37.672494+0000","last_clean":"2024-05-09T09:08:37.672494+0000","last_became_active":"2024-05-09T09:06:41.135724+0000","last_became_peered":"2024-05-09T09:06:41.135724+0000","last_unstale":"2024-05-09T09:08:37.672494+0000","last_undegraded":"2024-05-09T09:08:37.672494+0000","last_fullsized":"2024-05-09T09:08:37.672494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T09:06:40.089423+0000","last_clean_scrub_stamp":"2024-05-09T09:06:40.089423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:12:46.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70794,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":749,"num_read_kb":449,"num_write":173,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":208068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":522662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612657,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1176615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":526972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1119268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":518052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":579401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":519440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":586019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":518246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":523179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1683456,"data_stored":1114379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T09:09:10.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:09:10.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 -- 172.21.15.64:0/2153680140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78102340 msgr2=0x7f5a78108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/2153680140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78102340 0x7f5a78108bd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009ca0 tx=0x7f5a7002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 -- 172.21.15.64:0/2153680140 shutdown_connections 2024-05-09T09:09:10.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/2153680140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78102340 0x7f5a78108bd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/2153680140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78101980 0x7f5a78101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/2153680140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78100790 0x7f5a78100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.598+0000 7f5a7ff09640 1 -- 172.21.15.64:0/2153680140 >> 172.21.15.64:0/2153680140 conn(0x7f5a780fbd20 msgr2=0x7f5a780fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 -- 172.21.15.64:0/2153680140 shutdown_connections 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 -- 172.21.15.64:0/2153680140 wait complete. 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 Processor -- start 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 -- start start 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78100790 0x7f5a781046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 0x7f5a78102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78102340 0x7f5a781032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c2c0 con 0x7f5a78101980 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c140 con 0x7f5a78102340 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7ff09640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c440 con 0x7f5a78100790 2024-05-09T09:09:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7d4a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 0x7f5a78102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7d4a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 0x7f5a78102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39114/0 (socket says 172.21.15.64:39114) 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.599+0000 7f5a7d4a4640 1 -- 172.21.15.64:0/3914708731 learned_addr learned my addr 172.21.15.64:0/3914708731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7e4a6640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78102340 0x7f5a781032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7dca5640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78100790 0x7f5a781046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7d4a4640 1 -- 172.21.15.64:0/3914708731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78100790 msgr2=0x7f5a781046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7d4a4640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78100790 0x7f5a781046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7d4a4640 1 -- 172.21.15.64:0/3914708731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78102340 msgr2=0x7f5a781032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7d4a4640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78102340 0x7f5a781032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7d4a4640 1 -- 172.21.15.64:0/3914708731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a78103b60 con 0x7f5a78101980 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7e4a6640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78102340 0x7f5a781032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7dca5640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78100790 0x7f5a781046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7d4a4640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 0x7f5a78102d60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800d770 tx=0x7f5a6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68017070 con 0x7f5a78101980 2024-05-09T09:09:10.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a781a1b20 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a680054d0 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004cf0 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.600+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a781a2040 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.601+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a781a2210 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.602+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a68004e90 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.603+0000 7f5a6effd640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a640a2b90 0x7f5a640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.603+0000 7f5a7dca5640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a640a2b90 0x7f5a640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.603+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a680cf260 con 0x7f5a78101980 2024-05-09T09:09:10.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.604+0000 7f5a7dca5640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a640a2b90 0x7f5a640a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5a781017e0 tx=0x7f5a7400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:10.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.606+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a6809be30 con 0x7f5a78101980 2024-05-09T09:09:10.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.794+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a781a25a0 con 0x7f5a78101980 2024-05-09T09:09:10.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.795+0000 7f5a6effd640 1 -- 172.21.15.64:0/3914708731 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f5a6809b730 con 0x7f5a78101980 2024-05-09T09:09:10.795 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:10.795 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:09:10.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a640a2b90 msgr2=0x7f5a640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a640a2b90 0x7f5a640a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5a781017e0 tx=0x7f5a7400a5e0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 msgr2=0x7f5a78102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 0x7f5a78102d60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800d770 tx=0x7f5a6800dc30 comp rx=0 tx=0).stop 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 shutdown_connections 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a640a2b90 0x7f5a640a5050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a78102340 0x7f5a781032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a78101980 0x7f5a78102d60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 --2- 172.21.15.64:0/3914708731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a78100790 0x7f5a781046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 >> 172.21.15.64:0/3914708731 conn(0x7f5a780fbd20 msgr2=0x7f5a780fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 shutdown_connections 2024-05-09T09:09:10.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.797+0000 7f5a7ff09640 1 -- 172.21.15.64:0/3914708731 wait complete. 2024-05-09T09:09:10.808 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:09:10.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:09:10.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 -- 172.21.15.64:0/1208081512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710107570 msgr2=0x7fa7101099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1208081512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710107570 0x7fa7101099c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa708009ca0 tx=0x7fa70802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 -- 172.21.15.64:0/1208081512 shutdown_connections 2024-05-09T09:09:10.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1208081512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710107570 0x7fa7101099c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1208081512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa710104c40 0x7fa710107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1208081512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa710102310 0x7fa710104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 -- 172.21.15.64:0/1208081512 >> 172.21.15.64:0/1208081512 conn(0x7fa7100fbd20 msgr2=0x7fa7100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 -- 172.21.15.64:0/1208081512 shutdown_connections 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.918+0000 7fa717bb7640 1 -- 172.21.15.64:0/1208081512 wait complete. 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 Processor -- start 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 -- start start 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa710102310 0x7fa710197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 0x7fa710197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa710107570 0x7fa71019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa71010d3c0 con 0x7fa710104c40 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa71010d240 con 0x7fa710102310 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa717bb7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa71010d540 con 0x7fa710107570 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 0x7fa710197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 0x7fa710197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39134/0 (socket says 172.21.15.64:39134) 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 -- 172.21.15.64:0/1195224284 learned_addr learned my addr 172.21.15.64:0/1195224284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715953640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa710102310 0x7fa710197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa716154640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa710107570 0x7fa71019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 -- 172.21.15.64:0/1195224284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa710107570 msgr2=0x7fa71019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa710107570 0x7fa71019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 -- 172.21.15.64:0/1195224284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa710102310 msgr2=0x7fa710197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:10.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa710102310 0x7fa710197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 -- 172.21.15.64:0/1195224284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71019f620 con 0x7fa710104c40 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.919+0000 7fa715152640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 0x7fa710197f40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c970 tx=0x7fa70c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.920+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c014070 con 0x7fa710104c40 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.920+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa70c004440 con 0x7fa710104c40 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.920+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71019f910 con 0x7fa710104c40 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.920+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c003e50 con 0x7fa710104c40 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.920+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71019fe60 con 0x7fa710104c40 2024-05-09T09:09:10.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.921+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7100fd810 con 0x7fa710104c40 2024-05-09T09:09:10.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.922+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa70c017070 con 0x7fa710104c40 2024-05-09T09:09:10.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.923+0000 7fa702ffd640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6fc0a2d30 0x7fa6fc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:10.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.923+0000 7fa715953640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6fc0a2d30 0x7fa6fc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:10.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.923+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa70c0cf1b0 con 0x7fa710104c40 2024-05-09T09:09:10.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.923+0000 7fa715953640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6fc0a2d30 0x7fa6fc0a51f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa704005d00 tx=0x7fa70400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:10.927+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa70c01e020 con 0x7fa710104c40 2024-05-09T09:09:11.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.111+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa710062430 con 0x7fa710104c40 2024-05-09T09:09:11.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.112+0000 7fa702ffd640 1 -- 172.21.15.64:0/1195224284 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fa70c09bd80 con 0x7fa710104c40 2024-05-09T09:09:11.111 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6fc0a2d30 msgr2=0x7fa6fc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6fc0a2d30 0x7fa6fc0a51f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa704005d00 tx=0x7fa70400b410 comp rx=0 tx=0).stop 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 msgr2=0x7fa710197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 0x7fa710197f40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c970 tx=0x7fa70c00ce30 comp rx=0 tx=0).stop 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 shutdown_connections 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6fc0a2d30 0x7fa6fc0a51f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa710107570 0x7fa71019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa710104c40 0x7fa710197f40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 --2- 172.21.15.64:0/1195224284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa710102310 0x7fa710197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 >> 172.21.15.64:0/1195224284 conn(0x7fa7100fbd20 msgr2=0x7fa710102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 shutdown_connections 2024-05-09T09:09:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.114+0000 7fa717bb7640 1 -- 172.21.15.64:0/1195224284 wait complete. 2024-05-09T09:09:11.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:10 smithi064 ceph-mon[30149]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T09:09:11.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:10 smithi064 ceph-mon[30149]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:11.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:10 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3914708731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:11.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:09:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:10 smithi179 ceph-mon[34086]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T09:09:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:10 smithi179 ceph-mon[34086]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:10 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3914708731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:11.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 -- 172.21.15.64:0/874513247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214101410 msgr2=0x7f7214108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 --2- 172.21.15.64:0/874513247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214101410 0x7f7214108cb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f720c009ca0 tx=0x7f720c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 -- 172.21.15.64:0/874513247 shutdown_connections 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 --2- 172.21.15.64:0/874513247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214101410 0x7f7214108cb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 --2- 172.21.15.64:0/874513247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7214100a50 0x7f7214100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 --2- 172.21.15.64:0/874513247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 0x7f7214102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 -- 172.21.15.64:0/874513247 >> 172.21.15.64:0/874513247 conn(0x7f72140fb460 msgr2=0x7f72140fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 -- 172.21.15.64:0/874513247 shutdown_connections 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.234+0000 7f7219083640 1 -- 172.21.15.64:0/874513247 wait complete. 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 Processor -- start 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 -- start start 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 0x7f7214197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7214101410 0x7f7214197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 0x7f721419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f721410c2d0 con 0x7f7214100a50 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f721410c150 con 0x7f7214101d20 2024-05-09T09:09:11.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7219083640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f721410c450 con 0x7f7214101410 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 0x7f7214197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 0x7f7214197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39162/0 (socket says 172.21.15.64:39162) 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 -- 172.21.15.64:0/2115771896 learned_addr learned my addr 172.21.15.64:0/2115771896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f720bfff640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7214101410 0x7f7214197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f721359e640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 0x7f721419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 -- 172.21.15.64:0/2115771896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7214101410 msgr2=0x7f7214197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7214101410 0x7f7214197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 -- 172.21.15.64:0/2115771896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 msgr2=0x7f721419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 0x7f721419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f7212d9d640 1 -- 172.21.15.64:0/2115771896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721419f6c0 con 0x7f7214100a50 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.235+0000 7f721359e640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 0x7f721419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.236+0000 7f7212d9d640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 0x7f7214197aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f720400d950 tx=0x7f720400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.236+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204017020 con 0x7f7214100a50 2024-05-09T09:09:11.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.236+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7204009c30 con 0x7f7214100a50 2024-05-09T09:09:11.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.236+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721419f9b0 con 0x7f7214100a50 2024-05-09T09:09:11.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.236+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720400f6b0 con 0x7f7214100a50 2024-05-09T09:09:11.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.236+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721419ff50 con 0x7f7214100a50 2024-05-09T09:09:11.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.237+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72141a01e0 con 0x7f7214100a50 2024-05-09T09:09:11.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.238+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f720401a020 con 0x7f7214100a50 2024-05-09T09:09:11.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.239+0000 7f7210d99640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72000a2c10 0x7f72000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.239+0000 7f720bfff640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72000a2c10 0x7f72000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.239+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72040cf2b0 con 0x7f7214100a50 2024-05-09T09:09:11.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.239+0000 7f720bfff640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72000a2c10 0x7f72000a50d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005fd0 tx=0x7f71fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:11.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.243+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7204021020 con 0x7f7214100a50 2024-05-09T09:09:11.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:10 smithi139 ceph-mon[33599]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T09:09:11.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:10 smithi139 ceph-mon[33599]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:11.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:10 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3914708731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:11.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.428+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f72141a03b0 con 0x7f7214100a50 2024-05-09T09:09:11.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.429+0000 7f7210d99640 1 -- 172.21.15.64:0/2115771896 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f7204009de0 con 0x7f7214100a50 2024-05-09T09:09:11.428 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:09:11.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72000a2c10 msgr2=0x7f72000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72000a2c10 0x7f72000a50d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005fd0 tx=0x7f71fc005850 comp rx=0 tx=0).stop 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 msgr2=0x7f7214197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 0x7f7214197aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f720400d950 tx=0x7f720400de10 comp rx=0 tx=0).stop 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 shutdown_connections 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72000a2c10 0x7f72000a50d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214101d20 0x7f721419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7214101410 0x7f7214197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 --2- 172.21.15.64:0/2115771896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7214100a50 0x7f7214197aa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.430+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 >> 172.21.15.64:0/2115771896 conn(0x7f72140fb460 msgr2=0x7f72140fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.431+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 shutdown_connections 2024-05-09T09:09:11.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.431+0000 7f7219083640 1 -- 172.21.15.64:0/2115771896 wait complete. 2024-05-09T09:09:11.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:09:11.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 -- 172.21.15.64:0/3157050077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28101960 msgr2=0x7ffa28101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 --2- 172.21.15.64:0/3157050077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28101960 0x7ffa28101de0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009900 tx=0x7ffa1c02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 -- 172.21.15.64:0/3157050077 shutdown_connections 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 --2- 172.21.15.64:0/3157050077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa28102320 0x7ffa28108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 --2- 172.21.15.64:0/3157050077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28101960 0x7ffa28101de0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 --2- 172.21.15.64:0/3157050077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa28100770 0x7ffa28100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 -- 172.21.15.64:0/3157050077 >> 172.21.15.64:0/3157050077 conn(0x7ffa280fbce0 msgr2=0x7ffa280fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 -- 172.21.15.64:0/3157050077 shutdown_connections 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 -- 172.21.15.64:0/3157050077 wait complete. 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 Processor -- start 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.550+0000 7ffa2d453640 1 -- start start 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa28100770 0x7ffa28069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa28101960 0x7ffa2819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 0x7ffa2819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2810c280 con 0x7ffa28102320 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2810c100 con 0x7ffa28100770 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2810c400 con 0x7ffa28101960 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 0x7ffa2819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 0x7ffa2819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39182/0 (socket says 172.21.15.64:39182) 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 -- 172.21.15.64:0/2279309593 learned_addr learned my addr 172.21.15.64:0/2279309593 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa26ffd640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa28100770 0x7ffa28069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa267fc640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa28101960 0x7ffa2819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 -- 172.21.15.64:0/2279309593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa28101960 msgr2=0x7ffa2819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa28101960 0x7ffa2819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 -- 172.21.15.64:0/2279309593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa28100770 msgr2=0x7ffa28069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa28100770 0x7ffa28069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 -- 172.21.15.64:0/2279309593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2819f6b0 con 0x7ffa28102320 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa277fe640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 0x7ffa2819edf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000cce0 tx=0x7ffa2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000f450 con 0x7ffa28102320 2024-05-09T09:09:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa2000faa0 con 0x7ffa28102320 2024-05-09T09:09:11.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2819f9a0 con 0x7ffa28102320 2024-05-09T09:09:11.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.551+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20018b70 con 0x7ffa28102320 2024-05-09T09:09:11.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.552+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2819fe70 con 0x7ffa28102320 2024-05-09T09:09:11.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.553+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa28107b60 con 0x7ffa28102320 2024-05-09T09:09:11.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.555+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa20018d10 con 0x7ffa28102320 2024-05-09T09:09:11.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.555+0000 7ffa0ffff640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa100a2b90 0x7ffa100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.555+0000 7ffa26ffd640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa100a2b90 0x7ffa100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.555+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa200cf780 con 0x7ffa28102320 2024-05-09T09:09:11.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.556+0000 7ffa26ffd640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa100a2b90 0x7ffa100a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005fd0 tx=0x7ffa1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.559+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa20014030 con 0x7ffa28102320 2024-05-09T09:09:11.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.743+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa2802d350 con 0x7ffa28102320 2024-05-09T09:09:11.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.743+0000 7ffa0ffff640 1 -- 172.21.15.64:0/2279309593 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ffa2009c3e0 con 0x7ffa28102320 2024-05-09T09:09:11.743 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa100a2b90 msgr2=0x7ffa100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa100a2b90 0x7ffa100a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005fd0 tx=0x7ffa1800b410 comp rx=0 tx=0).stop 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 msgr2=0x7ffa2819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 0x7ffa2819edf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000cce0 tx=0x7ffa2000a620 comp rx=0 tx=0).stop 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 shutdown_connections 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa100a2b90 0x7ffa100a5050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa28102320 0x7ffa2819edf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa28101960 0x7ffa2819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 --2- 172.21.15.64:0/2279309593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa28100770 0x7ffa28069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 >> 172.21.15.64:0/2279309593 conn(0x7ffa280fbce0 msgr2=0x7ffa28104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:11.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 shutdown_connections 2024-05-09T09:09:11.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.745+0000 7ffa2d453640 1 -- 172.21.15.64:0/2279309593 wait complete. 2024-05-09T09:09:11.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 -- 172.21.15.64:0/2230540610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c1023b0 msgr2=0x7f060c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 --2- 172.21.15.64:0/2230540610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c1023b0 0x7f060c1047a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009900 tx=0x7f05fc02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 -- 172.21.15.64:0/2230540610 shutdown_connections 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 --2- 172.21.15.64:0/2230540610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060c107610 0x7f060c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 --2- 172.21.15.64:0/2230540610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c104ce0 0x7f060c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 --2- 172.21.15.64:0/2230540610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c1023b0 0x7f060c1047a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 -- 172.21.15.64:0/2230540610 >> 172.21.15.64:0/2230540610 conn(0x7f060c0fbce0 msgr2=0x7f060c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 -- 172.21.15.64:0/2230540610 shutdown_connections 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.865+0000 7f0612514640 1 -- 172.21.15.64:0/2230540610 wait complete. 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 Processor -- start 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 -- start start 2024-05-09T09:09:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060c1023b0 0x7f060c101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c104ce0 0x7f060c100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 0x7f060c1008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10d3d0 con 0x7f060c107610 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10d250 con 0x7f060c1023b0 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0612514640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10d550 con 0x7f060c104ce0 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0610ab1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 0x7f060c1008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0610ab1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 0x7f060c1008a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39202/0 (socket says 172.21.15.64:39202) 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f0610ab1640 1 -- 172.21.15.64:0/3723406362 learned_addr learned my addr 172.21.15.64:0/3723406362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f060bfff640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060c1023b0 0x7f060c101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.866+0000 7f060b7fe640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c104ce0 0x7f060c100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.867+0000 7f0610ab1640 1 -- 172.21.15.64:0/3723406362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c104ce0 msgr2=0x7f060c100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.867+0000 7f0610ab1640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c104ce0 0x7f060c100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.867+0000 7f0610ab1640 1 -- 172.21.15.64:0/3723406362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060c1023b0 msgr2=0x7f060c101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.867+0000 7f0610ab1640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060c1023b0 0x7f060c101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.867+0000 7f0610ab1640 1 -- 172.21.15.64:0/3723406362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c101130 con 0x7f060c107610 2024-05-09T09:09:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.867+0000 7f0610ab1640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 0x7f060c1008a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f060400eaa0 tx=0x7f060400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.868+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060400cdb0 con 0x7f060c107610 2024-05-09T09:09:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.868+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0604005020 con 0x7f060c107610 2024-05-09T09:09:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.868+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c1a1c30 con 0x7f060c107610 2024-05-09T09:09:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.868+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040116d0 con 0x7f060c107610 2024-05-09T09:09:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.868+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c1a2180 con 0x7f060c107610 2024-05-09T09:09:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.869+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c0fd7d0 con 0x7f060c107610 2024-05-09T09:09:11.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.870+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06040051d0 con 0x7f060c107610 2024-05-09T09:09:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.871+0000 7f06097fa640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05f40a2c10 0x7f05f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.871+0000 7f060bfff640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05f40a2c10 0x7f05f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.871+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06040cf4f0 con 0x7f060c107610 2024-05-09T09:09:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.872+0000 7f060bfff640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05f40a2c10 0x7f05f40a50d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05fc006010 tx=0x7f05fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:11.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:11.877+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0604016030 con 0x7f060c107610 2024-05-09T09:09:12.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.062+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f060c0624d0 con 0x7f060c107610 2024-05-09T09:09:12.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.062+0000 7f06097fa640 1 -- 172.21.15.64:0/3723406362 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f060409ba70 con 0x7f060c107610 2024-05-09T09:09:12.061 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05f40a2c10 msgr2=0x7f05f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05f40a2c10 0x7f05f40a50d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05fc006010 tx=0x7f05fc000bc0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 msgr2=0x7f060c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 0x7f060c1008a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f060400eaa0 tx=0x7f060400ef60 comp rx=0 tx=0).stop 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 shutdown_connections 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05f40a2c10 0x7f05f40a50d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060c107610 0x7f060c1008a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c104ce0 0x7f060c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 --2- 172.21.15.64:0/3723406362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060c1023b0 0x7f060c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 >> 172.21.15.64:0/3723406362 conn(0x7f060c0fbce0 msgr2=0x7f060c102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 shutdown_connections 2024-05-09T09:09:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.064+0000 7f0612514640 1 -- 172.21.15.64:0/3723406362 wait complete. 2024-05-09T09:09:12.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:11 smithi064 ceph-mon[30149]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:12.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:11 smithi064 ceph-mon[30149]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:12.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1195224284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2115771896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:11 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2279309593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 -- 172.21.15.64:0/2079100691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 msgr2=0x7f6f6c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 --2- 172.21.15.64:0/2079100691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c101ca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009900 tx=0x7f6f5c02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 -- 172.21.15.64:0/2079100691 shutdown_connections 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 --2- 172.21.15.64:0/2079100691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 0x7f6f6c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 --2- 172.21.15.64:0/2079100691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c102a70 0x7f6f6c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 --2- 172.21.15.64:0/2079100691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c101ca0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 -- 172.21.15.64:0/2079100691 >> 172.21.15.64:0/2079100691 conn(0x7f6f6c0fce30 msgr2=0x7f6f6c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 -- 172.21.15.64:0/2079100691 shutdown_connections 2024-05-09T09:09:12.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.182+0000 7f6f72429640 1 -- 172.21.15.64:0/2079100691 wait complete. 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 Processor -- start 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 -- start start 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c102a70 0x7f6f6c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 0x7f6f6c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10d4d0 con 0x7f6f6c101880 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10d350 con 0x7f6f6c102a70 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f72429640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10d650 con 0x7f6f6c103430 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39230/0 (socket says 172.21.15.64:39230) 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 -- 172.21.15.64:0/2597883912 learned_addr learned my addr 172.21.15.64:0/2597883912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6b7fe640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c102a70 0x7f6f6c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f709c6640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 0x7f6f6c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 -- 172.21.15.64:0/2597883912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 msgr2=0x7f6f6c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 0x7f6f6c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 -- 172.21.15.64:0/2597883912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c102a70 msgr2=0x7f6f6c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c102a70 0x7f6f6c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f6bfff640 1 -- 172.21.15.64:0/2597883912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c19f890 con 0x7f6f6c101880 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.183+0000 7f6f709c6640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 0x7f6f6c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.184+0000 7f6f6bfff640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c197c80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0314e0 tx=0x7f6f5c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.184+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c03f070 con 0x7f6f6c101880 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.184+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f5c031e40 con 0x7f6f6c101880 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.184+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c19fb20 con 0x7f6f6c101880 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.184+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c035030 con 0x7f6f6c101880 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.184+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1a0010 con 0x7f6f6c101880 2024-05-09T09:09:12.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.185+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c108cc0 con 0x7f6f6c101880 2024-05-09T09:09:12.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.186+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f5c042070 con 0x7f6f6c101880 2024-05-09T09:09:12.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.187+0000 7f6f697fa640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f540a2b90 0x7f6f540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.187+0000 7f6f6b7fe640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f540a2b90 0x7f6f540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.187+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f5c0eff60 con 0x7f6f6c101880 2024-05-09T09:09:12.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.187+0000 7f6f6b7fe640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f540a2b90 0x7f6f540a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009890 tx=0x7f6f60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.191+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f5c0bcad0 con 0x7f6f6c101880 2024-05-09T09:09:12.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:11 smithi179 ceph-mon[34086]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:11 smithi179 ceph-mon[34086]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:11 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1195224284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:11 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2115771896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:11 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2279309593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:11 smithi139 ceph-mon[33599]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:12.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:11 smithi139 ceph-mon[33599]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T09:09:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:11 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1195224284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:11 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2115771896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:11 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2279309593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:09:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.414+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f6c0624d0 con 0x7f6f6c101880 2024-05-09T09:09:12.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.415+0000 7f6f697fa640 1 -- 172.21.15.64:0/2597883912 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7f6f5c0bc3d0 con 0x7f6f6c101880 2024-05-09T09:09:12.414 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:12.414 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:09:04.976828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:09:12.414 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f540a2b90 msgr2=0x7f6f540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f540a2b90 0x7f6f540a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009890 tx=0x7f6f60008040 comp rx=0 tx=0).stop 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 msgr2=0x7f6f6c197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c197c80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0314e0 tx=0x7f6f5c003d50 comp rx=0 tx=0).stop 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 shutdown_connections 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f540a2b90 0x7f6f540a5050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f6c103430 0x7f6f6c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c102a70 0x7f6f6c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 --2- 172.21.15.64:0/2597883912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f6c101880 0x7f6f6c197c80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 >> 172.21.15.64:0/2597883912 conn(0x7f6f6c0fce30 msgr2=0x7f6f6c105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 shutdown_connections 2024-05-09T09:09:12.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.417+0000 7f6f72429640 1 -- 172.21.15.64:0/2597883912 wait complete. 2024-05-09T09:09:12.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 -- 172.21.15.64:0/2893441641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 msgr2=0x7f1b281099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2893441641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b281099c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009ca0 tx=0x7f1b2002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 -- 172.21.15.64:0/2893441641 shutdown_connections 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2893441641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b281099c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2893441641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28104c40 0x7f1b28107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2893441641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28102310 0x7f1b28104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 -- 172.21.15.64:0/2893441641 >> 172.21.15.64:0/2893441641 conn(0x7f1b280fbd20 msgr2=0x7f1b280fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 -- 172.21.15.64:0/2893441641 shutdown_connections 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.533+0000 7f1b3001c640 1 -- 172.21.15.64:0/2893441641 wait complete. 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 Processor -- start 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 -- start start 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28102310 0x7f1b28101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28104c40 0x7f1b281002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b28100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2810d3c0 con 0x7f1b28107570 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2810d240 con 0x7f1b28104c40 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b3001c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2810d540 con 0x7f1b28102310 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b28100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b28100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39258/0 (socket says 172.21.15.64:39258) 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 -- 172.21.15.64:0/2386054927 learned_addr learned my addr 172.21.15.64:0/2386054927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2d5b7640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28104c40 0x7f1b281002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2ddb8640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28102310 0x7f1b28101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 -- 172.21.15.64:0/2386054927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28102310 msgr2=0x7f1b28101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28102310 0x7f1b28101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 -- 172.21.15.64:0/2386054927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28104c40 msgr2=0x7f1b281002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28104c40 0x7f1b281002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 -- 172.21.15.64:0/2386054927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b28101090 con 0x7f1b28107570 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2ddb8640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28102310 0x7f1b28101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2d5b7640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28104c40 0x7f1b281002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.534+0000 7f1b2e5b9640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b28100800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009dc0 tx=0x7f1b20003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.535+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20004040 con 0x7f1b28107570 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.535+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b20004690 con 0x7f1b28107570 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.535+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b281a1a70 con 0x7f1b28107570 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.535+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2002f4c0 con 0x7f1b28107570 2024-05-09T09:09:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.535+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b281a1fc0 con 0x7f1b28107570 2024-05-09T09:09:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.536+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b280fd810 con 0x7f1b28107570 2024-05-09T09:09:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.537+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b20030650 con 0x7f1b28107570 2024-05-09T09:09:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.538+0000 7f1b1effd640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b140a2b90 0x7f1b140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.538+0000 7f1b2ddb8640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b140a2b90 0x7f1b140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.538+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1b200f0c00 con 0x7f1b28107570 2024-05-09T09:09:12.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.538+0000 7f1b2ddb8640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b140a2b90 0x7f1b140a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400abe0 tx=0x7f1b2400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.542+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b200f7080 con 0x7f1b28107570 2024-05-09T09:09:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.729+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b28062430 con 0x7f1b28107570 2024-05-09T09:09:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.730+0000 7f1b1effd640 1 -- 172.21.15.64:0/2386054927 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f1b200bd7d0 con 0x7f1b28107570 2024-05-09T09:09:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:12.729 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:09:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b140a2b90 msgr2=0x7f1b140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b140a2b90 0x7f1b140a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400abe0 tx=0x7f1b2400a5c0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 msgr2=0x7f1b28100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b28100800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009dc0 tx=0x7f1b20003ba0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 shutdown_connections 2024-05-09T09:09:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b140a2b90 0x7f1b140a5050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28107570 0x7f1b28100800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28104c40 0x7f1b281002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 --2- 172.21.15.64:0/2386054927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b28102310 0x7f1b28101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 >> 172.21.15.64:0/2386054927 conn(0x7f1b280fbd20 msgr2=0x7f1b28102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 shutdown_connections 2024-05-09T09:09:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.732+0000 7f1b3001c640 1 -- 172.21.15.64:0/2386054927 wait complete. 2024-05-09T09:09:12.743 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T09:09:12.744 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-09T09:09:12.757 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:09:12.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-09T09:09:12.758 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T09:09:12.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:09:12.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:12.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:09:12.761 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T09:09:12.838 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T09:09:12.838 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:09:12.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-09T09:09:12.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3720935257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c102340 msgr2=0x7f9e9c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3720935257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c102340 0x7f9e9c108bd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009ca0 tx=0x7f9e9402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3720935257 shutdown_connections 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3720935257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c102340 0x7f9e9c108bd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3720935257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e9c101980 0x7f9e9c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3720935257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e9c100790 0x7f9e9c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3720935257 >> 172.21.15.64:0/3720935257 conn(0x7f9e9c0fbd20 msgr2=0x7f9e9c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3720935257 shutdown_connections 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3720935257 wait complete. 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 Processor -- start 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.871+0000 7f9ea2a4c640 1 -- start start 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea2a4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e9c100790 0x7f9e9c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea2a4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 0x7f9e9c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea2a4c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e9c102340 0x7f9e9c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea2a4c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c10c2c0 con 0x7f9e9c101980 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea2a4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c10c140 con 0x7f9e9c100790 2024-05-09T09:09:12.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea2a4c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c10c440 con 0x7f9e9c102340 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 0x7f9e9c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 0x7f9e9c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39266/0 (socket says 172.21.15.64:39266) 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 -- 172.21.15.64:0/3147196992 learned_addr learned my addr 172.21.15.64:0/3147196992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e9c100790 0x7f9e9c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e9c102340 0x7f9e9c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 -- 172.21.15.64:0/3147196992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e9c102340 msgr2=0x7f9e9c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e9c102340 0x7f9e9c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 -- 172.21.15.64:0/3147196992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e9c100790 msgr2=0x7f9e9c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e9c100790 0x7f9e9c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 -- 172.21.15.64:0/3147196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c19f790 con 0x7f9e9c101980 2024-05-09T09:09:12.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.872+0000 7f9e93fff640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 0x7f9e9c1980b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9f0 tx=0x7f9e8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.874+0000 7f9e99ffb640 1 -- 172.21.15.64:0/3147196992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400cd80 con 0x7f9e9c101980 2024-05-09T09:09:12.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.874+0000 7f9e99ffb640 1 -- 172.21.15.64:0/3147196992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e840108e0 con 0x7f9e9c101980 2024-05-09T09:09:12.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.874+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9c19fa80 con 0x7f9e9c101980 2024-05-09T09:09:12.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.874+0000 7f9e99ffb640 1 -- 172.21.15.64:0/3147196992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84018920 con 0x7f9e9c101980 2024-05-09T09:09:12.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.874+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9c1a0020 con 0x7f9e9c101980 2024-05-09T09:09:12.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.876+0000 7f9e99ffb640 1 -- 172.21.15.64:0/3147196992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e84010a90 con 0x7f9e9c101980 2024-05-09T09:09:12.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 Processor -- start 2024-05-09T09:09:12.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 -- start start 2024-05-09T09:09:12.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f33e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f3439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f344020 0x55bf2f3464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf2f2a2990 con 0x55bf2f33e4c0 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf2f24a5b0 con 0x55bf2f344020 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea5830740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf2f29e950 con 0x55bf2f33eeb0 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f33e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f33e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39270/0 (socket says 172.21.15.64:39270) 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/697224526 learned_addr learned my addr 172.21.15.64:0/697224526 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9e93fff640 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f344020 0x55bf2f3464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f3439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9e99ffb640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e880a2c10 0x7f9e880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/697224526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 msgr2=0x55bf2f3439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f3439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/697224526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f344020 msgr2=0x55bf2f3464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f344020 0x55bf2f3464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/697224526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2f07e7e0 con 0x55bf2f33e4c0 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.877+0000 7f9e99ffb640 1 -- 172.21.15.64:0/3147196992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e840cf600 con 0x7f9e9c101980 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.878+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e880a2c10 0x7f9e880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.878+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f33e8e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e94035000 tx=0x7f9e94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8021afa2499d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.878+0000 7f9e927fc640 1 -- 172.21.15.64:0/697224526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94042040 con 0x55bf2f33e4c0 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.878+0000 7f9e927fc640 1 -- 172.21.15.64:0/697224526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e94036070 con 0x55bf2f33e4c0 2024-05-09T09:09:12.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.878+0000 7f9e927fc640 1 -- 172.21.15.64:0/697224526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940305c0 con 0x55bf2f33e4c0 2024-05-09T09:09:12.880 INFO:teuthology.orchestra.run.smithi064.stdout:total 0 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 -- 172.21.15.64:0/697224526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 msgr2=0x55bf2f33e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e880a2c10 0x7f9e880a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9e9404d000 tx=0x7f9e9404e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f33e8e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e94035000 tx=0x7f9e94004a40 comp rx=0 tx=0).stop 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 -- 172.21.15.64:0/697224526 shutdown_connections 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f344020 0x55bf2f3464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f3439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 --2- 172.21.15.64:0/697224526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f33e8e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 -- 172.21.15.64:0/697224526 >> 172.21.15.64:0/697224526 conn(0x55bf2f3060a0 msgr2=0x55bf2f339f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 -- 172.21.15.64:0/697224526 shutdown_connections 2024-05-09T09:09:12.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 -- 172.21.15.64:0/697224526 wait complete. 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 -- start start 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.879+0000 7f9ea5830740 1 Processor -- start 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f348970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f348eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f344020 0x55bf2f34fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf2f346fd0 con 0x55bf2f33e4c0 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf2f25db40 con 0x55bf2f33eeb0 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf2f347150 con 0x55bf2f344020 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea5830740 10 client.0 _ll_register_callbacks cb 0x55bf2f2520f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f348970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f348970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39282/0 (socket says 172.21.15.64:39282) 2024-05-09T09:09:12.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 -- 172.21.15.64:0/3903256526 learned_addr learned my addr 172.21.15.64:0/3903256526 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e93fff640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f348eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f344020 0x55bf2f34fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 -- 172.21.15.64:0/3903256526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f344020 msgr2=0x55bf2f34fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f344020 0x55bf2f34fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 -- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 msgr2=0x55bf2f348eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f348eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 -- 172.21.15.64:0/3903256526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2f0def70 con 0x55bf2f33e4c0 2024-05-09T09:09:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f348970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c012700 tx=0x7f9e8c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8797dcc8e7e303bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.880+0000 7f9e7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/3903256526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c004030 con 0x55bf2f33e4c0 2024-05-09T09:09:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e94036070 con 0x55bf2f33e4c0 2024-05-09T09:09:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/3903256526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e8c0041d0 con 0x55bf2f33e4c0 2024-05-09T09:09:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/3903256526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c025050 con 0x55bf2f33e4c0 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf2ef76170 con 0x55bf2f33e4c0 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/3903256526 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e94036070 con 0x55bf2f33e4c0 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/3903256526 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e8c0038e0 con 0x55bf2f33e4c0 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 20 client.15600 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f354540 0x55bf2f356a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 4 client.15600 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.881+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf2f2b00c0 con 0x55bf2f354540 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.882+0000 7f9e93fff640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f354540 0x55bf2f356a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.882+0000 7f9e7b7fe640 10 client.15600 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.882+0000 7f9e93fff640 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f354540 0x55bf2f356a00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83dd93e400bcbeb6 server_cookie=d9d02a8efbf6ea3f in_seq=0 out_seq=0 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/3903256526 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf2f2b00c0 con 0x55bf2f354540 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9e7b7fe640 10 client.15600 handle_command_reply: tid=0 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 client.15600 shutdown 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 msgr2=0x55bf2f348970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f348970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c012700 tx=0x7f9e8c012bc0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 shutdown_connections 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f354540 0x55bf2f356a00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f344020 0x55bf2f34fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9e7b7fe640 0 client.15600 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f348eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 --2- 172.21.15.64:0/3903256526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f33e4c0 0x55bf2f348970 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 >> 172.21.15.64:0/3903256526 conn(0x55bf2f3060a0 msgr2=0x55bf2f339080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 shutdown_connections 2024-05-09T09:09:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 1 -- 172.21.15.64:0/3903256526 wait complete. 2024-05-09T09:09:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.883+0000 7f9ea5830740 20 client.15600 trim_cache size 0 max 16384 2024-05-09T09:09:12.892 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-09T09:09:12.893 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-fuse 1.1T 384M 1.1T 1% /tmp/cephfs 2024-05-09T09:09:12.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-09T09:09:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 Processor -- start 2024-05-09T09:09:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 -- start start 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33e4c0 0x55bf2f35c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f35c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1accd0 con 0x55bf2f344020 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1aa5a0 con 0x55bf2f33e4c0 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.905+0000 7f9ea5830740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1a83f0 con 0x55bf2f33eeb0 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39298/0 (socket says 172.21.15.64:39298) 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/294144944 learned_addr learned my addr 172.21.15.64:0/294144944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33e4c0 0x55bf2f35c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9e93fff640 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f35c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/294144944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 msgr2=0x55bf2f35c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f35c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/294144944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33e4c0 msgr2=0x55bf2f35c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33e4c0 0x55bf2f35c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/294144944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2f343580 con 0x55bf2f344020 2024-05-09T09:09:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33e4c0 0x55bf2f35c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34feb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9e94035000 tx=0x7f9e94051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d521b7065c6aa67d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9e91ffb640 1 -- 172.21.15.64:0/294144944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94042040 con 0x55bf2f344020 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9e91ffb640 1 -- 172.21.15.64:0/294144944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e94036070 con 0x55bf2f344020 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.906+0000 7f9e91ffb640 1 -- 172.21.15.64:0/294144944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004040 con 0x55bf2f344020 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 -- 172.21.15.64:0/294144944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 msgr2=0x55bf2f34feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34feb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9e94035000 tx=0x7f9e94051000 comp rx=0 tx=0).stop 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 -- 172.21.15.64:0/294144944 shutdown_connections 2024-05-09T09:09:12.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34feb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33eeb0 0x55bf2f35c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 --2- 172.21.15.64:0/294144944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33e4c0 0x55bf2f35c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 -- 172.21.15.64:0/294144944 >> 172.21.15.64:0/294144944 conn(0x55bf2f3060a0 msgr2=0x55bf2f33cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 -- 172.21.15.64:0/294144944 shutdown_connections 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 -- 172.21.15.64:0/294144944 wait complete. 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.907+0000 7f9ea5830740 1 -- start start 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 Processor -- start 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33e4c0 0x55bf2f34d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f34de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1a8a60 con 0x55bf2f344020 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1a9ed0 con 0x55bf2f33eeb0 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c1a9130 con 0x55bf2f33e4c0 2024-05-09T09:09:12.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea5830740 10 client.0 _ll_register_callbacks cb 0x7f9e8c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39306/0 (socket says 172.21.15.64:39306) 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/2465531882 learned_addr learned my addr 172.21.15.64:0/2465531882 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33e4c0 0x55bf2f34d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9e93fff640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f34de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/2465531882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33e4c0 msgr2=0x55bf2f34d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33e4c0 0x55bf2f34d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 msgr2=0x55bf2f34de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f34de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.908+0000 7f9ea0fe9640 1 -- 172.21.15.64:0/2465531882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2f34f420 con 0x55bf2f344020 2024-05-09T09:09:12.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea0fe9640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34ee30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e9402a900 tx=0x7f9e940305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32695e087dbf959 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9e7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/2465531882 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94042030 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/2465531882 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e94003c30 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e94036070 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/2465531882 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e94030bc0 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf2f3bae00 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/2465531882 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e94036070 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/2465531882 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e9402f650 con 0x55bf2f344020 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 20 client.15606 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f3bb2b0 0x55bf2f3bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 4 client.15606 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.909+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bf2f2aa410 con 0x55bf2f3bb2b0 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.910+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f3bb2b0 0x55bf2f3bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.910+0000 7f9e9bfff640 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f3bb2b0 0x55bf2f3bd6d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316b6c064155435f server_cookie=d270351b3f9fd4cc in_seq=0 out_seq=0 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.910+0000 7f9e7b7fe640 10 client.15606 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9e7b7fe640 1 -- 172.21.15.64:0/2465531882 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bf2f2aa410 con 0x55bf2f3bb2b0 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9e7b7fe640 10 client.15606 handle_command_reply: tid=0 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 client.15606 shutdown 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 msgr2=0x55bf2f34ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34ee30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e9402a900 tx=0x7f9e940305c0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 shutdown_connections 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf2f3bb2b0 0x55bf2f3bd6d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf2f344020 0x55bf2f34ee30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf2f33eeb0 0x55bf2f34de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9e7b7fe640 0 client.15606 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.911+0000 7f9ea5830740 1 --2- 172.21.15.64:0/2465531882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf2f33e4c0 0x55bf2f34d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 >> 172.21.15.64:0/2465531882 conn(0x55bf2f3060a0 msgr2=0x55bf2f33d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 shutdown_connections 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea5830740 1 -- 172.21.15.64:0/2465531882 wait complete. 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea5830740 20 client.15606 trim_cache size 0 max 16384 2024-05-09T09:09:12.917 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:09:12.918 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:09:12.918 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "86574db8-c607-48b4-8dde-9472172efdf1", 2024-05-09T09:09:12.918 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:09:12.918 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e880a2c10 msgr2=0x7f9e880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e880a2c10 0x7f9e880a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9e9404d000 tx=0x7f9e9404e000 comp rx=0 tx=0).stop 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 msgr2=0x7f9e9c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 0x7f9e9c1980b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9f0 tx=0x7f9e8400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 shutdown_connections 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e880a2c10 0x7f9e880a50d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e9c102340 0x7f9e9c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e9c101980 0x7f9e9c1980b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 --2- 172.21.15.64:0/3147196992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e9c100790 0x7f9e9c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:12.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.912+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 >> 172.21.15.64:0/3147196992 conn(0x7f9e9c0fbd20 msgr2=0x7f9e9c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:12.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.913+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 shutdown_connections 2024-05-09T09:09:12.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:12.913+0000 7f9ea2a4c640 1 -- 172.21.15.64:0/3147196992 wait complete. 2024-05-09T09:09:12.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:09:12.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:09:12.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-09T09:09:13.020 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:12 smithi064 ceph-mon[30149]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T09:09:13.020 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:12 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3723406362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:13.020 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:12 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2597883912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:13.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:12 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2386054927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:13.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 -- 172.21.15.64:0/2081337897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec102a70 msgr2=0x7fdfec102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/2081337897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec102a70 0x7fdfec102ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009900 tx=0x7fdfd402a410 comp rx=0 tx=0).stop 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 -- 172.21.15.64:0/2081337897 shutdown_connections 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/2081337897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfec103430 0x7fdfec109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/2081337897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec102a70 0x7fdfec102ef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/2081337897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfec101880 0x7fdfec101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 -- 172.21.15.64:0/2081337897 >> 172.21.15.64:0/2081337897 conn(0x7fdfec0fce10 msgr2=0x7fdfec0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:13.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 -- 172.21.15.64:0/2081337897 shutdown_connections 2024-05-09T09:09:13.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.108+0000 7fdff0fb7640 1 -- 172.21.15.64:0/2081337897 wait complete. 2024-05-09T09:09:13.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 Processor -- start 2024-05-09T09:09:13.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 -- start start 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 0x7fdfec197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfec102a70 0x7fdfec198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfec103430 0x7fdfec19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec10d3b0 con 0x7fdfec101880 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec10d230 con 0x7fdfec102a70 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdff0fb7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec10d530 con 0x7fdfec103430 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 0x7fdfec197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 0x7fdfec197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39324/0 (socket says 172.21.15.64:39324) 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 -- 172.21.15.64:0/1949146993 learned_addr learned my addr 172.21.15.64:0/1949146993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfec102a70 0x7fdfec198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfead9d640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfec103430 0x7fdfec19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 -- 172.21.15.64:0/1949146993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfec103430 msgr2=0x7fdfec19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfec103430 0x7fdfec19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 -- 172.21.15.64:0/1949146993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfec102a70 msgr2=0x7fdfec198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfec102a70 0x7fdfec198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.109+0000 7fdfea59c640 1 -- 172.21.15.64:0/1949146993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec19f730 con 0x7fdfec101880 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.110+0000 7fdfea59c640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 0x7fdfec197b10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00eb70 tx=0x7fdfdc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.110+0000 7fdfe37fe640 1 -- 172.21.15.64:0/1949146993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc003b70 con 0x7fdfec101880 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.110+0000 7fdfe37fe640 1 -- 172.21.15.64:0/1949146993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfdc00ee70 con 0x7fdfec101880 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.110+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec19fa20 con 0x7fdfec101880 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.110+0000 7fdfe37fe640 1 -- 172.21.15.64:0/1949146993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc010650 con 0x7fdfec101880 2024-05-09T09:09:13.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.110+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec19fef0 con 0x7fdfec101880 2024-05-09T09:09:13.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.112+0000 7fdfe37fe640 1 -- 172.21.15.64:0/1949146993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfdc004020 con 0x7fdfec101880 2024-05-09T09:09:13.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.113+0000 7fdfe37fe640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfd80a2b90 0x7fdfd80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.113+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfd80a2b90 0x7fdfd80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.113+0000 7fdfe37fe640 1 -- 172.21.15.64:0/1949146993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfdc0cfa40 con 0x7fdfec101880 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 Processor -- start 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfd80a2b90 0x7fdfd80a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4006010 tx=0x7fdfd4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 -- start start 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091bd9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bde420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560091b3dd00 con 0x560091bd9920 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560091ae54c0 con 0x560091bd8f30 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdff3da7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560091b39cc0 con 0x560091bdea90 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bde420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bde420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39326/0 (socket says 172.21.15.64:39326) 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1459624606 learned_addr learned my addr 172.21.15.64:0/1459624606 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdfead9d640 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.114+0000 7fdfea59c640 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091bd9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1459624606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 msgr2=0x560091be0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1459624606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 msgr2=0x560091bd9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091bd9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1459624606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009191b7d0 con 0x560091bd9920 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfea59c640 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091bd9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bde420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009900 tx=0x7fdfd403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789290765a4c29ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:13.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe0ff9640 1 -- 172.21.15.64:0/1459624606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd404f070 con 0x560091bd9920 2024-05-09T09:09:13.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe0ff9640 1 -- 172.21.15.64:0/1459624606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfd40035f0 con 0x560091bd9920 2024-05-09T09:09:13.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.115+0000 7fdfe0ff9640 1 -- 172.21.15.64:0/1459624606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4035050 con 0x560091bd9920 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 -- 172.21.15.64:0/1459624606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 msgr2=0x560091bde420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bde420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009900 tx=0x7fdfd403e000 comp rx=0 tx=0).stop 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 -- 172.21.15.64:0/1459624606 shutdown_connections 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bde420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1459624606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091bd9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 -- 172.21.15.64:0/1459624606 >> 172.21.15.64:0/1459624606 conn(0x560091bd45a0 msgr2=0x560091bd49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 -- 172.21.15.64:0/1459624606 shutdown_connections 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.116+0000 7fdff3da7740 1 -- 172.21.15.64:0/1459624606 wait complete. 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 -- start start 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 Processor -- start 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd9920 0x560091be3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 0x560091bea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560091be19e0 con 0x560091bdea90 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560091af8b90 con 0x560091bd8f30 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560091be1b60 con 0x560091bd9920 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdff3da7740 10 client.0 _ll_register_callbacks cb 0x560091b34da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 0x560091bea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 0x560091bea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39332/0 (socket says 172.21.15.64:39332) 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 -- 172.21.15.64:0/883026324 learned_addr learned my addr 172.21.15.64:0/883026324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfea59c640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd9920 0x560091be3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 -- 172.21.15.64:0/883026324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd9920 msgr2=0x560091be3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd9920 0x560091be3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 -- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 msgr2=0x560091be32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.117+0000 7fdfead9d640 1 -- 172.21.15.64:0/883026324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600918513f0 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfead9d640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 0x560091bea870 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b930 tx=0x7fdfe400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3767c96aa012746a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/883026324 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401b440 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/883026324 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfe401b5e0 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd40035f0 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/883026324 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfe4024050 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600918110f0 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/883026324 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfd40035f0 con 0x560091bdea90 2024-05-09T09:09:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/883026324 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdfe4019020 con 0x560091bdea90 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 20 client.15618 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091bf1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 4 client.15618 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.118+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560091b4b430 con 0x560091bef1b0 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.119+0000 7fdfea59c640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091bf1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.119+0000 7fdfea59c640 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091bf1670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582eb07261caf9a6 server_cookie=4e0c7efaf4987c36 in_seq=0 out_seq=0 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.119+0000 7fdfd1ffb640 10 client.15618 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/883026324 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560091b4b430 con 0x560091bef1b0 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdfd1ffb640 10 client.15618 handle_command_reply: tid=0 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 client.15618 shutdown 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 msgr2=0x560091bea870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 0x560091bea870 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b930 tx=0x7fdfe400bdf0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 shutdown_connections 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091bf1670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdfd1ffb640 0 client.15618 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bdea90 0x560091bea870 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd9920 0x560091be3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 --2- 172.21.15.64:0/883026324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 >> 172.21.15.64:0/883026324 conn(0x560091bd45a0 msgr2=0x560091bd37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 shutdown_connections 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 1 -- 172.21.15.64:0/883026324 wait complete. 2024-05-09T09:09:13.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.120+0000 7fdff3da7740 20 client.15618 trim_cache size 0 max 16384 2024-05-09T09:09:13.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 Processor -- start 2024-05-09T09:09:13.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 -- start start 2024-05-09T09:09:13.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bdea90 0x560091be3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1aaae0 con 0x560091bd9920 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1a83b0 con 0x560091bdea90 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdff3da7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1a6200 con 0x560091bd8f30 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39336/0 (socket says 172.21.15.64:39336) 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1162873477 learned_addr learned my addr 172.21.15.64:0/1162873477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.141+0000 7fdfea59c640 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfead9d640 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bdea90 0x560091be3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1162873477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd8f30 msgr2=0x560091be32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1162873477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bdea90 msgr2=0x560091be3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bdea90 0x560091be3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/1162873477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560091bd43d0 con 0x560091bd9920 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bea830 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdfd404d040 tx=0x7fdfd4031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac46d16009a89f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfd3fff640 1 -- 172.21.15.64:0/1162873477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4057020 con 0x560091bd9920 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfd3fff640 1 -- 172.21.15.64:0/1162873477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfd4004240 con 0x560091bd9920 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.142+0000 7fdfd3fff640 1 -- 172.21.15.64:0/1162873477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4035050 con 0x560091bd9920 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 -- 172.21.15.64:0/1162873477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 msgr2=0x560091bea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bea830 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdfd404d040 tx=0x7fdfd4031a10 comp rx=0 tx=0).stop 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 -- 172.21.15.64:0/1162873477 shutdown_connections 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bdea90 0x560091be3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091bea830 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 --2- 172.21.15.64:0/1162873477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bd8f30 0x560091be32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 -- 172.21.15.64:0/1162873477 >> 172.21.15.64:0/1162873477 conn(0x560091bd45a0 msgr2=0x560091bd4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 -- 172.21.15.64:0/1162873477 shutdown_connections 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 -- 172.21.15.64:0/1162873477 wait complete. 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 -- start start 2024-05-09T09:09:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.143+0000 7fdff3da7740 1 Processor -- start 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091be8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1a6870 con 0x560091bd9920 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1a7ce0 con 0x560091bd8f30 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1a6f40 con 0x560091bdea90 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdff3da7740 10 client.0 _ll_register_callbacks cb 0x7fdfe4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091be8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091be8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39340/0 (socket says 172.21.15.64:39340) 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/4233562814 learned_addr learned my addr 172.21.15.64:0/4233562814 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfea59c640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfead9d640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/4233562814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 msgr2=0x560091be9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 msgr2=0x560091be8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 -- 172.21.15.64:0/4233562814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560091be9b60 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfe9d9b640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091be8560 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009800 tx=0x7fdfd402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897316f7a007a293 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.144+0000 7fdfd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/4233562814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4057030 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/4233562814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfd40035f0 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd4004240 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/4233562814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfd4035020 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560091c55810 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/4233562814 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfd4004240 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/4233562814 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdfd4052030 con 0x560091bd9920 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 20 client.15624 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091c57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:13.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 4 client.15624 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:09:13.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.145+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560091b45780 con 0x560091bef1b0 2024-05-09T09:09:13.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.146+0000 7fdfea59c640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091c57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:13.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.146+0000 7fdfea59c640 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091c57d70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8419f9b9fa4eb0d0 server_cookie=6fbdfef286638476 in_seq=0 out_seq=0 2024-05-09T09:09:13.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.146+0000 7fdfd1ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:13.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.146+0000 7fdfd1ffb640 1 -- 172.21.15.64:0/4233562814 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560091b45780 con 0x560091bef1b0 2024-05-09T09:09:13.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.146+0000 7fdfd1ffb640 10 client.15624 handle_command_reply: tid=0 2024-05-09T09:09:13.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.146+0000 7fdff3da7740 1 client.15624 shutdown 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 msgr2=0x560091be8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091be8560 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009800 tx=0x7fdfd402a410 comp rx=0 tx=0).stop 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 shutdown_connections 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560091bef1b0 0x560091c57d70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560091bdea90 0x560091be9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560091bd9920 0x560091be8560 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 --2- 172.21.15.64:0/4233562814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560091bd8f30 0x560091be8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdfd1ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 >> 172.21.15.64:0/4233562814 conn(0x560091bd45a0 msgr2=0x560091be32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 shutdown_connections 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 1 -- 172.21.15.64:0/4233562814 wait complete. 2024-05-09T09:09:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff3da7740 20 client.15624 trim_cache size 0 max 16384 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "86574db8-c607-48b4-8dde-9472172efdf1": { 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "86574db8-c607-48b4-8dde-9472172efdf1", 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:09:13.152 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfd80a2b90 msgr2=0x7fdfd80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfd80a2b90 0x7fdfd80a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4006010 tx=0x7fdfd4000bc0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 msgr2=0x7fdfec197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 0x7fdfec197b10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00eb70 tx=0x7fdfdc00c6a0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 shutdown_connections 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfd80a2b90 0x7fdfd80a5050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfec103430 0x7fdfec19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfec102a70 0x7fdfec198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 --2- 172.21.15.64:0/1949146993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfec101880 0x7fdfec197b10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.147+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 >> 172.21.15.64:0/1949146993 conn(0x7fdfec0fce10 msgr2=0x7fdfec105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.148+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 shutdown_connections 2024-05-09T09:09:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:13.148+0000 7fdff0fb7640 1 -- 172.21.15.64:0/1949146993 wait complete. 2024-05-09T09:09:13.158 INFO:tasks.cephfs.filesystem:scrub status for tag:86574db8-c607-48b4-8dde-9472172efdf1 - {'path': '/', 'tag': '86574db8-c607-48b4-8dde-9472172efdf1', 'options': 'recursive,force'} 2024-05-09T09:09:13.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:12 smithi179 ceph-mon[34086]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T09:09:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:12 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3723406362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:12 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2597883912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:12 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2386054927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:13.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:12 smithi139 ceph-mon[33599]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T09:09:13.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:12 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3723406362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:09:13.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:12 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2597883912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:12 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2386054927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:09:14.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:13 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:09:14.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:13 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:13 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:09:14.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:13 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:14.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:13 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:09:14.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:13 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:14.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:13 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:09:14.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:13 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:14.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:13 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:09:14.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:13 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:14.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:13 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:09:14.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:13 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:14.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-09T09:09:14.504 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-09T09:09:14.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-09T09:09:14.587 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-09T09:09:14.670 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/: 2024-05-09T09:09:14.670 INFO:teuthology.orchestra.run.smithi064.stdout:total 2 2024-05-09T09:09:14.670 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 .. 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 data 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data: 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:14.671 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 .. 2024-05-09T09:09:14.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-09T09:09:14.758 INFO:teuthology.orchestra.run.smithi064.stdout:[Service] 2024-05-09T09:09:14.758 INFO:teuthology.orchestra.run.smithi064.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-09T09:09:14.758 INFO:teuthology.orchestra.run.smithi064.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-09T09:09:14.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-09T09:09:14.843 INFO:teuthology.orchestra.run.smithi064.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-09T09:09:14.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:14 smithi064 ceph-mon[30149]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T09:09:15.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:14 smithi179 ceph-mon[34086]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T09:09:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:14 smithi139 ceph-mon[33599]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T09:09:17.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:16 smithi064 ceph-mon[30149]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T09:09:17.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:16 smithi179 ceph-mon[34086]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T09:09:17.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:16 smithi139 ceph-mon[33599]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T09:09:19.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:18 smithi064 ceph-mon[30149]: pgmap v375: 129 pgs: 129 active+clean; 5.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 413 KiB/s wr, 7 op/s 2024-05-09T09:09:19.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:18 smithi179 ceph-mon[34086]: pgmap v375: 129 pgs: 129 active+clean; 5.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 413 KiB/s wr, 7 op/s 2024-05-09T09:09:19.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:18 smithi139 ceph-mon[33599]: pgmap v375: 129 pgs: 129 active+clean; 5.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 413 KiB/s wr, 7 op/s 2024-05-09T09:09:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:20 smithi064 ceph-mon[30149]: pgmap v376: 129 pgs: 129 active+clean; 23 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.9 MiB/s wr, 33 op/s 2024-05-09T09:09:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:20 smithi179 ceph-mon[34086]: pgmap v376: 129 pgs: 129 active+clean; 23 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.9 MiB/s wr, 33 op/s 2024-05-09T09:09:21.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:20 smithi139 ceph-mon[33599]: pgmap v376: 129 pgs: 129 active+clean; 23 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.9 MiB/s wr, 33 op/s 2024-05-09T09:09:22.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:21 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:09:22.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:21 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:09:22.242 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:21 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:09:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:22 smithi064 ceph-mon[30149]: pgmap v377: 129 pgs: 129 active+clean; 27 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.5 MiB/s wr, 49 op/s 2024-05-09T09:09:23.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:22 smithi179 ceph-mon[34086]: pgmap v377: 129 pgs: 129 active+clean; 27 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.5 MiB/s wr, 49 op/s 2024-05-09T09:09:23.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:22 smithi139 ceph-mon[33599]: pgmap v377: 129 pgs: 129 active+clean; 27 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.5 MiB/s wr, 49 op/s 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: pgmap v378: 129 pgs: 129 active+clean; 34 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:09:24.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:09:24.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: pgmap v378: 129 pgs: 129 active+clean; 34 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-05-09T09:09:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:09:24.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:09:24.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: pgmap v378: 129 pgs: 129 active+clean; 34 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.4 MiB/s wr, 77 op/s 2024-05-09T09:09:24.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:09:24.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:09:25.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:09:25.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:09:25.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:09:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 -- 172.21.15.64:0/368530941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30102340 msgr2=0x7faf30108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 --2- 172.21.15.64:0/368530941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30102340 0x7faf30108bd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faf28009ca0 tx=0x7faf2802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 -- 172.21.15.64:0/368530941 shutdown_connections 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 --2- 172.21.15.64:0/368530941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30102340 0x7faf30108bd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 --2- 172.21.15.64:0/368530941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30101980 0x7faf30101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 --2- 172.21.15.64:0/368530941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100790 0x7faf30100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.910+0000 7faf36f77640 1 -- 172.21.15.64:0/368530941 >> 172.21.15.64:0/368530941 conn(0x7faf300fbd00 msgr2=0x7faf300fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 -- 172.21.15.64:0/368530941 shutdown_connections 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 -- 172.21.15.64:0/368530941 wait complete. 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 Processor -- start 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 -- start start 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30100790 0x7faf301979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 0x7faf30197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30102340 0x7faf3019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf3010c2a0 con 0x7faf30101980 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf3010c120 con 0x7faf30100790 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf36f77640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf3010c420 con 0x7faf30102340 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.911+0000 7faf2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 0x7faf30197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 0x7faf30197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53230/0 (socket says 172.21.15.64:53230) 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 -- 172.21.15.64:0/883696053 learned_addr learned my addr 172.21.15.64:0/883696053 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf35514640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30102340 0x7faf3019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf34d13640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30100790 0x7faf301979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 -- 172.21.15.64:0/883696053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30102340 msgr2=0x7faf3019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30102340 0x7faf3019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 -- 172.21.15.64:0/883696053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30100790 msgr2=0x7faf301979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30100790 0x7faf301979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 -- 172.21.15.64:0/883696053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3019f5d0 con 0x7faf30101980 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf35514640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30102340 0x7faf3019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf34d13640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30100790 0x7faf301979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2ffff640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 0x7faf30197ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00ea60 tx=0x7faf1c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2dffb640 1 -- 172.21.15.64:0/883696053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00cde0 con 0x7faf30101980 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2dffb640 1 -- 172.21.15.64:0/883696053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf1c0035f0 con 0x7faf30101980 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3019f8c0 con 0x7faf30101980 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.912+0000 7faf2dffb640 1 -- 172.21.15.64:0/883696053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c010690 con 0x7faf30101980 2024-05-09T09:09:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.913+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3019fe10 con 0x7faf30101980 2024-05-09T09:09:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.915+0000 7faf2dffb640 1 -- 172.21.15.64:0/883696053 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf1c0037a0 con 0x7faf30101980 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.915+0000 7faf39d63740 1 Processor -- start 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.915+0000 7faf39d63740 1 -- start start 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf39d63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc868670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf39d63740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc86d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf39d63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc870240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf39d63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9bc7cc340 con 0x55d9bc86ddb0 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf39d63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9bc774290 con 0x55d9bc868250 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf39d63740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9bc7c8300 con 0x55d9bc868c40 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc868670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc868670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:38104/0 (socket says 172.21.15.64:38104) 2024-05-09T09:09:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 -- 172.21.15.64:0/3508699766 learned_addr learned my addr 172.21.15.64:0/3508699766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf2ffff640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc870240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf34d13640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc86d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 -- 172.21.15.64:0/3508699766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 msgr2=0x55d9bc86d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc86d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 -- 172.21.15.64:0/3508699766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 msgr2=0x55d9bc870240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc870240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 -- 172.21.15.64:0/3508699766 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bc7988e0 con 0x55d9bc868250 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf2ffff640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc870240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf34d13640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc86d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf2dffb640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf200a2c30 0x7faf200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf34d13640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf200a2c30 0x7faf200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf2dffb640 1 -- 172.21.15.64:0/883696053 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faf1c0cf660 con 0x7faf30101980 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.916+0000 7faf35514640 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc868670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf28009ca0 tx=0x7faf28009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f4e9bc8268b5ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf1affd640 1 -- 172.21.15.64:0/3508699766 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28031d40 con 0x55d9bc868250 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf1affd640 1 -- 172.21.15.64:0/3508699766 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf28036030 con 0x55d9bc868250 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf1affd640 1 -- 172.21.15.64:0/3508699766 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28004850 con 0x55d9bc868250 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 -- 172.21.15.64:0/3508699766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 msgr2=0x55d9bc868670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf34d13640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf200a2c30 0x7faf200a50f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faf240095f0 tx=0x7faf24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc868670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf28009ca0 tx=0x7faf28009a20 comp rx=0 tx=0).stop 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 -- 172.21.15.64:0/3508699766 shutdown_connections 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc870240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc86d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 --2- 172.21.15.64:0/3508699766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc868670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 -- 172.21.15.64:0/3508699766 >> 172.21.15.64:0/3508699766 conn(0x55d9bc8638c0 msgr2=0x55d9bc863cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 -- 172.21.15.64:0/3508699766 shutdown_connections 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.917+0000 7faf39d63740 1 -- 172.21.15.64:0/3508699766 wait complete. 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 -- start start 2024-05-09T09:09:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 Processor -- start 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc876640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc876b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9bc874f90 con 0x55d9bc86ddb0 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9bc787790 con 0x55d9bc868250 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.918+0000 7faf39d63740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9bc875110 con 0x55d9bc868c40 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf39d63740 10 client.0 _ll_register_callbacks cb 0x55d9bc7c2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc876640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf35514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf2ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc876b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc876640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:38112/0 (socket says 172.21.15.64:38112) 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 -- 172.21.15.64:0/3999493198 learned_addr learned my addr 172.21.15.64:0/3999493198 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 -- 172.21.15.64:0/3999493198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 msgr2=0x55d9bc876b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc876b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 -- 172.21.15.64:0/3999493198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 msgr2=0x55d9bc87dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 -- 172.21.15.64:0/3999493198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bc4e0210 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf2ffff640 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc876b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf34d13640 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc876640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf2400fb30 tx=0x7faf24004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d331343b3a0d444 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf0ffff640 1 -- 172.21.15.64:0/3999493198 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24005020 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf28036030 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.919+0000 7faf0ffff640 1 -- 172.21.15.64:0/3999493198 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faf24018030 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf0ffff640 1 -- 172.21.15.64:0/3999493198 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf24034050 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 10 client.25214 fetch_fsmap learned FSMap version 26 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9bc5ac620 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf0ffff640 1 -- 172.21.15.64:0/3999493198 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf28036030 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf0ffff640 1 -- 172.21.15.64:0/3999493198 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7faf2401e5a0 con 0x55d9bc868250 2024-05-09T09:09:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 10 client.25214 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 10 client.25214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 20 client.25214 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc882120 0x55d9bc8845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 4 client.25214 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9bc7d9a70 con 0x55d9bc882120 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.920+0000 7faf2ffff640 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc882120 0x55d9bc8845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf2ffff640 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc882120 0x55d9bc8845c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df13c8759b4fc96c server_cookie=25b03f5a955e499e in_seq=0 out_seq=0 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf0ffff640 10 client.25214 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf0ffff640 1 -- 172.21.15.64:0/3999493198 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9bc7d9a70 con 0x55d9bc882120 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf0ffff640 10 client.25214 handle_command_reply: tid=0 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 client.25214 shutdown 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 msgr2=0x55d9bc876640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc876640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf2400fb30 tx=0x7faf24004730 comp rx=0 tx=0).stop 2024-05-09T09:09:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 shutdown_connections 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc882120 0x55d9bc8845c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc876b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 --2- 172.21.15.64:0/3999493198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc876640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf0ffff640 0 client.25214 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.921+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 >> 172.21.15.64:0/3999493198 conn(0x55d9bc8638c0 msgr2=0x55d9bc862a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.922+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 shutdown_connections 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.922+0000 7faf39d63740 1 -- 172.21.15.64:0/3999493198 wait complete. 2024-05-09T09:09:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.922+0000 7faf39d63740 20 client.25214 trim_cache size 0 max 16384 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 Processor -- start 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 -- start start 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 0x55d9bc8671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868c40 0x55d9bc87db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc86ddb0 0x55d9bc863ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf301aa960 con 0x55d9bc868250 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf301a8230 con 0x55d9bc868c40 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf39d63740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf301a6080 con 0x55d9bc86ddb0 2024-05-09T09:09:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 0x55d9bc8671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 0x55d9bc8671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53246/0 (socket says 172.21.15.64:53246) 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 -- 172.21.15.64:0/911327549 learned_addr learned my addr 172.21.15.64:0/911327549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf35514640 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc86ddb0 0x55d9bc863ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf2ffff640 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868c40 0x55d9bc87db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 -- 172.21.15.64:0/911327549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc86ddb0 msgr2=0x55d9bc863ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc86ddb0 0x55d9bc863ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 -- 172.21.15.64:0/911327549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868c40 msgr2=0x55d9bc87db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868c40 0x55d9bc87db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.940+0000 7faf34d13640 1 -- 172.21.15.64:0/911327549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bc867e90 con 0x55d9bc868250 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.941+0000 7faf34d13640 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 0x55d9bc8671e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf24012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a4c4cdca629e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.941+0000 7faf1a7fc640 1 -- 172.21.15.64:0/911327549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2403d070 con 0x55d9bc868250 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.941+0000 7faf1a7fc640 1 -- 172.21.15.64:0/911327549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf2401ed10 con 0x55d9bc868250 2024-05-09T09:09:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.941+0000 7faf1a7fc640 1 -- 172.21.15.64:0/911327549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240119e0 con 0x55d9bc868250 2024-05-09T09:09:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 -- 172.21.15.64:0/911327549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 msgr2=0x55d9bc8671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 0x55d9bc8671e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf24012f50 comp rx=0 tx=0).stop 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 -- 172.21.15.64:0/911327549 shutdown_connections 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc86ddb0 0x55d9bc863ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868c40 0x55d9bc87db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 --2- 172.21.15.64:0/911327549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc868250 0x55d9bc8671e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 -- 172.21.15.64:0/911327549 >> 172.21.15.64:0/911327549 conn(0x55d9bc8638c0 msgr2=0x55d9bc875470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 -- 172.21.15.64:0/911327549 shutdown_connections 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 -- 172.21.15.64:0/911327549 wait complete. 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 -- start start 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.942+0000 7faf39d63740 1 Processor -- start 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc87b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc87bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf301a66f0 con 0x55d9bc86ddb0 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf301a7b60 con 0x55d9bc868250 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf301a6dc0 con 0x55d9bc868c40 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf39d63740 10 client.0 _ll_register_callbacks cb 0x7faf24005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc87b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf2ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc87bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc87b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:38116/0 (socket says 172.21.15.64:38116) 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf35514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 -- 172.21.15.64:0/1570458467 learned_addr learned my addr 172.21.15.64:0/1570458467 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 -- 172.21.15.64:0/1570458467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 msgr2=0x55d9bc87bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc87bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 -- 172.21.15.64:0/1570458467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 msgr2=0x55d9bc87cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 -- 172.21.15.64:0/1570458467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bc87d1a0 con 0x55d9bc868250 2024-05-09T09:09:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf34d13640 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc87b660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf24005eb0 tx=0x7faf2401e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e25d0da9107bee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.943+0000 7faf0ffff640 1 -- 172.21.15.64:0/1570458467 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2403d070 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2401ed10 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf0ffff640 1 -- 172.21.15.64:0/1570458467 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faf24005250 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf0ffff640 1 -- 172.21.15.64:0/1570458467 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf2400f900 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 10 client.25220 fetch_fsmap learned FSMap version 26 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9bc8e8a70 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf0ffff640 1 -- 172.21.15.64:0/1570458467 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf2401ed10 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf0ffff640 1 -- 172.21.15.64:0/1570458467 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7faf24043480 con 0x55d9bc868250 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 10 client.25220 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 20 client.25220 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc8e93e0 0x55d9bc8eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 4 client.25220 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:09:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9bc7d3dc0 con 0x55d9bc8e93e0 2024-05-09T09:09:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.944+0000 7faf2ffff640 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc8e93e0 0x55d9bc8eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.945+0000 7faf2ffff640 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc8e93e0 0x55d9bc8eb800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e7a42331f76d30 server_cookie=93d9f2eec452e94a in_seq=0 out_seq=0 2024-05-09T09:09:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:25.945+0000 7faf0ffff640 10 client.25220 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf0ffff640 1 -- 172.21.15.64:0/1570458467 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9bc7d3dc0 con 0x55d9bc8e93e0 2024-05-09T09:09:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf0ffff640 10 client.25220 handle_command_reply: tid=0 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 client.25220 shutdown 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 msgr2=0x55d9bc87b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc87b660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf24005eb0 tx=0x7faf2401e7d0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 shutdown_connections 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9bc8e93e0 0x55d9bc8eb800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bc86ddb0 0x55d9bc87cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf0ffff640 0 client.25220 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9bc868c40 0x55d9bc87bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 --2- 172.21.15.64:0/1570458467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9bc868250 0x55d9bc87b660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 >> 172.21.15.64:0/1570458467 conn(0x55d9bc8638c0 msgr2=0x55d9bc863ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 shutdown_connections 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 1 -- 172.21.15.64:0/1570458467 wait complete. 2024-05-09T09:09:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.279+0000 7faf39d63740 20 client.25220 trim_cache size 0 max 16384 2024-05-09T09:09:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:09:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:09:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:09:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf200a2c30 msgr2=0x7faf200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf200a2c30 0x7faf200a50f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faf240095f0 tx=0x7faf24005f90 comp rx=0 tx=0).stop 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 msgr2=0x7faf30197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 0x7faf30197ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00ea60 tx=0x7faf1c00ef20 comp rx=0 tx=0).stop 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 shutdown_connections 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf200a2c30 0x7faf200a50f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30102340 0x7faf3019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30101980 0x7faf30197ef0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 --2- 172.21.15.64:0/883696053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30100790 0x7faf301979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 >> 172.21.15.64:0/883696053 conn(0x7faf300fbd00 msgr2=0x7faf300fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 shutdown_connections 2024-05-09T09:09:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:26.280+0000 7faf36f77640 1 -- 172.21.15.64:0/883696053 wait complete. 2024-05-09T09:09:26.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:26 smithi179 ceph-mon[34086]: pgmap v379: 129 pgs: 129 active+clean; 38 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 3.9 MiB/s wr, 94 op/s 2024-05-09T09:09:26.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:26 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:26 smithi139 ceph-mon[33599]: pgmap v379: 129 pgs: 129 active+clean; 38 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 3.9 MiB/s wr, 94 op/s 2024-05-09T09:09:26.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:26 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:26.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:26 smithi064 ceph-mon[30149]: pgmap v379: 129 pgs: 129 active+clean; 38 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 3.9 MiB/s wr, 94 op/s 2024-05-09T09:09:26.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:26 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:28.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:28 smithi179 ceph-mon[34086]: pgmap v380: 129 pgs: 129 active+clean; 40 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.2 MiB/s wr, 98 op/s 2024-05-09T09:09:28.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:28 smithi139 ceph-mon[33599]: pgmap v380: 129 pgs: 129 active+clean; 40 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.2 MiB/s wr, 98 op/s 2024-05-09T09:09:28.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:28 smithi064 ceph-mon[30149]: pgmap v380: 129 pgs: 129 active+clean; 40 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.2 MiB/s wr, 98 op/s 2024-05-09T09:09:29.303 INFO:teuthology.orchestra.run.smithi064.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-09T09:09:29.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/: 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:total 2 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 . 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 data 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data: 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:total 45 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 . 2024-05-09T09:09:29.619 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 9 09:09 base 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 9 09:09 global 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 log 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_commit_ts 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_dynshmem 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 9 09:09 pg_hba.conf 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 9 09:09 pg_ident.conf 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:09 pg_logical 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 pg_multixact 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_notify 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_replslot 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_serial 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_snapshots 2024-05-09T09:09:29.620 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_stat 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_stat_tmp 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 pg_subtrans 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_tblspc 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_twophase 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:09 pg_wal 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 pg_xact 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 9 09:09 postgresql.auto.conf 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 9 09:09 postgresql.conf 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base: 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:total 3 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 9 09:09 . 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:29.621 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 1 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 13493 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 13494 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:total 7778 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 . 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 9 09:09 .. 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 112 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 113 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 1247 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1247_fsm 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1247_vm 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 09:09 1249 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1249_fsm 2024-05-09T09:09:29.622 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1249_vm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 09:09 1255 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1255_fsm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1255_vm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 09:09 1259 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1259_fsm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1259_vm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 09:09 13344 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13344_fsm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13344_vm 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13346 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13348 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349 2024-05-09T09:09:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13349_fsm 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349_vm 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13351 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13353 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13354_fsm 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354_vm 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13356 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13358 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13359_fsm 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359_vm 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13361 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13363 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1417 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1418 2024-05-09T09:09:29.624 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 174 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 175 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2187 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2224 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2328 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2336 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2337 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2579 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2600 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2600_fsm 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2600_vm 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2601_fsm 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601_vm 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2602 2024-05-09T09:09:29.625 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2602_fsm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2602_vm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2603 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2603_fsm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2603_vm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2604 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2605 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2605_fsm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2605_vm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2606_fsm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606_vm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2607 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2607_fsm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2607_vm 2024-05-09T09:09:29.626 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 09:09 2608 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2608_fsm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2608_vm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 09:09 2609 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2609_fsm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2609_vm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2610 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2610_fsm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2610_vm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2611 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2612_fsm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612_vm 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2613 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615 2024-05-09T09:09:29.627 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2615_fsm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615_vm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2616 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2616_fsm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2616_vm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 09:09 2617 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2617_fsm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2617_vm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 09:09 2618 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2618_fsm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2618_vm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 09:09 2619 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2619_fsm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2619_vm 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2620 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2650 2024-05-09T09:09:29.628 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2651 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2652 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2653 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2654 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2655 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2656 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2657 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 09:09 2658 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 2659 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2660 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2661 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2662 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2663 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2664 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2665 2024-05-09T09:09:29.629 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2666 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2667 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2668 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2669 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2670 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 09:09 2673 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 09:09 2674 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 2675 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2678 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2679 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2680 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2681 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2682 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2683 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2684 2024-05-09T09:09:29.630 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2685 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2686 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2687 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2688 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2689 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:09 2690 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 09:09 2691 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2692 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2693 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2696 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2699 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2701 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2702 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2703 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2704 2024-05-09T09:09:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2753 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2753_fsm 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2753_vm 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2754 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2755 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2756 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2757 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2830 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2831 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2832 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2833 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2834 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2835 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2836 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2837 2024-05-09T09:09:29.632 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 09:09 2838 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2838_fsm 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2838_vm 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2839 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840_fsm 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2840_vm 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2841 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2995 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2996 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3079_fsm 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079_vm 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3080 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3081 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3085 2024-05-09T09:09:29.633 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3118 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3119 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 09:09 3164 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3256 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3257 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3258 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3350 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3351 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3379 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3380 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3381 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3394 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3394_fsm 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3394_vm 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3395 2024-05-09T09:09:29.634 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3429 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3430 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3431 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3433 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3439 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3440 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3455 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 09:09 3456 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3456_fsm 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3456_vm 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3466 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3467 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3468 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3501 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3502 2024-05-09T09:09:29.635 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3503 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3534 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3541_fsm 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541_vm 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3542 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3574 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3575 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3576 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3596 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3597 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3598 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3599 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3600_fsm 2024-05-09T09:09:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600_vm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3601_fsm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601_vm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3602_fsm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602_vm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603_fsm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3603_vm 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3604 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3605 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3606 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3607 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3608 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 3609 2024-05-09T09:09:29.637 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3712 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3764_fsm 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764_vm 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3766 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3767 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3997 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4143 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4144 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4145 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4146 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4147 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4148 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4149 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4150 2024-05-09T09:09:29.638 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4151 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4152 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4153 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4154 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4155 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4156 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4157 2024-05-09T09:09:29.639 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4158 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4159 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4160 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4161 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4162 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4163 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4164 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4165 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4166 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4167 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4168 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4169 2024-05-09T09:09:29.866 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4170 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4171 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4172 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4173 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4174 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 5002 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 548 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 549 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6102 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6104 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6106 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6110 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6111 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6112 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6113 2024-05-09T09:09:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6117 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 826 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 827 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 828 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:total 7778 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 . 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 9 09:09 .. 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 112 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 113 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 1247 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1247_fsm 2024-05-09T09:09:29.868 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1247_vm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 09:09 1249 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1249_fsm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1249_vm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 09:09 1255 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1255_fsm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1255_vm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 09:09 1259 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1259_fsm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1259_vm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 09:09 13344 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13344_fsm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13344_vm 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13346 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13348 2024-05-09T09:09:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13349_fsm 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349_vm 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13351 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13353 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13354_fsm 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354_vm 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13356 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13358 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13359_fsm 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359_vm 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13361 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13363 2024-05-09T09:09:29.870 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1417 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1418 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 174 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 175 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2187 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2224 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2328 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2336 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2337 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2579 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2600 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2600_fsm 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2600_vm 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2601_fsm 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601_vm 2024-05-09T09:09:29.871 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2602 2024-05-09T09:09:29.872 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2602_fsm 2024-05-09T09:09:29.872 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2602_vm 2024-05-09T09:09:29.872 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2603 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2603_fsm 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2603_vm 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2604 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2605 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2605_fsm 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2605_vm 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606 2024-05-09T09:09:29.873 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2606_fsm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606_vm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2607 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2607_fsm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2607_vm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 09:09 2608 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2608_fsm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2608_vm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 09:09 2609 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2609_fsm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2609_vm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2610 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2610_fsm 2024-05-09T09:09:29.874 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2610_vm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2611 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2612_fsm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612_vm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2613 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2615_fsm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615_vm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2616 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2616_fsm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2616_vm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 09:09 2617 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2617_fsm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2617_vm 2024-05-09T09:09:29.875 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 09:09 2618 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2618_fsm 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2618_vm 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 09:09 2619 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2619_fsm 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2619_vm 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2620 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2650 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2651 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2652 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2653 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2654 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2655 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2656 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2657 2024-05-09T09:09:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 09:09 2658 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 2659 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2660 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2661 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2662 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2663 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2664 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2665 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2666 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2667 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2668 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2669 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2670 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 09:09 2673 2024-05-09T09:09:29.877 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 09:09 2674 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 2675 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2678 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2679 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2680 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2681 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2682 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2683 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2684 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2685 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2686 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2687 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2688 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2689 2024-05-09T09:09:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:09 2690 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 09:09 2691 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2692 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2693 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2696 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2699 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2701 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2702 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2703 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2704 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2753 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2753_fsm 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2753_vm 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2754 2024-05-09T09:09:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2755 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2756 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2757 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2830 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2831 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2832 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2833 2024-05-09T09:09:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2834 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2835 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2836 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2837 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 09:09 2838 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2838_fsm 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2838_vm 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2839 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840_fsm 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2840_vm 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2841 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2995 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2996 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3079_fsm 2024-05-09T09:09:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079_vm 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3080 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3081 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3085 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3118 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3119 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 09:09 3164 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3256 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3257 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3258 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3350 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3351 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3379 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3380 2024-05-09T09:09:29.882 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3381 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3394 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3394_fsm 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3394_vm 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3395 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3429 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3430 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3431 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3433 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3439 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3440 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3455 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 09:09 3456 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3456_fsm 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3456_vm 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3466 2024-05-09T09:09:29.883 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3467 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3468 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3501 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3502 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3503 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3534 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3541_fsm 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541_vm 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3542 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3574 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3575 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3576 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3596 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3597 2024-05-09T09:09:29.884 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3598 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3599 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3600_fsm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600_vm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3601_fsm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601_vm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3602_fsm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602_vm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603_fsm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3603_vm 2024-05-09T09:09:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3604 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3605 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3606 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3607 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3608 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 3609 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3712 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3764_fsm 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764_vm 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3766 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3767 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3997 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4143 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4144 2024-05-09T09:09:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4145 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4146 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4147 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4148 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4149 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4150 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4151 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4152 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4153 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4154 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4155 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4156 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4157 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4158 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4159 2024-05-09T09:09:29.887 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4160 2024-05-09T09:09:29.888 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4161 2024-05-09T09:09:29.888 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4162 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4163 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4164 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4165 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4166 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4167 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4168 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4169 2024-05-09T09:09:30.111 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4170 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4171 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4172 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4173 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4174 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 5002 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 548 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 549 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6102 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6104 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6106 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6110 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6111 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6112 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6113 2024-05-09T09:09:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6117 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 826 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 827 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 828 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:total 7778 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 . 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 9 09:09 .. 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 112 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 113 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 1247 2024-05-09T09:09:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1247_fsm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1247_vm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 09:09 1249 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1249_fsm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1249_vm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 09:09 1255 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1255_fsm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1255_vm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 09:09 1259 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1259_fsm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1259_vm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 09:09 13344 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13344_fsm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13344_vm 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13346 2024-05-09T09:09:30.114 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13348 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13349_fsm 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349_vm 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13351 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13353 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13354_fsm 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354_vm 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13356 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13358 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13359_fsm 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359_vm 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13361 2024-05-09T09:09:30.115 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13363 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1417 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1418 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 174 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 175 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2187 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2224 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2328 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2336 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2337 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2579 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2600 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2600_fsm 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2600_vm 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601 2024-05-09T09:09:30.116 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2601_fsm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601_vm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2602 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2602_fsm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2602_vm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2603 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2603_fsm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2603_vm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2604 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2605 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2605_fsm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2605_vm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2606_fsm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606_vm 2024-05-09T09:09:30.117 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2607 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2607_fsm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2607_vm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 09:09 2608 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2608_fsm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2608_vm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 09:09 2609 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2609_fsm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2609_vm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2610 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2610_fsm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2610_vm 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2611 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612 2024-05-09T09:09:30.118 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2612_fsm 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612_vm 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2613 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2615_fsm 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615_vm 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2616 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2616_fsm 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2616_vm 2024-05-09T09:09:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 09:09 2617 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2617_fsm 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2617_vm 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 09:09 2618 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2618_fsm 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2618_vm 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 09:09 2619 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2619_fsm 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2619_vm 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2620 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2650 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2651 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2652 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2653 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2654 2024-05-09T09:09:30.120 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2655 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2656 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2657 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 09:09 2658 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 2659 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2660 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2661 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2662 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2663 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2664 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2665 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2666 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2667 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2668 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2669 2024-05-09T09:09:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2670 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 09:09 2673 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 09:09 2674 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 2675 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2678 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2679 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2680 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2681 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2682 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2683 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2684 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2685 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2686 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2687 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2688 2024-05-09T09:09:30.122 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2689 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:09 2690 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 09:09 2691 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2692 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2693 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2696 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2699 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2701 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2702 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2703 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2704 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2753 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2753_fsm 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2753_vm 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2754 2024-05-09T09:09:30.123 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2755 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2756 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2757 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2830 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2831 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2832 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2833 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2834 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2835 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2836 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2837 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 09:09 2838 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2838_fsm 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2838_vm 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2839 2024-05-09T09:09:30.124 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840_fsm 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2840_vm 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2841 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2995 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2996 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3079_fsm 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079_vm 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3080 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3081 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3085 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3118 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3119 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 09:09 3164 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3256 2024-05-09T09:09:30.125 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3257 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3258 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3350 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3351 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3379 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3380 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3381 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3394 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3394_fsm 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3394_vm 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3395 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3429 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3430 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3431 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3433 2024-05-09T09:09:30.126 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3439 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3440 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3455 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 09:09 3456 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3456_fsm 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3456_vm 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3466 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3467 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3468 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3501 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3502 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3503 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3534 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3541_fsm 2024-05-09T09:09:30.127 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541_vm 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3542 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3574 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3575 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3576 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3596 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3597 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3598 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3599 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3600_fsm 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600_vm 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3601_fsm 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601_vm 2024-05-09T09:09:30.128 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3602_fsm 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602_vm 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603_fsm 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3603_vm 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3604 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3605 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3606 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3607 2024-05-09T09:09:30.129 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3608 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 3609 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3712 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3764_fsm 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764_vm 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3766 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3767 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3997 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4143 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4144 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4145 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4146 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4147 2024-05-09T09:09:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4148 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4149 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4150 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4151 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4152 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4153 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4154 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4155 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4156 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4157 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4158 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4159 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4160 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4161 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4162 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4163 2024-05-09T09:09:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4164 2024-05-09T09:09:30.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4165 2024-05-09T09:09:30.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4166 2024-05-09T09:09:30.132 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4167 2024-05-09T09:09:30.149 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4168 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4169 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4170 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4171 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4172 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4173 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4174 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 5002 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 548 2024-05-09T09:09:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 549 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6102 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6104 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6106 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6110 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6111 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6112 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6113 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6117 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 826 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 827 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 828 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.151 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/global: 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:total 538 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 9 09:09 . 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1213 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1213_fsm 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1213_vm 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1214 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1214_fsm 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1214_vm 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 1232 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 1233 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1260 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1260_fsm 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1260_vm 2024-05-09T09:09:30.152 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1261 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1261_fsm 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1261_vm 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1262 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1262_fsm 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1262_vm 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2396 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2396_fsm 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2396_vm 2024-05-09T09:09:30.153 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2397 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2671 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2672 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2676 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2677 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2694 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2695 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2697 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2698 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2846 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2847 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2964 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2965 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2966 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2967 2024-05-09T09:09:30.154 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3592 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3593 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4060 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4061 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4175 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4176 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4177 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4178 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4181 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4182 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4183 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4184 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4185 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4186 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6000 2024-05-09T09:09:30.155 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6001 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6002 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6100 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6114 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6115 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 pg_control 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/log: 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.156 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:total 3 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:09 . 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 mappings 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:09 replorigin_checkpoint 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 snapshots 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:09 .. 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:09 .. 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:total 2 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 . 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 members 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 offsets 2024-05-09T09:09:30.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.212 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-09T09:09:30.212 INFO:teuthology.orchestra.run.smithi064.stdout:total 9 2024-05-09T09:09:30.212 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 . 2024-05-09T09:09:30.212 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 .. 2024-05-09T09:09:30.212 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 0000 2024-05-09T09:09:30.212 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:total 9 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 . 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 .. 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 0000 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.213 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.215 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-09T09:09:30.224 INFO:teuthology.orchestra.run.smithi064.stdout:total 9 2024-05-09T09:09:30.224 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 . 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 0000 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-09T09:09:30.225 INFO:teuthology.orchestra.run.smithi064.stdout:total 16386 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:09 . 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:09 000000010000000000000001 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 archive_status 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:09 .. 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:total 9 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 . 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 9 09:09 .. 2024-05-09T09:09:30.226 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 0000 2024-05-09T09:09:30.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-09T09:09:30.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:30 smithi064 ceph-mon[30149]: pgmap v381: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.7 MiB/s wr, 311 op/s 2024-05-09T09:09:30.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-09T09:09:30.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:30 smithi179 ceph-mon[34086]: pgmap v381: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.7 MiB/s wr, 311 op/s 2024-05-09T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:30 smithi139 ceph-mon[33599]: pgmap v381: 129 pgs: 129 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.7 MiB/s wr, 311 op/s 2024-05-09T09:09:31.240 INFO:teuthology.orchestra.run.smithi064.stderr:dropping old tables... 2024-05-09T09:09:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-09T09:09:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-09T09:09:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-09T09:09:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-09T09:09:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:creating tables... 2024-05-09T09:09:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:generating data (client-side)... 2024-05-09T09:09:31.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:31 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:31.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:31 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:31.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:31 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:09:31.808 INFO:teuthology.orchestra.run.smithi064.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.48 s, remaining 14.81 s) 2024-05-09T09:09:32.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:32 smithi179 ceph-mon[34086]: pgmap v382: 129 pgs: 129 active+clean; 47 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-05-09T09:09:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:32 smithi139 ceph-mon[33599]: pgmap v382: 129 pgs: 129 active+clean; 47 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-05-09T09:09:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:32 smithi064 ceph-mon[30149]: pgmap v382: 129 pgs: 129 active+clean; 47 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-05-09T09:09:33.000 INFO:teuthology.orchestra.run.smithi064.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.67 s, remaining 25.05 s) 2024-05-09T09:09:34.224 INFO:teuthology.orchestra.run.smithi064.stderr:300000 of 3200000 tuples (9%) done (elapsed 2.89 s, remaining 27.97 s) 2024-05-09T09:09:34.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:34 smithi179 ceph-mon[34086]: pgmap v383: 129 pgs: 129 active+clean; 61 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 4.2 MiB/s wr, 379 op/s 2024-05-09T09:09:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:34 smithi139 ceph-mon[33599]: pgmap v383: 129 pgs: 129 active+clean; 61 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 4.2 MiB/s wr, 379 op/s 2024-05-09T09:09:34.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:34 smithi064 ceph-mon[30149]: pgmap v383: 129 pgs: 129 active+clean; 61 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 4.2 MiB/s wr, 379 op/s 2024-05-09T09:09:34.983 INFO:teuthology.orchestra.run.smithi064.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.65 s, remaining 25.56 s) 2024-05-09T09:09:36.193 INFO:teuthology.orchestra.run.smithi064.stderr:500000 of 3200000 tuples (15%) done (elapsed 4.86 s, remaining 26.26 s) 2024-05-09T09:09:36.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:36 smithi179 ceph-mon[34086]: pgmap v384: 129 pgs: 129 active+clean; 81 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 6.5 MiB/s wr, 359 op/s 2024-05-09T09:09:36.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:36 smithi139 ceph-mon[33599]: pgmap v384: 129 pgs: 129 active+clean; 81 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 6.5 MiB/s wr, 359 op/s 2024-05-09T09:09:36.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:36 smithi064 ceph-mon[30149]: pgmap v384: 129 pgs: 129 active+clean; 81 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 6.5 MiB/s wr, 359 op/s 2024-05-09T09:09:37.041 INFO:teuthology.orchestra.run.smithi064.stderr:600000 of 3200000 tuples (18%) done (elapsed 5.71 s, remaining 24.75 s) 2024-05-09T09:09:38.254 INFO:teuthology.orchestra.run.smithi064.stderr:700000 of 3200000 tuples (21%) done (elapsed 6.92 s, remaining 24.73 s) 2024-05-09T09:09:38.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:38 smithi179 ceph-mon[34086]: pgmap v385: 129 pgs: 129 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 8.2 MiB/s wr, 341 op/s 2024-05-09T09:09:38.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:38 smithi139 ceph-mon[33599]: pgmap v385: 129 pgs: 129 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 8.2 MiB/s wr, 341 op/s 2024-05-09T09:09:38.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:38 smithi064 ceph-mon[30149]: pgmap v385: 129 pgs: 129 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 8.2 MiB/s wr, 341 op/s 2024-05-09T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 -- 172.21.15.64:0/2750182185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c1018c0 msgr2=0x7fad1c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 --2- 172.21.15.64:0/2750182185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c1018c0 0x7fad1c101ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009900 tx=0x7fad0c02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 -- 172.21.15.64:0/2750182185 shutdown_connections 2024-05-09T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 --2- 172.21.15.64:0/2750182185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad1c103470 0x7fad1c109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 --2- 172.21.15.64:0/2750182185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c102ab0 0x7fad1c102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 --2- 172.21.15.64:0/2750182185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c1018c0 0x7fad1c101ce0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 -- 172.21.15.64:0/2750182185 >> 172.21.15.64:0/2750182185 conn(0x7fad1c0fce50 msgr2=0x7fad1c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 -- 172.21.15.64:0/2750182185 shutdown_connections 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 -- 172.21.15.64:0/2750182185 wait complete. 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.379+0000 7fad22314640 1 Processor -- start 2024-05-09T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 -- start start 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1018c0 0x7fad1c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 0x7fad1c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad1c103470 0x7fad1c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c10d3c0 con 0x7fad1c102ab0 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c10d240 con 0x7fad1c103470 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad22314640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c10d540 con 0x7fad1c1018c0 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 0x7fad1c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 0x7fad1c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55190/0 (socket says 172.21.15.64:55190) 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1621223287 learned_addr learned my addr 172.21.15.64:0/1621223287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad1bfff640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1018c0 0x7fad1c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.380+0000 7fad208b1640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad1c103470 0x7fad1c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1621223287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1018c0 msgr2=0x7fad1c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1018c0 0x7fad1c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1621223287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad1c103470 msgr2=0x7fad1c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad1c103470 0x7fad1c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1621223287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c19f650 con 0x7fad1c102ab0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 0x7fad1c197f70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad1000e9f0 tx=0x7fad1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad197fa640 1 -- 172.21.15.64:0/1621223287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1000cd80 con 0x7fad1c102ab0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c19f940 con 0x7fad1c102ab0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad197fa640 1 -- 172.21.15.64:0/1621223287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad100108e0 con 0x7fad1c102ab0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad197fa640 1 -- 172.21.15.64:0/1621223287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10018920 con 0x7fad1c102ab0 2024-05-09T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.381+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c19fee0 con 0x7fad1c102ab0 2024-05-09T09:09:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad197fa640 1 -- 172.21.15.64:0/1621223287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad10010a90 con 0x7fad1c102ab0 2024-05-09T09:09:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 Processor -- start 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 -- start start 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b8f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b94db0 0x55a794b97240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a794af3340 con 0x55a794b8fc40 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a794a9b290 con 0x55a794b8f250 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad25028740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a794aef300 con 0x55a794b94db0 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.384+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b94740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55198/0 (socket says 172.21.15.64:55198) 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 -- 172.21.15.64:0/3236422861 learned_addr learned my addr 172.21.15.64:0/3236422861 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b94db0 0x55a794b97240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad208b1640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b8f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 -- 172.21.15.64:0/3236422861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b94db0 msgr2=0x55a794b97240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b94db0 0x55a794b97240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 -- 172.21.15.64:0/3236422861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 msgr2=0x55a794b8f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b8f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 -- 172.21.15.64:0/3236422861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a794abf8e0 con 0x55a794b8fc40 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad208b1640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b8f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b94db0 0x55a794b97240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad197fa640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad040a2c30 0x7fad040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad1bfff640 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b94740 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009900 tx=0x7fad0c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b595a1f24a177 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad208b1640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad040a2c30 0x7fad040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad197fa640 1 -- 172.21.15.64:0/1621223287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad100cf6a0 con 0x7fad1c102ab0 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad027fc640 1 -- 172.21.15.64:0/3236422861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c035030 con 0x55a794b8fc40 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad027fc640 1 -- 172.21.15.64:0/3236422861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0c040070 con 0x55a794b8fc40 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.385+0000 7fad027fc640 1 -- 172.21.15.64:0/3236422861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0039f0 con 0x55a794b8fc40 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 -- 172.21.15.64:0/3236422861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 msgr2=0x55a794b94740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad208b1640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad040a2c30 0x7fad040a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55a794b904a0 tx=0x7fad14009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b94740 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009900 tx=0x7fad0c005c00 comp rx=0 tx=0).stop 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 -- 172.21.15.64:0/3236422861 shutdown_connections 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b94db0 0x55a794b97240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b94740 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 --2- 172.21.15.64:0/3236422861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b8f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 -- 172.21.15.64:0/3236422861 >> 172.21.15.64:0/3236422861 conn(0x55a794b8a8c0 msgr2=0x55a794b8acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 -- 172.21.15.64:0/3236422861 shutdown_connections 2024-05-09T09:09:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.386+0000 7fad25028740 1 -- 172.21.15.64:0/3236422861 wait complete. 2024-05-09T09:09:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 -- start start 2024-05-09T09:09:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 Processor -- start 2024-05-09T09:09:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b99970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794ba09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a794b97d00 con 0x55a794b8fc40 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a794aae790 con 0x55a794b94db0 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a794b97e80 con 0x55a794b8f250 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad25028740 10 client.0 _ll_register_callbacks cb 0x55a794ae9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b99970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b99970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55202/0 (socket says 172.21.15.64:55202) 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1599681088 learned_addr learned my addr 172.21.15.64:0/1599681088 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.387+0000 7fad1bfff640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad208b1640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794ba09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1599681088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 msgr2=0x55a794b99430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b99430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 msgr2=0x55a794ba09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794ba09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad1b7fe640 1 -- 172.21.15.64:0/1599681088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a794807210 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad208b1640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794ba09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b99970 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fad100a42b0 tx=0x7fad1009b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534c0de9ff1c1fae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7facfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7facfb7fe640 1 -- 172.21.15.64:0/1599681088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1009ff20 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7facfb7fe640 1 -- 172.21.15.64:0/1599681088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad100cfb50 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0c040070 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7facfb7fe640 1 -- 172.21.15.64:0/1599681088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad100a47e0 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad25028740 10 client.15654 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7948d3620 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.388+0000 7facfb7fe640 1 -- 172.21.15.64:0/1599681088 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0c040070 con 0x55a794b8fc40 2024-05-09T09:09:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7facfb7fe640 1 -- 172.21.15.64:0/1599681088 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fad10097970 con 0x55a794b8fc40 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad25028740 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad25028740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad25028740 20 client.15654 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad25028740 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794ba4fc0 0x55a794ba7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad25028740 4 client.15654 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a794b00a70 con 0x55a794ba4fc0 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad1bfff640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794ba4fc0 0x55a794ba7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7fad1bfff640 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794ba4fc0 0x55a794ba7410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e743e3c4973123a8 server_cookie=8e0afd01aea7442d in_seq=0 out_seq=0 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.389+0000 7facfb7fe640 10 client.15654 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7facfb7fe640 1 -- 172.21.15.64:0/1599681088 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a794b00a70 con 0x55a794ba4fc0 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7facfb7fe640 10 client.15654 handle_command_reply: tid=0 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 client.15654 shutdown 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 msgr2=0x55a794b99970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b99970 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fad100a42b0 tx=0x7fad1009b860 comp rx=0 tx=0).stop 2024-05-09T09:09:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 shutdown_connections 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794ba4fc0 0x55a794ba7410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794ba09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b99970 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7facfb7fe640 0 client.15654 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 --2- 172.21.15.64:0/1599681088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b99430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 >> 172.21.15.64:0/1599681088 conn(0x55a794b8a8c0 msgr2=0x55a794b89a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.390+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 shutdown_connections 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.391+0000 7fad25028740 1 -- 172.21.15.64:0/1599681088 wait complete. 2024-05-09T09:09:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.391+0000 7fad25028740 20 client.15654 trim_cache size 0 max 16384 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.411+0000 7fad25028740 1 Processor -- start 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.411+0000 7fad25028740 1 -- start start 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad25028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b90e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad25028740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8fc40 0x55a794ba0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad25028740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 0x55a794b91360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad25028740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1acc40 con 0x55a794b94db0 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad25028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1aa510 con 0x55a794b8f250 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad25028740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1a8360 con 0x55a794b8fc40 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 0x55a794b91360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 0x55a794b91360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55204/0 (socket says 172.21.15.64:55204) 2024-05-09T09:09:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 -- 172.21.15.64:0/2902298560 learned_addr learned my addr 172.21.15.64:0/2902298560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad1bfff640 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b90e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8fc40 0x55a794ba0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 -- 172.21.15.64:0/2902298560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8fc40 msgr2=0x55a794ba0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8fc40 0x55a794ba0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 -- 172.21.15.64:0/2902298560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 msgr2=0x55a794b90e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b90e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad208b1640 1 -- 172.21.15.64:0/2902298560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a794b90910 con 0x55a794b94db0 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.412+0000 7fad1bfff640 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b90e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.413+0000 7fad208b1640 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 0x55a794b91360 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fad14005cc0 tx=0x7fad14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491fde3071ea5d20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.413+0000 7fad01ffb640 1 -- 172.21.15.64:0/2902298560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14004d20 con 0x55a794b94db0 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.413+0000 7fad01ffb640 1 -- 172.21.15.64:0/2902298560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad1401c9f0 con 0x55a794b94db0 2024-05-09T09:09:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.413+0000 7fad01ffb640 1 -- 172.21.15.64:0/2902298560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14005650 con 0x55a794b94db0 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.413+0000 7fad25028740 1 -- 172.21.15.64:0/2902298560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 msgr2=0x55a794b91360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.413+0000 7fad25028740 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 0x55a794b91360 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fad14005cc0 tx=0x7fad14019000 comp rx=0 tx=0).stop 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- 172.21.15.64:0/2902298560 shutdown_connections 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b94db0 0x55a794b91360 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8fc40 0x55a794ba0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 --2- 172.21.15.64:0/2902298560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b8f250 0x55a794b90e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- 172.21.15.64:0/2902298560 >> 172.21.15.64:0/2902298560 conn(0x55a794b8a8c0 msgr2=0x55a794b8aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- 172.21.15.64:0/2902298560 shutdown_connections 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- 172.21.15.64:0/2902298560 wait complete. 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- start start 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 Processor -- start 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b9de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794b9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1a89d0 con 0x55a794b8fc40 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1a9e40 con 0x55a794b94db0 2024-05-09T09:09:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1a90a0 con 0x55a794b8f250 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad25028740 10 client.0 _ll_register_callbacks cb 0x7fad1009b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.414+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b9e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55208/0 (socket says 172.21.15.64:55208) 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 -- 172.21.15.64:0/479581272 learned_addr learned my addr 172.21.15.64:0/479581272 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1bfff640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b9de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad208b1640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794b9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 -- 172.21.15.64:0/479581272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 msgr2=0x55a794b9de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b9de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 -- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 msgr2=0x55a794b9f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794b9f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 -- 172.21.15.64:0/479581272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a794b9f9d0 con 0x55a794b8fc40 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1bfff640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b9de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad1b7fe640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b9e3d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fad1009b8d0 tx=0x7fad1009b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0f53416a095550 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7facfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7facfb7fe640 1 -- 172.21.15.64:0/479581272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10010de0 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7facfb7fe640 1 -- 172.21.15.64:0/479581272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fad100cf3e0 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad1401c9f0 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7facfb7fe640 1 -- 172.21.15.64:0/479581272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad100ba2b0 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad25028740 10 client.15660 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.415+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a794c0b590 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7facfb7fe640 1 -- 172.21.15.64:0/479581272 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad1401c9f0 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7facfb7fe640 1 -- 172.21.15.64:0/479581272 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fad10018ac0 con 0x55a794b8fc40 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad25028740 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad25028740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad25028740 20 client.15660 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad25028740 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794c0bb40 0x55a794c0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad25028740 4 client.15660 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:09:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a794afadc0 con 0x55a794c0bb40 2024-05-09T09:09:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.416+0000 7fad1bfff640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794c0bb40 0x55a794c0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.417+0000 7facfb7fe640 10 client.15660 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.417+0000 7fad1bfff640 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794c0bb40 0x55a794c0df60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150f8afb8a9d3674 server_cookie=b336e68165d5aaf4 in_seq=0 out_seq=0 2024-05-09T09:09:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:800000 of 3200000 tuples (25%) done (elapsed 8.11 s, remaining 24.33 s) 2024-05-09T09:09:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7facfb7fe640 1 -- 172.21.15.64:0/479581272 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a794afadc0 con 0x55a794c0bb40 2024-05-09T09:09:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7facfb7fe640 10 client.15660 handle_command_reply: tid=0 2024-05-09T09:09:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 client.15660 shutdown 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 msgr2=0x55a794b9e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b9e3d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fad1009b8d0 tx=0x7fad1009b7f0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 shutdown_connections 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a794c0bb40 0x55a794c0df60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a794b94db0 0x55a794b9f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a794b8fc40 0x55a794b9e3d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 --2- 172.21.15.64:0/479581272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a794b8f250 0x55a794b9de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7facfb7fe640 0 client.15660 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 >> 172.21.15.64:0/479581272 conn(0x55a794b8a8c0 msgr2=0x55a794b993d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 shutdown_connections 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 1 -- 172.21.15.64:0/479581272 wait complete. 2024-05-09T09:09:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.444+0000 7fad25028740 20 client.15660 trim_cache size 0 max 16384 2024-05-09T09:09:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:09:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:09:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:09:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad040a2c30 msgr2=0x7fad040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad040a2c30 0x7fad040a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55a794b904a0 tx=0x7fad14009600 comp rx=0 tx=0).stop 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 msgr2=0x7fad1c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 0x7fad1c197f70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad1000e9f0 tx=0x7fad1000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 shutdown_connections 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fad040a2c30 0x7fad040a50f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad1c103470 0x7fad1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c102ab0 0x7fad1c197f70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 --2- 172.21.15.64:0/1621223287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1018c0 0x7fad1c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 >> 172.21.15.64:0/1621223287 conn(0x7fad1c0fce50 msgr2=0x7fad1c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 shutdown_connections 2024-05-09T09:09:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:39.445+0000 7fad22314640 1 -- 172.21.15.64:0/1621223287 wait complete. 2024-05-09T09:09:40.323 INFO:teuthology.orchestra.run.smithi064.stderr:900000 of 3200000 tuples (28%) done (elapsed 8.99 s, remaining 22.98 s) 2024-05-09T09:09:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:40 smithi179 ceph-mon[34086]: pgmap v386: 129 pgs: 129 active+clean; 181 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 22 MiB/s wr, 350 op/s 2024-05-09T09:09:40.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:40 smithi139 ceph-mon[33599]: pgmap v386: 129 pgs: 129 active+clean; 181 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 22 MiB/s wr, 350 op/s 2024-05-09T09:09:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:40 smithi064 ceph-mon[30149]: pgmap v386: 129 pgs: 129 active+clean; 181 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 22 MiB/s wr, 350 op/s 2024-05-09T09:09:41.519 INFO:teuthology.orchestra.run.smithi064.stderr:1000000 of 3200000 tuples (31%) done (elapsed 10.19 s, remaining 22.41 s) 2024-05-09T09:09:42.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:42 smithi179 ceph-mon[34086]: pgmap v387: 129 pgs: 129 active+clean; 210 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 26 MiB/s wr, 146 op/s 2024-05-09T09:09:42.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:42 smithi139 ceph-mon[33599]: pgmap v387: 129 pgs: 129 active+clean; 210 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 26 MiB/s wr, 146 op/s 2024-05-09T09:09:42.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:42 smithi064 ceph-mon[30149]: pgmap v387: 129 pgs: 129 active+clean; 210 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 26 MiB/s wr, 146 op/s 2024-05-09T09:09:42.721 INFO:teuthology.orchestra.run.smithi064.stderr:1100000 of 3200000 tuples (34%) done (elapsed 11.39 s, remaining 21.74 s) 2024-05-09T09:09:43.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:09:43.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.283+0000 7f6c550ca640 1 -- 172.21.15.64:0/1457316929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c50102b10 msgr2=0x7f6c50102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.283+0000 7f6c550ca640 1 --2- 172.21.15.64:0/1457316929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c50102b10 0x7f6c50102f90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009930 tx=0x7f6c4402a420 comp rx=0 tx=0).stop 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 -- 172.21.15.64:0/1457316929 shutdown_connections 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 --2- 172.21.15.64:0/1457316929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c501034d0 0x7f6c50109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 --2- 172.21.15.64:0/1457316929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c50102b10 0x7f6c50102f90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 --2- 172.21.15.64:0/1457316929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c500ffb40 0x7f6c500fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 -- 172.21.15.64:0/1457316929 >> 172.21.15.64:0/1457316929 conn(0x7f6c500fb470 msgr2=0x7f6c500fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 -- 172.21.15.64:0/1457316929 shutdown_connections 2024-05-09T09:09:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 -- 172.21.15.64:0/1457316929 wait complete. 2024-05-09T09:09:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.284+0000 7f6c550ca640 1 Processor -- start 2024-05-09T09:09:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 -- start start 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 0x7f6c50199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c50102b10 0x7f6c5019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c501034d0 0x7f6c501a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5010d3d0 con 0x7f6c500ffb40 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5010d250 con 0x7f6c501034d0 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c550ca640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5010d550 con 0x7f6c50102b10 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 0x7f6c50199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 0x7f6c50199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55218/0 (socket says 172.21.15.64:55218) 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2359755654 learned_addr learned my addr 172.21.15.64:0/2359755654 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c50102b10 0x7f6c5019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c501034d0 0x7f6c501a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2359755654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c50102b10 msgr2=0x7f6c5019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c50102b10 0x7f6c5019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2359755654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c501034d0 msgr2=0x7f6c501a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c501034d0 0x7f6c501a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2359755654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c501a1970 con 0x7f6c500ffb40 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.285+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c50102b10 0x7f6c5019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.286+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 0x7f6c50199d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000e9c0 tx=0x7f6c4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.286+0000 7f6c37fff640 1 -- 172.21.15.64:0/2359755654 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000cd80 con 0x7f6c500ffb40 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.286+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c501a1c60 con 0x7f6c500ffb40 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.286+0000 7f6c37fff640 1 -- 172.21.15.64:0/2359755654 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c400035f0 con 0x7f6c500ffb40 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.286+0000 7f6c37fff640 1 -- 172.21.15.64:0/2359755654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40010690 con 0x7f6c500ffb40 2024-05-09T09:09:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.286+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501a2180 con 0x7f6c500ffb40 2024-05-09T09:09:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.288+0000 7f6c37fff640 1 -- 172.21.15.64:0/2359755654 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c400037a0 con 0x7f6c500ffb40 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 Processor -- start 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 -- start start 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca385350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca38cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c2ca2e9d00 con 0x55c2ca38aa90 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c2ca2914c0 con 0x55c2ca385920 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c2ca2e5cc0 con 0x55c2ca384f30 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca38cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca38cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55230/0 (socket says 172.21.15.64:55230) 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4e59c640 1 -- 172.21.15.64:0/3287695698 learned_addr learned my addr 172.21.15.64:0/3287695698 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c37fff640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c380a2b90 0x7f6c380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca385350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c380a2b90 0x7f6c380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c37fff640 1 -- 172.21.15.64:0/2359755654 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c400cfa40 con 0x7f6c500ffb40 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4e59c640 1 -- 172.21.15.64:0/3287695698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 msgr2=0x55c2ca385350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca385350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.289+0000 7f6c4e59c640 1 -- 172.21.15.64:0/3287695698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 msgr2=0x55c2ca38a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c4e59c640 1 -- 172.21.15.64:0/3287695698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ca0c77d0 con 0x55c2ca38aa90 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca38cf20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6c440314e0 tx=0x7f6c44031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337cb5b9b30b6de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c380a2b90 0x7f6c380a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005c40 tx=0x7f6c48005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c357fa640 1 -- 172.21.15.64:0/3287695698 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44035030 con 0x55c2ca38aa90 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c357fa640 1 -- 172.21.15.64:0/3287695698 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c44031cb0 con 0x55c2ca38aa90 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.290+0000 7f6c357fa640 1 -- 172.21.15.64:0/3287695698 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c440046c0 con 0x55c2ca38aa90 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- 172.21.15.64:0/3287695698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 msgr2=0x55c2ca38cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca38cf20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6c440314e0 tx=0x7f6c44031630 comp rx=0 tx=0).stop 2024-05-09T09:09:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- 172.21.15.64:0/3287695698 shutdown_connections 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca38cf20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/3287695698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca385350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- 172.21.15.64:0/3287695698 >> 172.21.15.64:0/3287695698 conn(0x55c2ca3805a0 msgr2=0x55c2ca3809a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- 172.21.15.64:0/3287695698 shutdown_connections 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- 172.21.15.64:0/3287695698 wait complete. 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- start start 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 Processor -- start 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca38f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca396840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.291+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c2ca38d9e0 con 0x55c2ca38aa90 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c2ca2a4b90 con 0x55c2ca385920 2024-05-09T09:09:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c2ca38db60 con 0x55c2ca384f30 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c57dd7740 10 client.0 _ll_register_callbacks cb 0x55c2ca2e0da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca396840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca38f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca396840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55242/0 (socket says 172.21.15.64:55242) 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca38f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39058/0 (socket says 172.21.15.64:39058) 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4f59e640 1 -- 172.21.15.64:0/2464583298 learned_addr learned my addr 172.21.15.64:0/2464583298 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 msgr2=0x55c2ca38f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2464583298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 msgr2=0x55c2ca396840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca396840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/2464583298 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c9ffd3f0 con 0x55c2ca384f30 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca38f2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cd200 tx=0x7f6c4009b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdadca4242bf4e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:09:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c32ff5640 1 -- 172.21.15.64:0/2464583298 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400bad50 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c32ff5640 1 -- 172.21.15.64:0/2464583298 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c4009b360 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.292+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c44031cb0 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c32ff5640 1 -- 172.21.15.64:0/2464583298 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c4009bf10 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 10 client.25348 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2c9fbd0f0 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c32ff5640 1 -- 172.21.15.64:0/2464583298 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c44031cb0 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c32ff5640 1 -- 172.21.15.64:0/2464583298 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c4009add0 con 0x55c2ca384f30 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 10 client.25348 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 10 client.25348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 20 client.25348 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca39b160 0x55c2ca39d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 4 client.25348 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2ca2f7430 con 0x55c2ca39b160 2024-05-09T09:09:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.293+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca39b160 0x55c2ca39d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.294+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca39b160 0x55c2ca39d620 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add741ace6d74284 server_cookie=2d43f08c59bc1fd8 in_seq=0 out_seq=0 2024-05-09T09:09:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.294+0000 7f6c32ff5640 10 client.25348 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c32ff5640 1 -- 172.21.15.64:0/2464583298 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2ca2f7430 con 0x55c2ca39b160 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c32ff5640 10 client.25348 handle_command_reply: tid=0 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 client.25348 shutdown 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 msgr2=0x55c2ca38f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca38f2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cd200 tx=0x7f6c4009b870 comp rx=0 tx=0).stop 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 shutdown_connections 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca39b160 0x55c2ca39d620 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca38aa90 0x55c2ca396840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca385920 0x55c2ca38f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c32ff5640 0 client.25348 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/2464583298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca38f2c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 >> 172.21.15.64:0/2464583298 conn(0x55c2ca3805a0 msgr2=0x55c2ca37f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 shutdown_connections 2024-05-09T09:09:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 1 -- 172.21.15.64:0/2464583298 wait complete. 2024-05-09T09:09:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.295+0000 7f6c57dd7740 20 client.25348 trim_cache size 0 max 16384 2024-05-09T09:09:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 Processor -- start 2024-05-09T09:09:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 -- start start 2024-05-09T09:09:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 0x55c2ca38f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca385920 0x55c2ca396800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca37b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501aee90 con 0x55c2ca384f30 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ac760 con 0x55c2ca38aa90 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.315+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501aa5b0 con 0x55c2ca385920 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 0x55c2ca38f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 0x55c2ca38f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55244/0 (socket says 172.21.15.64:55244) 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/823158583 learned_addr learned my addr 172.21.15.64:0/823158583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca385920 0x55c2ca396800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca37b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 -- 172.21.15.64:0/823158583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 msgr2=0x55c2ca37b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca37b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 -- 172.21.15.64:0/823158583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 msgr2=0x55c2ca38f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 0x55c2ca38f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 -- 172.21.15.64:0/823158583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ca3803d0 con 0x55c2ca385920 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 0x55c2ca38f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca385920 0x55c2ca396800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c4402a940 tx=0x7f6c44031da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a666f308c8c529 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c34ff9640 1 -- 172.21.15.64:0/823158583 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44035040 con 0x55c2ca385920 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c34ff9640 1 -- 172.21.15.64:0/823158583 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c440054f0 con 0x55c2ca385920 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.316+0000 7f6c34ff9640 1 -- 172.21.15.64:0/823158583 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44004990 con 0x55c2ca385920 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 -- 172.21.15.64:0/823158583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca385920 msgr2=0x55c2ca396800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca385920 0x55c2ca396800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c4402a940 tx=0x7f6c44031da0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 -- 172.21.15.64:0/823158583 shutdown_connections 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca37b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca385920 0x55c2ca396800 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/823158583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca384f30 0x55c2ca38f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 -- 172.21.15.64:0/823158583 >> 172.21.15.64:0/823158583 conn(0x55c2ca3805a0 msgr2=0x55c2ca37ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 -- 172.21.15.64:0/823158583 shutdown_connections 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 -- 172.21.15.64:0/823158583 wait complete. 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.317+0000 7f6c57dd7740 1 -- start start 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 Processor -- start 2024-05-09T09:09:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca394010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 0x55c2ca394550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca395510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501aac20 con 0x55c2ca385920 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ac090 con 0x55c2ca38aa90 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ab2f0 con 0x55c2ca384f30 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c57dd7740 10 client.0 _ll_register_callbacks cb 0x7f6c4009bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 0x55c2ca394550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 0x55c2ca394550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55248/0 (socket says 172.21.15.64:55248) 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca394010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4e59c640 1 -- 172.21.15.64:0/1079009208 learned_addr learned my addr 172.21.15.64:0/1079009208 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca395510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 msgr2=0x55c2ca395510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca395510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/1079009208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 msgr2=0x55c2ca394550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 0x55c2ca394550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 -- 172.21.15.64:0/1079009208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ca395b00 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 0x55c2ca394550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4f59e640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca395510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c4ed9d640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca394010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cd520 tx=0x7f6c4009b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6383cb2b391160 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.318+0000 7f6c32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c32ff5640 1 -- 172.21.15.64:0/1079009208 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400981b0 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c32ff5640 1 -- 172.21.15.64:0/1079009208 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c400bab50 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c440054f0 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c32ff5640 1 -- 172.21.15.64:0/1079009208 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c400a02c0 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 10 client.25354 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2ca401850 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c32ff5640 1 -- 172.21.15.64:0/1079009208 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c440054f0 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c32ff5640 1 -- 172.21.15.64:0/1079009208 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c400ba020 con 0x55c2ca384f30 2024-05-09T09:09:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 10 client.25354 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 20 client.25354 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca401e50 0x55c2ca404290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 4 client.25354 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2ca2f1780 con 0x55c2ca401e50 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.319+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca401e50 0x55c2ca404290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c4e59c640 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca401e50 0x55c2ca404290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e8be5e5eb168e10 server_cookie=46a87b0b23ab8141 in_seq=0 out_seq=0 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c32ff5640 10 client.25354 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c32ff5640 1 -- 172.21.15.64:0/1079009208 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c2ca2f1780 con 0x55c2ca401e50 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c32ff5640 10 client.25354 handle_command_reply: tid=0 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 client.25354 shutdown 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 msgr2=0x55c2ca394010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca394010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cd520 tx=0x7f6c4009b9a0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 shutdown_connections 2024-05-09T09:09:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c2ca401e50 0x55c2ca404290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c2ca38aa90 0x55c2ca395510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c32ff5640 0 client.25354 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2ca385920 0x55c2ca394550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 --2- 172.21.15.64:0/1079009208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2ca384f30 0x55c2ca394010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 >> 172.21.15.64:0/1079009208 conn(0x55c2ca3805a0 msgr2=0x55c2ca38f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 shutdown_connections 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 1 -- 172.21.15.64:0/1079009208 wait complete. 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.320+0000 7f6c57dd7740 20 client.25354 trim_cache size 0 max 16384 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:09:43.326 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:09:43.327 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:09:43.327 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:09:43.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c380a2b90 msgr2=0x7f6c380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c380a2b90 0x7f6c380a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005c40 tx=0x7f6c48005bd0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 msgr2=0x7f6c50199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 0x7f6c50199d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000e9c0 tx=0x7f6c4000ee80 comp rx=0 tx=0).stop 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 shutdown_connections 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c380a2b90 0x7f6c380a5050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c501034d0 0x7f6c501a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c50102b10 0x7f6c5019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 --2- 172.21.15.64:0/2359755654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c500ffb40 0x7f6c50199d50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 >> 172.21.15.64:0/2359755654 conn(0x7f6c500fb470 msgr2=0x7f6c50100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 shutdown_connections 2024-05-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.321+0000 7f6c550ca640 1 -- 172.21.15.64:0/2359755654 wait complete. 2024-05-09T09:09:43.334 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:09:43.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:09:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 -- 172.21.15.64:0/3851633516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c103430 msgr2=0x7f903c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 --2- 172.21.15.64:0/3851633516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c103430 0x7f903c109cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9030009900 tx=0x7f903002a410 comp rx=0 tx=0).stop 2024-05-09T09:09:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 -- 172.21.15.64:0/3851633516 shutdown_connections 2024-05-09T09:09:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 --2- 172.21.15.64:0/3851633516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c103430 0x7f903c109cc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 --2- 172.21.15.64:0/3851633516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c102a70 0x7f903c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 --2- 172.21.15.64:0/3851633516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c101880 0x7f903c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 -- 172.21.15.64:0/3851633516 >> 172.21.15.64:0/3851633516 conn(0x7f903c0fce10 msgr2=0x7f903c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 -- 172.21.15.64:0/3851633516 shutdown_connections 2024-05-09T09:09:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.462+0000 7f9040df9640 1 -- 172.21.15.64:0/3851633516 wait complete. 2024-05-09T09:09:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 Processor -- start 2024-05-09T09:09:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 -- start start 2024-05-09T09:09:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c101880 0x7f903c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c102a70 0x7f903c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c103430 0x7f903c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10b1d0 con 0x7f903c101880 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10b050 con 0x7f903c103430 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f9040df9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10b350 con 0x7f903c102a70 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f903a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c101880 0x7f903c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f903a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c101880 0x7f903c195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55278/0 (socket says 172.21.15.64:55278) 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.463+0000 7f903a59c640 1 -- 172.21.15.64:0/1170660801 learned_addr learned my addr 172.21.15.64:0/1170660801 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c102a70 0x7f903c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f903ad9d640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c103430 0x7f903c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 -- 172.21.15.64:0/1170660801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c103430 msgr2=0x7f903c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c103430 0x7f903c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 -- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c101880 msgr2=0x7f903c195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c101880 0x7f903c195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 -- 172.21.15.64:0/1170660801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c19d450 con 0x7f903c102a70 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f903ad9d640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c103430 0x7f903c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9039d9b640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c102a70 0x7f903c195d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9034012f90 tx=0x7f9034010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903400d2d0 con 0x7f903c102a70 2024-05-09T09:09:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c19d6e0 con 0x7f903c102a70 2024-05-09T09:09:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.464+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c19dc80 con 0x7f903c102a70 2024-05-09T09:09:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.465+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f903401d070 con 0x7f903c102a70 2024-05-09T09:09:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.465+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340186e0 con 0x7f903c102a70 2024-05-09T09:09:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.466+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901c0052c0 con 0x7f903c102a70 2024-05-09T09:09:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.467+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9034016050 con 0x7f903c102a70 2024-05-09T09:09:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.468+0000 7f902b7fe640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90240a2be0 0x7f90240a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.468+0000 7f903a59c640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90240a2be0 0x7f90240a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.468+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f90340d32d0 con 0x7f903c102a70 2024-05-09T09:09:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.468+0000 7f903a59c640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90240a2be0 0x7f90240a50a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f902c0058c0 tx=0x7f902c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.472+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9034028020 con 0x7f903c102a70 2024-05-09T09:09:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:1200000 of 3200000 tuples (37%) done (elapsed 12.24 s, remaining 20.39 s) 2024-05-09T09:09:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.756+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f901c0055b0 con 0x7f903c102a70 2024-05-09T09:09:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.757+0000 7f902b7fe640 1 -- 172.21.15.64:0/1170660801 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11882 (secure 0 0 0) 0x7f9034014040 con 0x7f903c102a70 2024-05-09T09:09:43.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:09:43.756 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:09:30.076721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:09:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2024-05-09T09:09:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90240a2be0 msgr2=0x7f90240a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90240a2be0 0x7f90240a50a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f902c0058c0 tx=0x7f902c005f90 comp rx=0 tx=0).stop 2024-05-09T09:09:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c102a70 msgr2=0x7f903c195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c102a70 0x7f903c195d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9034012f90 tx=0x7f9034010490 comp rx=0 tx=0).stop 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 shutdown_connections 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f90240a2be0 0x7f90240a50a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f903c103430 0x7f903c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f903c102a70 0x7f903c195d70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 --2- 172.21.15.64:0/1170660801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f903c101880 0x7f903c195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 >> 172.21.15.64:0/1170660801 conn(0x7f903c0fce10 msgr2=0x7f903c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 shutdown_connections 2024-05-09T09:09:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.760+0000 7f9040df9640 1 -- 172.21.15.64:0/1170660801 wait complete. 2024-05-09T09:09:43.773 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:09:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 -- 172.21.15.64:0/3047758136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880101960 msgr2=0x7f2880101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 --2- 172.21.15.64:0/3047758136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880101960 0x7f2880101de0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2874009900 tx=0x7f287402a410 comp rx=0 tx=0).stop 2024-05-09T09:09:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 -- 172.21.15.64:0/3047758136 shutdown_connections 2024-05-09T09:09:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 --2- 172.21.15.64:0/3047758136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2880102320 0x7f2880108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 --2- 172.21.15.64:0/3047758136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880101960 0x7f2880101de0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 --2- 172.21.15.64:0/3047758136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880100770 0x7f2880100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 -- 172.21.15.64:0/3047758136 >> 172.21.15.64:0/3047758136 conn(0x7f28800fbce0 msgr2=0x7f28800fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 -- 172.21.15.64:0/3047758136 shutdown_connections 2024-05-09T09:09:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 -- 172.21.15.64:0/3047758136 wait complete. 2024-05-09T09:09:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.897+0000 7f2885f63640 1 Processor -- start 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 -- start start 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2880100770 0x7f288019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 0x7f288019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880102320 0x7f28801a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f288010c280 con 0x7f2880102320 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f288010c100 con 0x7f2880100770 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f2885f63640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f288010c400 con 0x7f2880101960 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f287effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 0x7f288019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f287effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 0x7f288019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39112/0 (socket says 172.21.15.64:39112) 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f287effd640 1 -- 172.21.15.64:0/3846533992 learned_addr learned my addr 172.21.15.64:0/3846533992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.898+0000 7f287f7fe640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2880100770 0x7f288019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287ffff640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880102320 0x7f28801a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287effd640 1 -- 172.21.15.64:0/3846533992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2880100770 msgr2=0x7f288019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287effd640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2880100770 0x7f288019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287effd640 1 -- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880102320 msgr2=0x7f28801a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287effd640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880102320 0x7f28801a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287effd640 1 -- 172.21.15.64:0/3846533992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28801a5d60 con 0x7f2880101960 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287ffff640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880102320 0x7f28801a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287effd640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 0x7f288019ece0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2874004770 tx=0x7f28740042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287cff9640 1 -- 172.21.15.64:0/3846533992 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287403f030 con 0x7f2880101960 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287cff9640 1 -- 172.21.15.64:0/3846533992 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2874003cb0 con 0x7f2880101960 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28801a5ff0 con 0x7f2880101960 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f287cff9640 1 -- 172.21.15.64:0/3846533992 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874035070 con 0x7f2880101960 2024-05-09T09:09:43.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.899+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28801a6450 con 0x7f2880101960 2024-05-09T09:09:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.902+0000 7f287cff9640 1 -- 172.21.15.64:0/3846533992 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2874042030 con 0x7f2880101960 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 Processor -- start 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 -- start start 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94944f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949454430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949456f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94944f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94944f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55304/0 (socket says 172.21.15.64:55304) 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287ffff640 1 -- 172.21.15.64:0/1758125865 learned_addr learned my addr 172.21.15.64:0/1758125865 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9493b3c40 con 0x55e94944ef40 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e94935b580 con 0x55e94944f930 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287f7fe640 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949454430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e9493afc00 con 0x55e949454aa0 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287cff9640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28680a2b90 0x7f28680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287f7fe640 1 -- 172.21.15.64:0/1758125865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 msgr2=0x55e949456f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287f7fe640 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949456f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287f7fe640 1 -- 172.21.15.64:0/1758125865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 msgr2=0x55e94944f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287f7fe640 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94944f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.903+0000 7f287f7fe640 1 -- 172.21.15.64:0/1758125865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9490c73f0 con 0x55e94944f930 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.904+0000 7f287ffff640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28680a2b90 0x7f28680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.904+0000 7f287cff9640 1 -- 172.21.15.64:0/3846533992 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28740efd70 con 0x7f2880101960 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.904+0000 7f287f7fe640 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949454430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28801017c0 tx=0x7f28700107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f16e48cf399236 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.904+0000 7f2865ffb640 1 -- 172.21.15.64:0/1758125865 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870004020 con 0x55e94944f930 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.904+0000 7f2865ffb640 1 -- 172.21.15.64:0/1758125865 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28700041c0 con 0x55e94944f930 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.904+0000 7f2865ffb640 1 -- 172.21.15.64:0/1758125865 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28700116a0 con 0x55e94944f930 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 msgr2=0x55e949454430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949454430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28801017c0 tx=0x7f28700107a0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f287ffff640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28680a2b90 0x7f28680a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e949450190 tx=0x7f2878004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 shutdown_connections 2024-05-09T09:09:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949456f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949454430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 --2- 172.21.15.64:0/1758125865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94944f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 >> 172.21.15.64:0/1758125865 conn(0x55e94944a5b0 msgr2=0x55e94944a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 shutdown_connections 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 -- 172.21.15.64:0/1758125865 wait complete. 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 -- start start 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.905+0000 7f2888c7a740 1 Processor -- start 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e94944ef40 0x55e949459420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949459960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 0x55e9494609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e949457a50 con 0x55e949454aa0 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e94936ec40 con 0x55e94944f930 2024-05-09T09:09:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e949457bd0 con 0x55e94944ef40 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f2888c7a740 10 client.0 _ll_register_callbacks cb 0x55e9490c6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 0x55e9494609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e94944ef40 0x55e949459420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e94944ef40 0x55e949459420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39138/0 (socket says 172.21.15.64:39138) 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 0x55e9494609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55314/0 (socket says 172.21.15.64:55314) 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287f7fe640 1 -- 172.21.15.64:0/4138504162 learned_addr learned my addr 172.21.15.64:0/4138504162 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287effd640 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949459960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 -- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e94944ef40 msgr2=0x55e949459420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e94944ef40 0x55e949459420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 -- 172.21.15.64:0/4138504162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 msgr2=0x55e949459960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949459960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 -- 172.21.15.64:0/4138504162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9491917d0 con 0x55e949454aa0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f287ffff640 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 0x55e9494609a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f28780095a0 tx=0x7f2878004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6cb785f9585632 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f285effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.906+0000 7f285effd640 1 -- 172.21.15.64:0/4138504162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287801edb0 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28700041c0 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f285effd640 1 -- 172.21.15.64:0/4138504162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f287801b410 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f285effd640 1 -- 172.21.15.64:0/4138504162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f287801b610 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e949463e30 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f285effd640 1 -- 172.21.15.64:0/4138504162 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28700041c0 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f285effd640 1 -- 172.21.15.64:0/4138504162 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2878025070 con 0x55e949454aa0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 10 client.15693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 20 client.15693 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494654f0 0x55e949467910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 4 client.15693 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.907+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9493c1370 con 0x55e9494654f0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.908+0000 7f287f7fe640 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494654f0 0x55e949467910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.908+0000 7f285effd640 10 client.15693 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:09:43.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.908+0000 7f287f7fe640 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494654f0 0x55e949467910 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=909fa1eefe047bfa server_cookie=bf5fa2ad41e9f5a in_seq=0 out_seq=0 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f285effd640 1 -- 172.21.15.64:0/4138504162 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9493c1370 con 0x55e9494654f0 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f285effd640 10 client.15693 handle_command_reply: tid=0 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 client.15693 shutdown 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 msgr2=0x55e9494609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 0x55e9494609a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f28780095a0 tx=0x7f2878004ab0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 shutdown_connections 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494654f0 0x55e949467910 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e949454aa0 0x55e9494609a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e949459960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 --2- 172.21.15.64:0/4138504162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e94944ef40 0x55e949459420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f285effd640 0 client.15693 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 >> 172.21.15.64:0/4138504162 conn(0x55e94944a5b0 msgr2=0x55e9494497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 shutdown_connections 2024-05-09T09:09:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 1 -- 172.21.15.64:0/4138504162 wait complete. 2024-05-09T09:09:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.909+0000 7f2888c7a740 20 client.15693 trim_cache size 0 max 16384 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 Processor -- start 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 -- start start 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944ef40 0x55e949458a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 0x55e949460960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949448f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28801b3380 con 0x55e94944f930 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28801b0c50 con 0x55e94944ef40 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f2888c7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28801aeaa0 con 0x55e949454aa0 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 0x55e949460960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 0x55e949460960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55316/0 (socket says 172.21.15.64:55316) 2024-05-09T09:09:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 -- 172.21.15.64:0/3530056895 learned_addr learned my addr 172.21.15.64:0/3530056895 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287ffff640 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949448f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287f7fe640 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944ef40 0x55e949458a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 -- 172.21.15.64:0/3530056895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 msgr2=0x55e949448f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949448f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 -- 172.21.15.64:0/3530056895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944ef40 msgr2=0x55e949458a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944ef40 0x55e949458a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 -- 172.21.15.64:0/3530056895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94944a150 con 0x55e94944f930 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287ffff640 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949448f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.924+0000 7f287effd640 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 0x55e949460960 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2874003fd0 tx=0x7f28740bbe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f7d5598dd2368b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.925+0000 7f28657fa640 1 -- 172.21.15.64:0/3530056895 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874049040 con 0x55e94944f930 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.925+0000 7f28657fa640 1 -- 172.21.15.64:0/3530056895 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28740bc940 con 0x55e94944f930 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.925+0000 7f28657fa640 1 -- 172.21.15.64:0/3530056895 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287403bc90 con 0x55e94944f930 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.925+0000 7f2888c7a740 1 -- 172.21.15.64:0/3530056895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 msgr2=0x55e949460960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.925+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 0x55e949460960 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2874003fd0 tx=0x7f28740bbe90 comp rx=0 tx=0).stop 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.925+0000 7f2888c7a740 1 -- 172.21.15.64:0/3530056895 shutdown_connections 2024-05-09T09:09:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e949448f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944f930 0x55e949460960 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3530056895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944ef40 0x55e949458a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- 172.21.15.64:0/3530056895 >> 172.21.15.64:0/3530056895 conn(0x55e94944a5b0 msgr2=0x55e949450d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- 172.21.15.64:0/3530056895 shutdown_connections 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- 172.21.15.64:0/3530056895 wait complete. 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- start start 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 Processor -- start 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94945e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e94945eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e94945fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28801af110 con 0x55e94944ef40 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28801b0580 con 0x55e94944f930 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28801af7e0 con 0x55e949454aa0 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f2888c7a740 10 client.0 _ll_register_callbacks cb 0x7f28780094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.926+0000 7f287f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94945e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94945e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55328/0 (socket says 172.21.15.64:55328) 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 -- 172.21.15.64:0/3057132866 learned_addr learned my addr 172.21.15.64:0/3057132866 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287effd640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e94945eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287ffff640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e94945fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 -- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 msgr2=0x55e94945fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e94945fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 -- 172.21.15.64:0/3057132866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 msgr2=0x55e94945eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e94945eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 -- 172.21.15.64:0/3057132866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9494602a0 con 0x55e94944ef40 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f287f7fe640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94945e760 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2870005890 tx=0x7f287000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af06d379164bee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f285effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f285effd640 1 -- 172.21.15.64:0/3057132866 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287002e070 con 0x55e94944ef40 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f285effd640 1 -- 172.21.15.64:0/3057132866 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2870005240 con 0x55e94944ef40 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28740bc940 con 0x55e94944ef40 2024-05-09T09:09:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f285effd640 1 -- 172.21.15.64:0/3057132866 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f287003a050 con 0x55e94944ef40 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f2888c7a740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.927+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9494cc1a0 con 0x55e94944ef40 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f285effd640 1 -- 172.21.15.64:0/3057132866 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28740bc940 con 0x55e94944ef40 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f285effd640 1 -- 172.21.15.64:0/3057132866 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f287002a070 con 0x55e94944ef40 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f2888c7a740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f2888c7a740 10 client.15699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f2888c7a740 20 client.15699 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494cc8c0 0x55e9494cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f2888c7a740 4 client.15699 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9493bb6c0 con 0x55e9494cc8c0 2024-05-09T09:09:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f287effd640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494cc8c0 0x55e9494cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f287effd640 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494cc8c0 0x55e9494cece0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46a366a988c5b88d server_cookie=55b86e5027c431a6 in_seq=0 out_seq=0 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.928+0000 7f285effd640 10 client.15699 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f285effd640 1 -- 172.21.15.64:0/3057132866 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9493bb6c0 con 0x55e9494cc8c0 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f285effd640 10 client.15699 handle_command_reply: tid=0 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 client.15699 shutdown 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 msgr2=0x55e94945e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94945e760 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2870005890 tx=0x7f287000b400 comp rx=0 tx=0).stop 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 shutdown_connections 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e9494cc8c0 0x55e9494cece0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f285effd640 0 client.15699 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e949454aa0 0x55e94945fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e94944f930 0x55e94945eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 --2- 172.21.15.64:0/3057132866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e94944ef40 0x55e94945e760 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 >> 172.21.15.64:0/3057132866 conn(0x55e94944a5b0 msgr2=0x55e94944d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 shutdown_connections 2024-05-09T09:09:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 1 -- 172.21.15.64:0/3057132866 wait complete. 2024-05-09T09:09:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.929+0000 7f2888c7a740 20 client.15699 trim_cache size 0 max 16384 2024-05-09T09:09:43.934 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28680a2b90 msgr2=0x7f28680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28680a2b90 0x7f28680a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e949450190 tx=0x7f2878004a40 comp rx=0 tx=0).stop 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 msgr2=0x7f288019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 0x7f288019ece0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2874004770 tx=0x7f28740042f0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 shutdown_connections 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28680a2b90 0x7f28680a5050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880102320 0x7f28801a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2880101960 0x7f288019ece0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 --2- 172.21.15.64:0/3846533992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2880100770 0x7f288019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 >> 172.21.15.64:0/3846533992 conn(0x7f28800fbce0 msgr2=0x7f28800689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 shutdown_connections 2024-05-09T09:09:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:43.930+0000 7f2885f63640 1 -- 172.21.15.64:0/3846533992 wait complete. 2024-05-09T09:09:43.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:09:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 -- 172.21.15.64:0/2785471914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 msgr2=0x7f22b0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 --2- 172.21.15.64:0/2785471914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b0109cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009ca0 tx=0x7f22a802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 -- 172.21.15.64:0/2785471914 shutdown_connections 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 --2- 172.21.15.64:0/2785471914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b0109cb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 --2- 172.21.15.64:0/2785471914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 0x7f22b0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 --2- 172.21.15.64:0/2785471914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22b0101870 0x7f22b0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 -- 172.21.15.64:0/2785471914 >> 172.21.15.64:0/2785471914 conn(0x7f22b00fcde0 msgr2=0x7f22b00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 -- 172.21.15.64:0/2785471914 shutdown_connections 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.068+0000 7f22b628c640 1 -- 172.21.15.64:0/2785471914 wait complete. 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 Processor -- start 2024-05-09T09:09:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 -- start start 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22b0101870 0x7f22b0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 0x7f22b0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010d3a0 con 0x7f22b0103420 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010d220 con 0x7f22b0101870 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b628c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010d520 con 0x7f22b0102a60 2024-05-09T09:09:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b4829640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b4829640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55354/0 (socket says 172.21.15.64:55354) 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b4829640 1 -- 172.21.15.64:0/4291409585 learned_addr learned my addr 172.21.15.64:0/4291409585 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22af7fe640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 0x7f22b0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22affff640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22b0101870 0x7f22b0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.069+0000 7f22b4829640 1 -- 172.21.15.64:0/4291409585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 msgr2=0x7f22b0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b4829640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 0x7f22b0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b4829640 1 -- 172.21.15.64:0/4291409585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22b0101870 msgr2=0x7f22b0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b4829640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22b0101870 0x7f22b0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b4829640 1 -- 172.21.15.64:0/4291409585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b019f690 con 0x7f22b0103420 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22af7fe640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 0x7f22b0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b4829640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b019f020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f22a8031970 tx=0x7f22a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22ad7fa640 1 -- 172.21.15.64:0/4291409585 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8004040 con 0x7f22b0103420 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b019f920 con 0x7f22b0103420 2024-05-09T09:09:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22ad7fa640 1 -- 172.21.15.64:0/4291409585 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a8004690 con 0x7f22b0103420 2024-05-09T09:09:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22ad7fa640 1 -- 172.21.15.64:0/4291409585 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8030c60 con 0x7f22b0103420 2024-05-09T09:09:44.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.070+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b019fec0 con 0x7f22b0103420 2024-05-09T09:09:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.073+0000 7f22ad7fa640 1 -- 172.21.15.64:0/4291409585 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22a80041e0 con 0x7f22b0103420 2024-05-09T09:09:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.073+0000 7f22b8f96740 1 Processor -- start 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.073+0000 7f22b8f96740 1 -- start start 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be0461770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461d40 0x562be0466840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be0469340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b8f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562be03c1c40 con 0x562be0461350 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b8f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562be0369580 con 0x562be0466eb0 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b8f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562be03bdc00 con 0x562be0461d40 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be0461770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461d40 0x562be0466840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be0461770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55368/0 (socket says 172.21.15.64:55368) 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22af7fe640 1 -- 172.21.15.64:0/1240137324 learned_addr learned my addr 172.21.15.64:0/1240137324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22af7fe640 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be0469340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22ad7fa640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22980a2b90 0x7f22980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 -- 172.21.15.64:0/1240137324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 msgr2=0x562be0469340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be0469340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 -- 172.21.15.64:0/1240137324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 msgr2=0x562be0461770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be0461770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 -- 172.21.15.64:0/1240137324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be00d53f0 con 0x562be0461d40 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22af7fe640 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be0469340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22b4829640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22980a2b90 0x7f22980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.074+0000 7f22affff640 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461d40 0x562be0466840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22a000bbb0 tx=0x7f22a000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ab433170e74e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22ad7fa640 1 -- 172.21.15.64:0/4291409585 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22a80f09c0 con 0x7f22b0103420 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22967fc640 1 -- 172.21.15.64:0/1240137324 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0015070 con 0x562be0461d40 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22967fc640 1 -- 172.21.15.64:0/1240137324 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0005020 con 0x562be0461d40 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22967fc640 1 -- 172.21.15.64:0/1240137324 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00049a0 con 0x562be0461d40 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22b8f96740 1 -- 172.21.15.64:0/1240137324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461d40 msgr2=0x562be0466840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22b8f96740 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461d40 0x562be0466840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22a000bbb0 tx=0x7f22a000f730 comp rx=0 tx=0).stop 2024-05-09T09:09:44.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.075+0000 7f22b4829640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22980a2b90 0x7f22980a5050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f22a80b92f0 tx=0x7f22a80e7a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 -- 172.21.15.64:0/1240137324 shutdown_connections 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be0469340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461d40 0x562be0466840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 --2- 172.21.15.64:0/1240137324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be0461770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 -- 172.21.15.64:0/1240137324 >> 172.21.15.64:0/1240137324 conn(0x562be04585b0 msgr2=0x562be04589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 -- 172.21.15.64:0/1240137324 shutdown_connections 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 -- 172.21.15.64:0/1240137324 wait complete. 2024-05-09T09:09:44.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 -- start start 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 Processor -- start 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.076+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be046b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be046bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0466eb0 0x562be0472cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562be0469e60 con 0x562be0461350 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562be037cc40 con 0x562be0461d40 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562be0469fe0 con 0x562be0466eb0 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 10 client.0 _ll_register_callbacks cb 0x562be00d4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be046b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be046b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55380/0 (socket says 172.21.15.64:55380) 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 -- 172.21.15.64:0/4151142770 learned_addr learned my addr 172.21.15.64:0/4151142770 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22affff640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be046bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0466eb0 0x562be0472cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 -- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0466eb0 msgr2=0x562be0472cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0466eb0 0x562be0472cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 -- 172.21.15.64:0/4151142770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 msgr2=0x562be046bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be046bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 -- 172.21.15.64:0/4151142770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be019f7d0 con 0x562be0461350 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22af7fe640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be046b740 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f22a4015f20 tx=0x7f22a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b1046a8e6f78e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f228f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f228f7fe640 1 -- 172.21.15.64:0/4151142770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a40150d0 con 0x562be0461350 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f228f7fe640 1 -- 172.21.15.64:0/4151142770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22a4015270 con 0x562be0461350 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.077+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a0005020 con 0x562be0461350 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f228f7fe640 1 -- 172.21.15.64:0/4151142770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22a4005020 con 0x562be0461350 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562be0476010 con 0x562be0461350 2024-05-09T09:09:44.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f228f7fe640 1 -- 172.21.15.64:0/4151142770 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0005020 con 0x562be0461350 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f228f7fe640 1 -- 172.21.15.64:0/4151142770 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22a4004ca0 con 0x562be0461350 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 10 client.15711 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 20 client.15711 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04776e0 0x562be0479b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 4 client.15711 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562be03cf370 con 0x562be04776e0 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.078+0000 7f22affff640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04776e0 0x562be0479b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.079+0000 7f228f7fe640 10 client.15711 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.079+0000 7f22affff640 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04776e0 0x562be0479b80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2597e8d51a0ef66e server_cookie=24744f9e0e5d865d in_seq=0 out_seq=0 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.079+0000 7f228f7fe640 1 -- 172.21.15.64:0/4151142770 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562be03cf370 con 0x562be04776e0 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.079+0000 7f228f7fe640 10 client.15711 handle_command_reply: tid=0 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.079+0000 7f22b8f96740 1 client.15711 shutdown 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 msgr2=0x562be046b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be046b740 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f22a4015f20 tx=0x7f22a4005f90 comp rx=0 tx=0).stop 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 shutdown_connections 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0466eb0 0x562be0472cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be046bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04776e0 0x562be0479b80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 --2- 172.21.15.64:0/4151142770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461350 0x562be046b740 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f228f7fe640 0 client.15711 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 >> 172.21.15.64:0/4151142770 conn(0x562be04585b0 msgr2=0x562be04577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 shutdown_connections 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 1 -- 172.21.15.64:0/4151142770 wait complete. 2024-05-09T09:09:44.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.080+0000 7f22b8f96740 20 client.15711 trim_cache size 0 max 16384 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 Processor -- start 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 -- start start 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be046ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 0x562be0472c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be045bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01aaa60 con 0x562be0461d40 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a8330 con 0x562be0466eb0 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22b8f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a6180 con 0x562be0461350 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.096+0000 7f22affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 0x562be0472c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 0x562be0472c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55390/0 (socket says 172.21.15.64:55390) 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 -- 172.21.15.64:0/3679659536 learned_addr learned my addr 172.21.15.64:0/3679659536 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22af7fe640 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be046ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22af7fe640 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be045bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 -- 172.21.15.64:0/3679659536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 msgr2=0x562be046ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be046ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 -- 172.21.15.64:0/3679659536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 msgr2=0x562be045bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be045bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 -- 172.21.15.64:0/3679659536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be0449d20 con 0x562be0461d40 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22af7fe640 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be046ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f22affff640 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 0x562be0472c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f22a000bbb0 tx=0x7f22a001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e033617819be4ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f2295ffb640 1 -- 172.21.15.64:0/3679659536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a002d070 con 0x562be0461d40 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f2295ffb640 1 -- 172.21.15.64:0/3679659536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0028780 con 0x562be0461d40 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.097+0000 7f2295ffb640 1 -- 172.21.15.64:0/3679659536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0015040 con 0x562be0461d40 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 -- 172.21.15.64:0/3679659536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 msgr2=0x562be0472c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 0x562be0472c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f22a000bbb0 tx=0x7f22a001d040 comp rx=0 tx=0).stop 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 -- 172.21.15.64:0/3679659536 shutdown_connections 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0466eb0 0x562be045bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0461d40 0x562be0472c80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 --2- 172.21.15.64:0/3679659536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be046ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 -- 172.21.15.64:0/3679659536 >> 172.21.15.64:0/3679659536 conn(0x562be04585b0 msgr2=0x562be045b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 -- 172.21.15.64:0/3679659536 shutdown_connections 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 -- 172.21.15.64:0/3679659536 wait complete. 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.098+0000 7f22b8f96740 1 -- start start 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 Processor -- start 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be04705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be0470b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 0x562be0471b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a67f0 con 0x562be0466eb0 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a7c60 con 0x562be0461d40 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a6ec0 con 0x562be0461350 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22b8f96740 10 client.0 _ll_register_callbacks cb 0x7f22a4015db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 0x562be0471b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 0x562be0471b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55398/0 (socket says 172.21.15.64:55398) 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22af7fe640 1 -- 172.21.15.64:0/429588213 learned_addr learned my addr 172.21.15.64:0/429588213 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22affff640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be0470b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22af7fe640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be04705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22affff640 1 -- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 msgr2=0x562be04705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22affff640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be04705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22affff640 1 -- 172.21.15.64:0/429588213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 msgr2=0x562be0471b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22affff640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 0x562be0471b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22affff640 1 -- 172.21.15.64:0/429588213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be0472130 con 0x562be0461d40 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.099+0000 7f22af7fe640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 0x562be0471b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22affff640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be0470b30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x562be03bcb40 tx=0x7f22a000fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3147fd2815e760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f228f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:09:44.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f228f7fe640 1 -- 172.21.15.64:0/429588213 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a002d030 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a0028780 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f228f7fe640 1 -- 172.21.15.64:0/429588213 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22a00266e0 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f228f7fe640 1 -- 172.21.15.64:0/429588213 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22a0015050 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 10 client.25292 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562be04dde40 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f228f7fe640 1 -- 172.21.15.64:0/429588213 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0028780 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f228f7fe640 1 -- 172.21.15.64:0/429588213 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22a00118c0 con 0x562be0461d40 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 10 client.25292 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 10 client.25292 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 20 client.25292 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.100+0000 7f22b8f96740 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04de5f0 0x562be04e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f22b8f96740 4 client.25292 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562be03c96c0 con 0x562be04de5f0 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f22af7fe640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04de5f0 0x562be04e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f22af7fe640 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04de5f0 0x562be04e0a10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51dd6a71777a33e1 server_cookie=f24adcfad8bc1c39 in_seq=0 out_seq=0 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f228f7fe640 10 client.25292 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f228f7fe640 1 -- 172.21.15.64:0/429588213 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562be03c96c0 con 0x562be04de5f0 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f228f7fe640 10 client.25292 handle_command_reply: tid=0 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.101+0000 7f22b8f96740 1 client.25292 shutdown 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 msgr2=0x562be0470b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be0470b30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x562be03bcb40 tx=0x7f22a000fce0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 shutdown_connections 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562be04de5f0 0x562be04e0a10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562be0466eb0 0x562be0471b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562be0461d40 0x562be0470b30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 --2- 172.21.15.64:0/429588213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562be0461350 0x562be04705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f228f7fe640 0 client.25292 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 >> 172.21.15.64:0/429588213 conn(0x562be04585b0 msgr2=0x562be046aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 shutdown_connections 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 1 -- 172.21.15.64:0/429588213 wait complete. 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b8f96740 20 client.25292 trim_cache size 0 max 16384 2024-05-09T09:09:44.107 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22980a2b90 msgr2=0x7f22980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b628c640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22980a2b90 0x7f22980a5050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f22a80b92f0 tx=0x7f22a80e7a20 comp rx=0 tx=0).stop 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 msgr2=0x7f22b019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.102+0000 7f22b628c640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b019f020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f22a8031970 tx=0x7f22a8003ba0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 shutdown_connections 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f22980a2b90 0x7f22980a5050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0103420 0x7f22b019f020 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0102a60 0x7f22b0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 --2- 172.21.15.64:0/4291409585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22b0101870 0x7f22b0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 >> 172.21.15.64:0/4291409585 conn(0x7f22b00fcde0 msgr2=0x7f22b0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 shutdown_connections 2024-05-09T09:09:44.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.103+0000 7f22b628c640 1 -- 172.21.15.64:0/4291409585 wait complete. 2024-05-09T09:09:44.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:09:44.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 -- 172.21.15.64:0/1674664003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0101970 msgr2=0x7f7eb0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 --2- 172.21.15.64:0/1674664003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0101970 0x7f7eb0101df0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009900 tx=0x7f7ea002a410 comp rx=0 tx=0).stop 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 -- 172.21.15.64:0/1674664003 shutdown_connections 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 --2- 172.21.15.64:0/1674664003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb0102330 0x7f7eb0108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 --2- 172.21.15.64:0/1674664003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0101970 0x7f7eb0101df0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 --2- 172.21.15.64:0/1674664003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb0100780 0x7f7eb0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 -- 172.21.15.64:0/1674664003 >> 172.21.15.64:0/1674664003 conn(0x7f7eb00fbcf0 msgr2=0x7f7eb00fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.250+0000 7f7eb7410640 1 -- 172.21.15.64:0/1674664003 shutdown_connections 2024-05-09T09:09:44.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.251+0000 7f7eb7410640 1 -- 172.21.15.64:0/1674664003 wait complete. 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.251+0000 7f7eb7410640 1 Processor -- start 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.251+0000 7f7eb7410640 1 -- start start 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 0x7f7eb0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb0101970 0x7f7eb019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb0102330 0x7f7eb01a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010c2b0 con 0x7f7eb0100780 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010c130 con 0x7f7eb0102330 2024-05-09T09:09:44.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010c430 con 0x7f7eb0101970 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 0x7f7eb0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 0x7f7eb0199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55404/0 (socket says 172.21.15.64:55404) 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 -- 172.21.15.64:0/523085251 learned_addr learned my addr 172.21.15.64:0/523085251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb49ab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb0101970 0x7f7eb019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb0102330 0x7f7eb01a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 -- 172.21.15.64:0/523085251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb0101970 msgr2=0x7f7eb019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb0101970 0x7f7eb019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 -- 172.21.15.64:0/523085251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb0102330 msgr2=0x7f7eb01a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb0102330 0x7f7eb01a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 -- 172.21.15.64:0/523085251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01a1930 con 0x7f7eb0100780 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 0x7f7eb0199d20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9c0 tx=0x7f7eac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7ea67fc640 1 -- 172.21.15.64:0/523085251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00cd80 con 0x7f7eb0100780 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01a1c20 con 0x7f7eb0100780 2024-05-09T09:09:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.252+0000 7f7ea67fc640 1 -- 172.21.15.64:0/523085251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac0035f0 con 0x7f7eb0100780 2024-05-09T09:09:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.253+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01a21c0 con 0x7f7eb0100780 2024-05-09T09:09:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.253+0000 7f7ea67fc640 1 -- 172.21.15.64:0/523085251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac018630 con 0x7f7eb0100780 2024-05-09T09:09:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7ea67fc640 1 -- 172.21.15.64:0/523085251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7eac0037a0 con 0x7f7eb0100780 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 Processor -- start 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 -- start start 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575da930 0x5569575df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575dfaa0 0x5569575e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55695753ec40 con 0x5569575da930 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5569574e6580 con 0x5569575d9f40 2024-05-09T09:09:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55695753ac00 con 0x5569575dfaa0 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575da930 0x5569575df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.256+0000 7f7eb59ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575da360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48014/0 (socket says 172.21.15.64:48014) 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 -- 172.21.15.64:0/2392083635 learned_addr learned my addr 172.21.15.64:0/2392083635 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575dfaa0 0x5569575e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 -- 172.21.15.64:0/2392083635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575dfaa0 msgr2=0x5569575e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575dfaa0 0x5569575e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 -- 172.21.15.64:0/2392083635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575da930 msgr2=0x5569575df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575da930 0x5569575df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7ea67fc640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7e9c0a2b90 0x7f7e9c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 -- 172.21.15.64:0/2392083635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569572523f0 con 0x5569575d9f40 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7e9c0a2b90 0x7f7e9c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7ea67fc640 1 -- 172.21.15.64:0/523085251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7eac0cf7d0 con 0x7f7eb0100780 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575da360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8010370 tx=0x7f7ea8010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9fe895703e6bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.257+0000 7f7e9b7fe640 1 -- 172.21.15.64:0/2392083635 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8022070 con 0x5569575d9f40 2024-05-09T09:09:44.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7e9b7fe640 1 -- 172.21.15.64:0/2392083635 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea80113e0 con 0x5569575d9f40 2024-05-09T09:09:44.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7e9b7fe640 1 -- 172.21.15.64:0/2392083635 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8016070 con 0x5569575d9f40 2024-05-09T09:09:44.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2392083635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 msgr2=0x5569575da360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575da360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8010370 tx=0x7f7ea8010830 comp rx=0 tx=0).stop 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7e9c0a2b90 0x7f7e9c0a5050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ea801f000 tx=0x7f7ea8022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2392083635 shutdown_connections 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575dfaa0 0x5569575e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575da930 0x5569575df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2392083635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575da360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2392083635 >> 172.21.15.64:0/2392083635 conn(0x5569575d55b0 msgr2=0x5569575d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.258+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2392083635 shutdown_connections 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2392083635 wait complete. 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 -- start start 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 Processor -- start 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 0x5569575e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575da930 0x5569575e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5569575e2a50 con 0x5569575dfaa0 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5569574f9c40 con 0x5569575da930 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5569575e2bd0 con 0x5569575d9f40 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eba1f5740 10 client.0 _ll_register_callbacks cb 0x556957251be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eb59ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 0x5569575e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eb59ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575eb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55422/0 (socket says 172.21.15.64:55422) 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 0x5569575e4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39256/0 (socket says 172.21.15.64:39256) 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.259+0000 7f7eb59ad640 1 -- 172.21.15.64:0/2594180292 learned_addr learned my addr 172.21.15.64:0/2594180292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575da930 0x5569575e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575da930 msgr2=0x5569575e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575da930 0x5569575e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2594180292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 msgr2=0x5569575eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2594180292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55695731c7d0 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 0x5569575e4310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0ccf90 tx=0x7f7eac009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109b2479b0830a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7e98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7e98ff9640 1 -- 172.21.15.64:0/2594180292 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac09ba70 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7e98ff9640 1 -- 172.21.15.64:0/2594180292 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7eac09b0f0 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea80113e0 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7e98ff9640 1 -- 172.21.15.64:0/2594180292 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7eac099de0 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eba1f5740 10 client.25405 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.260+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569575eecc0 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7e98ff9640 1 -- 172.21.15.64:0/2594180292 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea80113e0 con 0x5569575d9f40 2024-05-09T09:09:44.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7e98ff9640 1 -- 172.21.15.64:0/2594180292 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7eac09ab60 con 0x5569575d9f40 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eba1f5740 10 client.25405 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eba1f5740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eba1f5740 20 client.25405 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569575f0350 0x5569575f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eba1f5740 4 client.25405 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55695754c370 con 0x5569575f0350 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569575f0350 0x5569575f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7e98ff9640 10 client.25405 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.261+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569575f0350 0x5569575f27e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae197ecd5f448fad server_cookie=e32bca921354ff5c in_seq=0 out_seq=0 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.262+0000 7f7e98ff9640 1 -- 172.21.15.64:0/2594180292 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55695754c370 con 0x5569575f0350 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.262+0000 7f7e98ff9640 10 client.25405 handle_command_reply: tid=0 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.262+0000 7f7eba1f5740 1 client.25405 shutdown 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.262+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 msgr2=0x5569575e4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.262+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 0x5569575e4310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0ccf90 tx=0x7f7eac009770 comp rx=0 tx=0).stop 2024-05-09T09:09:44.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 shutdown_connections 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569575f0350 0x5569575f27e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575da930 0x5569575e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7e98ff9640 0 client.25405 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2594180292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575d9f40 0x5569575e4310 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 >> 172.21.15.64:0/2594180292 conn(0x5569575d55b0 msgr2=0x5569575d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 shutdown_connections 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2594180292 wait complete. 2024-05-09T09:09:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.263+0000 7f7eba1f5740 20 client.25405 trim_cache size 0 max 16384 2024-05-09T09:09:44.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.276+0000 7f7eba1f5740 1 Processor -- start 2024-05-09T09:09:44.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 -- start start 2024-05-09T09:09:44.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 0x5569575e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575dfaa0 0x5569575d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01aee80 con 0x5569575d9f40 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01ac750 con 0x5569575dfaa0 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01aa5a0 con 0x5569575da930 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 0x5569575e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 0x5569575e47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55436/0 (socket says 172.21.15.64:55436) 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2221755554 learned_addr learned my addr 172.21.15.64:0/2221755554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575dfaa0 0x5569575d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2221755554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 msgr2=0x5569575eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2221755554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575dfaa0 msgr2=0x5569575d4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575dfaa0 0x5569575d4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 -- 172.21.15.64:0/2221755554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569575c6d20 con 0x5569575d9f40 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 0x5569575e47b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c6a0 tx=0x7f7eac003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc336f2158c06b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.277+0000 7f7e9affd640 1 -- 172.21.15.64:0/2221755554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac09b600 con 0x5569575d9f40 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7e9affd640 1 -- 172.21.15.64:0/2221755554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac003b30 con 0x5569575d9f40 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7e9affd640 1 -- 172.21.15.64:0/2221755554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac09b7e0 con 0x5569575d9f40 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2221755554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 msgr2=0x5569575e47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 0x5569575e47b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c6a0 tx=0x7f7eac003e30 comp rx=0 tx=0).stop 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2221755554 shutdown_connections 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575dfaa0 0x5569575d4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/2221755554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575d9f40 0x5569575e47b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2221755554 >> 172.21.15.64:0/2221755554 conn(0x5569575d55b0 msgr2=0x5569575d3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2221755554 shutdown_connections 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.278+0000 7f7eba1f5740 1 -- 172.21.15.64:0/2221755554 wait complete. 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 -- start start 2024-05-09T09:09:44.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 Processor -- start 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01aac10 con 0x5569575dfaa0 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01ac080 con 0x5569575d9f40 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01ab2e0 con 0x5569575da930 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eba1f5740 10 client.0 _ll_register_callbacks cb 0x7f7eac0d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb59ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48048/0 (socket says 172.21.15.64:48048) 2024-05-09T09:09:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb59ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575ea020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55442/0 (socket says 172.21.15.64:55442) 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 -- 172.21.15.64:0/3522517513 learned_addr learned my addr 172.21.15.64:0/3522517513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 -- 172.21.15.64:0/3522517513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 msgr2=0x5569575e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 -- 172.21.15.64:0/3522517513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 msgr2=0x5569575ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.279+0000 7f7eb51ac640 1 -- 172.21.15.64:0/3522517513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569575ea610 con 0x5569575d9f40 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eb59ad640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eb51ac640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575e8ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00cfd0 tx=0x7f7eac09c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d358a41f73b7558 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7e98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7e98ff9640 1 -- 172.21.15.64:0/3522517513 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac09b7e0 con 0x5569575d9f40 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5569575466c0 con 0x5569575d9f40 2024-05-09T09:09:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7e98ff9640 1 -- 172.21.15.64:0/3522517513 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7eac097960 con 0x5569575d9f40 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7e98ff9640 1 -- 172.21.15.64:0/3522517513 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7eac09b600 con 0x5569575d9f40 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eba1f5740 10 client.25310 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569575ea610 con 0x5569575d9f40 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7e98ff9640 1 -- 172.21.15.64:0/3522517513 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5569575466c0 con 0x5569575d9f40 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7e98ff9640 1 -- 172.21.15.64:0/3522517513 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7eac082070 con 0x5569575d9f40 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eba1f5740 10 client.25310 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eba1f5740 10 client.25310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.280+0000 7f7eba1f5740 20 client.25310 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569576569e0 0x556957658dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 4 client.25310 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569574c3820 con 0x5569576569e0 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569576569e0 0x556957658dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eb49ab640 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569576569e0 0x556957658dd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7150d31b1da3609 server_cookie=28162d81efb359ff in_seq=0 out_seq=0 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7e98ff9640 10 client.25310 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7e98ff9640 1 -- 172.21.15.64:0/3522517513 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569574c3820 con 0x5569576569e0 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7e98ff9640 10 client.25310 handle_command_reply: tid=0 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 client.25310 shutdown 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 msgr2=0x5569575e8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575e8ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00cfd0 tx=0x7f7eac09c3a0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 shutdown_connections 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5569576569e0 0x556957658dd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569575dfaa0 0x5569575ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569575da930 0x5569575e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 --2- 172.21.15.64:0/3522517513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5569575d9f40 0x5569575e8ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7e98ff9640 0 client.25310 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.281+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 >> 172.21.15.64:0/3522517513 conn(0x5569575d55b0 msgr2=0x5569575e3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 shutdown_connections 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eba1f5740 1 -- 172.21.15.64:0/3522517513 wait complete. 2024-05-09T09:09:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eba1f5740 20 client.25310 trim_cache size 0 max 16384 2024-05-09T09:09:44.287 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:09:44.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7e9c0a2b90 msgr2=0x7f7e9c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7e9c0a2b90 0x7f7e9c0a5050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ea801f000 tx=0x7f7ea8022000 comp rx=0 tx=0).stop 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 msgr2=0x7f7eb0199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 0x7f7eb0199d20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9c0 tx=0x7f7eac00ee80 comp rx=0 tx=0).stop 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 shutdown_connections 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7e9c0a2b90 0x7f7e9c0a5050 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb0102330 0x7f7eb01a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb0101970 0x7f7eb019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 --2- 172.21.15.64:0/523085251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100780 0x7f7eb0199d20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 >> 172.21.15.64:0/523085251 conn(0x7f7eb00fbcf0 msgr2=0x7f7eb0069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 shutdown_connections 2024-05-09T09:09:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:44.282+0000 7f7eb7410640 1 -- 172.21.15.64:0/523085251 wait complete. 2024-05-09T09:09:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:44 smithi064 ceph-mon[30149]: pgmap v388: 129 pgs: 129 active+clean; 234 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 31 MiB/s wr, 88 op/s 2024-05-09T09:09:44.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:44 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1170660801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:44.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:44 smithi179 ceph-mon[34086]: pgmap v388: 129 pgs: 129 active+clean; 234 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 31 MiB/s wr, 88 op/s 2024-05-09T09:09:44.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:44 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1170660801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:44 smithi139 ceph-mon[33599]: pgmap v388: 129 pgs: 129 active+clean; 234 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 31 MiB/s wr, 88 op/s 2024-05-09T09:09:44.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:44 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1170660801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:09:44.777 INFO:teuthology.orchestra.run.smithi064.stderr:1300000 of 3200000 tuples (40%) done (elapsed 13.45 s, remaining 19.65 s) 2024-05-09T09:09:45.293 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:09:45.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:09:45.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:09:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 -- 172.21.15.64:0/3653794828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 msgr2=0x7fea7c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 --2- 172.21.15.64:0/3653794828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c101ca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009900 tx=0x7fea6c02a410 comp rx=0 tx=0).stop 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 -- 172.21.15.64:0/3653794828 shutdown_connections 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 --2- 172.21.15.64:0/3653794828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7c103430 0x7fea7c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 --2- 172.21.15.64:0/3653794828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c102a70 0x7fea7c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 --2- 172.21.15.64:0/3653794828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c101ca0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 -- 172.21.15.64:0/3653794828 >> 172.21.15.64:0/3653794828 conn(0x7fea7c0fce10 msgr2=0x7fea7c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 -- 172.21.15.64:0/3653794828 shutdown_connections 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.420+0000 7fea8155a640 1 -- 172.21.15.64:0/3653794828 wait complete. 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 Processor -- start 2024-05-09T09:09:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 -- start start 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7c102a70 0x7fea7c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c103430 0x7fea7c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c10d380 con 0x7fea7c101880 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c10d200 con 0x7fea7c103430 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea8155a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c10d500 con 0x7fea7c102a70 2024-05-09T09:09:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47026/0 (socket says 172.21.15.64:47026) 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea7affd640 1 -- 172.21.15.64:0/2530212807 learned_addr learned my addr 172.21.15.64:0/2530212807 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c103430 0x7fea7c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.421+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7c102a70 0x7fea7c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7affd640 1 -- 172.21.15.64:0/2530212807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7c102a70 msgr2=0x7fea7c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7affd640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7c102a70 0x7fea7c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7affd640 1 -- 172.21.15.64:0/2530212807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c103430 msgr2=0x7fea7c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7affd640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c103430 0x7fea7c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7affd640 1 -- 172.21.15.64:0/2530212807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c19f730 con 0x7fea7c101880 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c103430 0x7fea7c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea7affd640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c197b10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0314e0 tx=0x7fea6c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea63fff640 1 -- 172.21.15.64:0/2530212807 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c03f030 con 0x7fea7c101880 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c19f9c0 con 0x7fea7c101880 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea63fff640 1 -- 172.21.15.64:0/2530212807 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea6c0044b0 con 0x7fea7c101880 2024-05-09T09:09:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea63fff640 1 -- 172.21.15.64:0/2530212807 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c035070 con 0x7fea7c101880 2024-05-09T09:09:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.422+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c19feb0 con 0x7fea7c101880 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea63fff640 1 -- 172.21.15.64:0/2530212807 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea6c042030 con 0x7fea7c101880 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 Processor -- start 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 -- start start 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a5068848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a5068899b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 0x55a50688c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5067e8990 con 0x55a50688a020 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a5067905b0 con 0x55a506884eb0 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea84262740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a5067e4950 con 0x55a5068844c0 2024-05-09T09:09:45.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea7a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 0x55a50688c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea7a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 0x55a50688c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47038/0 (socket says 172.21.15.64:47038) 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea7a7fc640 1 -- 172.21.15.64:0/3458416896 learned_addr learned my addr 172.21.15.64:0/3458416896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a5068848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea63fff640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea640a2b90 0x7fea640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.425+0000 7fea7affd640 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a5068899b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7a7fc640 1 -- 172.21.15.64:0/3458416896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 msgr2=0x55a5068848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a5068848e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7a7fc640 1 -- 172.21.15.64:0/3458416896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 msgr2=0x55a5068899b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a5068899b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7a7fc640 1 -- 172.21.15.64:0/3458416896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5065c47e0 con 0x55a50688a020 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea640a2b90 0x7fea640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea63fff640 1 -- 172.21.15.64:0/2530212807 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea6c0ef890 con 0x7fea7c101880 2024-05-09T09:09:45.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 0x55a50688c4b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fea70009c90 tx=0x7fea70011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3035954fdae5cde0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea617fa640 1 -- 172.21.15.64:0/3458416896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7001e070 con 0x55a50688a020 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea617fa640 1 -- 172.21.15.64:0/3458416896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea70019ae0 con 0x55a50688a020 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea617fa640 1 -- 172.21.15.64:0/3458416896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7000ea40 con 0x55a50688a020 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.426+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea640a2b90 0x7fea640a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea74009fd0 tx=0x7fea74004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 -- 172.21.15.64:0/3458416896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 msgr2=0x55a50688c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 0x55a50688c4b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fea70009c90 tx=0x7fea70011810 comp rx=0 tx=0).stop 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 -- 172.21.15.64:0/3458416896 shutdown_connections 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50688a020 0x55a50688c4b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a5068899b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 --2- 172.21.15.64:0/3458416896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a5068848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 -- 172.21.15.64:0/3458416896 >> 172.21.15.64:0/3458416896 conn(0x55a50684c0a0 msgr2=0x55a50687ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 -- 172.21.15.64:0/3458416896 shutdown_connections 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 -- 172.21.15.64:0/3458416896 wait complete. 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.427+0000 7fea84262740 1 -- start start 2024-05-09T09:09:45.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 Processor -- start 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 0x55a50688e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a50688ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50688a020 0x55a506895ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a50688cfd0 con 0x55a5068844c0 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a5067a3b40 con 0x55a506884eb0 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a50688d150 con 0x55a50688a020 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea84262740 10 client.0 _ll_register_callbacks cb 0x55a5067980f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 0x55a50688e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 0x55a50688e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47046/0 (socket says 172.21.15.64:47046) 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 -- 172.21.15.64:0/1974746198 learned_addr learned my addr 172.21.15.64:0/1974746198 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50688a020 0x55a506895ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a50688ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 -- 172.21.15.64:0/1974746198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50688a020 msgr2=0x55a506895ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50688a020 0x55a506895ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 -- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 msgr2=0x55a50688ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a50688ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7affd640 1 -- 172.21.15.64:0/1974746198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a506624f70 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.428+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a50688ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea7affd640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 0x55a50688e920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fea6c000c00 tx=0x7fea6c0bb770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e221e1cdb45b27fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea5eff5640 1 -- 172.21.15.64:0/1974746198 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c049040 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea5eff5640 1 -- 172.21.15.64:0/1974746198 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea6c0b8480 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea70019ae0 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea5eff5640 1 -- 172.21.15.64:0/1974746198 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea6c003e20 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5064bc170 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea5eff5640 1 -- 172.21.15.64:0/1974746198 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea70019ae0 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea5eff5640 1 -- 172.21.15.64:0/1974746198 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fea6c03bae0 con 0x55a5068844c0 2024-05-09T09:09:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 20 client.15747 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.429+0000 7fea84262740 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50689a460 0x55a50689c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.430+0000 7fea84262740 4 client.15747 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.430+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5067f60c0 con 0x55a50689a460 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.430+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50689a460 0x55a50689c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.430+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50689a460 0x55a50689c920 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df125b54600e193b server_cookie=284fc002f64fb684 in_seq=0 out_seq=0 2024-05-09T09:09:45.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.430+0000 7fea5eff5640 10 client.15747 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea5eff5640 1 -- 172.21.15.64:0/1974746198 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5067f60c0 con 0x55a50689a460 2024-05-09T09:09:45.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea5eff5640 10 client.15747 handle_command_reply: tid=0 2024-05-09T09:09:45.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 client.15747 shutdown 2024-05-09T09:09:45.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 msgr2=0x55a50688e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 0x55a50688e920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fea6c000c00 tx=0x7fea6c0bb770 comp rx=0 tx=0).stop 2024-05-09T09:09:45.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 shutdown_connections 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50689a460 0x55a50689c920 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50688a020 0x55a506895ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea5eff5640 0 client.15747 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a50688ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 --2- 172.21.15.64:0/1974746198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5068844c0 0x55a50688e920 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 >> 172.21.15.64:0/1974746198 conn(0x55a50684c0a0 msgr2=0x55a50687f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 shutdown_connections 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 1 -- 172.21.15.64:0/1974746198 wait complete. 2024-05-09T09:09:45.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.431+0000 7fea84262740 20 client.15747 trim_cache size 0 max 16384 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.454+0000 7fea84262740 1 Processor -- start 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 -- start start 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50684c0a0 0x55a50687ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5068844c0 0x55a50687f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 0x55a506895e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1accb0 con 0x55a506884eb0 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1aa580 con 0x55a5068844c0 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea84262740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1a83d0 con 0x55a50684c0a0 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 0x55a506895e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 0x55a506895e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47060/0 (socket says 172.21.15.64:47060) 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 -- 172.21.15.64:0/1941260000 learned_addr learned my addr 172.21.15.64:0/1941260000 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5068844c0 0x55a50687f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7affd640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50684c0a0 0x55a50687ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 -- 172.21.15.64:0/1941260000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50684c0a0 msgr2=0x55a50687ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50684c0a0 0x55a50687ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 -- 172.21.15.64:0/1941260000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5068844c0 msgr2=0x55a50687f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5068844c0 0x55a50687f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7b7fe640 1 -- 172.21.15.64:0/1941260000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50687e470 con 0x55a506884eb0 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5068844c0 0x55a50687f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.455+0000 7fea7affd640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50684c0a0 0x55a50687ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.456+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 0x55a506895e60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fea74013b30 tx=0x7fea74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3efd2667c00af03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.456+0000 7fea60ff9640 1 -- 172.21.15.64:0/1941260000 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74011ac0 con 0x55a506884eb0 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.456+0000 7fea60ff9640 1 -- 172.21.15.64:0/1941260000 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea74003e20 con 0x55a506884eb0 2024-05-09T09:09:45.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.456+0000 7fea60ff9640 1 -- 172.21.15.64:0/1941260000 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74004760 con 0x55a506884eb0 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 -- 172.21.15.64:0/1941260000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 msgr2=0x55a506895e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 0x55a506895e60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fea74013b30 tx=0x7fea74016000 comp rx=0 tx=0).stop 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 -- 172.21.15.64:0/1941260000 shutdown_connections 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a506884eb0 0x55a506895e60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5068844c0 0x55a50687f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 --2- 172.21.15.64:0/1941260000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a50684c0a0 0x55a50687ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 -- 172.21.15.64:0/1941260000 >> 172.21.15.64:0/1941260000 conn(0x55a506885bc0 msgr2=0x55a50687e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 -- 172.21.15.64:0/1941260000 shutdown_connections 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 -- 172.21.15.64:0/1941260000 wait complete. 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 -- start start 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.457+0000 7fea84262740 1 Processor -- start 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 0x55a506895830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a506895d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a506893e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1a8a40 con 0x55a50684c0a0 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1a9eb0 con 0x55a506884eb0 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1a9110 con 0x55a5068844c0 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea84262740 10 client.0 _ll_register_callbacks cb 0x7fea6c0bbcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 0x55a506895830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 0x55a506895830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47070/0 (socket says 172.21.15.64:47070) 2024-05-09T09:09:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 -- 172.21.15.64:0/2546605720 learned_addr learned my addr 172.21.15.64:0/2546605720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7b7fe640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a506893e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a506895d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 -- 172.21.15.64:0/2546605720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 msgr2=0x55a506895d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a506895d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 -- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 msgr2=0x55a506893e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a506893e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 -- 172.21.15.64:0/2546605720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a506894670 con 0x55a50684c0a0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea7affd640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 0x55a506895830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0ba0c0 tx=0x7fea6c0bc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153ba057a6497739 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.458+0000 7fea5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea5eff5640 1 -- 172.21.15.64:0/2546605720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c049070 con 0x55a50684c0a0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea5eff5640 1 -- 172.21.15.64:0/2546605720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea6c0c4740 con 0x55a50684c0a0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea74003e20 con 0x55a50684c0a0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea5eff5640 1 -- 172.21.15.64:0/2546605720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea6c0bbe10 con 0x55a50684c0a0 2024-05-09T09:09:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 10 client.15753 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a506901670 con 0x55a50684c0a0 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea5eff5640 1 -- 172.21.15.64:0/2546605720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea74003e20 con 0x55a50684c0a0 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea5eff5640 1 -- 172.21.15.64:0/2546605720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fea6c0bacd0 con 0x55a50684c0a0 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 10 client.15753 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 20 client.15753 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50688a020 0x55a506903f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 4 client.15753 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a5067f0410 con 0x55a50688a020 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.459+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50688a020 0x55a506903f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.460+0000 7fea5eff5640 10 client.15753 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.460+0000 7fea7a7fc640 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50688a020 0x55a506903f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1266c473e83c7a0f server_cookie=61ee1bd827154122 in_seq=0 out_seq=0 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea5eff5640 1 -- 172.21.15.64:0/2546605720 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a5067f0410 con 0x55a50688a020 2024-05-09T09:09:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea5eff5640 10 client.15753 handle_command_reply: tid=0 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 client.15753 shutdown 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 msgr2=0x55a506895830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 0x55a506895830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0ba0c0 tx=0x7fea6c0bc070 comp rx=0 tx=0).stop 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 shutdown_connections 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a50688a020 0x55a506903f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a506884eb0 0x55a506893e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea5eff5640 0 client.15753 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5068844c0 0x55a506895d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 --2- 172.21.15.64:0/2546605720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a50684c0a0 0x55a506895830 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 >> 172.21.15.64:0/2546605720 conn(0x55a506885bc0 msgr2=0x55a50687f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.461+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 shutdown_connections 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea84262740 1 -- 172.21.15.64:0/2546605720 wait complete. 2024-05-09T09:09:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea84262740 20 client.15753 trim_cache size 0 max 16384 2024-05-09T09:09:45.466 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:09:45.466 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:09:45.466 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "9a3458c9-3bc9-416e-93fc-2070bcef57fb", 2024-05-09T09:09:45.466 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:09:45.466 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea640a2b90 msgr2=0x7fea640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea640a2b90 0x7fea640a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea74009fd0 tx=0x7fea74004a40 comp rx=0 tx=0).stop 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 msgr2=0x7fea7c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c197b10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0314e0 tx=0x7fea6c031a30 comp rx=0 tx=0).stop 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 shutdown_connections 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea640a2b90 0x7fea640a5050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c103430 0x7fea7c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7c102a70 0x7fea7c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 --2- 172.21.15.64:0/2530212807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea7c101880 0x7fea7c197b10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 >> 172.21.15.64:0/2530212807 conn(0x7fea7c0fce10 msgr2=0x7fea7c105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 shutdown_connections 2024-05-09T09:09:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.462+0000 7fea8155a640 1 -- 172.21.15.64:0/2530212807 wait complete. 2024-05-09T09:09:45.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:09:45.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:09:45.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 -- 172.21.15.64:0/2153658902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264101870 msgr2=0x7f8264101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 --2- 172.21.15.64:0/2153658902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264101870 0x7f8264101c90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8258009900 tx=0x7f825802a410 comp rx=0 tx=0).stop 2024-05-09T09:09:45.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 -- 172.21.15.64:0/2153658902 shutdown_connections 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 --2- 172.21.15.64:0/2153658902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8264103420 0x7f8264109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 --2- 172.21.15.64:0/2153658902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8264102a60 0x7f8264102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 --2- 172.21.15.64:0/2153658902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264101870 0x7f8264101c90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 -- 172.21.15.64:0/2153658902 >> 172.21.15.64:0/2153658902 conn(0x7f82640fcde0 msgr2=0x7f82640ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 -- 172.21.15.64:0/2153658902 shutdown_connections 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.622+0000 7f826afa5640 1 -- 172.21.15.64:0/2153658902 wait complete. 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 Processor -- start 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 -- start start 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8264101870 0x7f8264197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 0x7f8264198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8264103420 0x7f826419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f826410d390 con 0x7f8264102a60 2024-05-09T09:09:45.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f826410d210 con 0x7f8264103420 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f826afa5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f826410d510 con 0x7f8264101870 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f8263fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 0x7f8264198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f8268d41640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8264101870 0x7f8264197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f8263fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 0x7f8264198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47082/0 (socket says 172.21.15.64:47082) 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f8263fff640 1 -- 172.21.15.64:0/2694392412 learned_addr learned my addr 172.21.15.64:0/2694392412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.623+0000 7f8269542640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8264103420 0x7f826419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8263fff640 1 -- 172.21.15.64:0/2694392412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8264101870 msgr2=0x7f8264197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8263fff640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8264101870 0x7f8264197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8263fff640 1 -- 172.21.15.64:0/2694392412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8264103420 msgr2=0x7f826419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8263fff640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8264103420 0x7f826419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8263fff640 1 -- 172.21.15.64:0/2694392412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826419f740 con 0x7f8264102a60 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8263fff640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 0x7f8264198060 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f825000e9f0 tx=0x7f825000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8261ffb640 1 -- 172.21.15.64:0/2694392412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825000cd80 con 0x7f8264102a60 2024-05-09T09:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8261ffb640 1 -- 172.21.15.64:0/2694392412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82500054d0 con 0x7f8264102a60 2024-05-09T09:09:45.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826419fa30 con 0x7f8264102a60 2024-05-09T09:09:45.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f8261ffb640 1 -- 172.21.15.64:0/2694392412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250010490 con 0x7f8264102a60 2024-05-09T09:09:45.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.624+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826419ff30 con 0x7f8264102a60 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f8261ffb640 1 -- 172.21.15.64:0/2694392412 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8250005020 con 0x7f8264102a60 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 Processor -- start 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 -- start start 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f8261ffb640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82540a2b90 0x7f82540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b9051f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90524830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 0x557b90527330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557b9047fd00 con 0x557b90524ea0 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557b904274c0 con 0x557b9051f340 2024-05-09T09:09:45.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.627+0000 7f826dd8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557b9047bcc0 con 0x557b9051fd30 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8269542640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82540a2b90 0x7f82540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8261ffb640 1 -- 172.21.15.64:0/2694392412 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f82500cf730 con 0x7f8264102a60 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8269542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 0x557b90527330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90524830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8268d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b9051f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90524830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43038/0 (socket says 172.21.15.64:43038) 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 -- 172.21.15.64:0/1435822099 learned_addr learned my addr 172.21.15.64:0/1435822099 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8269542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 0x557b90527330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47096/0 (socket says 172.21.15.64:47096) 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 -- 172.21.15.64:0/1435822099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 msgr2=0x557b9051f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b9051f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 -- 172.21.15.64:0/1435822099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 msgr2=0x557b90527330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 0x557b90527330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 -- 172.21.15.64:0/1435822099 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b9025d7d0 con 0x557b9051fd30 2024-05-09T09:09:45.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8268d41640 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b9051f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8269542640 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 0x557b90527330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8269542640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82540a2b90 0x7f82540a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f825c0098b0 tx=0x7f825c009e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.628+0000 7f8263fff640 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90524830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8250098060 tx=0x7f825009b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef23e9a0812adc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.629+0000 7f824effd640 1 -- 172.21.15.64:0/1435822099 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825001fd70 con 0x557b9051fd30 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.629+0000 7f824effd640 1 -- 172.21.15.64:0/1435822099 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82500b2070 con 0x557b9051fd30 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.629+0000 7f824effd640 1 -- 172.21.15.64:0/1435822099 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825009ffb0 con 0x557b9051fd30 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 -- 172.21.15.64:0/1435822099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 msgr2=0x557b90524830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90524830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8250098060 tx=0x7f825009b620 comp rx=0 tx=0).stop 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 -- 172.21.15.64:0/1435822099 shutdown_connections 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b90524ea0 0x557b90527330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90524830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1435822099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b9051f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 -- 172.21.15.64:0/1435822099 >> 172.21.15.64:0/1435822099 conn(0x557b905165a0 msgr2=0x557b905169a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 -- 172.21.15.64:0/1435822099 shutdown_connections 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 -- 172.21.15.64:0/1435822099 wait complete. 2024-05-09T09:09:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 -- start start 2024-05-09T09:09:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.630+0000 7f826dd8c740 1 Processor -- start 2024-05-09T09:09:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b905296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 0x557b90529bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b90530c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557b90527df0 con 0x557b9051fd30 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557b9043ab90 con 0x557b9051f340 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557b90527f70 con 0x557b90524ea0 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f826dd8c740 10 client.0 _ll_register_callbacks cb 0x557b90476da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 0x557b90529bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8268d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b905296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 0x557b90529bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47098/0 (socket says 172.21.15.64:47098) 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 -- 172.21.15.64:0/2520266784 learned_addr learned my addr 172.21.15.64:0/2520266784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8269542640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b90530c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 -- 172.21.15.64:0/2520266784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 msgr2=0x557b90530c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b90530c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 -- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 msgr2=0x557b905296b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b905296b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 -- 172.21.15.64:0/2520266784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b901933f0 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8268d41640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b905296b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8263fff640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 0x557b90529bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f825000c6a0 tx=0x7f8250009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a3890fa66af0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.631+0000 7f8243fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f8243fff640 1 -- 172.21.15.64:0/2520266784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250005680 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f8243fff640 1 -- 172.21.15.64:0/2520266784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8250099f00 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82500b2070 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f8243fff640 1 -- 172.21.15.64:0/2520266784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f825008d050 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b901530f0 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f8243fff640 1 -- 172.21.15.64:0/2520266784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82500b2070 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f8243fff640 1 -- 172.21.15.64:0/2520266784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8250088ce0 con 0x557b9051fd30 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 20 client.15765 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b90535120 0x557b90537510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 4 client.15765 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.632+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b9048d430 con 0x557b90535120 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.633+0000 7f8268d41640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b90535120 0x557b90537510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.633+0000 7f8268d41640 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b90535120 0x557b90537510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c9d8aeae3db817 server_cookie=a0bd05bfb09759ec in_seq=0 out_seq=0 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.633+0000 7f8243fff640 10 client.15765 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f8243fff640 1 -- 172.21.15.64:0/2520266784 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b9048d430 con 0x557b90535120 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f8243fff640 10 client.15765 handle_command_reply: tid=0 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 client.15765 shutdown 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 msgr2=0x557b90529bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 0x557b90529bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f825000c6a0 tx=0x7f8250009740 comp rx=0 tx=0).stop 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 shutdown_connections 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b90535120 0x557b90537510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b90530c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051fd30 0x557b90529bf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2520266784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051f340 0x557b905296b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 >> 172.21.15.64:0/2520266784 conn(0x557b905165a0 msgr2=0x557b905157e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f8243fff640 0 client.15765 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 shutdown_connections 2024-05-09T09:09:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 1 -- 172.21.15.64:0/2520266784 wait complete. 2024-05-09T09:09:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.634+0000 7f826dd8c740 20 client.15765 trim_cache size 0 max 16384 2024-05-09T09:09:45.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 Processor -- start 2024-05-09T09:09:45.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 -- start start 2024-05-09T09:09:45.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b90515660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90530bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b90524ea0 0x557b9053bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82641aaa90 con 0x557b9051f340 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f82641a8360 con 0x557b90524ea0 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.662+0000 7f826dd8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f82641a61b0 con 0x557b9051fd30 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b90515660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b90515660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47112/0 (socket says 172.21.15.64:47112) 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 -- 172.21.15.64:0/2833258778 learned_addr learned my addr 172.21.15.64:0/2833258778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8269542640 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b90524ea0 0x557b9053bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8263fff640 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90530bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 -- 172.21.15.64:0/2833258778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 msgr2=0x557b90530bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90530bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 -- 172.21.15.64:0/2833258778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b90524ea0 msgr2=0x557b9053bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b90524ea0 0x557b9053bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 -- 172.21.15.64:0/2833258778 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b90520110 con 0x557b9051f340 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f8268d41640 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b90515660 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8258009900 tx=0x7f8258031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8163aebf693e9079 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f824e7fc640 1 -- 172.21.15.64:0/2833258778 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258035040 con 0x557b9051f340 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f824e7fc640 1 -- 172.21.15.64:0/2833258778 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8258046870 con 0x557b9051f340 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.663+0000 7f824e7fc640 1 -- 172.21.15.64:0/2833258778 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258003b90 con 0x557b9051f340 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 -- 172.21.15.64:0/2833258778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 msgr2=0x557b90515660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b90515660 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8258009900 tx=0x7f8258031980 comp rx=0 tx=0).stop 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 -- 172.21.15.64:0/2833258778 shutdown_connections 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b90524ea0 0x557b9053bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b9051fd30 0x557b90530bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 --2- 172.21.15.64:0/2833258778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b90515660 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 -- 172.21.15.64:0/2833258778 >> 172.21.15.64:0/2833258778 conn(0x557b905165a0 msgr2=0x557b90514f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 -- 172.21.15.64:0/2833258778 shutdown_connections 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.664+0000 7f826dd8c740 1 -- 172.21.15.64:0/2833258778 wait complete. 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 -- start start 2024-05-09T09:09:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 Processor -- start 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b9052e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051fd30 0x557b9052e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b9052f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82641a6820 con 0x557b9051f340 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f82641a7c90 con 0x557b9051fd30 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f82641a6ef0 con 0x557b90524ea0 2024-05-09T09:09:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f826dd8c740 10 client.0 _ll_register_callbacks cb 0x7f825c012770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b9052e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b9052e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47120/0 (socket says 172.21.15.64:47120) 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 -- 172.21.15.64:0/1027605901 learned_addr learned my addr 172.21.15.64:0/1027605901 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8269542640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b9052f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8263fff640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051fd30 0x557b9052e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 -- 172.21.15.64:0/1027605901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 msgr2=0x557b9052f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b9052f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 -- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051fd30 msgr2=0x557b9052e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.665+0000 7f8268d41640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051fd30 0x557b9052e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8268d41640 1 -- 172.21.15.64:0/1027605901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b9052ff70 con 0x557b9051f340 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8269542640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b9052f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8263fff640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051fd30 0x557b9052e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8268d41640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b9052e430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f825802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafb606d0f046315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8243fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8243fff640 1 -- 172.21.15.64:0/1027605901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258054070 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8243fff640 1 -- 172.21.15.64:0/1027605901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8258059460 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8258046870 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8243fff640 1 -- 172.21.15.64:0/1027605901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8258062050 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f826dd8c740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b9059b650 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.666+0000 7f8243fff640 1 -- 172.21.15.64:0/1027605901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8258046870 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f8243fff640 1 -- 172.21.15.64:0/1027605901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f825803c6d0 con 0x557b9051f340 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f826dd8c740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f826dd8c740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f826dd8c740 20 client.15771 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b9059c080 0x557b9059e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f826dd8c740 4 client.15771 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b90487780 con 0x557b9059c080 2024-05-09T09:09:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f8263fff640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b9059c080 0x557b9059e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f8243fff640 10 client.15771 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.667+0000 7f8263fff640 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b9059c080 0x557b9059e4a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9ce66c6545d77e server_cookie=284e2856a53c9925 in_seq=0 out_seq=0 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f8243fff640 1 -- 172.21.15.64:0/1027605901 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557b90487780 con 0x557b9059c080 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f8243fff640 10 client.15771 handle_command_reply: tid=0 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 client.15771 shutdown 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 msgr2=0x557b9052e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b9052e430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f825802ab90 comp rx=0 tx=0).stop 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 shutdown_connections 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557b9059c080 0x557b9059e4a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557b90524ea0 0x557b9052f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b9051fd30 0x557b9052e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f8243fff640 0 client.15771 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 --2- 172.21.15.64:0/1027605901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b9051f340 0x557b9052e430 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 >> 172.21.15.64:0/1027605901 conn(0x557b905165a0 msgr2=0x557b90519de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 shutdown_connections 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 1 -- 172.21.15.64:0/1027605901 wait complete. 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.668+0000 7f826dd8c740 20 client.15771 trim_cache size 0 max 16384 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: "9a3458c9-3bc9-416e-93fc-2070bcef57fb": { 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "9a3458c9-3bc9-416e-93fc-2070bcef57fb", 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:09:45.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:09:45.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:09:45.674 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:09:45.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82540a2b90 msgr2=0x7f82540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82540a2b90 0x7f82540a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f825c0098b0 tx=0x7f825c009e20 comp rx=0 tx=0).stop 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 msgr2=0x7f8264198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 0x7f8264198060 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f825000e9f0 tx=0x7f825000edf0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 shutdown_connections 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82540a2b90 0x7f82540a5050 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8264103420 0x7f826419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8264102a60 0x7f8264198060 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 --2- 172.21.15.64:0/2694392412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8264101870 0x7f8264197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 >> 172.21.15.64:0/2694392412 conn(0x7f82640fcde0 msgr2=0x7f8264105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 shutdown_connections 2024-05-09T09:09:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:09:45.669+0000 7f826afa5640 1 -- 172.21.15.64:0/2694392412 wait complete. 2024-05-09T09:09:45.681 INFO:tasks.cephfs.filesystem:scrub status for tag:9a3458c9-3bc9-416e-93fc-2070bcef57fb - {'path': '/', 'tag': '9a3458c9-3bc9-416e-93fc-2070bcef57fb', 'options': 'recursive,force'} 2024-05-09T09:09:45.966 INFO:teuthology.orchestra.run.smithi064.stderr:1400000 of 3200000 tuples (43%) done (elapsed 14.64 s, remaining 18.82 s) 2024-05-09T09:09:46.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: pgmap v389: 129 pgs: 129 active+clean; 278 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 38 MiB/s wr, 51 op/s 2024-05-09T09:09:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:09:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:09:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:09:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:46 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: pgmap v389: 129 pgs: 129 active+clean; 278 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 38 MiB/s wr, 51 op/s 2024-05-09T09:09:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:09:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:09:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:09:46.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:46 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: pgmap v389: 129 pgs: 129 active+clean; 278 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 38 MiB/s wr, 51 op/s 2024-05-09T09:09:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:09:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:09:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:09:46.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:46 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:46.802 INFO:teuthology.orchestra.run.smithi064.stderr:1500000 of 3200000 tuples (46%) done (elapsed 15.47 s, remaining 17.53 s) 2024-05-09T09:09:47.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:47 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:09:47.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:47 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:47.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:47 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:09:47.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:47 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:47.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:47 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:09:47.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:47 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:09:48.001 INFO:teuthology.orchestra.run.smithi064.stderr:1600000 of 3200000 tuples (50%) done (elapsed 16.67 s, remaining 16.67 s) 2024-05-09T09:09:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:48 smithi179 ceph-mon[34086]: pgmap v390: 129 pgs: 129 active+clean; 298 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 38 MiB/s wr, 56 op/s 2024-05-09T09:09:48.552 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:48 smithi064 ceph-mon[30149]: pgmap v390: 129 pgs: 129 active+clean; 298 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 38 MiB/s wr, 56 op/s 2024-05-09T09:09:48.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:48 smithi139 ceph-mon[33599]: pgmap v390: 129 pgs: 129 active+clean; 298 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 38 MiB/s wr, 56 op/s 2024-05-09T09:09:48.751 INFO:teuthology.orchestra.run.smithi064.stderr:1700000 of 3200000 tuples (53%) done (elapsed 17.42 s, remaining 15.37 s) 2024-05-09T09:09:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:1800000 of 3200000 tuples (56%) done (elapsed 18.67 s, remaining 14.52 s) 2024-05-09T09:09:50.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:50 smithi179 ceph-mon[34086]: pgmap v391: 129 pgs: 129 active+clean; 400 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 51 MiB/s wr, 121 op/s 2024-05-09T09:09:50.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:50 smithi139 ceph-mon[33599]: pgmap v391: 129 pgs: 129 active+clean; 400 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 51 MiB/s wr, 121 op/s 2024-05-09T09:09:50.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:50 smithi064 ceph-mon[30149]: pgmap v391: 129 pgs: 129 active+clean; 400 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 51 MiB/s wr, 121 op/s 2024-05-09T09:09:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:1900000 of 3200000 tuples (59%) done (elapsed 19.90 s, remaining 13.61 s) 2024-05-09T09:09:52.084 INFO:teuthology.orchestra.run.smithi064.stderr:2000000 of 3200000 tuples (62%) done (elapsed 20.75 s, remaining 12.45 s) 2024-05-09T09:09:52.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:52 smithi179 ceph-mon[34086]: pgmap v392: 129 pgs: 129 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 40 MiB/s wr, 120 op/s 2024-05-09T09:09:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 -- 172.21.15.64:0/400124772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4104cc0 msgr2=0x7ff6f41070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/400124772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4104cc0 0x7ff6f41070b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009900 tx=0x7ff6e802a410 comp rx=0 tx=0).stop 2024-05-09T09:09:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 -- 172.21.15.64:0/400124772 shutdown_connections 2024-05-09T09:09:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/400124772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 0x7ff6f41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/400124772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4104cc0 0x7ff6f41070b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/400124772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4102390 0x7ff6f4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 -- 172.21.15.64:0/400124772 >> 172.21.15.64:0/400124772 conn(0x7ff6f40fbd00 msgr2=0x7ff6f40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 -- 172.21.15.64:0/400124772 shutdown_connections 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 -- 172.21.15.64:0/400124772 wait complete. 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.547+0000 7ff6f9eee640 1 Processor -- start 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 -- start start 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 0x7ff6f4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4104cc0 0x7ff6f4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 0x7ff6f419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f410d390 con 0x7ff6f4102390 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f410d210 con 0x7ff6f41075f0 2024-05-09T09:09:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f9eee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f410d510 con 0x7ff6f4104cc0 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 0x7ff6f4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 0x7ff6f4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47136/0 (socket says 172.21.15.64:47136) 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3193839899 learned_addr learned my addr 172.21.15.64:0/3193839899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4104cc0 0x7ff6f4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 0x7ff6f419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3193839899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4104cc0 msgr2=0x7ff6f4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4104cc0 0x7ff6f4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3193839899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 msgr2=0x7ff6f419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 0x7ff6f419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3193839899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f419f660 con 0x7ff6f4102390 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4104cc0 0x7ff6f4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.548+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 0x7ff6f419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.549+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 0x7ff6f4197a40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400d950 tx=0x7ff6e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.549+0000 7ff6f0ff9640 1 -- 172.21.15.64:0/3193839899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4017070 con 0x7ff6f4102390 2024-05-09T09:09:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.549+0000 7ff6f0ff9640 1 -- 172.21.15.64:0/3193839899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e4009c30 con 0x7ff6f4102390 2024-05-09T09:09:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.549+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f419f950 con 0x7ff6f4102390 2024-05-09T09:09:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.549+0000 7ff6f0ff9640 1 -- 172.21.15.64:0/3193839899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400f620 con 0x7ff6f4102390 2024-05-09T09:09:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.549+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f419fe70 con 0x7ff6f4102390 2024-05-09T09:09:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 Processor -- start 2024-05-09T09:09:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 -- start start 2024-05-09T09:09:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f0ff9640 1 -- 172.21.15.64:0/3193839899 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff6e40109a0 con 0x7ff6f4102390 2024-05-09T09:09:52.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134250 0x55c824134670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 0x55c82413c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c824098340 con 0x55c824139db0 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c824040290 con 0x55c824134250 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c824094300 con 0x55c824134c40 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 0x55c82413c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 0x55c82413c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47148/0 (socket says 172.21.15.64:47148) 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/613209673 learned_addr learned my addr 172.21.15.64:0/613209673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134250 0x55c824134670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/613209673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 msgr2=0x55c824139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824139740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/613209673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134250 msgr2=0x55c824134670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134250 0x55c824134670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/613209673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8240648e0 con 0x55c824139db0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.552+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134250 0x55c824134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 0x55c82413c240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80314e0 tx=0x7ff6e8032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cee064e2a085d85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6f0ff9640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6dc0a2c30 0x7ff6dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6d9ffb640 1 -- 172.21.15.64:0/613209673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e803f070 con 0x55c824139db0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6dc0a2c30 0x7ff6dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6d9ffb640 1 -- 172.21.15.64:0/613209673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e8031780 con 0x55c824139db0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6f0ff9640 1 -- 172.21.15.64:0/3193839899 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6e40cf0d0 con 0x7ff6f4102390 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.553+0000 7ff6d9ffb640 1 -- 172.21.15.64:0/613209673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8035070 con 0x55c824139db0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/613209673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 msgr2=0x55c82413c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6dc0a2c30 0x7ff6dc0a50f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c8241354a0 tx=0x7ff6ec0099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 0x55c82413c240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80314e0 tx=0x7ff6e8032c60 comp rx=0 tx=0).stop 2024-05-09T09:09:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/613209673 shutdown_connections 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824139db0 0x55c82413c240 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/613209673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134250 0x55c824134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/613209673 >> 172.21.15.64:0/613209673 conn(0x55c82412f8c0 msgr2=0x55c82412fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/613209673 shutdown_connections 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/613209673 wait complete. 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 -- start start 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.554+0000 7ff6fcbfb740 1 Processor -- start 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134c40 0x55c82413e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824139db0 0x55c8241459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c82413cd00 con 0x55c824134250 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c824053790 con 0x55c824134c40 2024-05-09T09:09:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c82413ce80 con 0x55c824139db0 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6fcbfb740 10 client.0 _ll_register_callbacks cb 0x55c82408ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47154/0 (socket says 172.21.15.64:47154) 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 -- 172.21.15.64:0/4261085910 learned_addr learned my addr 172.21.15.64:0/4261085910 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f3fff640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824139db0 0x55c8241459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134c40 0x55c82413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 -- 172.21.15.64:0/4261085910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824139db0 msgr2=0x55c8241459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824139db0 0x55c8241459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 -- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134c40 msgr2=0x55c82413e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134c40 0x55c82413e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 -- 172.21.15.64:0/4261085910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c823dac210 con 0x55c824134250 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134c40 0x55c82413e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413e430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4097a00 tx=0x7ff6e40d27a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104fdcd195155b71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.555+0000 7ff6d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/4261085910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4020070 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/4261085910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6e40d2d60 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e8031780 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/4261085910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6e409af00 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6fcbfb740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c823e78620 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/4261085910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e8031780 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/4261085910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6e4097c10 con 0x55c824134250 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6fcbfb740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6fcbfb740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.556+0000 7ff6fcbfb740 20 client.15783 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.557+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c824149f50 0x55c82414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.557+0000 7ff6fcbfb740 4 client.15783 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.557+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8240a5a70 con 0x55c824149f50 2024-05-09T09:09:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.557+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c824149f50 0x55c82414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.557+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c824149f50 0x55c82414c3a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa55750af79b3217 server_cookie=b25bf0931ab187a2 in_seq=0 out_seq=0 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.557+0000 7ff6d2ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/4261085910 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8240a5a70 con 0x55c824149f50 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6d2ffd640 10 client.15783 handle_command_reply: tid=0 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 client.15783 shutdown 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 msgr2=0x55c82413e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413e430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4097a00 tx=0x7ff6e40d27a0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 shutdown_connections 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c824149f50 0x55c82414c3a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824139db0 0x55c8241459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824134c40 0x55c82413e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6d2ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/4261085910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413e430 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 >> 172.21.15.64:0/4261085910 conn(0x55c82412f8c0 msgr2=0x55c82412ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 shutdown_connections 2024-05-09T09:09:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/4261085910 wait complete. 2024-05-09T09:09:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.558+0000 7ff6fcbfb740 20 client.15783 trim_cache size 0 max 16384 2024-05-09T09:09:52.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:52 smithi139 ceph-mon[33599]: pgmap v392: 129 pgs: 129 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 40 MiB/s wr, 120 op/s 2024-05-09T09:09:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 Processor -- start 2024-05-09T09:09:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 -- start start 2024-05-09T09:09:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c8241521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41acb50 con 0x55c824134250 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41aa420 con 0x55c824139db0 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41a8270 con 0x55c824134c40 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47156/0 (socket says 172.21.15.64:47156) 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.579+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3992177455 learned_addr learned my addr 172.21.15.64:0/3992177455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3992177455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 msgr2=0x55c824151cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824151cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3992177455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 msgr2=0x55c8241521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c8241521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6f37fe640 1 -- 172.21.15.64:0/3992177455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c824135020 con 0x55c824134250 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413d220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40a3ce0 tx=0x7ff6e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ced52df4e02b26e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6d97fa640 1 -- 172.21.15.64:0/3992177455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4020070 con 0x55c824134250 2024-05-09T09:09:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6d97fa640 1 -- 172.21.15.64:0/3992177455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e4009de0 con 0x55c824134250 2024-05-09T09:09:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.580+0000 7ff6d97fa640 1 -- 172.21.15.64:0/3992177455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4097c10 con 0x55c824134250 2024-05-09T09:09:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/3992177455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 msgr2=0x55c82413d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413d220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40a3ce0 tx=0x7ff6e4000bc0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/3992177455 shutdown_connections 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c8241521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134c40 0x55c824151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/3992177455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134250 0x55c82413d220 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/3992177455 >> 172.21.15.64:0/3992177455 conn(0x55c82412f8c0 msgr2=0x55c82412fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/3992177455 shutdown_connections 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/3992177455 wait complete. 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.581+0000 7ff6fcbfb740 1 -- start start 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 Processor -- start 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134250 0x55c824142960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 0x55c824142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c824143e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41a88e0 con 0x55c824134c40 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41a9d50 con 0x55c824139db0 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41a8fb0 con 0x55c824134250 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6fcbfb740 10 client.0 _ll_register_callbacks cb 0x7ff6e400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 0x55c824142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 0x55c824142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47162/0 (socket says 172.21.15.64:47162) 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/215135318 learned_addr learned my addr 172.21.15.64:0/215135318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c824143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134250 0x55c824142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/215135318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134250 msgr2=0x55c824142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134250 0x55c824142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 msgr2=0x55c824143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c824143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f2ffd640 1 -- 172.21.15.64:0/215135318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c824144450 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.582+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134250 0x55c824142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6f2ffd640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 0x55c824142ea0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80314e0 tx=0x7ff6e80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60180a75944ed147 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/215135318 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8053060 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/215135318 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff6e8004240 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e4009de0 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/215135318 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6e805a050 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8241b0910 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/215135318 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e4009de0 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/215135318 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6e804b070 con 0x55c824134c40 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 20 client.15789 populate_metadata read hostname 'smithi064' 2024-05-09T09:09:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c8241b0f70 0x55c8241b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:09:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 4 client.15789 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:09:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.583+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c82409fdc0 con 0x55c8241b0f70 2024-05-09T09:09:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.584+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c8241b0f70 0x55c8241b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:09:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.584+0000 7ff6f37fe640 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c8241b0f70 0x55c8241b3360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd54063d59817b9 server_cookie=984a48b888b1d696 in_seq=0 out_seq=0 2024-05-09T09:09:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.584+0000 7ff6d2ffd640 10 client.15789 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6d2ffd640 1 -- 172.21.15.64:0/215135318 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c82409fdc0 con 0x55c8241b0f70 2024-05-09T09:09:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6d2ffd640 10 client.15789 handle_command_reply: tid=0 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 client.15789 shutdown 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 msgr2=0x55c824142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 0x55c824142ea0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80314e0 tx=0x7ff6e80058d0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 shutdown_connections 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c8241b0f70 0x55c8241b3360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c824139db0 0x55c824143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c824134c40 0x55c824142ea0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 --2- 172.21.15.64:0/215135318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c824134250 0x55c824142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 >> 172.21.15.64:0/215135318 conn(0x55c82412f8c0 msgr2=0x55c82413d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6d2ffd640 0 client.15789 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.597+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 shutdown_connections 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6fcbfb740 1 -- 172.21.15.64:0/215135318 wait complete. 2024-05-09T09:09:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6fcbfb740 20 client.15789 trim_cache size 0 max 16384 2024-05-09T09:09:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:09:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:09:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:09:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6dc0a2c30 msgr2=0x7ff6dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6dc0a2c30 0x7ff6dc0a50f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c8241354a0 tx=0x7ff6ec0099e0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 msgr2=0x7ff6f4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 0x7ff6f4197a40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400d950 tx=0x7ff6e400de10 comp rx=0 tx=0).stop 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 shutdown_connections 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff6dc0a2c30 0x7ff6dc0a50f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6f41075f0 0x7ff6f419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6f4104cc0 0x7ff6f4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 --2- 172.21.15.64:0/3193839899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4102390 0x7ff6f4197a40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 >> 172.21.15.64:0/3193839899 conn(0x7ff6f40fbd00 msgr2=0x7ff6f4102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:09:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 shutdown_connections 2024-05-09T09:09:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:09:52.598+0000 7ff6f9eee640 1 -- 172.21.15.64:0/3193839899 wait complete. 2024-05-09T09:09:52.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:52 smithi064 ceph-mon[30149]: pgmap v392: 129 pgs: 129 active+clean; 420 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 40 MiB/s wr, 120 op/s 2024-05-09T09:09:53.250 INFO:teuthology.orchestra.run.smithi064.stderr:2100000 of 3200000 tuples (65%) done (elapsed 21.92 s, remaining 11.48 s) 2024-05-09T09:09:54.437 INFO:teuthology.orchestra.run.smithi064.stderr:2200000 of 3200000 tuples (68%) done (elapsed 23.11 s, remaining 10.50 s) 2024-05-09T09:09:54.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:54 smithi179 ceph-mon[34086]: pgmap v393: 129 pgs: 129 active+clean; 462 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 43 MiB/s wr, 119 op/s 2024-05-09T09:09:54.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:54 smithi139 ceph-mon[33599]: pgmap v393: 129 pgs: 129 active+clean; 462 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 43 MiB/s wr, 119 op/s 2024-05-09T09:09:54.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:54 smithi064 ceph-mon[30149]: pgmap v393: 129 pgs: 129 active+clean; 462 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 43 MiB/s wr, 119 op/s 2024-05-09T09:09:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2300000 of 3200000 tuples (71%) done (elapsed 23.94 s, remaining 9.37 s) 2024-05-09T09:09:55.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:55 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:09:55.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:55 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:09:55.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:55 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:09:56.479 INFO:teuthology.orchestra.run.smithi064.stderr:2400000 of 3200000 tuples (75%) done (elapsed 25.15 s, remaining 8.38 s) 2024-05-09T09:09:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:56 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:09:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:56 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:09:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:56 smithi179 ceph-mon[34086]: pgmap v394: 129 pgs: 129 active+clean; 502 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 44 MiB/s wr, 114 op/s 2024-05-09T09:09:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:56 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:09:56.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:56 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:09:56.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:56 smithi139 ceph-mon[33599]: pgmap v394: 129 pgs: 129 active+clean; 502 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 44 MiB/s wr, 114 op/s 2024-05-09T09:09:56.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:56 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:09:56.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:56 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:09:56.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:56 smithi064 ceph-mon[30149]: pgmap v394: 129 pgs: 129 active+clean; 502 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 44 MiB/s wr, 114 op/s 2024-05-09T09:09:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2500000 of 3200000 tuples (78%) done (elapsed 26.34 s, remaining 7.38 s) 2024-05-09T09:09:58.445 INFO:teuthology.orchestra.run.smithi064.stderr:2600000 of 3200000 tuples (81%) done (elapsed 27.11 s, remaining 6.26 s) 2024-05-09T09:09:58.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:09:58 smithi179 ceph-mon[34086]: pgmap v395: 129 pgs: 129 active+clean; 518 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 39 MiB/s wr, 108 op/s 2024-05-09T09:09:58.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:09:58 smithi139 ceph-mon[33599]: pgmap v395: 129 pgs: 129 active+clean; 518 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 39 MiB/s wr, 108 op/s 2024-05-09T09:09:58.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:09:58 smithi064 ceph-mon[30149]: pgmap v395: 129 pgs: 129 active+clean; 518 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 39 MiB/s wr, 108 op/s 2024-05-09T09:09:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:2700000 of 3200000 tuples (84%) done (elapsed 28.32 s, remaining 5.24 s) 2024-05-09T09:10:00.432 INFO:teuthology.orchestra.run.smithi064.stderr:2800000 of 3200000 tuples (87%) done (elapsed 29.10 s, remaining 4.16 s) 2024-05-09T09:10:00.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:00 smithi179 ceph-mon[34086]: pgmap v396: 129 pgs: 129 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 53 MiB/s wr, 107 op/s 2024-05-09T09:10:00.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:00 smithi179 ceph-mon[34086]: overall HEALTH_OK 2024-05-09T09:10:00.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:00 smithi139 ceph-mon[33599]: pgmap v396: 129 pgs: 129 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 53 MiB/s wr, 107 op/s 2024-05-09T09:10:00.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:00 smithi139 ceph-mon[33599]: overall HEALTH_OK 2024-05-09T09:10:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:00 smithi064 ceph-mon[30149]: pgmap v396: 129 pgs: 129 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 53 MiB/s wr, 107 op/s 2024-05-09T09:10:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:00 smithi064 ceph-mon[30149]: overall HEALTH_OK 2024-05-09T09:10:01.584 INFO:teuthology.orchestra.run.smithi064.stderr:2900000 of 3200000 tuples (90%) done (elapsed 30.25 s, remaining 3.13 s) 2024-05-09T09:10:02.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:02 smithi179 ceph-mon[34086]: pgmap v397: 129 pgs: 129 active+clean; 638 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 42 op/s 2024-05-09T09:10:02.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:02 smithi139 ceph-mon[33599]: pgmap v397: 129 pgs: 129 active+clean; 638 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 42 op/s 2024-05-09T09:10:02.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:02 smithi064 ceph-mon[30149]: pgmap v397: 129 pgs: 129 active+clean; 638 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 42 op/s 2024-05-09T09:10:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:3000000 of 3200000 tuples (93%) done (elapsed 31.41 s, remaining 2.09 s) 2024-05-09T09:10:03.597 INFO:teuthology.orchestra.run.smithi064.stderr:3100000 of 3200000 tuples (96%) done (elapsed 32.27 s, remaining 1.04 s) 2024-05-09T09:10:04.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:04 smithi139 ceph-mon[33599]: pgmap v398: 129 pgs: 129 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 MiB/s wr, 41 op/s 2024-05-09T09:10:04.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:04 smithi064 ceph-mon[30149]: pgmap v398: 129 pgs: 129 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 MiB/s wr, 41 op/s 2024-05-09T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:04 smithi179 ceph-mon[34086]: pgmap v398: 129 pgs: 129 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 MiB/s wr, 41 op/s 2024-05-09T09:10:04.852 INFO:teuthology.orchestra.run.smithi064.stderr:3200000 of 3200000 tuples (100%) done (elapsed 33.52 s, remaining 0.00 s) 2024-05-09T09:10:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:vacuuming... 2024-05-09T09:10:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 -- 172.21.15.64:0/510649423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 msgr2=0x7f6698101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 --2- 172.21.15.64:0/510649423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698101e00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6680009900 tx=0x7f668002a410 comp rx=0 tx=0).stop 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 -- 172.21.15.64:0/510649423 shutdown_connections 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 --2- 172.21.15.64:0/510649423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698102340 0x7f6698108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 --2- 172.21.15.64:0/510649423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698101e00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 --2- 172.21.15.64:0/510649423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698100790 0x7f6698100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.705+0000 7f669d264640 1 -- 172.21.15.64:0/510649423 >> 172.21.15.64:0/510649423 conn(0x7f66980fbd00 msgr2=0x7f66980fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 -- 172.21.15.64:0/510649423 shutdown_connections 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 -- 172.21.15.64:0/510649423 wait complete. 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 Processor -- start 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 -- start start 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698100790 0x7f6698197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698102340 0x7f669819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f669810c310 con 0x7f6698101980 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f669810c190 con 0x7f6698102340 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.706+0000 7f669d264640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f669810c490 con 0x7f6698100790 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54428/0 (socket says 172.21.15.64:54428) 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 -- 172.21.15.64:0/3136492177 learned_addr learned my addr 172.21.15.64:0/3136492177 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698100790 0x7f6698197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f66977fe640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698102340 0x7f669819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 -- 172.21.15.64:0/3136492177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698100790 msgr2=0x7f6698197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698100790 0x7f6698197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 -- 172.21.15.64:0/3136492177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698102340 msgr2=0x7f669819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698102340 0x7f669819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 -- 172.21.15.64:0/3136492177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819f670 con 0x7f6698101980 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698100790 0x7f6698197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f66977fe640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698102340 0x7f669819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f668ffff640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698197f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f66800314e0 tx=0x7f6680003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f6694ff9640 1 -- 172.21.15.64:0/3136492177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668003f070 con 0x7f6698101980 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819f900 con 0x7f6698101980 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f6694ff9640 1 -- 172.21.15.64:0/3136492177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6680031e40 con 0x7f6698101980 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f6694ff9640 1 -- 172.21.15.64:0/3136492177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680035030 con 0x7f6698101980 2024-05-09T09:10:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.707+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669819fe40 con 0x7f6698101980 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.710+0000 7f6694ff9640 1 -- 172.21.15.64:0/3136492177 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6680042070 con 0x7f6698101980 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.710+0000 7f669ff74740 1 Processor -- start 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.710+0000 7f669ff74740 1 -- start start 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f669ff74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a546ff9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f669ff74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9c40 0x55a546ffe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f669ff74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ffedb0 0x55a547001240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f669ff74740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a546f5d340 con 0x55a546ff9250 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f669ff74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a546f05290 con 0x55a546ff9c40 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f669ff74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a546f59300 con 0x55a546ffedb0 2024-05-09T09:10:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a546ff9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a546ff9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54432/0 (socket says 172.21.15.64:54432) 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 -- 172.21.15.64:0/1305150875 learned_addr learned my addr 172.21.15.64:0/1305150875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f668ffff640 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ffedb0 0x55a547001240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f6696ffd640 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9c40 0x55a546ffe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 -- 172.21.15.64:0/1305150875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ffedb0 msgr2=0x55a547001240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ffedb0 0x55a547001240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 -- 172.21.15.64:0/1305150875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9c40 msgr2=0x55a546ffe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9c40 0x55a546ffe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f66977fe640 1 -- 172.21.15.64:0/1305150875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a546f298e0 con 0x55a546ff9250 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.711+0000 7f6694ff9640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66840a2cb0 0x7f66840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.712+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66840a2cb0 0x7f66840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.712+0000 7f6694ff9640 1 -- 172.21.15.64:0/3136492177 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66800efe00 con 0x7f6698101980 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.712+0000 7f66977fe640 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a546ff9670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6690009f30 tx=0x7f6690010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb8a14e27ebe66f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.712+0000 7f668d7fa640 1 -- 172.21.15.64:0/1305150875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690016070 con 0x55a546ff9250 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.712+0000 7f668d7fa640 1 -- 172.21.15.64:0/1305150875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669001bb10 con 0x55a546ff9250 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.712+0000 7f668d7fa640 1 -- 172.21.15.64:0/1305150875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690012c30 con 0x55a546ff9250 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 -- 172.21.15.64:0/1305150875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 msgr2=0x55a546ff9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66840a2cb0 0x7f66840a5170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a546ffa040 tx=0x7f668800b5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a546ff9670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6690009f30 tx=0x7f6690010800 comp rx=0 tx=0).stop 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 -- 172.21.15.64:0/1305150875 shutdown_connections 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ffedb0 0x55a547001240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9c40 0x55a546ffe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 --2- 172.21.15.64:0/1305150875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a546ff9670 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 -- 172.21.15.64:0/1305150875 >> 172.21.15.64:0/1305150875 conn(0x55a546ff48c0 msgr2=0x55a546ff4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 -- 172.21.15.64:0/1305150875 shutdown_connections 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 -- 172.21.15.64:0/1305150875 wait complete. 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 -- start start 2024-05-09T09:10:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.713+0000 7f669ff74740 1 Processor -- start 2024-05-09T09:10:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a547003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 0x55a54700aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a547001d60 con 0x55a546ffedb0 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a546f18790 con 0x55a546ff9250 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a547001ee0 con 0x55a546ff9c40 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f669ff74740 10 client.0 _ll_register_callbacks cb 0x55a546f53a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 0x55a54700aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 0x55a54700aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54446/0 (socket says 172.21.15.64:54446) 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 -- 172.21.15.64:0/3520752040 learned_addr learned my addr 172.21.15.64:0/3520752040 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f668ffff640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a547003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 -- 172.21.15.64:0/3520752040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 msgr2=0x55a547003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a547003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 -- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9250 msgr2=0x55a5470034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 -- 172.21.15.64:0/3520752040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a546c71210 con 0x55a546ffedb0 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f66977fe640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 0x55a54700aa40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6690009c70 tx=0x7f6690012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a565b228d99be9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f667e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f667e7fc640 1 -- 172.21.15.64:0/3520752040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690029060 con 0x55a546ffedb0 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.714+0000 7f667e7fc640 1 -- 172.21.15.64:0/3520752040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6690004d50 con 0x55a546ffedb0 2024-05-09T09:10:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669001bb10 con 0x55a546ffedb0 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f667e7fc640 1 -- 172.21.15.64:0/3520752040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6690016070 con 0x55a546ffedb0 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a546d3d620 con 0x55a546ffedb0 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f667e7fc640 1 -- 172.21.15.64:0/3520752040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669001bb10 con 0x55a546ffedb0 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f667e7fc640 1 -- 172.21.15.64:0/3520752040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f669000f040 con 0x55a546ffedb0 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 20 client.15801 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a54700ec50 0x55a547011040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 4 client.15801 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a546f6aa70 con 0x55a54700ec50 2024-05-09T09:10:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.715+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a54700ec50 0x55a547011040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.716+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a54700ec50 0x55a547011040 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6ac9bb701aaa25 server_cookie=714c35c03f8ac79b in_seq=0 out_seq=0 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.716+0000 7f667e7fc640 10 client.15801 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.716+0000 7f667e7fc640 1 -- 172.21.15.64:0/3520752040 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a546f6aa70 con 0x55a54700ec50 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.716+0000 7f667e7fc640 10 client.15801 handle_command_reply: tid=0 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 client.15801 shutdown 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 msgr2=0x55a54700aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 0x55a54700aa40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6690009c70 tx=0x7f6690012700 comp rx=0 tx=0).stop 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 shutdown_connections 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a54700ec50 0x55a547011040 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ffedb0 0x55a54700aa40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a547003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 --2- 172.21.15.64:0/3520752040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f667e7fc640 0 client.15801 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 >> 172.21.15.64:0/3520752040 conn(0x55a546ff48c0 msgr2=0x55a546ff3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 shutdown_connections 2024-05-09T09:10:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 1 -- 172.21.15.64:0/3520752040 wait complete. 2024-05-09T09:10:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.717+0000 7f669ff74740 20 client.15801 trim_cache size 0 max 16384 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 Processor -- start 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 -- start start 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a54700aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a547003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66981acc00 con 0x55a546ff9250 2024-05-09T09:10:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66981aa4d0 con 0x55a546ffedb0 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.733+0000 7f669ff74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66981a8320 con 0x55a546ff9c40 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f6696ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54452/0 (socket says 172.21.15.64:54452) 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f6696ffd640 1 -- 172.21.15.64:0/3286936819 learned_addr learned my addr 172.21.15.64:0/3286936819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a547003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f668ffff640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a54700aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 -- 172.21.15.64:0/3286936819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 msgr2=0x55a54700aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a54700aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 -- 172.21.15.64:0/3286936819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 msgr2=0x55a5470034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 -- 172.21.15.64:0/3286936819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a546ffa4a0 con 0x55a546ffedb0 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f668ffff640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a54700aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f6696ffd640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f66977fe640 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a547003a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6690012530 tx=0x7f6690010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeae726942fb8db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f668cff9640 1 -- 172.21.15.64:0/3286936819 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690029070 con 0x55a546ffedb0 2024-05-09T09:10:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f668cff9640 1 -- 172.21.15.64:0/3286936819 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6690005850 con 0x55a546ffedb0 2024-05-09T09:10:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.734+0000 7f668cff9640 1 -- 172.21.15.64:0/3286936819 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690030030 con 0x55a546ffedb0 2024-05-09T09:10:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 -- 172.21.15.64:0/3286936819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 msgr2=0x55a547003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a547003a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6690012530 tx=0x7f6690010fd0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 -- 172.21.15.64:0/3286936819 shutdown_connections 2024-05-09T09:10:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a547003a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9c40 0x55a54700aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 --2- 172.21.15.64:0/3286936819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9250 0x55a5470034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 -- 172.21.15.64:0/3286936819 >> 172.21.15.64:0/3286936819 conn(0x55a546ff48c0 msgr2=0x55a546ff4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 -- 172.21.15.64:0/3286936819 shutdown_connections 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.735+0000 7f669ff74740 1 -- 172.21.15.64:0/3286936819 wait complete. 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 -- start start 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 Processor -- start 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9250 0x55a547007ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 0x55a5470080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a5470090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66981a8990 con 0x55a546ff9c40 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66981a9e00 con 0x55a546ffedb0 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66981a9060 con 0x55a546ff9250 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f669ff74740 10 client.0 _ll_register_callbacks cb 0x7f668800b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 0x55a5470080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 0x55a5470080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54464/0 (socket says 172.21.15.64:54464) 2024-05-09T09:10:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 -- 172.21.15.64:0/2288202798 learned_addr learned my addr 172.21.15.64:0/2288202798 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f6696ffd640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9250 0x55a547007ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f66977fe640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a5470090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 -- 172.21.15.64:0/2288202798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9250 msgr2=0x55a547007ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9250 0x55a547007ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 -- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 msgr2=0x55a5470090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.736+0000 7f668ffff640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a5470090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f668ffff640 1 -- 172.21.15.64:0/2288202798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5470096e0 con 0x55a546ff9c40 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f6696ffd640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9250 0x55a547007ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f668ffff640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 0x55a5470080e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6680000c00 tx=0x7f6680005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240d346aada44e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f667e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f667e7fc640 1 -- 172.21.15.64:0/2288202798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680049040 con 0x55a546ff9c40 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f667e7fc640 1 -- 172.21.15.64:0/2288202798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66800bc4d0 con 0x55a546ff9c40 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6690005850 con 0x55a546ff9c40 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f667e7fc640 1 -- 172.21.15.64:0/2288202798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66800bbdb0 con 0x55a546ff9c40 2024-05-09T09:10:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f669ff74740 10 client.15807 fetch_fsmap learned FSMap version 27 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a54700aa60 con 0x55a546ff9c40 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f667e7fc640 1 -- 172.21.15.64:0/2288202798 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6690005850 con 0x55a546ff9c40 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.737+0000 7f667e7fc640 1 -- 172.21.15.64:0/2288202798 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66800d50d0 con 0x55a546ff9c40 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f669ff74740 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f669ff74740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f669ff74740 20 client.15807 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f669ff74740 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a547075730 0x55a547077b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f669ff74740 4 client.15807 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a546f64dc0 con 0x55a547075730 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f6696ffd640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a547075730 0x55a547077b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f667e7fc640 10 client.15807 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.738+0000 7f6696ffd640 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a547075730 0x55a547077b50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc566588a6f24993 server_cookie=a1b9528c14d2338c in_seq=0 out_seq=0 2024-05-09T09:10:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f667e7fc640 1 -- 172.21.15.64:0/2288202798 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a546f64dc0 con 0x55a547075730 2024-05-09T09:10:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f667e7fc640 10 client.15807 handle_command_reply: tid=0 2024-05-09T09:10:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 client.15807 shutdown 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 msgr2=0x55a5470080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 0x55a5470080e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6680000c00 tx=0x7f6680005fd0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 shutdown_connections 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a547075730 0x55a547077b50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a546ffedb0 0x55a5470090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f667e7fc640 0 client.15807 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a546ff9c40 0x55a5470080e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 --2- 172.21.15.64:0/2288202798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a546ff9250 0x55a547007ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 >> 172.21.15.64:0/2288202798 conn(0x55a546ff48c0 msgr2=0x55a546ffae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 shutdown_connections 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 1 -- 172.21.15.64:0/2288202798 wait complete. 2024-05-09T09:10:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.753+0000 7f669ff74740 20 client.15807 trim_cache size 0 max 16384 2024-05-09T09:10:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:10:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:10:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:10:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66840a2cb0 msgr2=0x7f66840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66840a2cb0 0x7f66840a5170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a546ffa040 tx=0x7f668800b5e0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 msgr2=0x7f6698197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698197f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f66800314e0 tx=0x7f6680003c80 comp rx=0 tx=0).stop 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 shutdown_connections 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66840a2cb0 0x7f66840a5170 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6698102340 0x7f669819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6698101980 0x7f6698197f90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 --2- 172.21.15.64:0/3136492177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6698100790 0x7f6698197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 >> 172.21.15.64:0/3136492177 conn(0x7f66980fbd00 msgr2=0x7f66980fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 shutdown_connections 2024-05-09T09:10:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:05.754+0000 7f669d264640 1 -- 172.21.15.64:0/3136492177 wait complete. 2024-05-09T09:10:06.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:06 smithi139 ceph-mon[33599]: pgmap v399: 129 pgs: 129 active+clean; 734 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 MiB/s wr, 36 op/s 2024-05-09T09:10:06.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:06 smithi064 ceph-mon[30149]: pgmap v399: 129 pgs: 129 active+clean; 734 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 MiB/s wr, 36 op/s 2024-05-09T09:10:06.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:06 smithi179 ceph-mon[34086]: pgmap v399: 129 pgs: 129 active+clean; 734 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 MiB/s wr, 36 op/s 2024-05-09T09:10:08.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:08 smithi139 ceph-mon[33599]: pgmap v400: 129 pgs: 129 active+clean; 742 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 MiB/s wr, 32 op/s 2024-05-09T09:10:08.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:08 smithi064 ceph-mon[30149]: pgmap v400: 129 pgs: 129 active+clean; 742 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 MiB/s wr, 32 op/s 2024-05-09T09:10:08.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:08 smithi179 ceph-mon[34086]: pgmap v400: 129 pgs: 129 active+clean; 742 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 MiB/s wr, 32 op/s 2024-05-09T09:10:10.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:10 smithi139 ceph-mon[33599]: pgmap v401: 129 pgs: 129 active+clean; 772 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 46 op/s 2024-05-09T09:10:10.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:10 smithi064 ceph-mon[30149]: pgmap v401: 129 pgs: 129 active+clean; 772 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 46 op/s 2024-05-09T09:10:10.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:10 smithi179 ceph-mon[34086]: pgmap v401: 129 pgs: 129 active+clean; 772 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 46 op/s 2024-05-09T09:10:12.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:12 smithi139 ceph-mon[33599]: pgmap v402: 129 pgs: 129 active+clean; 773 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 39 op/s 2024-05-09T09:10:12.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:12 smithi064 ceph-mon[30149]: pgmap v402: 129 pgs: 129 active+clean; 773 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 39 op/s 2024-05-09T09:10:12.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:12 smithi179 ceph-mon[34086]: pgmap v402: 129 pgs: 129 active+clean; 773 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 39 op/s 2024-05-09T09:10:13.066 INFO:teuthology.orchestra.run.smithi064.stderr:creating primary keys... 2024-05-09T09:10:14.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:14 smithi139 ceph-mon[33599]: pgmap v403: 129 pgs: 129 active+clean; 777 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 43 op/s 2024-05-09T09:10:14.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:14 smithi064 ceph-mon[30149]: pgmap v403: 129 pgs: 129 active+clean; 777 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 43 op/s 2024-05-09T09:10:14.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:14 smithi179 ceph-mon[34086]: pgmap v403: 129 pgs: 129 active+clean; 777 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 43 op/s 2024-05-09T09:10:15.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:10:15.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:10:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 -- 172.21.15.64:0/2228245880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68102320 msgr2=0x7f4b68108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/2228245880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68102320 0x7f4b68108bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009ca0 tx=0x7f4b6002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 -- 172.21.15.64:0/2228245880 shutdown_connections 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/2228245880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68102320 0x7f4b68108bb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/2228245880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b68101960 0x7f4b68101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/2228245880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68100770 0x7f4b68100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.816+0000 7f4b6f57e640 1 -- 172.21.15.64:0/2228245880 >> 172.21.15.64:0/2228245880 conn(0x7f4b680fbce0 msgr2=0x7f4b680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.817+0000 7f4b6f57e640 1 -- 172.21.15.64:0/2228245880 shutdown_connections 2024-05-09T09:10:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.817+0000 7f4b6f57e640 1 -- 172.21.15.64:0/2228245880 wait complete. 2024-05-09T09:10:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.817+0000 7f4b6f57e640 1 Processor -- start 2024-05-09T09:10:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.817+0000 7f4b6f57e640 1 -- start start 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 0x7f4b68199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68101960 0x7f4b6819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b68102320 0x7f4b681a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6810c280 con 0x7f4b68100770 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6810c100 con 0x7f4b68101960 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6810c400 con 0x7f4b68102320 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 0x7f4b68199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 0x7f4b68199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44010/0 (socket says 172.21.15.64:44010) 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 -- 172.21.15.64:0/3384315481 learned_addr learned my addr 172.21.15.64:0/3384315481 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b68102320 0x7f4b681a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68101960 0x7f4b6819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 -- 172.21.15.64:0/3384315481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b68102320 msgr2=0x7f4b681a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b68102320 0x7f4b681a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 -- 172.21.15.64:0/3384315481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68101960 msgr2=0x7f4b6819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68101960 0x7f4b6819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 -- 172.21.15.64:0/3384315481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b681a18d0 con 0x7f4b68100770 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68101960 0x7f4b6819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 0x7f4b68199c90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9c0 tx=0x7f4b6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b5e7fc640 1 -- 172.21.15.64:0/3384315481 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400cd80 con 0x7f4b68100770 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b5e7fc640 1 -- 172.21.15.64:0/3384315481 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b640035f0 con 0x7f4b68100770 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b681a1bc0 con 0x7f4b68100770 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.818+0000 7f4b5e7fc640 1 -- 172.21.15.64:0/3384315481 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64010690 con 0x7f4b68100770 2024-05-09T09:10:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.819+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b681a2110 con 0x7f4b68100770 2024-05-09T09:10:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.821+0000 7f4b5e7fc640 1 -- 172.21.15.64:0/3384315481 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b640037a0 con 0x7f4b68100770 2024-05-09T09:10:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 Processor -- start 2024-05-09T09:10:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 -- start start 2024-05-09T09:10:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4243350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf4248420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf424af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bf41a7d00 con 0x555bf4242f30 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555bf414f4c0 con 0x555bf4243920 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b7236e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555bf41a3cc0 con 0x555bf4248a90 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4243350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4243350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44020/0 (socket says 172.21.15.64:44020) 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 -- 172.21.15.64:0/2535046909 learned_addr learned my addr 172.21.15.64:0/2535046909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf424af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf4248420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b5e7fc640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b540a2b90 0x7f4b540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 -- 172.21.15.64:0/2535046909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 msgr2=0x555bf424af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf424af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 -- 172.21.15.64:0/2535046909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 msgr2=0x555bf4248420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf4248420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.822+0000 7f4b6db1b640 1 -- 172.21.15.64:0/2535046909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf3f857d0 con 0x555bf4242f30 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.823+0000 7f4b5e7fc640 1 -- 172.21.15.64:0/3384315481 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b640cf820 con 0x7f4b68100770 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.823+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4243350 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009ca0 tx=0x7f4b60009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a079d843bce5bfcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.823+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b540a2b90 0x7f4b540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.823+0000 7f4b537fe640 1 -- 172.21.15.64:0/2535046909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60031b40 con 0x555bf4242f30 2024-05-09T09:10:15.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.823+0000 7f4b537fe640 1 -- 172.21.15.64:0/2535046909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b60036030 con 0x555bf4242f30 2024-05-09T09:10:15.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.823+0000 7f4b537fe640 1 -- 172.21.15.64:0/2535046909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60004860 con 0x555bf4242f30 2024-05-09T09:10:15.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 -- 172.21.15.64:0/2535046909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 msgr2=0x555bf4243350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4243350 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009ca0 tx=0x7f4b60009a20 comp rx=0 tx=0).stop 2024-05-09T09:10:15.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b540a2b90 0x7f4b540a5050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b60004af0 tx=0x7f4b6004b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 -- 172.21.15.64:0/2535046909 shutdown_connections 2024-05-09T09:10:15.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf424af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf4248420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2535046909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4243350 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 -- 172.21.15.64:0/2535046909 >> 172.21.15.64:0/2535046909 conn(0x555bf423e5a0 msgr2=0x555bf423e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 -- 172.21.15.64:0/2535046909 shutdown_connections 2024-05-09T09:10:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 -- 172.21.15.64:0/2535046909 wait complete. 2024-05-09T09:10:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 -- start start 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.824+0000 7f4b7236e740 1 Processor -- start 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf424d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf424d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf4254730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bf424b8c0 con 0x555bf4243920 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555bf4162b90 con 0x555bf4242f30 2024-05-09T09:10:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555bf424ba40 con 0x555bf4248a90 2024-05-09T09:10:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b7236e740 10 client.0 _ll_register_callbacks cb 0x555bf419eda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6db1b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf4254730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6cb19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf424d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6db1b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf4254730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50466/0 (socket says 172.21.15.64:50466) 2024-05-09T09:10:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6db1b640 1 -- 172.21.15.64:0/3075297073 learned_addr learned my addr 172.21.15.64:0/3075297073 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf424d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6cb19640 1 -- 172.21.15.64:0/3075297073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 msgr2=0x555bf4254730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf4254730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6cb19640 1 -- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 msgr2=0x555bf424d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.825+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf424d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b6cb19640 1 -- 172.21.15.64:0/3075297073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf3ebb3f0 con 0x555bf4243920 2024-05-09T09:10:15.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf4254730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf424d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf424d6f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x555bf423f470 tx=0x7f4b58011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0226465c3886a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b50ff9640 1 -- 172.21.15.64:0/3075297073 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58011ea0 con 0x555bf4243920 2024-05-09T09:10:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b50ff9640 1 -- 172.21.15.64:0/3075297073 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b58004030 con 0x555bf4243920 2024-05-09T09:10:15.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b60036030 con 0x555bf4243920 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b50ff9640 1 -- 172.21.15.64:0/3075297073 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b58025050 con 0x555bf4243920 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b7236e740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bf3e7b0f0 con 0x555bf4243920 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.826+0000 7f4b50ff9640 1 -- 172.21.15.64:0/3075297073 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b60036030 con 0x555bf4243920 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b50ff9640 1 -- 172.21.15.64:0/3075297073 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b58003850 con 0x555bf4243920 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b7236e740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:10:15.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b7236e740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b7236e740 20 client.15819 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b7236e740 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf425b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b7236e740 4 client.15819 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf41b5430 con 0x555bf4259190 2024-05-09T09:10:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf425b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b50ff9640 10 client.15819 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.827+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf425b5f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cffd77ead3a550c4 server_cookie=58b2cb5121badba4 in_seq=0 out_seq=0 2024-05-09T09:10:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b50ff9640 1 -- 172.21.15.64:0/3075297073 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bf41b5430 con 0x555bf4259190 2024-05-09T09:10:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b50ff9640 10 client.15819 handle_command_reply: tid=0 2024-05-09T09:10:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 client.15819 shutdown 2024-05-09T09:10:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 msgr2=0x555bf424d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf424d6f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x555bf423f470 tx=0x7f4b58011b70 comp rx=0 tx=0).stop 2024-05-09T09:10:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 shutdown_connections 2024-05-09T09:10:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf425b5f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf4254730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf424d6f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b50ff9640 0 client.15819 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 --2- 172.21.15.64:0/3075297073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf424d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.828+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 >> 172.21.15.64:0/3075297073 conn(0x555bf423e5a0 msgr2=0x555bf423d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.829+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 shutdown_connections 2024-05-09T09:10:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.829+0000 7f4b7236e740 1 -- 172.21.15.64:0/3075297073 wait complete. 2024-05-09T09:10:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.829+0000 7f4b7236e740 20 client.15819 trim_cache size 0 max 16384 2024-05-09T09:10:15.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 Processor -- start 2024-05-09T09:10:15.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 -- start start 2024-05-09T09:10:15.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf4244ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf4245220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b681aeeb0 con 0x555bf4243920 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b681ac780 con 0x555bf4242f30 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b7236e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b681aa5d0 con 0x555bf4248a90 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b6cb19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf4245220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b6cb19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf4245220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44048/0 (socket says 172.21.15.64:44048) 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b6cb19640 1 -- 172.21.15.64:0/1336473046 learned_addr learned my addr 172.21.15.64:0/1336473046 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.849+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf4244ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6cb19640 1 -- 172.21.15.64:0/1336473046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 msgr2=0x555bf42546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6cb19640 1 -- 172.21.15.64:0/1336473046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 msgr2=0x555bf4244ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf4244ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6cb19640 1 -- 172.21.15.64:0/1336473046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf4242a90 con 0x555bf4243920 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf4244ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf4245220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4b58003e30 tx=0x7f4b5800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bb63f2ce5198b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b52ffd640 1 -- 172.21.15.64:0/1336473046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58011ea0 con 0x555bf4243920 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b52ffd640 1 -- 172.21.15.64:0/1336473046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b58022040 con 0x555bf4243920 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.850+0000 7f4b52ffd640 1 -- 172.21.15.64:0/1336473046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58004110 con 0x555bf4243920 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 -- 172.21.15.64:0/1336473046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 msgr2=0x555bf4245220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf4245220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4b58003e30 tx=0x7f4b5800bd00 comp rx=0 tx=0).stop 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 -- 172.21.15.64:0/1336473046 shutdown_connections 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4243920 0x555bf4245220 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 --2- 172.21.15.64:0/1336473046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4242f30 0x555bf4244ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 -- 172.21.15.64:0/1336473046 >> 172.21.15.64:0/1336473046 conn(0x555bf423e5a0 msgr2=0x555bf424d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 -- 172.21.15.64:0/1336473046 shutdown_connections 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 -- 172.21.15.64:0/1336473046 wait complete. 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 -- start start 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.851+0000 7f4b7236e740 1 Processor -- start 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4251f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf42524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b681aac40 con 0x555bf4242f30 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b681ac0b0 con 0x555bf4243920 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b681ab310 con 0x555bf4248a90 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b7236e740 10 client.0 _ll_register_callbacks cb 0x7f4b5800b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4251f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4251f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44050/0 (socket says 172.21.15.64:44050) 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 -- 172.21.15.64:0/2563510568 learned_addr learned my addr 172.21.15.64:0/2563510568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:15.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6db1b640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf42524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 -- 172.21.15.64:0/2563510568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 msgr2=0x555bf42534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 -- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 msgr2=0x555bf42524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf42524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 -- 172.21.15.64:0/2563510568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf4253ad0 con 0x555bf4242f30 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b6d31a640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4251f90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4b640cd300 tx=0x7f4b6409b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d352a918626fd3d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.852+0000 7f4b50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b50ff9640 1 -- 172.21.15.64:0/2563510568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6409bcf0 con 0x555bf4242f30 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b50ff9640 1 -- 172.21.15.64:0/2563510568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b640a44d0 con 0x555bf4242f30 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b58022040 con 0x555bf4242f30 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b50ff9640 1 -- 172.21.15.64:0/2563510568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b640a00a0 con 0x555bf4242f30 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 10 client.15825 fetch_fsmap learned FSMap version 27 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bf42bf860 con 0x555bf4242f30 2024-05-09T09:10:15.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b50ff9640 1 -- 172.21.15.64:0/2563510568 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b58022040 con 0x555bf4242f30 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b50ff9640 1 -- 172.21.15.64:0/2563510568 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b6409abb0 con 0x555bf4242f30 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 10 client.15825 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 20 client.15825 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf42c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 4 client.15825 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bf41af780 con 0x555bf4259190 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.853+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf42c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b6cb19640 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf42c2200 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b602573bcf6a445 server_cookie=2260e08602e61f4c in_seq=0 out_seq=0 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b50ff9640 10 client.15825 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b50ff9640 1 -- 172.21.15.64:0/2563510568 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bf41af780 con 0x555bf4259190 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b50ff9640 10 client.15825 handle_command_reply: tid=0 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 client.15825 shutdown 2024-05-09T09:10:15.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 msgr2=0x555bf4251f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4251f90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4b640cd300 tx=0x7f4b6409b710 comp rx=0 tx=0).stop 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 shutdown_connections 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555bf4259190 0x555bf42c2200 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555bf4248a90 0x555bf42534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bf4243920 0x555bf42524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b50ff9640 0 client.15825 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 --2- 172.21.15.64:0/2563510568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf4242f30 0x555bf4251f90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 >> 172.21.15.64:0/2563510568 conn(0x555bf423e5a0 msgr2=0x555bf424d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 shutdown_connections 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 1 -- 172.21.15.64:0/2563510568 wait complete. 2024-05-09T09:10:15.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.854+0000 7f4b7236e740 20 client.15825 trim_cache size 0 max 16384 2024-05-09T09:10:15.864 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:10:15.864 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:10:15.864 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:10:15.864 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b540a2b90 msgr2=0x7f4b540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b540a2b90 0x7f4b540a5050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b60004af0 tx=0x7f4b6004b000 comp rx=0 tx=0).stop 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 msgr2=0x7f4b68199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 0x7f4b68199c90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9c0 tx=0x7f4b6400ee80 comp rx=0 tx=0).stop 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 shutdown_connections 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b540a2b90 0x7f4b540a5050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b68102320 0x7f4b681a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b68101960 0x7f4b6819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 --2- 172.21.15.64:0/3384315481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b68100770 0x7f4b68199c90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 >> 172.21.15.64:0/3384315481 conn(0x7f4b680fbce0 msgr2=0x7f4b680689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 shutdown_connections 2024-05-09T09:10:15.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.855+0000 7f4b6f57e640 1 -- 172.21.15.64:0/3384315481 wait complete. 2024-05-09T09:10:15.866 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:10:15.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:10:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1094348503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 msgr2=0x7f78bc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1094348503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc102f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009900 tx=0x7f78a802a410 comp rx=0 tx=0).stop 2024-05-09T09:10:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1094348503 shutdown_connections 2024-05-09T09:10:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1094348503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 0x7f78bc109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1094348503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc102f00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1094348503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 0x7f78bc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1094348503 >> 172.21.15.64:0/1094348503 conn(0x7f78bc0fcde0 msgr2=0x7f78bc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1094348503 shutdown_connections 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.995+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1094348503 wait complete. 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 Processor -- start 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 -- start start 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 0x7f78bc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 0x7f78bc19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10d3a0 con 0x7f78bc102a80 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10d220 con 0x7f78bc101890 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c2fa9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10d520 con 0x7f78bc103440 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44076/0 (socket says 172.21.15.64:44076) 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78bbfff640 1 -- 172.21.15.64:0/1865921607 learned_addr learned my addr 172.21.15.64:0/1865921607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c0d45640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 0x7f78bc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.996+0000 7f78c1546640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 0x7f78bc19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78bbfff640 1 -- 172.21.15.64:0/1865921607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 msgr2=0x7f78bc19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78bbfff640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 0x7f78bc19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78bbfff640 1 -- 172.21.15.64:0/1865921607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 msgr2=0x7f78bc197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78bbfff640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 0x7f78bc197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78bbfff640 1 -- 172.21.15.64:0/1865921607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc19f6f0 con 0x7f78bc102a80 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78c1546640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 0x7f78bc19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78c0d45640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 0x7f78bc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78bbfff640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc198010 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005940 tx=0x7f78a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a803f030 con 0x7f78bc102a80 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78a8003cb0 con 0x7f78bc102a80 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc19f980 con 0x7f78bc102a80 2024-05-09T09:10:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8035070 con 0x7f78bc102a80 2024-05-09T09:10:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.997+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc19fe70 con 0x7f78bc102a80 2024-05-09T09:10:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:15.998+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc108bd0 con 0x7f78bc102a80 2024-05-09T09:10:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.000+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f78a8042030 con 0x7f78bc102a80 2024-05-09T09:10:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.000+0000 7f78b9ffb640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78ac0a2b90 0x7f78ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.001+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f78a80efa60 con 0x7f78bc102a80 2024-05-09T09:10:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.001+0000 7f78c0d45640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78ac0a2b90 0x7f78ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.001+0000 7f78c0d45640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78ac0a2b90 0x7f78ac0a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78b00058c0 tx=0x7f78b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.005+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f78a80bc630 con 0x7f78bc102a80 2024-05-09T09:10:16.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.285+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78bc062430 con 0x7f78bc102a80 2024-05-09T09:10:16.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.286+0000 7f78b9ffb640 1 -- 172.21.15.64:0/1865921607 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11883 (secure 0 0 0) 0x7f78a80bbf30 con 0x7f78bc102a80 2024-05-09T09:10:16.285 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:10:16.285 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:10:16.239035+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:10:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 28 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78ac0a2b90 msgr2=0x7f78ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78ac0a2b90 0x7f78ac0a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78b00058c0 tx=0x7f78b0005850 comp rx=0 tx=0).stop 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 msgr2=0x7f78bc198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc198010 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005940 tx=0x7f78a80042f0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 shutdown_connections 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f78ac0a2b90 0x7f78ac0a5050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc103440 0x7f78bc19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc102a80 0x7f78bc198010 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 --2- 172.21.15.64:0/1865921607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc101890 0x7f78bc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 >> 172.21.15.64:0/1865921607 conn(0x7f78bc0fcde0 msgr2=0x7f78bc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 shutdown_connections 2024-05-09T09:10:16.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.288+0000 7f78c2fa9640 1 -- 172.21.15.64:0/1865921607 wait complete. 2024-05-09T09:10:16.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:10:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 -- 172.21.15.64:0/2600619460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360100770 msgr2=0x7f3360100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 --2- 172.21.15.64:0/2600619460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360100770 0x7f3360100b90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3354009900 tx=0x7f335402a410 comp rx=0 tx=0).stop 2024-05-09T09:10:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 -- 172.21.15.64:0/2600619460 shutdown_connections 2024-05-09T09:10:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 --2- 172.21.15.64:0/2600619460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3360102320 0x7f3360108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 --2- 172.21.15.64:0/2600619460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3360101960 0x7f3360101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 --2- 172.21.15.64:0/2600619460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360100770 0x7f3360100b90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 -- 172.21.15.64:0/2600619460 >> 172.21.15.64:0/2600619460 conn(0x7f33600fbce0 msgr2=0x7f33600fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 -- 172.21.15.64:0/2600619460 shutdown_connections 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.426+0000 7f3366cba640 1 -- 172.21.15.64:0/2600619460 wait complete. 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 Processor -- start 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 -- start start 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3360100770 0x7f3360197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 0x7f3360198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3360102320 0x7f336019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f336010c290 con 0x7f3360101960 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f336010c110 con 0x7f3360102320 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3366cba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f336010c410 con 0x7f3360100770 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 0x7f3360198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 0x7f3360198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44096/0 (socket says 172.21.15.64:44096) 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 -- 172.21.15.64:0/3105994493 learned_addr learned my addr 172.21.15.64:0/3105994493 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3365257640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3360102320 0x7f336019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f3364a56640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3360100770 0x7f3360197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 -- 172.21.15.64:0/3105994493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3360100770 msgr2=0x7f3360197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3360100770 0x7f3360197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 -- 172.21.15.64:0/3105994493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3360102320 msgr2=0x7f336019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3360102320 0x7f336019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 -- 172.21.15.64:0/3105994493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336019f760 con 0x7f3360101960 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.427+0000 7f335ffff640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 0x7f3360198080 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f334c00d770 tx=0x7f334c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.428+0000 7f335dffb640 1 -- 172.21.15.64:0/3105994493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c017070 con 0x7f3360101960 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.428+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336019fa50 con 0x7f3360101960 2024-05-09T09:10:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.428+0000 7f335dffb640 1 -- 172.21.15.64:0/3105994493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f334c0054d0 con 0x7f3360101960 2024-05-09T09:10:16.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.428+0000 7f335dffb640 1 -- 172.21.15.64:0/3105994493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c004cf0 con 0x7f3360101960 2024-05-09T09:10:16.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.428+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336019ffa0 con 0x7f3360101960 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.430+0000 7f335dffb640 1 -- 172.21.15.64:0/3105994493 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f334c004e90 con 0x7f3360101960 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 Processor -- start 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 -- start start 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e22c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e231430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e233f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b34e190c40 con 0x55b34e231aa0 2024-05-09T09:10:16.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b34e138580 con 0x55b34e22bf40 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f33699c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b34e18cc00 con 0x55b34e22c930 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f335ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e233f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f3365257640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e22c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f335dffb640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33500a2b90 0x7f33500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f3365257640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e22c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37140/0 (socket says 172.21.15.64:37140) 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f3365257640 1 -- 172.21.15.64:0/916881670 learned_addr learned my addr 172.21.15.64:0/916881670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f3364a56640 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e231430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f3365257640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33500a2b90 0x7f33500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.431+0000 7f335dffb640 1 -- 172.21.15.64:0/3105994493 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f334c0ced90 con 0x7f3360101960 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 -- 172.21.15.64:0/916881670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 msgr2=0x55b34e231430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e231430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 -- 172.21.15.64:0/916881670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 msgr2=0x55b34e233f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e233f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 -- 172.21.15.64:0/916881670 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34dea43f0 con 0x55b34e22bf40 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f335ffff640 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e233f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e22c360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3358005c40 tx=0x7f335800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78b29b444953bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f3365257640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33500a2b90 0x7f33500a5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3358008000 tx=0x7f335801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f334affd640 1 -- 172.21.15.64:0/916881670 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335800fd30 con 0x55b34e22bf40 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f334affd640 1 -- 172.21.15.64:0/916881670 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3358022030 con 0x55b34e22bf40 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.432+0000 7f334affd640 1 -- 172.21.15.64:0/916881670 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358010830 con 0x55b34e22bf40 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 -- 172.21.15.64:0/916881670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 msgr2=0x55b34e22c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e22c360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3358005c40 tx=0x7f335800f3c0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 -- 172.21.15.64:0/916881670 shutdown_connections 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e233f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e231430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 --2- 172.21.15.64:0/916881670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e22c360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 -- 172.21.15.64:0/916881670 >> 172.21.15.64:0/916881670 conn(0x55b34e2275b0 msgr2=0x55b34e2279b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 -- 172.21.15.64:0/916881670 shutdown_connections 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 -- 172.21.15.64:0/916881670 wait complete. 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 -- start start 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.433+0000 7f33699c7740 1 Processor -- start 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 0x55b34e236370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22c930 0x55b34e2368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b34e2349f0 con 0x55b34e231aa0 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b34e14bc40 con 0x55b34e22c930 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b34e234b70 con 0x55b34e22bf40 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f33699c7740 10 client.0 _ll_register_callbacks cb 0x55b34dea3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3365257640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 0x55b34e236370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3365257640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44122/0 (socket says 172.21.15.64:44122) 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3365257640 1 -- 172.21.15.64:0/780420442 learned_addr learned my addr 172.21.15.64:0/780420442 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 0x55b34e236370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50534/0 (socket says 172.21.15.64:50534) 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f335ffff640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22c930 0x55b34e2368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 -- 172.21.15.64:0/780420442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22c930 msgr2=0x55b34e2368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22c930 0x55b34e2368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 -- 172.21.15.64:0/780420442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 msgr2=0x55b34e23d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.434+0000 7f3364a56640 1 -- 172.21.15.64:0/780420442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34df6e7d0 con 0x55b34e22bf40 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f3364a56640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 0x55b34e236370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3354009230 tx=0x7f3354009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2213f4f7101fdaa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f333ffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f333ffff640 1 -- 172.21.15.64:0/780420442 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354035040 con 0x55b34e22bf40 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3358022030 con 0x55b34e22bf40 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f333ffff640 1 -- 172.21.15.64:0/780420442 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3354004240 con 0x55b34e22bf40 2024-05-09T09:10:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f333ffff640 1 -- 172.21.15.64:0/780420442 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f335404e050 con 0x55b34e22bf40 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 10 client.25516 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b34e240d20 con 0x55b34e22bf40 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f333ffff640 1 -- 172.21.15.64:0/780420442 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3358022030 con 0x55b34e22bf40 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f333ffff640 1 -- 172.21.15.64:0/780420442 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f335403c440 con 0x55b34e22bf40 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 10 client.25516 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 10 client.25516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 20 client.25516 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e242390 0x55b34e2447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 4 client.25516 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.435+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b34e19e370 con 0x55b34e242390 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.436+0000 7f335ffff640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e242390 0x55b34e2447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.436+0000 7f335ffff640 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e242390 0x55b34e2447e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ca0a14af8f9664a server_cookie=b0d7efa99e11c615 in_seq=0 out_seq=0 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.436+0000 7f333ffff640 10 client.25516 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f333ffff640 1 -- 172.21.15.64:0/780420442 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f334c080ae0 con 0x55b34e242390 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f333ffff640 10 client.25516 handle_command_reply: tid=0 2024-05-09T09:10:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 client.25516 shutdown 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 msgr2=0x55b34e236370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 0x55b34e236370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3354009230 tx=0x7f3354009670 comp rx=0 tx=0).stop 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 shutdown_connections 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e242390 0x55b34e2447e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22c930 0x55b34e2368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f333ffff640 0 client.25516 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 --2- 172.21.15.64:0/780420442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22bf40 0x55b34e236370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 >> 172.21.15.64:0/780420442 conn(0x55b34e2275b0 msgr2=0x55b34e2267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 shutdown_connections 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 1 -- 172.21.15.64:0/780420442 wait complete. 2024-05-09T09:10:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.437+0000 7f33699c7740 20 client.25516 trim_cache size 0 max 16384 2024-05-09T09:10:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 Processor -- start 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 -- start start 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e235990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 0x55b34e23d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e231aa0 0x55b34e235ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33601acc00 con 0x55b34e22c930 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33601aa4a0 con 0x55b34e22bf40 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f33699c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33601a82f0 con 0x55b34e231aa0 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f335ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 0x55b34e23d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f3364a56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e235990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f335ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 0x55b34e23d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44126/0 (socket says 172.21.15.64:44126) 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.454+0000 7f335ffff640 1 -- 172.21.15.64:0/2177096596 learned_addr learned my addr 172.21.15.64:0/2177096596 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f3365257640 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e231aa0 0x55b34e235ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f335ffff640 1 -- 172.21.15.64:0/2177096596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e231aa0 msgr2=0x55b34e235ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f335ffff640 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e231aa0 0x55b34e235ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f335ffff640 1 -- 172.21.15.64:0/2177096596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 msgr2=0x55b34e235990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f335ffff640 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e235990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f335ffff640 1 -- 172.21.15.64:0/2177096596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34e218d20 con 0x55b34e22c930 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f3364a56640 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e235990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f335ffff640 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 0x55b34e23d8b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f334c09b2a0 tx=0x7f334c09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeefaf45b7fbfb38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f334a7fc640 1 -- 172.21.15.64:0/2177096596 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c0b4070 con 0x55b34e22c930 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f334a7fc640 1 -- 172.21.15.64:0/2177096596 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b34e19e370 con 0x55b34e22c930 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.455+0000 7f334a7fc640 1 -- 172.21.15.64:0/2177096596 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c020040 con 0x55b34e22c930 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 -- 172.21.15.64:0/2177096596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 msgr2=0x55b34e23d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 0x55b34e23d8b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f334c09b2a0 tx=0x7f334c09f040 comp rx=0 tx=0).stop 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 -- 172.21.15.64:0/2177096596 shutdown_connections 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e231aa0 0x55b34e235ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e22c930 0x55b34e23d8b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 --2- 172.21.15.64:0/2177096596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e235990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 -- 172.21.15.64:0/2177096596 >> 172.21.15.64:0/2177096596 conn(0x55b34e2275b0 msgr2=0x55b34e22abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 -- 172.21.15.64:0/2177096596 shutdown_connections 2024-05-09T09:10:16.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 -- 172.21.15.64:0/2177096596 wait complete. 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 -- start start 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.456+0000 7f33699c7740 1 Processor -- start 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e23aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e23b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33601a8960 con 0x55b34e231aa0 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33601a9dd0 con 0x55b34e22bf40 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33601a9030 con 0x55b34e22c930 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f33699c7740 10 client.0 _ll_register_callbacks cb 0x7f3354031c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44136/0 (socket says 172.21.15.64:44136) 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 -- 172.21.15.64:0/1151502758 learned_addr learned my addr 172.21.15.64:0/1151502758 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3364a56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e23aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f335ffff640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e23b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 -- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 msgr2=0x55b34e23b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e23b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 -- 172.21.15.64:0/1151502758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 msgr2=0x55b34e23aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e23aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 -- 172.21.15.64:0/1151502758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34e23c9e0 con 0x55b34e231aa0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3364a56640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e23aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f3365257640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23c3f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3358006010 tx=0x7f3358013aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894e22172c09f808 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.457+0000 7f333ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f333ffff640 1 -- 172.21.15.64:0/1151502758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358013120 con 0x55b34e231aa0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f333ffff640 1 -- 172.21.15.64:0/1151502758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3358010aa0 con 0x55b34e231aa0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b34e19e370 con 0x55b34e231aa0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f333ffff640 1 -- 172.21.15.64:0/1151502758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33580132c0 con 0x55b34e231aa0 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:16.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b34e2a88f0 con 0x55b34e231aa0 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f333ffff640 1 -- 172.21.15.64:0/1151502758 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b34e19e370 con 0x55b34e231aa0 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f333ffff640 1 -- 172.21.15.64:0/1151502758 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f33580050d0 con 0x55b34e231aa0 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 10 client.15849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 20 client.15849 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e2a9120 0x55b34e2ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 4 client.15849 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.458+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b34e1986c0 con 0x55b34e2a9120 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f3364a56640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e2a9120 0x55b34e2ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f3364a56640 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e2a9120 0x55b34e2ab540 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c0cbf6be1a5e41 server_cookie=e6b8f35455fe1114 in_seq=0 out_seq=0 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f333ffff640 10 client.15849 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f333ffff640 1 -- 172.21.15.64:0/1151502758 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b34e1986c0 con 0x55b34e2a9120 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f333ffff640 10 client.15849 handle_command_reply: tid=0 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 client.15849 shutdown 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 msgr2=0x55b34e23c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23c3f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3358006010 tx=0x7f3358013aa0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 shutdown_connections 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b34e2a9120 0x55b34e2ab540 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f333ffff640 0 client.15849 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b34e231aa0 0x55b34e23c3f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b34e22c930 0x55b34e23b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 --2- 172.21.15.64:0/1151502758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b34e22bf40 0x55b34e23aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 >> 172.21.15.64:0/1151502758 conn(0x55b34e2275b0 msgr2=0x55b34e225f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 shutdown_connections 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.459+0000 7f33699c7740 1 -- 172.21.15.64:0/1151502758 wait complete. 2024-05-09T09:10:16.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f33699c7740 20 client.15849 trim_cache size 0 max 16384 2024-05-09T09:10:16.465 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:10:16.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33500a2b90 msgr2=0x7f33500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33500a2b90 0x7f33500a5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3358008000 tx=0x7f335801f000 comp rx=0 tx=0).stop 2024-05-09T09:10:16.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 msgr2=0x7f3360198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 0x7f3360198080 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f334c00d770 tx=0x7f334c00dc30 comp rx=0 tx=0).stop 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 shutdown_connections 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33500a2b90 0x7f33500a5050 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3360102320 0x7f336019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3360101960 0x7f3360198080 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 --2- 172.21.15.64:0/3105994493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3360100770 0x7f3360197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 >> 172.21.15.64:0/3105994493 conn(0x7f33600fbce0 msgr2=0x7f33600689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 shutdown_connections 2024-05-09T09:10:16.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.460+0000 7f3366cba640 1 -- 172.21.15.64:0/3105994493 wait complete. 2024-05-09T09:10:16.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:10:16.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:16 smithi139 ceph-mon[33599]: pgmap v404: 129 pgs: 129 active+clean; 797 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 MiB/s wr, 43 op/s 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 -- 172.21.15.64:0/381040100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370103430 msgr2=0x7fe370109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 --2- 172.21.15.64:0/381040100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370103430 0x7fe370109cc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ca0 tx=0x7fe36802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 -- 172.21.15.64:0/381040100 shutdown_connections 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 --2- 172.21.15.64:0/381040100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370103430 0x7fe370109cc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 --2- 172.21.15.64:0/381040100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370102a70 0x7fe370102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 --2- 172.21.15.64:0/381040100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370101880 0x7fe370101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 -- 172.21.15.64:0/381040100 >> 172.21.15.64:0/381040100 conn(0x7fe3700fce10 msgr2=0x7fe3700ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 -- 172.21.15.64:0/381040100 shutdown_connections 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 -- 172.21.15.64:0/381040100 wait complete. 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.591+0000 7fe376b68640 1 Processor -- start 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 -- start start 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 0x7fe370197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370102a70 0x7fe370197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370103430 0x7fe37019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe37010d390 con 0x7fe370101880 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe37010d210 con 0x7fe370102a70 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe376b68640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe37010d510 con 0x7fe370103430 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 0x7fe370197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 0x7fe370197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44146/0 (socket says 172.21.15.64:44146) 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 -- 172.21.15.64:0/1614265756 learned_addr learned my addr 172.21.15.64:0/1614265756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe36ffff640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370102a70 0x7fe370197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe375105640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370103430 0x7fe37019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 -- 172.21.15.64:0/1614265756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370103430 msgr2=0x7fe37019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370103430 0x7fe37019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 -- 172.21.15.64:0/1614265756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370102a70 msgr2=0x7fe370197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370102a70 0x7fe370197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 -- 172.21.15.64:0/1614265756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37019f640 con 0x7fe370101880 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe36ffff640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370102a70 0x7fe370197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe375105640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370103430 0x7fe37019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.592+0000 7fe374904640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 0x7fe370197a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe36400e9c0 tx=0x7fe36400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.593+0000 7fe36dffb640 1 -- 172.21.15.64:0/1614265756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36400cd80 con 0x7fe370101880 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.593+0000 7fe36dffb640 1 -- 172.21.15.64:0/1614265756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3640035f0 con 0x7fe370101880 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.593+0000 7fe36dffb640 1 -- 172.21.15.64:0/1614265756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364010690 con 0x7fe370101880 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.593+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37019f930 con 0x7fe370101880 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.593+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37019fe80 con 0x7fe370101880 2024-05-09T09:10:16.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.595+0000 7fe36dffb640 1 -- 172.21.15.64:0/1614265756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3640037a0 con 0x7fe370101880 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 Processor -- start 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe36dffb640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3600a2b90 0x7fe3600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 -- start start 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe36dffb640 1 -- 172.21.15.64:0/1614265756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe3640cf820 con 0x7fe370101880 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe375105640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3600a2b90 0x7fe3600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4610770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4615840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4618340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562ef4570c40 con 0x562ef4610d40 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562ef4518580 con 0x562ef4610350 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.596+0000 7fe379872740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562ef456cc00 con 0x562ef4615eb0 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4615840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4610770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4615840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44156/0 (socket says 172.21.15.64:44156) 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe36ffff640 1 -- 172.21.15.64:0/3725839177 learned_addr learned my addr 172.21.15.64:0/3725839177 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe375105640 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4618340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 -- 172.21.15.64:0/3725839177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 msgr2=0x562ef4618340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4618340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 -- 172.21.15.64:0/3725839177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 msgr2=0x562ef4615840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4615840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 -- 172.21.15.64:0/3725839177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef42843f0 con 0x562ef4610350 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe375105640 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4618340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe375105640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3600a2b90 0x7fe3600a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe368004af0 tx=0x7fe3680098c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe374904640 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4610770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ccfe0 tx=0x7fe36409b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce88a649a0c7f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe35affd640 1 -- 172.21.15.64:0/3725839177 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36409bcf0 con 0x562ef4610350 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe35affd640 1 -- 172.21.15.64:0/3725839177 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3640a00a0 con 0x562ef4610350 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.597+0000 7fe35affd640 1 -- 172.21.15.64:0/3725839177 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640a4730 con 0x562ef4610350 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 -- 172.21.15.64:0/3725839177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 msgr2=0x562ef4610770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4610770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ccfe0 tx=0x7fe36409b8f0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 -- 172.21.15.64:0/3725839177 shutdown_connections 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4618340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4615840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 --2- 172.21.15.64:0/3725839177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4610770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 -- 172.21.15.64:0/3725839177 >> 172.21.15.64:0/3725839177 conn(0x562ef46075b0 msgr2=0x562ef46079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 -- 172.21.15.64:0/3725839177 shutdown_connections 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 -- 172.21.15.64:0/3725839177 wait complete. 2024-05-09T09:10:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.598+0000 7fe379872740 1 -- start start 2024-05-09T09:10:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 Processor -- start 2024-05-09T09:10:16.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef461a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562ef4618da0 con 0x562ef4610d40 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562ef452bc40 con 0x562ef4610350 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562ef4618f20 con 0x562ef4615eb0 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe379872740 10 client.0 _ll_register_callbacks cb 0x562ef4283be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe374904640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef461a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44158/0 (socket says 172.21.15.64:44158) 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe36ffff640 1 -- 172.21.15.64:0/3968645089 learned_addr learned my addr 172.21.15.64:0/3968645089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe375105640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe36ffff640 1 -- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 msgr2=0x562ef4621d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.599+0000 7fe36ffff640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe36ffff640 1 -- 172.21.15.64:0/3968645089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 msgr2=0x562ef461a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe36ffff640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef461a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe36ffff640 1 -- 172.21.15.64:0/3968645089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef434e7d0 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe374904640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef461a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe375105640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe36ffff640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461acc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0116a0 tx=0x7fe35c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7ff53eab7e3d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe34ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe34ffff640 1 -- 172.21.15.64:0/3968645089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c010c00 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe34ffff640 1 -- 172.21.15.64:0/3968645089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe35c010da0 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3640a00a0 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe34ffff640 1 -- 172.21.15.64:0/3968645089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe35c025050 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe379872740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ef4625050 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.600+0000 7fe34ffff640 1 -- 172.21.15.64:0/3968645089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3640a00a0 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe34ffff640 1 -- 172.21.15.64:0/3968645089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe35c00f450 con 0x562ef4610d40 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe379872740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe379872740 10 client.15861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe379872740 20 client.15861 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe379872740 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef4628f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe379872740 4 client.15861 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef457e370 con 0x562ef4626aa0 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe374904640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef4628f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe374904640 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef4628f40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff5c88112a6421a2 server_cookie=11e730bc7ac4e001 in_seq=0 out_seq=0 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.601+0000 7fe34ffff640 10 client.15861 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe34ffff640 1 -- 172.21.15.64:0/3968645089 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ef457e370 con 0x562ef4626aa0 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe34ffff640 10 client.15861 handle_command_reply: tid=0 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 client.15861 shutdown 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 msgr2=0x562ef461acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461acc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0116a0 tx=0x7fe35c011b60 comp rx=0 tx=0).stop 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 shutdown_connections 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef4628f40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe34ffff640 0 client.15861 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461acc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 --2- 172.21.15.64:0/3968645089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef461a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 >> 172.21.15.64:0/3968645089 conn(0x562ef46075b0 msgr2=0x562ef46067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 shutdown_connections 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 1 -- 172.21.15.64:0/3968645089 wait complete. 2024-05-09T09:10:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.602+0000 7fe379872740 20 client.15861 trim_cache size 0 max 16384 2024-05-09T09:10:16.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.616+0000 7fe379872740 1 Processor -- start 2024-05-09T09:10:16.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.616+0000 7fe379872740 1 -- start start 2024-05-09T09:10:16.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe379872740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4612240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe379872740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4612780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe379872740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe379872740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701aaaf0 con 0x562ef4610d40 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe379872740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a8390 con 0x562ef4610350 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe379872740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a61e0 con 0x562ef4615eb0 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4612780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4612780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44172/0 (socket says 172.21.15.64:44172) 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4612240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe36ffff640 1 -- 172.21.15.64:0/277369063 learned_addr learned my addr 172.21.15.64:0/277369063 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe375105640 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 -- 172.21.15.64:0/277369063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 msgr2=0x562ef4621cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 -- 172.21.15.64:0/277369063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 msgr2=0x562ef4612780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4612780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 -- 172.21.15.64:0/277369063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef4618d40 con 0x562ef4610350 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe36ffff640 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4612780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe374904640 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4612240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c6a0 tx=0x7fe364003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a1d28aad84ba92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.617+0000 7fe35a7fc640 1 -- 172.21.15.64:0/277369063 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640a00a0 con 0x562ef4610350 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe35a7fc640 1 -- 172.21.15.64:0/277369063 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36409bcf0 con 0x562ef4610350 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe35a7fc640 1 -- 172.21.15.64:0/277369063 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364097ca0 con 0x562ef4610350 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 -- 172.21.15.64:0/277369063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 msgr2=0x562ef4612240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4612240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c6a0 tx=0x7fe364003e30 comp rx=0 tx=0).stop 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 -- 172.21.15.64:0/277369063 shutdown_connections 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4615eb0 0x562ef4621cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef4612780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 --2- 172.21.15.64:0/277369063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4610350 0x562ef4612240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 -- 172.21.15.64:0/277369063 >> 172.21.15.64:0/277369063 conn(0x562ef46075b0 msgr2=0x562ef461a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 -- 172.21.15.64:0/277369063 shutdown_connections 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.618+0000 7fe379872740 1 -- 172.21.15.64:0/277369063 wait complete. 2024-05-09T09:10:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 -- start start 2024-05-09T09:10:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 Processor -- start 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 0x562ef461f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4615eb0 0x562ef4620710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a6850 con 0x562ef4610d40 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a7cc0 con 0x562ef4615eb0 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701a6f20 con 0x562ef4610350 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe379872740 10 client.0 _ll_register_callbacks cb 0x7fe35c00bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe374904640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 0x562ef461f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe375105640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4615eb0 0x562ef4620710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe374904640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 0x562ef461f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50600/0 (socket says 172.21.15.64:50600) 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44184/0 (socket says 172.21.15.64:44184) 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.619+0000 7fe374904640 1 -- 172.21.15.64:0/3514517972 learned_addr learned my addr 172.21.15.64:0/3514517972 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe374904640 1 -- 172.21.15.64:0/3514517972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4615eb0 msgr2=0x562ef4620710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe374904640 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4615eb0 0x562ef4620710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe374904640 1 -- 172.21.15.64:0/3514517972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 msgr2=0x562ef461f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe374904640 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe374904640 1 -- 172.21.15.64:0/3514517972 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef4620d00 con 0x562ef4610350 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe375105640 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4615eb0 0x562ef4620710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe374904640 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 0x562ef461f210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe364009830 tx=0x7fe36409c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=176623d007da23c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe34ffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe34ffff640 1 -- 172.21.15.64:0/3514517972 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640a00a0 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36409bcf0 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe34ffff640 1 -- 172.21.15.64:0/3514517972 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe36407ab60 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe34ffff640 1 -- 172.21.15.64:0/3514517972 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe364085020 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe379872740 10 client.25537 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.620+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ef4622000 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe34ffff640 1 -- 172.21.15.64:0/3514517972 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36409bcf0 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe34ffff640 1 -- 172.21.15.64:0/3514517972 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe36409abb0 con 0x562ef4610350 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe379872740 10 client.25537 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe379872740 10 client.25537 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe379872740 20 client.25537 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe379872740 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef468f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe379872740 4 client.25537 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ef45786c0 con 0x562ef4626aa0 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe36ffff640 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef468f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe34ffff640 10 client.25537 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.621+0000 7fe36ffff640 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef468f380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63f702954bbaacda server_cookie=659617712b65990 in_seq=0 out_seq=0 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe34ffff640 1 -- 172.21.15.64:0/3514517972 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ef45786c0 con 0x562ef4626aa0 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe34ffff640 10 client.25537 handle_command_reply: tid=0 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 client.25537 shutdown 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 msgr2=0x562ef461f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 0x562ef461f210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe364009830 tx=0x7fe36409c540 comp rx=0 tx=0).stop 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 shutdown_connections 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ef4615eb0 0x562ef4620710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562ef4626aa0 0x562ef468f380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ef4610d40 0x562ef461f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe34ffff640 0 client.25537 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 --2- 172.21.15.64:0/3514517972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562ef4610350 0x562ef461f210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 >> 172.21.15.64:0/3514517972 conn(0x562ef46075b0 msgr2=0x562ef461a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 shutdown_connections 2024-05-09T09:10:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 1 -- 172.21.15.64:0/3514517972 wait complete. 2024-05-09T09:10:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.622+0000 7fe379872740 20 client.25537 trim_cache size 0 max 16384 2024-05-09T09:10:16.627 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3600a2b90 msgr2=0x7fe3600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3600a2b90 0x7fe3600a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe368004af0 tx=0x7fe3680098c0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 msgr2=0x7fe370197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 0x7fe370197a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe36400e9c0 tx=0x7fe36400ee80 comp rx=0 tx=0).stop 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 shutdown_connections 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3600a2b90 0x7fe3600a5050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe370103430 0x7fe37019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe370102a70 0x7fe370197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 --2- 172.21.15.64:0/1614265756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe370101880 0x7fe370197a20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 >> 172.21.15.64:0/1614265756 conn(0x7fe3700fce10 msgr2=0x7fe370105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 shutdown_connections 2024-05-09T09:10:16.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.623+0000 7fe376b68640 1 -- 172.21.15.64:0/1614265756 wait complete. 2024-05-09T09:10:16.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:16 smithi064 ceph-mon[30149]: pgmap v404: 129 pgs: 129 active+clean; 797 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 MiB/s wr, 43 op/s 2024-05-09T09:10:16.633 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:10:16.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:16 smithi179 ceph-mon[34086]: pgmap v404: 129 pgs: 129 active+clean; 797 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 MiB/s wr, 43 op/s 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 -- 172.21.15.64:0/4283167263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c100770 msgr2=0x7fe82c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 --2- 172.21.15.64:0/4283167263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c100770 0x7fe82c100b90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe828009ca0 tx=0x7fe82802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 -- 172.21.15.64:0/4283167263 shutdown_connections 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 --2- 172.21.15.64:0/4283167263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c102320 0x7fe82c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 --2- 172.21.15.64:0/4283167263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c101960 0x7fe82c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 --2- 172.21.15.64:0/4283167263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c100770 0x7fe82c100b90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 -- 172.21.15.64:0/4283167263 >> 172.21.15.64:0/4283167263 conn(0x7fe82c0fbce0 msgr2=0x7fe82c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 -- 172.21.15.64:0/4283167263 shutdown_connections 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.757+0000 7fe833e27640 1 -- 172.21.15.64:0/4283167263 wait complete. 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 Processor -- start 2024-05-09T09:10:16.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 -- start start 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c100770 0x7fe82c1046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 0x7fe82c102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c102320 0x7fe82c103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c10c270 con 0x7fe82c101960 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c10c0f0 con 0x7fe82c100770 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe833e27640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c10c3f0 con 0x7fe82c102320 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 0x7fe82c102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 0x7fe82c102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44200/0 (socket says 172.21.15.64:44200) 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 -- 172.21.15.64:0/3092921470 learned_addr learned my addr 172.21.15.64:0/3092921470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe831bc3640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c100770 0x7fe82c1046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8323c4640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c102320 0x7fe82c103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 -- 172.21.15.64:0/3092921470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c102320 msgr2=0x7fe82c103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c102320 0x7fe82c103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 -- 172.21.15.64:0/3092921470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c100770 msgr2=0x7fe82c1046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c100770 0x7fe82c1046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe8313c2640 1 -- 172.21.15.64:0/3092921470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c103b40 con 0x7fe82c101960 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.758+0000 7fe831bc3640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c100770 0x7fe82c1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.759+0000 7fe8313c2640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 0x7fe82c102d40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00ea60 tx=0x7fe81c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.759+0000 7fe822ffd640 1 -- 172.21.15.64:0/3092921470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00cde0 con 0x7fe82c101960 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.759+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c1a1c70 con 0x7fe82c101960 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.759+0000 7fe822ffd640 1 -- 172.21.15.64:0/3092921470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe81c0035f0 con 0x7fe82c101960 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.759+0000 7fe822ffd640 1 -- 172.21.15.64:0/3092921470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c010690 con 0x7fe82c101960 2024-05-09T09:10:16.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.759+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c1a2190 con 0x7fe82c101960 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 Processor -- start 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe822ffd640 1 -- 172.21.15.64:0/3092921470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe81c0037a0 con 0x7fe82c101960 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 -- start start 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a5aa0 0x55a5720a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a572004c40 con 0x55a5720a0930 2024-05-09T09:10:16.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a571fac580 con 0x55a5720a5aa0 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe836c0d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a572000c00 con 0x55a57209ff40 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe831bc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe8313c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a5aa0 0x55a5720a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe831bc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44214/0 (socket says 172.21.15.64:44214) 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe831bc3640 1 -- 172.21.15.64:0/3962205996 learned_addr learned my addr 172.21.15.64:0/3962205996 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.762+0000 7fe8323c4640 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe8323c4640 1 -- 172.21.15.64:0/3962205996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a5aa0 msgr2=0x55a5720a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe8323c4640 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a5aa0 0x55a5720a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe8323c4640 1 -- 172.21.15.64:0/3962205996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 msgr2=0x55a5720a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe8323c4640 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe8323c4640 1 -- 172.21.15.64:0/3962205996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a571d183f0 con 0x55a57209ff40 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe822ffd640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8180a2b90 0x7fe8180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe831bc3640 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe822ffd640 1 -- 172.21.15.64:0/3092921470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe81c0cf6e0 con 0x7fe82c101960 2024-05-09T09:10:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe831bc3640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8180a2b90 0x7fe8180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe8323c4640 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720a0360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe824009af0 tx=0x7fe8240113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=140c93b2bb37a0f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe817fff640 1 -- 172.21.15.64:0/3962205996 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824011e10 con 0x55a57209ff40 2024-05-09T09:10:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe817fff640 1 -- 172.21.15.64:0/3962205996 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe824012900 con 0x55a57209ff40 2024-05-09T09:10:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.763+0000 7fe817fff640 1 -- 172.21.15.64:0/3962205996 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82400f9f0 con 0x55a57209ff40 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 -- 172.21.15.64:0/3962205996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 msgr2=0x55a5720a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe831bc3640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8180a2b90 0x7fe8180a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55a5720a0d30 tx=0x7fe828031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720a0360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe824009af0 tx=0x7fe8240113e0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 -- 172.21.15.64:0/3962205996 shutdown_connections 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a5aa0 0x55a5720a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 --2- 172.21.15.64:0/3962205996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720a0360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 -- 172.21.15.64:0/3962205996 >> 172.21.15.64:0/3962205996 conn(0x55a57209b5b0 msgr2=0x55a57209b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 -- 172.21.15.64:0/3962205996 shutdown_connections 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 -- 172.21.15.64:0/3962205996 wait complete. 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.764+0000 7fe836c0d740 1 -- start start 2024-05-09T09:10:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 Processor -- start 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5720a8a50 con 0x55a5720a5aa0 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a571fbfc40 con 0x55a5720a0930 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a5720a8bd0 con 0x55a57209ff40 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe836c0d740 10 client.0 _ll_register_callbacks cb 0x55a571d17be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe8323c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe8323c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44226/0 (socket says 172.21.15.64:44226) 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe8323c4640 1 -- 172.21.15.64:0/4078101493 learned_addr learned my addr 172.21.15.64:0/4078101493 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe831bc3640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.765+0000 7fe8313c2640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8323c4640 1 -- 172.21.15.64:0/4078101493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 msgr2=0x55a5720aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8323c4640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8323c4640 1 -- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 msgr2=0x55a5720aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8323c4640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8323c4640 1 -- 172.21.15.64:0/4078101493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a571de27d0 con 0x55a5720a5aa0 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8323c4640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b1910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe824000c00 tx=0x7fe824012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc33125f892e02ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8157fa640 1 -- 172.21.15.64:0/4078101493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824028070 con 0x55a5720a5aa0 2024-05-09T09:10:16.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8157fa640 1 -- 172.21.15.64:0/4078101493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe824005040 con 0x55a5720a5aa0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe824012900 con 0x55a5720a5aa0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8157fa640 1 -- 172.21.15.64:0/4078101493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe824034050 con 0x55a5720a5aa0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe836c0d740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5720b4c80 con 0x55a5720a5aa0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.766+0000 7fe8157fa640 1 -- 172.21.15.64:0/4078101493 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe824012900 con 0x55a5720a5aa0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe8157fa640 1 -- 172.21.15.64:0/4078101493 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe82400fbf0 con 0x55a5720a5aa0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe836c0d740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe836c0d740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe836c0d740 20 client.15879 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe836c0d740 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a5720b6080 0x55a5720b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe836c0d740 4 client.15879 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a572012370 con 0x55a5720b6080 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe831bc3640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a5720b6080 0x55a5720b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe8157fa640 10 client.15879 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.767+0000 7fe831bc3640 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a5720b6080 0x55a5720b8470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a633826866ba400 server_cookie=5dda57325bbd72f7 in_seq=0 out_seq=0 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.768+0000 7fe8157fa640 1 -- 172.21.15.64:0/4078101493 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a572012370 con 0x55a5720b6080 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.768+0000 7fe8157fa640 10 client.15879 handle_command_reply: tid=0 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.768+0000 7fe836c0d740 1 client.15879 shutdown 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.768+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 msgr2=0x55a5720b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.768+0000 7fe836c0d740 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b1910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe824000c00 tx=0x7fe824012700 comp rx=0 tx=0).stop 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 shutdown_connections 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a5720b6080 0x55a5720b8470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe8157fa640 0 client.15879 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b1910 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 --2- 172.21.15.64:0/4078101493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 >> 172.21.15.64:0/4078101493 conn(0x55a57209b5b0 msgr2=0x55a57209a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 shutdown_connections 2024-05-09T09:10:16.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 1 -- 172.21.15.64:0/4078101493 wait complete. 2024-05-09T09:10:16.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.769+0000 7fe836c0d740 20 client.15879 trim_cache size 0 max 16384 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.786+0000 7fe836c0d740 1 Processor -- start 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 -- start start 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5720a5aa0 0x55a5720aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1af000 con 0x55a5720a0930 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1ac8a0 con 0x55a57209ff40 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe836c0d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1aa6f0 con 0x55a5720a5aa0 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44242/0 (socket says 172.21.15.64:44242) 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 -- 172.21.15.64:0/2536143411 learned_addr learned my addr 172.21.15.64:0/2536143411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe831bc3640 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8323c4640 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5720a5aa0 0x55a5720aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 -- 172.21.15.64:0/2536143411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5720a5aa0 msgr2=0x55a5720aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5720a5aa0 0x55a5720aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 -- 172.21.15.64:0/2536143411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57209ff40 msgr2=0x55a5720aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 -- 172.21.15.64:0/2536143411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57208cd20 con 0x55a5720a0930 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.787+0000 7fe8313c2640 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720b18d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe81c098010 tx=0x7fe81c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f5e109ab3e96da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe8177fe640 1 -- 172.21.15.64:0/2536143411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c09b6f0 con 0x55a5720a0930 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe8177fe640 1 -- 172.21.15.64:0/2536143411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe81c003b30 con 0x55a5720a0930 2024-05-09T09:10:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe8177fe640 1 -- 172.21.15.64:0/2536143411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c09b510 con 0x55a5720a0930 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 -- 172.21.15.64:0/2536143411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 msgr2=0x55a5720b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720b18d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe81c098010 tx=0x7fe81c009590 comp rx=0 tx=0).stop 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 -- 172.21.15.64:0/2536143411 shutdown_connections 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5720a5aa0 0x55a5720aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a0930 0x55a5720b18d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 --2- 172.21.15.64:0/2536143411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57209ff40 0x55a5720aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.788+0000 7fe836c0d740 1 -- 172.21.15.64:0/2536143411 >> 172.21.15.64:0/2536143411 conn(0x55a57209b5b0 msgr2=0x55a5720a1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 -- 172.21.15.64:0/2536143411 shutdown_connections 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 -- 172.21.15.64:0/2536143411 wait complete. 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 -- start start 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 Processor -- start 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1aad60 con 0x55a5720a5aa0 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1ac1d0 con 0x55a5720a0930 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1ab430 con 0x55a57209ff40 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.789+0000 7fe836c0d740 10 client.0 _ll_register_callbacks cb 0x7fe81c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8313c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe831bc3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44248/0 (socket says 172.21.15.64:44248) 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 -- 172.21.15.64:0/1482311185 learned_addr learned my addr 172.21.15.64:0/1482311185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8313c2640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37284/0 (socket says 172.21.15.64:37284) 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 -- 172.21.15.64:0/1482311185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 msgr2=0x55a5720aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 -- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 msgr2=0x55a5720af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 -- 172.21.15.64:0/1482311185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5720b07e0 con 0x55a5720a5aa0 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe831bc3640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8313c2640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8323c4640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b01f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe82400a010 tx=0x7fe824009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e0649d925c1141 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8157fa640 1 -- 172.21.15.64:0/1482311185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824028040 con 0x55a5720a5aa0 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8157fa640 1 -- 172.21.15.64:0/1482311185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe824004b00 con 0x55a5720a5aa0 2024-05-09T09:10:16.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a57200c6c0 con 0x55a5720a5aa0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe8157fa640 1 -- 172.21.15.64:0/1482311185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe824009700 con 0x55a5720a5aa0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe836c0d740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.790+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5720b1ae0 con 0x55a5720a5aa0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe8157fa640 1 -- 172.21.15.64:0/1482311185 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a57200c6c0 con 0x55a5720a5aa0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe8157fa640 1 -- 172.21.15.64:0/1482311185 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe824003b90 con 0x55a5720a5aa0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe836c0d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe836c0d740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe836c0d740 20 client.15885 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe836c0d740 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a57211cc90 0x55a57211f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe836c0d740 4 client.15885 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a571f89820 con 0x55a57211cc90 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.791+0000 7fe831bc3640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a57211cc90 0x55a57211f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe831bc3640 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a57211cc90 0x55a57211f0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c770318c0fe2615 server_cookie=e1d1fccfbb16863 in_seq=0 out_seq=0 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe8157fa640 10 client.15885 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe8157fa640 1 -- 172.21.15.64:0/1482311185 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a571f89820 con 0x55a57211cc90 2024-05-09T09:10:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe8157fa640 10 client.15885 handle_command_reply: tid=0 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 client.15885 shutdown 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 msgr2=0x55a5720b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b01f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe82400a010 tx=0x7fe824009ab0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 shutdown_connections 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a57211cc90 0x55a57211f0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5720a5aa0 0x55a5720b01f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe8157fa640 0 client.15885 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a5720a0930 0x55a5720af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 --2- 172.21.15.64:0/1482311185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a57209ff40 0x55a5720aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 >> 172.21.15.64:0/1482311185 conn(0x55a57209b5b0 msgr2=0x55a57209a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 shutdown_connections 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 1 -- 172.21.15.64:0/1482311185 wait complete. 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.792+0000 7fe836c0d740 20 client.15885 trim_cache size 0 max 16384 2024-05-09T09:10:16.797 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:10:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8180a2b90 msgr2=0x7fe8180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8180a2b90 0x7fe8180a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55a5720a0d30 tx=0x7fe828031000 comp rx=0 tx=0).stop 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 msgr2=0x7fe82c102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 0x7fe82c102d40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00ea60 tx=0x7fe81c00ef20 comp rx=0 tx=0).stop 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 shutdown_connections 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8180a2b90 0x7fe8180a5050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c102320 0x7fe82c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82c101960 0x7fe82c102d40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 --2- 172.21.15.64:0/3092921470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe82c100770 0x7fe82c1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 >> 172.21.15.64:0/3092921470 conn(0x7fe82c0fbce0 msgr2=0x7fe82c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 shutdown_connections 2024-05-09T09:10:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:16.793+0000 7fe833e27640 1 -- 172.21.15.64:0/3092921470 wait complete. 2024-05-09T09:10:17.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:17 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:17.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:17 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1865921607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:10:17.629 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:17 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:17.630 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:17 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1865921607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:10:17.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:17 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:17.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:17 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1865921607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:10:17.806 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:10:17.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:10:17.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:10:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 -- 172.21.15.64:0/1388434609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090101960 msgr2=0x7fe090101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 --2- 172.21.15.64:0/1388434609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090101960 0x7fe090101de0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe084009900 tx=0x7fe08402a410 comp rx=0 tx=0).stop 2024-05-09T09:10:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 -- 172.21.15.64:0/1388434609 shutdown_connections 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 --2- 172.21.15.64:0/1388434609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 0x7fe090108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 --2- 172.21.15.64:0/1388434609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090101960 0x7fe090101de0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 --2- 172.21.15.64:0/1388434609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe090100770 0x7fe090100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 -- 172.21.15.64:0/1388434609 >> 172.21.15.64:0/1388434609 conn(0x7fe0900fbce0 msgr2=0x7fe0900fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 -- 172.21.15.64:0/1388434609 shutdown_connections 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 -- 172.21.15.64:0/1388434609 wait complete. 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.933+0000 7fe0952a5640 1 Processor -- start 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 -- start start 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 0x7fe090069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe090101960 0x7fe09019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 0x7fe0901a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe09010c280 con 0x7fe090100770 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe09010c100 con 0x7fe090101960 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe0952a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe09010c400 con 0x7fe090102320 2024-05-09T09:10:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 0x7fe090069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 0x7fe090069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44270/0 (socket says 172.21.15.64:44270) 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 -- 172.21.15.64:0/2229949730 learned_addr learned my addr 172.21.15.64:0/2229949730 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08e7fc640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe090101960 0x7fe09019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 0x7fe0901a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 -- 172.21.15.64:0/2229949730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 msgr2=0x7fe0901a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 0x7fe0901a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 -- 172.21.15.64:0/2229949730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe090101960 msgr2=0x7fe09019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe090101960 0x7fe09019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08effd640 1 -- 172.21.15.64:0/2229949730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0901a1910 con 0x7fe090100770 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.934+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 0x7fe0901a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.935+0000 7fe08effd640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 0x7fe090069530 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe08000d950 tx=0x7fe08000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.935+0000 7fe077fff640 1 -- 172.21.15.64:0/2229949730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080017020 con 0x7fe090100770 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.935+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0901a1ba0 con 0x7fe090100770 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.935+0000 7fe077fff640 1 -- 172.21.15.64:0/2229949730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe080009c30 con 0x7fe090100770 2024-05-09T09:10:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.935+0000 7fe077fff640 1 -- 172.21.15.64:0/2229949730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08000f6b0 con 0x7fe090100770 2024-05-09T09:10:17.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.935+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0901a2110 con 0x7fe090100770 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 Processor -- start 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe077fff640 1 -- 172.21.15.64:0/2229949730 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe08001a020 con 0x7fe090100770 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 -- start start 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a6008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a6059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a6084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e0a564990 con 0x555e0a606020 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e0a50c5b0 con 0x555e0a600eb0 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe097fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555e0a560950 con 0x555e0a6004c0 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe08e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a6084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe08e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a6084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44282/0 (socket says 172.21.15.64:44282) 2024-05-09T09:10:17.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe08e7fc640 1 -- 172.21.15.64:0/3050508772 learned_addr learned my addr 172.21.15.64:0/3050508772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a6008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.938+0000 7fe08effd640 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a6059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08e7fc640 1 -- 172.21.15.64:0/3050508772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 msgr2=0x555e0a6008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08e7fc640 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a6008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08e7fc640 1 -- 172.21.15.64:0/3050508772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 msgr2=0x555e0a6059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08e7fc640 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a6059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08e7fc640 1 -- 172.21.15.64:0/3050508772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0a3407e0 con 0x555e0a606020 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a6008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe077fff640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe0780a2bc0 0x7fe0780a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08e7fc640 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a6084b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe084009900 tx=0x7fe084031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb378af38a2c98b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe0780a2bc0 0x7fe0780a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe077fff640 1 -- 172.21.15.64:0/2229949730 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe0800cedb0 con 0x7fe090100770 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe0757fa640 1 -- 172.21.15.64:0/3050508772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084035030 con 0x555e0a606020 2024-05-09T09:10:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe0757fa640 1 -- 172.21.15.64:0/3050508772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe084040070 con 0x555e0a606020 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.939+0000 7fe0757fa640 1 -- 172.21.15.64:0/3050508772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084003c90 con 0x555e0a606020 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 -- 172.21.15.64:0/3050508772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 msgr2=0x555e0a6084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a6084b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe084009900 tx=0x7fe084031dc0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe0780a2bc0 0x7fe0780a5080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x555e0a601710 tx=0x7fe088007540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 -- 172.21.15.64:0/3050508772 shutdown_connections 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a6084b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a6059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 --2- 172.21.15.64:0/3050508772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a6008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 -- 172.21.15.64:0/3050508772 >> 172.21.15.64:0/3050508772 conn(0x555e0a5c80a0 msgr2=0x555e0a5fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 -- 172.21.15.64:0/3050508772 shutdown_connections 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.940+0000 7fe097fb3740 1 -- 172.21.15.64:0/3050508772 wait complete. 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 -- start start 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 Processor -- start 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a60ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a611e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e0a608f70 con 0x555e0a606020 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e0a51fb40 con 0x555e0a600eb0 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555e0a6090f0 con 0x555e0a6004c0 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe097fb3740 10 client.0 _ll_register_callbacks cb 0x555e0a5140f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a611e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a611e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44284/0 (socket says 172.21.15.64:44284) 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 -- 172.21.15.64:0/62190779 learned_addr learned my addr 172.21.15.64:0/62190779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08effd640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08e7fc640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a60ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 -- 172.21.15.64:0/62190779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 msgr2=0x555e0a60a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 -- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 msgr2=0x555e0a60ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a60ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.941+0000 7fe08f7fe640 1 -- 172.21.15.64:0/62190779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0a3a0f70 con 0x555e0a606020 2024-05-09T09:10:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a611e70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe088007720 tx=0x7fe088004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddf6b0b565d2a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe072ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe072ff5640 1 -- 172.21.15.64:0/62190779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08801e620 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe072ff5640 1 -- 172.21.15.64:0/62190779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe08801e7c0 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe084040070 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe072ff5640 1 -- 172.21.15.64:0/62190779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe08802f050 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e0a238170 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe072ff5640 1 -- 172.21.15.64:0/62190779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe084040070 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe072ff5640 1 -- 172.21.15.64:0/62190779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe08801c8d0 con 0x555e0a606020 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 20 client.15897 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a618920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 4 client.15897 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.942+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e0a5720c0 con 0x555e0a6164c0 2024-05-09T09:10:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.943+0000 7fe08effd640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a618920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.943+0000 7fe072ff5640 10 client.15897 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.943+0000 7fe08effd640 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a618920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba9f78a18cddbfa server_cookie=5e9e9fbb0598865b in_seq=0 out_seq=0 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe072ff5640 1 -- 172.21.15.64:0/62190779 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e0a5720c0 con 0x555e0a6164c0 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe072ff5640 10 client.15897 handle_command_reply: tid=0 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 client.15897 shutdown 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 msgr2=0x555e0a611e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a611e70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe088007720 tx=0x7fe088004a40 comp rx=0 tx=0).stop 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 shutdown_connections 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a618920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe072ff5640 0 client.15897 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a611e70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a60ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 --2- 172.21.15.64:0/62190779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 >> 172.21.15.64:0/62190779 conn(0x555e0a5c80a0 msgr2=0x555e0a5fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:17.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 shutdown_connections 2024-05-09T09:10:17.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 1 -- 172.21.15.64:0/62190779 wait complete. 2024-05-09T09:10:17.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.944+0000 7fe097fb3740 20 client.15897 trim_cache size 0 max 16384 2024-05-09T09:10:17.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.963+0000 7fe097fb3740 1 Processor -- start 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.963+0000 7fe097fb3740 1 -- start start 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a5fee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a611e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a5ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe097fb3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901aef20 con 0x555e0a606020 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe097fb3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901ac7c0 con 0x555e0a600eb0 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe097fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901aa610 con 0x555e0a6004c0 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a5ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a5ff370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44286/0 (socket says 172.21.15.64:44286) 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a5fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 -- 172.21.15.64:0/2346436129 learned_addr learned my addr 172.21.15.64:0/2346436129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 -- 172.21.15.64:0/2346436129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 msgr2=0x555e0a5fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a5fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 -- 172.21.15.64:0/2346436129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 msgr2=0x555e0a611e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a611e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 -- 172.21.15.64:0/2346436129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0a609200 con 0x555e0a606020 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a5ff370 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe088011010 tx=0x7fe0880075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f60a499062b894e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.964+0000 7fe074ff9640 1 -- 172.21.15.64:0/2346436129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088004cb0 con 0x555e0a606020 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe074ff9640 1 -- 172.21.15.64:0/2346436129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe088004e50 con 0x555e0a606020 2024-05-09T09:10:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe074ff9640 1 -- 172.21.15.64:0/2346436129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088005320 con 0x555e0a606020 2024-05-09T09:10:17.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 -- 172.21.15.64:0/2346436129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 msgr2=0x555e0a5ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a5ff370 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe088011010 tx=0x7fe0880075d0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 -- 172.21.15.64:0/2346436129 shutdown_connections 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a5ff370 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a611e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 --2- 172.21.15.64:0/2346436129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a5fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.965+0000 7fe097fb3740 1 -- 172.21.15.64:0/2346436129 >> 172.21.15.64:0/2346436129 conn(0x555e0a5c80a0 msgr2=0x555e0a602310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 -- 172.21.15.64:0/2346436129 shutdown_connections 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 -- 172.21.15.64:0/2346436129 wait complete. 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 -- start start 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 Processor -- start 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a610000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a610ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901aac80 con 0x555e0a606020 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901ac0f0 con 0x555e0a600eb0 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0901ab350 con 0x555e0a6004c0 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe097fb3740 10 client.0 _ll_register_callbacks cb 0x7fe088007b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe08f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a610ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe08f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a610ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44300/0 (socket says 172.21.15.64:44300) 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe08f7fe640 1 -- 172.21.15.64:0/4142564068 learned_addr learned my addr 172.21.15.64:0/4142564068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe08effd640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.966+0000 7fe08e7fc640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a610000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe08f7fe640 1 -- 172.21.15.64:0/4142564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 msgr2=0x555e0a60fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe08f7fe640 1 -- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 msgr2=0x555e0a610000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a610000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe08f7fe640 1 -- 172.21.15.64:0/4142564068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0a6114c0 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe08f7fe640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a610ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe08801cc70 tx=0x7fe08801c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234440b61ed9017 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe072ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe072ff5640 1 -- 172.21.15.64:0/4142564068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088043070 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe072ff5640 1 -- 172.21.15.64:0/4142564068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe0880050a0 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe088004e50 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe072ff5640 1 -- 172.21.15.64:0/4142564068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe08804f050 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe097fb3740 10 client.15903 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.967+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e0a67cf40 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe072ff5640 1 -- 172.21.15.64:0/4142564068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe088004e50 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe072ff5640 1 -- 172.21.15.64:0/4142564068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe088032070 con 0x555e0a606020 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe097fb3740 10 client.15903 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe097fb3740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe097fb3740 20 client.15903 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe097fb3740 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a67f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe097fb3740 4 client.15903 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555e0a56c410 con 0x555e0a6164c0 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe08effd640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a67f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe072ff5640 10 client.15903 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.968+0000 7fe08effd640 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a67f8e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a88757c9f915a6a server_cookie=cfbdf7f742d0b9a0 in_seq=0 out_seq=0 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.969+0000 7fe072ff5640 1 -- 172.21.15.64:0/4142564068 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555e0a56c410 con 0x555e0a6164c0 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.969+0000 7fe072ff5640 10 client.15903 handle_command_reply: tid=0 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 client.15903 shutdown 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 msgr2=0x555e0a610ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a610ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe08801cc70 tx=0x7fe08801c5c0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 shutdown_connections 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e0a6164c0 0x555e0a67f8e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e0a606020 0x555e0a610ed0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e0a600eb0 0x555e0a610000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 --2- 172.21.15.64:0/4142564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e0a6004c0 0x555e0a60fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe072ff5640 0 client.15903 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 >> 172.21.15.64:0/4142564068 conn(0x555e0a5c80a0 msgr2=0x555e0a5fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 shutdown_connections 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 1 -- 172.21.15.64:0/4142564068 wait complete. 2024-05-09T09:10:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.970+0000 7fe097fb3740 20 client.15903 trim_cache size 0 max 16384 2024-05-09T09:10:17.975 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:10:17.975 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:10:17.975 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "c5acb97c-51fe-4910-9178-f7a0ad45400c", 2024-05-09T09:10:17.975 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:10:17.975 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe0780a2bc0 msgr2=0x7fe0780a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe0780a2bc0 0x7fe0780a5080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x555e0a601710 tx=0x7fe088007540 comp rx=0 tx=0).stop 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 msgr2=0x7fe090069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 0x7fe090069530 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe08000d950 tx=0x7fe08000de10 comp rx=0 tx=0).stop 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 shutdown_connections 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe0780a2bc0 0x7fe0780a5080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe090102320 0x7fe0901a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe090101960 0x7fe09019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 --2- 172.21.15.64:0/2229949730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe090100770 0x7fe090069530 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 >> 172.21.15.64:0/2229949730 conn(0x7fe0900fbce0 msgr2=0x7fe090104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:17.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 shutdown_connections 2024-05-09T09:10:17.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:17.971+0000 7fe0952a5640 1 -- 172.21.15.64:0/2229949730 wait complete. 2024-05-09T09:10:17.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:10:17.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 -- 172.21.15.64:0/890048849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8100770 msgr2=0x7ffaa8100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 --2- 172.21.15.64:0/890048849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8100770 0x7ffaa8100b90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009900 tx=0x7ffaa402a410 comp rx=0 tx=0).stop 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 -- 172.21.15.64:0/890048849 shutdown_connections 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 --2- 172.21.15.64:0/890048849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa8102320 0x7ffaa8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 --2- 172.21.15.64:0/890048849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa8101960 0x7ffaa8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 --2- 172.21.15.64:0/890048849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8100770 0x7ffaa8100b90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 -- 172.21.15.64:0/890048849 >> 172.21.15.64:0/890048849 conn(0x7ffaa80fbce0 msgr2=0x7ffaa80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 -- 172.21.15.64:0/890048849 shutdown_connections 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.122+0000 7ffaaff90640 1 -- 172.21.15.64:0/890048849 wait complete. 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 Processor -- start 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 -- start start 2024-05-09T09:10:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa8100770 0x7ffaa8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa8101960 0x7ffaa819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 0x7ffaa819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa810c2a0 con 0x7ffaa8102320 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa810c120 con 0x7ffaa8100770 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaaff90640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa810c420 con 0x7ffaa8101960 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 0x7ffaa819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 0x7ffaa819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44322/0 (socket says 172.21.15.64:44322) 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 -- 172.21.15.64:0/1732488519 learned_addr learned my addr 172.21.15.64:0/1732488519 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa8100770 0x7ffaa8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaad52b640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa8101960 0x7ffaa819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 -- 172.21.15.64:0/1732488519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa8101960 msgr2=0x7ffaa819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa8101960 0x7ffaa819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 -- 172.21.15.64:0/1732488519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa8100770 msgr2=0x7ffaa8069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa8100770 0x7ffaa8069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 -- 172.21.15.64:0/1732488519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa819f740 con 0x7ffaa8102320 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.123+0000 7ffaae52d640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 0x7ffaa819ee80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000eb10 tx=0x7ffaa000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.124+0000 7ffa9effd640 1 -- 172.21.15.64:0/1732488519 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0004b20 con 0x7ffaa8102320 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.124+0000 7ffa9effd640 1 -- 172.21.15.64:0/1732488519 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaa0004cc0 con 0x7ffaa8102320 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.124+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa819fa30 con 0x7ffaa8102320 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.124+0000 7ffa9effd640 1 -- 172.21.15.64:0/1732488519 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0010690 con 0x7ffaa8102320 2024-05-09T09:10:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.124+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa819ff80 con 0x7ffaa8102320 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.126+0000 7ffa9effd640 1 -- 172.21.15.64:0/1732488519 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffaa0005020 con 0x7ffaa8102320 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.126+0000 7ffab2d7e740 1 Processor -- start 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 -- start start 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 0x55df7d6b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7d30 0x55df7d6bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55df7d617d00 con 0x55df7d6b7340 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55df7d5bf4c0 con 0x55df7d6bcea0 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55df7d613cc0 con 0x55df7d6b7d30 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffa9effd640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa940a2b90 0x7ffa940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 0x55df7d6b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7d30 0x55df7d6bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 0x55df7d6b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44326/0 (socket says 172.21.15.64:44326) 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 -- 172.21.15.64:0/3990689609 learned_addr learned my addr 172.21.15.64:0/3990689609 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaae52d640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa940a2b90 0x7ffa940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffa9effd640 1 -- 172.21.15.64:0/1732488519 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffaa00cfa90 con 0x7ffaa8102320 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 -- 172.21.15.64:0/3990689609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7d30 msgr2=0x55df7d6bc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7d30 0x55df7d6bc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 -- 172.21.15.64:0/3990689609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 msgr2=0x55df7d6bf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6bf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaad52b640 1 -- 172.21.15.64:0/3990689609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7d3f57d0 con 0x55df7d6b7340 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.127+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7d30 0x55df7d6bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.128+0000 7ffaae52d640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa940a2b90 0x7ffa940a5050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffaa009bc60 tx=0x7ffaa006aee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.128+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 0x55df7d6b7760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffa98013630 tx=0x7ffa98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da470a7ff2ebf963 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.128+0000 7ffa93fff640 1 -- 172.21.15.64:0/3990689609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800f500 con 0x55df7d6b7340 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.128+0000 7ffa93fff640 1 -- 172.21.15.64:0/3990689609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa9800fb50 con 0x55df7d6b7340 2024-05-09T09:10:18.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.128+0000 7ffa93fff640 1 -- 172.21.15.64:0/3990689609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980109b0 con 0x55df7d6b7340 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3990689609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 msgr2=0x55df7d6b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 0x55df7d6b7760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffa98013630 tx=0x7ffa98005f90 comp rx=0 tx=0).stop 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3990689609 shutdown_connections 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6bf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7d30 0x55df7d6bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3990689609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7340 0x55df7d6b7760 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3990689609 >> 172.21.15.64:0/3990689609 conn(0x55df7d6ae5a0 msgr2=0x55df7d6ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3990689609 shutdown_connections 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3990689609 wait complete. 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 -- start start 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.129+0000 7ffab2d7e740 1 Processor -- start 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55df7d6bfe50 con 0x55df7d6b7d30 2024-05-09T09:10:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55df7d5d2b90 con 0x55df7d6bcea0 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55df7d6bffd0 con 0x55df7d6b7340 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 10 client.0 _ll_register_callbacks cb 0x55df7d60eda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaad52b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44342/0 (socket says 172.21.15.64:44342) 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3745154788 learned_addr learned my addr 172.21.15.64:0/3745154788 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3745154788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 msgr2=0x55df7d6c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 msgr2=0x55df7d6c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3745154788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7d32b3f0 con 0x55df7d6b7d30 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c1c10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffaa81017c0 tx=0x7ffaa4031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577ac1fc9ac1a8d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffa917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffa917fa640 1 -- 172.21.15.64:0/3745154788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4035040 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffa917fa640 1 -- 172.21.15.64:0/3745154788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffaa4004040 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.130+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa9800fb50 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffa917fa640 1 -- 172.21.15.64:0/3745154788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4004240 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df7d2eb0f0 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffa917fa640 1 -- 172.21.15.64:0/3745154788 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa9800fb50 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffa917fa640 1 -- 172.21.15.64:0/3745154788 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ffaa4044020 con 0x55df7d6b7d30 2024-05-09T09:10:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 20 client.15915 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d6cd620 0x55df7d6cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 4 client.15915 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df7d625430 con 0x55df7d6cd620 2024-05-09T09:10:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.131+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d6cd620 0x55df7d6cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.132+0000 7ffa917fa640 10 client.15915 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.132+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d6cd620 0x55df7d6cfa60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b890f167857088ed server_cookie=d1cef2698be23bb in_seq=0 out_seq=0 2024-05-09T09:10:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffa917fa640 1 -- 172.21.15.64:0/3745154788 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df7d625430 con 0x55df7d6cd620 2024-05-09T09:10:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffa917fa640 10 client.15915 handle_command_reply: tid=0 2024-05-09T09:10:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 client.15915 shutdown 2024-05-09T09:10:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 msgr2=0x55df7d6c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c1c10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffaa81017c0 tx=0x7ffaa4031e60 comp rx=0 tx=0).stop 2024-05-09T09:10:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 shutdown_connections 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d6cd620 0x55df7d6cfa60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c1c10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3745154788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffa917fa640 0 client.15915 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 >> 172.21.15.64:0/3745154788 conn(0x55df7d6ae5a0 msgr2=0x55df7d6ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 shutdown_connections 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3745154788 wait complete. 2024-05-09T09:10:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.133+0000 7ffab2d7e740 20 client.15915 trim_cache size 0 max 16384 2024-05-09T09:10:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.152+0000 7ffab2d7e740 1 Processor -- start 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 -- start start 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6b7340 0x55df7d6b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6bcea0 0x55df7d6b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81aab70 con 0x55df7d6b7d30 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81a8410 con 0x55df7d6b7340 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81a6260 con 0x55df7d6bcea0 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44354/0 (socket says 172.21.15.64:44354) 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaad52b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6b7340 0x55df7d6b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaadd2c640 1 -- 172.21.15.64:0/2060802418 learned_addr learned my addr 172.21.15.64:0/2060802418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaad52b640 1 -- 172.21.15.64:0/2060802418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6bcea0 msgr2=0x55df7d6b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaad52b640 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6bcea0 0x55df7d6b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaad52b640 1 -- 172.21.15.64:0/2060802418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 msgr2=0x55df7d6c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaad52b640 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaad52b640 1 -- 172.21.15.64:0/2060802418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7d6d00e0 con 0x55df7d6b7340 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.153+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.154+0000 7ffaad52b640 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6b7340 0x55df7d6b90b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55df7d6d0310 tx=0x7ffa98009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d53f8bab28da4ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.154+0000 7ffa937fe640 1 -- 172.21.15.64:0/2060802418 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980050c0 con 0x55df7d6b7340 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.154+0000 7ffa937fe640 1 -- 172.21.15.64:0/2060802418 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa98009a40 con 0x55df7d6b7340 2024-05-09T09:10:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.154+0000 7ffa937fe640 1 -- 172.21.15.64:0/2060802418 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980035f0 con 0x55df7d6b7340 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 -- 172.21.15.64:0/2060802418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6b7340 msgr2=0x55df7d6b90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6b7340 0x55df7d6b90b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55df7d6d0310 tx=0x7ffa98009310 comp rx=0 tx=0).stop 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 -- 172.21.15.64:0/2060802418 shutdown_connections 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6bcea0 0x55df7d6b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/2060802418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6b7340 0x55df7d6b90b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 -- 172.21.15.64:0/2060802418 >> 172.21.15.64:0/2060802418 conn(0x55df7d6ae5a0 msgr2=0x55df7d6c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 -- 172.21.15.64:0/2060802418 shutdown_connections 2024-05-09T09:10:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 -- 172.21.15.64:0/2060802418 wait complete. 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 -- start start 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.155+0000 7ffab2d7e740 1 Processor -- start 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81a68d0 con 0x55df7d6b7d30 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81a7d40 con 0x55df7d6bcea0 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81a6fa0 con 0x55df7d6b7340 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffab2d7e740 10 client.0 _ll_register_callbacks cb 0x7ffaa4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44364/0 (socket says 172.21.15.64:44364) 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3268433400 learned_addr learned my addr 172.21.15.64:0/3268433400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3268433400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 msgr2=0x55df7d6c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 msgr2=0x55df7d6c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaadd2c640 1 -- 172.21.15.64:0/3268433400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7d6c7e10 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.156+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffaadd2c640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c6810 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffaa402a930 tx=0x7ffaa4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbeacc78b8a651b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffa917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffa917fa640 1 -- 172.21.15.64:0/3268433400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4058060 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffa917fa640 1 -- 172.21.15.64:0/3268433400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffaa403cd70 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa98009a40 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffa917fa640 1 -- 172.21.15.64:0/3268433400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4035070 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffab2d7e740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df7d733c50 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffa917fa640 1 -- 172.21.15.64:0/3268433400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa98009a40 con 0x55df7d6b7d30 2024-05-09T09:10:18.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.157+0000 7ffa917fa640 1 -- 172.21.15.64:0/3268433400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ffaa4044040 con 0x55df7d6b7d30 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffab2d7e740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffab2d7e740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffab2d7e740 20 client.15921 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d7346e0 0x55df7d736b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffab2d7e740 4 client.15921 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df7d61f780 con 0x55df7d7346e0 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d7346e0 0x55df7d736b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffa917fa640 10 client.15921 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.158+0000 7ffaad52b640 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d7346e0 0x55df7d736b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b672f98d96a769 server_cookie=2cfe707e351dafdf in_seq=0 out_seq=0 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffa917fa640 1 -- 172.21.15.64:0/3268433400 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55df7d61f780 con 0x55df7d7346e0 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffa917fa640 10 client.15921 handle_command_reply: tid=0 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 client.15921 shutdown 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 msgr2=0x55df7d6c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c6810 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffaa402a930 tx=0x7ffaa4004610 comp rx=0 tx=0).stop 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 shutdown_connections 2024-05-09T09:10:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55df7d7346e0 0x55df7d736b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df7d6bcea0 0x55df7d6c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df7d6b7d30 0x55df7d6c6810 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 --2- 172.21.15.64:0/3268433400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df7d6b7340 0x55df7d6c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffa917fa640 0 client.15921 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 >> 172.21.15.64:0/3268433400 conn(0x55df7d6ae5a0 msgr2=0x55df7d6c1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 shutdown_connections 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 1 -- 172.21.15.64:0/3268433400 wait complete. 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.159+0000 7ffab2d7e740 20 client.15921 trim_cache size 0 max 16384 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:10:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "c5acb97c-51fe-4910-9178-f7a0ad45400c": { 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "c5acb97c-51fe-4910-9178-f7a0ad45400c", 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:10:18.164 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa940a2b90 msgr2=0x7ffa940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa940a2b90 0x7ffa940a5050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffaa009bc60 tx=0x7ffaa006aee0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 msgr2=0x7ffaa819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 0x7ffaa819ee80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000eb10 tx=0x7ffaa000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 shutdown_connections 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa940a2b90 0x7ffa940a5050 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffaa8102320 0x7ffaa819ee80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa8101960 0x7ffaa819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 --2- 172.21.15.64:0/1732488519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa8100770 0x7ffaa8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 >> 172.21.15.64:0/1732488519 conn(0x7ffaa80fbce0 msgr2=0x7ffaa8104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 shutdown_connections 2024-05-09T09:10:18.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:18.160+0000 7ffaaff90640 1 -- 172.21.15.64:0/1732488519 wait complete. 2024-05-09T09:10:18.171 INFO:tasks.cephfs.filesystem:scrub status for tag:c5acb97c-51fe-4910-9178-f7a0ad45400c - {'path': '/', 'tag': 'c5acb97c-51fe-4910-9178-f7a0ad45400c', 'options': 'recursive,force'} 2024-05-09T09:10:18.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:18 smithi139 ceph-mon[33599]: pgmap v405: 129 pgs: 129 active+clean; 817 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 41 op/s 2024-05-09T09:10:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:18 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:10:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:18 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:18 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:18.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:18 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:18.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:18 smithi064 ceph-mon[30149]: pgmap v405: 129 pgs: 129 active+clean; 817 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 41 op/s 2024-05-09T09:10:18.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:18 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:10:18.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:18 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:18.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:18 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:18.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:18 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:18.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:18 smithi179 ceph-mon[34086]: pgmap v405: 129 pgs: 129 active+clean; 817 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 41 op/s 2024-05-09T09:10:18.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:18 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:10:18.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:18 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:18.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:18 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:18.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:18 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 -- 172.21.15.64:0/2579183764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0868102370 msgr2=0x7f0868104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 --2- 172.21.15.64:0/2579183764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0868102370 0x7f0868104760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0858009900 tx=0x7f085802a410 comp rx=0 tx=0).stop 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 -- 172.21.15.64:0/2579183764 shutdown_connections 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 --2- 172.21.15.64:0/2579183764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681075d0 0x7f08681099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 --2- 172.21.15.64:0/2579183764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868104ca0 0x7f0868107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 --2- 172.21.15.64:0/2579183764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0868102370 0x7f0868104760 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.856+0000 7f086e493640 1 -- 172.21.15.64:0/2579183764 >> 172.21.15.64:0/2579183764 conn(0x7f08680fbd00 msgr2=0x7f08680fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.857+0000 7f086e493640 1 -- 172.21.15.64:0/2579183764 shutdown_connections 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.857+0000 7f086e493640 1 -- 172.21.15.64:0/2579183764 wait complete. 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.857+0000 7f086e493640 1 Processor -- start 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.857+0000 7f086e493640 1 -- start start 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086e493640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 0x7f0868197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086e493640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868104ca0 0x7f0868197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086e493640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08681075d0 0x7f086819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086e493640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f086810d3e0 con 0x7f08681075d0 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086e493640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f086810d260 con 0x7f0868104ca0 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086e493640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f086810d560 con 0x7f0868102370 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08681075d0 0x7f086819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 0x7f0868197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 0x7f0868197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50800/0 (socket says 172.21.15.64:50800) 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 -- 172.21.15.64:0/2577677089 learned_addr learned my addr 172.21.15.64:0/2577677089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f08677fe640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868104ca0 0x7f0868197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 -- 172.21.15.64:0/2577677089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868104ca0 msgr2=0x7f0868197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868104ca0 0x7f0868197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 -- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08681075d0 msgr2=0x7f086819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08681075d0 0x7f086819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 -- 172.21.15.64:0/2577677089 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086819f640 con 0x7f0868102370 2024-05-09T09:10:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f086ca30640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08681075d0 0x7f086819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.858+0000 7f0867fff640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 0x7f0868197a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0858005940 tx=0x7f08580042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.859+0000 7f08657fa640 1 -- 172.21.15.64:0/2577677089 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085803f030 con 0x7f0868102370 2024-05-09T09:10:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.859+0000 7f08657fa640 1 -- 172.21.15.64:0/2577677089 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08580318f0 con 0x7f0868102370 2024-05-09T09:10:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.859+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086819f8d0 con 0x7f0868102370 2024-05-09T09:10:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.859+0000 7f08657fa640 1 -- 172.21.15.64:0/2577677089 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858035070 con 0x7f0868102370 2024-05-09T09:10:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.859+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086819fdc0 con 0x7f0868102370 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 Processor -- start 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08657fa640 1 -- 172.21.15.64:0/2577677089 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0858042030 con 0x7f0868102370 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 -- start start 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a4459c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a445a1db0 0x560a445a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a44500340 con 0x560a4459cc40 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a444a8290 con 0x560a445a1db0 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08711a0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a444fc300 con 0x560a4459c250 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f0867fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f086ca30640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a4459c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f0867fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44376/0 (socket says 172.21.15.64:44376) 2024-05-09T09:10:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f0867fff640 1 -- 172.21.15.64:0/1732726307 learned_addr learned my addr 172.21.15.64:0/1732726307 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f08677fe640 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a445a1db0 0x560a445a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f086ca30640 1 -- 172.21.15.64:0/1732726307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a445a1db0 msgr2=0x560a445a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f086ca30640 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a445a1db0 0x560a445a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f086ca30640 1 -- 172.21.15.64:0/1732726307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 msgr2=0x560a445a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f086ca30640 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.862+0000 7f086ca30640 1 -- 172.21.15.64:0/1732726307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a444cc8e0 con 0x560a4459c250 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f08657fa640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08500a2c30 0x7f08500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f0867fff640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08500a2c30 0x7f08500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f086ca30640 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a4459c670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0860005fd0 tx=0x7f0860011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5264f6c57bf10d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f08657fa640 1 -- 172.21.15.64:0/2577677089 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f08580efc90 con 0x7f0868102370 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f084e7fc640 1 -- 172.21.15.64:0/1732726307 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004d20 con 0x560a4459c250 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f084e7fc640 1 -- 172.21.15.64:0/1732726307 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086000f9a0 con 0x560a4459c250 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.863+0000 7f084e7fc640 1 -- 172.21.15.64:0/1732726307 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004470 con 0x560a4459c250 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 -- 172.21.15.64:0/1732726307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 msgr2=0x560a4459c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f0867fff640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08500a2c30 0x7f08500a50f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f08580b8b30 tx=0x7f08580f3770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a4459c670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0860005fd0 tx=0x7f0860011bf0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 -- 172.21.15.64:0/1732726307 shutdown_connections 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a445a1db0 0x560a445a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 --2- 172.21.15.64:0/1732726307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a4459c670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 -- 172.21.15.64:0/1732726307 >> 172.21.15.64:0/1732726307 conn(0x560a445978c0 msgr2=0x560a44597cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 -- 172.21.15.64:0/1732726307 shutdown_connections 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 -- 172.21.15.64:0/1732726307 wait complete. 2024-05-09T09:10:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.864+0000 7f08711a0740 1 -- start start 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 Processor -- start 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a445a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459cc40 0x560a445a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a445a4d60 con 0x560a445a1db0 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a444bb790 con 0x560a4459cc40 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a445a4ee0 con 0x560a4459c250 2024-05-09T09:10:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08711a0740 10 client.0 _ll_register_callbacks cb 0x560a444f6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f086ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f086ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44380/0 (socket says 172.21.15.64:44380) 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f086ca30640 1 -- 172.21.15.64:0/1976741171 learned_addr learned my addr 172.21.15.64:0/1976741171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08677fe640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459cc40 0x560a445a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f086ca30640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a445a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08677fe640 1 -- 172.21.15.64:0/1976741171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 msgr2=0x560a445a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08677fe640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a445a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08677fe640 1 -- 172.21.15.64:0/1976741171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 msgr2=0x560a445ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08677fe640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.865+0000 7f08677fe640 1 -- 172.21.15.64:0/1976741171 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a44214210 con 0x560a4459cc40 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08677fe640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459cc40 0x560a445a6a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0868198f30 tx=0x7f085c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f12ce459afeda5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08477fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08477fe640 1 -- 172.21.15.64:0/1976741171 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c004320 con 0x560a4459cc40 2024-05-09T09:10:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08477fe640 1 -- 172.21.15.64:0/1976741171 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f085c011970 con 0x560a4459cc40 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086000f9a0 con 0x560a4459cc40 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08477fe640 1 -- 172.21.15.64:0/1976741171 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f085c025050 con 0x560a4459cc40 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08711a0740 10 client.25496 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a442e0620 con 0x560a4459cc40 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.866+0000 7f08477fe640 1 -- 172.21.15.64:0/1976741171 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086000f9a0 con 0x560a4459cc40 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08477fe640 1 -- 172.21.15.64:0/1976741171 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f085c00f440 con 0x560a4459cc40 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08711a0740 10 client.25496 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08711a0740 10 client.25496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08711a0740 20 client.25496 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08711a0740 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a445b2030 0x560a445b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08711a0740 4 client.25496 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a4450da70 con 0x560a445b2030 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f086ca30640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a445b2030 0x560a445b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f08477fe640 10 client.25496 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.867+0000 7f086ca30640 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a445b2030 0x560a445b4470 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e9bb3fc8623487 server_cookie=40b148d56b85d8e in_seq=0 out_seq=0 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08477fe640 1 -- 172.21.15.64:0/1976741171 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a4450da70 con 0x560a445b2030 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08477fe640 10 client.25496 handle_command_reply: tid=0 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 client.25496 shutdown 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459cc40 msgr2=0x560a445a6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459cc40 0x560a445a6a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0868198f30 tx=0x7f085c012b60 comp rx=0 tx=0).stop 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 shutdown_connections 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a445b2030 0x560a445b4470 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459cc40 0x560a445a6a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08477fe640 0 client.25496 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 --2- 172.21.15.64:0/1976741171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459c250 0x560a445a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 >> 172.21.15.64:0/1976741171 conn(0x560a445978c0 msgr2=0x560a44596a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 shutdown_connections 2024-05-09T09:10:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 1 -- 172.21.15.64:0/1976741171 wait complete. 2024-05-09T09:10:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.868+0000 7f08711a0740 20 client.25496 trim_cache size 0 max 16384 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.889+0000 7f08711a0740 1 Processor -- start 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.889+0000 7f08711a0740 1 -- start start 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08711a0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a44597ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08711a0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459cc40 0x560a445ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08711a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08711a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f08681acca0 con 0x560a445a1db0 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08711a0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08681aa540 con 0x560a4459c250 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08711a0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f08681a8390 con 0x560a4459cc40 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445b9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44382/0 (socket says 172.21.15.64:44382) 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 -- 172.21.15.64:0/258567813 learned_addr learned my addr 172.21.15.64:0/258567813 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f08677fe640 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459cc40 0x560a445ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a44597ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 -- 172.21.15.64:0/258567813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459cc40 msgr2=0x560a445ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459cc40 0x560a445ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 -- 172.21.15.64:0/258567813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 msgr2=0x560a44597ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a44597ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 -- 172.21.15.64:0/258567813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a445b4af0 con 0x560a445a1db0 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.890+0000 7f086ca30640 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445b9d80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0860014040 tx=0x7f0860009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7e4de56862caff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.891+0000 7f084dffb640 1 -- 172.21.15.64:0/258567813 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086001e040 con 0x560a445a1db0 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.891+0000 7f084dffb640 1 -- 172.21.15.64:0/258567813 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086000f9a0 con 0x560a445a1db0 2024-05-09T09:10:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.891+0000 7f084dffb640 1 -- 172.21.15.64:0/258567813 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860010d70 con 0x560a445a1db0 2024-05-09T09:10:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 -- 172.21.15.64:0/258567813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 msgr2=0x560a445b9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445b9d80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0860014040 tx=0x7f0860009500 comp rx=0 tx=0).stop 2024-05-09T09:10:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 -- 172.21.15.64:0/258567813 shutdown_connections 2024-05-09T09:10:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a445a1db0 0x560a445b9d80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4459cc40 0x560a445ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- 172.21.15.64:0/258567813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a44597ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 -- 172.21.15.64:0/258567813 >> 172.21.15.64:0/258567813 conn(0x560a445978c0 msgr2=0x560a445a1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 -- 172.21.15.64:0/258567813 shutdown_connections 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 -- 172.21.15.64:0/258567813 wait complete. 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 -- start start 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 Processor -- start 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a445aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.892+0000 7f08711a0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a445a1db0 0x560a445ac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08711a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f08681a8a00 con 0x560a4459cc40 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08711a0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08681a9e70 con 0x560a4459c250 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08711a0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f08681a90d0 con 0x560a445a1db0 2024-05-09T09:10:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08711a0740 10 client.0 _ll_register_callbacks cb 0x7f085c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44398/0 (socket says 172.21.15.64:44398) 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 -- 172.21.15.64:0/3974309766 learned_addr learned my addr 172.21.15.64:0/3974309766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f086ca30640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a445aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f086ca30640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a445a1db0 0x560a445ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 -- 172.21.15.64:0/3974309766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a445a1db0 msgr2=0x560a445ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a445a1db0 0x560a445ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 -- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 msgr2=0x560a445aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a445aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 -- 172.21.15.64:0/3974309766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4459be90 con 0x560a4459cc40 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08677fe640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445ab280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x560a44595fc0 tx=0x7f085c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844b36463a85d57a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08477fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08477fe640 1 -- 172.21.15.64:0/3974309766 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c022040 con 0x560a4459cc40 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08477fe640 1 -- 172.21.15.64:0/3974309766 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f085c005390 con 0x560a4459cc40 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.893+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086000f9a0 con 0x560a4459cc40 2024-05-09T09:10:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08477fe640 1 -- 172.21.15.64:0/3974309766 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f085c005590 con 0x560a4459cc40 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a445ad9f0 con 0x560a4459cc40 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08477fe640 1 -- 172.21.15.64:0/3974309766 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086000f9a0 con 0x560a4459cc40 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08477fe640 1 -- 172.21.15.64:0/3974309766 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f085c031020 con 0x560a4459cc40 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 20 client.15939 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a44618ad0 0x560a4461aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 4 client.15939 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a44507dc0 con 0x560a44618ad0 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.894+0000 7f086ca30640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a44618ad0 0x560a4461aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.895+0000 7f086ca30640 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a44618ad0 0x560a4461aef0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faae5447ab8a506 server_cookie=63681aa581bc128d in_seq=0 out_seq=0 2024-05-09T09:10:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.895+0000 7f08477fe640 10 client.15939 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08477fe640 1 -- 172.21.15.64:0/3974309766 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a44507dc0 con 0x560a44618ad0 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08477fe640 10 client.15939 handle_command_reply: tid=0 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 client.15939 shutdown 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 msgr2=0x560a445ab280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445ab280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x560a44595fc0 tx=0x7f085c0096c0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 shutdown_connections 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a44618ad0 0x560a4461aef0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a445a1db0 0x560a445ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a4459cc40 0x560a445ab280 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 --2- 172.21.15.64:0/3974309766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4459c250 0x560a445aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08477fe640 0 client.15939 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 >> 172.21.15.64:0/3974309766 conn(0x560a445978c0 msgr2=0x560a445a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 shutdown_connections 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 1 -- 172.21.15.64:0/3974309766 wait complete. 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.913+0000 7f08711a0740 20 client.15939 trim_cache size 0 max 16384 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:10:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:10:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08500a2c30 msgr2=0x7f08500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08500a2c30 0x7f08500a50f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f08580b8b30 tx=0x7f08580f3770 comp rx=0 tx=0).stop 2024-05-09T09:10:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 msgr2=0x7f0868197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 0x7f0868197a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0858005940 tx=0x7f08580042f0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 shutdown_connections 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08500a2c30 0x7f08500a50f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08681075d0 0x7f086819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868104ca0 0x7f0868197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 --2- 172.21.15.64:0/2577677089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0868102370 0x7f0868197a20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 >> 172.21.15.64:0/2577677089 conn(0x7f08680fbd00 msgr2=0x7f0868102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 shutdown_connections 2024-05-09T09:10:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:18.914+0000 7f086e493640 1 -- 172.21.15.64:0/2577677089 wait complete. 2024-05-09T09:10:19.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:19 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:19.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:19 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:19 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:19.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:19 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:19.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:19 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:19 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:19 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:19 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:19 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:19 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:19 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:19 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:19 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:19.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:19 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:19 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:19.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:19 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:19.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:19 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:19 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:19.796 INFO:teuthology.orchestra.run.smithi064.stderr:done in 48.56 s (drop tables 0.01 s, create tables 0.06 s, client-side generate 33.63 s, vacuum 8.13 s, primary keys 6.73 s). 2024-05-09T09:10:19.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-09T09:10:20.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:20 smithi139 ceph-mon[33599]: pgmap v406: 129 pgs: 129 active+clean; 882 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 54 MiB/s wr, 81 op/s 2024-05-09T09:10:20.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:20 smithi064 ceph-mon[30149]: pgmap v406: 129 pgs: 129 active+clean; 882 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 54 MiB/s wr, 81 op/s 2024-05-09T09:10:20.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:20 smithi179 ceph-mon[34086]: pgmap v406: 129 pgs: 129 active+clean; 882 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 54 MiB/s wr, 81 op/s 2024-05-09T09:10:21.351 INFO:teuthology.orchestra.run.smithi064.stderr:starting vacuum...end. 2024-05-09T09:10:22.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:22 smithi139 ceph-mon[33599]: pgmap v407: 129 pgs: 129 active+clean; 922 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 46 MiB/s wr, 93 op/s 2024-05-09T09:10:22.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:22 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:22.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:22 smithi064 ceph-mon[30149]: pgmap v407: 129 pgs: 129 active+clean; 922 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 46 MiB/s wr, 93 op/s 2024-05-09T09:10:22.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:22 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:22.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:22 smithi179 ceph-mon[34086]: pgmap v407: 129 pgs: 129 active+clean; 922 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 46 MiB/s wr, 93 op/s 2024-05-09T09:10:22.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:22 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:24 smithi139 ceph-mon[33599]: pgmap v408: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 44 MiB/s wr, 121 op/s 2024-05-09T09:10:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:24 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:10:24.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:24 smithi064 ceph-mon[30149]: pgmap v408: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 44 MiB/s wr, 121 op/s 2024-05-09T09:10:24.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:24 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:10:24.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:24 smithi179 ceph-mon[34086]: pgmap v408: 129 pgs: 129 active+clean; 936 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 44 MiB/s wr, 121 op/s 2024-05-09T09:10:24.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:24 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:10:25.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:25 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:10:26.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:10:26.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:10:26.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:25 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:25 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 5.0 s, 1303.2 tps, lat 71.624 ms stddev 85.155 2024-05-09T09:10:26.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:26 smithi179 ceph-mon[34086]: pgmap v409: 129 pgs: 129 active+clean; 931 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 36 MiB/s wr, 146 op/s 2024-05-09T09:10:26.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:26 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:10:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:10:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:26 smithi139 ceph-mon[33599]: pgmap v409: 129 pgs: 129 active+clean; 931 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 36 MiB/s wr, 146 op/s 2024-05-09T09:10:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:26 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:10:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:10:27.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:26 smithi064 ceph-mon[30149]: pgmap v409: 129 pgs: 129 active+clean; 931 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 36 MiB/s wr, 146 op/s 2024-05-09T09:10:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:26 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:10:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:10:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:10:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:10:28.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:28 smithi179 ceph-mon[34086]: pgmap v410: 129 pgs: 129 active+clean; 919 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 27 MiB/s wr, 194 op/s 2024-05-09T09:10:29.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:28 smithi139 ceph-mon[33599]: pgmap v410: 129 pgs: 129 active+clean; 919 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 27 MiB/s wr, 194 op/s 2024-05-09T09:10:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:28 smithi064 ceph-mon[30149]: pgmap v410: 129 pgs: 129 active+clean; 919 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 27 MiB/s wr, 194 op/s 2024-05-09T09:10:30.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:30 smithi179 ceph-mon[34086]: pgmap v411: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 25 MiB/s wr, 205 op/s 2024-05-09T09:10:31.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:30 smithi139 ceph-mon[33599]: pgmap v411: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 25 MiB/s wr, 205 op/s 2024-05-09T09:10:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:30 smithi064 ceph-mon[30149]: pgmap v411: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 25 MiB/s wr, 205 op/s 2024-05-09T09:10:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 10.0 s, 1348.0 tps, lat 73.937 ms stddev 26.219 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 -- 172.21.15.64:0/1660724927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 msgr2=0x7f6da8072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/1660724927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da8072f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009be0 tx=0x7f6da002a6c0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 -- 172.21.15.64:0/1660724927 shutdown_connections 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/1660724927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 0x7f6da810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/1660724927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8108e40 0x7f6da81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/1660724927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da8072f80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 -- 172.21.15.64:0/1660724927 >> 172.21.15.64:0/1660724927 conn(0x7f6da806e6d0 msgr2=0x7f6da8070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.066+0000 7f6dae9a5640 1 -- 172.21.15.64:0/1660724927 shutdown_connections 2024-05-09T09:10:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.067+0000 7f6dae9a5640 1 -- 172.21.15.64:0/1660724927 wait complete. 2024-05-09T09:10:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.068+0000 7f6dae9a5640 1 Processor -- start 2024-05-09T09:10:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.069+0000 7f6dae9a5640 1 -- start start 2024-05-09T09:10:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.069+0000 7f6dae9a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da81a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.069+0000 7f6dae9a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8108e40 0x7f6da81a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.069+0000 7f6dae9a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 0x7f6da81adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.069+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da81a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.069+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da81a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41156/0 (socket says 172.21.15.64:41156) 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dad9a3640 1 -- 172.21.15.64:0/2119791456 learned_addr learned my addr 172.21.15.64:0/2119791456 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6da811c010 con 0x7f6da8109800 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6da811be90 con 0x7f6da8072b60 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6da811c190 con 0x7f6da8108e40 2024-05-09T09:10:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 0x7f6da81adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8108e40 0x7f6da81a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dad9a3640 1 -- 172.21.15.64:0/2119791456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8108e40 msgr2=0x7f6da81a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8108e40 0x7f6da81a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.070+0000 7f6dad9a3640 1 -- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 msgr2=0x7f6da81adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.071+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 0x7f6da81adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.071+0000 7f6dad9a3640 1 -- 172.21.15.64:0/2119791456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da81ae340 con 0x7f6da8072b60 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.071+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 0x7f6da81adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:10:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.071+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da81a6720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6da002acb0 tx=0x7f6da0030f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.071+0000 7f6d9effd640 1 -- 172.21.15.64:0/2119791456 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0041020 con 0x7f6da8072b60 2024-05-09T09:10:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.072+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81ae5d0 con 0x7f6da8072b60 2024-05-09T09:10:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.072+0000 7f6d9effd640 1 -- 172.21.15.64:0/2119791456 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da0037040 con 0x7f6da8072b60 2024-05-09T09:10:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.072+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da81aea50 con 0x7f6da8072b60 2024-05-09T09:10:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.072+0000 7f6d9effd640 1 -- 172.21.15.64:0/2119791456 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0036070 con 0x7f6da8072b60 2024-05-09T09:10:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.075+0000 7f6d9effd640 1 -- 172.21.15.64:0/2119791456 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6da0044020 con 0x7f6da8072b60 2024-05-09T09:10:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.077+0000 7f6d9effd640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d940a2c80 0x7f6d940a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.077+0000 7f6d9effd640 1 -- 172.21.15.64:0/2119791456 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6da00f0040 con 0x7f6da8072b60 2024-05-09T09:10:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.077+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d940a2c80 0x7f6d940a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.078+0000 7f6db29f2740 1 Processor -- start 2024-05-09T09:10:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.078+0000 7f6db29f2740 1 -- start start 2024-05-09T09:10:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.078+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d940a2c80 0x7f6d940a5140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6da81a7c30 tx=0x7f6da4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 0x5637ecf678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf67f30 0x5637ecf6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6db29f2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5637ecec6340 con 0x5637ecf623d0 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6db29f2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5637ece6e290 con 0x5637ecf67f30 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6db29f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5637ecec2300 con 0x5637ecf62dc0 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 0x5637ecf678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 0x5637ecf678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:41726/0 (socket says 172.21.15.64:41726) 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 -- 172.21.15.64:0/3140190443 learned_addr learned my addr 172.21.15.64:0/3140190443 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 -- 172.21.15.64:0/3140190443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf67f30 msgr2=0x5637ecf6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf67f30 0x5637ecf6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 -- 172.21.15.64:0/3140190443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 msgr2=0x5637ecf627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad1a2640 1 -- 172.21.15.64:0/3140190443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ece928e0 con 0x5637ecf62dc0 2024-05-09T09:10:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.079+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.080+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 0x5637ecf678c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6da4011f50 tx=0x7f6da4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87dda0582f1fb858 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.080+0000 7f6d93fff640 1 -- 172.21.15.64:0/3140190443 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da401f620 con 0x5637ecf62dc0 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.080+0000 7f6d93fff640 1 -- 172.21.15.64:0/3140190443 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da401fc70 con 0x5637ecf62dc0 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.080+0000 7f6d93fff640 1 -- 172.21.15.64:0/3140190443 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4027be0 con 0x5637ecf62dc0 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 -- 172.21.15.64:0/3140190443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 msgr2=0x5637ecf678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 0x5637ecf678c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6da4011f50 tx=0x7f6da4014000 comp rx=0 tx=0).stop 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 -- 172.21.15.64:0/3140190443 shutdown_connections 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf67f30 0x5637ecf6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf62dc0 0x5637ecf678c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3140190443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 -- 172.21.15.64:0/3140190443 >> 172.21.15.64:0/3140190443 conn(0x5637ecf5da40 msgr2=0x5637ecf5de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 -- 172.21.15.64:0/3140190443 shutdown_connections 2024-05-09T09:10:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.081+0000 7f6db29f2740 1 -- 172.21.15.64:0/3140190443 wait complete. 2024-05-09T09:10:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.082+0000 7f6db29f2740 1 -- start start 2024-05-09T09:10:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.082+0000 7f6db29f2740 1 Processor -- start 2024-05-09T09:10:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf6cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6db29f2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5637ecf6aee0 con 0x5637ecf62dc0 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf6c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41180/0 (socket says 172.21.15.64:41180) 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6dad9a3640 1 -- 172.21.15.64:0/3990678775 learned_addr learned my addr 172.21.15.64:0/3990678775 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5637ece81790 con 0x5637ecf623d0 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.083+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5637ecf6b060 con 0x5637ecf67f30 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf6cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6db29f2740 10 client.0 _ll_register_callbacks cb 0x5637ecebca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dae1a4640 1 -- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 msgr2=0x5637ecf6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dae1a4640 1 -- 172.21.15.64:0/3990678775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 msgr2=0x5637ecf6cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf6cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.084+0000 7f6dae1a4640 1 -- 172.21.15.64:0/3990678775 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ecbda210 con 0x5637ecf67f30 2024-05-09T09:10:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.085+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf6cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.085+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf73bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d98010370 tx=0x7f6d98010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ab42477c7061b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.085+0000 7f6d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.085+0000 7f6d917fa640 1 -- 172.21.15.64:0/3990678775 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005220 con 0x5637ecf67f30 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da401fc70 con 0x5637ecf67f30 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6d917fa640 1 -- 172.21.15.64:0/3990678775 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d980053c0 con 0x5637ecf67f30 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6d917fa640 1 -- 172.21.15.64:0/3990678775 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d98023050 con 0x5637ecf67f30 2024-05-09T09:10:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6db29f2740 10 client.25615 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637ecca6620 con 0x5637ecf67f30 2024-05-09T09:10:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.086+0000 7f6d917fa640 1 -- 172.21.15.64:0/3990678775 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da401fc70 con 0x5637ecf67f30 2024-05-09T09:10:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6d917fa640 1 -- 172.21.15.64:0/3990678775 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6d9800f040 con 0x5637ecf67f30 2024-05-09T09:10:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6db29f2740 10 client.25615 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6db29f2740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6db29f2740 20 client.25615 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecf77f20 0x5637ecf7a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6db29f2740 4 client.25615 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637eced3a70 con 0x5637ecf77f20 2024-05-09T09:10:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.087+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecf77f20 0x5637ecf7a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.088+0000 7f6d917fa640 10 client.25615 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.088+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecf77f20 0x5637ecf7a330 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca4f86b72cf9314 server_cookie=e5c317853f746b7 in_seq=0 out_seq=0 2024-05-09T09:10:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.089+0000 7f6d917fa640 1 -- 172.21.15.64:0/3990678775 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637eced3a70 con 0x5637ecf77f20 2024-05-09T09:10:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.089+0000 7f6d917fa640 10 client.25615 handle_command_reply: tid=0 2024-05-09T09:10:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.089+0000 7f6db29f2740 1 client.25615 shutdown 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.089+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 msgr2=0x5637ecf73bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.089+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf73bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d98010370 tx=0x7f6d98010830 comp rx=0 tx=0).stop 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 shutdown_connections 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecf77f20 0x5637ecf7a330 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf73bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf6cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 --2- 172.21.15.64:0/3990678775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 >> 172.21.15.64:0/3990678775 conn(0x5637ecf5da40 msgr2=0x5637ecf5cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6d917fa640 0 client.25615 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 shutdown_connections 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 1 -- 172.21.15.64:0/3990678775 wait complete. 2024-05-09T09:10:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.090+0000 7f6db29f2740 20 client.25615 trim_cache size 0 max 16384 2024-05-09T09:10:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.121+0000 7f6db29f2740 1 Processor -- start 2024-05-09T09:10:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.121+0000 7f6db29f2740 1 -- start start 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.121+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf73bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf7a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf79d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41194/0 (socket says 172.21.15.64:41194) 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 -- 172.21.15.64:0/1064486230 learned_addr learned my addr 172.21.15.64:0/1064486230 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad1a2640 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf73bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf7a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81bba50 con 0x5637ecf62dc0 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b9290 con 0x5637ecf623d0 2024-05-09T09:10:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b70e0 con 0x5637ecf67f30 2024-05-09T09:10:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 -- 172.21.15.64:0/1064486230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 msgr2=0x5637ecf7a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf7a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 -- 172.21.15.64:0/1064486230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 msgr2=0x5637ecf73bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf73bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.122+0000 7f6dad9a3640 1 -- 172.21.15.64:0/1064486230 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ecf4f330 con 0x5637ecf623d0 2024-05-09T09:10:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.123+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf79d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6da00bc440 tx=0x7f6da0031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da426edf1606109 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.124+0000 7f6d937fe640 1 -- 172.21.15.64:0/1064486230 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00bc4b0 con 0x5637ecf623d0 2024-05-09T09:10:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.124+0000 7f6d937fe640 1 -- 172.21.15.64:0/1064486230 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da00c0860 con 0x5637ecf623d0 2024-05-09T09:10:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.124+0000 7f6d937fe640 1 -- 172.21.15.64:0/1064486230 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0004670 con 0x5637ecf623d0 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 msgr2=0x5637ecf79d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf79d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6da00bc440 tx=0x7f6da0031000 comp rx=0 tx=0).stop 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 shutdown_connections 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf7a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf62dc0 0x5637ecf73bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 --2- 172.21.15.64:0/1064486230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf623d0 0x5637ecf79d00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 >> 172.21.15.64:0/1064486230 conn(0x5637ecf5da40 msgr2=0x5637ecf5de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 shutdown_connections 2024-05-09T09:10:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 -- 172.21.15.64:0/1064486230 wait complete. 2024-05-09T09:10:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.125+0000 7f6db29f2740 1 -- start start 2024-05-09T09:10:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.126+0000 7f6db29f2740 1 Processor -- start 2024-05-09T09:10:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf70c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf62dc0 0x5637ecf71150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf70c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6dad9a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf70c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56930/0 (socket says 172.21.15.64:56930) 2024-05-09T09:10:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6dad1a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf62dc0 0x5637ecf71150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6dad1a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf62dc0 0x5637ecf71150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41200/0 (socket says 172.21.15.64:41200) 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b7750 con 0x5637ecf623d0 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b8bc0 con 0x5637ecf62dc0 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b7e20 con 0x5637ecf67f30 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6db29f2740 10 client.0 _ll_register_callbacks cb 0x7f6d98009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.127+0000 7f6dad9a3640 1 -- 172.21.15.64:0/2652728630 learned_addr learned my addr 172.21.15.64:0/2652728630 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.128+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.128+0000 7f6dae1a4640 1 -- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf62dc0 msgr2=0x5637ecf71150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.128+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf62dc0 0x5637ecf71150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.128+0000 7f6dae1a4640 1 -- 172.21.15.64:0/2652728630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 msgr2=0x5637ecf70c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.128+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf70c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.128+0000 7f6dae1a4640 1 -- 172.21.15.64:0/2652728630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ecf72700 con 0x5637ecf67f30 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.129+0000 7f6dae1a4640 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf72110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6d98012aa0 tx=0x7f6d98009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=792448df95c9339c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.129+0000 7f6d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.129+0000 7f6d917fa640 1 -- 172.21.15.64:0/2652728630 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98020040 con 0x5637ecf67f30 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.129+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da00c0860 con 0x5637ecf67f30 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.129+0000 7f6d917fa640 1 -- 172.21.15.64:0/2652728630 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d98005240 con 0x5637ecf67f30 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.130+0000 7f6d917fa640 1 -- 172.21.15.64:0/2652728630 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d98039050 con 0x5637ecf67f30 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.130+0000 7f6db29f2740 10 client.25621 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.130+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637ecf73a80 con 0x5637ecf67f30 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.130+0000 7f6d917fa640 1 -- 172.21.15.64:0/2652728630 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d9802c070 con 0x5637ecf67f30 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6d917fa640 1 -- 172.21.15.64:0/2652728630 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6d9801a430 con 0x5637ecf67f30 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6db29f2740 10 client.25621 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6db29f2740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6db29f2740 20 client.25621 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6db29f2740 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecfde7f0 0x5637ecfe0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6db29f2740 4 client.25621 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:10:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.131+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6da00c0860 con 0x5637ecfde7f0 2024-05-09T09:10:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.132+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecfde7f0 0x5637ecfe0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.132+0000 7f6dad9a3640 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecfde7f0 0x5637ecfe0c10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac06dc663cc6a521 server_cookie=65bbfbdb9a39d08c in_seq=0 out_seq=0 2024-05-09T09:10:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.132+0000 7f6d917fa640 10 client.25621 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.166+0000 7f6d917fa640 1 -- 172.21.15.64:0/2652728630 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6da00c0860 con 0x5637ecfde7f0 2024-05-09T09:10:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.166+0000 7f6d917fa640 10 client.25621 handle_command_reply: tid=0 2024-05-09T09:10:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.167+0000 7f6db29f2740 1 client.25621 shutdown 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.168+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 msgr2=0x5637ecf72110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.168+0000 7f6db29f2740 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf72110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6d98012aa0 tx=0x7f6d98009b90 comp rx=0 tx=0).stop 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 shutdown_connections 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5637ecfde7f0 0x5637ecfe0c10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5637ecf67f30 0x5637ecf72110 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5637ecf62dc0 0x5637ecf71150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 --2- 172.21.15.64:0/2652728630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637ecf623d0 0x5637ecf70c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 >> 172.21.15.64:0/2652728630 conn(0x5637ecf5da40 msgr2=0x5637ecf79d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6d917fa640 0 client.25621 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 shutdown_connections 2024-05-09T09:10:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.169+0000 7f6db29f2740 1 -- 172.21.15.64:0/2652728630 wait complete. 2024-05-09T09:10:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.170+0000 7f6db29f2740 20 client.25621 trim_cache size 0 max 16384 2024-05-09T09:10:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:10:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:10:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:10:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.173+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d940a2c80 msgr2=0x7f6d940a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.173+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d940a2c80 0x7f6d940a5140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6da81a7c30 tx=0x7f6da4009260 comp rx=0 tx=0).stop 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.173+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 msgr2=0x7f6da81a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.173+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da81a6720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6da002acb0 tx=0x7f6da0030f10 comp rx=0 tx=0).stop 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.174+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 shutdown_connections 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.174+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d940a2c80 0x7f6d940a5140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.174+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6da8109800 0x7f6da81adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.174+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8108e40 0x7f6da81a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.174+0000 7f6dae9a5640 1 --2- 172.21.15.64:0/2119791456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6da8072b60 0x7f6da81a6720 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.174+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 >> 172.21.15.64:0/2119791456 conn(0x7f6da806e6d0 msgr2=0x7f6da8070460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.175+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 shutdown_connections 2024-05-09T09:10:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:32.175+0000 7f6dae9a5640 1 -- 172.21.15.64:0/2119791456 wait complete. 2024-05-09T09:10:32.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:32 smithi179 ceph-mon[34086]: pgmap v412: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 229 op/s 2024-05-09T09:10:33.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:32 smithi139 ceph-mon[33599]: pgmap v412: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 229 op/s 2024-05-09T09:10:33.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:32 smithi064 ceph-mon[30149]: pgmap v412: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 229 op/s 2024-05-09T09:10:34.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:34 smithi179 ceph-mon[34086]: pgmap v413: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 452 op/s 2024-05-09T09:10:35.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:34 smithi139 ceph-mon[33599]: pgmap v413: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 452 op/s 2024-05-09T09:10:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:34 smithi064 ceph-mon[30149]: pgmap v413: 129 pgs: 129 active+clean; 931 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 452 op/s 2024-05-09T09:10:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 15.0 s, 1021.2 tps, lat 97.854 ms stddev 40.039 2024-05-09T09:10:36.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:36 smithi179 ceph-mon[34086]: pgmap v414: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.8 MiB/s wr, 604 op/s 2024-05-09T09:10:37.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:36 smithi139 ceph-mon[33599]: pgmap v414: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.8 MiB/s wr, 604 op/s 2024-05-09T09:10:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:36 smithi064 ceph-mon[30149]: pgmap v414: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.8 MiB/s wr, 604 op/s 2024-05-09T09:10:39.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:38 smithi139 ceph-mon[33599]: pgmap v415: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.0 MiB/s wr, 622 op/s 2024-05-09T09:10:39.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:38 smithi064 ceph-mon[30149]: pgmap v415: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.0 MiB/s wr, 622 op/s 2024-05-09T09:10:39.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:38 smithi179 ceph-mon[34086]: pgmap v415: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.0 MiB/s wr, 622 op/s 2024-05-09T09:10:41.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:40 smithi139 ceph-mon[33599]: pgmap v416: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-05-09T09:10:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:40 smithi064 ceph-mon[30149]: pgmap v416: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-05-09T09:10:41.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:40 smithi179 ceph-mon[34086]: pgmap v416: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-05-09T09:10:41.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 20.0 s, 1137.2 tps, lat 87.995 ms stddev 23.823 2024-05-09T09:10:43.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:42 smithi139 ceph-mon[33599]: pgmap v417: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:10:43.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:42 smithi064 ceph-mon[30149]: pgmap v417: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:10:43.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:42 smithi179 ceph-mon[34086]: pgmap v417: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:10:45.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:44 smithi139 ceph-mon[33599]: pgmap v418: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-05-09T09:10:45.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:44 smithi064 ceph-mon[30149]: pgmap v418: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-05-09T09:10:45.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:44 smithi179 ceph-mon[34086]: pgmap v418: 129 pgs: 129 active+clean; 932 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 -- 172.21.15.64:0/570352491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8072da0 msgr2=0x7faea8073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 --2- 172.21.15.64:0/570352491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8072da0 0x7faea8073220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae9c013080 tx=0x7fae9c0326f0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 -- 172.21.15.64:0/570352491 shutdown_connections 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 --2- 172.21.15.64:0/570352491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea810fb70 0x7faea8111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 --2- 172.21.15.64:0/570352491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8072da0 0x7faea8073220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 --2- 172.21.15.64:0/570352491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea8109440 0x7faea8072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 -- 172.21.15.64:0/570352491 >> 172.21.15.64:0/570352491 conn(0x7faea806e6d0 msgr2=0x7faea8070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 -- 172.21.15.64:0/570352491 shutdown_connections 2024-05-09T09:10:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.332+0000 7faead30e640 1 -- 172.21.15.64:0/570352491 wait complete. 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.333+0000 7faead30e640 1 Processor -- start 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.333+0000 7faead30e640 1 -- start start 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faead30e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 0x7faea819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faead30e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8109440 0x7faea819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faead30e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea810fb70 0x7faea81a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faead30e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faea8113830 con 0x7faea8072da0 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faead30e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faea81136b0 con 0x7faea810fb70 2024-05-09T09:10:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faead30e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faea81139b0 con 0x7faea8109440 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 0x7faea819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 0x7faea819df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35446/0 (socket says 172.21.15.64:35446) 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea7fff640 1 -- 172.21.15.64:0/366664728 learned_addr learned my addr 172.21.15.64:0/366664728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea77fe640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8109440 0x7faea819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faeacb0d640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea810fb70 0x7faea81a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea7fff640 1 -- 172.21.15.64:0/366664728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8109440 msgr2=0x7faea819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea7fff640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8109440 0x7faea819e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.334+0000 7faea7fff640 1 -- 172.21.15.64:0/366664728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea810fb70 msgr2=0x7faea81a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faea7fff640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea810fb70 0x7faea81a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faea7fff640 1 -- 172.21.15.64:0/366664728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea81a5ba0 con 0x7faea8072da0 2024-05-09T09:10:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faea7fff640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 0x7faea819df80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faea000efb0 tx=0x7faea000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faea57fa640 1 -- 172.21.15.64:0/366664728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea001b070 con 0x7faea8072da0 2024-05-09T09:10:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faea57fa640 1 -- 172.21.15.64:0/366664728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea000ba20 con 0x7faea8072da0 2024-05-09T09:10:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faea57fa640 1 -- 172.21.15.64:0/366664728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0016b30 con 0x7faea8072da0 2024-05-09T09:10:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea81a5e90 con 0x7faea8072da0 2024-05-09T09:10:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.335+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a62a0 con 0x7faea8072da0 2024-05-09T09:10:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.339+0000 7faeb1284740 1 Processor -- start 2024-05-09T09:10:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faea57fa640 1 -- 172.21.15.64:0/366664728 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faea001e070 con 0x7faea8072da0 2024-05-09T09:10:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 -- start start 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4be27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4be78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 0x5581d4bea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5581d4b46340 con 0x5581d4be23d0 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5581d4aee290 con 0x5581d4be7f30 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.340+0000 7faeb1284740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5581d4b42300 con 0x5581d4be2dc0 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.341+0000 7faea77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 0x5581d4bea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.341+0000 7faea7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4be78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.341+0000 7faea77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 0x5581d4bea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35578/0 (socket says 172.21.15.64:35578) 2024-05-09T09:10:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.341+0000 7faea77fe640 1 -- 172.21.15.64:0/3894547376 learned_addr learned my addr 172.21.15.64:0/3894547376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.341+0000 7faeacb0d640 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4be27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea77fe640 1 -- 172.21.15.64:0/3894547376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 msgr2=0x5581d4be78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea77fe640 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4be78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea77fe640 1 -- 172.21.15.64:0/3894547376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 msgr2=0x5581d4be27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea77fe640 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4be27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea77fe640 1 -- 172.21.15.64:0/3894547376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d4b128e0 con 0x5581d4be7f30 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea57fa640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fae900a2d90 0x7fae900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faeacb0d640 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4be27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faea57fa640 1 -- 172.21.15.64:0/366664728 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faea00cf340 con 0x7faea8072da0 2024-05-09T09:10:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.342+0000 7faeacb0d640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fae900a2d90 0x7fae900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.343+0000 7faea77fe640 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 0x5581d4bea3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faea8109160 tx=0x7fae9c039400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0af6b8e36ec880 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.343+0000 7fae8e7fc640 1 -- 172.21.15.64:0/3894547376 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c049070 con 0x5581d4be7f30 2024-05-09T09:10:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.343+0000 7fae8e7fc640 1 -- 172.21.15.64:0/3894547376 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae9c03f040 con 0x5581d4be7f30 2024-05-09T09:10:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.343+0000 7fae8e7fc640 1 -- 172.21.15.64:0/3894547376 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c03e070 con 0x5581d4be7f30 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.344+0000 7faeb1284740 1 -- 172.21.15.64:0/3894547376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 msgr2=0x5581d4bea3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.344+0000 7faeacb0d640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fae900a2d90 0x7fae900a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae98005fd0 tx=0x7fae980117b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.344+0000 7faeb1284740 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 0x5581d4bea3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faea8109160 tx=0x7fae9c039400 comp rx=0 tx=0).stop 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 -- 172.21.15.64:0/3894547376 shutdown_connections 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be7f30 0x5581d4bea3c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4be78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 --2- 172.21.15.64:0/3894547376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4be27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 -- 172.21.15.64:0/3894547376 >> 172.21.15.64:0/3894547376 conn(0x5581d4bdda40 msgr2=0x5581d4bdde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 -- 172.21.15.64:0/3894547376 shutdown_connections 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.345+0000 7faeb1284740 1 -- 172.21.15.64:0/3894547376 wait complete. 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 -- start start 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 Processor -- start 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 0x5581d4bec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4becbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be7f30 0x5581d4bf3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5581d4beae80 con 0x5581d4be7f30 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5581d4b01790 con 0x5581d4be23d0 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5581d4beb000 con 0x5581d4be2dc0 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faeb1284740 10 client.0 _ll_register_callbacks cb 0x5581d4b3ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faea7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 0x5581d4bec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faea7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 0x5581d4bec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35588/0 (socket says 172.21.15.64:35588) 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.346+0000 7faea7fff640 1 -- 172.21.15.64:0/1672969973 learned_addr learned my addr 172.21.15.64:0/1672969973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea77fe640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4becbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faeacb0d640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be7f30 0x5581d4bf3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea7fff640 1 -- 172.21.15.64:0/1672969973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 msgr2=0x5581d4becbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea7fff640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4becbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea7fff640 1 -- 172.21.15.64:0/1672969973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be7f30 msgr2=0x5581d4bf3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea7fff640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be7f30 0x5581d4bf3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea7fff640 1 -- 172.21.15.64:0/1672969973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d485a210 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea77fe640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4becbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7faea7fff640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 0x5581d4bec6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faea0097c10 tx=0x7faea00d2f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b304a6ab222cce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7fae877fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7fae877fe640 1 -- 172.21.15.64:0/1672969973 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea009b110 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.347+0000 7fae877fe640 1 -- 172.21.15.64:0/1672969973 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faea0099ab0 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.348+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c03f040 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.348+0000 7fae877fe640 1 -- 172.21.15.64:0/1672969973 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faea009beb0 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.348+0000 7faeb1284740 10 client.25529 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.348+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5581d4926620 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.348+0000 7fae877fe640 1 -- 172.21.15.64:0/1672969973 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae9c03f040 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7fae877fe640 1 -- 172.21.15.64:0/1672969973 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7faea000b040 con 0x5581d4be23d0 2024-05-09T09:10:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faeb1284740 10 client.25529 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faeb1284740 10 client.25529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faeb1284740 20 client.25529 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faeb1284740 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4bf8110 0x5581d4bfa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faeb1284740 4 client.25529 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d4b53a70 con 0x5581d4bf8110 2024-05-09T09:10:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.349+0000 7faea77fe640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4bf8110 0x5581d4bfa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.350+0000 7faea77fe640 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4bf8110 0x5581d4bfa5a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec05229cda39f56 server_cookie=6222a03515ff1b3a in_seq=0 out_seq=0 2024-05-09T09:10:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.350+0000 7fae877fe640 10 client.25529 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.351+0000 7fae877fe640 1 -- 172.21.15.64:0/1672969973 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581d4b53a70 con 0x5581d4bf8110 2024-05-09T09:10:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.351+0000 7fae877fe640 10 client.25529 handle_command_reply: tid=0 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.351+0000 7faeb1284740 1 client.25529 shutdown 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.351+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 msgr2=0x5581d4bec6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.351+0000 7faeb1284740 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 0x5581d4bec6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faea0097c10 tx=0x7faea00d2f80 comp rx=0 tx=0).stop 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 shutdown_connections 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4bf8110 0x5581d4bfa5a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be7f30 0x5581d4bf3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7fae877fe640 0 client.25529 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4becbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 --2- 172.21.15.64:0/1672969973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be23d0 0x5581d4bec6a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 >> 172.21.15.64:0/1672969973 conn(0x5581d4bdda40 msgr2=0x5581d4bdcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 shutdown_connections 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 1 -- 172.21.15.64:0/1672969973 wait complete. 2024-05-09T09:10:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.352+0000 7faeb1284740 20 client.25529 trim_cache size 0 max 16384 2024-05-09T09:10:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.385+0000 7faeb1284740 1 Processor -- start 2024-05-09T09:10:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.385+0000 7faeb1284740 1 -- start start 2024-05-09T09:10:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faeb1284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4bdda40 0x5581d4be1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faeb1284740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be23d0 0x5581d4bec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faeb1284740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be2dc0 0x5581d4bec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faeb1284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faea81b3240 con 0x5581d4bdda40 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faeb1284740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faea81b0a80 con 0x5581d4be2dc0 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faeb1284740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ae8d0 con 0x5581d4be23d0 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faea77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be23d0 0x5581d4bec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faea7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4bdda40 0x5581d4be1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faea77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be23d0 0x5581d4bec2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49428/0 (socket says 172.21.15.64:49428) 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.386+0000 7faea77fe640 1 -- 172.21.15.64:0/2013586123 learned_addr learned my addr 172.21.15.64:0/2013586123 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.387+0000 7faea7fff640 1 -- 172.21.15.64:0/2013586123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be23d0 msgr2=0x5581d4bec2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.387+0000 7faea7fff640 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be23d0 0x5581d4bec2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.387+0000 7faea7fff640 1 -- 172.21.15.64:0/2013586123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be2dc0 msgr2=0x5581d4bec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.387+0000 7faea7fff640 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be2dc0 0x5581d4bec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.387+0000 7faea7fff640 1 -- 172.21.15.64:0/2013586123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d4bcf330 con 0x5581d4bdda40 2024-05-09T09:10:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.388+0000 7faea7fff640 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4bdda40 0x5581d4be1100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faea0098130 tx=0x7faea0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd0fad984c6910d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.388+0000 7fae8dffb640 1 -- 172.21.15.64:0/2013586123 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea00d29f0 con 0x5581d4bdda40 2024-05-09T09:10:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.389+0000 7fae8dffb640 1 -- 172.21.15.64:0/2013586123 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea009b7b0 con 0x5581d4bdda40 2024-05-09T09:10:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.389+0000 7fae8dffb640 1 -- 172.21.15.64:0/2013586123 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000b040 con 0x5581d4bdda40 2024-05-09T09:10:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.389+0000 7faeb1284740 1 -- 172.21.15.64:0/2013586123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4bdda40 msgr2=0x5581d4be1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.389+0000 7faeb1284740 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4bdda40 0x5581d4be1100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faea0098130 tx=0x7faea0009b50 comp rx=0 tx=0).stop 2024-05-09T09:10:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.390+0000 7faeb1284740 1 -- 172.21.15.64:0/2013586123 shutdown_connections 2024-05-09T09:10:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.390+0000 7faeb1284740 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4be2dc0 0x5581d4bec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.390+0000 7faeb1284740 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be23d0 0x5581d4bec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.390+0000 7faeb1284740 1 --2- 172.21.15.64:0/2013586123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4bdda40 0x5581d4be1100 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.390+0000 7faeb1284740 1 -- 172.21.15.64:0/2013586123 >> 172.21.15.64:0/2013586123 conn(0x5581d4be39c0 msgr2=0x5581d4be09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.391+0000 7faeb1284740 1 -- 172.21.15.64:0/2013586123 shutdown_connections 2024-05-09T09:10:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.391+0000 7faeb1284740 1 -- 172.21.15.64:0/2013586123 wait complete. 2024-05-09T09:10:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.391+0000 7faeb1284740 1 -- start start 2024-05-09T09:10:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.391+0000 7faeb1284740 1 Processor -- start 2024-05-09T09:10:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 0x5581d4bf0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4bf1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4bf2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aef40 con 0x5581d4be23d0 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faea81b03b0 con 0x5581d4bdda40 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faea81af610 con 0x5581d4be2dc0 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faeb1284740 10 client.0 _ll_register_callbacks cb 0x7faea0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faea7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 0x5581d4bf0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faea7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 0x5581d4bf0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35604/0 (socket says 172.21.15.64:35604) 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faea7fff640 1 -- 172.21.15.64:0/1558326160 learned_addr learned my addr 172.21.15.64:0/1558326160 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.392+0000 7faea77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4bf1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faea7fff640 1 -- 172.21.15.64:0/1558326160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 msgr2=0x5581d4bf2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faea7fff640 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4bf2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faea7fff640 1 -- 172.21.15.64:0/1558326160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 msgr2=0x5581d4bf1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faea7fff640 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4bf1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faea7fff640 1 -- 172.21.15.64:0/1558326160 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d4bf2920 con 0x5581d4bdda40 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faea7fff640 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 0x5581d4bf0de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faea00a3ef0 tx=0x7faea00a4420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29b48fb773f85c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7fae877fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7fae877fe640 1 -- 172.21.15.64:0/1558326160 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000e0b0 con 0x5581d4bdda40 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea009b7b0 con 0x5581d4bdda40 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.393+0000 7fae877fe640 1 -- 172.21.15.64:0/1558326160 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faea000b570 con 0x5581d4bdda40 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7fae877fe640 1 -- 172.21.15.64:0/1558326160 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faea00ab050 con 0x5581d4bdda40 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 10 client.25532 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5581d4bf2920 con 0x5581d4bdda40 2024-05-09T09:10:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7fae877fe640 1 -- 172.21.15.64:0/1558326160 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea009b7b0 con 0x5581d4bdda40 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7fae877fe640 1 -- 172.21.15.64:0/1558326160 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7faea009a670 con 0x5581d4bdda40 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 10 client.25532 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 10 client.25532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 20 client.25532 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4be7f30 0x5581d4c613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 4 client.25532 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581d4b4ddc0 con 0x5581d4be7f30 2024-05-09T09:10:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.394+0000 7faea77fe640 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4be7f30 0x5581d4c613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.395+0000 7fae877fe640 10 client.25532 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.395+0000 7faea77fe640 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4be7f30 0x5581d4c613b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f319b9de731a20 server_cookie=d4d26334ad98371d in_seq=0 out_seq=0 2024-05-09T09:10:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.413+0000 7fae877fe640 1 -- 172.21.15.64:0/1558326160 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581d4b4ddc0 con 0x5581d4be7f30 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.413+0000 7fae877fe640 10 client.25532 handle_command_reply: tid=0 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.413+0000 7faeb1284740 1 client.25532 shutdown 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 msgr2=0x5581d4bf0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 0x5581d4bf0de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faea00a3ef0 tx=0x7faea00a4420 comp rx=0 tx=0).stop 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 shutdown_connections 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5581d4be7f30 0x5581d4c613b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581d4be2dc0 0x5581d4bf2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d4be23d0 0x5581d4bf1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 --2- 172.21.15.64:0/1558326160 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5581d4bdda40 0x5581d4bf0de0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7fae877fe640 0 client.25532 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 >> 172.21.15.64:0/1558326160 conn(0x5581d4be39c0 msgr2=0x5581d4be09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 shutdown_connections 2024-05-09T09:10:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.414+0000 7faeb1284740 1 -- 172.21.15.64:0/1558326160 wait complete. 2024-05-09T09:10:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.415+0000 7faeb1284740 20 client.25532 trim_cache size 0 max 16384 2024-05-09T09:10:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:10:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:10:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:10:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fae900a2d90 msgr2=0x7fae900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fae900a2d90 0x7fae900a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae98005fd0 tx=0x7fae980117b0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 msgr2=0x7faea819df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 0x7faea819df80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faea000efb0 tx=0x7faea000c800 comp rx=0 tx=0).stop 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 shutdown_connections 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fae900a2d90 0x7fae900a5250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea810fb70 0x7faea81a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faea8109440 0x7faea819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 --2- 172.21.15.64:0/366664728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faea8072da0 0x7faea819df80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.416+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 >> 172.21.15.64:0/366664728 conn(0x7faea806e6d0 msgr2=0x7faea810bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.417+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 shutdown_connections 2024-05-09T09:10:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:45.417+0000 7faead30e640 1 -- 172.21.15.64:0/366664728 wait complete. 2024-05-09T09:10:46.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 25.0 s, 1245.6 tps, lat 80.218 ms stddev 21.208 2024-05-09T09:10:47.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:46 smithi179 ceph-mon[34086]: pgmap v419: 129 pgs: 129 active+clean; 932 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-05-09T09:10:47.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:46 smithi139 ceph-mon[33599]: pgmap v419: 129 pgs: 129 active+clean; 932 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-05-09T09:10:47.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:46 smithi064 ceph-mon[30149]: pgmap v419: 129 pgs: 129 active+clean; 932 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-05-09T09:10:48.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:10:48.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:10:48.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 -- 172.21.15.64:0/1093873489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 msgr2=0x7fa468104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 --2- 172.21.15.64:0/1093873489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 0x7fa468104fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa450009900 tx=0x7fa45002a410 comp rx=0 tx=0).stop 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 -- 172.21.15.64:0/1093873489 shutdown_connections 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 --2- 172.21.15.64:0/1093873489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa46810bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 --2- 172.21.15.64:0/1093873489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 0x7fa468104fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 --2- 172.21.15.64:0/1093873489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa468103970 0x7fa468103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 -- 172.21.15.64:0/1093873489 >> 172.21.15.64:0/1093873489 conn(0x7fa4680feee0 msgr2=0x7fa468101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.367+0000 7fa46d668640 1 -- 172.21.15.64:0/1093873489 shutdown_connections 2024-05-09T09:10:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.368+0000 7fa46d668640 1 -- 172.21.15.64:0/1093873489 wait complete. 2024-05-09T09:10:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.368+0000 7fa46d668640 1 Processor -- start 2024-05-09T09:10:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 -- start start 2024-05-09T09:10:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa468103970 0x7fa468199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 0x7fa468199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa4681a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810f4d0 con 0x7fa468105520 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810f350 con 0x7fa468104b60 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46d668640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810f650 con 0x7fa468103970 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46ce67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa4681a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46ce67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa4681a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35508/0 (socket says 172.21.15.64:35508) 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa46ce67640 1 -- 172.21.15.64:0/2908121400 learned_addr learned my addr 172.21.15.64:0/2908121400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.369+0000 7fa467fff640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa468103970 0x7fa468199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa46ce67640 1 -- 172.21.15.64:0/2908121400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa468103970 msgr2=0x7fa468199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa46ce67640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa468103970 0x7fa468199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa46ce67640 1 -- 172.21.15.64:0/2908121400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 msgr2=0x7fa468199ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa46ce67640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 0x7fa468199ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa46ce67640 1 -- 172.21.15.64:0/2908121400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4681a15b0 con 0x7fa468105520 2024-05-09T09:10:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa46ce67640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa4681a0f40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa46000e970 tx=0x7fa46000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa4657fa640 1 -- 172.21.15.64:0/2908121400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46000ccc0 con 0x7fa468105520 2024-05-09T09:10:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa4657fa640 1 -- 172.21.15.64:0/2908121400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa46000ce60 con 0x7fa468105520 2024-05-09T09:10:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.370+0000 7fa4657fa640 1 -- 172.21.15.64:0/2908121400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460010430 con 0x7fa468105520 2024-05-09T09:10:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.371+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa468075cd0 con 0x7fa468105520 2024-05-09T09:10:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.371+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4680761d0 con 0x7fa468105520 2024-05-09T09:10:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.376+0000 7fa4657fa640 1 -- 172.21.15.64:0/2908121400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa460004ff0 con 0x7fa468105520 2024-05-09T09:10:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.376+0000 7fa4715d7740 1 Processor -- start 2024-05-09T09:10:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.376+0000 7fa4715d7740 1 -- start start 2024-05-09T09:10:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.376+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48b120 0x56264c48b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.376+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 0x56264c490610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c493110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4715d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56264c3efd00 con 0x56264c48b120 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4715d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56264c3974c0 con 0x56264c490c80 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4715d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56264c3ebcc0 con 0x56264c48bb10 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4657fa640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4540a2ce0 0x7fa4540a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa467fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 0x56264c490610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa467fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 0x56264c490610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49452/0 (socket says 172.21.15.64:49452) 2024-05-09T09:10:48.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa467fff640 1 -- 172.21.15.64:0/797187480 learned_addr learned my addr 172.21.15.64:0/797187480 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4677fe640 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c493110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4677fe640 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48b120 0x56264c48b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa4657fa640 1 -- 172.21.15.64:0/2908121400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4600cf830 con 0x7fa468105520 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.377+0000 7fa46ce67640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4540a2ce0 0x7fa4540a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa467fff640 1 -- 172.21.15.64:0/797187480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 msgr2=0x56264c493110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa467fff640 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c493110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa467fff640 1 -- 172.21.15.64:0/797187480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48b120 msgr2=0x56264c48b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa467fff640 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48b120 0x56264c48b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa467fff640 1 -- 172.21.15.64:0/797187480 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264c1cd7d0 con 0x56264c48bb10 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa4677fe640 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c493110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:48.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa46ce67640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4540a2ce0 0x7fa4540a51a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa460009380 tx=0x7fa460009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.378+0000 7fa467fff640 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 0x56264c490610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa458009f40 tx=0x7fa458010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abdf83440eca7b25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.379+0000 7fa45e7fc640 1 -- 172.21.15.64:0/797187480 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458016070 con 0x56264c48bb10 2024-05-09T09:10:48.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.379+0000 7fa45e7fc640 1 -- 172.21.15.64:0/797187480 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa458004d20 con 0x56264c48bb10 2024-05-09T09:10:48.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.379+0000 7fa45e7fc640 1 -- 172.21.15.64:0/797187480 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458016070 con 0x56264c48bb10 2024-05-09T09:10:48.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.380+0000 7fa4715d7740 1 -- 172.21.15.64:0/797187480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 msgr2=0x56264c490610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.380+0000 7fa4715d7740 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 0x56264c490610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa458009f40 tx=0x7fa458010800 comp rx=0 tx=0).stop 2024-05-09T09:10:48.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.382+0000 7fa4715d7740 1 -- 172.21.15.64:0/797187480 shutdown_connections 2024-05-09T09:10:48.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.382+0000 7fa4715d7740 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c493110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.382+0000 7fa4715d7740 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48bb10 0x56264c490610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.382+0000 7fa4715d7740 1 --2- 172.21.15.64:0/797187480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48b120 0x56264c48b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.382+0000 7fa4715d7740 1 -- 172.21.15.64:0/797187480 >> 172.21.15.64:0/797187480 conn(0x56264c486790 msgr2=0x56264c486b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.383+0000 7fa4715d7740 1 -- 172.21.15.64:0/797187480 shutdown_connections 2024-05-09T09:10:48.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.383+0000 7fa4715d7740 1 -- 172.21.15.64:0/797187480 wait complete. 2024-05-09T09:10:48.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.383+0000 7fa4715d7740 1 -- start start 2024-05-09T09:10:48.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.383+0000 7fa4715d7740 1 Processor -- start 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c495590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c495ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c49cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56264c493c30 con 0x56264c48bb10 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56264c3aab90 con 0x56264c490c80 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56264c493db0 con 0x56264c48b120 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4715d7740 10 client.0 _ll_register_callbacks cb 0x56264c3e6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4677fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c495590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4677fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c495590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49462/0 (socket says 172.21.15.64:49462) 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4677fe640 1 -- 172.21.15.64:0/1059127803 learned_addr learned my addr 172.21.15.64:0/1059127803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.384+0000 7fa4677fe640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c49cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.385+0000 7fa467fff640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c495ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.385+0000 7fa4677fe640 1 -- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 msgr2=0x56264c49cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.385+0000 7fa4677fe640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c49cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.385+0000 7fa4677fe640 1 -- 172.21.15.64:0/1059127803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 msgr2=0x56264c495ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.385+0000 7fa4677fe640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c495ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.385+0000 7fa4677fe640 1 -- 172.21.15.64:0/1059127803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56264c1033f0 con 0x56264c48b120 2024-05-09T09:10:48.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4677fe640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c495590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa450009340 tx=0x7fa450005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f70148570e8ae2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4477fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:48.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4477fe640 1 -- 172.21.15.64:0/1059127803 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450035040 con 0x56264c48b120 2024-05-09T09:10:48.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa458004d20 con 0x56264c48b120 2024-05-09T09:10:48.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4477fe640 1 -- 172.21.15.64:0/1059127803 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa45003d9c0 con 0x56264c48b120 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4477fe640 1 -- 172.21.15.64:0/1059127803 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa450052050 con 0x56264c48b120 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4715d7740 10 client.25648 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.386+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56264c0c30f0 con 0x56264c48b120 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4477fe640 1 -- 172.21.15.64:0/1059127803 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa458004d20 con 0x56264c48b120 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4477fe640 1 -- 172.21.15.64:0/1059127803 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fa45003a040 con 0x56264c48b120 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4715d7740 10 client.25648 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4715d7740 10 client.25648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4715d7740 20 client.25648 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4715d7740 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c4a18c0 0x56264c4a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4715d7740 4 client.25648 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56264c3fd430 con 0x56264c4a18c0 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.387+0000 7fa467fff640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c4a18c0 0x56264c4a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.388+0000 7fa4477fe640 10 client.25648 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:48.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.388+0000 7fa467fff640 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c4a18c0 0x56264c4a3d40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3700ec22df1ae6d6 server_cookie=d04642bda879a967 in_seq=0 out_seq=0 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.389+0000 7fa4477fe640 1 -- 172.21.15.64:0/1059127803 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56264c3fd430 con 0x56264c4a18c0 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4477fe640 10 client.25648 handle_command_reply: tid=0 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 client.25648 shutdown 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 msgr2=0x56264c495590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c495590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa450009340 tx=0x7fa450005dd0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 shutdown_connections 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c4a18c0 0x56264c4a3d40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c490c80 0x56264c49cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c495ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 --2- 172.21.15.64:0/1059127803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c495590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 >> 172.21.15.64:0/1059127803 conn(0x56264c486790 msgr2=0x56264c485a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.390+0000 7fa4477fe640 0 client.25648 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.391+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 shutdown_connections 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.391+0000 7fa4715d7740 1 -- 172.21.15.64:0/1059127803 wait complete. 2024-05-09T09:10:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.391+0000 7fa4715d7740 20 client.25648 trim_cache size 0 max 16384 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.425+0000 7fa4715d7740 1 Processor -- start 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 -- start start 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48b120 0x56264c495a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c49cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c490c80 0x56264c4818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681b2f10 con 0x56264c48bb10 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681b0750 con 0x56264c48b120 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4715d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681ae5a0 con 0x56264c490c80 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa467fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c49cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa467fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c49cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35538/0 (socket says 172.21.15.64:35538) 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa467fff640 1 -- 172.21.15.64:0/163052763 learned_addr learned my addr 172.21.15.64:0/163052763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4677fe640 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48b120 0x56264c495a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.426+0000 7fa4677fe640 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c490c80 0x56264c4818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.427+0000 7fa467fff640 1 -- 172.21.15.64:0/163052763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c490c80 msgr2=0x56264c4818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.427+0000 7fa467fff640 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c490c80 0x56264c4818c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.427+0000 7fa467fff640 1 -- 172.21.15.64:0/163052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48b120 msgr2=0x56264c495a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.427+0000 7fa467fff640 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48b120 0x56264c495a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.427+0000 7fa467fff640 1 -- 172.21.15.64:0/163052763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264c4865c0 con 0x56264c48bb10 2024-05-09T09:10:48.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.427+0000 7fa467fff640 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c49cad0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa458009f40 tx=0x7fa458009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c29a4d9538a709 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.428+0000 7fa45dffb640 1 -- 172.21.15.64:0/163052763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45802d070 con 0x56264c48bb10 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.428+0000 7fa45dffb640 1 -- 172.21.15.64:0/163052763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa458012b90 con 0x56264c48bb10 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.428+0000 7fa45dffb640 1 -- 172.21.15.64:0/163052763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458016040 con 0x56264c48bb10 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 -- 172.21.15.64:0/163052763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 msgr2=0x56264c49cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c49cad0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa458009f40 tx=0x7fa458009860 comp rx=0 tx=0).stop 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 -- 172.21.15.64:0/163052763 shutdown_connections 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c490c80 0x56264c4818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c48bb10 0x56264c49cad0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 --2- 172.21.15.64:0/163052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48b120 0x56264c495a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.429+0000 7fa4715d7740 1 -- 172.21.15.64:0/163052763 >> 172.21.15.64:0/163052763 conn(0x56264c486790 msgr2=0x56264c486b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.430+0000 7fa4715d7740 1 -- 172.21.15.64:0/163052763 shutdown_connections 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.430+0000 7fa4715d7740 1 -- 172.21.15.64:0/163052763 wait complete. 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.430+0000 7fa4715d7740 1 -- start start 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 Processor -- start 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c49a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48bb10 0x56264c49a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 0x56264c49b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681aec10 con 0x56264c490c80 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681b0080 con 0x56264c48bb10 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681af2e0 con 0x56264c48b120 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4715d7740 10 client.0 _ll_register_callbacks cb 0x7fa450009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:48.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa4677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 0x56264c49b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 0x56264c49b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35540/0 (socket says 172.21.15.64:35540) 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 -- 172.21.15.64:0/3075400545 learned_addr learned my addr 172.21.15.64:0/3075400545 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c49a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.431+0000 7fa467fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48bb10 0x56264c49a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 -- 172.21.15.64:0/3075400545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 msgr2=0x56264c49a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c49a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 -- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48bb10 msgr2=0x56264c49a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48bb10 0x56264c49a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 -- 172.21.15.64:0/3075400545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56264c477930 con 0x56264c490c80 2024-05-09T09:10:48.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa467fff640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48bb10 0x56264c49a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:48.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.432+0000 7fa4677fe640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 0x56264c49b500 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa450031fd0 tx=0x7fa4500092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbce092dd84492d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.433+0000 7fa4477fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:48.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.433+0000 7fa4477fe640 1 -- 172.21.15.64:0/3075400545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450035040 con 0x56264c490c80 2024-05-09T09:10:48.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.433+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa458012b90 con 0x56264c490c80 2024-05-09T09:10:48.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.433+0000 7fa4477fe640 1 -- 172.21.15.64:0/3075400545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa450046040 con 0x56264c490c80 2024-05-09T09:10:48.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.434+0000 7fa4477fe640 1 -- 172.21.15.64:0/3075400545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa45003be60 con 0x56264c490c80 2024-05-09T09:10:48.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.434+0000 7fa4715d7740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:48.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.434+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56264c49cca0 con 0x56264c490c80 2024-05-09T09:10:48.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4477fe640 1 -- 172.21.15.64:0/3075400545 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa458012b90 con 0x56264c490c80 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4477fe640 1 -- 172.21.15.64:0/3075400545 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fa45003b400 con 0x56264c490c80 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4715d7740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4715d7740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4715d7740 20 client.15987 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4715d7740 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c5193b0 0x56264c51b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4715d7740 4 client.15987 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56264c3f7780 con 0x56264c5193b0 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.435+0000 7fa467fff640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c5193b0 0x56264c51b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.436+0000 7fa4477fe640 10 client.15987 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:48.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.436+0000 7fa467fff640 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c5193b0 0x56264c51b7d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea126ca51724af server_cookie=46d254a21a1cac12 in_seq=0 out_seq=0 2024-05-09T09:10:48.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.437+0000 7fa4477fe640 1 -- 172.21.15.64:0/3075400545 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56264c3f7780 con 0x56264c5193b0 2024-05-09T09:10:48.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.437+0000 7fa4477fe640 10 client.15987 handle_command_reply: tid=0 2024-05-09T09:10:48.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.437+0000 7fa4715d7740 1 client.15987 shutdown 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 msgr2=0x56264c49b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 0x56264c49b500 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa450031fd0 tx=0x7fa4500092d0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 shutdown_connections 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56264c5193b0 0x56264c51b7d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56264c490c80 0x56264c49b500 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56264c48bb10 0x56264c49a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 --2- 172.21.15.64:0/3075400545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56264c48b120 0x56264c49a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 >> 172.21.15.64:0/3075400545 conn(0x56264c486790 msgr2=0x56264c4818c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4477fe640 0 client.15987 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 shutdown_connections 2024-05-09T09:10:48.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 1 -- 172.21.15.64:0/3075400545 wait complete. 2024-05-09T09:10:48.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.438+0000 7fa4715d7740 20 client.15987 trim_cache size 0 max 16384 2024-05-09T09:10:48.438 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:10:48.438 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:10:48.438 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:10:48.438 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4540a2ce0 msgr2=0x7fa4540a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4540a2ce0 0x7fa4540a51a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa460009380 tx=0x7fa460009720 comp rx=0 tx=0).stop 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 msgr2=0x7fa4681a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa4681a0f40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa46000e970 tx=0x7fa46000ed70 comp rx=0 tx=0).stop 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 shutdown_connections 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4540a2ce0 0x7fa4540a51a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa468105520 0x7fa4681a0f40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa468104b60 0x7fa468199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 --2- 172.21.15.64:0/2908121400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa468103970 0x7fa468199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.439+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 >> 172.21.15.64:0/2908121400 conn(0x7fa4680feee0 msgr2=0x7fa468107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.440+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 shutdown_connections 2024-05-09T09:10:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.440+0000 7fa46d668640 1 -- 172.21.15.64:0/2908121400 wait complete. 2024-05-09T09:10:48.455 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:10:48.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:10:48.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 -- 172.21.15.64:0/1180658465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 msgr2=0x7f5d84108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 --2- 172.21.15.64:0/1180658465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d84108bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009ca0 tx=0x7f5d7c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 -- 172.21.15.64:0/1180658465 shutdown_connections 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 --2- 172.21.15.64:0/1180658465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d84108bb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 --2- 172.21.15.64:0/1180658465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84101960 0x7f5d84101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 --2- 172.21.15.64:0/1180658465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84100770 0x7f5d84100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 -- 172.21.15.64:0/1180658465 >> 172.21.15.64:0/1180658465 conn(0x7f5d840fbce0 msgr2=0x7f5d840fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 -- 172.21.15.64:0/1180658465 shutdown_connections 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.585+0000 7f5d89710640 1 -- 172.21.15.64:0/1180658465 wait complete. 2024-05-09T09:10:48.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 Processor -- start 2024-05-09T09:10:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 -- start start 2024-05-09T09:10:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84100770 0x7f5d84199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84101960 0x7f5d8419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d841a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410c280 con 0x7f5d84102320 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410c100 con 0x7f5d84101960 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.586+0000 7f5d89710640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410c400 con 0x7f5d84100770 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d841a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d841a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35558/0 (socket says 172.21.15.64:35558) 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d837fe640 1 -- 172.21.15.64:0/199743980 learned_addr learned my addr 172.21.15.64:0/199743980 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84101960 0x7f5d8419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84100770 0x7f5d84199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 -- 172.21.15.64:0/199743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84100770 msgr2=0x7f5d84199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84100770 0x7f5d84199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 -- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 msgr2=0x7f5d841a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d841a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 -- 172.21.15.64:0/199743980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d841a1870 con 0x7f5d84101960 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d837fe640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d841a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84100770 0x7f5d84199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d827fc640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84101960 0x7f5d8419a1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800e9f0 tx=0x7f5d7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800ccc0 con 0x7f5d84101960 2024-05-09T09:10:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.587+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d841a1b60 con 0x7f5d84101960 2024-05-09T09:10:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.588+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d780108e0 con 0x7f5d84101960 2024-05-09T09:10:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.588+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78018880 con 0x7f5d84101960 2024-05-09T09:10:48.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.588+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a2100 con 0x7f5d84101960 2024-05-09T09:10:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.589+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d841a22d0 con 0x7f5d84101960 2024-05-09T09:10:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.590+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d78010a90 con 0x7f5d84101960 2024-05-09T09:10:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.591+0000 7f5d6bfff640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.591+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5d780cf7b0 con 0x7f5d84101960 2024-05-09T09:10:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.591+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:48.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.592+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d8419b130 tx=0x7f5d74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:48.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.596+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d7809bd30 con 0x7f5d84101960 2024-05-09T09:10:48.745 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:48 smithi064 ceph-mon[30149]: pgmap v420: 129 pgs: 129 active+clean; 932 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-05-09T09:10:48.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.936+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d841a24a0 con 0x7f5d84101960 2024-05-09T09:10:48.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.937+0000 7f5d6bfff640 1 -- 172.21.15.64:0/199743980 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11886 (secure 0 0 0) 0x7f5d780a0030 con 0x7f5d84101960 2024-05-09T09:10:48.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:10:48.936 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:10:25.554787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:10:48.936 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 30 2024-05-09T09:10:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d6c0a2b90 msgr2=0x7f5d6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d8419b130 tx=0x7f5d74005f90 comp rx=0 tx=0).stop 2024-05-09T09:10:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84101960 msgr2=0x7f5d8419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84101960 0x7f5d8419a1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800e9f0 tx=0x7f5d7800eeb0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 shutdown_connections 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d84102320 0x7f5d841a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d84101960 0x7f5d8419a1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 --2- 172.21.15.64:0/199743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d84100770 0x7f5d84199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 >> 172.21.15.64:0/199743980 conn(0x7f5d840fbce0 msgr2=0x7f5d840689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 shutdown_connections 2024-05-09T09:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:48.940+0000 7f5d89710640 1 -- 172.21.15.64:0/199743980 wait complete. 2024-05-09T09:10:48.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:10:49.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:48 smithi139 ceph-mon[33599]: pgmap v420: 129 pgs: 129 active+clean; 932 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-05-09T09:10:49.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 -- 172.21.15.64:0/2811283374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c074290 msgr2=0x7f539c074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 --2- 172.21.15.64:0/2811283374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c074290 0x7f539c074710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5390009860 tx=0x7f539002a340 comp rx=0 tx=0).stop 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 -- 172.21.15.64:0/2811283374 shutdown_connections 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 --2- 172.21.15.64:0/2811283374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c10de50 0x7f539c110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 --2- 172.21.15.64:0/2811283374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c074290 0x7f539c074710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 --2- 172.21.15.64:0/2811283374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c075870 0x7f539c073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 -- 172.21.15.64:0/2811283374 >> 172.21.15.64:0/2811283374 conn(0x7f539c0fce10 msgr2=0x7f539c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 -- 172.21.15.64:0/2811283374 shutdown_connections 2024-05-09T09:10:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.138+0000 7f53a2bab640 1 -- 172.21.15.64:0/2811283374 wait complete. 2024-05-09T09:10:49.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.139+0000 7f53a2bab640 1 Processor -- start 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.139+0000 7f53a2bab640 1 -- start start 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a2bab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c074290 0x7f539c10a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a2bab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c075870 0x7f539c10abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a2bab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 0x7f539c1ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a2bab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1139d0 con 0x7f539c074290 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a2bab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f539c113850 con 0x7f539c075870 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a2bab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f539c113b50 con 0x7f539c10de50 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a1148640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 0x7f539c1ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a1148640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 0x7f539c1ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49512/0 (socket says 172.21.15.64:49512) 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a0947640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c074290 0x7f539c10a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f53a1148640 1 -- 172.21.15.64:0/175233495 learned_addr learned my addr 172.21.15.64:0/175233495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.140+0000 7f539bfff640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c075870 0x7f539c10abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a1148640 1 -- 172.21.15.64:0/175233495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c075870 msgr2=0x7f539c10abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a1148640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c075870 0x7f539c10abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a1148640 1 -- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c074290 msgr2=0x7f539c10a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a1148640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c074290 0x7f539c10a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a1148640 1 -- 172.21.15.64:0/175233495 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c1ae620 con 0x7f539c10de50 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a0947640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c074290 0x7f539c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.141+0000 7f53a1148640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 0x7f539c1ae0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f539400eb10 tx=0x7f539400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.142+0000 7f5399ffb640 1 -- 172.21.15.64:0/175233495 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394004b20 con 0x7f539c10de50 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.142+0000 7f5399ffb640 1 -- 172.21.15.64:0/175233495 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5394004cc0 con 0x7f539c10de50 2024-05-09T09:10:49.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.142+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c1ae850 con 0x7f539c10de50 2024-05-09T09:10:49.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.142+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1aed20 con 0x7f539c10de50 2024-05-09T09:10:49.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.142+0000 7f5399ffb640 1 -- 172.21.15.64:0/175233495 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394010690 con 0x7f539c10de50 2024-05-09T09:10:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.146+0000 7f53a598b740 1 Processor -- start 2024-05-09T09:10:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.146+0000 7f5399ffb640 1 -- 172.21.15.64:0/175233495 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5394005020 con 0x7f539c10de50 2024-05-09T09:10:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.146+0000 7f53a598b740 1 -- start start 2024-05-09T09:10:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a598b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322068360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a598b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x55932206d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a598b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x55932206ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a598b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559321fccc40 con 0x559322067f40 2024-05-09T09:10:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a598b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559321f74580 con 0x559322068930 2024-05-09T09:10:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a598b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559321fc8c00 con 0x55932206daa0 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a0947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x55932206d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a0947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x55932206d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35720/0 (socket says 172.21.15.64:35720) 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.147+0000 7f53a0947640 1 -- 172.21.15.64:0/3114684553 learned_addr learned my addr 172.21.15.64:0/3114684553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f539bfff640 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x55932206ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f539bfff640 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322068360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f53a0947640 1 -- 172.21.15.64:0/3114684553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 msgr2=0x55932206ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f53a0947640 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x55932206ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f53a0947640 1 -- 172.21.15.64:0/3114684553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 msgr2=0x559322068360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f53a0947640 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322068360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f53a0947640 1 -- 172.21.15.64:0/3114684553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559321ce03f0 con 0x559322068930 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f539bfff640 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x55932206ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f53a0947640 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x55932206d430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f538c009cf0 tx=0x7f538c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01808173319e40f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.148+0000 7f5399ffb640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53840a2c60 0x7f53840a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.149+0000 7f5382ffd640 1 -- 172.21.15.64:0/3114684553 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c012450 con 0x559322068930 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.149+0000 7f5382ffd640 1 -- 172.21.15.64:0/3114684553 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f538c012aa0 con 0x559322068930 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.149+0000 7f539bfff640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53840a2c60 0x7f53840a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.149+0000 7f5399ffb640 1 -- 172.21.15.64:0/175233495 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f53940cfb30 con 0x7f539c10de50 2024-05-09T09:10:49.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.149+0000 7f5382ffd640 1 -- 172.21.15.64:0/3114684553 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c00fc60 con 0x559322068930 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 -- 172.21.15.64:0/3114684553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 msgr2=0x55932206d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x55932206d430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f538c009cf0 tx=0x7f538c0117b0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f539bfff640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53840a2c60 0x7f53840a5120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f539002a930 tx=0x7f539002a340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 -- 172.21.15.64:0/3114684553 shutdown_connections 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x55932206ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x55932206d430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 --2- 172.21.15.64:0/3114684553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322068360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 -- 172.21.15.64:0/3114684553 >> 172.21.15.64:0/3114684553 conn(0x5593220635b0 msgr2=0x5593220639b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 -- 172.21.15.64:0/3114684553 shutdown_connections 2024-05-09T09:10:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.150+0000 7f53a598b740 1 -- 172.21.15.64:0/3114684553 wait complete. 2024-05-09T09:10:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.151+0000 7f53a598b740 1 -- start start 2024-05-09T09:10:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.151+0000 7f53a598b740 1 Processor -- start 2024-05-09T09:10:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 0x559322072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322068930 0x559322072890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x5593220798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559322070aa0 con 0x559322068930 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559321f87c40 con 0x559322067f40 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559322070c20 con 0x55932206daa0 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a598b740 10 client.0 _ll_register_callbacks cb 0x559321cdfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 0x559322072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 0x559322072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35734/0 (socket says 172.21.15.64:35734) 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 -- 172.21.15.64:0/3493920551 learned_addr learned my addr 172.21.15.64:0/3493920551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f539bfff640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322068930 0x559322072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a1148640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x5593220798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 -- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 msgr2=0x5593220798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x5593220798d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 -- 172.21.15.64:0/3493920551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322068930 msgr2=0x559322072890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.152+0000 7f53a0947640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322068930 0x559322072890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.153+0000 7f53a0947640 1 -- 172.21.15.64:0/3493920551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559321daa7d0 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.153+0000 7f53a0947640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 0x559322072350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f538c000c00 tx=0x7f538c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d4ff7b3650d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.153+0000 7f537bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.153+0000 7f537bfff640 1 -- 172.21.15.64:0/3493920551 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c028070 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.153+0000 7f537bfff640 1 -- 172.21.15.64:0/3493920551 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f538c004240 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.153+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538c012aa0 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f537bfff640 1 -- 172.21.15.64:0/3493920551 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f538c034050 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 10 client.25565 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55932207cc40 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f537bfff640 1 -- 172.21.15.64:0/3493920551 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f538c012aa0 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f537bfff640 1 -- 172.21.15.64:0/3493920551 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f538c0035f0 con 0x559322067f40 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 10 client.25565 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 10 client.25565 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:10:49.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 20 client.25565 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55932207e640 0x559322080ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 4 client.25565 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559321fda370 con 0x55932207e640 2024-05-09T09:10:49.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.154+0000 7f539bfff640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55932207e640 0x559322080ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.155+0000 7f539bfff640 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55932207e640 0x559322080ae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c38e6f3aea1a6f7d server_cookie=34edecce0cf49723 in_seq=0 out_seq=0 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.155+0000 7f537bfff640 10 client.25565 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f537bfff640 1 -- 172.21.15.64:0/3493920551 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559321fda370 con 0x55932207e640 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f537bfff640 10 client.25565 handle_command_reply: tid=0 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 client.25565 shutdown 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 msgr2=0x559322072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 0x559322072350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f538c000c00 tx=0x7f538c005740 comp rx=0 tx=0).stop 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 shutdown_connections 2024-05-09T09:10:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55932207e640 0x559322080ae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55932206daa0 0x5593220798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322068930 0x559322072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f537bfff640 0 client.25565 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 --2- 172.21.15.64:0/3493920551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322067f40 0x559322072350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.156+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 >> 172.21.15.64:0/3493920551 conn(0x5593220635b0 msgr2=0x5593220627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.157+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 shutdown_connections 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.157+0000 7f53a598b740 1 -- 172.21.15.64:0/3493920551 wait complete. 2024-05-09T09:10:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.157+0000 7f53a598b740 20 client.25565 trim_cache size 0 max 16384 2024-05-09T09:10:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.182+0000 7f53a598b740 1 Processor -- start 2024-05-09T09:10:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 -- start start 2024-05-09T09:10:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322067f40 0x559322071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x559322079890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55932206daa0 0x559322071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1bbce0 con 0x55932206daa0 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f539bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x559322079890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f539bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x559322079890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35738/0 (socket says 172.21.15.64:35738) 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f539bfff640 1 -- 172.21.15.64:0/704639194 learned_addr learned my addr 172.21.15.64:0/704639194 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a1148640 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55932206daa0 0x559322071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a0947640 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322067f40 0x559322071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1b9520 con 0x559322068930 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.183+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1b7370 con 0x559322067f40 2024-05-09T09:10:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a1148640 1 -- 172.21.15.64:0/704639194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322067f40 msgr2=0x559322071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a1148640 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322067f40 0x559322071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a1148640 1 -- 172.21.15.64:0/704639194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 msgr2=0x559322079890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a1148640 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x559322079890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a1148640 1 -- 172.21.15.64:0/704639194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593220630f0 con 0x55932206daa0 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a0947640 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322067f40 0x559322071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53a1148640 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55932206daa0 0x559322071eb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f53940cd240 tx=0x7f539409b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd453c16e3c6971b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53827fc640 1 -- 172.21.15.64:0/704639194 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940a0350 con 0x55932206daa0 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53827fc640 1 -- 172.21.15.64:0/704639194 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53940a4c10 con 0x55932206daa0 2024-05-09T09:10:49.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.184+0000 7f53827fc640 1 -- 172.21.15.64:0/704639194 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539409c200 con 0x55932206daa0 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.185+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55932206daa0 msgr2=0x559322071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.185+0000 7f53a598b740 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55932206daa0 0x559322071eb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f53940cd240 tx=0x7f539409b900 comp rx=0 tx=0).stop 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 shutdown_connections 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55932206daa0 0x559322071eb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559322068930 0x559322079890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 --2- 172.21.15.64:0/704639194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322067f40 0x559322071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 >> 172.21.15.64:0/704639194 conn(0x5593220635b0 msgr2=0x5593220627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 shutdown_connections 2024-05-09T09:10:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 -- 172.21.15.64:0/704639194 wait complete. 2024-05-09T09:10:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 -- start start 2024-05-09T09:10:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.186+0000 7f53a598b740 1 Processor -- start 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.187+0000 7f53a598b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322076ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.187+0000 7f53a0947640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322076ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.187+0000 7f53a0947640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322076ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35602/0 (socket says 172.21.15.64:35602) 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.187+0000 7f53a598b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322068930 0x559322077020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a0947640 1 -- 172.21.15.64:0/3075293644 learned_addr learned my addr 172.21.15.64:0/3075293644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55932206daa0 0x559322078030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1b79e0 con 0x559322067f40 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1b8e50 con 0x55932206daa0 2024-05-09T09:10:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1b80b0 con 0x559322068930 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a1148640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55932206daa0 0x559322078030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f539bfff640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322068930 0x559322077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f53a598b740 10 client.0 _ll_register_callbacks cb 0x7f538c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f539bfff640 1 -- 172.21.15.64:0/3075293644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55932206daa0 msgr2=0x559322078030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f539bfff640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55932206daa0 0x559322078030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f539bfff640 1 -- 172.21.15.64:0/3075293644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 msgr2=0x559322076ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f539bfff640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322076ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.188+0000 7f539bfff640 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559322073080 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f539bfff640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322068930 0x559322077020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53900306f0 tx=0x7f5390005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d1817377d0e357 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f537bfff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f537bfff640 1 -- 172.21.15.64:0/3075293644 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390048030 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f537bfff640 1 -- 172.21.15.64:0/3075293644 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5390004300 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53940a4c10 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f537bfff640 1 -- 172.21.15.64:0/3075293644 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5390062050 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f53a598b740 10 client.25678 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55932206d780 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.189+0000 7f537bfff640 1 -- 172.21.15.64:0/3075293644 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53940a4c10 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f537bfff640 1 -- 172.21.15.64:0/3075293644 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f53900046c0 con 0x559322068930 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a598b740 10 client.25678 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a598b740 10 client.25678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a598b740 20 client.25678 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5593220e4e00 0x5593220e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a598b740 4 client.25678 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559321fd46c0 con 0x5593220e4e00 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f53a0947640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5593220e4e00 0x5593220e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.190+0000 7f537bfff640 10 client.25678 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a0947640 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5593220e4e00 0x5593220e71f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71e4bfa0fa32d38 server_cookie=c8c034ade287a2b5 in_seq=0 out_seq=0 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f537bfff640 1 -- 172.21.15.64:0/3075293644 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559321fd46c0 con 0x5593220e4e00 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f537bfff640 10 client.25678 handle_command_reply: tid=0 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 client.25678 shutdown 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322068930 msgr2=0x559322077020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322068930 0x559322077020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53900306f0 tx=0x7f5390005c90 comp rx=0 tx=0).stop 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 shutdown_connections 2024-05-09T09:10:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5593220e4e00 0x5593220e71f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55932206daa0 0x559322078030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559322068930 0x559322077020 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 --2- 172.21.15.64:0/3075293644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559322067f40 0x559322076ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 >> 172.21.15.64:0/3075293644 conn(0x5593220635b0 msgr2=0x559322066330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f537bfff640 0 client.25678 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.191+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 shutdown_connections 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.192+0000 7f53a598b740 1 -- 172.21.15.64:0/3075293644 wait complete. 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.192+0000 7f53a598b740 20 client.25678 trim_cache size 0 max 16384 2024-05-09T09:10:49.193 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53840a2c60 msgr2=0x7f53840a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53840a2c60 0x7f53840a5120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f539002a930 tx=0x7f539002a340 comp rx=0 tx=0).stop 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 msgr2=0x7f539c1ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 0x7f539c1ae0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f539400eb10 tx=0x7f539400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 shutdown_connections 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53840a2c60 0x7f53840a5120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f539c10de50 0x7f539c1ae0e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c075870 0x7f539c10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 --2- 172.21.15.64:0/175233495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c074290 0x7f539c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 >> 172.21.15.64:0/175233495 conn(0x7f539c0fce10 msgr2=0x7f539c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 shutdown_connections 2024-05-09T09:10:49.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.193+0000 7f53a2bab640 1 -- 172.21.15.64:0/175233495 wait complete. 2024-05-09T09:10:49.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:10:49.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:48 smithi179 ceph-mon[34086]: pgmap v420: 129 pgs: 129 active+clean; 932 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-05-09T09:10:49.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1188810240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 msgr2=0x7fd314073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1188810240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd314073cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009ca0 tx=0x7fd30c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1188810240 shutdown_connections 2024-05-09T09:10:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1188810240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314109620 0x7fd31410bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1188810240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 0x7fd314074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1188810240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd314073cc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1188810240 >> 172.21.15.64:0/1188810240 conn(0x7fd3140fccd0 msgr2=0x7fd3140ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1188810240 shutdown_connections 2024-05-09T09:10:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.360+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1188810240 wait complete. 2024-05-09T09:10:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 Processor -- start 2024-05-09T09:10:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 -- start start 2024-05-09T09:10:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 0x7fd314199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd31419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314109620 0x7fd3141a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410f4a0 con 0x7fd314074290 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410f320 con 0x7fd314109620 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.361+0000 7fd31b2bb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410f620 con 0x7fd314075870 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd31419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd31419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49556/0 (socket says 172.21.15.64:49556) 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 -- 172.21.15.64:0/1338970831 learned_addr learned my addr 172.21.15.64:0/1338970831 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd31aaba640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314109620 0x7fd3141a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 0x7fd314199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 -- 172.21.15.64:0/1338970831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314109620 msgr2=0x7fd3141a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314109620 0x7fd3141a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 -- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 msgr2=0x7fd314199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 0x7fd314199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 -- 172.21.15.64:0/1338970831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd314101190 con 0x7fd314075870 2024-05-09T09:10:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 0x7fd314199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.362+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd31419a0a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd31000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.363+0000 7fd30b7fe640 1 -- 172.21.15.64:0/1338970831 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310017070 con 0x7fd314075870 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.363+0000 7fd30b7fe640 1 -- 172.21.15.64:0/1338970831 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd310009c30 con 0x7fd314075870 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.363+0000 7fd30b7fe640 1 -- 172.21.15.64:0/1338970831 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31000f620 con 0x7fd314075870 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.363+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd314101480 con 0x7fd314075870 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.363+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd314101980 con 0x7fd314075870 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 Processor -- start 2024-05-09T09:10:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 -- start start 2024-05-09T09:10:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78d5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78d5af0 0x55a3f78da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78dac60 0x55a3f78dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd30b7fe640 1 -- 172.21.15.64:0/1338970831 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3100109a0 con 0x7fd314075870 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a3f7839c40 con 0x55a3f78dac60 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a3f77e1580 con 0x55a3f78d5af0 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31f221740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a3f7835c00 con 0x55a3f78d5100 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.367+0000 7fd31aaba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78d5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78d5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49572/0 (socket says 172.21.15.64:49572) 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 -- 172.21.15.64:0/4184250912 learned_addr learned my addr 172.21.15.64:0/4184250912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78d5af0 0x55a3f78da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd319ab8640 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78dac60 0x55a3f78dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 -- 172.21.15.64:0/4184250912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78d5af0 msgr2=0x55a3f78da5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78d5af0 0x55a3f78da5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 -- 172.21.15.64:0/4184250912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78dac60 msgr2=0x55a3f78dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78dac60 0x55a3f78dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 -- 172.21.15.64:0/4184250912 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f754d3f0 con 0x55a3f78d5100 2024-05-09T09:10:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.368+0000 7fd31aaba640 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78d5520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd304009c80 tx=0x7fd304011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d77279b9df651 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.369+0000 7fd308bfe640 1 -- 172.21.15.64:0/4184250912 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304004220 con 0x55a3f78d5100 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.369+0000 7fd308bfe640 1 -- 172.21.15.64:0/4184250912 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd304009e30 con 0x55a3f78d5100 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.369+0000 7fd30b7fe640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd3000a2d60 0x7fd3000a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.369+0000 7fd30b7fe640 1 -- 172.21.15.64:0/1338970831 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd3100cef90 con 0x7fd314075870 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.369+0000 7fd308bfe640 1 -- 172.21.15.64:0/4184250912 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304019760 con 0x55a3f78d5100 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.369+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd3000a2d60 0x7fd3000a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 -- 172.21.15.64:0/4184250912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 msgr2=0x55a3f78d5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78d5520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd304009c80 tx=0x7fd304011810 comp rx=0 tx=0).stop 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd3000a2d60 0x7fd3000a5220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f80 tx=0x7fd30c02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 -- 172.21.15.64:0/4184250912 shutdown_connections 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78dac60 0x55a3f78dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78d5af0 0x55a3f78da5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 --2- 172.21.15.64:0/4184250912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78d5520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.370+0000 7fd31f221740 1 -- 172.21.15.64:0/4184250912 >> 172.21.15.64:0/4184250912 conn(0x55a3f78d0770 msgr2=0x55a3f78d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.371+0000 7fd31f221740 1 -- 172.21.15.64:0/4184250912 shutdown_connections 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.371+0000 7fd31f221740 1 -- 172.21.15.64:0/4184250912 wait complete. 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 -- start start 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 Processor -- start 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78df5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a3f78ddcb0 con 0x55a3f78d5af0 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a3f77f4c40 con 0x55a3f78dac60 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.372+0000 7fd31f221740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a3f78dde30 con 0x55a3f78d5100 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31f221740 10 client.0 _ll_register_callbacks cb 0x55a3f754cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31a2b9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78df5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd319ab8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35774/0 (socket says 172.21.15.64:35774) 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31a2b9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78df5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49578/0 (socket says 172.21.15.64:49578) 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 -- 172.21.15.64:0/3505515226 learned_addr learned my addr 172.21.15.64:0/3505515226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 -- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 msgr2=0x55a3f78df5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78df5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 -- 172.21.15.64:0/3505515226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 msgr2=0x55a3f78dfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78dfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 -- 172.21.15.64:0/3505515226 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f76177d0 con 0x55a3f78dac60 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd319ab8640 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd31aaba640 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e6b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd304003850 tx=0x7fd304003880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa5c3af870dbc37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.373+0000 7fd2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.374+0000 7fd2fdffb640 1 -- 172.21.15.64:0/3505515226 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304027020 con 0x55a3f78dac60 2024-05-09T09:10:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.374+0000 7fd2fdffb640 1 -- 172.21.15.64:0/3505515226 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd304005250 con 0x55a3f78dac60 2024-05-09T09:10:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.374+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd304009e30 con 0x55a3f78dac60 2024-05-09T09:10:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.374+0000 7fd2fdffb640 1 -- 172.21.15.64:0/3505515226 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd304005450 con 0x55a3f78dac60 2024-05-09T09:10:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.374+0000 7fd31f221740 10 client.25583 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.374+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a3f78e9f00 con 0x55a3f78dac60 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd2fdffb640 1 -- 172.21.15.64:0/3505515226 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3040055f0 con 0x55a3f78dac60 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd2fdffb640 1 -- 172.21.15.64:0/3505515226 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd3040195c0 con 0x55a3f78dac60 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd31f221740 10 client.25583 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd31f221740 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd31f221740 20 client.25583 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd31f221740 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f78eb3a0 0x55a3f78ed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd31f221740 4 client.25583 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.375+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd304009e30 con 0x55a3f78eb3a0 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.376+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f78eb3a0 0x55a3f78ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.377+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f78eb3a0 0x55a3f78ed790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb353bdcfb08c5d0 server_cookie=f0788dba3362ee90 in_seq=0 out_seq=0 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.377+0000 7fd2fdffb640 10 client.25583 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.377+0000 7fd2fdffb640 1 -- 172.21.15.64:0/3505515226 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fd304009e30 con 0x55a3f78eb3a0 2024-05-09T09:10:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.377+0000 7fd2fdffb640 10 client.25583 handle_command_reply: tid=0 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.377+0000 7fd31f221740 1 client.25583 shutdown 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 msgr2=0x55a3f78e6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e6b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd304003850 tx=0x7fd304003880 comp rx=0 tx=0).stop 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 shutdown_connections 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e6b40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f78eb3a0 0x55a3f78ed790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 --2- 172.21.15.64:0/3505515226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78df5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd2fdffb640 0 client.25583 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 >> 172.21.15.64:0/3505515226 conn(0x55a3f78d0770 msgr2=0x55a3f78cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 shutdown_connections 2024-05-09T09:10:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 1 -- 172.21.15.64:0/3505515226 wait complete. 2024-05-09T09:10:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.378+0000 7fd31f221740 20 client.25583 trim_cache size 0 max 16384 2024-05-09T09:10:49.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.409+0000 7fd31f221740 1 Processor -- start 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.409+0000 7fd31f221740 1 -- start start 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.410+0000 7fd31f221740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 0x55a3f78d6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.410+0000 7fd31f221740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5af0 0x55a3f78e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.410+0000 7fd31a2b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 0x55a3f78d6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.410+0000 7fd31a2b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 0x55a3f78d6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35652/0 (socket says 172.21.15.64:35652) 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.410+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1147848812 learned_addr learned my addr 172.21.15.64:0/1147848812 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.410+0000 7fd31f221740 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5af0 0x55a3f78e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141aaa00 con 0x55a3f78d5100 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd314071d90 con 0x55a3f78dac60 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd31406fbe0 con 0x55a3f78d5af0 2024-05-09T09:10:49.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31aaba640 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1147848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5af0 msgr2=0x55a3f78e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5af0 0x55a3f78e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1147848812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 msgr2=0x55a3f78d7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78d7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.411+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1147848812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f78d14e0 con 0x55a3f78d5100 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.412+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 0x55a3f78d6ef0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0318d0 tx=0x7fd30c052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25fc375f0b75acd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.412+0000 7fd2fffff640 1 -- 172.21.15.64:0/1147848812 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c003a30 con 0x55a3f78d5100 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.412+0000 7fd2fffff640 1 -- 172.21.15.64:0/1147848812 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd30c06b070 con 0x55a3f78d5100 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.412+0000 7fd2fffff640 1 -- 172.21.15.64:0/1147848812 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0043a0 con 0x55a3f78d5100 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 msgr2=0x55a3f78d6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 0x55a3f78d6ef0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0318d0 tx=0x7fd30c052000 comp rx=0 tx=0).stop 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 shutdown_connections 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5af0 0x55a3f78e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 --2- 172.21.15.64:0/1147848812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5100 0x55a3f78d6ef0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.413+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 >> 172.21.15.64:0/1147848812 conn(0x55a3f78d0770 msgr2=0x55a3f78cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.414+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 shutdown_connections 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.414+0000 7fd31f221740 1 -- 172.21.15.64:0/1147848812 wait complete. 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.414+0000 7fd31f221740 1 -- start start 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.415+0000 7fd31f221740 1 Processor -- start 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.415+0000 7fd31f221740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.415+0000 7fd31f221740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31f221740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31a2b9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31a2b9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49594/0 (socket says 172.21.15.64:49594) 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1908713946 learned_addr learned my addr 172.21.15.64:0/1908713946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31aaba640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd314070250 con 0x55a3f78d5af0 2024-05-09T09:10:49.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140716c0 con 0x55a3f78dac60 2024-05-09T09:10:49.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd314070920 con 0x55a3f78d5100 2024-05-09T09:10:49.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.416+0000 7fd31f221740 10 client.0 _ll_register_callbacks cb 0x7fd304005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:49.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1908713946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 msgr2=0x55a3f78e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1908713946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 msgr2=0x55a3f78e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31a2b9640 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f794f970 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31a2b9640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78e6be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd30c02aeb0 tx=0x7fd30c031610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8197af5d1497985b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd2fdffb640 1 -- 172.21.15.64:0/1908713946 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c073070 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd2fdffb640 1 -- 172.21.15.64:0/1908713946 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd30c003b70 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.417+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c06b070 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd2fdffb640 1 -- 172.21.15.64:0/1908713946 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd30c07f050 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd31f221740 10 client.25696 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a3f7950c70 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd2fdffb640 1 -- 172.21.15.64:0/1908713946 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd30c036040 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd2fdffb640 1 -- 172.21.15.64:0/1908713946 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd30c004560 con 0x55a3f78d5100 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd31f221740 10 client.25696 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:49.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd31f221740 10 client.25696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:10:49.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.418+0000 7fd31f221740 20 client.25696 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:49.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.419+0000 7fd31f221740 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f79516b0 0x55a3f7953ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.419+0000 7fd31f221740 4 client.25696 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:10:49.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.419+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd30c06b070 con 0x55a3f79516b0 2024-05-09T09:10:49.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.419+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f79516b0 0x55a3f7953ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.419+0000 7fd2fdffb640 10 client.25696 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:49.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd319ab8640 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f79516b0 0x55a3f7953ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=162665f90d483d59 server_cookie=2f04ccb78c25c712 in_seq=0 out_seq=0 2024-05-09T09:10:49.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd2fdffb640 1 -- 172.21.15.64:0/1908713946 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd30c06b070 con 0x55a3f79516b0 2024-05-09T09:10:49.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd2fdffb640 10 client.25696 handle_command_reply: tid=0 2024-05-09T09:10:49.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd31f221740 1 client.25696 shutdown 2024-05-09T09:10:49.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 msgr2=0x55a3f78e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd31f221740 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78e6be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd30c02aeb0 tx=0x7fd30c031610 comp rx=0 tx=0).stop 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 shutdown_connections 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd31f221740 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f78dac60 0x55a3f78e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.420+0000 7fd31f221740 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55a3f79516b0 0x55a3f7953ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd31f221740 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f78d5af0 0x55a3f78e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd31f221740 1 --2- 172.21.15.64:0/1908713946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f78d5100 0x55a3f78e6be0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd2fdffb640 0 client.25696 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 >> 172.21.15.64:0/1908713946 conn(0x55a3f78d0770 msgr2=0x55a3f78d6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 shutdown_connections 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd31f221740 1 -- 172.21.15.64:0/1908713946 wait complete. 2024-05-09T09:10:49.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.421+0000 7fd31f221740 20 client.25696 trim_cache size 0 max 16384 2024-05-09T09:10:49.422 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd3000a2d60 msgr2=0x7fd3000a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd3000a2d60 0x7fd3000a5220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f80 tx=0x7fd30c02f310 comp rx=0 tx=0).stop 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 msgr2=0x7fd31419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd31419a0a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd31000de10 comp rx=0 tx=0).stop 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 shutdown_connections 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd3000a2d60 0x7fd3000a5220 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314109620 0x7fd3141a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314075870 0x7fd31419a0a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 --2- 172.21.15.64:0/1338970831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314074290 0x7fd314199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 >> 172.21.15.64:0/1338970831 conn(0x7fd3140fccd0 msgr2=0x7fd314105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 shutdown_connections 2024-05-09T09:10:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.422+0000 7fd31b2bb640 1 -- 172.21.15.64:0/1338970831 wait complete. 2024-05-09T09:10:49.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:10:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.611+0000 7f5323e38640 1 -- 172.21.15.64:0/1693758793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c0761f0 msgr2=0x7f531c076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.611+0000 7f5323e38640 1 --2- 172.21.15.64:0/1693758793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c0761f0 0x7f531c076610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f530802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 -- 172.21.15.64:0/1693758793 shutdown_connections 2024-05-09T09:10:49.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 --2- 172.21.15.64:0/1693758793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c077dd0 0x7f531c078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 --2- 172.21.15.64:0/1693758793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 --2- 172.21.15.64:0/1693758793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c0761f0 0x7f531c076610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 -- 172.21.15.64:0/1693758793 >> 172.21.15.64:0/1693758793 conn(0x7f531c0ff020 msgr2=0x7f531c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 -- 172.21.15.64:0/1693758793 shutdown_connections 2024-05-09T09:10:49.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.612+0000 7f5323e38640 1 -- 172.21.15.64:0/1693758793 wait complete. 2024-05-09T09:10:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.613+0000 7f5323e38640 1 Processor -- start 2024-05-09T09:10:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.613+0000 7f5323e38640 1 -- start start 2024-05-09T09:10:49.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5323e38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c0761f0 0x7f531c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5323e38640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5323e38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c077dd0 0x7f531c1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5323e38640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1139d0 con 0x7f531c0761f0 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5323e38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f531c113850 con 0x7f531c077dd0 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5323e38640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f531c113b50 con 0x7f531c077430 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c0761f0 0x7f531c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49620/0 (socket says 172.21.15.64:49620) 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 -- 172.21.15.64:0/2611766036 learned_addr learned my addr 172.21.15.64:0/2611766036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c0761f0 0x7f531c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35696/0 (socket says 172.21.15.64:35696) 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53223d5640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c077dd0 0x7f531c1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 -- 172.21.15.64:0/2611766036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c077dd0 msgr2=0x7f531c1a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c077dd0 0x7f531c1a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 -- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c0761f0 msgr2=0x7f531c19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c0761f0 0x7f531c19e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.614+0000 7f53213d3640 1 -- 172.21.15.64:0/2611766036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c1a5d20 con 0x7f531c077430 2024-05-09T09:10:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.615+0000 7f53213d3640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c19e640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f531000ebc0 tx=0x7f531000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.615+0000 7f531affd640 1 -- 172.21.15.64:0/2611766036 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310010630 con 0x7f531c077430 2024-05-09T09:10:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.615+0000 7f531affd640 1 -- 172.21.15.64:0/2611766036 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5310010c80 con 0x7f531c077430 2024-05-09T09:10:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.615+0000 7f531affd640 1 -- 172.21.15.64:0/2611766036 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310018b30 con 0x7f531c077430 2024-05-09T09:10:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.615+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c1a5fb0 con 0x7f531c077430 2024-05-09T09:10:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.615+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c1a6480 con 0x7f531c077430 2024-05-09T09:10:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.619+0000 7f531affd640 1 -- 172.21.15.64:0/2611766036 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5310018cd0 con 0x7f531c077430 2024-05-09T09:10:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.620+0000 7f5326c26740 1 Processor -- start 2024-05-09T09:10:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.620+0000 7f5326c26740 1 -- start start 2024-05-09T09:10:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.620+0000 7f5326c26740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 0x5578f320f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.620+0000 7f5326c26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f3214430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.620+0000 7f5326c26740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f3214aa0 0x5578f3216f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 0x5578f320f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f5326c26740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5578f3173c40 con 0x5578f320f930 2024-05-09T09:10:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f5326c26740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5578f311b580 con 0x5578f320ef40 2024-05-09T09:10:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f5326c26740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5578f316fc00 con 0x5578f3214aa0 2024-05-09T09:10:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 0x5578f320f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35804/0 (socket says 172.21.15.64:35804) 2024-05-09T09:10:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 -- 172.21.15.64:0/1287573661 learned_addr learned my addr 172.21.15.64:0/1287573661 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f5321bd4640 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f3214430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53213d3640 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f3214aa0 0x5578f3216f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f531affd640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f530c0a2c60 0x7f530c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 -- 172.21.15.64:0/1287573661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f3214aa0 msgr2=0x5578f3216f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f3214aa0 0x5578f3216f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 -- 172.21.15.64:0/1287573661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 msgr2=0x5578f3214430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f3214430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f53223d5640 1 -- 172.21.15.64:0/1287573661 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f2e873f0 con 0x5578f320ef40 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f5321bd4640 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f3214430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.621+0000 7f531affd640 1 -- 172.21.15.64:0/2611766036 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f53100cfa10 con 0x7f531c077430 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.622+0000 7f53223d5640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f530c0a2c60 0x7f530c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.622+0000 7f53223d5640 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 0x5578f320f360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f531400dc30 tx=0x7f531400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a95d149596c166 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.622+0000 7f5307fff640 1 -- 172.21.15.64:0/1287573661 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314011700 con 0x5578f320ef40 2024-05-09T09:10:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.622+0000 7f5307fff640 1 -- 172.21.15.64:0/1287573661 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5314011dd0 con 0x5578f320ef40 2024-05-09T09:10:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.622+0000 7f5307fff640 1 -- 172.21.15.64:0/1287573661 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314010630 con 0x5578f320ef40 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f53223d5640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f530c0a2c60 0x7f530c0a5120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f531401e000 tx=0x7f5314010cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 -- 172.21.15.64:0/1287573661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 msgr2=0x5578f320f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 0x5578f320f360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f531400dc30 tx=0x7f531400f730 comp rx=0 tx=0).stop 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 -- 172.21.15.64:0/1287573661 shutdown_connections 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f3214aa0 0x5578f3216f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f3214430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 --2- 172.21.15.64:0/1287573661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320ef40 0x5578f320f360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.623+0000 7f5326c26740 1 -- 172.21.15.64:0/1287573661 >> 172.21.15.64:0/1287573661 conn(0x5578f320a5b0 msgr2=0x5578f320a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.624+0000 7f5326c26740 1 -- 172.21.15.64:0/1287573661 shutdown_connections 2024-05-09T09:10:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.624+0000 7f5326c26740 1 -- 172.21.15.64:0/1287573661 wait complete. 2024-05-09T09:10:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.624+0000 7f5326c26740 1 -- start start 2024-05-09T09:10:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.624+0000 7f5326c26740 1 Processor -- start 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f32191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f32196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f32191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f53213d3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f32196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f32191a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49642/0 (socket says 172.21.15.64:49642) 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f53213d3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f32196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35716/0 (socket says 172.21.15.64:35716) 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5321bd4640 1 -- 172.21.15.64:0/1509525359 learned_addr learned my addr 172.21.15.64:0/1509525359 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3220720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5578f3217a50 con 0x5578f320f930 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5578f312ec40 con 0x5578f3214aa0 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5578f3217bd0 con 0x5578f320ef40 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f5326c26740 10 client.0 _ll_register_callbacks cb 0x5578f2e86be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.625+0000 7f53223d5640 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3220720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53223d5640 1 -- 172.21.15.64:0/1509525359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 msgr2=0x5578f32191a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53223d5640 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f32191a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53223d5640 1 -- 172.21.15.64:0/1509525359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 msgr2=0x5578f32196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53223d5640 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f32196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53223d5640 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f2f517d0 con 0x5578f3214aa0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53223d5640 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3220720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f531400dc30 tx=0x7f5314016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b106913b886067 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.626+0000 7f53057fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.627+0000 7f53057fa640 1 -- 172.21.15.64:0/1509525359 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314037060 con 0x5578f3214aa0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.627+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5314011dd0 con 0x5578f3214aa0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.627+0000 7f53057fa640 1 -- 172.21.15.64:0/1509525359 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5314010e00 con 0x5578f3214aa0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.627+0000 7f53057fa640 1 -- 172.21.15.64:0/1509525359 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f531403e050 con 0x5578f3214aa0 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.627+0000 7f5326c26740 10 client.25601 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.627+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578f3223b50 con 0x5578f3214aa0 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f53057fa640 1 -- 172.21.15.64:0/1509525359 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5314011dd0 con 0x5578f3214aa0 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f53057fa640 1 -- 172.21.15.64:0/1509525359 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f531401cca0 con 0x5578f3214aa0 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5326c26740 10 client.25601 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5326c26740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5326c26740 20 client.25601 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f3224fa0 0x5578f32273d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5326c26740 4 client.25601 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578f3181370 con 0x5578f3224fa0 2024-05-09T09:10:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.628+0000 7f5321bd4640 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f3224fa0 0x5578f32273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.629+0000 7f5321bd4640 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f3224fa0 0x5578f32273d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2daeeadd36bfad0a server_cookie=8fe74ed27634df80 in_seq=0 out_seq=0 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.629+0000 7f53057fa640 10 client.25601 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.630+0000 7f53057fa640 1 -- 172.21.15.64:0/1509525359 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578f3181370 con 0x5578f3224fa0 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.630+0000 7f53057fa640 10 client.25601 handle_command_reply: tid=0 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.630+0000 7f5326c26740 1 client.25601 shutdown 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.630+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 msgr2=0x5578f3220720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.630+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3220720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f531400dc30 tx=0x7f5314016000 comp rx=0 tx=0).stop 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 shutdown_connections 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f3224fa0 0x5578f32273d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3220720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f32196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 --2- 172.21.15.64:0/1509525359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f32191a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 >> 172.21.15.64:0/1509525359 conn(0x5578f320a5b0 msgr2=0x5578f32097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f53057fa640 0 client.25601 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 shutdown_connections 2024-05-09T09:10:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 1 -- 172.21.15.64:0/1509525359 wait complete. 2024-05-09T09:10:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.631+0000 7f5326c26740 20 client.25601 trim_cache size 0 max 16384 2024-05-09T09:10:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.657+0000 7f5326c26740 1 Processor -- start 2024-05-09T09:10:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.658+0000 7f5326c26740 1 -- start start 2024-05-09T09:10:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.658+0000 7f5326c26740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f3210bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.658+0000 7f5326c26740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320f930 0x5578f32206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.658+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f3210bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.658+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f3210bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49644/0 (socket says 172.21.15.64:49644) 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.658+0000 7f5326c26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f3214aa0 0x5578f3211100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f53213d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320f930 0x5578f32206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 -- 172.21.15.64:0/3206542867 learned_addr learned my addr 172.21.15.64:0/3206542867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1b3340 con 0x5578f3214aa0 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1b0b80 con 0x5578f320f930 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1ae9d0 con 0x5578f320ef40 2024-05-09T09:10:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f53223d5640 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f3214aa0 0x5578f3211100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 -- 172.21.15.64:0/3206542867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320f930 msgr2=0x5578f32206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320f930 0x5578f32206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 -- 172.21.15.64:0/3206542867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f3214aa0 msgr2=0x5578f3211100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f3214aa0 0x5578f3211100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 -- 172.21.15.64:0/3206542867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f320fd10 con 0x5578f320ef40 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f53213d3640 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320f930 0x5578f32206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.659+0000 7f5321bd4640 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f3210bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5308009850 tx=0x7f5308005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37dd3f175d3dffe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.660+0000 7f53077fe640 1 -- 172.21.15.64:0/3206542867 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308035040 con 0x5578f320ef40 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.660+0000 7f53077fe640 1 -- 172.21.15.64:0/3206542867 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53080476f0 con 0x5578f320ef40 2024-05-09T09:10:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.660+0000 7f53077fe640 1 -- 172.21.15.64:0/3206542867 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308003970 con 0x5578f320ef40 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 msgr2=0x5578f3210bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f3210bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5308009850 tx=0x7f5308005fd0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 shutdown_connections 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f3214aa0 0x5578f3211100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f320f930 0x5578f32206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 --2- 172.21.15.64:0/3206542867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f3210bc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 >> 172.21.15.64:0/3206542867 conn(0x5578f320a5b0 msgr2=0x5578f32191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 shutdown_connections 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 -- 172.21.15.64:0/3206542867 wait complete. 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.661+0000 7f5326c26740 1 -- start start 2024-05-09T09:10:49.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5326c26740 1 Processor -- start 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5326c26740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f321d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5326c26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f321da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5326c26740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3289500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f321d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5321bd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f321d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49652/0 (socket says 172.21.15.64:49652) 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f53213d3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f321da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f53213d3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f321da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35726/0 (socket says 172.21.15.64:35726) 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.662+0000 7f5321bd4640 1 -- 172.21.15.64:0/2425151769 learned_addr learned my addr 172.21.15.64:0/2425151769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1af040 con 0x5578f320f930 2024-05-09T09:10:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1b04b0 con 0x5578f3214aa0 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1af710 con 0x5578f320ef40 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f53223d5640 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3289500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5326c26740 10 client.0 _ll_register_callbacks cb 0x7f5314015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5321bd4640 1 -- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 msgr2=0x5578f3289500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5321bd4640 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3289500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5321bd4640 1 -- 172.21.15.64:0/2425151769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 msgr2=0x5578f321da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5321bd4640 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f321da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5321bd4640 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f321e040 con 0x5578f320ef40 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53080476f0 con 0x5578f320ef40 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f5321bd4640 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f321d4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f5308047670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c693935c6b144a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f53057fa640 10 client.25714 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.663+0000 7f53057fa640 1 -- 172.21.15.64:0/2425151769 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308060050 con 0x5578f320ef40 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.664+0000 7f53057fa640 1 -- 172.21.15.64:0/2425151769 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5308031ad0 con 0x5578f320ef40 2024-05-09T09:10:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.664+0000 7f53057fa640 1 -- 172.21.15.64:0/2425151769 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5308056070 con 0x5578f320ef40 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.664+0000 7f5326c26740 10 client.25714 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.664+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5578f328a9e0 con 0x5578f320ef40 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.664+0000 7f53057fa640 1 -- 172.21.15.64:0/2425151769 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f53080036f0 con 0x5578f320ef40 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f5326c26740 10 client.25714 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f5326c26740 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f5326c26740 20 client.25714 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f5326c26740 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f328b360 0x5578f328d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f5326c26740 4 client.25714 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f53080476f0 con 0x5578f328b360 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.665+0000 7f53213d3640 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f328b360 0x5578f328d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f53213d3640 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f328b360 0x5578f328d780 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85a05dddb18bc server_cookie=e871c961eaf51ce5 in_seq=0 out_seq=0 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f53057fa640 10 client.25714 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f53057fa640 1 -- 172.21.15.64:0/2425151769 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f53080476f0 con 0x5578f328b360 2024-05-09T09:10:49.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f53057fa640 10 client.25714 handle_command_reply: tid=0 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 client.25714 shutdown 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 msgr2=0x5578f321d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f321d4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f5308047670 comp rx=0 tx=0).stop 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 shutdown_connections 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5578f328b360 0x5578f328d780 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5578f3214aa0 0x5578f3289500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f53057fa640 0 client.25714 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578f320f930 0x5578f321da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 --2- 172.21.15.64:0/2425151769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5578f320ef40 0x5578f321d4f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 >> 172.21.15.64:0/2425151769 conn(0x5578f320a5b0 msgr2=0x5578f32191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 shutdown_connections 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.666+0000 7f5326c26740 1 -- 172.21.15.64:0/2425151769 wait complete. 2024-05-09T09:10:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.667+0000 7f5326c26740 20 client.25714 trim_cache size 0 max 16384 2024-05-09T09:10:49.672 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:10:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.667+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f530c0a2c60 msgr2=0x7f530c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.667+0000 7f5323e38640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f530c0a2c60 0x7f530c0a5120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f531401e000 tx=0x7f5314010cd0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.667+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 msgr2=0x7f531c19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.667+0000 7f5323e38640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c19e640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f531000ebc0 tx=0x7f531000c6a0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 shutdown_connections 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f530c0a2c60 0x7f530c0a5120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f531c077dd0 0x7f531c1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531c077430 0x7f531c19e640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 --2- 172.21.15.64:0/2611766036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f531c0761f0 0x7f531c19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 >> 172.21.15.64:0/2611766036 conn(0x7f531c0ff020 msgr2=0x7f531c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 shutdown_connections 2024-05-09T09:10:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:49.668+0000 7f5323e38640 1 -- 172.21.15.64:0/2611766036 wait complete. 2024-05-09T09:10:50.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:49 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/199743980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:10:50.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:49 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/199743980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:10:50.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:49 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/199743980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:10:50.685 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:10:50.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:10:50.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:10:50.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 -- 172.21.15.64:0/83170550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c0ff710 msgr2=0x7f6f3c0ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 --2- 172.21.15.64:0/83170550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c0ffb30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009900 tx=0x7f6f2c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 -- 172.21.15.64:0/83170550 shutdown_connections 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 --2- 172.21.15.64:0/83170550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c1011e0 0x7f6f3c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 --2- 172.21.15.64:0/83170550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c100840 0x7f6f3c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 --2- 172.21.15.64:0/83170550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c0ffb30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.879+0000 7f6f40f90640 1 -- 172.21.15.64:0/83170550 >> 172.21.15.64:0/83170550 conn(0x7f6f3c0fb030 msgr2=0x7f6f3c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.880+0000 7f6f40f90640 1 -- 172.21.15.64:0/83170550 shutdown_connections 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.880+0000 7f6f40f90640 1 -- 172.21.15.64:0/83170550 wait complete. 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 Processor -- start 2024-05-09T09:10:50.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 -- start start 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c100840 0x7f6f3c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c1011e0 0x7f6f3c1a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1139c0 con 0x7f6f3c100840 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c113840 con 0x7f6f3c0ff710 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f40f90640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c113b40 con 0x7f6f3c1011e0 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c1011e0 0x7f6f3c1a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c100840 0x7f6f3c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f3a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f3a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c19e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35828/0 (socket says 172.21.15.64:35828) 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f3a59c640 1 -- 172.21.15.64:0/3642682440 learned_addr learned my addr 172.21.15.64:0/3642682440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.881+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c100840 0x7f6f3c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35736/0 (socket says 172.21.15.64:35736) 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3a59c640 1 -- 172.21.15.64:0/3642682440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c1011e0 msgr2=0x7f6f3c1a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c1011e0 0x7f6f3c1a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3a59c640 1 -- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c100840 msgr2=0x7f6f3c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c100840 0x7f6f3c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3a59c640 1 -- 172.21.15.64:0/3642682440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c1a5d00 con 0x7f6f3c0ff710 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c1011e0 0x7f6f3c1a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.882+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c19e0e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0314e0 tx=0x7f6f2c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.883+0000 7f6f337fe640 1 -- 172.21.15.64:0/3642682440 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c03f070 con 0x7f6f3c0ff710 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.883+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c1a5f90 con 0x7f6f3c0ff710 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.883+0000 7f6f337fe640 1 -- 172.21.15.64:0/3642682440 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2c003e40 con 0x7f6f3c0ff710 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.883+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c1a63f0 con 0x7f6f3c0ff710 2024-05-09T09:10:50.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.883+0000 7f6f337fe640 1 -- 172.21.15.64:0/3642682440 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c035030 con 0x7f6f3c0ff710 2024-05-09T09:10:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.887+0000 7f6f337fe640 1 -- 172.21.15.64:0/3642682440 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f2c042070 con 0x7f6f3c0ff710 2024-05-09T09:10:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.887+0000 7f6f43d7c740 1 Processor -- start 2024-05-09T09:10:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.888+0000 7f6f43d7c740 1 -- start start 2024-05-09T09:10:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.888+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.888+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f37eb0 0x55f493f3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.888+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f3f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f493e9b990 con 0x55f493f374c0 2024-05-09T09:10:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f493e435b0 con 0x55f493f37eb0 2024-05-09T09:10:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f493e97950 con 0x55f493f3d020 2024-05-09T09:10:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f378e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35738/0 (socket says 172.21.15.64:35738) 2024-05-09T09:10:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3467173261 learned_addr learned my addr 172.21.15.64:0/3467173261 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.889+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f37eb0 0x55f493f3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3467173261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 msgr2=0x55f493f3f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f3f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3467173261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f37eb0 msgr2=0x55f493f3c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f37eb0 0x55f493f3c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3467173261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f493c777e0 con 0x55f493f374c0 2024-05-09T09:10:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f37eb0 0x55f493f3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:10:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f337fe640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f280a2c60 0x7f6f280a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f378e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400f600 tx=0x7f6f3400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09f00336b132bbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.890+0000 7f6f337fe640 1 -- 172.21.15.64:0/3642682440 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f2c0efb20 con 0x7f6f3c0ff710 2024-05-09T09:10:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.891+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f280a2c60 0x7f6f280a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.891+0000 7f6f30ff9640 1 -- 172.21.15.64:0/3467173261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34003b70 con 0x55f493f374c0 2024-05-09T09:10:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.891+0000 7f6f30ff9640 1 -- 172.21.15.64:0/3467173261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f34003d10 con 0x55f493f374c0 2024-05-09T09:10:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.891+0000 7f6f30ff9640 1 -- 172.21.15.64:0/3467173261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34004740 con 0x55f493f374c0 2024-05-09T09:10:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3467173261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 msgr2=0x55f493f378e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f378e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400f600 tx=0x7f6f3400fac0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f3a59c640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f280a2c60 0x7f6f280a5120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0ed600 tx=0x7f6f2c02a3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3467173261 shutdown_connections 2024-05-09T09:10:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f3f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f37eb0 0x55f493f3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3467173261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f378e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.892+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3467173261 >> 172.21.15.64:0/3467173261 conn(0x55f493eff0a0 msgr2=0x55f493f32f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.893+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3467173261 shutdown_connections 2024-05-09T09:10:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.894+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3467173261 wait complete. 2024-05-09T09:10:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.894+0000 7f6f43d7c740 1 -- start start 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.894+0000 7f6f43d7c740 1 Processor -- start 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f41910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f37eb0 0x55f493f41e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f3d020 0x55f493f48e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f493f3ffd0 con 0x55f493f374c0 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f493e56b40 con 0x55f493f3d020 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f493f40150 con 0x55f493f37eb0 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f43d7c740 10 client.0 _ll_register_callbacks cb 0x55f493e4b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f3d020 0x55f493f48e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f41910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f41910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35754/0 (socket says 172.21.15.64:35754) 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/1539945635 learned_addr learned my addr 172.21.15.64:0/1539945635 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.895+0000 7f6f33fff640 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f37eb0 0x55f493f41e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/1539945635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f37eb0 msgr2=0x55f493f41e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f37eb0 0x55f493f41e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/1539945635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 msgr2=0x55f493f41910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f41910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/1539945635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f493cd7f70 con 0x55f493f3d020 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f3d020 0x55f493f48e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55f493f38710 tx=0x7f6f3400a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc50223870af9c2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f21ffb640 1 -- 172.21.15.64:0/1539945635 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3402d060 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.896+0000 7f6f21ffb640 1 -- 172.21.15.64:0/1539945635 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f3400dcc0 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.897+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f34003d10 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.897+0000 7f6f21ffb640 1 -- 172.21.15.64:0/1539945635 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6f34004a40 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.897+0000 7f6f43d7c740 10 client.25613 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.897+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f493b6f170 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.898+0000 7f6f21ffb640 1 -- 172.21.15.64:0/1539945635 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f34003d10 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.899+0000 7f6f21ffb640 1 -- 172.21.15.64:0/1539945635 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6f34024040 con 0x55f493f3d020 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.899+0000 7f6f43d7c740 10 client.25613 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.899+0000 7f6f43d7c740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.899+0000 7f6f43d7c740 20 client.25613 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.900+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493f4d0a0 0x55f493f4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.900+0000 7f6f43d7c740 4 client.25613 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.900+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f493ea90c0 con 0x55f493f4d0a0 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.900+0000 7f6f33fff640 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493f4d0a0 0x55f493f4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.901+0000 7f6f33fff640 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493f4d0a0 0x55f493f4f490 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac609ce7446e1ad server_cookie=7464a6582638dd12 in_seq=0 out_seq=0 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.901+0000 7f6f21ffb640 10 client.25613 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.902+0000 7f6f21ffb640 1 -- 172.21.15.64:0/1539945635 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f493ea90c0 con 0x55f493f4d0a0 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.902+0000 7f6f21ffb640 10 client.25613 handle_command_reply: tid=0 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.902+0000 7f6f43d7c740 1 client.25613 shutdown 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f3d020 msgr2=0x55f493f48e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f3d020 0x55f493f48e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55f493f38710 tx=0x7f6f3400a2b0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 shutdown_connections 2024-05-09T09:10:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493f4d0a0 0x55f493f4f490 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f3d020 0x55f493f48e90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f37eb0 0x55f493f41e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/1539945635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f374c0 0x55f493f41910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 >> 172.21.15.64:0/1539945635 conn(0x55f493eff0a0 msgr2=0x55f493f32080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f21ffb640 0 client.25613 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 shutdown_connections 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.903+0000 7f6f43d7c740 1 -- 172.21.15.64:0/1539945635 wait complete. 2024-05-09T09:10:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.904+0000 7f6f43d7c740 20 client.25613 trim_cache size 0 max 16384 2024-05-09T09:10:50.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.939+0000 7f6f43d7c740 1 Processor -- start 2024-05-09T09:10:50.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.939+0000 7f6f43d7c740 1 -- start start 2024-05-09T09:10:50.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f31f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f32440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f48e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c012630 con 0x55f493f37eb0 2024-05-09T09:10:50.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00fe70 con 0x55f493f374c0 2024-05-09T09:10:50.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00dcc0 con 0x55f493f3d020 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f31f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f31f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35860/0 (socket says 172.21.15.64:35860) 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3071318386 learned_addr learned my addr 172.21.15.64:0/3071318386 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f33fff640 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f32440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3071318386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 msgr2=0x55f493f48e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f48e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3071318386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 msgr2=0x55f493f32440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f32440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3071318386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f493f33190 con 0x55f493f374c0 2024-05-09T09:10:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.940+0000 7f6f33fff640 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f32440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:10:50.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.941+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f31f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400d640 tx=0x7f6f3400add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f068e15aa9a63c65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:50.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.941+0000 7f6f23fff640 1 -- 172.21.15.64:0/3071318386 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3402d070 con 0x55f493f374c0 2024-05-09T09:10:50.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.942+0000 7f6f23fff640 1 -- 172.21.15.64:0/3071318386 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f34005a10 con 0x55f493f374c0 2024-05-09T09:10:50.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.942+0000 7f6f23fff640 1 -- 172.21.15.64:0/3071318386 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34027ca0 con 0x55f493f374c0 2024-05-09T09:10:50.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3071318386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 msgr2=0x55f493f31f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f31f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400d640 tx=0x7f6f3400add0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3071318386 shutdown_connections 2024-05-09T09:10:50.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f48e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f32440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3071318386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f31f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3071318386 >> 172.21.15.64:0/3071318386 conn(0x55f493eff0a0 msgr2=0x55f493f317f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3071318386 shutdown_connections 2024-05-09T09:10:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3071318386 wait complete. 2024-05-09T09:10:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.943+0000 7f6f43d7c740 1 -- start start 2024-05-09T09:10:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 Processor -- start 2024-05-09T09:10:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f46df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f47d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f3ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35866/0 (socket says 172.21.15.64:35866) 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3135937068 learned_addr learned my addr 172.21.15.64:0/3135937068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00e330 con 0x55f493f37eb0 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00f7a0 con 0x55f493f374c0 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.944+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00ea00 con 0x55f493f3d020 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f33fff640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f46df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f47d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3135937068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 msgr2=0x55f493f47d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f47d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3135937068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 msgr2=0x55f493f46df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f46df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 -- 172.21.15.64:0/3135937068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f493f48380 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f43d7c740 10 client.0 _ll_register_callbacks cb 0x7f6f3400f450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f34005a10 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f3ad9d640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f468b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005be0 tx=0x7f6f3400a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8967e523e3e069 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f21ffb640 10 client.25619 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.945+0000 7f6f21ffb640 1 -- 172.21.15.64:0/3135937068 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3402d040 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f21ffb640 1 -- 172.21.15.64:0/3135937068 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f340275a0 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f21ffb640 1 -- 172.21.15.64:0/3135937068 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6f34027a00 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 10 client.25619 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55f493fb3650 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f21ffb640 1 -- 172.21.15.64:0/3135937068 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6f34027ba0 con 0x55f493f374c0 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 10 client.25619 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:50.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 20 client.25619 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493fb3ec0 0x55f493fb62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 4 client.25619 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:10:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f493ea3410 con 0x55f493fb3ec0 2024-05-09T09:10:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.946+0000 7f6f33fff640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493fb3ec0 0x55f493fb62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.947+0000 7f6f33fff640 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493fb3ec0 0x55f493fb62e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1870f6405eaa8ade server_cookie=fbb9d285da754114 in_seq=0 out_seq=0 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.947+0000 7f6f21ffb640 10 client.25619 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.949+0000 7f6f21ffb640 1 -- 172.21.15.64:0/3135937068 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f493ea3410 con 0x55f493fb3ec0 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.949+0000 7f6f21ffb640 10 client.25619 handle_command_reply: tid=0 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.949+0000 7f6f43d7c740 1 client.25619 shutdown 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 msgr2=0x55f493f468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f468b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005be0 tx=0x7f6f3400a710 comp rx=0 tx=0).stop 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 shutdown_connections 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f493fb3ec0 0x55f493fb62e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f493f3d020 0x55f493f47d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f493f37eb0 0x55f493f46df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 --2- 172.21.15.64:0/3135937068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f493f374c0 0x55f493f468b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 >> 172.21.15.64:0/3135937068 conn(0x55f493eff0a0 msgr2=0x55f493f32080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f21ffb640 0 client.25619 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 shutdown_connections 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 1 -- 172.21.15.64:0/3135937068 wait complete. 2024-05-09T09:10:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.950+0000 7f6f43d7c740 20 client.25619 trim_cache size 0 max 16384 2024-05-09T09:10:50.951 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:10:50.951 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:10:50.951 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "a1ed9c7c-995d-4e9b-bd0f-eca02fd104f8", 2024-05-09T09:10:50.951 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:10:50.951 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f280a2c60 msgr2=0x7f6f280a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f280a2c60 0x7f6f280a5120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0ed600 tx=0x7f6f2c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 msgr2=0x7f6f3c19e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c19e0e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0314e0 tx=0x7f6f2c003cd0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 shutdown_connections 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f280a2c60 0x7f6f280a5120 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f3c1011e0 0x7f6f3c1a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3c100840 0x7f6f3c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 --2- 172.21.15.64:0/3642682440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f3c0ff710 0x7f6f3c19e0e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.952+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 >> 172.21.15.64:0/3642682440 conn(0x7f6f3c0fb030 msgr2=0x7f6f3c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.953+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 shutdown_connections 2024-05-09T09:10:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:50.953+0000 7f6f40f90640 1 -- 172.21.15.64:0/3642682440 wait complete. 2024-05-09T09:10:50.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:10:50.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:10:51.010 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:50 smithi064 ceph-mon[30149]: pgmap v421: 129 pgs: 129 active+clean; 936 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-09T09:10:51.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:50 smithi139 ceph-mon[33599]: pgmap v421: 129 pgs: 129 active+clean; 936 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-09T09:10:51.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.172+0000 7fd9ac83c640 1 -- 172.21.15.64:0/803741928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a8108f10 msgr2=0x7fd9a8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.172+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/803741928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a8108f10 0x7fd9a8109370 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009ca0 tx=0x7fd9a002a8d0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:50 smithi179 ceph-mon[34086]: pgmap v421: 129 pgs: 129 active+clean; 936 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-09T09:10:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.174+0000 7fd9ac83c640 1 -- 172.21.15.64:0/803741928 shutdown_connections 2024-05-09T09:10:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.174+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/803741928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a81098b0 0x7fd9a810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.174+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/803741928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a8108f10 0x7fd9a8109370 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.174+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/803741928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8072c70 0x7fd9a8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.174+0000 7fd9ac83c640 1 -- 172.21.15.64:0/803741928 >> 172.21.15.64:0/803741928 conn(0x7fd9a806e740 msgr2=0x7fd9a8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.174+0000 7fd9ac83c640 1 -- 172.21.15.64:0/803741928 shutdown_connections 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.175+0000 7fd9ac83c640 1 -- 172.21.15.64:0/803741928 wait complete. 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.176+0000 7fd9ac83c640 1 Processor -- start 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.176+0000 7fd9ac83c640 1 -- start start 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9ac83c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a8072c70 0x7fd9a8117de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9ac83c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8108f10 0x7fd9a8118320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9ac83c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 0x7fd9a8119360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9ac83c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8110530 con 0x7fd9a81098b0 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9ac83c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81103b0 con 0x7fd9a8072c70 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9ac83c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81106b0 con 0x7fd9a8108f10 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 0x7fd9a8119360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8108f10 0x7fd9a8118320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8108f10 0x7fd9a8118320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49734/0 (socket says 172.21.15.64:49734) 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9a6ffd640 1 -- 172.21.15.64:0/4106083182 learned_addr learned my addr 172.21.15.64:0/4106083182 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.177+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 0x7fd9a8119360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35786/0 (socket says 172.21.15.64:35786) 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.178+0000 7fd9a7fff640 1 -- 172.21.15.64:0/4106083182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8108f10 msgr2=0x7fd9a8118320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.178+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8108f10 0x7fd9a8118320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.178+0000 7fd9a7fff640 1 -- 172.21.15.64:0/4106083182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a8072c70 msgr2=0x7fd9a8117de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.178+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a8072c70 0x7fd9a8117de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.178+0000 7fd9a7fff640 1 -- 172.21.15.64:0/4106083182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a81199d0 con 0x7fd9a81098b0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.179+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 0x7fd9a8119360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c970 tx=0x7fd99800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.179+0000 7fd9a4ff9640 1 -- 172.21.15.64:0/4106083182 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998007bb0 con 0x7fd9a81098b0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.179+0000 7fd9a4ff9640 1 -- 172.21.15.64:0/4106083182 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd998007d50 con 0x7fd9a81098b0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.179+0000 7fd9a4ff9640 1 -- 172.21.15.64:0/4106083182 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980186a0 con 0x7fd9a81098b0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.181+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a8119cc0 con 0x7fd9a81098b0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.181+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a811a190 con 0x7fd9a81098b0 2024-05-09T09:10:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.189+0000 7fd9b07ae740 1 Processor -- start 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.189+0000 7fd9a4ff9640 1 -- 172.21.15.64:0/4106083182 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd99800f990 con 0x7fd9a81098b0 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.190+0000 7fd9a4ff9640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9900a2da0 0x7fd9900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.190+0000 7fd9a4ff9640 1 -- 172.21.15.64:0/4106083182 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd9980cf830 con 0x7fd9a81098b0 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 -- start start 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 0x558a613a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a613aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613aac80 0x558a613ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558a61309d00 con 0x558a613a5b10 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558a612b14c0 con 0x558a613aac80 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558a61305cc0 con 0x558a613a5120 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 0x558a613a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 0x558a613a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49738/0 (socket says 172.21.15.64:49738) 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.193+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2765806849 learned_addr learned my addr 172.21.15.64:0/2765806849 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9900a2da0 0x7fd9900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2765806849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613aac80 msgr2=0x558a613ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613aac80 0x558a613ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2765806849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 msgr2=0x558a613aa610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a613aa610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2765806849 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a610e77d0 con 0x558a613a5120 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.194+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 0x558a613a5540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00a980 tx=0x7fd99c00ae40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a06d8d7c6f1f542 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.196+0000 7fd98dffb640 1 -- 172.21.15.64:0/2765806849 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00cd80 con 0x558a613a5120 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.196+0000 7fd98dffb640 1 -- 172.21.15.64:0/2765806849 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd99c00f900 con 0x558a613a5120 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd98dffb640 1 -- 172.21.15.64:0/2765806849 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00e8d0 con 0x558a613a5120 2024-05-09T09:10:51.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2765806849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 msgr2=0x558a613a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 0x558a613a5540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00a980 tx=0x7fd99c00ae40 comp rx=0 tx=0).stop 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2765806849 shutdown_connections 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613aac80 0x558a613ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a613aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2765806849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a5120 0x558a613a5540 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.197+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2765806849 >> 172.21.15.64:0/2765806849 conn(0x558a613a0790 msgr2=0x558a613a0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.198+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2765806849 shutdown_connections 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.198+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9900a2da0 0x7fd9900a5260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x558a613a5f10 tx=0x7fd9a0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.198+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2765806849 wait complete. 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.198+0000 7fd9b07ae740 1 -- start start 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 Processor -- start 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613aac80 0x558a613b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558a613adc80 con 0x558a613a5120 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558a612c4b90 con 0x558a613a5b10 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558a613ade00 con 0x558a613aac80 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.199+0000 7fd9b07ae740 10 client.0 _ll_register_callbacks cb 0x558a61300da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613aac80 0x558a613b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35802/0 (socket says 172.21.15.64:35802) 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2530142610 learned_addr learned my addr 172.21.15.64:0/2530142610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2530142610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613aac80 msgr2=0x558a613b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613aac80 0x558a613b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 msgr2=0x558a613afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.200+0000 7fd9a77fe640 1 -- 172.21.15.64:0/2530142610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6101d3f0 con 0x558a613a5120 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.201+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613af610 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005e00 tx=0x7fd99c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ccce738edb1eba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:51.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.201+0000 7fd986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.201+0000 7fd986ffd640 1 -- 172.21.15.64:0/2530142610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00cea0 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.201+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c00f900 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.201+0000 7fd986ffd640 1 -- 172.21.15.64:0/2530142610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd99c005020 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.201+0000 7fd986ffd640 1 -- 172.21.15.64:0/2530142610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd99c005240 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.202+0000 7fd9b07ae740 10 client.16074 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.202+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a60fdd0f0 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd986ffd640 1 -- 172.21.15.64:0/2530142610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd99c00f900 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd986ffd640 1 -- 172.21.15.64:0/2530142610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd99c00e920 con 0x558a613a5120 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9b07ae740 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9b07ae740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9b07ae740 20 client.16074 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:51.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613bb730 0x558a613bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9b07ae740 4 client.16074 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a61317430 con 0x558a613bb730 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.203+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613bb730 0x558a613bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.204+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613bb730 0x558a613bdb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404b5a1c24e5a8a4 server_cookie=c9e647bab3488aff in_seq=0 out_seq=0 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.204+0000 7fd986ffd640 10 client.16074 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.205+0000 7fd986ffd640 1 -- 172.21.15.64:0/2530142610 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a61317430 con 0x558a613bb730 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.205+0000 7fd986ffd640 10 client.16074 handle_command_reply: tid=0 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.205+0000 7fd9b07ae740 1 client.16074 shutdown 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.206+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 msgr2=0x558a613af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.206+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613af610 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005e00 tx=0x7fd99c00cdb0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.206+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 shutdown_connections 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.206+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613bb730 0x558a613bdb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.206+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613aac80 0x558a613b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd986ffd640 0 client.16074 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/2530142610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613af610 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 >> 172.21.15.64:0/2530142610 conn(0x558a613a0790 msgr2=0x558a6139fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 shutdown_connections 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd9b07ae740 1 -- 172.21.15.64:0/2530142610 wait complete. 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.207+0000 7fd9b07ae740 20 client.16074 trim_cache size 0 max 16384 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.240+0000 7fd9b07ae740 1 Processor -- start 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.240+0000 7fd9b07ae740 1 -- start start 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.240+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49764/0 (socket says 172.21.15.64:49764) 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9a77fe640 1 -- 172.21.15.64:0/3638594227 learned_addr learned my addr 172.21.15.64:0/3638594227 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81c40d0 con 0x558a613a5120 2024-05-09T09:10:51.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81c1910 con 0x558a613a5b10 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.241+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81bf760 con 0x558a613a0790 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a77fe640 1 -- 172.21.15.64:0/3638594227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 msgr2=0x558a613aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a77fe640 1 -- 172.21.15.64:0/3638594227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 msgr2=0x558a613b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a77fe640 1 -- 172.21.15.64:0/3638594227 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a613be210 con 0x558a613a0790 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.242+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613ae8e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005e30 tx=0x7fd99c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3858b5d72eb293e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.243+0000 7fd98d7fa640 1 -- 172.21.15.64:0/3638594227 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c030080 con 0x558a613a0790 2024-05-09T09:10:51.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.243+0000 7fd98d7fa640 1 -- 172.21.15.64:0/3638594227 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd99c022410 con 0x558a613a0790 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.243+0000 7fd98d7fa640 1 -- 172.21.15.64:0/3638594227 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0226b0 con 0x558a613a0790 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 msgr2=0x558a613ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613ae8e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005e30 tx=0x7fd99c005a90 comp rx=0 tx=0).stop 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 shutdown_connections 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5b10 0x558a613aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5120 0x558a613b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/3638594227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613ae8e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 >> 172.21.15.64:0/3638594227 conn(0x558a6139bc30 msgr2=0x558a6139fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 shutdown_connections 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 -- 172.21.15.64:0/3638594227 wait complete. 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.245+0000 7fd9b07ae740 1 -- start start 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.246+0000 7fd9b07ae740 1 Processor -- start 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.246+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5120 0x558a613b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a6141fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49766/0 (socket says 172.21.15.64:49766) 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5120 0x558a613b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5120 0x558a613b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35938/0 (socket says 172.21.15.64:35938) 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9a6ffd640 1 -- 172.21.15.64:0/1922519784 learned_addr learned my addr 172.21.15.64:0/1922519784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81bfdd0 con 0x558a613a5b10 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81c1240 con 0x558a613a5120 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81c04a0 con 0x558a613a0790 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9b07ae740 10 client.0 _ll_register_callbacks cb 0x7fd99c00a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.247+0000 7fd9a7fff640 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a6141fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd9a77fe640 1 -- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5120 msgr2=0x558a613b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5120 0x558a613b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd9a77fe640 1 -- 172.21.15.64:0/1922519784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 msgr2=0x558a6141fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a6141fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd9a77fe640 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a61391930 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd9a77fe640 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613b3cb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x558a613a6320 tx=0x7fd99c00e5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9253f693437522c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd986ffd640 1 -- 172.21.15.64:0/1922519784 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c042060 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.248+0000 7fd986ffd640 1 -- 172.21.15.64:0/1922519784 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd99c022660 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.249+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c022410 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.249+0000 7fd986ffd640 1 -- 172.21.15.64:0/1922519784 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd99c030050 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.250+0000 7fd9b07ae740 10 client.25735 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.250+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a613a05c0 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd986ffd640 1 -- 172.21.15.64:0/1922519784 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd99c00ea80 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd986ffd640 1 -- 172.21.15.64:0/1922519784 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd99c03da50 con 0x558a613a0790 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9b07ae740 10 client.25735 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9b07ae740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9b07ae740 20 client.25735 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:51.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613aac80 0x558a61423b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9b07ae740 4 client.25735 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd99c022410 con 0x558a613aac80 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.251+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613aac80 0x558a61423b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.252+0000 7fd986ffd640 10 client.25735 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd9a6ffd640 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613aac80 0x558a61423b70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9863e08ea8ca7e32 server_cookie=eda69020edeb177 in_seq=0 out_seq=0 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd986ffd640 1 -- 172.21.15.64:0/1922519784 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fd99c022410 con 0x558a613aac80 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd986ffd640 10 client.25735 handle_command_reply: tid=0 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd9b07ae740 1 client.25735 shutdown 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 msgr2=0x558a613b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613b3cb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x558a613a6320 tx=0x7fd99c00e5c0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.253+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 shutdown_connections 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558a613aac80 0x558a61423b70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a613a5b10 0x558a6141fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a613a5120 0x558a613b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 --2- 172.21.15.64:0/1922519784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a613a0790 0x558a613b3cb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 >> 172.21.15.64:0/1922519784 conn(0x558a6139bc30 msgr2=0x558a613a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd986ffd640 0 client.25735 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 shutdown_connections 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 1 -- 172.21.15.64:0/1922519784 wait complete. 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.254+0000 7fd9b07ae740 20 client.25735 trim_cache size 0 max 16384 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stdout: "a1ed9c7c-995d-4e9b-bd0f-eca02fd104f8": { 2024-05-09T09:10:51.273 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:10:51.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "a1ed9c7c-995d-4e9b-bd0f-eca02fd104f8", 2024-05-09T09:10:51.274 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:10:51.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:10:51.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:10:51.274 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:10:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9900a2da0 msgr2=0x7fd9900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9900a2da0 0x7fd9900a5260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x558a613a5f10 tx=0x7fd9a0000f40 comp rx=0 tx=0).stop 2024-05-09T09:10:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 msgr2=0x7fd9a8119360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 0x7fd9a8119360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c970 tx=0x7fd99800ce30 comp rx=0 tx=0).stop 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 shutdown_connections 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9900a2da0 0x7fd9900a5260 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9a81098b0 0x7fd9a8119360 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9a8108f10 0x7fd9a8118320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 --2- 172.21.15.64:0/4106083182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a8072c70 0x7fd9a8117de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.256+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 >> 172.21.15.64:0/4106083182 conn(0x7fd9a806e740 msgr2=0x7fd9a810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.257+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 shutdown_connections 2024-05-09T09:10:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:10:51.257+0000 7fd9ac83c640 1 -- 172.21.15.64:0/4106083182 wait complete. 2024-05-09T09:10:51.277 INFO:tasks.cephfs.filesystem:scrub status for tag:a1ed9c7c-995d-4e9b-bd0f-eca02fd104f8 - {'path': '/', 'tag': 'a1ed9c7c-995d-4e9b-bd0f-eca02fd104f8', 'options': 'recursive,force'} 2024-05-09T09:10:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 30.0 s, 926.6 tps, lat 100.396 ms stddev 113.148 2024-05-09T09:10:52.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:10:52.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:51 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:10:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:52.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:51 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:52.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:51 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:10:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:52 smithi139 ceph-mon[33599]: pgmap v422: 129 pgs: 129 active+clean; 948 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-09T09:10:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:52 smithi064 ceph-mon[30149]: pgmap v422: 129 pgs: 129 active+clean; 948 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-09T09:10:53.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:52 smithi179 ceph-mon[34086]: pgmap v422: 129 pgs: 129 active+clean; 948 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-09T09:10:55.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:54 smithi139 ceph-mon[33599]: pgmap v423: 129 pgs: 129 active+clean; 950 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-09T09:10:55.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:54 smithi064 ceph-mon[30149]: pgmap v423: 129 pgs: 129 active+clean; 950 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-09T09:10:55.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:54 smithi179 ceph-mon[34086]: pgmap v423: 129 pgs: 129 active+clean; 950 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-09T09:10:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 35.0 s, 1172.0 tps, lat 91.327 ms stddev 54.445 2024-05-09T09:10:57.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:56 smithi139 ceph-mon[33599]: pgmap v424: 129 pgs: 129 active+clean; 951 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-09T09:10:57.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:56 smithi064 ceph-mon[30149]: pgmap v424: 129 pgs: 129 active+clean; 951 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-09T09:10:57.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:56 smithi179 ceph-mon[34086]: pgmap v424: 129 pgs: 129 active+clean; 951 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-09T09:10:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 -- 172.21.15.64:0/1437824159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66640ff970 msgr2=0x7f66640ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 --2- 172.21.15.64:0/1437824159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66640ff970 0x7f66640ffd90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6660009900 tx=0x7f666002a410 comp rx=0 tx=0).stop 2024-05-09T09:10:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 -- 172.21.15.64:0/1437824159 shutdown_connections 2024-05-09T09:10:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 --2- 172.21.15.64:0/1437824159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6664101090 0x7f6664101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 --2- 172.21.15.64:0/1437824159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66641006d0 0x7f6664100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 --2- 172.21.15.64:0/1437824159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66640ff970 0x7f66640ffd90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.577+0000 7f666ad8b640 1 -- 172.21.15.64:0/1437824159 >> 172.21.15.64:0/1437824159 conn(0x7f66640fb2c0 msgr2=0x7f66640fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.578+0000 7f666ad8b640 1 -- 172.21.15.64:0/1437824159 shutdown_connections 2024-05-09T09:10:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.578+0000 7f666ad8b640 1 -- 172.21.15.64:0/1437824159 wait complete. 2024-05-09T09:10:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.579+0000 7f666ad8b640 1 Processor -- start 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.579+0000 7f666ad8b640 1 -- start start 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f666ad8b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 0x7f666419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f666ad8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66641006d0 0x7f666419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f666ad8b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6664101090 0x7f66641a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f666ad8b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6664103b10 con 0x7f66641006d0 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f666ad8b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6664103990 con 0x7f66640ff970 2024-05-09T09:10:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f666ad8b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6664103c90 con 0x7f6664101090 2024-05-09T09:10:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f6669d89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 0x7f666419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f6669d89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 0x7f666419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59854/0 (socket says 172.21.15.64:59854) 2024-05-09T09:10:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.580+0000 7f6669d89640 1 -- 172.21.15.64:0/2889514171 learned_addr learned my addr 172.21.15.64:0/2889514171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.581+0000 7f6669d89640 1 -- 172.21.15.64:0/2889514171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6664101090 msgr2=0x7f66641a52b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.581+0000 7f6669d89640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6664101090 0x7f66641a52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.581+0000 7f6669d89640 1 -- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66641006d0 msgr2=0x7f666419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.581+0000 7f6669d89640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66641006d0 0x7f666419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.581+0000 7f6669d89640 1 -- 172.21.15.64:0/2889514171 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666410e380 con 0x7f66640ff970 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.582+0000 7f6669d89640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 0x7f666419dd40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6660009850 tx=0x7f6660003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.582+0000 7f665affd640 1 -- 172.21.15.64:0/2889514171 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666003f030 con 0x7f66640ff970 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.582+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666410e610 con 0x7f66640ff970 2024-05-09T09:10:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.582+0000 7f665affd640 1 -- 172.21.15.64:0/2889514171 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6660032c60 con 0x7f66640ff970 2024-05-09T09:10:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.582+0000 7f665affd640 1 -- 172.21.15.64:0/2889514171 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660035070 con 0x7f66640ff970 2024-05-09T09:10:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.582+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666410ea90 con 0x7f66640ff970 2024-05-09T09:10:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.586+0000 7f666edcf740 1 Processor -- start 2024-05-09T09:10:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.586+0000 7f665affd640 1 -- 172.21.15.64:0/2889514171 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6660042030 con 0x7f66640ff970 2024-05-09T09:10:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 -- start start 2024-05-09T09:10:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 0x55afa7e4e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4edf0 0x55afa7e538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afa7db2340 con 0x55afa7e4edf0 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666a58a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 0x55afa7e4e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f6669d89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4edf0 0x55afa7e538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666a58a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 0x55afa7e4e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39408/0 (socket says 172.21.15.64:39408) 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666a58a640 1 -- 172.21.15.64:0/2420386130 learned_addr learned my addr 172.21.15.64:0/2420386130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55afa7d5a290 con 0x55afa7e53f60 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.587+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55afa7dae300 con 0x55afa7e4e400 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f666a58a640 1 -- 172.21.15.64:0/2420386130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 msgr2=0x55afa7e563f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f666a58a640 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e563f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f666a58a640 1 -- 172.21.15.64:0/2420386130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4edf0 msgr2=0x55afa7e538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f666a58a640 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4edf0 0x55afa7e538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f666a58a640 1 -- 172.21.15.64:0/2420386130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa7d7e8e0 con 0x55afa7e4e400 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f6669d89640 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4edf0 0x55afa7e538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.588+0000 7f666a58a640 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 0x55afa7e4e820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f665c009db0 tx=0x7f665c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317764f0e52b36b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.589+0000 7f665affd640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66500a2cc0 0x7f66500a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.589+0000 7f664ffff640 1 -- 172.21.15.64:0/2420386130 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c004b20 con 0x55afa7e4e400 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.589+0000 7f664ffff640 1 -- 172.21.15.64:0/2420386130 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665c004cc0 con 0x55afa7e4e400 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.589+0000 7f664ffff640 1 -- 172.21.15.64:0/2420386130 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0187d0 con 0x55afa7e4e400 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.589+0000 7f6669d89640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66500a2cc0 0x7f66500a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.590+0000 7f665affd640 1 -- 172.21.15.64:0/2889514171 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66600efc50 con 0x7f66640ff970 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.590+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 msgr2=0x55afa7e4e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.590+0000 7f6669d89640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66500a2cc0 0x7f66500a5180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55afa7e4f1f0 tx=0x7f66600f3610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.590+0000 7f666edcf740 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 0x55afa7e4e820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f665c009db0 tx=0x7f665c00e720 comp rx=0 tx=0).stop 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 shutdown_connections 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4edf0 0x55afa7e538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 --2- 172.21.15.64:0/2420386130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4e400 0x55afa7e4e820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 >> 172.21.15.64:0/2420386130 conn(0x55afa7e49a70 msgr2=0x55afa7e49e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 shutdown_connections 2024-05-09T09:10:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.591+0000 7f666edcf740 1 -- 172.21.15.64:0/2420386130 wait complete. 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.592+0000 7f666edcf740 1 -- start start 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.592+0000 7f666edcf740 1 Processor -- start 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e58700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666a58a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e58700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666a58a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666a58a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e58700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58422/0 (socket says 172.21.15.64:58422) 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666a58a640 1 -- 172.21.15.64:0/560406786 learned_addr learned my addr 172.21.15.64:0/560406786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afa7e56f10 con 0x55afa7e4e400 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55afa7d6d790 con 0x55afa7e4edf0 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55afa7e57090 con 0x55afa7e53f60 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f666edcf740 10 client.0 _ll_register_callbacks cb 0x55afa7da8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.593+0000 7f6669588640 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.594+0000 7f6669588640 1 -- 172.21.15.64:0/560406786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 msgr2=0x55afa7e5fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.594+0000 7f6669588640 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.594+0000 7f6669588640 1 -- 172.21.15.64:0/560406786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 msgr2=0x55afa7e58700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.594+0000 7f6669588640 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e58700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.594+0000 7f6669588640 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa7ac6210 con 0x55afa7e4edf0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.594+0000 7f6669588640 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e58c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6654005d00 tx=0x7f665400d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6879fbc99f1c9828 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.595+0000 7f664d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.595+0000 7f664d7fa640 1 -- 172.21.15.64:0/560406786 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654016070 con 0x55afa7e4edf0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.595+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c004cc0 con 0x55afa7e4edf0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.595+0000 7f664d7fa640 1 -- 172.21.15.64:0/560406786 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6654004220 con 0x55afa7e4edf0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.595+0000 7f664d7fa640 1 -- 172.21.15.64:0/560406786 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6654021050 con 0x55afa7e4edf0 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.596+0000 7f666edcf740 10 client.25634 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.596+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55afa7b92620 con 0x55afa7e4edf0 2024-05-09T09:10:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f664d7fa640 1 -- 172.21.15.64:0/560406786 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665c004cc0 con 0x55afa7e4edf0 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f664d7fa640 1 -- 172.21.15.64:0/560406786 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f66540104c0 con 0x55afa7e4edf0 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f666edcf740 10 client.25634 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f666edcf740 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f666edcf740 20 client.25634 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f666edcf740 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7e63f40 0x55afa7e663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f666edcf740 4 client.25634 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.597+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afa7dbfa70 con 0x55afa7e63f40 2024-05-09T09:10:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.598+0000 7f666a58a640 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7e63f40 0x55afa7e663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.599+0000 7f666a58a640 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7e63f40 0x55afa7e663c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f465b04eaea130aa server_cookie=e1b35029ffde63ca in_seq=0 out_seq=0 2024-05-09T09:10:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.599+0000 7f664d7fa640 10 client.25634 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.600+0000 7f664d7fa640 1 -- 172.21.15.64:0/560406786 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afa7dbfa70 con 0x55afa7e63f40 2024-05-09T09:10:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.600+0000 7f664d7fa640 10 client.25634 handle_command_reply: tid=0 2024-05-09T09:10:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.600+0000 7f666edcf740 1 client.25634 shutdown 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 msgr2=0x55afa7e58c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e58c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6654005d00 tx=0x7f665400d6b0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 shutdown_connections 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7e63f40 0x55afa7e663c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e58c40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f664d7fa640 0 client.25634 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 --2- 172.21.15.64:0/560406786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e58700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 >> 172.21.15.64:0/560406786 conn(0x55afa7e49a70 msgr2=0x55afa7e48bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.601+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 shutdown_connections 2024-05-09T09:10:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.602+0000 7f666edcf740 1 -- 172.21.15.64:0/560406786 wait complete. 2024-05-09T09:10:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.602+0000 7f666edcf740 20 client.25634 trim_cache size 0 max 16384 2024-05-09T09:10:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.636+0000 7f666edcf740 1 Processor -- start 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.636+0000 7f666edcf740 1 -- start start 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666edcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e6bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666edcf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4edf0 0x55afa7e6c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666edcf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666a58a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e6bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666a58a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e6bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58432/0 (socket says 172.21.15.64:58432) 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666a58a640 1 -- 172.21.15.64:0/2878074164 learned_addr learned my addr 172.21.15.64:0/2878074164 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66641bb6d0 con 0x55afa7e4e400 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66641b8f10 con 0x55afa7e53f60 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6664116680 con 0x55afa7e4edf0 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f6669588640 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4edf0 0x55afa7e6c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.637+0000 7f666a58a640 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.638+0000 7f6669588640 1 -- 172.21.15.64:0/2878074164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 msgr2=0x55afa7e5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.638+0000 7f6669588640 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.638+0000 7f6669588640 1 -- 172.21.15.64:0/2878074164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 msgr2=0x55afa7e6bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.638+0000 7f6669588640 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e6bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.638+0000 7f6669588640 1 -- 172.21.15.64:0/2878074164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa7e4a7e0 con 0x55afa7e4edf0 2024-05-09T09:10:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.638+0000 7f6669588640 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4edf0 0x55afa7e6c1d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6654000c00 tx=0x7f665400b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d60b6badb78df9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.639+0000 7f664f7fe640 1 -- 172.21.15.64:0/2878074164 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654016040 con 0x55afa7e4edf0 2024-05-09T09:10:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.639+0000 7f664f7fe640 1 -- 172.21.15.64:0/2878074164 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665402a070 con 0x55afa7e4edf0 2024-05-09T09:10:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.639+0000 7f664f7fe640 1 -- 172.21.15.64:0/2878074164 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654025b00 con 0x55afa7e4edf0 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.640+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4edf0 msgr2=0x55afa7e6c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.640+0000 7f666edcf740 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4edf0 0x55afa7e6c1d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6654000c00 tx=0x7f665400b470 comp rx=0 tx=0).stop 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.640+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 shutdown_connections 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.641+0000 7f666edcf740 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e53f60 0x55afa7e5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.641+0000 7f666edcf740 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e4edf0 0x55afa7e6c1d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.641+0000 7f666edcf740 1 --2- 172.21.15.64:0/2878074164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e6bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.641+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 >> 172.21.15.64:0/2878074164 conn(0x55afa7e49a70 msgr2=0x55afa7e49e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.641+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 shutdown_connections 2024-05-09T09:10:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.641+0000 7f666edcf740 1 -- 172.21.15.64:0/2878074164 wait complete. 2024-05-09T09:10:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.642+0000 7f666edcf740 1 -- start start 2024-05-09T09:10:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.642+0000 7f666edcf740 1 Processor -- start 2024-05-09T09:10:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66641b73d0 con 0x55afa7e4e400 2024-05-09T09:10:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66641b8840 con 0x55afa7e4edf0 2024-05-09T09:10:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66641b7aa0 con 0x55afa7e53f60 2024-05-09T09:10:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f666edcf740 10 client.0 _ll_register_callbacks cb 0x7f665400b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:10:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f6669588640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f6669588640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e5d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59892/0 (socket says 172.21.15.64:59892) 2024-05-09T09:10:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.643+0000 7f6669588640 1 -- 172.21.15.64:0/1409481720 learned_addr learned my addr 172.21.15.64:0/1409481720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:10:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.644+0000 7f666a58a640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.645+0000 7f666a58a640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.645+0000 7f666a58a640 1 -- 172.21.15.64:0/1409481720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 msgr2=0x55afa7e5e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.645+0000 7f666a58a640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.645+0000 7f666a58a640 1 -- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 msgr2=0x55afa7e5d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.645+0000 7f666a58a640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e5d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.645+0000 7f666a58a640 1 -- 172.21.15.64:0/1409481720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa7e5ee60 con 0x55afa7e4e400 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.646+0000 7f666a58a640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e5d320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55afa7e58670 tx=0x7f665c009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22b39b51f091144 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:10:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.646+0000 7f664d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:10:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.646+0000 7f664d7fa640 1 -- 172.21.15.64:0/1409481720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0288c0 con 0x55afa7e4e400 2024-05-09T09:10:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.647+0000 7f664d7fa640 1 -- 172.21.15.64:0/1409481720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f665c003970 con 0x55afa7e4e400 2024-05-09T09:10:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.647+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665402a070 con 0x55afa7e4e400 2024-05-09T09:10:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.648+0000 7f664d7fa640 1 -- 172.21.15.64:0/1409481720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f665c003bb0 con 0x55afa7e4e400 2024-05-09T09:10:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.648+0000 7f666edcf740 10 client.16095 fetch_fsmap learned FSMap version 30 2024-05-09T09:10:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.648+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55afa7eca500 con 0x55afa7e4e400 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f664d7fa640 1 -- 172.21.15.64:0/1409481720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665c003d50 con 0x55afa7e4e400 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f664d7fa640 1 -- 172.21.15.64:0/1409481720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f665c026070 con 0x55afa7e4e400 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f666edcf740 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f666edcf740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f666edcf740 20 client.16095 populate_metadata read hostname 'smithi064' 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f666edcf740 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7ecaca0 0x55afa7ecd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f666edcf740 4 client.16095 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f665402a070 con 0x55afa7ecaca0 2024-05-09T09:10:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.649+0000 7f6669588640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7ecaca0 0x55afa7ecd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:10:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.650+0000 7f664d7fa640 10 client.16095 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.650+0000 7f6669588640 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7ecaca0 0x55afa7ecd0c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca103d9cb85210a server_cookie=3ab7196faaf269e7 in_seq=0 out_seq=0 2024-05-09T09:10:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.664+0000 7f664d7fa640 1 -- 172.21.15.64:0/1409481720 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f665402a070 con 0x55afa7ecaca0 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.664+0000 7f664d7fa640 10 client.16095 handle_command_reply: tid=0 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.664+0000 7f666edcf740 1 client.16095 shutdown 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.664+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 msgr2=0x55afa7e5d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.664+0000 7f666edcf740 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e5d320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55afa7e58670 tx=0x7f665c009ab0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 shutdown_connections 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55afa7ecaca0 0x55afa7ecd0c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afa7e53f60 0x55afa7e5e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afa7e4edf0 0x55afa7e5d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 --2- 172.21.15.64:0/1409481720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afa7e4e400 0x55afa7e5d320 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 >> 172.21.15.64:0/1409481720 conn(0x55afa7e49a70 msgr2=0x55afa7e4c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f664d7fa640 0 client.16095 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:10:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 shutdown_connections 2024-05-09T09:10:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.665+0000 7f666edcf740 1 -- 172.21.15.64:0/1409481720 wait complete. 2024-05-09T09:10:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.666+0000 7f666edcf740 20 client.16095 trim_cache size 0 max 16384 2024-05-09T09:10:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:10:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:10:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:10:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:10:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66500a2cc0 msgr2=0x7f66500a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66500a2cc0 0x7f66500a5180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55afa7e4f1f0 tx=0x7f66600f3610 comp rx=0 tx=0).stop 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 msgr2=0x7f666419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 0x7f666419dd40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6660009850 tx=0x7f6660003f00 comp rx=0 tx=0).stop 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 shutdown_connections 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66500a2cc0 0x7f66500a5180 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6664101090 0x7f66641a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66641006d0 0x7f666419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 --2- 172.21.15.64:0/2889514171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66640ff970 0x7f666419dd40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.667+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 >> 172.21.15.64:0/2889514171 conn(0x7f66640fb2c0 msgr2=0x7f66640fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.668+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 shutdown_connections 2024-05-09T09:10:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:10:58.668+0000 7f666ad8b640 1 -- 172.21.15.64:0/2889514171 wait complete. 2024-05-09T09:10:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:10:58 smithi139 ceph-mon[33599]: pgmap v425: 129 pgs: 129 active+clean; 951 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-09T09:10:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:10:58 smithi064 ceph-mon[30149]: pgmap v425: 129 pgs: 129 active+clean; 951 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-09T09:10:59.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:10:58 smithi179 ceph-mon[34086]: pgmap v425: 129 pgs: 129 active+clean; 951 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-09T09:11:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:00 smithi139 ceph-mon[33599]: pgmap v426: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 1.62k op/s 2024-05-09T09:11:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:11:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:11:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:11:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:00 smithi064 ceph-mon[30149]: pgmap v426: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 1.62k op/s 2024-05-09T09:11:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:11:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:11:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:11:01.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:00 smithi179 ceph-mon[34086]: pgmap v426: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 1.62k op/s 2024-05-09T09:11:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:11:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:11:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:11:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 40.0 s, 1019.8 tps, lat 97.946 ms stddev 24.686 2024-05-09T09:11:03.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:02 smithi139 ceph-mon[33599]: pgmap v427: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:11:03.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:02 smithi064 ceph-mon[30149]: pgmap v427: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:11:03.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:02 smithi179 ceph-mon[34086]: pgmap v427: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:11:05.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:04 smithi139 ceph-mon[33599]: pgmap v428: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-05-09T09:11:05.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:04 smithi064 ceph-mon[30149]: pgmap v428: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-05-09T09:11:05.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:04 smithi179 ceph-mon[34086]: pgmap v428: 129 pgs: 129 active+clean; 951 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-05-09T09:11:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 45.0 s, 946.4 tps, lat 75.541 ms stddev 29.498 2024-05-09T09:11:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:06 smithi139 ceph-mon[33599]: pgmap v429: 129 pgs: 129 active+clean; 951 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:06 smithi064 ceph-mon[30149]: pgmap v429: 129 pgs: 129 active+clean; 951 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:07.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:06 smithi179 ceph-mon[34086]: pgmap v429: 129 pgs: 129 active+clean; 951 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:08 smithi139 ceph-mon[33599]: pgmap v430: 129 pgs: 129 active+clean; 951 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.4 MiB/s wr, 1.12k op/s 2024-05-09T09:11:09.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:08 smithi064 ceph-mon[30149]: pgmap v430: 129 pgs: 129 active+clean; 951 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.4 MiB/s wr, 1.12k op/s 2024-05-09T09:11:09.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:08 smithi179 ceph-mon[34086]: pgmap v430: 129 pgs: 129 active+clean; 951 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.4 MiB/s wr, 1.12k op/s 2024-05-09T09:11:11.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:10 smithi139 ceph-mon[33599]: pgmap v431: 129 pgs: 129 active+clean; 952 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-09T09:11:11.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:10 smithi064 ceph-mon[30149]: pgmap v431: 129 pgs: 129 active+clean; 952 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-09T09:11:11.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:10 smithi179 ceph-mon[34086]: pgmap v431: 129 pgs: 129 active+clean; 952 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-09T09:11:11.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 50.0 s, 947.6 tps, lat 64.403 ms stddev 27.863 2024-05-09T09:11:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 -- 172.21.15.64:0/2783351699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 msgr2=0x7f9a00073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 --2- 172.21.15.64:0/2783351699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a00073220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009900 tx=0x7f99f002a400 comp rx=0 tx=0).stop 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 -- 172.21.15.64:0/2783351699 shutdown_connections 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 --2- 172.21.15.64:0/2783351699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a0010fb70 0x7f9a00111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 --2- 172.21.15.64:0/2783351699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a00073220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 --2- 172.21.15.64:0/2783351699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a00109440 0x7f9a00072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 -- 172.21.15.64:0/2783351699 >> 172.21.15.64:0/2783351699 conn(0x7f9a0006e6d0 msgr2=0x7f9a00070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.826+0000 7f9a05814640 1 -- 172.21.15.64:0/2783351699 shutdown_connections 2024-05-09T09:11:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.827+0000 7f9a05814640 1 -- 172.21.15.64:0/2783351699 wait complete. 2024-05-09T09:11:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.828+0000 7f9a05814640 1 Processor -- start 2024-05-09T09:11:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.828+0000 7f9a05814640 1 -- start start 2024-05-09T09:11:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.828+0000 7f9a05814640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a0019ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f9a05814640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 0x7f9a001a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f9a05814640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a0010fb70 0x7f9a001a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f9a04812640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a0019ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f99fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 0x7f9a001a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f99fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 0x7f9a001a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49110/0 (socket says 172.21.15.64:49110) 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f9a04812640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a0019ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52134/0 (socket says 172.21.15.64:52134) 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f99fffff640 1 -- 172.21.15.64:0/2773751341 learned_addr learned my addr 172.21.15.64:0/2773751341 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00115a00 con 0x7f9a0010fb70 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.829+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00115880 con 0x7f9a00109440 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f9a05013640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a0010fb70 0x7f9a001a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00115b80 con 0x7f9a00072da0 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f99fffff640 1 -- 172.21.15.64:0/2773751341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 msgr2=0x7f9a0019ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f99fffff640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a0019ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f99fffff640 1 -- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a0010fb70 msgr2=0x7f9a001a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f99fffff640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a0010fb70 0x7f9a001a7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.830+0000 7f99fffff640 1 -- 172.21.15.64:0/2773751341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a7c00 con 0x7f9a00109440 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.831+0000 7f99fffff640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 0x7f9a001a0520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f99f00314e0 tx=0x7f99f0003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.831+0000 7f99fdffb640 1 -- 172.21.15.64:0/2773751341 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0035030 con 0x7f9a00109440 2024-05-09T09:11:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.831+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a00109940 con 0x7f9a00109440 2024-05-09T09:11:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.831+0000 7f99fdffb640 1 -- 172.21.15.64:0/2773751341 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f0041070 con 0x7f9a00109440 2024-05-09T09:11:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.832+0000 7f99fdffb640 1 -- 172.21.15.64:0/2773751341 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f00303a0 con 0x7f9a00109440 2024-05-09T09:11:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.832+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a00109dc0 con 0x7f9a00109440 2024-05-09T09:11:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.836+0000 7f99fdffb640 1 -- 172.21.15.64:0/2773751341 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99f0031580 con 0x7f9a00109440 2024-05-09T09:11:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.837+0000 7f9a09788740 1 Processor -- start 2024-05-09T09:11:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.837+0000 7f9a09788740 1 -- start start 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a09788740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b6be0 0x55d9a18b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a09788740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a09788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18bebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a05013640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b6be0 0x55d9a18b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a09788740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9a1812340 con 0x55d9a18b6be0 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a09788740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9a17ba290 con 0x55d9a18b75d0 2024-05-09T09:11:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a09788740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9a180e300 con 0x55d9a18bc740 2024-05-09T09:11:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a05013640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b6be0 0x55d9a18b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43464/0 (socket says 172.21.15.64:43464) 2024-05-09T09:11:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a05013640 1 -- 172.21.15.64:0/483128405 learned_addr learned my addr 172.21.15.64:0/483128405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f9a04812640 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.838+0000 7f99fffff640 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18bebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f99fdffb640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e80a2cc0 0x7f99e80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f9a04812640 1 -- 172.21.15.64:0/483128405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 msgr2=0x55d9a18bebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f9a04812640 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18bebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f9a04812640 1 -- 172.21.15.64:0/483128405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b6be0 msgr2=0x55d9a18b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f9a04812640 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b6be0 0x55d9a18b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f9a04812640 1 -- 172.21.15.64:0/483128405 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a17de8e0 con 0x55d9a18b75d0 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.839+0000 7f9a04812640 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18bc0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f99f400cba0 tx=0x7f99f400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad73fc36fd99cc02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.840+0000 7f99e6ffd640 1 -- 172.21.15.64:0/483128405 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4004230 con 0x55d9a18b75d0 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.840+0000 7f99e6ffd640 1 -- 172.21.15.64:0/483128405 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40043d0 con 0x55d9a18b75d0 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.840+0000 7f9a05013640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e80a2cc0 0x7f99e80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.840+0000 7f99e6ffd640 1 -- 172.21.15.64:0/483128405 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4004cf0 con 0x55d9a18b75d0 2024-05-09T09:11:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f99fdffb640 1 -- 172.21.15.64:0/2773751341 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f00f3dd0 con 0x7f9a00109440 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f9a05013640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e80a2cc0 0x7f99e80a5180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55d9a18b7e30 tx=0x7f99f8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f9a09788740 1 -- 172.21.15.64:0/483128405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 msgr2=0x55d9a18bc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f9a09788740 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18bc0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f99f400cba0 tx=0x7f99f400e720 comp rx=0 tx=0).stop 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f9a09788740 1 -- 172.21.15.64:0/483128405 shutdown_connections 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f9a09788740 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18bebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.841+0000 7f9a09788740 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18bc0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.842+0000 7f9a09788740 1 --2- 172.21.15.64:0/483128405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b6be0 0x55d9a18b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.842+0000 7f9a09788740 1 -- 172.21.15.64:0/483128405 >> 172.21.15.64:0/483128405 conn(0x55d9a18b2250 msgr2=0x55d9a18b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.842+0000 7f9a09788740 1 -- 172.21.15.64:0/483128405 shutdown_connections 2024-05-09T09:11:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.842+0000 7f9a09788740 1 -- 172.21.15.64:0/483128405 wait complete. 2024-05-09T09:11:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.842+0000 7f9a09788740 1 -- start start 2024-05-09T09:11:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.842+0000 7f9a09788740 1 Processor -- start 2024-05-09T09:11:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a09788740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 0x55d9a18c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a09788740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b75d0 0x55d9a18c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a09788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a09788740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9a18bf6f0 con 0x55d9a18b75d0 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a04812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 0x55d9a18c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a09788740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9a17cd790 con 0x55d9a18b6be0 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a04812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 0x55d9a18c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49132/0 (socket says 172.21.15.64:49132) 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a04812640 1 -- 172.21.15.64:0/737812881 learned_addr learned my addr 172.21.15.64:0/737812881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f99fffff640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b75d0 0x55d9a18c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9a18bf870 con 0x55d9a18bc740 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.844+0000 7f9a05013640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a09788740 10 client.0 _ll_register_callbacks cb 0x55d9a1808a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a04812640 1 -- 172.21.15.64:0/737812881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 msgr2=0x55d9a18c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a04812640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a04812640 1 -- 172.21.15.64:0/737812881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b75d0 msgr2=0x55d9a18c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a04812640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b75d0 0x55d9a18c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a04812640 1 -- 172.21.15.64:0/737812881 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a1526210 con 0x55d9a18b6be0 2024-05-09T09:11:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.845+0000 7f9a04812640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 0x55d9a18c0e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99f400cb70 tx=0x7f99f401cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac3f52ae31b32b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f99dffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f99dffff640 1 -- 172.21.15.64:0/737812881 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f402a060 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f40043d0 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f99dffff640 1 -- 172.21.15.64:0/737812881 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f401c070 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f99dffff640 1 -- 172.21.15.64:0/737812881 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99f4031050 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f9a09788740 10 client.25649 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.846+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9a15f2620 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f99dffff640 1 -- 172.21.15.64:0/737812881 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40043d0 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f99dffff640 1 -- 172.21.15.64:0/737812881 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f99f401a900 con 0x55d9a18b6be0 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f9a09788740 10 client.25649 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f9a09788740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f9a09788740 20 client.25649 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f9a09788740 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18cc5f0 0x55d9a18ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.847+0000 7f9a09788740 4 client.25649 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.848+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a181fa70 con 0x55d9a18cc5f0 2024-05-09T09:11:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.848+0000 7f99fffff640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18cc5f0 0x55d9a18ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.848+0000 7f99dffff640 10 client.25649 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.849+0000 7f99fffff640 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18cc5f0 0x55d9a18ce9e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76f39785091bd9f server_cookie=867110e9b858b571 in_seq=0 out_seq=0 2024-05-09T09:11:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.850+0000 7f99dffff640 1 -- 172.21.15.64:0/737812881 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9a181fa70 con 0x55d9a18cc5f0 2024-05-09T09:11:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.850+0000 7f99dffff640 10 client.25649 handle_command_reply: tid=0 2024-05-09T09:11:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.850+0000 7f9a09788740 1 client.25649 shutdown 2024-05-09T09:11:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 msgr2=0x55d9a18c0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 0x55d9a18c0e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99f400cb70 tx=0x7f99f401cb40 comp rx=0 tx=0).stop 2024-05-09T09:11:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 shutdown_connections 2024-05-09T09:11:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18cc5f0 0x55d9a18ce9e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18bc740 0x55d9a18c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18b75d0 0x55d9a18c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 --2- 172.21.15.64:0/737812881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b6be0 0x55d9a18c0e80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 >> 172.21.15.64:0/737812881 conn(0x55d9a18b2250 msgr2=0x55d9a18a8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f99dffff640 0 client.25649 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.851+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 shutdown_connections 2024-05-09T09:11:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.852+0000 7f9a09788740 1 -- 172.21.15.64:0/737812881 wait complete. 2024-05-09T09:11:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.852+0000 7f9a09788740 20 client.25649 trim_cache size 0 max 16384 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a09788740 1 Processor -- start 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a09788740 1 -- start start 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a09788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a09788740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a09788740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a04812640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.884+0000 7f9a04812640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52168/0 (socket says 172.21.15.64:52168) 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f99fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f99fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49138/0 (socket says 172.21.15.64:49138) 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 -- 172.21.15.64:0/764487367 learned_addr learned my addr 172.21.15.64:0/764487367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001b3030 con 0x55d9a18bc740 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001b0870 con 0x55d9a18b75d0 2024-05-09T09:11:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ae6c0 con 0x55d9a18b6be0 2024-05-09T09:11:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a05013640 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 -- 172.21.15.64:0/764487367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 msgr2=0x55d9a18c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 -- 172.21.15.64:0/764487367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 msgr2=0x55d9a18b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 -- 172.21.15.64:0/764487367 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a18b2fc0 con 0x55d9a18b6be0 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.885+0000 7f9a04812640 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18b52c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99f400f010 tx=0x7f99f40058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434cbbc1b8439610 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.886+0000 7f99e67fc640 1 -- 172.21.15.64:0/764487367 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f402a040 con 0x55d9a18b6be0 2024-05-09T09:11:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.886+0000 7f99e67fc640 1 -- 172.21.15.64:0/764487367 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40054f0 con 0x55d9a18b6be0 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.886+0000 7f99e67fc640 1 -- 172.21.15.64:0/764487367 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40109d0 con 0x55d9a18b6be0 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 msgr2=0x55d9a18b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18b52c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99f400f010 tx=0x7f99f40058d0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 shutdown_connections 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 --2- 172.21.15.64:0/764487367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18b52c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.887+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 >> 172.21.15.64:0/764487367 conn(0x55d9a18b2250 msgr2=0x55d9a18b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.888+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 shutdown_connections 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.888+0000 7f9a09788740 1 -- 172.21.15.64:0/764487367 wait complete. 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.888+0000 7f9a09788740 1 -- start start 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.889+0000 7f9a09788740 1 Processor -- start 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.889+0000 7f9a09788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.889+0000 7f9a09788740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a09788740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a09788740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001aed30 con 0x55d9a18bc740 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a09788740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001b01a0 con 0x55d9a18b75d0 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a09788740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001af400 con 0x55d9a18b6be0 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f99fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f99fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49150/0 (socket says 172.21.15.64:49150) 2024-05-09T09:11:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f99fffff640 1 -- 172.21.15.64:0/3602603756 learned_addr learned my addr 172.21.15.64:0/3602603756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a04812640 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a09788740 10 client.0 _ll_register_callbacks cb 0x7f99f4014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a04812640 1 -- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 msgr2=0x55d9a18c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a04812640 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a04812640 1 -- 172.21.15.64:0/3602603756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 msgr2=0x55d9a18c74f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a04812640 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18c74f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.890+0000 7f9a04812640 1 -- 172.21.15.64:0/3602603756 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a18c7ae0 con 0x55d9a18b6be0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.891+0000 7f9a04812640 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18c5ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55d9a18c1e40 tx=0x7f99f401a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=894da624ec794e14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.891+0000 7f99dffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.891+0000 7f99dffff640 1 -- 172.21.15.64:0/3602603756 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f402a040 con 0x55d9a18b6be0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.891+0000 7f99dffff640 1 -- 172.21.15.64:0/3602603756 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f401cb20 con 0x55d9a18b6be0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.891+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f40054f0 con 0x55d9a18b6be0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f99dffff640 1 -- 172.21.15.64:0/3602603756 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99f4005300 con 0x55d9a18b6be0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 10 client.25768 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9a18ae7e0 con 0x55d9a18b6be0 2024-05-09T09:11:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f99dffff640 1 -- 172.21.15.64:0/3602603756 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40054f0 con 0x55d9a18b6be0 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f99dffff640 1 -- 172.21.15.64:0/3602603756 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f99f4010c10 con 0x55d9a18b6be0 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 10 client.25768 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 20 client.25768 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18af1d0 0x55d9a18b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 4 client.25768 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.892+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9a1819dc0 con 0x55d9a18af1d0 2024-05-09T09:11:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.893+0000 7f99fffff640 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18af1d0 0x55d9a18b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.893+0000 7f99fffff640 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18af1d0 0x55d9a18b15f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680c027b38bf5b86 server_cookie=11814c5eb974c26c in_seq=0 out_seq=0 2024-05-09T09:11:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.893+0000 7f99dffff640 10 client.25768 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f99dffff640 1 -- 172.21.15.64:0/3602603756 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9a1819dc0 con 0x55d9a18af1d0 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f99dffff640 10 client.25768 handle_command_reply: tid=0 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 client.25768 shutdown 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 msgr2=0x55d9a18c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18c5ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55d9a18c1e40 tx=0x7f99f401a680 comp rx=0 tx=0).stop 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 shutdown_connections 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d9a18af1d0 0x55d9a18b15f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9a18bc740 0x55d9a18c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9a18b75d0 0x55d9a18c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 --2- 172.21.15.64:0/3602603756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9a18b6be0 0x55d9a18c5ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 >> 172.21.15.64:0/3602603756 conn(0x55d9a18b2250 msgr2=0x55d9a18b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f99dffff640 0 client.25768 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 shutdown_connections 2024-05-09T09:11:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.907+0000 7f9a09788740 1 -- 172.21.15.64:0/3602603756 wait complete. 2024-05-09T09:11:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.908+0000 7f9a09788740 20 client.25768 trim_cache size 0 max 16384 2024-05-09T09:11:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:11:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:11:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:11:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.910+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e80a2cc0 msgr2=0x7f99e80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.910+0000 7f9a05814640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e80a2cc0 0x7f99e80a5180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55d9a18b7e30 tx=0x7f99f8004a40 comp rx=0 tx=0).stop 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 msgr2=0x7f9a001a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 0x7f9a001a0520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f99f00314e0 tx=0x7f99f0003d40 comp rx=0 tx=0).stop 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 shutdown_connections 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e80a2cc0 0x7f99e80a5180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a0010fb70 0x7f9a001a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a00109440 0x7f9a001a0520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 --2- 172.21.15.64:0/2773751341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a00072da0 0x7f9a0019ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 >> 172.21.15.64:0/2773751341 conn(0x7f9a0006e6d0 msgr2=0x7f9a0010bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 shutdown_connections 2024-05-09T09:11:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:11.911+0000 7f9a05814640 1 -- 172.21.15.64:0/2773751341 wait complete. 2024-05-09T09:11:13.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:12 smithi139 ceph-mon[33599]: pgmap v432: 129 pgs: 129 active+clean; 952 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-05-09T09:11:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:12 smithi064 ceph-mon[30149]: pgmap v432: 129 pgs: 129 active+clean; 952 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-05-09T09:11:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:12 smithi179 ceph-mon[34086]: pgmap v432: 129 pgs: 129 active+clean; 952 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-05-09T09:11:15.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:14 smithi139 ceph-mon[33599]: pgmap v433: 129 pgs: 129 active+clean; 954 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-05-09T09:11:15.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:14 smithi064 ceph-mon[30149]: pgmap v433: 129 pgs: 129 active+clean; 954 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-05-09T09:11:15.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:14 smithi179 ceph-mon[34086]: pgmap v433: 129 pgs: 129 active+clean; 954 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-05-09T09:11:16.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 55.0 s, 998.8 tps, lat 167.753 ms stddev 1096.829 2024-05-09T09:11:17.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:16 smithi139 ceph-mon[33599]: pgmap v434: 129 pgs: 129 active+clean; 954 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 1.13k op/s 2024-05-09T09:11:17.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:16 smithi064 ceph-mon[30149]: pgmap v434: 129 pgs: 129 active+clean; 954 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 1.13k op/s 2024-05-09T09:11:17.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:16 smithi179 ceph-mon[34086]: pgmap v434: 129 pgs: 129 active+clean; 954 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 1.13k op/s 2024-05-09T09:11:19.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:18 smithi139 ceph-mon[33599]: pgmap v435: 129 pgs: 129 active+clean; 954 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 1.08k op/s 2024-05-09T09:11:19.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:18 smithi064 ceph-mon[30149]: pgmap v435: 129 pgs: 129 active+clean; 954 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 1.08k op/s 2024-05-09T09:11:19.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:18 smithi179 ceph-mon[34086]: pgmap v435: 129 pgs: 129 active+clean; 954 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 1.08k op/s 2024-05-09T09:11:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:20 smithi139 ceph-mon[33599]: pgmap v436: 129 pgs: 129 active+clean; 956 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:20 smithi064 ceph-mon[30149]: pgmap v436: 129 pgs: 129 active+clean; 956 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:21.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:20 smithi179 ceph-mon[34086]: pgmap v436: 129 pgs: 129 active+clean; 956 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:21.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:11:21.278 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:11:21.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 60.0 s, 992.4 tps, lat 95.765 ms stddev 28.818 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.475+0000 7f289da26640 1 -- 172.21.15.64:0/2770868039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2898109940 msgr2=0x7f289810bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.475+0000 7f289da26640 1 --2- 172.21.15.64:0/2770868039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2898109940 0x7f289810bd30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2890009ca0 tx=0x7f289002a8b0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 -- 172.21.15.64:0/2770868039 shutdown_connections 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 --2- 172.21.15.64:0/2770868039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 0x7f2898110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 --2- 172.21.15.64:0/2770868039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f289810c270 0x7f289810e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 --2- 172.21.15.64:0/2770868039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2898109940 0x7f289810bd30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 -- 172.21.15.64:0/2770868039 >> 172.21.15.64:0/2770868039 conn(0x7f289806e740 msgr2=0x7f2898070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 -- 172.21.15.64:0/2770868039 shutdown_connections 2024-05-09T09:11:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.476+0000 7f289da26640 1 -- 172.21.15.64:0/2770868039 wait complete. 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.477+0000 7f289da26640 1 Processor -- start 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.477+0000 7f289da26640 1 -- start start 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289da26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 0x7f289819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289da26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f289810c270 0x7f289819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289da26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 0x7f28981a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289da26640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28981138c0 con 0x7f289810eba0 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289da26640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2898113740 con 0x7f2898109940 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289ca24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 0x7f289819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289da26640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2898113a40 con 0x7f289810c270 2024-05-09T09:11:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289ca24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 0x7f289819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33452/0 (socket says 172.21.15.64:33452) 2024-05-09T09:11:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289ca24640 1 -- 172.21.15.64:0/1841046862 learned_addr learned my addr 172.21.15.64:0/1841046862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f2897fff640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f289810c270 0x7f289819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.478+0000 7f289d225640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 0x7f28981a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289ca24640 1 -- 172.21.15.64:0/1841046862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f289810c270 msgr2=0x7f289819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289ca24640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f289810c270 0x7f289819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289ca24640 1 -- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 msgr2=0x7f28981a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289ca24640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 0x7f28981a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289ca24640 1 -- 172.21.15.64:0/1841046862 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981b6bb0 con 0x7f2898109940 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289d225640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 0x7f28981a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f2897fff640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f289810c270 0x7f289819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:21.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.479+0000 7f289ca24640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 0x7f289819dde0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2890003ba0 tx=0x7f2890003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.480+0000 7f2895ffb640 1 -- 172.21.15.64:0/1841046862 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890004240 con 0x7f2898109940 2024-05-09T09:11:21.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.480+0000 7f2895ffb640 1 -- 172.21.15.64:0/1841046862 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2890040070 con 0x7f2898109940 2024-05-09T09:11:21.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.480+0000 7f2895ffb640 1 -- 172.21.15.64:0/1841046862 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890031de0 con 0x7f2898109940 2024-05-09T09:11:21.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.481+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28981b6d80 con 0x7f2898109940 2024-05-09T09:11:21.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.481+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981b7190 con 0x7f2898109940 2024-05-09T09:11:21.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.485+0000 7f2895ffb640 1 -- 172.21.15.64:0/1841046862 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28900043e0 con 0x7f2898109940 2024-05-09T09:11:21.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.485+0000 7f28a1992740 1 Processor -- start 2024-05-09T09:11:21.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.485+0000 7f28a1992740 1 -- start start 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.485+0000 7f28a1992740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e757540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.485+0000 7f28a1992740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e75c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.485+0000 7f28a1992740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e75f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f28a1992740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56232e6bbd00 con 0x56232e75cc80 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f28a1992740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56232e6634c0 con 0x56232e757b10 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f28a1992740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56232e6b7cc0 con 0x56232e757120 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e757540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e757540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34666/0 (socket says 172.21.15.64:34666) 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 -- 172.21.15.64:0/2540623303 learned_addr learned my addr 172.21.15.64:0/2540623303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f2897fff640 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e75f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f2895ffb640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28800a2dd0 0x7f28800a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f2895ffb640 1 -- 172.21.15.64:0/1841046862 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2890031b80 con 0x7f2898109940 2024-05-09T09:11:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 -- 172.21.15.64:0/2540623303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 msgr2=0x56232e75c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289ca24640 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e75c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e75c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 -- 172.21.15.64:0/2540623303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 msgr2=0x56232e75f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.486+0000 7f289d225640 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e75f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.487+0000 7f289d225640 1 -- 172.21.15.64:0/2540623303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232e4997d0 con 0x56232e757120 2024-05-09T09:11:21.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.487+0000 7f289d225640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28800a2dd0 0x7f28800a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.487+0000 7f289d225640 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e757540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f288c011590 tx=0x7f288c011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7093e2f9b7e70457 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.487+0000 7f287effd640 1 -- 172.21.15.64:0/2540623303 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c017050 con 0x56232e757120 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.487+0000 7f289d225640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28800a2dd0 0x7f28800a5290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f288c023000 tx=0x7f288c024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.488+0000 7f287effd640 1 -- 172.21.15.64:0/2540623303 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f288c01bd80 con 0x56232e757120 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.488+0000 7f289ca24640 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e75c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.488+0000 7f287effd640 1 -- 172.21.15.64:0/2540623303 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c026c00 con 0x56232e757120 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 -- 172.21.15.64:0/2540623303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 msgr2=0x56232e757540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e757540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f288c011590 tx=0x7f288c011a50 comp rx=0 tx=0).stop 2024-05-09T09:11:21.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 -- 172.21.15.64:0/2540623303 shutdown_connections 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e75f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e75c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 --2- 172.21.15.64:0/2540623303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e757540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 -- 172.21.15.64:0/2540623303 >> 172.21.15.64:0/2540623303 conn(0x56232e752790 msgr2=0x56232e752b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 -- 172.21.15.64:0/2540623303 shutdown_connections 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.489+0000 7f28a1992740 1 -- 172.21.15.64:0/2540623303 wait complete. 2024-05-09T09:11:21.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 -- start start 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 Processor -- start 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e761af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e768ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56232e75fcd0 con 0x56232e75cc80 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56232e676b90 con 0x56232e757b10 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56232e75fe50 con 0x56232e757120 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.490+0000 7f28a1992740 10 client.0 _ll_register_callbacks cb 0x56232e6b2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34678/0 (socket says 172.21.15.64:34678) 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 -- 172.21.15.64:0/370888821 learned_addr learned my addr 172.21.15.64:0/370888821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f2897fff640 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e761af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 -- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 msgr2=0x56232e761af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e761af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 -- 172.21.15.64:0/370888821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 msgr2=0x56232e768ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e768ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f289ca24640 1 -- 172.21.15.64:0/370888821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232e3cf3f0 con 0x56232e757120 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.491+0000 7f2877fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.492+0000 7f289ca24640 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7615b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28900c5310 tx=0x7f28900bc530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126f59ff4f3e16eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.492+0000 7f2877fff640 1 -- 172.21.15.64:0/370888821 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900bcbd0 con 0x56232e757120 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.492+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288c01bd80 con 0x56232e757120 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.492+0000 7f2877fff640 1 -- 172.21.15.64:0/370888821 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28900314d0 con 0x56232e757120 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f2877fff640 1 -- 172.21.15.64:0/370888821 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28900c0f80 con 0x56232e757120 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 10 client.25780 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56232e38f0f0 con 0x56232e757120 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f2877fff640 1 -- 172.21.15.64:0/370888821 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f288c01bd80 con 0x56232e757120 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f2877fff640 1 -- 172.21.15.64:0/370888821 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f28900c53b0 con 0x56232e757120 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 10 client.25780 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 10 client.25780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 20 client.25780 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e76d300 0x56232e76f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 4 client.25780 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.493+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56232e6c9430 con 0x56232e76d300 2024-05-09T09:11:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.494+0000 7f2897fff640 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e76d300 0x56232e76f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.495+0000 7f2897fff640 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e76d300 0x56232e76f7c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4487de7159e85b server_cookie=7b0ca8498bc80835 in_seq=0 out_seq=0 2024-05-09T09:11:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.495+0000 7f2877fff640 10 client.25780 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.496+0000 7f2877fff640 1 -- 172.21.15.64:0/370888821 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56232e6c9430 con 0x56232e76d300 2024-05-09T09:11:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.496+0000 7f2877fff640 10 client.25780 handle_command_reply: tid=0 2024-05-09T09:11:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.496+0000 7f28a1992740 1 client.25780 shutdown 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.496+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 msgr2=0x56232e7615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.496+0000 7f28a1992740 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7615b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28900c5310 tx=0x7f28900bc530 comp rx=0 tx=0).stop 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 shutdown_connections 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e76d300 0x56232e76f7c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e768ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e761af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 --2- 172.21.15.64:0/370888821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7615b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 >> 172.21.15.64:0/370888821 conn(0x56232e752790 msgr2=0x56232e751a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f2877fff640 0 client.25780 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 shutdown_connections 2024-05-09T09:11:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 1 -- 172.21.15.64:0/370888821 wait complete. 2024-05-09T09:11:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.497+0000 7f28a1992740 20 client.25780 trim_cache size 0 max 16384 2024-05-09T09:11:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.532+0000 7f28a1992740 1 Processor -- start 2024-05-09T09:11:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.532+0000 7f28a1992740 1 -- start start 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.532+0000 7f28a1992740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.532+0000 7f28a1992740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e768a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f28a1992740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e7556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f289ca24640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f289ca24640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34680/0 (socket says 172.21.15.64:34680) 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f28a1992740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28981c3eb0 con 0x56232e75cc80 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f28a1992740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28981c16f0 con 0x56232e757b10 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f28a1992740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28981bf4e0 con 0x56232e757120 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e768a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e768a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33484/0 (socket says 172.21.15.64:33484) 2024-05-09T09:11:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 -- 172.21.15.64:0/1147159531 learned_addr learned my addr 172.21.15.64:0/1147159531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 -- 172.21.15.64:0/1147159531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 msgr2=0x56232e7618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 -- 172.21.15.64:0/1147159531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 msgr2=0x56232e7556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e7556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.533+0000 7f2897fff640 1 -- 172.21.15.64:0/1147159531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232e743930 con 0x56232e757b10 2024-05-09T09:11:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.534+0000 7f2897fff640 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e768a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2888005940 tx=0x7f288800d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c6124edca11564 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.534+0000 7f287e7fc640 1 -- 172.21.15.64:0/1147159531 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288801fd80 con 0x56232e757b10 2024-05-09T09:11:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.534+0000 7f287e7fc640 1 -- 172.21.15.64:0/1147159531 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f288801f250 con 0x56232e757b10 2024-05-09T09:11:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.534+0000 7f287e7fc640 1 -- 172.21.15.64:0/1147159531 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888003850 con 0x56232e757b10 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.535+0000 7f28a1992740 1 -- 172.21.15.64:0/1147159531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 msgr2=0x56232e768a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e768a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2888005940 tx=0x7f288800d040 comp rx=0 tx=0).stop 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 -- 172.21.15.64:0/1147159531 shutdown_connections 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e75cc80 0x56232e7556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e768a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 --2- 172.21.15.64:0/1147159531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e757120 0x56232e7618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 -- 172.21.15.64:0/1147159531 >> 172.21.15.64:0/1147159531 conn(0x56232e752790 msgr2=0x56232e7611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 -- 172.21.15.64:0/1147159531 shutdown_connections 2024-05-09T09:11:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 -- 172.21.15.64:0/1147159531 wait complete. 2024-05-09T09:11:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 -- start start 2024-05-09T09:11:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.536+0000 7f28a1992740 1 Processor -- start 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.537+0000 7f28a1992740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e757120 0x56232e7667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f28a1992740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e766d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f28a1992740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e75cc80 0x56232e767d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f28a1992740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28981bfbb0 con 0x56232e757120 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f28a1992740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28981c1020 con 0x56232e757b10 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f28a1992740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28981c0280 con 0x56232e75cc80 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f28a1992740 10 client.0 _ll_register_callbacks cb 0x7f2890005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e766d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e766d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33492/0 (socket says 172.21.15.64:33492) 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 -- 172.21.15.64:0/590027329 learned_addr learned my addr 172.21.15.64:0/590027329 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f289ca24640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e757120 0x56232e7667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f289d225640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e75cc80 0x56232e767d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 -- 172.21.15.64:0/590027329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e75cc80 msgr2=0x56232e767d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e75cc80 0x56232e767d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 -- 172.21.15.64:0/590027329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e757120 msgr2=0x56232e7667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e757120 0x56232e7667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f2897fff640 1 -- 172.21.15.64:0/590027329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232e74df30 con 0x56232e757b10 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.538+0000 7f289d225640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e75cc80 0x56232e767d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.539+0000 7f2897fff640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e766d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f288801f630 tx=0x7f288801f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42cd27764a80153a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.539+0000 7f2877fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.539+0000 7f2877fff640 1 -- 172.21.15.64:0/590027329 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288802b070 con 0x56232e757b10 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.539+0000 7f2877fff640 1 -- 172.21.15.64:0/590027329 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f288801f910 con 0x56232e757b10 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.539+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288801f250 con 0x56232e757b10 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f2877fff640 1 -- 172.21.15.64:0/590027329 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2888036050 con 0x56232e757b10 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 10 client.25670 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56232e7d3e30 con 0x56232e757b10 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f2877fff640 1 -- 172.21.15.64:0/590027329 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f288801f250 con 0x56232e757b10 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f2877fff640 1 -- 172.21.15.64:0/590027329 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2888018400 con 0x56232e757b10 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 20 client.25670 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e7d4790 0x56232e7d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 4 client.25670 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.540+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56232e6c3780 con 0x56232e7d4790 2024-05-09T09:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.541+0000 7f289ca24640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e7d4790 0x56232e7d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.542+0000 7f289ca24640 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e7d4790 0x56232e7d6bb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129f5190e8cb1bab server_cookie=2f4e17d2a2c44e2b in_seq=0 out_seq=0 2024-05-09T09:11:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.542+0000 7f2877fff640 10 client.25670 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f2877fff640 1 -- 172.21.15.64:0/590027329 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56232e6c3780 con 0x56232e7d4790 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f2877fff640 10 client.25670 handle_command_reply: tid=0 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 client.25670 shutdown 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 msgr2=0x56232e766d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e766d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f288801f630 tx=0x7f288801f660 comp rx=0 tx=0).stop 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 shutdown_connections 2024-05-09T09:11:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56232e7d4790 0x56232e7d6bb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56232e75cc80 0x56232e767d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56232e757b10 0x56232e766d10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 --2- 172.21.15.64:0/590027329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56232e757120 0x56232e7667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 >> 172.21.15.64:0/590027329 conn(0x56232e752790 msgr2=0x56232e7511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.545+0000 7f2877fff640 0 client.25670 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:21.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.548+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 shutdown_connections 2024-05-09T09:11:21.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.548+0000 7f28a1992740 1 -- 172.21.15.64:0/590027329 wait complete. 2024-05-09T09:11:21.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.548+0000 7f28a1992740 20 client.25670 trim_cache size 0 max 16384 2024-05-09T09:11:21.547 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:11:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:11:21.547 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28800a2dd0 msgr2=0x7f28800a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28800a2dd0 0x7f28800a5290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f288c023000 tx=0x7f288c024000 comp rx=0 tx=0).stop 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 msgr2=0x7f289819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 0x7f289819dde0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2890003ba0 tx=0x7f2890003bd0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 shutdown_connections 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28800a2dd0 0x7f28800a5290 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289810eba0 0x7f28981a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f289810c270 0x7f289819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 --2- 172.21.15.64:0/1841046862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2898109940 0x7f289819dde0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.549+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 >> 172.21.15.64:0/1841046862 conn(0x7f289806e740 msgr2=0x7f289810d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.550+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 shutdown_connections 2024-05-09T09:11:21.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.550+0000 7f289da26640 1 -- 172.21.15.64:0/1841046862 wait complete. 2024-05-09T09:11:21.569 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:11:21.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:11:21.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 -- 172.21.15.64:0/3949336877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 msgr2=0x7f05f8109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 --2- 172.21.15.64:0/3949336877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 0x7f05f8109ce0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009ca0 tx=0x7f05f002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 -- 172.21.15.64:0/3949336877 shutdown_connections 2024-05-09T09:11:21.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 --2- 172.21.15.64:0/3949336877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 0x7f05f8109ce0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 --2- 172.21.15.64:0/3949336877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f8102a90 0x7f05f8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 --2- 172.21.15.64:0/3949336877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05f81018a0 0x7f05f8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 -- 172.21.15.64:0/3949336877 >> 172.21.15.64:0/3949336877 conn(0x7f05f80fce10 msgr2=0x7f05f80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 -- 172.21.15.64:0/3949336877 shutdown_connections 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 -- 172.21.15.64:0/3949336877 wait complete. 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 Processor -- start 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 -- start start 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.729+0000 7f05fe35a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 0x7f05f8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fe35a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05f8102a90 0x7f05f819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fe35a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 0x7f05f81a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fe35a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05f810d420 con 0x7f05f8103450 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fe35a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05f810d2a0 con 0x7f05f81018a0 2024-05-09T09:11:21.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fe35a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05f810d5a0 con 0x7f05f8102a90 2024-05-09T09:11:21.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 0x7f05f8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fc8f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 0x7f05f81a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 0x7f05f8199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33514/0 (socket says 172.21.15.64:33514) 2024-05-09T09:11:21.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 -- 172.21.15.64:0/2915141911 learned_addr learned my addr 172.21.15.64:0/2915141911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:21.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05effff640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05f8102a90 0x7f05f819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 -- 172.21.15.64:0/2915141911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05f8102a90 msgr2=0x7f05f819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05f8102a90 0x7f05f819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 -- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 msgr2=0x7f05f81a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 0x7f05f81a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 -- 172.21.15.64:0/2915141911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f81a18f0 con 0x7f05f81018a0 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f7fff640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 0x7f05f8199cb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f05e800e9c0 tx=0x7f05e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e800cd80 con 0x7f05f81018a0 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f81a1be0 con 0x7f05f81018a0 2024-05-09T09:11:21.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.730+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e80035f0 con 0x7f05f81018a0 2024-05-09T09:11:21.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.731+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8010690 con 0x7f05f81018a0 2024-05-09T09:11:21.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.731+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f81a2130 con 0x7f05f81018a0 2024-05-09T09:11:21.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.732+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f81a2300 con 0x7f05f81018a0 2024-05-09T09:11:21.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.733+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05e80037a0 con 0x7f05f81018a0 2024-05-09T09:11:21.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.734+0000 7f05f5ffb640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05e40a2b90 0x7f05e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:21.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.734+0000 7f05effff640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05e40a2b90 0x7f05e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:21.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.734+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f05e80cf900 con 0x7f05f81018a0 2024-05-09T09:11:21.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.735+0000 7f05effff640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05e40a2b90 0x7f05e40a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f05f819b1b0 tx=0x7f05e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:21.738+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05e809c4d0 con 0x7f05f81018a0 2024-05-09T09:11:22.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.111+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05f81a24d0 con 0x7f05f81018a0 2024-05-09T09:11:22.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.112+0000 7f05f5ffb640 1 -- 172.21.15.64:0/2915141911 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11886 (secure 0 0 0) 0x7f05e809bdd0 con 0x7f05f81018a0 2024-05-09T09:11:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:11:22.111 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:10:25.554787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:11:22.111 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 30 2024-05-09T09:11:22.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05e40a2b90 msgr2=0x7f05e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05e40a2b90 0x7f05e40a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f05f819b1b0 tx=0x7f05e0005f90 comp rx=0 tx=0).stop 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 msgr2=0x7f05f8199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 0x7f05f8199cb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f05e800e9c0 tx=0x7f05e800ee80 comp rx=0 tx=0).stop 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 shutdown_connections 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05e40a2b90 0x7f05e40a5050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05f8103450 0x7f05f81a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05f8102a90 0x7f05f819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 --2- 172.21.15.64:0/2915141911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05f81018a0 0x7f05f8199cb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.115+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 >> 172.21.15.64:0/2915141911 conn(0x7f05f80fce10 msgr2=0x7f05f8073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.116+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 shutdown_connections 2024-05-09T09:11:22.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.116+0000 7f05fe35a640 1 -- 172.21.15.64:0/2915141911 wait complete. 2024-05-09T09:11:22.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:11:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 -- 172.21.15.64:0/3358206548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c09bac0 msgr2=0x7f9b2c09deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 --2- 172.21.15.64:0/3358206548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c09deb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9b34067530 tx=0x7f9b34095fe0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 -- 172.21.15.64:0/3358206548 shutdown_connections 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 --2- 172.21.15.64:0/3358206548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c09deb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 --2- 172.21.15.64:0/3358206548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c099190 0x7f9b2c09b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 --2- 172.21.15.64:0/3358206548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 0x7f9b2c098c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.306+0000 7f9b38b40640 1 -- 172.21.15.64:0/3358206548 >> 172.21.15.64:0/3358206548 conn(0x7f9b2c0901d0 msgr2=0x7f9b2c092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.307+0000 7f9b38b40640 1 -- 172.21.15.64:0/3358206548 shutdown_connections 2024-05-09T09:11:22.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.307+0000 7f9b38b40640 1 -- 172.21.15.64:0/3358206548 wait complete. 2024-05-09T09:11:22.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.308+0000 7f9b38b40640 1 Processor -- start 2024-05-09T09:11:22.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.308+0000 7f9b38b40640 1 -- start start 2024-05-09T09:11:22.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.308+0000 7f9b38b40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 0x7f9b2c132570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b38b40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c099190 0x7f9b2c132ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b38b40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c139b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b38b40640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0a7e10 con 0x7f9b2c096860 2024-05-09T09:11:22.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b38b40640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0a7c90 con 0x7f9b2c099190 2024-05-09T09:11:22.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b38b40640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0a7f90 con 0x7f9b2c09bac0 2024-05-09T09:11:22.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c139b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34722/0 (socket says 172.21.15.64:34722) 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 -- 172.21.15.64:0/783682248 learned_addr learned my addr 172.21.15.64:0/783682248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b337fe640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 0x7f9b2c132570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 -- 172.21.15.64:0/783682248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c099190 msgr2=0x7f9b2c132ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c099190 0x7f9b2c132ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c099190 0x7f9b2c132ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 -- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 msgr2=0x7f9b2c132570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 0x7f9b2c132570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.309+0000 7f9b33fff640 1 -- 172.21.15.64:0/783682248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c13a190 con 0x7f9b2c09bac0 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c099190 0x7f9b2c132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b337fe640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 0x7f9b2c132570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:22.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b33fff640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c139b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b34064d50 tx=0x7f9b3409df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b30ff9640 1 -- 172.21.15.64:0/783682248 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34096c30 con 0x7f9b2c09bac0 2024-05-09T09:11:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b30ff9640 1 -- 172.21.15.64:0/783682248 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b34096dd0 con 0x7f9b2c09bac0 2024-05-09T09:11:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b30ff9640 1 -- 172.21.15.64:0/783682248 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3409fc00 con 0x7f9b2c09bac0 2024-05-09T09:11:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c13a420 con 0x7f9b2c09bac0 2024-05-09T09:11:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.310+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c13a8f0 con 0x7f9b2c09bac0 2024-05-09T09:11:22.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.313+0000 7f9b30ff9640 1 -- 172.21.15.64:0/783682248 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9b34075070 con 0x7f9b2c09bac0 2024-05-09T09:11:22.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 Processor -- start 2024-05-09T09:11:22.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 -- start start 2024-05-09T09:11:22.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48100 0x5646e3a48520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5646e39acc40 con 0x5646e3a48100 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5646e3954580 con 0x5646e3a4dc60 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b3cb90740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646e39a8c00 con 0x5646e3a48af0 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a4d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34730/0 (socket says 172.21.15.64:34730) 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b337fe640 1 -- 172.21.15.64:0/4241576169 learned_addr learned my addr 172.21.15.64:0/4241576169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.314+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48100 0x5646e3a48520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b337fe640 1 -- 172.21.15.64:0/4241576169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 msgr2=0x5646e3a500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b337fe640 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b337fe640 1 -- 172.21.15.64:0/4241576169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48100 msgr2=0x5646e3a48520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b337fe640 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48100 0x5646e3a48520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b337fe640 1 -- 172.21.15.64:0/4241576169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646e36c03f0 con 0x5646e3a48af0 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48100 0x5646e3a48520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b337fe640 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a4d5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9b280114f0 tx=0x7f9b280119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a01419fe92eda7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.315+0000 7f9b1dffb640 1 -- 172.21.15.64:0/4241576169 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28011030 con 0x5646e3a48af0 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.316+0000 7f9b30ff9640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9b240a2d20 0x7f9b240a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.316+0000 7f9b1dffb640 1 -- 172.21.15.64:0/4241576169 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b280111d0 con 0x5646e3a48af0 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.316+0000 7f9b1dffb640 1 -- 172.21.15.64:0/4241576169 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800f5c0 con 0x5646e3a48af0 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.316+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9b240a2d20 0x7f9b240a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4241576169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 msgr2=0x5646e3a4d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a4d5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9b280114f0 tx=0x7f9b280119b0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4241576169 shutdown_connections 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b30ff9640 1 -- 172.21.15.64:0/783682248 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9b3415a700 con 0x7f9b2c09bac0 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a4d5f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4241576169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48100 0x5646e3a48520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4241576169 >> 172.21.15.64:0/4241576169 conn(0x5646e3a43770 msgr2=0x5646e3a43b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9b240a2d20 0x7f9b240a51e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005fd0 tx=0x7f9b2000d360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4241576169 shutdown_connections 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.317+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4241576169 wait complete. 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.318+0000 7f9b3cb90740 1 -- start start 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.318+0000 7f9b3cb90740 1 Processor -- start 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48100 0x5646e3a63510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48af0 0x5646e3a63a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48100 0x5646e3a63510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48100 0x5646e3a63510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34740/0 (socket says 172.21.15.64:34740) 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b337fe640 1 -- 172.21.15.64:0/3227200750 learned_addr learned my addr 172.21.15.64:0/3227200750 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5646e3a50c10 con 0x5646e3a48af0 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48af0 0x5646e3a63a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5646e3967c40 con 0x5646e3a4dc60 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646e3a50d90 con 0x5646e3a48100 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.319+0000 7f9b33fff640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b32ffd640 1 -- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48100 msgr2=0x5646e3a63510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48100 0x5646e3a63510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b32ffd640 1 -- 172.21.15.64:0/3227200750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 msgr2=0x5646e3a6aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a6aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b32ffd640 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646e378a7d0 con 0x5646e3a48af0 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b3cb90740 10 client.0 _ll_register_callbacks cb 0x5646e36bfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b280111d0 con 0x5646e3a48af0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48af0 0x5646e3a63a50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5646e3a49350 tx=0x7f9b20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82516806d3c5b698 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.320+0000 7f9b12ffd640 10 client.16137 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.321+0000 7f9b12ffd640 1 -- 172.21.15.64:0/3227200750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20011d40 con 0x5646e3a48af0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.321+0000 7f9b12ffd640 1 -- 172.21.15.64:0/3227200750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9b2002a470 con 0x5646e3a48af0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.321+0000 7f9b12ffd640 1 -- 172.21.15.64:0/3227200750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b2002a670 con 0x5646e3a48af0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.321+0000 7f9b3cb90740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.321+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5646e3a7ecb0 con 0x5646e3a48af0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b12ffd640 1 -- 172.21.15.64:0/3227200750 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9b2002a810 con 0x5646e3a48af0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b3cb90740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b3cb90740 10 client.16137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b3cb90740 20 client.16137 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3a803f0 0x5646e3a82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b3cb90740 4 client.16137 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646e39ba370 con 0x5646e3a803f0 2024-05-09T09:11:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.322+0000 7f9b337fe640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3a803f0 0x5646e3a82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.323+0000 7f9b337fe640 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3a803f0 0x5646e3a82890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61d61d0958c1ac33 server_cookie=b1fa291ef55d83b6 in_seq=0 out_seq=0 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.323+0000 7f9b12ffd640 10 client.16137 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.323+0000 7f9b12ffd640 1 -- 172.21.15.64:0/3227200750 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646e39ba370 con 0x5646e3a803f0 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.323+0000 7f9b12ffd640 10 client.16137 handle_command_reply: tid=0 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.323+0000 7f9b3cb90740 1 client.16137 shutdown 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48af0 msgr2=0x5646e3a63a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48af0 0x5646e3a63a50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5646e3a49350 tx=0x7f9b20004730 comp rx=0 tx=0).stop 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 shutdown_connections 2024-05-09T09:11:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3a803f0 0x5646e3a82890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a4dc60 0x5646e3a6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a48af0 0x5646e3a63a50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/3227200750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48100 0x5646e3a63510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 >> 172.21.15.64:0/3227200750 conn(0x5646e3a43770 msgr2=0x5646e3a42990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b12ffd640 0 client.16137 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 shutdown_connections 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 1 -- 172.21.15.64:0/3227200750 wait complete. 2024-05-09T09:11:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.324+0000 7f9b3cb90740 20 client.16137 trim_cache size 0 max 16384 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 Processor -- start 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 -- start start 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a6aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3a63870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c13ef10 con 0x5646e3a4dc60 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c13c750 con 0x5646e3a48100 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.348+0000 7f9b3cb90740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0a6f90 con 0x5646e3a48af0 2024-05-09T09:11:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a6aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a6aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34756/0 (socket says 172.21.15.64:34756) 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b32ffd640 1 -- 172.21.15.64:0/386857015 learned_addr learned my addr 172.21.15.64:0/386857015 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b33fff640 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3a63870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 -- 172.21.15.64:0/386857015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 msgr2=0x5646e3a6aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a6aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 -- 172.21.15.64:0/386857015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 msgr2=0x5646e3a63870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3a63870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 -- 172.21.15.64:0/386857015 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646e3a6b850 con 0x5646e3a48100 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.349+0000 7f9b337fe640 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a82400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b28000c00 tx=0x7f9b28009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b38f97306318d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.350+0000 7f9b1d7fa640 1 -- 172.21.15.64:0/386857015 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2802f070 con 0x5646e3a48100 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.350+0000 7f9b1d7fa640 1 -- 172.21.15.64:0/386857015 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b2802bbc0 con 0x5646e3a48100 2024-05-09T09:11:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.350+0000 7f9b1d7fa640 1 -- 172.21.15.64:0/386857015 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2802b7c0 con 0x5646e3a48100 2024-05-09T09:11:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.351+0000 7f9b3cb90740 1 -- 172.21.15.64:0/386857015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 msgr2=0x5646e3a82400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.351+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a82400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b28000c00 tx=0x7f9b28009260 comp rx=0 tx=0).stop 2024-05-09T09:11:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 -- 172.21.15.64:0/386857015 shutdown_connections 2024-05-09T09:11:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3a63870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/386857015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a82400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 -- 172.21.15.64:0/386857015 >> 172.21.15.64:0/386857015 conn(0x5646e3a43770 msgr2=0x5646e3a63160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 -- 172.21.15.64:0/386857015 shutdown_connections 2024-05-09T09:11:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.352+0000 7f9b3cb90740 1 -- 172.21.15.64:0/386857015 wait complete. 2024-05-09T09:11:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.353+0000 7f9b3cb90740 1 -- start start 2024-05-09T09:11:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.353+0000 7f9b3cb90740 1 Processor -- start 2024-05-09T09:11:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.353+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.353+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a67fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.353+0000 7f9b3cb90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3ae4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a67fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a67fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34772/0 (socket says 172.21.15.64:34772) 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a67a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33542/0 (socket says 172.21.15.64:33542) 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 -- 172.21.15.64:0/4205770895 learned_addr learned my addr 172.21.15.64:0/4205770895 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c13ac10 con 0x5646e3a4dc60 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c13c080 con 0x5646e3a48100 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c13b2e0 con 0x5646e3a48af0 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b3cb90740 10 client.0 _ll_register_callbacks cb 0x7f9b20012520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b33fff640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3ae4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 -- 172.21.15.64:0/4205770895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 msgr2=0x5646e3a67a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a67a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 -- 172.21.15.64:0/4205770895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 msgr2=0x5646e3ae4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.354+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3ae4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b32ffd640 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646e3ae4ff0 con 0x5646e3a48af0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b33fff640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3ae4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2802bbc0 con 0x5646e3a48af0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b32ffd640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a67fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000d8d0 tx=0x7f9b2001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c1c5a59a2876a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b12ffd640 10 client.25807 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b12ffd640 1 -- 172.21.15.64:0/4205770895 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2002a470 con 0x5646e3a48af0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b12ffd640 1 -- 172.21.15.64:0/4205770895 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9b20021650 con 0x5646e3a48af0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b12ffd640 1 -- 172.21.15.64:0/4205770895 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b20021870 con 0x5646e3a48af0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b3cb90740 10 client.25807 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.355+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5646e3ae6180 con 0x5646e3a48af0 2024-05-09T09:11:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b12ffd640 1 -- 172.21.15.64:0/4205770895 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9b20021a10 con 0x5646e3a48af0 2024-05-09T09:11:22.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b3cb90740 10 client.25807 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:22.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b3cb90740 10 client.25807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:11:22.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b3cb90740 20 client.25807 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:22.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3ae6a20 0x5646e3ae8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b3cb90740 4 client.25807 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:11:22.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.356+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646e39b46c0 con 0x5646e3ae6a20 2024-05-09T09:11:22.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.357+0000 7f9b337fe640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3ae6a20 0x5646e3ae8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.357+0000 7f9b337fe640 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3ae6a20 0x5646e3ae8e40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc1a60415ee52c2a server_cookie=bf30657d07ef6015 in_seq=0 out_seq=0 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.357+0000 7f9b12ffd640 10 client.25807 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b12ffd640 1 -- 172.21.15.64:0/4205770895 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646e39b46c0 con 0x5646e3ae6a20 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b12ffd640 10 client.25807 handle_command_reply: tid=0 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 client.25807 shutdown 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 msgr2=0x5646e3a67fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a67fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000d8d0 tx=0x7f9b2001b000 comp rx=0 tx=0).stop 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 shutdown_connections 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5646e3ae6a20 0x5646e3ae8e40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646e3a4dc60 0x5646e3ae4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b12ffd640 0 client.25807 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646e3a48af0 0x5646e3a67fc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 --2- 172.21.15.64:0/4205770895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646e3a48100 0x5646e3a67a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 >> 172.21.15.64:0/4205770895 conn(0x5646e3a43770 msgr2=0x5646e3a42100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 shutdown_connections 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.358+0000 7f9b3cb90740 1 -- 172.21.15.64:0/4205770895 wait complete. 2024-05-09T09:11:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.359+0000 7f9b3cb90740 20 client.25807 trim_cache size 0 max 16384 2024-05-09T09:11:22.363 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.359+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9b240a2d20 msgr2=0x7f9b240a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.359+0000 7f9b38b40640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9b240a2d20 0x7f9b240a51e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005fd0 tx=0x7f9b2000d360 comp rx=0 tx=0).stop 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 msgr2=0x7f9b2c139b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c139b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b34064d50 tx=0x7f9b3409df90 comp rx=0 tx=0).stop 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 shutdown_connections 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9b240a2d20 0x7f9b240a51e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b2c09bac0 0x7f9b2c139b20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2c099190 0x7f9b2c132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 --2- 172.21.15.64:0/783682248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2c096860 0x7f9b2c132570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 >> 172.21.15.64:0/783682248 conn(0x7f9b2c0901d0 msgr2=0x7f9b2c097160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 shutdown_connections 2024-05-09T09:11:22.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.360+0000 7f9b38b40640 1 -- 172.21.15.64:0/783682248 wait complete. 2024-05-09T09:11:22.378 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:11:22.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 -- 172.21.15.64:0/1949750269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 msgr2=0x7f06d8108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/1949750269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d8108c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009900 tx=0x7f06c802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 -- 172.21.15.64:0/1949750269 shutdown_connections 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/1949750269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d81100b0 0x7f06d81124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/1949750269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d8109190 0x7f06d810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/1949750269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d8108c50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 -- 172.21.15.64:0/1949750269 >> 172.21.15.64:0/1949750269 conn(0x7f06d806e6c0 msgr2=0x7f06d8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.555+0000 7f06dd3c8640 1 -- 172.21.15.64:0/1949750269 shutdown_connections 2024-05-09T09:11:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.556+0000 7f06dd3c8640 1 -- 172.21.15.64:0/1949750269 wait complete. 2024-05-09T09:11:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.556+0000 7f06dd3c8640 1 Processor -- start 2024-05-09T09:11:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 -- start start 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d81a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d8109190 0x7f06d81a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d81a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d81a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34802/0 (socket says 172.21.15.64:34802) 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06d6ffd640 1 -- 172.21.15.64:0/970646257 learned_addr learned my addr 172.21.15.64:0/970646257 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d81100b0 0x7f06d81aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06d67fc640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d8109190 0x7f06d81a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8115c20 con 0x7f06d81100b0 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8115aa0 con 0x7f06d8109190 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d77fe640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d81100b0 0x7f06d81aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.557+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8115da0 con 0x7f06d8108830 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d67fc640 1 -- 172.21.15.64:0/970646257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 msgr2=0x7f06d81a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d67fc640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d81a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d67fc640 1 -- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d81100b0 msgr2=0x7f06d81aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d67fc640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d81100b0 0x7f06d81aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d67fc640 1 -- 172.21.15.64:0/970646257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d81ae540 con 0x7f06d8109190 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d81a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06d67fc640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d8109190 0x7f06d81a6e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f06d8073030 tx=0x7f06cc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.558+0000 7f06bffff640 1 -- 172.21.15.64:0/970646257 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc017070 con 0x7f06d8109190 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.559+0000 7f06bffff640 1 -- 172.21.15.64:0/970646257 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06cc009c30 con 0x7f06d8109190 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.559+0000 7f06bffff640 1 -- 172.21.15.64:0/970646257 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00f620 con 0x7f06d8109190 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.559+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81ae830 con 0x7f06d8109190 2024-05-09T09:11:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.559+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d81aed30 con 0x7f06d8109190 2024-05-09T09:11:22.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.562+0000 7f06e01b1740 1 Processor -- start 2024-05-09T09:11:22.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.562+0000 7f06e01b1740 1 -- start start 2024-05-09T09:11:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.562+0000 7f06bffff640 1 -- 172.21.15.64:0/970646257 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06cc01a070 con 0x7f06d8109190 2024-05-09T09:11:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a68360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a6daa0 0x5607d5a6ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06e01b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5607d59ccc40 con 0x5607d5a68930 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06e01b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5607d5974580 con 0x5607d5a67f40 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06e01b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5607d59c8c00 con 0x5607d5a6daa0 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06d67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a6daa0 0x5607d5a6ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a68360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06d77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a68360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33572/0 (socket says 172.21.15.64:33572) 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06d77fe640 1 -- 172.21.15.64:0/3019959041 learned_addr learned my addr 172.21.15.64:0/3019959041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.563+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d77fe640 1 -- 172.21.15.64:0/3019959041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a6daa0 msgr2=0x5607d5a6ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d77fe640 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a6daa0 0x5607d5a6ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d77fe640 1 -- 172.21.15.64:0/3019959041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 msgr2=0x5607d5a6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d77fe640 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a6d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d77fe640 1 -- 172.21.15.64:0/3019959041 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d56e03f0 con 0x5607d5a67f40 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d67fc640 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a6daa0 0x5607d5a6ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.564+0000 7f06d77fe640 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a68360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f30 tx=0x7f06d00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba148a264ca740e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.565+0000 7f06bffff640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06c00a2c10 0x7f06c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.565+0000 7f06bffff640 1 -- 172.21.15.64:0/970646257 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06cc0ced30 con 0x7f06d8109190 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.565+0000 7f06bd7fa640 1 -- 172.21.15.64:0/3019959041 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004d20 con 0x5607d5a67f40 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.565+0000 7f06bd7fa640 1 -- 172.21.15.64:0/3019959041 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d0004050 con 0x5607d5a67f40 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.565+0000 7f06bd7fa640 1 -- 172.21.15.64:0/3019959041 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0005260 con 0x5607d5a67f40 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06c00a2c10 0x7f06c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 -- 172.21.15.64:0/3019959041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 msgr2=0x5607d5a68360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a68360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f30 tx=0x7f06d00118d0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06c00a2c10 0x7f06c00a50d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f06c8000c00 tx=0x7f06c8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 -- 172.21.15.64:0/3019959041 shutdown_connections 2024-05-09T09:11:22.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a6daa0 0x5607d5a6ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3019959041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a68360 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.566+0000 7f06e01b1740 1 -- 172.21.15.64:0/3019959041 >> 172.21.15.64:0/3019959041 conn(0x5607d5a635b0 msgr2=0x5607d5a639b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.567+0000 7f06e01b1740 1 -- 172.21.15.64:0/3019959041 shutdown_connections 2024-05-09T09:11:22.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.567+0000 7f06e01b1740 1 -- 172.21.15.64:0/3019959041 wait complete. 2024-05-09T09:11:22.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.567+0000 7f06e01b1740 1 -- start start 2024-05-09T09:11:22.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 Processor -- start 2024-05-09T09:11:22.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a68930 0x5607d5a72930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5607d5a70a50 con 0x5607d5a6daa0 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5607d5987c40 con 0x5607d5a68930 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06e01b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5607d5a70bd0 con 0x5607d5a67f40 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34822/0 (socket says 172.21.15.64:34822) 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06d6ffd640 1 -- 172.21.15.64:0/1180733315 learned_addr learned my addr 172.21.15.64:0/1180733315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06d77fe640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.568+0000 7f06d67fc640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a68930 0x5607d5a72930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06d77fe640 1 -- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 msgr2=0x5607d5a723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06d77fe640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06d77fe640 1 -- 172.21.15.64:0/1180733315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a68930 msgr2=0x5607d5a72930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06d77fe640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a68930 0x5607d5a72930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06d77fe640 1 -- 172.21.15.64:0/1180733315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d57aa7d0 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06e01b1740 10 client.0 _ll_register_callbacks cb 0x5607d56dfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d0004050 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06d77fe640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a79970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f06d0011dd0 tx=0x7f06d0003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83974a4274fb5ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.569+0000 7f06baff5640 10 client.16152 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.570+0000 7f06baff5640 1 -- 172.21.15.64:0/1180733315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0029020 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.570+0000 7f06baff5640 1 -- 172.21.15.64:0/1180733315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06d0004400 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.570+0000 7f06baff5640 1 -- 172.21.15.64:0/1180733315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06d0004600 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.570+0000 7f06e01b1740 10 client.16152 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.570+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5607d5a8db30 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06baff5640 1 -- 172.21.15.64:0/1180733315 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f06d0005040 con 0x5607d5a6daa0 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06e01b1740 10 client.16152 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06e01b1740 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:11:22.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06e01b1740 20 client.16152 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06e01b1740 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5a8f590 0x5607d5a91a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06e01b1740 4 client.16152 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607d59da370 con 0x5607d5a8f590 2024-05-09T09:11:22.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.571+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5a8f590 0x5607d5a91a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.572+0000 7f06baff5640 10 client.16152 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:22.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.572+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5a8f590 0x5607d5a91a50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87e6641e2dd6a19d server_cookie=81049a2aa936e in_seq=0 out_seq=0 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06baff5640 1 -- 172.21.15.64:0/1180733315 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607d59da370 con 0x5607d5a8f590 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06baff5640 10 client.16152 handle_command_reply: tid=0 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 client.16152 shutdown 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 msgr2=0x5607d5a79970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a79970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f06d0011dd0 tx=0x7f06d0003c60 comp rx=0 tx=0).stop 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 shutdown_connections 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5a8f590 0x5607d5a91a50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a79970 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a68930 0x5607d5a72930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 --2- 172.21.15.64:0/1180733315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 >> 172.21.15.64:0/1180733315 conn(0x5607d5a635b0 msgr2=0x5607d5a627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06baff5640 0 client.16152 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 shutdown_connections 2024-05-09T09:11:22.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.573+0000 7f06e01b1740 1 -- 172.21.15.64:0/1180733315 wait complete. 2024-05-09T09:11:22.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.574+0000 7f06e01b1740 20 client.16152 trim_cache size 0 max 16384 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 Processor -- start 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 -- start start 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 0x5607d5a79890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b3390 con 0x5607d5a6daa0 2024-05-09T09:11:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b0bd0 con 0x5607d5a67f40 2024-05-09T09:11:22.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06e01b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81a35b0 con 0x5607d5a68930 2024-05-09T09:11:22.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 0x5607d5a79890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 0x5607d5a79890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34838/0 (socket says 172.21.15.64:34838) 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 -- 172.21.15.64:0/801768874 learned_addr learned my addr 172.21.15.64:0/801768874 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 -- 172.21.15.64:0/801768874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 msgr2=0x5607d5a723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 -- 172.21.15.64:0/801768874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 msgr2=0x5607d5a728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.599+0000 7f06d67fc640 1 -- 172.21.15.64:0/801768874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d5a54d20 con 0x5607d5a68930 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.600+0000 7f06d77fe640 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.600+0000 7f06d67fc640 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 0x5607d5a79890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06cc09b240 tx=0x7f06cc0d2810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e04e3eb660b8fad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.600+0000 7f06bcff9640 1 -- 172.21.15.64:0/801768874 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc021040 con 0x5607d5a68930 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.600+0000 7f06bcff9640 1 -- 172.21.15.64:0/801768874 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06cc009de0 con 0x5607d5a68930 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.600+0000 7f06bcff9640 1 -- 172.21.15.64:0/801768874 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0995b0 con 0x5607d5a68930 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 -- 172.21.15.64:0/801768874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 msgr2=0x5607d5a79890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 0x5607d5a79890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06cc09b240 tx=0x7f06cc0d2810 comp rx=0 tx=0).stop 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 -- 172.21.15.64:0/801768874 shutdown_connections 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a6daa0 0x5607d5a728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a68930 0x5607d5a79890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 --2- 172.21.15.64:0/801768874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a67f40 0x5607d5a723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 -- 172.21.15.64:0/801768874 >> 172.21.15.64:0/801768874 conn(0x5607d5a635b0 msgr2=0x5607d5a69e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 -- 172.21.15.64:0/801768874 shutdown_connections 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 -- 172.21.15.64:0/801768874 wait complete. 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.601+0000 7f06e01b1740 1 -- start start 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 Processor -- start 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a79740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a6daa0 0x5607d5a76720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81af0b0 con 0x5607d5a68930 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b0500 con 0x5607d5a6daa0 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81af760 con 0x5607d5a67f40 2024-05-09T09:11:22.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06e01b1740 10 client.0 _ll_register_callbacks cb 0x7f06c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a79740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a79740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34846/0 (socket says 172.21.15.64:34846) 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.602+0000 7f06d6ffd640 1 -- 172.21.15.64:0/3455545310 learned_addr learned my addr 172.21.15.64:0/3455545310 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d67fc640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d77fe640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a6daa0 0x5607d5a76720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d6ffd640 1 -- 172.21.15.64:0/3455545310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a6daa0 msgr2=0x5607d5a76720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a6daa0 0x5607d5a76720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d6ffd640 1 -- 172.21.15.64:0/3455545310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 msgr2=0x5607d5a761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d6ffd640 1 -- 172.21.15.64:0/3455545310 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d5af3bc0 con 0x5607d5a67f40 2024-05-09T09:11:22.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06d6ffd640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a79740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06c8031490 tx=0x7f06c802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dcd6e257cc0d0a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06baff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.603+0000 7f06baff5640 1 -- 172.21.15.64:0/3455545310 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8035050 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06baff5640 1 -- 172.21.15.64:0/3455545310 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06c8031a80 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06cc009de0 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06baff5640 1 -- 172.21.15.64:0/3455545310 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06c8065050 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06e01b1740 10 client.25825 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607d5af4ee0 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06baff5640 1 -- 172.21.15.64:0/3455545310 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06cc009de0 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06baff5640 1 -- 172.21.15.64:0/3455545310 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f06c804a430 con 0x5607d5a67f40 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06e01b1740 10 client.25825 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:22.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06e01b1740 10 client.25825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.604+0000 7f06e01b1740 20 client.25825 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.605+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5af5890 0x5607d5af7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.605+0000 7f06e01b1740 4 client.25825 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.605+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607d59d46c0 con 0x5607d5af5890 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.605+0000 7f06d67fc640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5af5890 0x5607d5af7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.605+0000 7f06baff5640 10 client.25825 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.606+0000 7f06d67fc640 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5af5890 0x5607d5af7cb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5fe34ba5079c758 server_cookie=56b39e80fa0933d2 in_seq=0 out_seq=0 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06baff5640 1 -- 172.21.15.64:0/3455545310 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607d59d46c0 con 0x5607d5af5890 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06baff5640 10 client.25825 handle_command_reply: tid=0 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 client.25825 shutdown 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 msgr2=0x5607d5a79740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a79740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06c8031490 tx=0x7f06c802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 shutdown_connections 2024-05-09T09:11:22.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5607d5a6daa0 0x5607d5a76720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5607d5af5890 0x5607d5af7cb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607d5a68930 0x5607d5a761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 --2- 172.21.15.64:0/3455545310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607d5a67f40 0x5607d5a79740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 >> 172.21.15.64:0/3455545310 conn(0x5607d5a635b0 msgr2=0x5607d5a72330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.607+0000 7f06baff5640 0 client.25825 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.608+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 shutdown_connections 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.608+0000 7f06e01b1740 1 -- 172.21.15.64:0/3455545310 wait complete. 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.608+0000 7f06e01b1740 20 client.25825 trim_cache size 0 max 16384 2024-05-09T09:11:22.617 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:11:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06c00a2c10 msgr2=0x7f06c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06c00a2c10 0x7f06c00a50d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f06c8000c00 tx=0x7f06c8005a20 comp rx=0 tx=0).stop 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d8109190 msgr2=0x7f06d81a6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d8109190 0x7f06d81a6e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f06d8073030 tx=0x7f06cc00dde0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 shutdown_connections 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06c00a2c10 0x7f06c00a50d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06d81100b0 0x7f06d81aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d8109190 0x7f06d81a6e60 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f06d8073030 tx=0x7f06cc00dde0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 --2- 172.21.15.64:0/970646257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06d8108830 0x7f06d81a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 >> 172.21.15.64:0/970646257 conn(0x7f06d806e6c0 msgr2=0x7f06d8070320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 shutdown_connections 2024-05-09T09:11:22.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.609+0000 7f06dd3c8640 1 -- 172.21.15.64:0/970646257 wait complete. 2024-05-09T09:11:22.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:11:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 -- 172.21.15.64:0/2144109569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc101790 msgr2=0x7fb9bc101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 --2- 172.21.15.64:0/2144109569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc101790 0x7fb9bc101bb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009900 tx=0x7fb9b002a410 comp rx=0 tx=0).stop 2024-05-09T09:11:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 -- 172.21.15.64:0/2144109569 shutdown_connections 2024-05-09T09:11:22.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 --2- 172.21.15.64:0/2144109569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 0x7fb9bc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 --2- 172.21.15.64:0/2144109569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc102980 0x7fb9bc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 --2- 172.21.15.64:0/2144109569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc101790 0x7fb9bc101bb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 -- 172.21.15.64:0/2144109569 >> 172.21.15.64:0/2144109569 conn(0x7fb9bc0fcd00 msgr2=0x7fb9bc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 -- 172.21.15.64:0/2144109569 shutdown_connections 2024-05-09T09:11:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.800+0000 7fb9c1431640 1 -- 172.21.15.64:0/2144109569 wait complete. 2024-05-09T09:11:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 Processor -- start 2024-05-09T09:11:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 -- start start 2024-05-09T09:11:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc101790 0x7fb9bc195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102980 0x7fb9bc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 0x7fb9bc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc10b170 con 0x7fb9bc103340 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc10aff0 con 0x7fb9bc102980 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.801+0000 7fb9c1431640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc10b2f0 con 0x7fb9bc101790 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc101790 0x7fb9bc195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc101790 0x7fb9bc195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34868/0 (socket says 172.21.15.64:34868) 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9baffd640 1 -- 172.21.15.64:0/321361492 learned_addr learned my addr 172.21.15.64:0/321361492 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9bb7fe640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 0x7fb9bc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102980 0x7fb9bc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9bb7fe640 1 -- 172.21.15.64:0/321361492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc101790 msgr2=0x7fb9bc195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9bb7fe640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc101790 0x7fb9bc195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9bb7fe640 1 -- 172.21.15.64:0/321361492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102980 msgr2=0x7fb9bc195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9bb7fe640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102980 0x7fb9bc195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9bb7fe640 1 -- 172.21.15.64:0/321361492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc19d490 con 0x7fb9bc103340 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.802+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102980 0x7fb9bc195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:11:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.803+0000 7fb9bb7fe640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 0x7fb9bc19ce20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400eef0 tx=0x7fb9b4012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.803+0000 7fb9a3fff640 1 -- 172.21.15.64:0/321361492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b401c030 con 0x7fb9bc103340 2024-05-09T09:11:22.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.803+0000 7fb9a3fff640 1 -- 172.21.15.64:0/321361492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9b4012e70 con 0x7fb9bc103340 2024-05-09T09:11:22.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.803+0000 7fb9a3fff640 1 -- 172.21.15.64:0/321361492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4018400 con 0x7fb9bc103340 2024-05-09T09:11:22.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.803+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc19d780 con 0x7fb9bc103340 2024-05-09T09:11:22.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.803+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc19dd50 con 0x7fb9bc103340 2024-05-09T09:11:22.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.807+0000 7fb9a3fff640 1 -- 172.21.15.64:0/321361492 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb9b401f070 con 0x7fb9bc103340 2024-05-09T09:11:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.807+0000 7fb9c421a740 1 Processor -- start 2024-05-09T09:11:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.807+0000 7fb9c421a740 1 -- start start 2024-05-09T09:11:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x557009997550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x55700999c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x55700999f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9c421a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5570098fbc40 con 0x55700999cc90 2024-05-09T09:11:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9c421a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5570098a3580 con 0x557009997b20 2024-05-09T09:11:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9c421a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5570098f7c00 con 0x557009997130 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9ba7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x557009997550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x55700999c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x55700999c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33624/0 (socket says 172.21.15.64:33624) 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9baffd640 1 -- 172.21.15.64:0/1092477862 learned_addr learned my addr 172.21.15.64:0/1092477862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9a3fff640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9a40a2c10 0x7fb9a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.808+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x55700999f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9a3fff640 1 -- 172.21.15.64:0/321361492 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9b40d2fe0 con 0x7fb9bc103340 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9baffd640 1 -- 172.21.15.64:0/1092477862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 msgr2=0x557009997550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x557009997550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9baffd640 1 -- 172.21.15.64:0/1092477862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 msgr2=0x55700999f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x55700999f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9bb7fe640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9a40a2c10 0x7fb9a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9baffd640 1 -- 172.21.15.64:0/1092477862 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700960f3f0 con 0x557009997b20 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x55700999f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x55700999c620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb9b002a930 tx=0x7fb9b0004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a28602218dd7be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.809+0000 7fb9bb7fe640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9a40a2c10 0x7fb9a40a50d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb9b409b8f0 tx=0x7fb9b4004020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.810+0000 7fb9a17fa640 1 -- 172.21.15.64:0/1092477862 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0035030 con 0x557009997b20 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.810+0000 7fb9a17fa640 1 -- 172.21.15.64:0/1092477862 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9b0040070 con 0x557009997b20 2024-05-09T09:11:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.810+0000 7fb9a17fa640 1 -- 172.21.15.64:0/1092477862 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0031b40 con 0x557009997b20 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.811+0000 7fb9c421a740 1 -- 172.21.15.64:0/1092477862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 msgr2=0x55700999c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.811+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x55700999c620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb9b002a930 tx=0x7fb9b0004180 comp rx=0 tx=0).stop 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 -- 172.21.15.64:0/1092477862 shutdown_connections 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x55700999f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x55700999c620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1092477862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x557009997550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 -- 172.21.15.64:0/1092477862 >> 172.21.15.64:0/1092477862 conn(0x5570099927a0 msgr2=0x557009992ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 -- 172.21.15.64:0/1092477862 shutdown_connections 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 -- 172.21.15.64:0/1092477862 wait complete. 2024-05-09T09:11:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.812+0000 7fb9c421a740 1 -- start start 2024-05-09T09:11:22.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 Processor -- start 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x5570099a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x5570099a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55700999fc30 con 0x557009997130 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5570098b6c40 con 0x55700999cc90 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55700999fdb0 con 0x557009997b20 2024-05-09T09:11:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9ba7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x5570099a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9ba7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x5570099a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38754/0 (socket says 172.21.15.64:38754) 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1279454671 learned_addr learned my addr 172.21.15.64:0/1279454671 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.813+0000 7fb9c421a740 10 client.0 _ll_register_callbacks cb 0x55700960ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x5570099a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1279454671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 msgr2=0x5570099a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1279454671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 msgr2=0x5570099a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x5570099a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1279454671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570096d97d0 con 0x55700999cc90 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x5570099a8960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac018c10 tx=0x7fb9ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c50b58b7c3c421d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb99eff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.814+0000 7fb99eff5640 1 -- 172.21.15.64:0/1279454671 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac016c00 con 0x55700999cc90 2024-05-09T09:11:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.815+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b0040070 con 0x55700999cc90 2024-05-09T09:11:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.815+0000 7fb99eff5640 1 -- 172.21.15.64:0/1279454671 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9ac016da0 con 0x55700999cc90 2024-05-09T09:11:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.815+0000 7fb99eff5640 1 -- 172.21.15.64:0/1279454671 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9ac028050 con 0x55700999cc90 2024-05-09T09:11:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.815+0000 7fb9c421a740 10 client.25721 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.815+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570099abd20 con 0x55700999cc90 2024-05-09T09:11:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.816+0000 7fb99eff5640 1 -- 172.21.15.64:0/1279454671 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9ac01f7e0 con 0x55700999cc90 2024-05-09T09:11:22.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.816+0000 7fb99eff5640 1 -- 172.21.15.64:0/1279454671 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb9ac020380 con 0x55700999cc90 2024-05-09T09:11:22.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.816+0000 7fb9c421a740 10 client.25721 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:22.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.816+0000 7fb9c421a740 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:22.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.816+0000 7fb9c421a740 20 client.25721 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:22.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.817+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x5570099af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.817+0000 7fb9c421a740 4 client.25721 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.817+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9b0040070 con 0x5570099ad330 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.817+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x5570099af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.818+0000 7fb99eff5640 10 client.25721 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.818+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x5570099af790 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3d601747ec3813 server_cookie=8d233fa8d5c63e2b in_seq=0 out_seq=0 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.818+0000 7fb99eff5640 1 -- 172.21.15.64:0/1279454671 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb9b0040070 con 0x5570099ad330 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.818+0000 7fb99eff5640 10 client.25721 handle_command_reply: tid=0 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.818+0000 7fb9c421a740 1 client.25721 shutdown 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 msgr2=0x5570099a8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x5570099a8960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac018c10 tx=0x7fb9ac004750 comp rx=0 tx=0).stop 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 shutdown_connections 2024-05-09T09:11:22.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x5570099af790 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb99eff5640 0 client.25721 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x5570099a8960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.819+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1279454671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x5570099a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.820+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 >> 172.21.15.64:0/1279454671 conn(0x5570099927a0 msgr2=0x5570099919c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.820+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 shutdown_connections 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.820+0000 7fb9c421a740 1 -- 172.21.15.64:0/1279454671 wait complete. 2024-05-09T09:11:22.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.820+0000 7fb9c421a740 20 client.25721 trim_cache size 0 max 16384 2024-05-09T09:11:22.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 Processor -- start 2024-05-09T09:11:22.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 -- start start 2024-05-09T09:11:22.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x557009995750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x557009995c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1aaaa0 con 0x557009997130 2024-05-09T09:11:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1a82e0 con 0x55700999cc90 2024-05-09T09:11:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9c421a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1a6130 con 0x557009997b20 2024-05-09T09:11:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9baffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34904/0 (socket says 172.21.15.64:34904) 2024-05-09T09:11:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9baffd640 1 -- 172.21.15.64:0/2869941663 learned_addr learned my addr 172.21.15.64:0/2869941663 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x557009995750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x557009995c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9baffd640 1 -- 172.21.15.64:0/2869941663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 msgr2=0x557009995c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.846+0000 7fb9baffd640 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x557009995c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9baffd640 1 -- 172.21.15.64:0/2869941663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 msgr2=0x557009995750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9baffd640 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x557009995750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9baffd640 1 -- 172.21.15.64:0/2869941663 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700998ce40 con 0x557009997b20 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x557009995c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9baffd640 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a8920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0006010 tx=0x7fb9b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6269d05298321c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9a0ff9640 1 -- 172.21.15.64:0/2869941663 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0035040 con 0x557009997b20 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9a0ff9640 1 -- 172.21.15.64:0/2869941663 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9b0040070 con 0x557009997b20 2024-05-09T09:11:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.847+0000 7fb9a0ff9640 1 -- 172.21.15.64:0/2869941663 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0003850 con 0x557009997b20 2024-05-09T09:11:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.848+0000 7fb9c421a740 1 -- 172.21.15.64:0/2869941663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 msgr2=0x5570099a8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.848+0000 7fb9c421a740 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a8920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0006010 tx=0x7fb9b0000bc0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.848+0000 7fb9c421a740 1 -- 172.21.15.64:0/2869941663 shutdown_connections 2024-05-09T09:11:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.848+0000 7fb9c421a740 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55700999cc90 0x557009995c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.848+0000 7fb9c421a740 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997b20 0x5570099a8920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.849+0000 7fb9c421a740 1 --2- 172.21.15.64:0/2869941663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557009997130 0x557009995750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.849+0000 7fb9c421a740 1 -- 172.21.15.64:0/2869941663 >> 172.21.15.64:0/2869941663 conn(0x5570099927a0 msgr2=0x5570099a13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.849+0000 7fb9c421a740 1 -- 172.21.15.64:0/2869941663 shutdown_connections 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.849+0000 7fb9c421a740 1 -- 172.21.15.64:0/2869941663 wait complete. 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.849+0000 7fb9c421a740 1 -- start start 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.849+0000 7fb9c421a740 1 Processor -- start 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x5570099a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x5570099a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x5570099a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1a67a0 con 0x55700999cc90 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1a7c10 con 0x557009997b20 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1a6e70 con 0x557009997130 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9c421a740 10 client.0 _ll_register_callbacks cb 0x7fb9ac009f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9ba7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x5570099a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9ba7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x5570099a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34914/0 (socket says 172.21.15.64:34914) 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1375261268 learned_addr learned my addr 172.21.15.64:0/1375261268 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x5570099a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.850+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x5570099a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 msgr2=0x5570099a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x5570099a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1375261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 msgr2=0x5570099a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x5570099a78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb9ba7fc640 1 -- 172.21.15.64:0/1375261268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570099a7e90 con 0x557009997130 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb99eff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb9ba7fc640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x5570099a6350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009f10 tx=0x7fb9ac018040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd6a9484eaec058 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb99eff5640 1 -- 172.21.15.64:0/1375261268 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac020440 con 0x557009997130 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.851+0000 7fb99eff5640 1 -- 172.21.15.64:0/1375261268 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9ac016bd0 con 0x557009997130 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b0040070 con 0x557009997130 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb99eff5640 1 -- 172.21.15.64:0/1375261268 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9ac0205e0 con 0x557009997130 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 10 client.25840 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557009a1c380 con 0x557009997130 2024-05-09T09:11:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb99eff5640 1 -- 172.21.15.64:0/1375261268 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9ac016dd0 con 0x557009997130 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb99eff5640 1 -- 172.21.15.64:0/1375261268 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb9ac004020 con 0x557009997130 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 10 client.25840 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 20 client.25840 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x557009a1edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 4 client.25840 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.852+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb9b0040070 con 0x5570099ad330 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.853+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x557009a1edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb9baffd640 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x557009a1edd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d8f2617cc164b server_cookie=f09d55ef1077ed08 in_seq=0 out_seq=0 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb99eff5640 10 client.25840 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb99eff5640 1 -- 172.21.15.64:0/1375261268 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb9b0040070 con 0x5570099ad330 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb99eff5640 10 client.25840 handle_command_reply: tid=0 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb9c421a740 1 client.25840 shutdown 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 msgr2=0x5570099a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x5570099a6350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009f10 tx=0x7fb9ac018040 comp rx=0 tx=0).stop 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 shutdown_connections 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.854+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570099ad330 0x557009a1edd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55700999cc90 0x5570099a78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557009997b20 0x5570099a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 1 --2- 172.21.15.64:0/1375261268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557009997130 0x5570099a6350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb99eff5640 0 client.25840 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 >> 172.21.15.64:0/1375261268 conn(0x5570099927a0 msgr2=0x557009995750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 shutdown_connections 2024-05-09T09:11:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 1 -- 172.21.15.64:0/1375261268 wait complete. 2024-05-09T09:11:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.855+0000 7fb9c421a740 20 client.25840 trim_cache size 0 max 16384 2024-05-09T09:11:22.862 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.856+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9a40a2c10 msgr2=0x7fb9a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.856+0000 7fb9c1431640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9a40a2c10 0x7fb9a40a50d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb9b409b8f0 tx=0x7fb9b4004020 comp rx=0 tx=0).stop 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.856+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 msgr2=0x7fb9bc19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.856+0000 7fb9c1431640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 0x7fb9bc19ce20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400eef0 tx=0x7fb9b4012490 comp rx=0 tx=0).stop 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 shutdown_connections 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9a40a2c10 0x7fb9a40a50d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9bc103340 0x7fb9bc19ce20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102980 0x7fb9bc195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 --2- 172.21.15.64:0/321361492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9bc101790 0x7fb9bc195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 >> 172.21.15.64:0/321361492 conn(0x7fb9bc0fcd00 msgr2=0x7fb9bc105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 shutdown_connections 2024-05-09T09:11:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:22.857+0000 7fb9c1431640 1 -- 172.21.15.64:0/321361492 wait complete. 2024-05-09T09:11:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:22 smithi064 ceph-mon[30149]: pgmap v437: 129 pgs: 129 active+clean; 956 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-05-09T09:11:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:22 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2915141911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:11:23.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:22 smithi179 ceph-mon[34086]: pgmap v437: 129 pgs: 129 active+clean; 956 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-05-09T09:11:23.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:22 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2915141911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:11:23.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:22 smithi139 ceph-mon[33599]: pgmap v437: 129 pgs: 129 active+clean; 956 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-05-09T09:11:23.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:22 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2915141911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:11:23.877 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:11:23.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:11:23.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:11:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 -- 172.21.15.64:0/3197238431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300103230 msgr2=0x7fc300109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 --2- 172.21.15.64:0/3197238431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300103230 0x7fc300109ac0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c30 tx=0x7fc2f802a8f0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 -- 172.21.15.64:0/3197238431 shutdown_connections 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 --2- 172.21.15.64:0/3197238431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300103230 0x7fc300109ac0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 --2- 172.21.15.64:0/3197238431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc300102870 0x7fc300102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 --2- 172.21.15.64:0/3197238431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300101680 0x7fc300101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.081+0000 7fc305a07640 1 -- 172.21.15.64:0/3197238431 >> 172.21.15.64:0/3197238431 conn(0x7fc3000fcc10 msgr2=0x7fc3000ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.082+0000 7fc305a07640 1 -- 172.21.15.64:0/3197238431 shutdown_connections 2024-05-09T09:11:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.082+0000 7fc305a07640 1 -- 172.21.15.64:0/3197238431 wait complete. 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 Processor -- start 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 -- start start 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc300101680 0x7fc30019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300102870 0x7fc30019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300103230 0x7fc3001a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc300113780 con 0x7fc300101680 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc300113600 con 0x7fc300102870 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.083+0000 7fc305a07640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc300113900 con 0x7fc300103230 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2fe7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300102870 0x7fc30019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300103230 0x7fc3001a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300103230 0x7fc3001a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34924/0 (socket says 172.21.15.64:34924) 2024-05-09T09:11:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2fffff640 1 -- 172.21.15.64:0/75773950 learned_addr learned my addr 172.21.15.64:0/75773950 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2feffd640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc300101680 0x7fc30019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2fe7fc640 1 -- 172.21.15.64:0/75773950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300103230 msgr2=0x7fc3001a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.084+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300103230 0x7fc3001a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc2fe7fc640 1 -- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc300101680 msgr2=0x7fc30019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc300101680 0x7fc30019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc2fe7fc640 1 -- 172.21.15.64:0/75773950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001a5a70 con 0x7fc300102870 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300102870 0x7fc30019e390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000e9f0 tx=0x7fc2f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc304a05640 1 -- 172.21.15.64:0/75773950 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000ccc0 con 0x7fc300102870 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001a5d60 con 0x7fc300102870 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc304a05640 1 -- 172.21.15.64:0/75773950 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f00108e0 con 0x7fc300102870 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.085+0000 7fc304a05640 1 -- 172.21.15.64:0/75773950 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0018880 con 0x7fc300102870 2024-05-09T09:11:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.086+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a6250 con 0x7fc300102870 2024-05-09T09:11:24.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.089+0000 7fc304a05640 1 -- 172.21.15.64:0/75773950 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2f0010430 con 0x7fc300102870 2024-05-09T09:11:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.090+0000 7fc3087f2740 1 Processor -- start 2024-05-09T09:11:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 -- start start 2024-05-09T09:11:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183136b0 0x561e18313ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 0x561e18318ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e18319210 0x561e1831b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561e18277990 con 0x561e18319210 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561e1821f5b0 con 0x561e183136b0 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc3087f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561e18273950 con 0x561e183140a0 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 0x561e18318ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc304a05640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc2e80a2da0 0x7fc2e80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 0x561e18318ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34938/0 (socket says 172.21.15.64:34938) 2024-05-09T09:11:24.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc2feffd640 1 -- 172.21.15.64:0/4101899196 learned_addr learned my addr 172.21.15.64:0/4101899196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.091+0000 7fc2fffff640 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183136b0 0x561e18313ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc304a05640 1 -- 172.21.15.64:0/75773950 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc2f00cf620 con 0x7fc300102870 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2feffd640 1 -- 172.21.15.64:0/4101899196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183136b0 msgr2=0x561e18313ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2feffd640 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183136b0 0x561e18313ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2feffd640 1 -- 172.21.15.64:0/4101899196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e18319210 msgr2=0x561e1831b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2feffd640 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e18319210 0x561e1831b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2feffd640 1 -- 172.21.15.64:0/4101899196 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e180537e0 con 0x561e183140a0 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2fffff640 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183136b0 0x561e18313ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2fffff640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc2e80a2da0 0x7fc2e80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.092+0000 7fc2feffd640 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 0x561e18318ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400dc80 tx=0x7fc2f400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a37914203785b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.093+0000 7fc2e5ffb640 1 -- 172.21.15.64:0/4101899196 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4005020 con 0x561e183140a0 2024-05-09T09:11:24.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.093+0000 7fc2e5ffb640 1 -- 172.21.15.64:0/4101899196 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f40047a0 con 0x561e183140a0 2024-05-09T09:11:24.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.093+0000 7fc2e5ffb640 1 -- 172.21.15.64:0/4101899196 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40116a0 con 0x561e183140a0 2024-05-09T09:11:24.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.094+0000 7fc3087f2740 1 -- 172.21.15.64:0/4101899196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 msgr2=0x561e18318ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.094+0000 7fc3087f2740 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 0x561e18318ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400dc80 tx=0x7fc2f400f730 comp rx=0 tx=0).stop 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.094+0000 7fc2fffff640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc2e80a2da0 0x7fc2e80a5260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x561e18314900 tx=0x7fc2f800a600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.095+0000 7fc3087f2740 1 -- 172.21.15.64:0/4101899196 shutdown_connections 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.095+0000 7fc3087f2740 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e18319210 0x561e1831b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.095+0000 7fc3087f2740 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183140a0 0x561e18318ba0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.095+0000 7fc3087f2740 1 --2- 172.21.15.64:0/4101899196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183136b0 0x561e18313ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.095+0000 7fc3087f2740 1 -- 172.21.15.64:0/4101899196 >> 172.21.15.64:0/4101899196 conn(0x561e182db0a0 msgr2=0x561e1830f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.095+0000 7fc3087f2740 1 -- 172.21.15.64:0/4101899196 shutdown_connections 2024-05-09T09:11:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.096+0000 7fc3087f2740 1 -- 172.21.15.64:0/4101899196 wait complete. 2024-05-09T09:11:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.096+0000 7fc3087f2740 1 -- start start 2024-05-09T09:11:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.096+0000 7fc3087f2740 1 Processor -- start 2024-05-09T09:11:24.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e1832ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183140a0 0x561e1832f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e18319210 0x561e183361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561e1832d1d0 con 0x561e183140a0 2024-05-09T09:11:24.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561e18232b40 con 0x561e18319210 2024-05-09T09:11:24.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561e1832d350 con 0x561e183136b0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.097+0000 7fc3087f2740 10 client.0 _ll_register_callbacks cb 0x561e182270f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.098+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e1832ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.098+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e1832ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34950/0 (socket says 172.21.15.64:34950) 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.098+0000 7fc2feffd640 1 -- 172.21.15.64:0/1155053907 learned_addr learned my addr 172.21.15.64:0/1155053907 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.098+0000 7fc2fffff640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e18319210 0x561e183361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.098+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183140a0 0x561e1832f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2feffd640 1 -- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e18319210 msgr2=0x561e183361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2feffd640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e18319210 0x561e183361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2feffd640 1 -- 172.21.15.64:0/1155053907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183140a0 msgr2=0x561e1832f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2feffd640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183140a0 0x561e1832f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2feffd640 1 -- 172.21.15.64:0/1155053907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e180b3f70 con 0x561e183136b0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2feffd640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e1832ec90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4019000 tx=0x7fc2f4004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d581d9d1aa02d833 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.099+0000 7fc2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.100+0000 7fc2deffd640 1 -- 172.21.15.64:0/1155053907 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f402a060 con 0x561e183136b0 2024-05-09T09:11:24.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.100+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f40047a0 con 0x561e183136b0 2024-05-09T09:11:24.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.100+0000 7fc2deffd640 1 -- 172.21.15.64:0/1155053907 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc2f400dd70 con 0x561e183136b0 2024-05-09T09:11:24.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.100+0000 7fc2deffd640 1 -- 172.21.15.64:0/1155053907 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2f4031050 con 0x561e183136b0 2024-05-09T09:11:24.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.100+0000 7fc3087f2740 10 client.25852 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:24.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.100+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e17f4b170 con 0x561e183136b0 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc2deffd640 1 -- 172.21.15.64:0/1155053907 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f40047a0 con 0x561e183136b0 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc2deffd640 1 -- 172.21.15.64:0/1155053907 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fc2f4003700 con 0x561e183136b0 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc3087f2740 10 client.25852 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc3087f2740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc3087f2740 20 client.25852 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc3087f2740 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e1834ba10 0x561e1834de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc3087f2740 4 client.25852 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e182850c0 con 0x561e1834ba10 2024-05-09T09:11:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.101+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e1834ba10 0x561e1834de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.102+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e1834ba10 0x561e1834de70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188d5177698a0df server_cookie=c2b4e88f5b4fbe8b in_seq=0 out_seq=0 2024-05-09T09:11:24.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.102+0000 7fc2deffd640 10 client.25852 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.103+0000 7fc2deffd640 1 -- 172.21.15.64:0/1155053907 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e182850c0 con 0x561e1834ba10 2024-05-09T09:11:24.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.103+0000 7fc2deffd640 10 client.25852 handle_command_reply: tid=0 2024-05-09T09:11:24.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.103+0000 7fc3087f2740 1 client.25852 shutdown 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 msgr2=0x561e1832ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e1832ec90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4019000 tx=0x7fc2f4004b80 comp rx=0 tx=0).stop 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 shutdown_connections 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e1834ba10 0x561e1834de70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e18319210 0x561e183361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183140a0 0x561e1832f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 --2- 172.21.15.64:0/1155053907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e1832ec90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 >> 172.21.15.64:0/1155053907 conn(0x561e182db0a0 msgr2=0x561e1830e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc2deffd640 0 client.25852 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 shutdown_connections 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 1 -- 172.21.15.64:0/1155053907 wait complete. 2024-05-09T09:11:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.104+0000 7fc3087f2740 20 client.25852 trim_cache size 0 max 16384 2024-05-09T09:11:24.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.142+0000 7fc3087f2740 1 Processor -- start 2024-05-09T09:11:24.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.142+0000 7fc3087f2740 1 -- start start 2024-05-09T09:11:24.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.142+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e182db0a0 0x561e18352770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e18336160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e18352cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc3087f2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec012510 con 0x561e182db0a0 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc3087f2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec00fd50 con 0x561e183140a0 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc3087f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec00dba0 con 0x561e183136b0 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e18352cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc2fe7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e18336160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e182db0a0 0x561e18352770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e18352cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33690/0 (socket says 172.21.15.64:33690) 2024-05-09T09:11:24.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc2fe7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e18336160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34952/0 (socket says 172.21.15.64:34952) 2024-05-09T09:11:24.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.143+0000 7fc2fffff640 1 -- 172.21.15.64:0/2470202641 learned_addr learned my addr 172.21.15.64:0/2470202641 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2fffff640 1 -- 172.21.15.64:0/2470202641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 msgr2=0x561e18336160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2fffff640 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e18336160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2fffff640 1 -- 172.21.15.64:0/2470202641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e182db0a0 msgr2=0x561e18352770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2fffff640 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e182db0a0 0x561e18352770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2fffff640 1 -- 172.21.15.64:0/2470202641 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1831ba60 con 0x561e183140a0 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2feffd640 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e182db0a0 0x561e18352770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2fffff640 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e18352cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8030640 tx=0x7fc2f8044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0003efce0512264 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2e57fa640 1 -- 172.21.15.64:0/2470202641 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802fcf0 con 0x561e183140a0 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2e57fa640 1 -- 172.21.15.64:0/2470202641 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f802d070 con 0x561e183140a0 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.144+0000 7fc2e57fa640 1 -- 172.21.15.64:0/2470202641 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8049690 con 0x561e183140a0 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.145+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470202641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 msgr2=0x561e18352cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.145+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e18352cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8030640 tx=0x7fc2f8044000 comp rx=0 tx=0).stop 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.146+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470202641 shutdown_connections 2024-05-09T09:11:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.146+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e18352cb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.146+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e183136b0 0x561e18336160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.146+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470202641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e182db0a0 0x561e18352770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.146+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470202641 >> 172.21.15.64:0/2470202641 conn(0x561e18314e80 msgr2=0x561e18312100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.147+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470202641 shutdown_connections 2024-05-09T09:11:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.147+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470202641 wait complete. 2024-05-09T09:11:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.148+0000 7fc3087f2740 1 -- start start 2024-05-09T09:11:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.149+0000 7fc3087f2740 1 Processor -- start 2024-05-09T09:11:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.149+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 0x561e18332f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.149+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183136b0 0x561e183334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.149+0000 7fc3087f2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e1831fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 0x561e18332f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 0x561e18332f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34954/0 (socket says 172.21.15.64:34954) 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 -- 172.21.15.64:0/2470872664 learned_addr learned my addr 172.21.15.64:0/2470872664 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183136b0 0x561e183334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2fffff640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e1831fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec00e210 con 0x561e183136b0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec00f680 con 0x561e183140a0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec00e8e0 con 0x561e182db0a0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc3087f2740 10 client.0 _ll_register_callbacks cb 0x7fc2f4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 -- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 msgr2=0x561e1831fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e1831fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 -- 172.21.15.64:0/2470872664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183136b0 msgr2=0x561e183334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183136b0 0x561e183334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.150+0000 7fc2feffd640 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e183202a0 con 0x561e182db0a0 2024-05-09T09:11:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.151+0000 7fc2feffd640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 0x561e18332f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005e40 tx=0x7fc2f400d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2fba95524cd51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.151+0000 7fc2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.151+0000 7fc2deffd640 1 -- 172.21.15.64:0/2470872664 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f401f020 con 0x561e182db0a0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.151+0000 7fc2deffd640 1 -- 172.21.15.64:0/2470872664 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc2f4004680 con 0x561e182db0a0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.151+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f802d070 con 0x561e182db0a0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.152+0000 7fc2deffd640 1 -- 172.21.15.64:0/2470872664 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2f402a070 con 0x561e182db0a0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.152+0000 7fc3087f2740 10 client.25858 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.152+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e18321630 con 0x561e182db0a0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc2deffd640 1 -- 172.21.15.64:0/2470872664 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f400fc30 con 0x561e182db0a0 2024-05-09T09:11:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc2deffd640 1 -- 172.21.15.64:0/2470872664 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fc2f4004e70 con 0x561e182db0a0 2024-05-09T09:11:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc3087f2740 10 client.25858 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc3087f2740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc3087f2740 20 client.25858 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e18319210 0x561e18324000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc3087f2740 4 client.25858 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:11:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.153+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc2f802d070 con 0x561e18319210 2024-05-09T09:11:24.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.154+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e18319210 0x561e18324000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.154+0000 7fc2deffd640 10 client.25858 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.154+0000 7fc2fe7fc640 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e18319210 0x561e18324000 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71486f244fd18a7f server_cookie=6c87a972caee4a2a in_seq=0 out_seq=0 2024-05-09T09:11:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.156+0000 7fc2deffd640 1 -- 172.21.15.64:0/2470872664 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc2f802d070 con 0x561e18319210 2024-05-09T09:11:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.156+0000 7fc2deffd640 10 client.25858 handle_command_reply: tid=0 2024-05-09T09:11:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.156+0000 7fc3087f2740 1 client.25858 shutdown 2024-05-09T09:11:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 msgr2=0x561e18332f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 0x561e18332f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005e40 tx=0x7fc2f400d9d0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 shutdown_connections 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e18319210 0x561e18324000 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc2deffd640 0 client.25858 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e183140a0 0x561e1831fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e183136b0 0x561e183334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 --2- 172.21.15.64:0/2470872664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e182db0a0 0x561e18332f70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.157+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 >> 172.21.15.64:0/2470872664 conn(0x561e18314e80 msgr2=0x561e1830d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.158+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 shutdown_connections 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.158+0000 7fc3087f2740 1 -- 172.21.15.64:0/2470872664 wait complete. 2024-05-09T09:11:24.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.158+0000 7fc3087f2740 20 client.25858 trim_cache size 0 max 16384 2024-05-09T09:11:24.159 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:11:24.159 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:11:24.159 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "f6d3e87d-f078-4913-9478-89695d3c4985", 2024-05-09T09:11:24.159 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:11:24.159 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc2e80a2da0 msgr2=0x7fc2e80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc2e80a2da0 0x7fc2e80a5260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x561e18314900 tx=0x7fc2f800a600 comp rx=0 tx=0).stop 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300102870 msgr2=0x7fc30019e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300102870 0x7fc30019e390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000e9f0 tx=0x7fc2f000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 shutdown_connections 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc2e80a2da0 0x7fc2e80a5260 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc300103230 0x7fc3001a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc300102870 0x7fc30019e390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 --2- 172.21.15.64:0/75773950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc300101680 0x7fc30019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.159+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 >> 172.21.15.64:0/75773950 conn(0x7fc3000fcc10 msgr2=0x7fc300105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.160+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 shutdown_connections 2024-05-09T09:11:24.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.160+0000 7fc305a07640 1 -- 172.21.15.64:0/75773950 wait complete. 2024-05-09T09:11:24.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:11:24.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:11:24.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.392+0000 7f017e15f640 1 -- 172.21.15.64:0/449609289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01781021b0 msgr2=0x7f01781045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.392+0000 7f017e15f640 1 --2- 172.21.15.64:0/449609289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01781021b0 0x7f01781045a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0174009ca0 tx=0x7f017402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.393+0000 7f017e15f640 1 -- 172.21.15.64:0/449609289 shutdown_connections 2024-05-09T09:11:24.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.393+0000 7f017e15f640 1 --2- 172.21.15.64:0/449609289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0178107410 0x7f0178109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.393+0000 7f017e15f640 1 --2- 172.21.15.64:0/449609289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0178104ae0 0x7f0178106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.393+0000 7f017e15f640 1 --2- 172.21.15.64:0/449609289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01781021b0 0x7f01781045a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.393+0000 7f017e15f640 1 -- 172.21.15.64:0/449609289 >> 172.21.15.64:0/449609289 conn(0x7f01780fbbe0 msgr2=0x7f01780fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.394+0000 7f017e15f640 1 -- 172.21.15.64:0/449609289 shutdown_connections 2024-05-09T09:11:24.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.394+0000 7f017e15f640 1 -- 172.21.15.64:0/449609289 wait complete. 2024-05-09T09:11:24.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.394+0000 7f017e15f640 1 Processor -- start 2024-05-09T09:11:24.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 -- start start 2024-05-09T09:11:24.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01781021b0 0x7f0178197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0178104ae0 0x7f0178197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0178107410 0x7f017819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f017810d280 con 0x7f0178107410 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f017810d100 con 0x7f0178104ae0 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017e15f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f017810d400 con 0x7f01781021b0 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017c95c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0178104ae0 0x7f0178197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017d15d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01781021b0 0x7f0178197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017c95c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0178104ae0 0x7f0178197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33710/0 (socket says 172.21.15.64:33710) 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017c95c640 1 -- 172.21.15.64:0/2358474566 learned_addr learned my addr 172.21.15.64:0/2358474566 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.395+0000 7f017d95e640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0178107410 0x7f017819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f017d15d640 1 -- 172.21.15.64:0/2358474566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0178104ae0 msgr2=0x7f0178197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f017d15d640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0178104ae0 0x7f0178197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f017d15d640 1 -- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0178107410 msgr2=0x7f017819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f017d15d640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0178107410 0x7f017819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f017d15d640 1 -- 172.21.15.64:0/2358474566 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01781000a0 con 0x7f01781021b0 2024-05-09T09:11:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f017d15d640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01781021b0 0x7f0178197980 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f017402ade0 tx=0x7f0174003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.396+0000 7f016e7fc640 1 -- 172.21.15.64:0/2358474566 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174004040 con 0x7f01781021b0 2024-05-09T09:11:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.397+0000 7f016e7fc640 1 -- 172.21.15.64:0/2358474566 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0174004690 con 0x7f01781021b0 2024-05-09T09:11:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.397+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0178100330 con 0x7f01781021b0 2024-05-09T09:11:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.397+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01781008d0 con 0x7f01781021b0 2024-05-09T09:11:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.397+0000 7f016e7fc640 1 -- 172.21.15.64:0/2358474566 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174030cd0 con 0x7f01781021b0 2024-05-09T09:11:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.402+0000 7f01821aa740 1 Processor -- start 2024-05-09T09:11:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.403+0000 7f01821aa740 1 -- start start 2024-05-09T09:11:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.403+0000 7f016e7fc640 1 -- 172.21.15.64:0/2358474566 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f017402f4c0 con 0x7f01781021b0 2024-05-09T09:11:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.403+0000 7f01821aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf39d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.403+0000 7f01821aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf3ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.403+0000 7f01821aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf39d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d15d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf3ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf39d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34984/0 (socket says 172.21.15.64:34984) 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d15d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf3ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38850/0 (socket says 172.21.15.64:38850) 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f01821aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55680ce95d00 con 0x55680cf3a2e0 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f01821aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55680ce3d4c0 con 0x55680cf3f450 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f01821aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55680ce91cc0 con 0x55680cf398f0 2024-05-09T09:11:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 -- 172.21.15.64:0/4001330346 learned_addr learned my addr 172.21.15.64:0/4001330346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 -- 172.21.15.64:0/4001330346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 msgr2=0x55680cf418e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:11:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf418e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 -- 172.21.15.64:0/4001330346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 msgr2=0x55680cf3ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf3ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 -- 172.21.15.64:0/4001330346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680cc737d0 con 0x55680cf398f0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.404+0000 7f017d95e640 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf39d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0170009c20 tx=0x7f0170011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3077bde4c8c9c836 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.405+0000 7f016e7fc640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f01640a2c90 0x7f01640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.405+0000 7f01637fe640 1 -- 172.21.15.64:0/4001330346 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017001e070 con 0x55680cf398f0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.406+0000 7f01637fe640 1 -- 172.21.15.64:0/4001330346 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0170019ae0 con 0x55680cf398f0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.406+0000 7f017d15d640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f01640a2c90 0x7f01640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.406+0000 7f01637fe640 1 -- 172.21.15.64:0/4001330346 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017000e940 con 0x55680cf398f0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.406+0000 7f016e7fc640 1 -- 172.21.15.64:0/2358474566 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f01740f0640 con 0x7f01781021b0 2024-05-09T09:11:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 -- 172.21.15.64:0/4001330346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 msgr2=0x55680cf39d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f017d15d640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f01640a2c90 0x7f01640a5150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0174000f80 tx=0x7f01740095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf39d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0170009c20 tx=0x7f0170011810 comp rx=0 tx=0).stop 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 -- 172.21.15.64:0/4001330346 shutdown_connections 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf3ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 --2- 172.21.15.64:0/4001330346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf39d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 -- 172.21.15.64:0/4001330346 >> 172.21.15.64:0/4001330346 conn(0x55680cf34f60 msgr2=0x55680cf35360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 -- 172.21.15.64:0/4001330346 shutdown_connections 2024-05-09T09:11:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.407+0000 7f01821aa740 1 -- 172.21.15.64:0/4001330346 wait complete. 2024-05-09T09:11:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.408+0000 7f01821aa740 1 -- start start 2024-05-09T09:11:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.408+0000 7f01821aa740 1 Processor -- start 2024-05-09T09:11:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf43e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf4b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55680cf42400 con 0x55680cf3a2e0 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55680ce50b90 con 0x55680cf3f450 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55680cf42580 con 0x55680cf398f0 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f017d95e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf43e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f01821aa740 10 client.0 _ll_register_callbacks cb 0x55680ce8cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f017d95e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf43e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34996/0 (socket says 172.21.15.64:34996) 2024-05-09T09:11:24.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f017d95e640 1 -- 172.21.15.64:0/2118353557 learned_addr learned my addr 172.21.15.64:0/2118353557 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.409+0000 7f017d95e640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf4b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017c95c640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017d95e640 1 -- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 msgr2=0x55680cf4b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017d95e640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf4b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017d95e640 1 -- 172.21.15.64:0/2118353557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 msgr2=0x55680cf443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017d95e640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017d95e640 1 -- 172.21.15.64:0/2118353557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55680cba93f0 con 0x55680cf398f0 2024-05-09T09:11:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017c95c640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f017d95e640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf43e70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0170009870 tx=0x7f0170009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=517275fe1d786f90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.410+0000 7f0160ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.411+0000 7f0160ff9640 1 -- 172.21.15.64:0/2118353557 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017001e040 con 0x55680cf398f0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.411+0000 7f0160ff9640 1 -- 172.21.15.64:0/2118353557 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0170005480 con 0x55680cf398f0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.411+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0170019ae0 con 0x55680cf398f0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.411+0000 7f0160ff9640 1 -- 172.21.15.64:0/2118353557 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0170035050 con 0x55680cf398f0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.411+0000 7f01821aa740 10 client.25870 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.411+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55680cb690f0 con 0x55680cf398f0 2024-05-09T09:11:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f0160ff9640 1 -- 172.21.15.64:0/2118353557 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0170019ae0 con 0x55680cf398f0 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f0160ff9640 1 -- 172.21.15.64:0/2118353557 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f017000e640 con 0x55680cf398f0 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f01821aa740 10 client.25870 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f01821aa740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f01821aa740 20 client.25870 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f01821aa740 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cf50020 0x55680cf52480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f01821aa740 4 client.25870 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55680cea3430 con 0x55680cf50020 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.412+0000 7f017c95c640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cf50020 0x55680cf52480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.413+0000 7f017c95c640 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cf50020 0x55680cf52480 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7509a691c75b4c server_cookie=f97dbbc0756f9932 in_seq=0 out_seq=0 2024-05-09T09:11:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.413+0000 7f0160ff9640 10 client.25870 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f0160ff9640 1 -- 172.21.15.64:0/2118353557 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55680cea3430 con 0x55680cf50020 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f0160ff9640 10 client.25870 handle_command_reply: tid=0 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 client.25870 shutdown 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 msgr2=0x55680cf43e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf43e70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0170009870 tx=0x7f0170009940 comp rx=0 tx=0).stop 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 shutdown_connections 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cf50020 0x55680cf52480 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf4b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 --2- 172.21.15.64:0/2118353557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf398f0 0x55680cf43e70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.414+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 >> 172.21.15.64:0/2118353557 conn(0x55680cf34f60 msgr2=0x55680cf35fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.415+0000 7f0160ff9640 0 client.25870 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.415+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 shutdown_connections 2024-05-09T09:11:24.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.415+0000 7f01821aa740 1 -- 172.21.15.64:0/2118353557 wait complete. 2024-05-09T09:11:24.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.415+0000 7f01821aa740 20 client.25870 trim_cache size 0 max 16384 2024-05-09T09:11:24.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.449+0000 7f01821aa740 1 Processor -- start 2024-05-09T09:11:24.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.449+0000 7f01821aa740 1 -- start start 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f01821aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 0x55680cf3ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f01821aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3a2e0 0x55680cf4b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f01821aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf2ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f01821aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0125f0 con 0x55680cf398f0 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f01821aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f015c00fe30 con 0x55680cf3f450 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f01821aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f015c00dc80 con 0x55680cf3a2e0 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f017d95e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 0x55680cf3ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f017c95c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3a2e0 0x55680cf4b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f017d95e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 0x55680cf3ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38866/0 (socket says 172.21.15.64:38866) 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.450+0000 7f017d95e640 1 -- 172.21.15.64:0/2966759780 learned_addr learned my addr 172.21.15.64:0/2966759780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017d95e640 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf2ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017d95e640 1 -- 172.21.15.64:0/2966759780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3a2e0 msgr2=0x55680cf4b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017d95e640 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3a2e0 0x55680cf4b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017d95e640 1 -- 172.21.15.64:0/2966759780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 msgr2=0x55680cf2ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017d95e640 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf2ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017d95e640 1 -- 172.21.15.64:0/2966759780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680cf52b00 con 0x55680cf398f0 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.451+0000 7f017c95c640 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3a2e0 0x55680cf4b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.452+0000 7f017d95e640 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 0x55680cf3ad80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0170000c00 tx=0x7f017001af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbd965c50c2944a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.452+0000 7f0162ffd640 1 -- 172.21.15.64:0/2966759780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017003e070 con 0x55680cf398f0 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.453+0000 7f0162ffd640 1 -- 172.21.15.64:0/2966759780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f017000edb0 con 0x55680cf398f0 2024-05-09T09:11:24.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.453+0000 7f0162ffd640 1 -- 172.21.15.64:0/2966759780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017001e070 con 0x55680cf398f0 2024-05-09T09:11:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.453+0000 7f01821aa740 1 -- 172.21.15.64:0/2966759780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 msgr2=0x55680cf3ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 0x55680cf3ad80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0170000c00 tx=0x7f017001af00 comp rx=0 tx=0).stop 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 -- 172.21.15.64:0/2966759780 shutdown_connections 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf3f450 0x55680cf2ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3a2e0 0x55680cf4b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 --2- 172.21.15.64:0/2966759780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf398f0 0x55680cf3ad80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 -- 172.21.15.64:0/2966759780 >> 172.21.15.64:0/2966759780 conn(0x55680cf34f60 msgr2=0x55680cf440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 -- 172.21.15.64:0/2966759780 shutdown_connections 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.454+0000 7f01821aa740 1 -- 172.21.15.64:0/2966759780 wait complete. 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.455+0000 7f01821aa740 1 -- start start 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.455+0000 7f01821aa740 1 Processor -- start 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 0x55680cf481f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3f450 0x55680cfb45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f015c00e2f0 con 0x55680cf3a2e0 2024-05-09T09:11:24.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f015c00f760 con 0x55680cf398f0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f015c00e9c0 con 0x55680cf3f450 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f01821aa740 10 client.0 _ll_register_callbacks cb 0x7f017001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 0x55680cf481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 0x55680cf481f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33754/0 (socket says 172.21.15.64:33754) 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 -- 172.21.15.64:0/2567736141 learned_addr learned my addr 172.21.15.64:0/2567736141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3f450 0x55680cfb45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017c95c640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 -- 172.21.15.64:0/2567736141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3f450 msgr2=0x55680cfb45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3f450 0x55680cfb45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 -- 172.21.15.64:0/2567736141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 msgr2=0x55680cf48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 -- 172.21.15.64:0/2567736141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55680cfb4b30 con 0x55680cf398f0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.456+0000 7f017d95e640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 0x55680cf481f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0170009a30 tx=0x7f0170005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1b351cc20d9c07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.457+0000 7f0160ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:11:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.457+0000 7f0160ff9640 1 -- 172.21.15.64:0/2567736141 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017003e040 con 0x55680cf398f0 2024-05-09T09:11:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.457+0000 7f0160ff9640 1 -- 172.21.15.64:0/2567736141 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0170019ce0 con 0x55680cf398f0 2024-05-09T09:11:24.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.457+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f017000edb0 con 0x55680cf398f0 2024-05-09T09:11:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.458+0000 7f0160ff9640 1 -- 172.21.15.64:0/2567736141 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f017004a050 con 0x55680cf398f0 2024-05-09T09:11:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.458+0000 7f01821aa740 10 client.25754 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.458+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55680cfb5d00 con 0x55680cf398f0 2024-05-09T09:11:24.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f0160ff9640 1 -- 172.21.15.64:0/2567736141 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f017000edb0 con 0x55680cf398f0 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f0160ff9640 1 -- 172.21.15.64:0/2567736141 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0170003b70 con 0x55680cf398f0 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f01821aa740 10 client.25754 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f01821aa740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f01821aa740 20 client.25754 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f01821aa740 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cfb6710 0x55680cfb8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f01821aa740 4 client.25754 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.459+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55680ce9d780 con 0x55680cfb6710 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.460+0000 7f017c95c640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cfb6710 0x55680cfb8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.460+0000 7f017c95c640 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cfb6710 0x55680cfb8b30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d78be486e60f49 server_cookie=6ccb82531685dcd5 in_seq=0 out_seq=0 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.461+0000 7f0160ff9640 10 client.25754 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.461+0000 7f0160ff9640 1 -- 172.21.15.64:0/2567736141 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55680ce9d780 con 0x55680cfb6710 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.461+0000 7f0160ff9640 10 client.25754 handle_command_reply: tid=0 2024-05-09T09:11:24.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.461+0000 7f01821aa740 1 client.25754 shutdown 2024-05-09T09:11:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.463+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 msgr2=0x55680cf481f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.463+0000 7f01821aa740 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 0x55680cf481f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0170009a30 tx=0x7f0170005b90 comp rx=0 tx=0).stop 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 shutdown_connections 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f01821aa740 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55680cfb6710 0x55680cfb8b30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f0160ff9640 0 client.25754 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f01821aa740 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55680cf3f450 0x55680cfb45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f01821aa740 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55680cf3a2e0 0x55680cf48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f01821aa740 1 --2- 172.21.15.64:0/2567736141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55680cf398f0 0x55680cf481f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.464+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 >> 172.21.15.64:0/2567736141 conn(0x55680cf34f60 msgr2=0x55680cf42980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.466+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 shutdown_connections 2024-05-09T09:11:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.466+0000 7f01821aa740 1 -- 172.21.15.64:0/2567736141 wait complete. 2024-05-09T09:11:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.466+0000 7f01821aa740 20 client.25754 trim_cache size 0 max 16384 2024-05-09T09:11:24.465 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:11:24.465 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:11:24.465 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:11:24.465 INFO:teuthology.orchestra.run.smithi064.stdout: "f6d3e87d-f078-4913-9478-89695d3c4985": { 2024-05-09T09:11:24.465 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:11:24.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "f6d3e87d-f078-4913-9478-89695d3c4985", 2024-05-09T09:11:24.466 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:11:24.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:11:24.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:11:24.466 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f01640a2c90 msgr2=0x7f01640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f01640a2c90 0x7f01640a5150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0174000f80 tx=0x7f01740095c0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01781021b0 msgr2=0x7f0178197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01781021b0 0x7f0178197980 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f017402ade0 tx=0x7f0174003ba0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 shutdown_connections 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f01640a2c90 0x7f01640a5150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0178107410 0x7f017819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0178104ae0 0x7f0178197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 --2- 172.21.15.64:0/2358474566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01781021b0 0x7f0178197980 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 >> 172.21.15.64:0/2358474566 conn(0x7f01780fbbe0 msgr2=0x7f0178102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 shutdown_connections 2024-05-09T09:11:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:24.469+0000 7f017e15f640 1 -- 172.21.15.64:0/2358474566 wait complete. 2024-05-09T09:11:24.485 INFO:tasks.cephfs.filesystem:scrub status for tag:f6d3e87d-f078-4913-9478-89695d3c4985 - {'path': '/', 'tag': 'f6d3e87d-f078-4913-9478-89695d3c4985', 'options': 'recursive,force'} 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.069+0000 7f5a04789640 1 -- 172.21.15.64:0/2677780194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc101040 msgr2=0x7f59fc105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.069+0000 7f5a04789640 1 --2- 172.21.15.64:0/2677780194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc101040 0x7f59fc105900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec02a400 comp rx=0 tx=0).stop 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 -- 172.21.15.64:0/2677780194 shutdown_connections 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 --2- 172.21.15.64:0/2677780194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc101040 0x7f59fc105900 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 --2- 172.21.15.64:0/2677780194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc1006a0 0x7f59fc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 --2- 172.21.15.64:0/2677780194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc0ff570 0x7f59fc0ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 -- 172.21.15.64:0/2677780194 >> 172.21.15.64:0/2677780194 conn(0x7f59fc0fae90 msgr2=0x7f59fc0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 -- 172.21.15.64:0/2677780194 shutdown_connections 2024-05-09T09:11:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.070+0000 7f5a04789640 1 -- 172.21.15.64:0/2677780194 wait complete. 2024-05-09T09:11:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.071+0000 7f5a04789640 1 Processor -- start 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.071+0000 7f5a04789640 1 -- start start 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a04789640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 0x7f59fc19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a04789640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc1006a0 0x7f59fc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a04789640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101040 0x7f59fc1a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a04789640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc113880 con 0x7f59fc101040 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a04789640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc113700 con 0x7f59fc1006a0 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a04789640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc113a00 con 0x7f59fc0ff570 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a02525640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 0x7f59fc19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a02525640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 0x7f59fc19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52014/0 (socket says 172.21.15.64:52014) 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a02525640 1 -- 172.21.15.64:0/1548800967 learned_addr learned my addr 172.21.15.64:0/1548800967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a01d24640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc1006a0 0x7f59fc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.072+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101040 0x7f59fc1a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a02525640 1 -- 172.21.15.64:0/1548800967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc1006a0 msgr2=0x7f59fc19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a02525640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc1006a0 0x7f59fc19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a02525640 1 -- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101040 msgr2=0x7f59fc1a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a02525640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101040 0x7f59fc1a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a02525640 1 -- 172.21.15.64:0/1548800967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc1a5b20 con 0x7f59fc0ff570 2024-05-09T09:11:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a02525640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 0x7f59fc19df00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59f800b4a0 tx=0x7f59f800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f59f37fe640 1 -- 172.21.15.64:0/1548800967 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8014070 con 0x7f59fc0ff570 2024-05-09T09:11:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f59f37fe640 1 -- 172.21.15.64:0/1548800967 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59f8010ac0 con 0x7f59fc0ff570 2024-05-09T09:11:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.073+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc1a5e10 con 0x7f59fc0ff570 2024-05-09T09:11:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.074+0000 7f59f37fe640 1 -- 172.21.15.64:0/1548800967 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8017c30 con 0x7f59fc0ff570 2024-05-09T09:11:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.074+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a62f0 con 0x7f59fc0ff570 2024-05-09T09:11:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.078+0000 7f59f37fe640 1 -- 172.21.15.64:0/1548800967 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f59f8017070 con 0x7f59fc0ff570 2024-05-09T09:11:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.080+0000 7f5a07574740 1 Processor -- start 2024-05-09T09:11:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 -- start start 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4df0 0x55d8962b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d896218340 con 0x55d8962b9f60 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d8961c0290 con 0x55d8962b4df0 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a07574740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d896214300 con 0x55d8962b4400 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a02d26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a02d26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52020/0 (socket says 172.21.15.64:52020) 2024-05-09T09:11:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.081+0000 7f5a02d26640 1 -- 172.21.15.64:0/4035774646 learned_addr learned my addr 172.21.15.64:0/4035774646 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.082+0000 7f5a01d24640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.082+0000 7f5a02d26640 1 -- 172.21.15.64:0/4035774646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4df0 msgr2=0x55d8962b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.082+0000 7f5a02525640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4df0 0x55d8962b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.083+0000 7f59f37fe640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59e80a2cd0 0x7f59e80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.083+0000 7f59f37fe640 1 -- 172.21.15.64:0/1548800967 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59f80ce610 con 0x7f59fc0ff570 2024-05-09T09:11:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.083+0000 7f5a02d26640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4df0 0x55d8962b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.083+0000 7f5a02d26640 1 -- 172.21.15.64:0/4035774646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 msgr2=0x55d8962bc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.083+0000 7f5a02d26640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962bc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.083+0000 7f5a02d26640 1 -- 172.21.15.64:0/4035774646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8961e48e0 con 0x55d8962b4400 2024-05-09T09:11:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.084+0000 7f5a01d24640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:11:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.084+0000 7f5a02d26640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b4820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281b446e0be6be3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.084+0000 7f5a02525640 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4df0 0x55d8962b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.084+0000 7f59f0ff9640 1 -- 172.21.15.64:0/4035774646 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec035030 con 0x55d8962b4400 2024-05-09T09:11:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.084+0000 7f59f0ff9640 1 -- 172.21.15.64:0/4035774646 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ec040070 con 0x55d8962b4400 2024-05-09T09:11:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.084+0000 7f59f0ff9640 1 -- 172.21.15.64:0/4035774646 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec031ba0 con 0x55d8962b4400 2024-05-09T09:11:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.085+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59e80a2cd0 0x7f59e80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 -- 172.21.15.64:0/4035774646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 msgr2=0x55d8962b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b4820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec004180 comp rx=0 tx=0).stop 2024-05-09T09:11:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59e80a2cd0 0x7f59e80a5190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59ec031dd0 tx=0x7f59ec031d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 -- 172.21.15.64:0/4035774646 shutdown_connections 2024-05-09T09:11:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4df0 0x55d8962b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 --2- 172.21.15.64:0/4035774646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b4820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 -- 172.21.15.64:0/4035774646 >> 172.21.15.64:0/4035774646 conn(0x55d8962afa70 msgr2=0x55d8962afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 -- 172.21.15.64:0/4035774646 shutdown_connections 2024-05-09T09:11:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.086+0000 7f5a07574740 1 -- 172.21.15.64:0/4035774646 wait complete. 2024-05-09T09:11:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.087+0000 7f5a07574740 1 -- start start 2024-05-09T09:11:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.087+0000 7f5a07574740 1 Processor -- start 2024-05-09T09:11:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.087+0000 7f5a07574740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 0x55d8962be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.087+0000 7f5a07574740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4df0 0x55d8962beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.087+0000 7f5a07574740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a07574740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d8962bceb0 con 0x55d8962b9f60 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 0x55d8962be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 0x55d8962be640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48798/0 (socket says 172.21.15.64:48798) 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02525640 1 -- 172.21.15.64:0/1353155859 learned_addr learned my addr 172.21.15.64:0/1353155859 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d8961d3790 con 0x55d8962b4400 2024-05-09T09:11:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8962bd030 con 0x55d8962b4df0 2024-05-09T09:11:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a07574740 10 client.0 _ll_register_callbacks cb 0x55d89620ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02525640 1 -- 172.21.15.64:0/1353155859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4df0 msgr2=0x55d8962beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02525640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4df0 0x55d8962beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.088+0000 7f5a02525640 1 -- 172.21.15.64:0/1353155859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 msgr2=0x55d8962c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f5a02525640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f5a02525640 1 -- 172.21.15.64:0/1353155859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d895f2c210 con 0x55d8962b4400 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ec040070 con 0x55d8962b4400 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f5a02525640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 0x55d8962be640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009420 tx=0x7f59f80047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a94883af0a75809 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f59e5ffb640 10 client.25763 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1353155859 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f801e070 con 0x55d8962b4400 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.089+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1353155859 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59f80cbe30 con 0x55d8962b4400 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1353155859 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59f80ce300 con 0x55d8962b4400 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 10 client.25763 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55d895ff8620 con 0x55d8962b4400 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1353155859 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f59f80970f0 con 0x55d8962b4400 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 10 client.25763 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 20 client.25763 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d8962c9ef0 0x55d8962cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 4 client.25763 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.090+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d896225a70 con 0x55d8962c9ef0 2024-05-09T09:11:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.091+0000 7f5a01d24640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d8962c9ef0 0x55d8962cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.092+0000 7f5a01d24640 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d8962c9ef0 0x55d8962cc390 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c5855c325ab287 server_cookie=81662c1430194a0e in_seq=0 out_seq=0 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.092+0000 7f59e5ffb640 10 client.25763 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.092+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1353155859 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d896225a70 con 0x55d8962c9ef0 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.092+0000 7f59e5ffb640 10 client.25763 handle_command_reply: tid=0 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.093+0000 7f5a07574740 1 client.25763 shutdown 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.093+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 msgr2=0x55d8962be640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.093+0000 7f5a07574740 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 0x55d8962be640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009420 tx=0x7f59f80047d0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 shutdown_connections 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f5a07574740 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d8962c9ef0 0x55d8962cc390 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f5a07574740 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b9f60 0x55d8962c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f5a07574740 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4df0 0x55d8962beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f5a07574740 1 --2- 172.21.15.64:0/1353155859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b4400 0x55d8962be640 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 >> 172.21.15.64:0/1353155859 conn(0x55d8962afa70 msgr2=0x55d8962aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.094+0000 7f59e5ffb640 0 client.25763 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.095+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 shutdown_connections 2024-05-09T09:11:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.095+0000 7f5a07574740 1 -- 172.21.15.64:0/1353155859 wait complete. 2024-05-09T09:11:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.095+0000 7f5a07574740 20 client.25763 trim_cache size 0 max 16384 2024-05-09T09:11:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.126+0000 7f5a07574740 1 Processor -- start 2024-05-09T09:11:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.126+0000 7f5a07574740 1 -- start start 2024-05-09T09:11:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a07574740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a07574740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a07574740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a02525640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a02525640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52046/0 (socket says 172.21.15.64:52046) 2024-05-09T09:11:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a01d24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a01d24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60744/0 (socket says 172.21.15.64:60744) 2024-05-09T09:11:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.127+0000 7f5a02525640 1 -- 172.21.15.64:0/137839740 learned_addr learned my addr 172.21.15.64:0/137839740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00125e0 con 0x55d8962b4df0 2024-05-09T09:11:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59e000fe20 con 0x55d8962b9f60 2024-05-09T09:11:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59e000dc70 con 0x55d8962b4400 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a02525640 1 -- 172.21.15.64:0/137839740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 msgr2=0x55d8962b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a02525640 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a02525640 1 -- 172.21.15.64:0/137839740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 msgr2=0x55d8962c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a02525640 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a02525640 1 -- 172.21.15.64:0/137839740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8962b51d0 con 0x55d8962b4400 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.128+0000 7f5a02525640 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b2a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59f80ce900 tx=0x7f59f801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4656dd52485e3e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.129+0000 7f59e7fff640 1 -- 172.21.15.64:0/137839740 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8098d80 con 0x55d8962b4400 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.129+0000 7f59e7fff640 1 -- 172.21.15.64:0/137839740 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ec040070 con 0x55d8962b4400 2024-05-09T09:11:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.129+0000 7f59e7fff640 1 -- 172.21.15.64:0/137839740 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80a34c0 con 0x55d8962b4400 2024-05-09T09:11:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 msgr2=0x55d8962b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b2a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59f80ce900 tx=0x7f59f801e000 comp rx=0 tx=0).stop 2024-05-09T09:11:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 shutdown_connections 2024-05-09T09:11:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 --2- 172.21.15.64:0/137839740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962b4400 0x55d8962b2a20 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59f80ce900 tx=0x7f59f801e000 comp rx=0 tx=0).stop 2024-05-09T09:11:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.130+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 >> 172.21.15.64:0/137839740 conn(0x55d8962afa70 msgr2=0x55d8962afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.132+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 shutdown_connections 2024-05-09T09:11:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.134+0000 7f5a07574740 1 -- 172.21.15.64:0/137839740 wait complete. 2024-05-09T09:11:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.135+0000 7f5a07574740 1 -- start start 2024-05-09T09:11:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.135+0000 7f5a07574740 1 Processor -- start 2024-05-09T09:11:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.136+0000 7f5a07574740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.136+0000 7f5a07574740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.136+0000 7f5a07574740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 0x55d8962c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.136+0000 7f5a07574740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59e000e2e0 con 0x55d8962b4df0 2024-05-09T09:11:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a07574740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59e000f750 con 0x55d8962b9f60 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a02d26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 0x55d8962c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a02d26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 0x55d8962c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52054/0 (socket says 172.21.15.64:52054) 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a02d26640 1 -- 172.21.15.64:0/1380335850 learned_addr learned my addr 172.21.15.64:0/1380335850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a02525640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59e000e9b0 con 0x55d8962c41a0 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a07574740 10 client.0 _ll_register_callbacks cb 0x7f59f8096da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.137+0000 7f5a01d24640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.138+0000 7f5a02d26640 1 -- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 msgr2=0x55d8962c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.138+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.138+0000 7f5a02d26640 1 -- 172.21.15.64:0/1380335850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 msgr2=0x55d8962c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.138+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.138+0000 7f5a02d26640 1 -- 172.21.15.64:0/1380335850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8962c4b60 con 0x55d8962c41a0 2024-05-09T09:11:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.139+0000 7f5a02d26640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 0x55d8962c4620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59ec031fd0 tx=0x7f59ec031e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4ece95b880e436 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.140+0000 7f59e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.140+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1380335850 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec035040 con 0x55d8962c41a0 2024-05-09T09:11:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.140+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1380335850 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59ec031990 con 0x55d8962c41a0 2024-05-09T09:11:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.140+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ec040070 con 0x55d8962c41a0 2024-05-09T09:11:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.141+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1380335850 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59ec06a050 con 0x55d8962c41a0 2024-05-09T09:11:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.142+0000 7f5a07574740 10 client.25891 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.142+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8962c5e50 con 0x55d8962c41a0 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.143+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1380335850 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59ec061070 con 0x55d8962c41a0 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1380335850 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f59ec03c440 con 0x55d8962c41a0 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a07574740 10 client.25891 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a07574740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a07574740 20 client.25891 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a07574740 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d896330890 0x55d896332cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a07574740 4 client.25891 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:11:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f59ec040070 con 0x55d896330890 2024-05-09T09:11:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.144+0000 7f5a02525640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d896330890 0x55d896332cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.145+0000 7f5a02525640 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d896330890 0x55d896332cb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bdbc6eec4559561 server_cookie=8bcdf9c9505e6a0c in_seq=0 out_seq=0 2024-05-09T09:11:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.145+0000 7f59e5ffb640 10 client.25891 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: pgmap v438: 129 pgs: 129 active+clean; 964 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.159+0000 7f59e5ffb640 1 -- 172.21.15.64:0/1380335850 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f59ec040070 con 0x55d896330890 2024-05-09T09:11:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.159+0000 7f59e5ffb640 10 client.25891 handle_command_reply: tid=0 2024-05-09T09:11:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.159+0000 7f5a07574740 1 client.25891 shutdown 2024-05-09T09:11:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.159+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 msgr2=0x55d8962c4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.159+0000 7f5a07574740 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 0x55d8962c4620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59ec031fd0 tx=0x7f59ec031e00 comp rx=0 tx=0).stop 2024-05-09T09:11:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 shutdown_connections 2024-05-09T09:11:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d896330890 0x55d896332cb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8962c41a0 0x55d8962c4620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8962b9f60 0x55d8962c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 --2- 172.21.15.64:0/1380335850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8962b4df0 0x55d8962c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 >> 172.21.15.64:0/1380335850 conn(0x55d8962afa70 msgr2=0x55d8962b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f59e5ffb640 0 client.25891 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.160+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 shutdown_connections 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.161+0000 7f5a07574740 1 -- 172.21.15.64:0/1380335850 wait complete. 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:11:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.161+0000 7f5a07574740 20 client.25891 trim_cache size 0 max 16384 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.162+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59e80a2cd0 msgr2=0x7f59e80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.162+0000 7f5a04789640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59e80a2cd0 0x7f59e80a5190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59ec031dd0 tx=0x7f59ec031d40 comp rx=0 tx=0).stop 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.162+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 msgr2=0x7f59fc19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.162+0000 7f5a04789640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 0x7f59fc19df00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59f800b4a0 tx=0x7f59f800b960 comp rx=0 tx=0).stop 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.162+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 shutdown_connections 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59e80a2cd0 0x7f59e80a5190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101040 0x7f59fc1a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc1006a0 0x7f59fc19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 --2- 172.21.15.64:0/1548800967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59fc0ff570 0x7f59fc19df00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 >> 172.21.15.64:0/1548800967 conn(0x7f59fc0fae90 msgr2=0x7f59fc1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 shutdown_connections 2024-05-09T09:11:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:25.163+0000 7f5a04789640 1 -- 172.21.15.64:0/1548800967 wait complete. 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: pgmap v438: 129 pgs: 129 active+clean; 964 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: pgmap v438: 129 pgs: 129 active+clean; 964 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-05-09T09:11:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:25 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:11:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:25 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:25 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:25 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:25 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:11:26.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:25 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:25 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:26.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:25 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:25 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:11:26.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:25 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:25 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:26.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:25 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 65.0 s, 971.8 tps, lat 108.022 ms stddev 109.473 2024-05-09T09:11:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:26 smithi064 ceph-mon[30149]: pgmap v439: 129 pgs: 129 active+clean; 972 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-05-09T09:11:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:26 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:11:27.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:26 smithi179 ceph-mon[34086]: pgmap v439: 129 pgs: 129 active+clean; 972 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-05-09T09:11:27.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:26 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:11:27.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:26 smithi139 ceph-mon[33599]: pgmap v439: 129 pgs: 129 active+clean; 972 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-05-09T09:11:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:26 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:11:28.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:11:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:11:28.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:27 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:11:28.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:11:28.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:11:28.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:27 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:11:28.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:11:28.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:11:28.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:27 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:11:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:28 smithi064 ceph-mon[30149]: pgmap v440: 129 pgs: 129 active+clean; 972 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:11:29.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:28 smithi179 ceph-mon[34086]: pgmap v440: 129 pgs: 129 active+clean; 972 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:11:29.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:28 smithi139 ceph-mon[33599]: pgmap v440: 129 pgs: 129 active+clean; 972 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:11:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:30 smithi064 ceph-mon[30149]: pgmap v441: 129 pgs: 129 active+clean; 972 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-05-09T09:11:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:30 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:11:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:30 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:11:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:30 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:11:31.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:30 smithi179 ceph-mon[34086]: pgmap v441: 129 pgs: 129 active+clean; 972 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-05-09T09:11:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:30 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:11:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:30 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:11:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:30 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:11:31.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:30 smithi139 ceph-mon[33599]: pgmap v441: 129 pgs: 129 active+clean; 972 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-05-09T09:11:31.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:30 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:11:31.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:30 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:11:31.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:30 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:11:31.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 70.0 s, 1029.4 tps, lat 97.100 ms stddev 24.485 2024-05-09T09:11:33.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:32 smithi064 ceph-mon[30149]: pgmap v442: 129 pgs: 129 active+clean; 972 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:11:33.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:32 smithi179 ceph-mon[34086]: pgmap v442: 129 pgs: 129 active+clean; 972 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:11:33.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:32 smithi139 ceph-mon[33599]: pgmap v442: 129 pgs: 129 active+clean; 972 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:11:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:34 smithi064 ceph-mon[30149]: pgmap v443: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:11:35.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:34 smithi179 ceph-mon[34086]: pgmap v443: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:11:35.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:34 smithi139 ceph-mon[33599]: pgmap v443: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:11:36.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 75.0 s, 1122.0 tps, lat 89.189 ms stddev 24.801 2024-05-09T09:11:37.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:36 smithi064 ceph-mon[30149]: pgmap v444: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 1.25k op/s 2024-05-09T09:11:37.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:36 smithi179 ceph-mon[34086]: pgmap v444: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 1.25k op/s 2024-05-09T09:11:37.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:36 smithi139 ceph-mon[33599]: pgmap v444: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 10 MiB/s wr, 1.25k op/s 2024-05-09T09:11:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 -- 172.21.15.64:0/1026947702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100100 msgr2=0x7fd314100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 --2- 172.21.15.64:0/1026947702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100100 0x7fd314100520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd308009900 tx=0x7fd30802a400 comp rx=0 tx=0).stop 2024-05-09T09:11:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 -- 172.21.15.64:0/1026947702 shutdown_connections 2024-05-09T09:11:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 --2- 172.21.15.64:0/1026947702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314101400 0x7fd314101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 --2- 172.21.15.64:0/1026947702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314100a60 0x7fd314100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 --2- 172.21.15.64:0/1026947702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100100 0x7fd314100520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 -- 172.21.15.64:0/1026947702 >> 172.21.15.64:0/1026947702 conn(0x7fd3140fbd00 msgr2=0x7fd3140fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.331+0000 7fd31889e640 1 -- 172.21.15.64:0/1026947702 shutdown_connections 2024-05-09T09:11:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.332+0000 7fd31889e640 1 -- 172.21.15.64:0/1026947702 wait complete. 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.332+0000 7fd31889e640 1 Processor -- start 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 -- start start 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314100100 0x7fd31419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 0x7fd31419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314101400 0x7fd3141a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410f5d0 con 0x7fd314100100 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410f450 con 0x7fd314100a60 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31889e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410f750 con 0x7fd314101400 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd311d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 0x7fd31419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd311d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 0x7fd31419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43478/0 (socket says 172.21.15.64:43478) 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd311d9b640 1 -- 172.21.15.64:0/1580437005 learned_addr learned my addr 172.21.15.64:0/1580437005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd31259c640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314100100 0x7fd31419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.333+0000 7fd312d9d640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314101400 0x7fd3141a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd311d9b640 1 -- 172.21.15.64:0/1580437005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314101400 msgr2=0x7fd3141a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd311d9b640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314101400 0x7fd3141a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd311d9b640 1 -- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314100100 msgr2=0x7fd31419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd311d9b640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314100100 0x7fd31419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd311d9b640 1 -- 172.21.15.64:0/1580437005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141a5be0 con 0x7fd314100a60 2024-05-09T09:11:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd311d9b640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 0x7fd31419e500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d950 tx=0x7fd30400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.334+0000 7fd3037fe640 1 -- 172.21.15.64:0/1580437005 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304017030 con 0x7fd314100a60 2024-05-09T09:11:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.335+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3141a5ed0 con 0x7fd314100a60 2024-05-09T09:11:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.335+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3141a63e0 con 0x7fd314100a60 2024-05-09T09:11:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.335+0000 7fd3037fe640 1 -- 172.21.15.64:0/1580437005 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd304004020 con 0x7fd314100a60 2024-05-09T09:11:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.335+0000 7fd3037fe640 1 -- 172.21.15.64:0/1580437005 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304017030 con 0x7fd314100a60 2024-05-09T09:11:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.338+0000 7fd31b5ac740 1 Processor -- start 2024-05-09T09:11:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.338+0000 7fd31b5ac740 1 -- start start 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e385fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 0x561e38600740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e38600db0 0x561e38603240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31b5ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561e3855f340 con 0x561e385fb250 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31b5ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561e38507290 con 0x561e38600db0 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31b5ac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561e3855b300 con 0x561e385fbc40 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd3037fe640 1 -- 172.21.15.64:0/1580437005 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd304005460 con 0x7fd314100a60 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 0x561e38600740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 0x561e38600740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48444/0 (socket says 172.21.15.64:48444) 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd31259c640 1 -- 172.21.15.64:0/506359450 learned_addr learned my addr 172.21.15.64:0/506359450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.339+0000 7fd312d9d640 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e385fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.340+0000 7fd31259c640 1 -- 172.21.15.64:0/506359450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e38600db0 msgr2=0x561e38603240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:11:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.340+0000 7fd31259c640 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e38600db0 0x561e38603240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.340+0000 7fd31259c640 1 -- 172.21.15.64:0/506359450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 msgr2=0x561e385fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.340+0000 7fd31259c640 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e385fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.340+0000 7fd31259c640 1 -- 172.21.15.64:0/506359450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3852b8e0 con 0x561e385fbc40 2024-05-09T09:11:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.340+0000 7fd31259c640 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 0x561e38600740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd30802a970 tx=0x7fd3080040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b22418edf99f52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.341+0000 7fd300ff9640 1 -- 172.21.15.64:0/506359450 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308035030 con 0x561e385fbc40 2024-05-09T09:11:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.341+0000 7fd300ff9640 1 -- 172.21.15.64:0/506359450 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3080044c0 con 0x561e385fbc40 2024-05-09T09:11:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.341+0000 7fd3037fe640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd2fc0a2f10 0x7fd2fc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.341+0000 7fd300ff9640 1 -- 172.21.15.64:0/506359450 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3080035f0 con 0x561e385fbc40 2024-05-09T09:11:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.342+0000 7fd3037fe640 1 -- 172.21.15.64:0/1580437005 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd304009a90 con 0x7fd314100a60 2024-05-09T09:11:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.342+0000 7fd312d9d640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd2fc0a2f10 0x7fd2fc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.342+0000 7fd31b5ac740 1 -- 172.21.15.64:0/506359450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 msgr2=0x561e38600740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.342+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 0x561e38600740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd30802a970 tx=0x7fd3080040d0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.342+0000 7fd312d9d640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd2fc0a2f10 0x7fd2fc0a53d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x561e385fc4a0 tx=0x7fd30c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 -- 172.21.15.64:0/506359450 shutdown_connections 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e38600db0 0x561e38603240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fbc40 0x561e38600740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/506359450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e385fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 -- 172.21.15.64:0/506359450 >> 172.21.15.64:0/506359450 conn(0x561e385f68c0 msgr2=0x561e385f6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 -- 172.21.15.64:0/506359450 shutdown_connections 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.343+0000 7fd31b5ac740 1 -- 172.21.15.64:0/506359450 wait complete. 2024-05-09T09:11:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.344+0000 7fd31b5ac740 1 -- start start 2024-05-09T09:11:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.344+0000 7fd31b5ac740 1 Processor -- start 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 0x561e38605540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e38605a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 0x561e38605540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 0x561e38605540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48456/0 (socket says 172.21.15.64:48456) 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31259c640 1 -- 172.21.15.64:0/1396313731 learned_addr learned my addr 172.21.15.64:0/1396313731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd311d9b640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e38605a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e38600db0 0x561e3860cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.345+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561e38603d50 con 0x561e38600db0 2024-05-09T09:11:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561e3851a790 con 0x561e385fbc40 2024-05-09T09:11:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561e38603ed0 con 0x561e385fb250 2024-05-09T09:11:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31b5ac740 10 client.0 _ll_register_callbacks cb 0x561e38555a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd312d9d640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e38600db0 0x561e3860cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31259c640 1 -- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 msgr2=0x561e38605a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31259c640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e38605a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31259c640 1 -- 172.21.15.64:0/1396313731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e38600db0 msgr2=0x561e3860cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31259c640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e38600db0 0x561e3860cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.346+0000 7fd31259c640 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e38273210 con 0x561e385fb250 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.347+0000 7fd312d9d640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e38600db0 0x561e3860cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.347+0000 7fd31259c640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 0x561e38605540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd308009850 tx=0x7fd308004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd5e77599465640c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.347+0000 7fd2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.348+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/1396313731 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308051060 con 0x561e385fb250 2024-05-09T09:11:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.348+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/1396313731 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd308005210 con 0x561e385fb250 2024-05-09T09:11:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.348+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e3856ca70 con 0x561e385fb250 2024-05-09T09:11:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.348+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/1396313731 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd308035030 con 0x561e385fb250 2024-05-09T09:11:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.348+0000 7fd31b5ac740 10 client.25903 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.348+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e3833f620 con 0x561e385fb250 2024-05-09T09:11:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.349+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/1396313731 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561e3856ca70 con 0x561e385fb250 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.349+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/1396313731 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd308031580 con 0x561e385fb250 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.349+0000 7fd31b5ac740 10 client.25903 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.349+0000 7fd31b5ac740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.349+0000 7fd31b5ac740 20 client.25903 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.350+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e38612ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.350+0000 7fd31b5ac740 4 client.25903 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.350+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e38566dc0 con 0x561e38610c00 2024-05-09T09:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.350+0000 7fd311d9b640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e38612ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.351+0000 7fd2f9ffb640 10 client.25903 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.351+0000 7fd311d9b640 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e38612ff0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f0f7920e5a121f server_cookie=d040bcf0d36956c5 in_seq=0 out_seq=0 2024-05-09T09:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/1396313731 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e38566dc0 con 0x561e38610c00 2024-05-09T09:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd2f9ffb640 10 client.25903 handle_command_reply: tid=0 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 client.25903 shutdown 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 msgr2=0x561e38605540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 0x561e38605540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd308009850 tx=0x7fd308004f60 comp rx=0 tx=0).stop 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 shutdown_connections 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e38612ff0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e38600db0 0x561e3860cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e38605a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd2f9ffb640 0 client.25903 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/1396313731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e385fb250 0x561e38605540 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.352+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 >> 172.21.15.64:0/1396313731 conn(0x561e385f68c0 msgr2=0x561e385f5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.353+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 shutdown_connections 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.353+0000 7fd31b5ac740 1 -- 172.21.15.64:0/1396313731 wait complete. 2024-05-09T09:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.353+0000 7fd31b5ac740 20 client.25903 trim_cache size 0 max 16384 2024-05-09T09:11:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.382+0000 7fd31b5ac740 1 Processor -- start 2024-05-09T09:11:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.382+0000 7fd31b5ac740 1 -- start start 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38605820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e385f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e386175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31b5ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f40125a0 con 0x561e385fb250 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31b5ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400fde0 con 0x561e385fbc40 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38605820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31b5ac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400dc30 con 0x561e38600db0 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38605820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41040/0 (socket says 172.21.15.64:41040) 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd31259c640 1 -- 172.21.15.64:0/2713397396 learned_addr learned my addr 172.21.15.64:0/2713397396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd311d9b640 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e385f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd312d9d640 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e386175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd311d9b640 1 -- 172.21.15.64:0/2713397396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 msgr2=0x561e386175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd311d9b640 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e386175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd311d9b640 1 -- 172.21.15.64:0/2713397396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 msgr2=0x561e38605820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd311d9b640 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38605820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.383+0000 7fd311d9b640 1 -- 172.21.15.64:0/2713397396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e385f1bc0 con 0x561e385fbc40 2024-05-09T09:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.384+0000 7fd31259c640 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38605820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.384+0000 7fd311d9b640 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e385f9fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3040cc610 tx=0x7fd30409b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1551d475095fa057 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.385+0000 7fd2fbfff640 1 -- 172.21.15.64:0/2713397396 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30409f040 con 0x561e385fbc40 2024-05-09T09:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.385+0000 7fd2fbfff640 1 -- 172.21.15.64:0/2713397396 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd30407f070 con 0x561e385fbc40 2024-05-09T09:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.385+0000 7fd2fbfff640 1 -- 172.21.15.64:0/2713397396 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040041d0 con 0x561e385fbc40 2024-05-09T09:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.386+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2713397396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 msgr2=0x561e385f9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.386+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e385f9fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3040cc610 tx=0x7fd30409b850 comp rx=0 tx=0).stop 2024-05-09T09:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2713397396 shutdown_connections 2024-05-09T09:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e386175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e385f9fd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2713397396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38605820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2713397396 >> 172.21.15.64:0/2713397396 conn(0x561e385f68c0 msgr2=0x561e385f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2713397396 shutdown_connections 2024-05-09T09:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.387+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2713397396 wait complete. 2024-05-09T09:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.388+0000 7fd31b5ac740 1 -- start start 2024-05-09T09:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.388+0000 7fd31b5ac740 1 Processor -- start 2024-05-09T09:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38670f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e386734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd31b5ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e38675a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd311d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e386734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd311d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e386734e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43506/0 (socket says 172.21.15.64:43506) 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd31259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38670f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd31259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38670f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41044/0 (socket says 172.21.15.64:41044) 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd311d9b640 1 -- 172.21.15.64:0/2392537804 learned_addr learned my addr 172.21.15.64:0/2392537804 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.389+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400e2a0 con 0x561e385fb250 2024-05-09T09:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd312d9d640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e38675a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400f710 con 0x561e385fbc40 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400e970 con 0x561e38600db0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd312d9d640 1 -- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 msgr2=0x561e386734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd312d9d640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e386734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd312d9d640 1 -- 172.21.15.64:0/2392537804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 msgr2=0x561e38670f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd312d9d640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38670f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd312d9d640 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e38613360 con 0x561e38600db0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd31259c640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38670f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd31b5ac740 10 client.0 _ll_register_callbacks cb 0x7fd30c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.390+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30407f070 con 0x561e38600db0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd312d9d640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e38675a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561e38613330 tx=0x7fd30c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5eb2b824d41d78d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd2f9ffb640 10 client.25909 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/2392537804 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c02e060 con 0x561e38600db0 2024-05-09T09:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/2392537804 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd30c00f480 con 0x561e38600db0 2024-05-09T09:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/2392537804 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd30c00f6e0 con 0x561e38600db0 2024-05-09T09:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd31b5ac740 10 client.25909 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.391+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=30}) -- 0x561e38676f10 con 0x561e38600db0 2024-05-09T09:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/2392537804 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd30c028070 con 0x561e38600db0 2024-05-09T09:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd31b5ac740 10 client.25909 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd31b5ac740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd31b5ac740 20 client.25909 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e386798a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd31b5ac740 4 client.25909 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.392+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e384e48e0 con 0x561e38610c00 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.393+0000 7fd31259c640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e386798a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.394+0000 7fd31259c640 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e386798a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa8b50d1b6cbe21 server_cookie=f70ad7fe75ad7a9b in_seq=0 out_seq=0 2024-05-09T09:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.394+0000 7fd2f9ffb640 10 client.25909 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.422+0000 7fd2f9ffb640 1 -- 172.21.15.64:0/2392537804 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e384e48e0 con 0x561e38610c00 2024-05-09T09:11:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.422+0000 7fd2f9ffb640 10 client.25909 handle_command_reply: tid=0 2024-05-09T09:11:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.423+0000 7fd31b5ac740 1 client.25909 shutdown 2024-05-09T09:11:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.423+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 msgr2=0x561e38675a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.423+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e38675a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561e38613330 tx=0x7fd30c004ab0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 shutdown_connections 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561e38610c00 0x561e386798a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd2f9ffb640 0 client.25909 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e38600db0 0x561e38675a30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561e385fbc40 0x561e386734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd31b5ac740 1 --2- 172.21.15.64:0/2392537804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561e385fb250 0x561e38670f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.424+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 >> 172.21.15.64:0/2392537804 conn(0x561e385f68c0 msgr2=0x561e385f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.425+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 shutdown_connections 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.425+0000 7fd31b5ac740 1 -- 172.21.15.64:0/2392537804 wait complete. 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.425+0000 7fd31b5ac740 20 client.25909 trim_cache size 0 max 16384 2024-05-09T09:11:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd2fc0a2f10 msgr2=0x7fd2fc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd2fc0a2f10 0x7fd2fc0a53d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x561e385fc4a0 tx=0x7fd30c009500 comp rx=0 tx=0).stop 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 msgr2=0x7fd31419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 0x7fd31419e500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d950 tx=0x7fd30400de10 comp rx=0 tx=0).stop 2024-05-09T09:11:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 shutdown_connections 2024-05-09T09:11:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd2fc0a2f10 0x7fd2fc0a53d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd314101400 0x7fd3141a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd314100a60 0x7fd31419e500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 --2- 172.21.15.64:0/1580437005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd314100100 0x7fd31419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 >> 172.21.15.64:0/1580437005 conn(0x7fd3140fbd00 msgr2=0x7fd3140fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.427+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 shutdown_connections 2024-05-09T09:11:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:38.428+0000 7fd31889e640 1 -- 172.21.15.64:0/1580437005 wait complete. 2024-05-09T09:11:39.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:38 smithi064 ceph-mon[30149]: pgmap v445: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 1.15k op/s 2024-05-09T09:11:39.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:38 smithi179 ceph-mon[34086]: pgmap v445: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 1.15k op/s 2024-05-09T09:11:39.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:38 smithi139 ceph-mon[33599]: pgmap v445: 129 pgs: 129 active+clean; 972 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.1 MiB/s wr, 1.15k op/s 2024-05-09T09:11:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:40 smithi064 ceph-mon[30149]: pgmap v446: 129 pgs: 129 active+clean; 972 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 1.51k op/s 2024-05-09T09:11:41.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:40 smithi179 ceph-mon[34086]: pgmap v446: 129 pgs: 129 active+clean; 972 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 1.51k op/s 2024-05-09T09:11:41.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:40 smithi139 ceph-mon[33599]: pgmap v446: 129 pgs: 129 active+clean; 972 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 1.51k op/s 2024-05-09T09:11:41.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 80.0 s, 1021.0 tps, lat 97.875 ms stddev 40.455 2024-05-09T09:11:43.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:42 smithi064 ceph-mon[30149]: pgmap v447: 129 pgs: 129 active+clean; 972 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.6 MiB/s wr, 1.15k op/s 2024-05-09T09:11:43.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:42 smithi179 ceph-mon[34086]: pgmap v447: 129 pgs: 129 active+clean; 972 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.6 MiB/s wr, 1.15k op/s 2024-05-09T09:11:43.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:42 smithi139 ceph-mon[33599]: pgmap v447: 129 pgs: 129 active+clean; 972 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.6 MiB/s wr, 1.15k op/s 2024-05-09T09:11:45.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:44 smithi064 ceph-mon[30149]: pgmap v448: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:45.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:44 smithi179 ceph-mon[34086]: pgmap v448: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:45.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:44 smithi139 ceph-mon[33599]: pgmap v448: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-05-09T09:11:46.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 85.0 s, 1077.8 tps, lat 92.558 ms stddev 26.094 2024-05-09T09:11:47.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:46 smithi064 ceph-mon[30149]: pgmap v449: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T09:11:47.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:46 smithi179 ceph-mon[34086]: pgmap v449: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T09:11:47.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:46 smithi139 ceph-mon[33599]: pgmap v449: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T09:11:49.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:48 smithi064 ceph-mon[30149]: pgmap v450: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:11:49.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:48 smithi179 ceph-mon[34086]: pgmap v450: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:11:49.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:48 smithi139 ceph-mon[33599]: pgmap v450: 129 pgs: 129 active+clean; 972 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:11:51.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:50 smithi064 ceph-mon[30149]: pgmap v451: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-09T09:11:51.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:50 smithi179 ceph-mon[34086]: pgmap v451: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-09T09:11:51.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:50 smithi139 ceph-mon[33599]: pgmap v451: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-09T09:11:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 90.0 s, 1104.0 tps, lat 90.846 ms stddev 58.359 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 -- 172.21.15.64:0/1904983799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 msgr2=0x7f3cec00bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 --2- 172.21.15.64:0/1904983799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00bca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4066fa0 tx=0x7f3cf408d7f0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 -- 172.21.15.64:0/1904983799 shutdown_connections 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 --2- 172.21.15.64:0/1904983799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00bca0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 --2- 172.21.15.64:0/1904983799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0a4790 0x7f3cec0a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 --2- 172.21.15.64:0/1904983799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cec0a6160 0x7f3cec0a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.588+0000 7f3cf9534640 1 -- 172.21.15.64:0/1904983799 >> 172.21.15.64:0/1904983799 conn(0x7f3cec0a00d0 msgr2=0x7f3cec0a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.589+0000 7f3cf9534640 1 -- 172.21.15.64:0/1904983799 shutdown_connections 2024-05-09T09:11:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.592+0000 7f3cf9534640 1 -- 172.21.15.64:0/1904983799 wait complete. 2024-05-09T09:11:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.593+0000 7f3cf9534640 1 Processor -- start 2024-05-09T09:11:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 -- start start 2024-05-09T09:11:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0a4790 0x7f3cec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cec0a6160 0x7f3cec00a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec00f3d0 con 0x7f3cec0a6160 2024-05-09T09:11:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec00f250 con 0x7f3cec0a4790 2024-05-09T09:11:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf9534640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec00f550 con 0x7f3cec0a5150 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0a4790 0x7f3cec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35942/0 (socket says 172.21.15.64:35942) 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf37fe640 1 -- 172.21.15.64:0/4145665993 learned_addr learned my addr 172.21.15.64:0/4145665993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.594+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cec0a6160 0x7f3cec00a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.595+0000 7f3cf37fe640 1 -- 172.21.15.64:0/4145665993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0a4790 msgr2=0x7f3cec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.595+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0a4790 0x7f3cec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.595+0000 7f3cf37fe640 1 -- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cec0a6160 msgr2=0x7f3cec00a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.595+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cec0a6160 0x7f3cec00a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.595+0000 7f3cf37fe640 1 -- 172.21.15.64:0/4145665993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec00ac80 con 0x7f3cec0a5150 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.595+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00a010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8030a70 tx=0x7f3ce8030f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.596+0000 7f3cf17fa640 1 -- 172.21.15.64:0/4145665993 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce802edf0 con 0x7f3cec0a5150 2024-05-09T09:11:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.596+0000 7f3cf17fa640 1 -- 172.21.15.64:0/4145665993 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce80035f0 con 0x7f3cec0a5150 2024-05-09T09:11:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.596+0000 7f3cf17fa640 1 -- 172.21.15.64:0/4145665993 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce80326a0 con 0x7f3cec0a5150 2024-05-09T09:11:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.597+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec153ce0 con 0x7f3cec0a5150 2024-05-09T09:11:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.597+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec154150 con 0x7f3cec0a5150 2024-05-09T09:11:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 Processor -- start 2024-05-09T09:11:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 -- start start 2024-05-09T09:11:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9a400 0x555c2ec9a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2ec9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9ff60 0x555c2eca23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c2ebfe340 con 0x555c2ec9ff60 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555c2eba6290 con 0x555c2ec9adf0 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cfd577740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555c2ebfa300 con 0x555c2ec9a400 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.603+0000 7f3cf17fa640 1 -- 172.21.15.64:0/4145665993 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ce8004000 con 0x7f3cec0a5150 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.604+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2ec9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.604+0000 7f3cf8d33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9a400 0x555c2ec9a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.604+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2ec9f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37434/0 (socket says 172.21.15.64:37434) 2024-05-09T09:11:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.604+0000 7f3cf3fff640 1 -- 172.21.15.64:0/1128856100 learned_addr learned my addr 172.21.15.64:0/1128856100 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.604+0000 7f3cf8d33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9a400 0x555c2ec9a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35950/0 (socket says 172.21.15.64:35950) 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.604+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9ff60 0x555c2eca23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf3fff640 1 -- 172.21.15.64:0/1128856100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9a400 msgr2=0x555c2ec9a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9a400 0x555c2ec9a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf3fff640 1 -- 172.21.15.64:0/1128856100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9ff60 msgr2=0x555c2eca23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9ff60 0x555c2eca23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf3fff640 1 -- 172.21.15.64:0/1128856100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2ebca8e0 con 0x555c2ec9adf0 2024-05-09T09:11:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf17fa640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3cdc0a2d10 0x7f3cdc0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2ec9f8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400bcb0 tx=0x7f3ce40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b231dbcd05e6343b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.605+0000 7f3cf17fa640 1 -- 172.21.15.64:0/4145665993 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ce80f1670 con 0x7f3cec0a5150 2024-05-09T09:11:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.606+0000 7f3cda7fc640 1 -- 172.21.15.64:0/1128856100 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4004020 con 0x555c2ec9adf0 2024-05-09T09:11:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.606+0000 7f3cda7fc640 1 -- 172.21.15.64:0/1128856100 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce40054d0 con 0x555c2ec9adf0 2024-05-09T09:11:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.606+0000 7f3cda7fc640 1 -- 172.21.15.64:0/1128856100 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4004cc0 con 0x555c2ec9adf0 2024-05-09T09:11:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.606+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3cdc0a2d10 0x7f3cdc0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.607+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3cdc0a2d10 0x7f3cdc0a51d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40649a0 tx=0x7f3cf4064aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 -- 172.21.15.64:0/1128856100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 msgr2=0x555c2ec9f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2ec9f8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400bcb0 tx=0x7f3ce40107a0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 -- 172.21.15.64:0/1128856100 shutdown_connections 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9ff60 0x555c2eca23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2ec9f8f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 --2- 172.21.15.64:0/1128856100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9a400 0x555c2ec9a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 -- 172.21.15.64:0/1128856100 >> 172.21.15.64:0/1128856100 conn(0x555c2ec95a70 msgr2=0x555c2ec95e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 -- 172.21.15.64:0/1128856100 shutdown_connections 2024-05-09T09:11:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 -- 172.21.15.64:0/1128856100 wait complete. 2024-05-09T09:11:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.608+0000 7f3cfd577740 1 -- start start 2024-05-09T09:11:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.609+0000 7f3cfd577740 1 Processor -- start 2024-05-09T09:11:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.609+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.609+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2eca4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59730/0 (socket says 172.21.15.64:59730) 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf3fff640 1 -- 172.21.15.64:0/3427975381 learned_addr learned my addr 172.21.15.64:0/3427975381 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ecabbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c2eca2f10 con 0x555c2ec9a400 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555c2ebb9790 con 0x555c2ec9adf0 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555c2eca3090 con 0x555c2ec9ff60 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ecabbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cfd577740 10 client.0 _ll_register_callbacks cb 0x555c2ebf4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf8d33640 1 -- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 msgr2=0x555c2eca4be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2eca4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf8d33640 1 -- 172.21.15.64:0/3427975381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 msgr2=0x555c2eca46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.610+0000 7f3cf8d33640 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c2e912210 con 0x555c2ec9ff60 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.611+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ecabbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3cf402d750 tx=0x7f3cf408d7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc75c1c2dc342d95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.611+0000 7f3cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.611+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3427975381 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40b1020 con 0x555c2ec9ff60 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.611+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce40054d0 con 0x555c2ec9ff60 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.611+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3427975381 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3cf409d040 con 0x555c2ec9ff60 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.612+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3427975381 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3cf40bc050 con 0x555c2ec9ff60 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.612+0000 7f3cfd577740 10 client.25921 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.613+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c2e9de620 con 0x555c2ec9ff60 2024-05-09T09:11:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3427975381 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cf40a28f0 con 0x555c2ec9ff60 2024-05-09T09:11:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3427975381 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3cf40ac070 con 0x555c2ec9ff60 2024-05-09T09:11:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cfd577740 10 client.25921 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cfd577740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cfd577740 20 client.25921 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ecb0120 0x555c2ecb2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cfd577740 4 client.25921 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.615+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ce40054d0 con 0x555c2ecb0120 2024-05-09T09:11:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.616+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ecb0120 0x555c2ecb2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.617+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ecb0120 0x555c2ecb2550 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d764ba714e8966bd server_cookie=a04e451963fe75d9 in_seq=0 out_seq=0 2024-05-09T09:11:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.617+0000 7f3cd37fe640 10 client.25921 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.618+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3427975381 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3ce40054d0 con 0x555c2ecb0120 2024-05-09T09:11:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.618+0000 7f3cd37fe640 10 client.25921 handle_command_reply: tid=0 2024-05-09T09:11:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.618+0000 7f3cfd577740 1 client.25921 shutdown 2024-05-09T09:11:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.618+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 msgr2=0x555c2ecabbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ecabbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3cf402d750 tx=0x7f3cf408d7f0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 shutdown_connections 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ecb0120 0x555c2ecb2550 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ecabbb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9adf0 0x555c2eca4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3427975381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 >> 172.21.15.64:0/3427975381 conn(0x555c2ec95a70 msgr2=0x555c2ec94bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cd37fe640 0 client.25921 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 shutdown_connections 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 1 -- 172.21.15.64:0/3427975381 wait complete. 2024-05-09T09:11:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.619+0000 7f3cfd577740 20 client.25921 trim_cache size 0 max 16384 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 Processor -- start 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 -- start start 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9a400 0x555c2ec98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9adf0 0x555c2ecabb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ec99040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1610c0 con 0x555c2ec9adf0 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec15e900 con 0x555c2ec9a400 2024-05-09T09:11:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.654+0000 7f3cfd577740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec15c730 con 0x555c2ec9ff60 2024-05-09T09:11:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf8d33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ec99040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9adf0 0x555c2ecabb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9adf0 0x555c2ecabb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59744/0 (socket says 172.21.15.64:59744) 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf37fe640 1 -- 172.21.15.64:0/4183103864 learned_addr learned my addr 172.21.15.64:0/4183103864 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf8d33640 1 -- 172.21.15.64:0/4183103864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9a400 msgr2=0x555c2ec98b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9a400 0x555c2ec98b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf8d33640 1 -- 172.21.15.64:0/4183103864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9adf0 msgr2=0x555c2ecabb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9adf0 0x555c2ecabb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.655+0000 7f3cf8d33640 1 -- 172.21.15.64:0/4183103864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2ec9b1d0 con 0x555c2ec9ff60 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.656+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ec99040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cf408dde0 tx=0x7f3cf4094b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a78699c44dc06bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.656+0000 7f3cd9ffb640 1 -- 172.21.15.64:0/4183103864 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40b1040 con 0x555c2ec9ff60 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.656+0000 7f3cd9ffb640 1 -- 172.21.15.64:0/4183103864 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cf40a28f0 con 0x555c2ec9ff60 2024-05-09T09:11:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.656+0000 7f3cd9ffb640 1 -- 172.21.15.64:0/4183103864 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40a6030 con 0x555c2ec9ff60 2024-05-09T09:11:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 -- 172.21.15.64:0/4183103864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 msgr2=0x555c2ec99040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ec99040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cf408dde0 tx=0x7f3cf4094b80 comp rx=0 tx=0).stop 2024-05-09T09:11:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 -- 172.21.15.64:0/4183103864 shutdown_connections 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9ff60 0x555c2ec99040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9adf0 0x555c2ecabb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 --2- 172.21.15.64:0/4183103864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9a400 0x555c2ec98b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 -- 172.21.15.64:0/4183103864 >> 172.21.15.64:0/4183103864 conn(0x555c2ec95a70 msgr2=0x555c2ec9c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 -- 172.21.15.64:0/4183103864 shutdown_connections 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.657+0000 7f3cfd577740 1 -- 172.21.15.64:0/4183103864 wait complete. 2024-05-09T09:11:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.658+0000 7f3cfd577740 1 -- start start 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.658+0000 7f3cfd577740 1 Processor -- start 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.659+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.659+0000 7f3cfd577740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9adf0 0x555c2eca9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.659+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.659+0000 7f3cf3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59754/0 (socket says 172.21.15.64:59754) 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.659+0000 7f3cf3fff640 1 -- 172.21.15.64:0/3529971784 learned_addr learned my addr 172.21.15.64:0/3529971784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9ff60 0x555c2ecaa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9adf0 0x555c2eca9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec15cdc0 con 0x555c2ec9a400 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec15e230 con 0x555c2ec9ff60 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cf8d33640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9ff60 0x555c2ecaa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec15d490 con 0x555c2ec9adf0 2024-05-09T09:11:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.660+0000 7f3cfd577740 10 client.0 _ll_register_callbacks cb 0x7f3cf4067200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cf37fe640 1 -- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9ff60 msgr2=0x555c2ecaa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9ff60 0x555c2ecaa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cf37fe640 1 -- 172.21.15.64:0/3529971784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 msgr2=0x555c2eca9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cf37fe640 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c2ecaac50 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf40a28f0 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cf37fe640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9adf0 0x555c2eca9650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8030a40 tx=0x7f3ce8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84bfdc948d58143 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cd37fe640 10 client.25927 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3529971784 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce80042f0 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3529971784 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ce8003af0 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3529971784 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ce8009470 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.661+0000 7f3cfd577740 10 client.25927 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=30}) -- 0x555c2ed166d0 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3529971784 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3ce80ba1b0 con 0x555c2ec9adf0 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 10 client.25927 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 20 client.25927 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ed16f40 0x555c2ed19360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 4 client.25927 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:11:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c2ec0ba70 con 0x555c2ed16f40 2024-05-09T09:11:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.662+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ed16f40 0x555c2ed19360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.663+0000 7f3cf3fff640 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ed16f40 0x555c2ed19360 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe1121323520626 server_cookie=79a3cdc7fba898d4 in_seq=0 out_seq=0 2024-05-09T09:11:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.663+0000 7f3cd37fe640 10 client.25927 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.689+0000 7f3cd37fe640 1 -- 172.21.15.64:0/3529971784 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c2ec0ba70 con 0x555c2ed16f40 2024-05-09T09:11:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.689+0000 7f3cd37fe640 10 client.25927 handle_command_reply: tid=0 2024-05-09T09:11:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.689+0000 7f3cfd577740 1 client.25927 shutdown 2024-05-09T09:11:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.690+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9adf0 msgr2=0x555c2eca9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.690+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9adf0 0x555c2eca9650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8030a40 tx=0x7f3ce8000bc0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.690+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 shutdown_connections 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c2ed16f40 0x555c2ed19360 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cd37fe640 0 client.25927 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c2ec9ff60 0x555c2ecaa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c2ec9adf0 0x555c2eca9650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 --2- 172.21.15.64:0/3529971784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c2ec9a400 0x555c2eca9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 >> 172.21.15.64:0/3529971784 conn(0x555c2ec95a70 msgr2=0x555c2eca45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 shutdown_connections 2024-05-09T09:11:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 1 -- 172.21.15.64:0/3529971784 wait complete. 2024-05-09T09:11:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.691+0000 7f3cfd577740 20 client.25927 trim_cache size 0 max 16384 2024-05-09T09:11:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:11:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:11:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:11:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:11:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3cdc0a2d10 msgr2=0x7f3cdc0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3cdc0a2d10 0x7f3cdc0a51d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40649a0 tx=0x7f3cf4064aa0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 msgr2=0x7f3cec00a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00a010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8030a70 tx=0x7f3ce8030f30 comp rx=0 tx=0).stop 2024-05-09T09:11:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 shutdown_connections 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3cdc0a2d10 0x7f3cdc0a51d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cec0a6160 0x7f3cec00a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cec0a5150 0x7f3cec00a010 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 --2- 172.21.15.64:0/4145665993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0a4790 0x7f3cec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 >> 172.21.15.64:0/4145665993 conn(0x7f3cec0a00d0 msgr2=0x7f3cec011f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 shutdown_connections 2024-05-09T09:11:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:11:51.693+0000 7f3cf9534640 1 -- 172.21.15.64:0/4145665993 wait complete. 2024-05-09T09:11:53.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:52 smithi179 ceph-mon[34086]: pgmap v452: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.7 MiB/s wr, 1.10k op/s 2024-05-09T09:11:53.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:52 smithi139 ceph-mon[33599]: pgmap v452: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.7 MiB/s wr, 1.10k op/s 2024-05-09T09:11:53.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:52 smithi064 ceph-mon[30149]: pgmap v452: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.7 MiB/s wr, 1.10k op/s 2024-05-09T09:11:54.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:11:54.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:11:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 -- 172.21.15.64:0/2646753045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009c3f0 msgr2=0x7f398009c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 --2- 172.21.15.64:0/2646753045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009c3f0 0x7f398009c810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f397802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 -- 172.21.15.64:0/2646753045 shutdown_connections 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 --2- 172.21.15.64:0/2646753045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 0x7f39800a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 --2- 172.21.15.64:0/2646753045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398009d590 0x7f398009da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 --2- 172.21.15.64:0/2646753045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009c3f0 0x7f398009c810 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f397802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.680+0000 7f398d42c640 1 -- 172.21.15.64:0/2646753045 >> 172.21.15.64:0/2646753045 conn(0x7f398000b340 msgr2=0x7f398000b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.681+0000 7f398d42c640 1 -- 172.21.15.64:0/2646753045 shutdown_connections 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.681+0000 7f398d42c640 1 -- 172.21.15.64:0/2646753045 wait complete. 2024-05-09T09:11:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.682+0000 7f398d42c640 1 Processor -- start 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.682+0000 7f398d42c640 1 -- start start 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f398d42c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009d590 0x7f3980132810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f398d42c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 0x7f3980132d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f398d42c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 0x7f398013a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f398d42c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39800a7ee0 con 0x7f3980137db0 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f39867fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 0x7f3980132d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f39877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 0x7f398013a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f39877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 0x7f398013a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52936/0 (socket says 172.21.15.64:52936) 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f39877fe640 1 -- 172.21.15.64:0/1679025253 learned_addr learned my addr 172.21.15.64:0/1679025253 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.683+0000 7f39867fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 0x7f3980132d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59728/0 (socket says 172.21.15.64:59728) 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.684+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39800a7d60 con 0x7f398009df50 2024-05-09T09:11:54.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.684+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39800a8060 con 0x7f398009d590 2024-05-09T09:11:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f39877fe640 1 -- 172.21.15.64:0/1679025253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009d590 msgr2=0x7f3980132810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:11:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f39877fe640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009d590 0x7f3980132810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f39877fe640 1 -- 172.21.15.64:0/1679025253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 msgr2=0x7f3980132d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f39877fe640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 0x7f3980132d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f39877fe640 1 -- 172.21.15.64:0/1679025253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398013a810 con 0x7f3980137db0 2024-05-09T09:11:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f39877fe640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 0x7f398013a1a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3988067000 tx=0x7f3988079ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.685+0000 7f396ffff640 1 -- 172.21.15.64:0/1679025253 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988084070 con 0x7f3980137db0 2024-05-09T09:11:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.686+0000 7f396ffff640 1 -- 172.21.15.64:0/1679025253 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f398807f3f0 con 0x7f3980137db0 2024-05-09T09:11:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.686+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398013ab00 con 0x7f3980137db0 2024-05-09T09:11:54.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.686+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398013afd0 con 0x7f3980137db0 2024-05-09T09:11:54.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.687+0000 7f396ffff640 1 -- 172.21.15.64:0/1679025253 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988069330 con 0x7f3980137db0 2024-05-09T09:11:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 Processor -- start 2024-05-09T09:11:54.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f396ffff640 1 -- 172.21.15.64:0/1679025253 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3988087070 con 0x7f3980137db0 2024-05-09T09:11:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 -- start start 2024-05-09T09:11:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f7120 0x55bc692f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc692fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692fcc80 0x55bc692ff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bc6925bd00 con 0x55bc692fcc80 2024-05-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bc692034c0 con 0x55bc692f7b10 2024-05-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.691+0000 7f3990210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bc69257cc0 con 0x55bc692f7120 2024-05-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.692+0000 7f3986ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc692fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.692+0000 7f3986ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc692fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59744/0 (socket says 172.21.15.64:59744) 2024-05-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.692+0000 7f3986ffd640 1 -- 172.21.15.64:0/441657455 learned_addr learned my addr 172.21.15.64:0/441657455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.693+0000 7f396ffff640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39700a2a70 0x7f39700a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.693+0000 7f396ffff640 1 -- 172.21.15.64:0/1679025253 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f398813a6a0 con 0x7f3980137db0 2024-05-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.693+0000 7f39877fe640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39700a2a70 0x7f39700a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.694+0000 7f39867fc640 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f7120 0x55bc692f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.694+0000 7f39867fc640 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692fcc80 0x55bc692ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.694+0000 7f39877fe640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39700a2a70 0x7f39700a4f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39880503a0 tx=0x7f3988073f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.694+0000 7f39867fc640 1 -- 172.21.15.64:0/441657455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 msgr2=0x55bc692fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.695+0000 7f39867fc640 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc692fc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.695+0000 7f39867fc640 1 -- 172.21.15.64:0/441657455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692fcc80 msgr2=0x55bc692ff110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.695+0000 7f39867fc640 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692fcc80 0x55bc692ff110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.695+0000 7f39867fc640 1 -- 172.21.15.64:0/441657455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc690397d0 con 0x55bc692f7120 2024-05-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.695+0000 7f39867fc640 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f7120 0x55bc692f7540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3980133d20 tx=0x7f397c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c55897ef63cd36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.696+0000 7f396d7fa640 1 -- 172.21.15.64:0/441657455 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c004050 con 0x55bc692f7120 2024-05-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.696+0000 7f396d7fa640 1 -- 172.21.15.64:0/441657455 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c0054d0 con 0x55bc692f7120 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f396d7fa640 1 -- 172.21.15.64:0/441657455 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c004cc0 con 0x55bc692f7120 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 -- 172.21.15.64:0/441657455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f7120 msgr2=0x55bc692f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f7120 0x55bc692f7540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3980133d20 tx=0x7f397c0096c0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 -- 172.21.15.64:0/441657455 shutdown_connections 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692fcc80 0x55bc692ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc692fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 --2- 172.21.15.64:0/441657455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f7120 0x55bc692f7540 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3980133d20 tx=0x7f397c0096c0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.697+0000 7f3990210740 1 -- 172.21.15.64:0/441657455 >> 172.21.15.64:0/441657455 conn(0x55bc692f2790 msgr2=0x55bc692f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.698+0000 7f3990210740 1 -- 172.21.15.64:0/441657455 shutdown_connections 2024-05-09T09:11:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.698+0000 7f3990210740 1 -- 172.21.15.64:0/441657455 wait complete. 2024-05-09T09:11:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.698+0000 7f3990210740 1 -- start start 2024-05-09T09:11:54.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.698+0000 7f3990210740 1 Processor -- start 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc69312830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 0x55bc69312d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc69317da0 0x55bc6931a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bc69310c90 con 0x55bc69317da0 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bc69216b90 con 0x55bc692f7b10 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bc69310e10 con 0x55bc692fcc80 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3990210740 10 client.0 _ll_register_callbacks cb 0x55bc69252da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 0x55bc69312d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 0x55bc69312d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40242/0 (socket says 172.21.15.64:40242) 2024-05-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 -- 172.21.15.64:0/1685056309 learned_addr learned my addr 172.21.15.64:0/1685056309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 -- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 msgr2=0x55bc69312830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc69312830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 -- 172.21.15.64:0/1685056309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc69317da0 msgr2=0x55bc6931a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.699+0000 7f3986ffd640 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc69317da0 0x55bc6931a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.700+0000 7f3986ffd640 1 -- 172.21.15.64:0/1685056309 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc68f6f3f0 con 0x55bc692fcc80 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.700+0000 7f3986ffd640 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 0x55bc69312d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3978006010 tx=0x7f3978032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abe4f26b583dd316 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f396aff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f396aff5640 1 -- 172.21.15.64:0/1685056309 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978040070 con 0x55bc692fcc80 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397c0054d0 con 0x55bc692fcc80 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f396aff5640 1 -- 172.21.15.64:0/1685056309 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3978004040 con 0x55bc692fcc80 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f396aff5640 1 -- 172.21.15.64:0/1685056309 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f397804e050 con 0x55bc692fcc80 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f3990210740 10 client.25936 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.701+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc68f2f0f0 con 0x55bc692fcc80 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.702+0000 7f396aff5640 1 -- 172.21.15.64:0/1685056309 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c0054d0 con 0x55bc692fcc80 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.702+0000 7f396aff5640 1 -- 172.21.15.64:0/1685056309 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f39780046c0 con 0x55bc692fcc80 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f3990210740 10 client.25936 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f3990210740 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f3990210740 20 client.25936 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f3990210740 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc6932fb10 0x55bc69331f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f3990210740 4 client.25936 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc69269430 con 0x55bc6932fb10 2024-05-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.703+0000 7f39867fc640 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc6932fb10 0x55bc69331f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.705+0000 7f39867fc640 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc6932fb10 0x55bc69331f70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b29c00d17597e74 server_cookie=91c8656d2fd67996 in_seq=0 out_seq=0 2024-05-09T09:11:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.705+0000 7f396aff5640 10 client.25936 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.706+0000 7f396aff5640 1 -- 172.21.15.64:0/1685056309 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc69269430 con 0x55bc6932fb10 2024-05-09T09:11:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.707+0000 7f396aff5640 10 client.25936 handle_command_reply: tid=0 2024-05-09T09:11:54.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.707+0000 7f3990210740 1 client.25936 shutdown 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.707+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 msgr2=0x55bc69312d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.707+0000 7f3990210740 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 0x55bc69312d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3978006010 tx=0x7f3978032d30 comp rx=0 tx=0).stop 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 shutdown_connections 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc6932fb10 0x55bc69331f70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc69317da0 0x55bc6931a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692fcc80 0x55bc69312d70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 --2- 172.21.15.64:0/1685056309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692f7b10 0x55bc69312830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 >> 172.21.15.64:0/1685056309 conn(0x55bc692f2790 msgr2=0x55bc692f1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f396aff5640 0 client.25936 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 shutdown_connections 2024-05-09T09:11:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 1 -- 172.21.15.64:0/1685056309 wait complete. 2024-05-09T09:11:54.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.708+0000 7f3990210740 20 client.25936 trim_cache size 0 max 16384 2024-05-09T09:11:54.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.744+0000 7f3990210740 1 Processor -- start 2024-05-09T09:11:54.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.744+0000 7f3990210740 1 -- start start 2024-05-09T09:11:54.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.745+0000 7f3990210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc692f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.745+0000 7f3990210740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc692f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.745+0000 7f3990210740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc692f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.745+0000 7f3990210740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3974012640 con 0x55bc692f7b10 2024-05-09T09:11:54.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.745+0000 7f3990210740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f397400fe80 con 0x55bc692fcc80 2024-05-09T09:11:54.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.745+0000 7f3990210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f397400dcd0 con 0x55bc692f2790 2024-05-09T09:11:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.748+0000 7f39867fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc692f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.748+0000 7f39867fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc692f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40254/0 (socket says 172.21.15.64:40254) 2024-05-09T09:11:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.748+0000 7f39867fc640 1 -- 172.21.15.64:0/2215290170 learned_addr learned my addr 172.21.15.64:0/2215290170 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.748+0000 7f39867fc640 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc692f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.748+0000 7f3986ffd640 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc692f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.749+0000 7f39867fc640 1 -- 172.21.15.64:0/2215290170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 msgr2=0x55bc692f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.749+0000 7f39867fc640 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc692f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.749+0000 7f39867fc640 1 -- 172.21.15.64:0/2215290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 msgr2=0x55bc692f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.749+0000 7f39867fc640 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc692f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.749+0000 7f39867fc640 1 -- 172.21.15.64:0/2215290170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc692ee150 con 0x55bc692f2790 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.749+0000 7f39867fc640 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc692f9120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f397c009250 tx=0x7f397c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f03134749cc2f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.750+0000 7f396cff9640 1 -- 172.21.15.64:0/2215290170 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c02a460 con 0x55bc692f2790 2024-05-09T09:11:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.750+0000 7f396cff9640 1 -- 172.21.15.64:0/2215290170 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c01e460 con 0x55bc692f2790 2024-05-09T09:11:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.751+0000 7f3990210740 1 -- 172.21.15.64:0/2215290170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 msgr2=0x55bc692f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.751+0000 7f3990210740 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc692f9120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f397c009250 tx=0x7f397c005f90 comp rx=0 tx=0).stop 2024-05-09T09:11:54.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.751+0000 7f3990210740 1 -- 172.21.15.64:0/2215290170 shutdown_connections 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.751+0000 7f3990210740 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc692f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.751+0000 7f3990210740 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc692f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.751+0000 7f3990210740 1 --2- 172.21.15.64:0/2215290170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc692f9120 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.752+0000 7f3990210740 1 -- 172.21.15.64:0/2215290170 >> 172.21.15.64:0/2215290170 conn(0x55bc692edc30 msgr2=0x55bc69312830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.752+0000 7f3990210740 1 -- 172.21.15.64:0/2215290170 shutdown_connections 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.752+0000 7f3990210740 1 -- 172.21.15.64:0/2215290170 wait complete. 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.752+0000 7f3990210740 1 -- start start 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.752+0000 7f3990210740 1 Processor -- start 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc69340670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc69340bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc693419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f397400e340 con 0x55bc692f7b10 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f397400f7b0 con 0x55bc692fcc80 2024-05-09T09:11:54.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f397400ea10 con 0x55bc692f2790 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3990210740 10 client.0 _ll_register_callbacks cb 0x7f3978009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3986ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc69340bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3986ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc69340bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52970/0 (socket says 172.21.15.64:52970) 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f3986ffd640 1 -- 172.21.15.64:0/2318993341 learned_addr learned my addr 172.21.15.64:0/2318993341 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.753+0000 7f39867fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc69340670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f39867fc640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc693419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f3986ffd640 1 -- 172.21.15.64:0/2318993341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 msgr2=0x55bc69340670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f3986ffd640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc69340670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f3986ffd640 1 -- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 msgr2=0x55bc693419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f3986ffd640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc693419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f3986ffd640 1 -- 172.21.15.64:0/2318993341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc69341fa0 con 0x55bc692f7b10 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.754+0000 7f39867fc640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc69340670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:54.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.755+0000 7f3986ffd640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc69340bb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55bc69313d10 tx=0x7f3978000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f08d2a9dc5c4167 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.755+0000 7f396aff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.756+0000 7f396aff5640 1 -- 172.21.15.64:0/2318993341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978040030 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.756+0000 7f396aff5640 1 -- 172.21.15.64:0/2318993341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f39780316b0 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.756+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397c01e460 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.756+0000 7f396aff5640 1 -- 172.21.15.64:0/2318993341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3978035040 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.756+0000 7f3990210740 10 client.16269 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.756+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc69343320 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.757+0000 7f396aff5640 1 -- 172.21.15.64:0/2318993341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c01e460 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.757+0000 7f396aff5640 1 -- 172.21.15.64:0/2318993341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3978003970 con 0x55bc692f7b10 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.758+0000 7f3990210740 10 client.16269 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.758+0000 7f3990210740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.758+0000 7f3990210740 20 client.16269 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:54.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.758+0000 7f3990210740 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc69317da0 0x55bc69397fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.758+0000 7f39867fc640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc69317da0 0x55bc69397fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.759+0000 7f3990210740 4 client.16269 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:11:54.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.759+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc69263780 con 0x55bc69317da0 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.759+0000 7f396aff5640 10 client.16269 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.759+0000 7f39867fc640 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc69317da0 0x55bc69397fe0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6cf754640fe66b6 server_cookie=18988fce2db09813 in_seq=0 out_seq=0 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f396aff5640 1 -- 172.21.15.64:0/2318993341 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc69263780 con 0x55bc69317da0 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f396aff5640 10 client.16269 handle_command_reply: tid=0 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f3990210740 1 client.16269 shutdown 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 msgr2=0x55bc69340bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f3990210740 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc69340bb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55bc69313d10 tx=0x7f3978000bc0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 shutdown_connections 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.760+0000 7f3990210740 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bc69317da0 0x55bc69397fe0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bc692fcc80 0x55bc693419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc692f7b10 0x55bc69340bb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 1 --2- 172.21.15.64:0/2318993341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bc692f2790 0x55bc69340670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 >> 172.21.15.64:0/2318993341 conn(0x55bc692edc30 msgr2=0x55bc69312830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f396aff5640 0 client.16269 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 shutdown_connections 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 1 -- 172.21.15.64:0/2318993341 wait complete. 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.761+0000 7f3990210740 20 client.16269 trim_cache size 0 max 16384 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:11:54.763 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.762+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39700a2a70 msgr2=0x7f39700a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39700a2a70 0x7f39700a4f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39880503a0 tx=0x7f3988073f90 comp rx=0 tx=0).stop 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 msgr2=0x7f398013a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 0x7f398013a1a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3988067000 tx=0x7f3988079ef0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 shutdown_connections 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f39700a2a70 0x7f39700a4f30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3980137db0 0x7f398013a1a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f398009df50 0x7f3980132d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 --2- 172.21.15.64:0/1679025253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f398009d590 0x7f3980132810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 >> 172.21.15.64:0/1679025253 conn(0x7f398000b340 msgr2=0x7f39800a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 shutdown_connections 2024-05-09T09:11:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.763+0000 7f398d42c640 1 -- 172.21.15.64:0/1679025253 wait complete. 2024-05-09T09:11:54.781 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:11:54.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:11:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 -- 172.21.15.64:0/1016420931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c107e60 msgr2=0x7fd97c108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 --2- 172.21.15.64:0/1016420931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c107e60 0x7fd97c108280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009900 tx=0x7fd96c02a420 comp rx=0 tx=0).stop 2024-05-09T09:11:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 -- 172.21.15.64:0/1016420931 shutdown_connections 2024-05-09T09:11:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 --2- 172.21.15.64:0/1016420931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c109a00 0x7fd97c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 --2- 172.21.15.64:0/1016420931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd97c109040 0x7fd97c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 --2- 172.21.15.64:0/1016420931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c107e60 0x7fd97c108280 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.976+0000 7fd980e22640 1 -- 172.21.15.64:0/1016420931 >> 172.21.15.64:0/1016420931 conn(0x7fd97c076e40 msgr2=0x7fd97c079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.977+0000 7fd980e22640 1 -- 172.21.15.64:0/1016420931 shutdown_connections 2024-05-09T09:11:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.978+0000 7fd980e22640 1 -- 172.21.15.64:0/1016420931 wait complete. 2024-05-09T09:11:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.979+0000 7fd980e22640 1 Processor -- start 2024-05-09T09:11:54.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.979+0000 7fd980e22640 1 -- start start 2024-05-09T09:11:54.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd980e22640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 0x7fd97c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd980e22640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c109040 0x7fd97c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd980e22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd97c109a00 0x7fd97c1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd980e22640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c113ac0 con 0x7fd97c109040 2024-05-09T09:11:54.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd980e22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c113940 con 0x7fd97c109a00 2024-05-09T09:11:54.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd980e22640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c113c40 con 0x7fd97c107e60 2024-05-09T09:11:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd97a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 0x7fd97c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd97a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 0x7fd97c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40274/0 (socket says 172.21.15.64:40274) 2024-05-09T09:11:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.980+0000 7fd97a59c640 1 -- 172.21.15.64:0/3265031355 learned_addr learned my addr 172.21.15.64:0/3265031355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd973fff640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c109040 0x7fd97c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd97ad9d640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd97c109a00 0x7fd97c1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd97a59c640 1 -- 172.21.15.64:0/3265031355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd97c109a00 msgr2=0x7fd97c1a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd97a59c640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd97c109a00 0x7fd97c1a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd97a59c640 1 -- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c109040 msgr2=0x7fd97c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd97a59c640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c109040 0x7fd97c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.981+0000 7fd97a59c640 1 -- 172.21.15.64:0/3265031355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1a5d20 con 0x7fd97c107e60 2024-05-09T09:11:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.982+0000 7fd97a59c640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 0x7fd97c19e100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009850 tx=0x7fd96c003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.982+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c03f070 con 0x7fd97c107e60 2024-05-09T09:11:54.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.982+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd96c031e40 con 0x7fd97c107e60 2024-05-09T09:11:54.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.982+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c035030 con 0x7fd97c107e60 2024-05-09T09:11:54.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.982+0000 7fd980e22640 1 -- 172.21.15.64:0/3265031355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a5fb0 con 0x7fd97c107e60 2024-05-09T09:11:54.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.982+0000 7fd980e22640 1 -- 172.21.15.64:0/3265031355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a63e0 con 0x7fd97c107e60 2024-05-09T09:11:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.984+0000 7fd980e22640 1 -- 172.21.15.64:0/3265031355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c10f200 con 0x7fd97c107e60 2024-05-09T09:11:54.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.986+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd96c042070 con 0x7fd97c107e60 2024-05-09T09:11:54.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.987+0000 7fd9737fe640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9680a2d10 0x7fd9680a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:54.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.987+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd96c03b460 con 0x7fd97c107e60 2024-05-09T09:11:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.988+0000 7fd973fff640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9680a2d10 0x7fd9680a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.989+0000 7fd973fff640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9680a2d10 0x7fd9680a51d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd97c19f610 tx=0x7fd96400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:54.996+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd96c0bca10 con 0x7fd97c107e60 2024-05-09T09:11:55.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:54 smithi064 ceph-mon[30149]: pgmap v453: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T09:11:55.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:54 smithi179 ceph-mon[34086]: pgmap v453: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T09:11:55.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:54 smithi139 ceph-mon[33599]: pgmap v453: 129 pgs: 129 active+clean; 976 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-05-09T09:11:55.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:11:55.479 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:10:25.554787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:11:55.480 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 30 2024-05-09T09:11:55.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.478+0000 7fd980e22640 1 -- 172.21.15.64:0/3265031355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97c02d1c0 con 0x7fd97c107e60 2024-05-09T09:11:55.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.480+0000 7fd9737fe640 1 -- 172.21.15.64:0/3265031355 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11886 (secure 0 0 0) 0x7fd96c0c06c0 con 0x7fd97c107e60 2024-05-09T09:11:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.484+0000 7fd9717fa640 1 -- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9680a2d10 msgr2=0x7fd9680a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.484+0000 7fd9717fa640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9680a2d10 0x7fd9680a51d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd97c19f610 tx=0x7fd96400a2d0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.484+0000 7fd9717fa640 1 -- 172.21.15.64:0/3265031355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 msgr2=0x7fd97c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.484+0000 7fd9717fa640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 0x7fd97c19e100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009850 tx=0x7fd96c003e10 comp rx=0 tx=0).stop 2024-05-09T09:11:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.485+0000 7fd9717fa640 1 -- 172.21.15.64:0/3265031355 shutdown_connections 2024-05-09T09:11:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.485+0000 7fd9717fa640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9680a2d10 0x7fd9680a51d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.485+0000 7fd9717fa640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd97c109a00 0x7fd97c1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.485+0000 7fd9717fa640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97c109040 0x7fd97c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.485+0000 7fd9717fa640 1 --2- 172.21.15.64:0/3265031355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd97c107e60 0x7fd97c19e100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.485+0000 7fd9717fa640 1 -- 172.21.15.64:0/3265031355 >> 172.21.15.64:0/3265031355 conn(0x7fd97c076e40 msgr2=0x7fd97c10c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.487+0000 7fd9717fa640 1 -- 172.21.15.64:0/3265031355 shutdown_connections 2024-05-09T09:11:55.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.487+0000 7fd9717fa640 1 -- 172.21.15.64:0/3265031355 wait complete. 2024-05-09T09:11:55.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:11:55.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 -- 172.21.15.64:0/981207260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24102320 msgr2=0x7f2e24108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 --2- 172.21.15.64:0/981207260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24102320 0x7f2e24108bb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009ca0 tx=0x7f2e1c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 -- 172.21.15.64:0/981207260 shutdown_connections 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 --2- 172.21.15.64:0/981207260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24102320 0x7f2e24108bb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 --2- 172.21.15.64:0/981207260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e24101960 0x7f2e24101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 --2- 172.21.15.64:0/981207260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e24100770 0x7f2e24100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 -- 172.21.15.64:0/981207260 >> 172.21.15.64:0/981207260 conn(0x7f2e240fbce0 msgr2=0x7f2e240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 -- 172.21.15.64:0/981207260 shutdown_connections 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.640+0000 7f2e2af09640 1 -- 172.21.15.64:0/981207260 wait complete. 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 Processor -- start 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 -- start start 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e24100770 0x7f2e241048a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 0x7f2e24102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e24102320 0x7f2e24103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410c2a0 con 0x7f2e24101960 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410c120 con 0x7f2e24100770 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e2af09640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410c420 con 0x7f2e24102320 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 0x7f2e24102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 0x7f2e24102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53022/0 (socket says 172.21.15.64:53022) 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2692796820 learned_addr learned my addr 172.21.15.64:0/2692796820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e24100770 0x7f2e241048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e294a6640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e24102320 0x7f2e24103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2692796820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e24102320 msgr2=0x7f2e24103470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e24102320 0x7f2e24103470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2692796820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e24100770 msgr2=0x7f2e241048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e24100770 0x7f2e241048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.641+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2692796820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e24103d30 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.642+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 0x7f2e24102f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400d950 tx=0x7f2e1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.642+0000 7f2e19ffb640 1 -- 172.21.15.64:0/2692796820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14017070 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.642+0000 7f2e19ffb640 1 -- 172.21.15.64:0/2692796820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e14009c30 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.642+0000 7f2e19ffb640 1 -- 172.21.15.64:0/2692796820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400f620 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.642+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e241a1c30 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.642+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241a21a0 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e19ffb640 1 -- 172.21.15.64:0/2692796820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e1400f7c0 con 0x7f2e24101960 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 Processor -- start 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 -- start start 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 0x55a8de1edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8de14ac40 con 0x55a8de1ebaa0 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a8de0f2580 con 0x55a8de1e5f40 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a8de146c00 con 0x55a8de1e6930 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 0x55a8de1edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e28ca5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 0x55a8de1edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53032/0 (socket says 172.21.15.64:53032) 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e28ca5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40308/0 (socket says 172.21.15.64:40308) 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2171238487 learned_addr learned my addr 172.21.15.64:0/2171238487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.645+0000 7f2e294a6640 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2171238487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 msgr2=0x55a8de1eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e19ffb640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e0c0a2c90 0x7f2e0c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2171238487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 msgr2=0x55a8de1e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e1bfff640 1 -- 172.21.15.64:0/2171238487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8dde5e3f0 con 0x55a8de1ebaa0 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e294a6640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e0c0a2c90 0x7f2e0c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e19ffb640 1 -- 172.21.15.64:0/2692796820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e140cf010 con 0x7f2e24101960 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 0x55a8de1edf30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e140997b0 tx=0x7f2e140d74b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63eaa053f20be38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e0affd640 1 -- 172.21.15.64:0/2171238487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14020030 con 0x55a8de1ebaa0 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e0affd640 1 -- 172.21.15.64:0/2171238487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e14099320 con 0x55a8de1ebaa0 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e294a6640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e0c0a2c90 0x7f2e0c0a5150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55a8de1e7190 tx=0x7f2e1c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.646+0000 7f2e0affd640 1 -- 172.21.15.64:0/2171238487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1409ade0 con 0x55a8de1ebaa0 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 -- 172.21.15.64:0/2171238487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 msgr2=0x55a8de1edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 0x55a8de1edf30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e140997b0 tx=0x7f2e140d74b0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 -- 172.21.15.64:0/2171238487 shutdown_connections 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1ebaa0 0x55a8de1edf30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/2171238487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 -- 172.21.15.64:0/2171238487 >> 172.21.15.64:0/2171238487 conn(0x55a8de1e15b0 msgr2=0x55a8de1e19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 -- 172.21.15.64:0/2171238487 shutdown_connections 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.647+0000 7f2e2dc13740 1 -- 172.21.15.64:0/2171238487 wait complete. 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 -- start start 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 Processor -- start 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de1f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8de1eeaa0 con 0x55a8de1f5bc0 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a8de105c40 con 0x55a8de1e5f40 2024-05-09T09:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a8de1eec20 con 0x55a8de1e6930 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e2dc13740 10 client.0 _ll_register_callbacks cb 0x55a8dde5dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e294a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de1f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e294a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de1f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53042/0 (socket says 172.21.15.64:53042) 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e294a6640 1 -- 172.21.15.64:0/4194930977 learned_addr learned my addr 172.21.15.64:0/4194930977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.648+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e294a6640 1 -- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 msgr2=0x55a8de1f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e294a6640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e294a6640 1 -- 172.21.15.64:0/4194930977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 msgr2=0x55a8de1f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e294a6640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e294a6640 1 -- 172.21.15.64:0/4194930977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ddf287d0 con 0x55a8de1f5bc0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e294a6640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de1f7fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c035000 tx=0x7f2e1c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184991402a9858b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e03fff640 1 -- 172.21.15.64:0/4194930977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c004450 con 0x55a8de1f5bc0 2024-05-09T09:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e14099320 con 0x55a8de1f5bc0 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e03fff640 1 -- 172.21.15.64:0/4194930977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e1c0045f0 con 0x55a8de1f5bc0 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e03fff640 1 -- 172.21.15.64:0/4194930977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c0047f0 con 0x55a8de1f5bc0 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e2dc13740 10 client.16287 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8de1fb320 con 0x55a8de1f5bc0 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.649+0000 7f2e03fff640 1 -- 172.21.15.64:0/4194930977 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e14099320 con 0x55a8de1f5bc0 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e03fff640 1 -- 172.21.15.64:0/4194930977 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2e1c02f5e0 con 0x55a8de1f5bc0 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e2dc13740 10 client.16287 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e2dc13740 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e2dc13740 20 client.16287 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de1fcad0 0x55a8de1fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e2dc13740 4 client.16287 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8de158370 con 0x55a8de1fcad0 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de1fcad0 0x55a8de1fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e03fff640 10 client.16287 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.650+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de1fcad0 0x55a8de1fef70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35384a3cb0436686 server_cookie=68c896cd6f2873a1 in_seq=0 out_seq=0 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e03fff640 1 -- 172.21.15.64:0/4194930977 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8de158370 con 0x55a8de1fcad0 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e03fff640 10 client.16287 handle_command_reply: tid=0 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 client.16287 shutdown 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 msgr2=0x55a8de1f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de1f7fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c035000 tx=0x7f2e1c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 shutdown_connections 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de1fcad0 0x55a8de1fef70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de1f7fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de1f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e03fff640 0 client.16287 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4194930977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 >> 172.21.15.64:0/4194930977 conn(0x55a8de1e15b0 msgr2=0x55a8de1e07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 shutdown_connections 2024-05-09T09:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.651+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4194930977 wait complete. 2024-05-09T09:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.652+0000 7f2e2dc13740 20 client.16287 trim_cache size 0 max 16384 2024-05-09T09:11:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 Processor -- start 2024-05-09T09:11:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 -- start start 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 0x55a8de1e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1f5bc0 0x55a8de1e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241aefb0 con 0x55a8de1e6930 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ac7f0 con 0x55a8de1e5f40 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241aa640 con 0x55a8de1f5bc0 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 0x55a8de1e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 0x55a8de1e48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53056/0 (socket says 172.21.15.64:53056) 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 -- 172.21.15.64:0/799562644 learned_addr learned my addr 172.21.15.64:0/799562644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e294a6640 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1f5bc0 0x55a8de1e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 -- 172.21.15.64:0/799562644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1f5bc0 msgr2=0x55a8de1e4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1f5bc0 0x55a8de1e4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 -- 172.21.15.64:0/799562644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 msgr2=0x55a8de1e43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.669+0000 7f2e1bfff640 1 -- 172.21.15.64:0/799562644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8de1ff5f0 con 0x55a8de1e6930 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.670+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 0x55a8de1e48e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2e1409b180 tx=0x7f2e14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476f177a7f21c8c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.670+0000 7f2e0a7fc640 1 -- 172.21.15.64:0/799562644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140b4070 con 0x55a8de1e6930 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.670+0000 7f2e0a7fc640 1 -- 172.21.15.64:0/799562644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e140d2820 con 0x55a8de1e6930 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.670+0000 7f2e0a7fc640 1 -- 172.21.15.64:0/799562644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14020040 con 0x55a8de1e6930 2024-05-09T09:11:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 -- 172.21.15.64:0/799562644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 msgr2=0x55a8de1e48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 0x55a8de1e48e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2e1409b180 tx=0x7f2e14005880 comp rx=0 tx=0).stop 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 -- 172.21.15.64:0/799562644 shutdown_connections 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1f5bc0 0x55a8de1e4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1e6930 0x55a8de1e48e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/799562644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de1e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 -- 172.21.15.64:0/799562644 >> 172.21.15.64:0/799562644 conn(0x55a8de1e15b0 msgr2=0x55a8de1ef050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 -- 172.21.15.64:0/799562644 shutdown_connections 2024-05-09T09:11:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 -- 172.21.15.64:0/799562644 wait complete. 2024-05-09T09:11:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 -- start start 2024-05-09T09:11:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.671+0000 7f2e2dc13740 1 Processor -- start 2024-05-09T09:11:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de20dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de20e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de20f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241aacb0 con 0x55a8de1f5bc0 2024-05-09T09:11:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ac120 con 0x55a8de1e5f40 2024-05-09T09:11:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ab380 con 0x55a8de1e6930 2024-05-09T09:11:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e2dc13740 10 client.0 _ll_register_callbacks cb 0x7f2e1c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de20f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de20f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53062/0 (socket says 172.21.15.64:53062) 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 -- 172.21.15.64:0/4129770199 learned_addr learned my addr 172.21.15.64:0/4129770199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de20e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de20dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 -- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 msgr2=0x55a8de20e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de20e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 -- 172.21.15.64:0/4129770199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 msgr2=0x55a8de20dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de20dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 -- 172.21.15.64:0/4129770199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8de20f700 con 0x55a8de1f5bc0 2024-05-09T09:11:55.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e1bfff640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de20e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:55.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de20dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:55.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e294a6640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de20f110 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0049d0 tx=0x7f2e1c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241e853b0f005c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.672+0000 7f2e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:11:55.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e03fff640 1 -- 172.21.15.64:0/4129770199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c064060 con 0x55a8de1f5bc0 2024-05-09T09:11:55.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e03fff640 1 -- 172.21.15.64:0/4129770199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e1c004250 con 0x55a8de1f5bc0 2024-05-09T09:11:55.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e140d2820 con 0x55a8de1f5bc0 2024-05-09T09:11:55.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e03fff640 1 -- 172.21.15.64:0/4129770199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c004450 con 0x55a8de1f5bc0 2024-05-09T09:11:55.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 10 client.16293 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:55.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8de262d00 con 0x55a8de1f5bc0 2024-05-09T09:11:55.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e03fff640 1 -- 172.21.15.64:0/4129770199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e140d2820 con 0x55a8de1f5bc0 2024-05-09T09:11:55.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e03fff640 1 -- 172.21.15.64:0/4129770199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2e1c05f450 con 0x55a8de1f5bc0 2024-05-09T09:11:55.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 10 client.16293 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:55.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 10 client.16293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:11:55.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 20 client.16293 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:55.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de263730 0x55a8de265b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 4 client.16293 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:11:55.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.673+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8de1526c0 con 0x55a8de263730 2024-05-09T09:11:55.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de263730 0x55a8de265b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e28ca5640 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de263730 0x55a8de265b50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f45dbe981ba4acbb server_cookie=2f4e88c05caebcf8 in_seq=0 out_seq=0 2024-05-09T09:11:55.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e03fff640 10 client.16293 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:55.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e03fff640 1 -- 172.21.15.64:0/4129770199 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8de1526c0 con 0x55a8de263730 2024-05-09T09:11:55.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e03fff640 10 client.16293 handle_command_reply: tid=0 2024-05-09T09:11:55.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 client.16293 shutdown 2024-05-09T09:11:55.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 msgr2=0x55a8de20f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de20f110 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0049d0 tx=0x7f2e1c009940 comp rx=0 tx=0).stop 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 shutdown_connections 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a8de263730 0x55a8de265b50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8de1f5bc0 0x55a8de20f110 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de1e6930 0x55a8de20e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e03fff640 0 client.16293 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 --2- 172.21.15.64:0/4129770199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8de1e5f40 0x55a8de20dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.674+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 >> 172.21.15.64:0/4129770199 conn(0x55a8de1e15b0 msgr2=0x55a8de1dff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 shutdown_connections 2024-05-09T09:11:55.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2dc13740 1 -- 172.21.15.64:0/4129770199 wait complete. 2024-05-09T09:11:55.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2dc13740 20 client.16293 trim_cache size 0 max 16384 2024-05-09T09:11:55.687 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:11:55.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e0c0a2c90 msgr2=0x7f2e0c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e0c0a2c90 0x7f2e0c0a5150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55a8de1e7190 tx=0x7f2e1c0095c0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 msgr2=0x7f2e24102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 0x7f2e24102f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400d950 tx=0x7f2e1400de10 comp rx=0 tx=0).stop 2024-05-09T09:11:55.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 shutdown_connections 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e0c0a2c90 0x7f2e0c0a5150 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e24102320 0x7f2e24103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e24101960 0x7f2e24102f30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 --2- 172.21.15.64:0/2692796820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e24100770 0x7f2e241048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.675+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 >> 172.21.15.64:0/2692796820 conn(0x7f2e240fbce0 msgr2=0x7f2e240689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.676+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 shutdown_connections 2024-05-09T09:11:55.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.676+0000 7f2e2af09640 1 -- 172.21.15.64:0/2692796820 wait complete. 2024-05-09T09:11:55.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:11:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.848+0000 7fa49bfff640 1 -- 172.21.15.64:0/2561204395 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c031880 con 0x7fa4a8100790 2024-05-09T09:11:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 -- 172.21.15.64:0/2561204395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 msgr2=0x7fa4a8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 --2- 172.21.15.64:0/2561204395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a8100bb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009900 tx=0x7fa49c02a400 comp rx=0 tx=0).stop 2024-05-09T09:11:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 -- 172.21.15.64:0/2561204395 shutdown_connections 2024-05-09T09:11:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 --2- 172.21.15.64:0/2561204395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 0x7fa4a8069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 --2- 172.21.15.64:0/2561204395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 --2- 172.21.15.64:0/2561204395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a8100bb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 -- 172.21.15.64:0/2561204395 >> 172.21.15.64:0/2561204395 conn(0x7fa4a80fbd00 msgr2=0x7fa4a80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.849+0000 7fa4af615640 1 -- 172.21.15.64:0/2561204395 shutdown_connections 2024-05-09T09:11:55.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.850+0000 7fa4af615640 1 -- 172.21.15.64:0/2561204395 wait complete. 2024-05-09T09:11:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.850+0000 7fa4af615640 1 Processor -- start 2024-05-09T09:11:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.850+0000 7fa4af615640 1 -- start start 2024-05-09T09:11:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4af615640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4af615640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4af615640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 0x7fa4a81a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4af615640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810b220 con 0x7fa4a8101980 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4acbb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4acbb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53082/0 (socket says 172.21.15.64:53082) 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4acbb0640 1 -- 172.21.15.64:0/2295421350 learned_addr learned my addr 172.21.15.64:0/2295421350 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810b0a0 con 0x7fa4a8100790 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810b3a0 con 0x7fa4a8102340 2024-05-09T09:11:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.851+0000 7fa4adbb2640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 0x7fa4a81a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4adbb2640 1 -- 172.21.15.64:0/2295421350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 msgr2=0x7fa4a819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4adbb2640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4adbb2640 1 -- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 msgr2=0x7fa4a819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4adbb2640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4adbb2640 1 -- 172.21.15.64:0/2295421350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a81a5d50 con 0x7fa4a8102340 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.852+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.853+0000 7fa4adbb2640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 0x7fa4a81a56e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fd0 tx=0x7fa4a000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.853+0000 7fa49a7fc640 1 -- 172.21.15.64:0/2295421350 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000ce20 con 0x7fa4a8102340 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.853+0000 7fa49a7fc640 1 -- 172.21.15.64:0/2295421350 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4a0005020 con 0x7fa4a8102340 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.853+0000 7fa49a7fc640 1 -- 172.21.15.64:0/2295421350 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0010690 con 0x7fa4a8102340 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.854+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a81a5fe0 con 0x7fa4a8102340 2024-05-09T09:11:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.854+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a64b0 con 0x7fa4a8102340 2024-05-09T09:11:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.859+0000 7fa4b23ff740 1 Processor -- start 2024-05-09T09:11:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.859+0000 7fa49a7fc640 1 -- 172.21.15.64:0/2295421350 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4a00051d0 con 0x7fa4a8102340 2024-05-09T09:11:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.859+0000 7fa4b23ff740 1 -- start start 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d4f40 0x5603228d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560322839c40 con 0x5603228d4f40 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5603227e1580 con 0x5603228daaa0 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560322835c00 con 0x5603228d5930 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228da430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40354/0 (socket says 172.21.15.64:40354) 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2240780824 learned_addr learned my addr 172.21.15.64:0/2240780824 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d4f40 0x5603228d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2240780824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 msgr2=0x5603228dcf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228dcf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2240780824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d4f40 msgr2=0x5603228d5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d4f40 0x5603228d5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2240780824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032254d3f0 con 0x5603228d5930 2024-05-09T09:11:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.860+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d4f40 0x5603228d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:11:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa49a7fc640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4940a2c60 0x7fa4940a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228da430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4a81017e0 tx=0x7fa49c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa14cc97eeb8be98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa49a7fc640 1 -- 172.21.15.64:0/2295421350 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4a00cf7d0 con 0x7fa4a8102340 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4940a2c60 0x7fa4940a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa4937fe640 1 -- 172.21.15.64:0/2240780824 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c040070 con 0x5603228d5930 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa4937fe640 1 -- 172.21.15.64:0/2240780824 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa49c003ca0 con 0x5603228d5930 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.861+0000 7fa4937fe640 1 -- 172.21.15.64:0/2240780824 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c035040 con 0x5603228d5930 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2240780824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 msgr2=0x5603228da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228da430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4a81017e0 tx=0x7fa49c005fd0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4940a2c60 0x7fa4940a5120 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5603228d6190 tx=0x7fa4a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2240780824 shutdown_connections 2024-05-09T09:11:55.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228dcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228da430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2240780824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d4f40 0x5603228d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.862+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2240780824 >> 172.21.15.64:0/2240780824 conn(0x5603228d05b0 msgr2=0x5603228d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.863+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2240780824 shutdown_connections 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.863+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2240780824 wait complete. 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.863+0000 7fa4b23ff740 1 -- start start 2024-05-09T09:11:55.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.863+0000 7fa4b23ff740 1 Processor -- start 2024-05-09T09:11:55.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228d4f40 0x5603228df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228daaa0 0x5603228e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5603228dda90 con 0x5603228daaa0 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5603227f4c40 con 0x5603228d4f40 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5603228ddc10 con 0x5603228d5930 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.864+0000 7fa4b23ff740 10 client.0 _ll_register_callbacks cb 0x56032254cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228d4f40 0x5603228df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4adbb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228daaa0 0x5603228e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228df8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40356/0 (socket says 172.21.15.64:40356) 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 -- 172.21.15.64:0/2572115881 learned_addr learned my addr 172.21.15.64:0/2572115881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 -- 172.21.15.64:0/2572115881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228d4f40 msgr2=0x5603228df370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228d4f40 0x5603228df370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 -- 172.21.15.64:0/2572115881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228daaa0 msgr2=0x5603228e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228daaa0 0x5603228e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.865+0000 7fa4acbb0640 1 -- 172.21.15.64:0/2572115881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603226177d0 con 0x5603228d5930 2024-05-09T09:11:55.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.866+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228df8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4011c60 tx=0x7fa4a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e54430fa2e9994 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.866+0000 7fa490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:55.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.866+0000 7fa490ff9640 1 -- 172.21.15.64:0/2572115881 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a402d060 con 0x5603228d5930 2024-05-09T09:11:55.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.866+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c003ca0 con 0x5603228d5930 2024-05-09T09:11:55.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.866+0000 7fa490ff9640 1 -- 172.21.15.64:0/2572115881 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4a4028420 con 0x5603228d5930 2024-05-09T09:11:55.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.867+0000 7fa490ff9640 1 -- 172.21.15.64:0/2572115881 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4a4034050 con 0x5603228d5930 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.867+0000 7fa4b23ff740 10 client.25975 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.867+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603228e9ca0 con 0x5603228d5930 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.867+0000 7fa490ff9640 1 -- 172.21.15.64:0/2572115881 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4a4011e90 con 0x5603228d5930 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa490ff9640 1 -- 172.21.15.64:0/2572115881 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fa4a400e790 con 0x5603228d5930 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4b23ff740 10 client.25975 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4b23ff740 10 client.25975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4b23ff740 20 client.25975 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x5603228ed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4b23ff740 4 client.25975 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa49c003ca0 con 0x5603228eb410 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.868+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x5603228ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.869+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x5603228ed840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92f0eb35c20301e1 server_cookie=5de981cba2ec4fe6 in_seq=0 out_seq=0 2024-05-09T09:11:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.869+0000 7fa490ff9640 10 client.25975 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa490ff9640 1 -- 172.21.15.64:0/2572115881 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa49c003ca0 con 0x5603228eb410 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa490ff9640 10 client.25975 handle_command_reply: tid=0 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 client.25975 shutdown 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 msgr2=0x5603228df8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228df8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4011c60 tx=0x7fa4a40046c0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 shutdown_connections 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x5603228ed840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228daaa0 0x5603228e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa490ff9640 0 client.25975 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d5930 0x5603228df8b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2572115881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228d4f40 0x5603228df370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 >> 172.21.15.64:0/2572115881 conn(0x5603228d05b0 msgr2=0x5603228cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 shutdown_connections 2024-05-09T09:11:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.870+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2572115881 wait complete. 2024-05-09T09:11:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.871+0000 7fa4b23ff740 20 client.25975 trim_cache size 0 max 16384 2024-05-09T09:11:55.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.901+0000 7fa4b23ff740 1 Processor -- start 2024-05-09T09:11:55.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.901+0000 7fa4b23ff740 1 -- start start 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.901+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.901+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b3380 con 0x5603228d5930 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b0bc0 con 0x5603228daaa0 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81aea10 con 0x5603228d4f40 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228ddfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40364/0 (socket says 172.21.15.64:40364) 2024-05-09T09:11:55.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/3070715909 learned_addr learned my addr 172.21.15.64:0/3070715909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/3070715909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 msgr2=0x5603228d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/3070715909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 msgr2=0x5603228e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.902+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/3070715909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603228c1d20 con 0x5603228d4f40 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.903+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228ddfb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5603228d63a0 tx=0x7fa49c02a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627eb512cf4a7c1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.903+0000 7fa492ffd640 1 -- 172.21.15.64:0/3070715909 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c055070 con 0x5603228d4f40 2024-05-09T09:11:55.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.903+0000 7fa492ffd640 1 -- 172.21.15.64:0/3070715909 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa49c031ca0 con 0x5603228d4f40 2024-05-09T09:11:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.903+0000 7fa492ffd640 1 -- 172.21.15.64:0/3070715909 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c035040 con 0x5603228d4f40 2024-05-09T09:11:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 -- 172.21.15.64:0/3070715909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 msgr2=0x5603228ddfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228ddfb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5603228d63a0 tx=0x7fa49c02a400 comp rx=0 tx=0).stop 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 -- 172.21.15.64:0/3070715909 shutdown_connections 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/3070715909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228ddfb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.904+0000 7fa4b23ff740 1 -- 172.21.15.64:0/3070715909 >> 172.21.15.64:0/3070715909 conn(0x5603228d05b0 msgr2=0x5603228d6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.905+0000 7fa4b23ff740 1 -- 172.21.15.64:0/3070715909 shutdown_connections 2024-05-09T09:11:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.905+0000 7fa4b23ff740 1 -- 172.21.15.64:0/3070715909 wait complete. 2024-05-09T09:11:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.905+0000 7fa4b23ff740 1 -- start start 2024-05-09T09:11:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.906+0000 7fa4b23ff740 1 Processor -- start 2024-05-09T09:11:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.906+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4b23ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81af080 con 0x5603228d5930 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b04f0 con 0x5603228daaa0 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4b23ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81af750 con 0x5603228d4f40 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4b23ff740 10 client.0 _ll_register_callbacks cb 0x7fa4a4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228e3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40374/0 (socket says 172.21.15.64:40374) 2024-05-09T09:11:55.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2696654057 learned_addr learned my addr 172.21.15.64:0/2696654057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4adbb2640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2696654057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 msgr2=0x5603228e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2696654057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 msgr2=0x5603228e4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.907+0000 7fa4ad3b1640 1 -- 172.21.15.64:0/2696654057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603228e5a20 con 0x5603228d4f40 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa4ad3b1640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228e3ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa49c005ab0 tx=0x7fa49c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3940584d79c9eced server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa490ff9640 1 -- 172.21.15.64:0/2696654057 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c055040 con 0x5603228d4f40 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa490ff9640 1 -- 172.21.15.64:0/2696654057 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa49c0035f0 con 0x5603228d4f40 2024-05-09T09:11:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c031ca0 con 0x5603228d4f40 2024-05-09T09:11:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa490ff9640 1 -- 172.21.15.64:0/2696654057 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa49c035040 con 0x5603228d4f40 2024-05-09T09:11:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa4b23ff740 10 client.25981 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603229519c0 con 0x5603228d4f40 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.908+0000 7fa490ff9640 1 -- 172.21.15.64:0/2696654057 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa49c031e50 con 0x5603228d4f40 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa490ff9640 1 -- 172.21.15.64:0/2696654057 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fa49c045460 con 0x5603228d4f40 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4b23ff740 10 client.25981 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4b23ff740 10 client.25981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4b23ff740 20 client.25981 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x560322953e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4b23ff740 4 client.25981 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa49c031ca0 con 0x5603228eb410 2024-05-09T09:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.909+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x560322953e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa490ff9640 10 client.25981 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4acbb0640 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x560322953e50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99709bdffa5317af server_cookie=a476cfcd56b0ae25 in_seq=0 out_seq=0 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa490ff9640 1 -- 172.21.15.64:0/2696654057 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa49c031ca0 con 0x5603228eb410 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa490ff9640 10 client.25981 handle_command_reply: tid=0 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 client.25981 shutdown 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 msgr2=0x5603228e3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228e3ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa49c005ab0 tx=0x7fa49c004610 comp rx=0 tx=0).stop 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 shutdown_connections 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5603228daaa0 0x5603228e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5603228eb410 0x560322953e50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603228d5930 0x5603228e4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 --2- 172.21.15.64:0/2696654057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603228d4f40 0x5603228e3ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.910+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 >> 172.21.15.64:0/2696654057 conn(0x5603228d05b0 msgr2=0x5603228cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.911+0000 7fa490ff9640 0 client.25981 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.911+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 shutdown_connections 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.911+0000 7fa4b23ff740 1 -- 172.21.15.64:0/2696654057 wait complete. 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.911+0000 7fa4b23ff740 20 client.25981 trim_cache size 0 max 16384 2024-05-09T09:11:55.913 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.912+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4940a2c60 msgr2=0x7fa4940a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.912+0000 7fa4af615640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4940a2c60 0x7fa4940a5120 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5603228d6190 tx=0x7fa4a4005f90 comp rx=0 tx=0).stop 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.912+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 msgr2=0x7fa4a81a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.912+0000 7fa4af615640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 0x7fa4a81a56e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fd0 tx=0x7fa4a000ef10 comp rx=0 tx=0).stop 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.913+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 shutdown_connections 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.913+0000 7fa4af615640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4940a2c60 0x7fa4940a5120 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.913+0000 7fa4af615640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8102340 0x7fa4a81a56e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.913+0000 7fa4af615640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a8101980 0x7fa4a819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.913+0000 7fa4af615640 1 --2- 172.21.15.64:0/2295421350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a8100790 0x7fa4a819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.913+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 >> 172.21.15.64:0/2295421350 conn(0x7fa4a80fbd00 msgr2=0x7fa4a80fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.914+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 shutdown_connections 2024-05-09T09:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:55.914+0000 7fa4af615640 1 -- 172.21.15.64:0/2295421350 wait complete. 2024-05-09T09:11:55.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 -- 172.21.15.64:0/2723139921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809c3b0 msgr2=0x7f396809c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 --2- 172.21.15.64:0/2723139921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809c3b0 0x7f396809c7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f396c009860 tx=0x7f396c02a340 comp rx=0 tx=0).stop 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 -- 172.21.15.64:0/2723139921 shutdown_connections 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 --2- 172.21.15.64:0/2723139921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f396809df10 0x7f39680a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 --2- 172.21.15.64:0/2723139921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809d550 0x7f396809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 --2- 172.21.15.64:0/2723139921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809c3b0 0x7f396809c7d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 -- 172.21.15.64:0/2723139921 >> 172.21.15.64:0/2723139921 conn(0x7f396800b520 msgr2=0x7f396809a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:56.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 -- 172.21.15.64:0/2723139921 shutdown_connections 2024-05-09T09:11:56.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.112+0000 7f39778da640 1 -- 172.21.15.64:0/2723139921 wait complete. 2024-05-09T09:11:56.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.113+0000 7f39778da640 1 Processor -- start 2024-05-09T09:11:56.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.113+0000 7f39778da640 1 -- start start 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.113+0000 7f39778da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 0x7f39681325f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f39778da640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f396809d550 0x7f3968132b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f39778da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809df10 0x7f3968139ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 0x7f39681325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 0x7f39681325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53126/0 (socket says 172.21.15.64:53126) 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f39778da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39680a7e70 con 0x7f396809c3b0 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3974e75640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f396809d550 0x7f3968132b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 -- 172.21.15.64:0/1291409294 learned_addr learned my addr 172.21.15.64:0/1291409294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39680a7cf0 con 0x7f396809df10 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39680a7ff0 con 0x7f396809d550 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975e77640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809df10 0x7f3968139ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 -- 172.21.15.64:0/1291409294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f396809d550 msgr2=0x7f3968132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f396809d550 0x7f3968132b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 -- 172.21.15.64:0/1291409294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809df10 msgr2=0x7f3968139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809df10 0x7f3968139ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.114+0000 7f3975676640 1 -- 172.21.15.64:0/1291409294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396813a210 con 0x7f396809c3b0 2024-05-09T09:11:56.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.115+0000 7f3975676640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 0x7f39681325f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f396c0057c0 tx=0x7f396c030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.115+0000 7f39667fc640 1 -- 172.21.15.64:0/1291409294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c040030 con 0x7f396809c3b0 2024-05-09T09:11:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.115+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396813a4a0 con 0x7f396809c3b0 2024-05-09T09:11:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.115+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396813a9e0 con 0x7f396809c3b0 2024-05-09T09:11:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.116+0000 7f39667fc640 1 -- 172.21.15.64:0/1291409294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f396c038040 con 0x7f396809c3b0 2024-05-09T09:11:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.116+0000 7f39667fc640 1 -- 172.21.15.64:0/1291409294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f396809c3b0 2024-05-09T09:11:56.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.119+0000 7f39667fc640 1 -- 172.21.15.64:0/1291409294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f396c043070 con 0x7f396809c3b0 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.119+0000 7f397a5ed740 1 Processor -- start 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.119+0000 7f397a5ed740 1 -- start start 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91406550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b9140b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b9140e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f397a5ed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b9136ac40 con 0x560b91406130 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f397a5ed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560b91312580 con 0x560b91406b20 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f397a5ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560b91366c00 con 0x560b9140bc90 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f3975e77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91406550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f3975e77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91406550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53138/0 (socket says 172.21.15.64:53138) 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f3975e77640 1 -- 172.21.15.64:0/1663063723 learned_addr learned my addr 172.21.15.64:0/1663063723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:56.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f3974e75640 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b9140e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.120+0000 7f3975676640 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b9140b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.121+0000 7f3975e77640 1 -- 172.21.15.64:0/1663063723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 msgr2=0x560b9140e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.121+0000 7f3975e77640 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b9140e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.121+0000 7f3975e77640 1 -- 172.21.15.64:0/1663063723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 msgr2=0x560b9140b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.121+0000 7f3975e77640 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b9140b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.121+0000 7f3975e77640 1 -- 172.21.15.64:0/1663063723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9107e3f0 con 0x560b91406130 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.121+0000 7f3975e77640 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91406550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3970064d50 tx=0x7f3970050670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ed364be15af237 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.122+0000 7f39667fc640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f395c0a2c10 0x7f395c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.122+0000 7f3975676640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f395c0a2c10 0x7f395c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.122+0000 7f395b7fe640 1 -- 172.21.15.64:0/1663063723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970086070 con 0x560b91406130 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.122+0000 7f395b7fe640 1 -- 172.21.15.64:0/1663063723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3970081af0 con 0x560b91406130 2024-05-09T09:11:56.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.122+0000 7f395b7fe640 1 -- 172.21.15.64:0/1663063723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700808d0 con 0x560b91406130 2024-05-09T09:11:56.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.122+0000 7f39667fc640 1 -- 172.21.15.64:0/1291409294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f396c0eedc0 con 0x7f396809c3b0 2024-05-09T09:11:56.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 -- 172.21.15.64:0/1663063723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 msgr2=0x560b91406550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91406550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3970064d50 tx=0x7f3970050670 comp rx=0 tx=0).stop 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f3975676640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f395c0a2c10 0x7f395c0a50d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f396c0ec8a0 tx=0x7f396c0f28a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 -- 172.21.15.64:0/1663063723 shutdown_connections 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b9140e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b9140b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 --2- 172.21.15.64:0/1663063723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91406550 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.123+0000 7f397a5ed740 1 -- 172.21.15.64:0/1663063723 >> 172.21.15.64:0/1663063723 conn(0x560b914017a0 msgr2=0x560b91401ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.124+0000 7f397a5ed740 1 -- 172.21.15.64:0/1663063723 shutdown_connections 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.124+0000 7f397a5ed740 1 -- 172.21.15.64:0/1663063723 wait complete. 2024-05-09T09:11:56.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.124+0000 7f397a5ed740 1 -- start start 2024-05-09T09:11:56.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.124+0000 7f397a5ed740 1 Processor -- start 2024-05-09T09:11:56.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91410520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b91410a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b91417aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b9140ec40 con 0x560b91406130 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560b91325c40 con 0x560b91406b20 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560b9140edc0 con 0x560b9140bc90 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f397a5ed740 10 client.0 _ll_register_callbacks cb 0x560b9107dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f3975e77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b91417aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f3975e77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b91417aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40396/0 (socket says 172.21.15.64:40396) 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f3975e77640 1 -- 172.21.15.64:0/3585893928 learned_addr learned my addr 172.21.15.64:0/3585893928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f3975e77640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91410520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.125+0000 7f3974e75640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b91410a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3975e77640 1 -- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 msgr2=0x560b91410a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3975e77640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b91410a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3975e77640 1 -- 172.21.15.64:0/3585893928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 msgr2=0x560b91410520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3975e77640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91410520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3975e77640 1 -- 172.21.15.64:0/3585893928 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b911487d0 con 0x560b9140bc90 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3975e77640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b91417aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3970064c00 tx=0x7f3970066fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54763e1cec30100 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3958ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f3958ff9640 1 -- 172.21.15.64:0/3585893928 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970086040 con 0x560b9140bc90 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.126+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3970081af0 con 0x560b9140bc90 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.127+0000 7f3958ff9640 1 -- 172.21.15.64:0/3585893928 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3970080ae0 con 0x560b9140bc90 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.127+0000 7f3958ff9640 1 -- 172.21.15.64:0/3585893928 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3970098050 con 0x560b9140bc90 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.127+0000 7f397a5ed740 10 client.25993 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.127+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b9141ae60 con 0x560b9140bc90 2024-05-09T09:11:56.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.127+0000 7f3958ff9640 1 -- 172.21.15.64:0/3585893928 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3970081af0 con 0x560b9140bc90 2024-05-09T09:11:56.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f3958ff9640 1 -- 172.21.15.64:0/3585893928 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f397008c040 con 0x560b9140bc90 2024-05-09T09:11:56.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f397a5ed740 10 client.25993 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:56.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f397a5ed740 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f397a5ed740 20 client.25993 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b9141c820 0x560b9141ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f397a5ed740 4 client.25993 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b91378370 con 0x560b9141c820 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.128+0000 7f3974e75640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b9141c820 0x560b9141ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.129+0000 7f3958ff9640 10 client.25993 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.129+0000 7f3974e75640 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b9141c820 0x560b9141ec60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc9c7cd8994b7b7 server_cookie=997fb1a1b6d1a7d3 in_seq=0 out_seq=0 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.129+0000 7f3958ff9640 1 -- 172.21.15.64:0/3585893928 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b91378370 con 0x560b9141c820 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f3958ff9640 10 client.25993 handle_command_reply: tid=0 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 client.25993 shutdown 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 msgr2=0x560b91417aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b91417aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3970064c00 tx=0x7f3970066fc0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 shutdown_connections 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b9141c820 0x560b9141ec60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b9140bc90 0x560b91417aa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406b20 0x560b91410a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3585893928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406130 0x560b91410520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 >> 172.21.15.64:0/3585893928 conn(0x560b914017a0 msgr2=0x560b914009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f3958ff9640 0 client.25993 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 shutdown_connections 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.130+0000 7f397a5ed740 1 -- 172.21.15.64:0/3585893928 wait complete. 2024-05-09T09:11:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.131+0000 7f397a5ed740 20 client.25993 trim_cache size 0 max 16384 2024-05-09T09:11:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.154+0000 7f397a5ed740 1 Processor -- start 2024-05-09T09:11:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.154+0000 7f397a5ed740 1 -- start start 2024-05-09T09:11:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 0x560b91400130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406b20 0x560b914179c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f3975e77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 0x560b91400130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f3975e77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 0x560b91400130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40400/0 (socket says 172.21.15.64:40400) 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b9140bc90 0x560b91400670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f3975e77640 1 -- 172.21.15.64:0/3295133338 learned_addr learned my addr 172.21.15.64:0/3295133338 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f3974e75640 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406b20 0x560b914179c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3968147810 con 0x560b91406b20 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3968145050 con 0x560b9140bc90 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3968142ea0 con 0x560b91406130 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.155+0000 7f3975e77640 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b9140bc90 0x560b91400670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f3975e77640 1 -- 172.21.15.64:0/3295133338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b9140bc90 msgr2=0x560b91400670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f3975e77640 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b9140bc90 0x560b91400670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f3975e77640 1 -- 172.21.15.64:0/3295133338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406b20 msgr2=0x560b914179c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f3975e77640 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406b20 0x560b914179c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f3975e77640 1 -- 172.21.15.64:0/3295133338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b91402510 con 0x560b91406130 2024-05-09T09:11:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f3975e77640 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 0x560b91400130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39700503a0 tx=0x7f3970050670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f876a7880a0ad64f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f395affd640 1 -- 172.21.15.64:0/3295133338 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970086040 con 0x560b91406130 2024-05-09T09:11:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f395affd640 1 -- 172.21.15.64:0/3295133338 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397004fe40 con 0x560b91406130 2024-05-09T09:11:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.156+0000 7f395affd640 1 -- 172.21.15.64:0/3295133338 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970080ae0 con 0x560b91406130 2024-05-09T09:11:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.157+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 msgr2=0x560b91400130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.157+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 0x560b91400130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39700503a0 tx=0x7f3970050670 comp rx=0 tx=0).stop 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.157+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 shutdown_connections 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.158+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b9140bc90 0x560b91400670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.158+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b91406b20 0x560b914179c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.158+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3295133338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406130 0x560b91400130 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.158+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 >> 172.21.15.64:0/3295133338 conn(0x560b914017a0 msgr2=0x560b91407ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.158+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 shutdown_connections 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.158+0000 7f397a5ed740 1 -- 172.21.15.64:0/3295133338 wait complete. 2024-05-09T09:11:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.159+0000 7f397a5ed740 1 -- start start 2024-05-09T09:11:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.159+0000 7f397a5ed740 1 Processor -- start 2024-05-09T09:11:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406130 0x560b914148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 0x560b91414de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b9140bc90 0x560b91480d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3968143510 con 0x560b9140bc90 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3968144980 con 0x560b91406130 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3968143be0 con 0x560b91406b20 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f397a5ed740 10 client.0 _ll_register_callbacks cb 0x7f3970084070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f3974e75640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 0x560b91414de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f3974e75640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 0x560b91414de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40404/0 (socket says 172.21.15.64:40404) 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f3974e75640 1 -- 172.21.15.64:0/3005384234 learned_addr learned my addr 172.21.15.64:0/3005384234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f3975e77640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406130 0x560b914148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.160+0000 7f3975e77640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b9140bc90 0x560b91480d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3974e75640 1 -- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406130 msgr2=0x560b914148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3974e75640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406130 0x560b914148a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3974e75640 1 -- 172.21.15.64:0/3005384234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b9140bc90 msgr2=0x560b91480d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:56.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3974e75640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b9140bc90 0x560b91480d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3974e75640 1 -- 172.21.15.64:0/3005384234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9140b970 con 0x560b91406b20 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3974e75640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 0x560b91414de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x560b91402710 tx=0x7f3960010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d8c1905db74771 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3958ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3958ff9640 1 -- 172.21.15.64:0/3005384234 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960004220 con 0x560b91406b20 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f3958ff9640 1 -- 172.21.15.64:0/3005384234 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f39600043c0 con 0x560b91406b20 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.161+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397004fe40 con 0x560b91406b20 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f3958ff9640 1 -- 172.21.15.64:0/3005384234 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f396002f050 con 0x560b91406b20 2024-05-09T09:11:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 10 client.25999 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b91482360 con 0x560b91406b20 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f3958ff9640 1 -- 172.21.15.64:0/3005384234 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397004fe40 con 0x560b91406b20 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f3958ff9640 1 -- 172.21.15.64:0/3005384234 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f39600047a0 con 0x560b91406b20 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 10 client.25999 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 20 client.25999 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b91482d80 0x560b914851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 4 client.25999 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.162+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b913726c0 con 0x560b91482d80 2024-05-09T09:11:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.163+0000 7f3975e77640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b91482d80 0x560b914851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.163+0000 7f3975e77640 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b91482d80 0x560b914851a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce91a1c03be2efd server_cookie=a9e35058edd5802e in_seq=0 out_seq=0 2024-05-09T09:11:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.163+0000 7f3958ff9640 10 client.25999 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.163+0000 7f3958ff9640 1 -- 172.21.15.64:0/3005384234 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b913726c0 con 0x560b91482d80 2024-05-09T09:11:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.163+0000 7f3958ff9640 10 client.25999 handle_command_reply: tid=0 2024-05-09T09:11:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.163+0000 7f397a5ed740 1 client.25999 shutdown 2024-05-09T09:11:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 msgr2=0x560b91414de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 0x560b91414de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x560b91402710 tx=0x7f3960010800 comp rx=0 tx=0).stop 2024-05-09T09:11:56.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 shutdown_connections 2024-05-09T09:11:56.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b91482d80 0x560b914851a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f3958ff9640 0 client.25999 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:56.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b9140bc90 0x560b91480d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b91406b20 0x560b91414de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 --2- 172.21.15.64:0/3005384234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b91406130 0x560b914148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 >> 172.21.15.64:0/3005384234 conn(0x560b914017a0 msgr2=0x560b91404750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 shutdown_connections 2024-05-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 1 -- 172.21.15.64:0/3005384234 wait complete. 2024-05-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.164+0000 7f397a5ed740 20 client.25999 trim_cache size 0 max 16384 2024-05-09T09:11:56.167 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.165+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f395c0a2c10 msgr2=0x7f395c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.165+0000 7f39778da640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f395c0a2c10 0x7f395c0a50d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f396c0ec8a0 tx=0x7f396c0f28a0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.165+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 msgr2=0x7f39681325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.165+0000 7f39778da640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 0x7f39681325f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f396c0057c0 tx=0x7f396c030e00 comp rx=0 tx=0).stop 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.165+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 shutdown_connections 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.165+0000 7f39778da640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f395c0a2c10 0x7f395c0a50d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.166+0000 7f39778da640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f396809df10 0x7f3968139ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.166+0000 7f39778da640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f396809d550 0x7f3968132b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.166+0000 7f39778da640 1 --2- 172.21.15.64:0/1291409294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f396809c3b0 0x7f39681325f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.166+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 >> 172.21.15.64:0/1291409294 conn(0x7f396800b520 msgr2=0x7f39680a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.166+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 shutdown_connections 2024-05-09T09:11:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:56.166+0000 7f39778da640 1 -- 172.21.15.64:0/1291409294 wait complete. 2024-05-09T09:11:56.181 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:55 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3265031355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:11:56.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:55 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3265031355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:11:56.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:55 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3265031355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:11:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 95.0 s, 1014.0 tps, lat 98.718 ms stddev 123.548 2024-05-09T09:11:57.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:11:57.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:11:57.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:11:57.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:56 smithi179 ceph-mon[34086]: pgmap v454: 129 pgs: 129 active+clean; 976 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:11:57.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:56 smithi064 ceph-mon[30149]: pgmap v454: 129 pgs: 129 active+clean; 976 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:11:57.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:56 smithi139 ceph-mon[33599]: pgmap v454: 129 pgs: 129 active+clean; 976 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:11:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.396+0000 7f9a80509640 1 -- 172.21.15.64:0/861358631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a78108830 msgr2=0x7f9a7810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.396+0000 7f9a80509640 1 --2- 172.21.15.64:0/861358631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a78108830 0x7f9a7810ac60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009900 tx=0x7f9a7402a410 comp rx=0 tx=0).stop 2024-05-09T09:11:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.397+0000 7f9a80509640 1 -- 172.21.15.64:0/861358631 shutdown_connections 2024-05-09T09:11:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.397+0000 7f9a80509640 1 --2- 172.21.15.64:0/861358631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a7810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.397+0000 7f9a80509640 1 --2- 172.21.15.64:0/861358631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a7810b1a0 0x7f9a7810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.397+0000 7f9a80509640 1 --2- 172.21.15.64:0/861358631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a78108830 0x7f9a7810ac60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.397+0000 7f9a80509640 1 -- 172.21.15.64:0/861358631 >> 172.21.15.64:0/861358631 conn(0x7f9a7806e6c0 msgr2=0x7f9a78070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.399+0000 7f9a80509640 1 -- 172.21.15.64:0/861358631 shutdown_connections 2024-05-09T09:11:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.399+0000 7f9a80509640 1 -- 172.21.15.64:0/861358631 wait complete. 2024-05-09T09:11:57.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.400+0000 7f9a80509640 1 Processor -- start 2024-05-09T09:11:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 -- start start 2024-05-09T09:11:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a78108830 0x7f9a781a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a7810b1a0 0x7f9a781a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a781add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7811c270 con 0x7f9a7810b1a0 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7811c0f0 con 0x7f9a78108830 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.401+0000 7f9a80509640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7811c3f0 con 0x7f9a7810db70 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.402+0000 7f9a7eaa6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a781add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.402+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a78108830 0x7f9a781a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.402+0000 7f9a7eaa6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a781add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40416/0 (socket says 172.21.15.64:40416) 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.402+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a78108830 0x7f9a781a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59948/0 (socket says 172.21.15.64:59948) 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.402+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/266719886 learned_addr learned my addr 172.21.15.64:0/266719886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.402+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a7810b1a0 0x7f9a781a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.403+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/266719886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a78108830 msgr2=0x7f9a781a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.403+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a78108830 0x7f9a781a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.403+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a7810b1a0 msgr2=0x7f9a781a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.403+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a7810b1a0 0x7f9a781a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.403+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/266719886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a781ae380 con 0x7f9a7810db70 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.403+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a781add10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009f10 tx=0x7f9a6800ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.404+0000 7f9a6f7fe640 1 -- 172.21.15.64:0/266719886 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800ccc0 con 0x7f9a7810db70 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.404+0000 7f9a6f7fe640 1 -- 172.21.15.64:0/266719886 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a680054d0 con 0x7f9a7810db70 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.404+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7810ffc0 con 0x7f9a7810db70 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.404+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a78110540 con 0x7f9a7810db70 2024-05-09T09:11:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.405+0000 7f9a6f7fe640 1 -- 172.21.15.64:0/266719886 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800ccc0 con 0x7f9a7810db70 2024-05-09T09:11:57.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.408+0000 7f9a6f7fe640 1 -- 172.21.15.64:0/266719886 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9a68004cf0 con 0x7f9a7810db70 2024-05-09T09:11:57.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.409+0000 7f9a6f7fe640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a640a3050 0x7f9a640a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.409+0000 7f9a6f7fe640 1 -- 172.21.15.64:0/266719886 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a6800cac0 con 0x7f9a7810db70 2024-05-09T09:11:57.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.410+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a640a3050 0x7f9a640a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.411+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a640a3050 0x7f9a640a5510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a740096c0 tx=0x7f9a74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.413+0000 7f9a8320d740 1 Processor -- start 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.414+0000 7f9a8320d740 1 -- start start 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.414+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 0x5615e53888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e5388eb0 0x5615e538d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e53904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a8320d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5615e52ec990 con 0x5615e53884c0 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a8320d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5615e52945b0 con 0x5615e5388eb0 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a8320d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5615e52e8950 con 0x5615e538e020 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 0x5615e53888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 0x5615e53888e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53186/0 (socket says 172.21.15.64:53186) 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/2498594755 learned_addr learned my addr 172.21.15.64:0/2498594755 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e53904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e5388eb0 0x5615e538d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/2498594755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e5388eb0 msgr2=0x5615e538d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e5388eb0 0x5615e538d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/2498594755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 msgr2=0x5615e53888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 0x5615e53888e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/2498594755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e50c87e0 con 0x5615e538e020 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.415+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 0x5615e53888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:57.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.416+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e53904b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a680dd890 tx=0x7f9a68097170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbca49c96c4adfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.416+0000 7f9a6cbfe640 1 -- 172.21.15.64:0/2498594755 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680c4020 con 0x5615e538e020 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.416+0000 7f9a6cbfe640 1 -- 172.21.15.64:0/2498594755 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a68005680 con 0x5615e538e020 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.416+0000 7f9a6cbfe640 1 -- 172.21.15.64:0/2498594755 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800ccc0 con 0x5615e538e020 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 -- 172.21.15.64:0/2498594755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 msgr2=0x5615e53904b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e53904b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a680dd890 tx=0x7f9a68097170 comp rx=0 tx=0).stop 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 -- 172.21.15.64:0/2498594755 shutdown_connections 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e53904b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e5388eb0 0x5615e538d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2498594755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e53884c0 0x5615e53888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.417+0000 7f9a8320d740 1 -- 172.21.15.64:0/2498594755 >> 172.21.15.64:0/2498594755 conn(0x5615e53500a0 msgr2=0x5615e5383f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.418+0000 7f9a8320d740 1 -- 172.21.15.64:0/2498594755 shutdown_connections 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.418+0000 7f9a8320d740 1 -- 172.21.15.64:0/2498594755 wait complete. 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.418+0000 7f9a8320d740 1 -- start start 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.419+0000 7f9a8320d740 1 Processor -- start 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.419+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 0x5615e53927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.419+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5392d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 0x5615e53927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 0x5615e53927d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40428/0 (socket says 172.21.15.64:40428) 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a7daa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5392d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/1938784411 learned_addr learned my addr 172.21.15.64:0/1938784411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e538e020 0x5615e5399d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5615e5390fd0 con 0x5615e5388eb0 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e538e020 0x5615e5399d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5615e52a7b40 con 0x5615e538e020 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.420+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5615e5391150 con 0x5615e53884c0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a8320d740 10 client.0 _ll_register_callbacks cb 0x5615e529c0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e538e020 msgr2=0x5615e5399d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e538e020 0x5615e5399d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/1938784411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 msgr2=0x5615e5392d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5392d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e5128f70 con 0x5615e53884c0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 0x5615e53927d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a7402aa00 tx=0x7f9a74005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa274ea26cbe884 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.421+0000 7f9a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.422+0000 7f9a61ffb640 1 -- 172.21.15.64:0/1938784411 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74035040 con 0x5615e53884c0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.422+0000 7f9a61ffb640 1 -- 172.21.15.64:0/1938784411 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a74004b30 con 0x5615e53884c0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.422+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a68005680 con 0x5615e53884c0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.423+0000 7f9a61ffb640 1 -- 172.21.15.64:0/1938784411 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a74004d30 con 0x5615e53884c0 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.423+0000 7f9a8320d740 10 client.26011 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:57.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.423+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615e4fc0170 con 0x5615e53884c0 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.423+0000 7f9a61ffb640 1 -- 172.21.15.64:0/1938784411 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a74003aa0 con 0x5615e53884c0 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a61ffb640 1 -- 172.21.15.64:0/1938784411 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9a74032760 con 0x5615e53884c0 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a8320d740 10 client.26011 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a8320d740 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a8320d740 20 client.26011 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e53a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a8320d740 4 client.26011 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a68005680 con 0x5615e539e360 2024-05-09T09:11:57.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.424+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e53a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.425+0000 7f9a61ffb640 10 client.26011 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.425+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e53a07e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11e63719fae85c4 server_cookie=e86df81b75c307c in_seq=0 out_seq=0 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.426+0000 7f9a61ffb640 1 -- 172.21.15.64:0/1938784411 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f9a68005680 con 0x5615e539e360 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.426+0000 7f9a61ffb640 10 client.26011 handle_command_reply: tid=0 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.426+0000 7f9a8320d740 1 client.26011 shutdown 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 msgr2=0x5615e53927d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 0x5615e53927d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a7402aa00 tx=0x7f9a74005fd0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 shutdown_connections 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e53a07e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e538e020 0x5615e5399d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a61ffb640 0 client.26011 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5392d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.427+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1938784411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e53884c0 0x5615e53927d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.428+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 >> 172.21.15.64:0/1938784411 conn(0x5615e53500a0 msgr2=0x5615e5383080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.428+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 shutdown_connections 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.428+0000 7f9a8320d740 1 -- 172.21.15.64:0/1938784411 wait complete. 2024-05-09T09:11:57.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.428+0000 7f9a8320d740 20 client.26011 trim_cache size 0 max 16384 2024-05-09T09:11:57.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 Processor -- start 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 -- start start 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5392a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e5388eb0 0x5615e5399ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e538e020 0x5615e5387540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b3220 con 0x5615e538e020 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b0a60 con 0x5615e53884c0 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.469+0000 7f9a8320d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781ae8d0 con 0x5615e5388eb0 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7eaa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e538e020 0x5615e5387540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7eaa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e538e020 0x5615e5387540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53202/0 (socket says 172.21.15.64:53202) 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7eaa6640 1 -- 172.21.15.64:0/1277196799 learned_addr learned my addr 172.21.15.64:0/1277196799 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e5388eb0 0x5615e5399ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5392a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 -- 172.21.15.64:0/1277196799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 msgr2=0x5615e5392a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5392a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 -- 172.21.15.64:0/1277196799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e538e020 msgr2=0x5615e5387540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e538e020 0x5615e5387540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 -- 172.21.15.64:0/1277196799 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e5384bf0 con 0x5615e5388eb0 2024-05-09T09:11:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.470+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e5388eb0 0x5615e5399ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5615e5384df0 tx=0x7f9a700096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92780959437de228 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.471+0000 7f9a63fff640 1 -- 172.21.15.64:0/1277196799 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a700178a0 con 0x5615e5388eb0 2024-05-09T09:11:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.471+0000 7f9a63fff640 1 -- 172.21.15.64:0/1277196799 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a70021030 con 0x5615e5388eb0 2024-05-09T09:11:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.471+0000 7f9a63fff640 1 -- 172.21.15.64:0/1277196799 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70025c50 con 0x5615e5388eb0 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 -- 172.21.15.64:0/1277196799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e5388eb0 msgr2=0x5615e5399ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e5388eb0 0x5615e5399ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5615e5384df0 tx=0x7f9a700096d0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 -- 172.21.15.64:0/1277196799 shutdown_connections 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e538e020 0x5615e5387540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e5388eb0 0x5615e5399ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 --2- 172.21.15.64:0/1277196799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5392a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.472+0000 7f9a8320d740 1 -- 172.21.15.64:0/1277196799 >> 172.21.15.64:0/1277196799 conn(0x5615e53500a0 msgr2=0x5615e5386e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.473+0000 7f9a8320d740 1 -- 172.21.15.64:0/1277196799 shutdown_connections 2024-05-09T09:11:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.473+0000 7f9a8320d740 1 -- 172.21.15.64:0/1277196799 wait complete. 2024-05-09T09:11:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.473+0000 7f9a8320d740 1 -- start start 2024-05-09T09:11:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.474+0000 7f9a8320d740 1 Processor -- start 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.474+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5399a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.474+0000 7f9a8320d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5398030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5399a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a7e2a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5399a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59978/0 (socket says 172.21.15.64:59978) 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/2274732301 learned_addr learned my addr 172.21.15.64:0/2274732301 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e5398570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781aef20 con 0x5615e5388eb0 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a7eaa6640 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e5398570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b0390 con 0x5615e53884c0 2024-05-09T09:11:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.475+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781af5f0 con 0x5615e538e020 2024-05-09T09:11:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a8320d740 10 client.0 _ll_register_callbacks cb 0x7f9a74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/2274732301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 msgr2=0x5615e5398570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e5398570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/2274732301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 msgr2=0x5615e5398030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5398030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a7e2a5640 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e5398ab0 con 0x5615e53884c0 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.476+0000 7f9a7e2a5640 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5399a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9a740058c0 tx=0x7f9a74032850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6581d33bc6a03210 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.477+0000 7f9a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.477+0000 7f9a61ffb640 1 -- 172.21.15.64:0/2274732301 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74035070 con 0x5615e53884c0 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.477+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a70021030 con 0x5615e53884c0 2024-05-09T09:11:57.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.477+0000 7f9a61ffb640 1 -- 172.21.15.64:0/2274732301 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a7403ca40 con 0x5615e53884c0 2024-05-09T09:11:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.477+0000 7f9a61ffb640 1 -- 172.21.15.64:0/2274732301 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a7406e050 con 0x5615e53884c0 2024-05-09T09:11:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.477+0000 7f9a8320d740 10 client.25868 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:57.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.478+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5615e54055b0 con 0x5615e53884c0 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a61ffb640 1 -- 172.21.15.64:0/2274732301 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9a7403cc40 con 0x5615e53884c0 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a8320d740 10 client.25868 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a8320d740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a8320d740 20 client.25868 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e5407df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a8320d740 4 client.25868 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:11:57.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.479+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f9a70021030 con 0x5615e539e360 2024-05-09T09:11:57.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.480+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e5407df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.482+0000 7f9a7daa4640 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e5407df0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31a041ff5a694a server_cookie=fbffa59d29a94ca7 in_seq=0 out_seq=0 2024-05-09T09:11:57.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.482+0000 7f9a61ffb640 10 client.25868 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.485+0000 7f9a61ffb640 1 -- 172.21.15.64:0/2274732301 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9a70021030 con 0x5615e539e360 2024-05-09T09:11:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.485+0000 7f9a61ffb640 10 client.25868 handle_command_reply: tid=0 2024-05-09T09:11:57.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.485+0000 7f9a8320d740 1 client.25868 shutdown 2024-05-09T09:11:57.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 msgr2=0x5615e5399a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5399a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9a740058c0 tx=0x7f9a74032850 comp rx=0 tx=0).stop 2024-05-09T09:11:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 shutdown_connections 2024-05-09T09:11:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615e539e360 0x5615e5407df0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615e538e020 0x5615e5398570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a61ffb640 0 client.25868 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5388eb0 0x5615e5398030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 --2- 172.21.15.64:0/2274732301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615e53884c0 0x5615e5399a20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.486+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 >> 172.21.15.64:0/2274732301 conn(0x5615e53500a0 msgr2=0x5615e53827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.487+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 shutdown_connections 2024-05-09T09:11:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.487+0000 7f9a8320d740 1 -- 172.21.15.64:0/2274732301 wait complete. 2024-05-09T09:11:57.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.488+0000 7f9a8320d740 20 client.25868 trim_cache size 0 max 16384 2024-05-09T09:11:57.487 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:11:57.487 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:11:57.487 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "7bf9ef14-4e50-41c5-92ef-4aefa1022887", 2024-05-09T09:11:57.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:11:57.487 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:11:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a640a3050 msgr2=0x7f9a640a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a640a3050 0x7f9a640a5510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a740096c0 tx=0x7f9a74000bc0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 msgr2=0x7f9a781add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a781add10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009f10 tx=0x7f9a6800ed10 comp rx=0 tx=0).stop 2024-05-09T09:11:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 shutdown_connections 2024-05-09T09:11:57.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a640a3050 0x7f9a640a5510 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a740096c0 tx=0x7f9a74000bc0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a7810db70 0x7f9a781add10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a7810b1a0 0x7f9a781a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 --2- 172.21.15.64:0/266719886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a78108830 0x7f9a781a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.489+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 >> 172.21.15.64:0/266719886 conn(0x7f9a7806e6c0 msgr2=0x7f9a7810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.491+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 shutdown_connections 2024-05-09T09:11:57.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.491+0000 7f9a80509640 1 -- 172.21.15.64:0/266719886 wait complete. 2024-05-09T09:11:57.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:11:57.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:11:57.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 -- 172.21.15.64:0/408573792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f68101150 msgr2=0x7f3f681059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/408573792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f68101150 0x7f3f681059f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009be0 tx=0x7f3f6002a6f0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 -- 172.21.15.64:0/408573792 shutdown_connections 2024-05-09T09:11:57.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/408573792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f68101150 0x7f3f681059f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/408573792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68100790 0x7f3f68100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/408573792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f680ffa30 0x7f3f680ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.748+0000 7f3f6e95f640 1 -- 172.21.15.64:0/408573792 >> 172.21.15.64:0/408573792 conn(0x7f3f680fb360 msgr2=0x7f3f680fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.749+0000 7f3f6e95f640 1 -- 172.21.15.64:0/408573792 shutdown_connections 2024-05-09T09:11:57.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.749+0000 7f3f6e95f640 1 -- 172.21.15.64:0/408573792 wait complete. 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 Processor -- start 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 -- start start 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f680ffa30 0x7f3f680714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f68100790 0x7f3f68071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 0x7f3f68072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6806bb30 con 0x7f3f680ffa30 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6806b9b0 con 0x7f3f68101150 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.750+0000 7f3f6e95f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6806bcb0 con 0x7f3f68100790 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 0x7f3f68072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6d95d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f680ffa30 0x7f3f680714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 0x7f3f68072a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59998/0 (socket says 172.21.15.64:59998) 2024-05-09T09:11:57.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6d95d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f680ffa30 0x7f3f680714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53228/0 (socket says 172.21.15.64:53228) 2024-05-09T09:11:57.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 -- 172.21.15.64:0/2552398711 learned_addr learned my addr 172.21.15.64:0/2552398711 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f68100790 0x7f3f68071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 -- 172.21.15.64:0/2552398711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f68100790 msgr2=0x7f3f68071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f68100790 0x7f3f68071a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 -- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f680ffa30 msgr2=0x7f3f680714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f680ffa30 0x7f3f680714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.751+0000 7f3f6e15e640 1 -- 172.21.15.64:0/2552398711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f68073090 con 0x7f3f68101150 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.752+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 0x7f3f68072a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f6002ac10 tx=0x7f3f60030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.752+0000 7f3f5effd640 1 -- 172.21.15.64:0/2552398711 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60041020 con 0x7f3f68101150 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.752+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f68073320 con 0x7f3f68101150 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.752+0000 7f3f5effd640 1 -- 172.21.15.64:0/2552398711 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f60037040 con 0x7f3f68101150 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.752+0000 7f3f5effd640 1 -- 172.21.15.64:0/2552398711 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60036070 con 0x7f3f68101150 2024-05-09T09:11:57.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.752+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f68073860 con 0x7f3f68101150 2024-05-09T09:11:57.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.757+0000 7f3f729aa740 1 Processor -- start 2024-05-09T09:11:57.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.759+0000 7f3f5effd640 1 -- 172.21.15.64:0/2552398711 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f60044020 con 0x7f3f68101150 2024-05-09T09:11:57.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.760+0000 7f3f5effd640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3f540a2c90 0x7f3f540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.760+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3f540a2c90 0x7f3f540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.761+0000 7f3f5effd640 1 -- 172.21.15.64:0/2552398711 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f600f01d0 con 0x7f3f68101150 2024-05-09T09:11:57.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 -- start start 2024-05-09T09:11:57.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15f7120 0x55a1a15f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 0x55a1a15fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a15ff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a1a155bd00 con 0x55a1a15f7120 2024-05-09T09:11:57.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a1a15034c0 con 0x55a1a15f7b10 2024-05-09T09:11:57.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f729aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1557cc0 con 0x55a1a15fcc80 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.762+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3f540a2c90 0x7f3f540a5150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f680729b0 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 0x55a1a15fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 0x55a1a15fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60000/0 (socket says 172.21.15.64:60000) 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 -- 172.21.15.64:0/4293427683 learned_addr learned my addr 172.21.15.64:0/4293427683 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15f7120 0x55a1a15f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 -- 172.21.15.64:0/4293427683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 msgr2=0x55a1a15ff110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a15ff110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 -- 172.21.15.64:0/4293427683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15f7120 msgr2=0x55a1a15f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15f7120 0x55a1a15f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.763+0000 7f3f6d15c640 1 -- 172.21.15.64:0/4293427683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a13397d0 con 0x55a1a15f7b10 2024-05-09T09:11:57.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.764+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 0x55a1a15fc610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400caf0 tx=0x7f3f6400f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e5f4e266c8ba19e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.764+0000 7f3f53fff640 1 -- 172.21.15.64:0/4293427683 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64019040 con 0x55a1a15f7b10 2024-05-09T09:11:57.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.764+0000 7f3f53fff640 1 -- 172.21.15.64:0/4293427683 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f64022070 con 0x55a1a15f7b10 2024-05-09T09:11:57.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.764+0000 7f3f53fff640 1 -- 172.21.15.64:0/4293427683 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f640113d0 con 0x55a1a15f7b10 2024-05-09T09:11:57.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.765+0000 7f3f729aa740 1 -- 172.21.15.64:0/4293427683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 msgr2=0x55a1a15fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 0x55a1a15fc610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400caf0 tx=0x7f3f6400f000 comp rx=0 tx=0).stop 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 -- 172.21.15.64:0/4293427683 shutdown_connections 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a15ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7b10 0x55a1a15fc610 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400caf0 tx=0x7f3f6400f000 comp rx=0 tx=0).stop 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 --2- 172.21.15.64:0/4293427683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15f7120 0x55a1a15f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 -- 172.21.15.64:0/4293427683 >> 172.21.15.64:0/4293427683 conn(0x55a1a15f2790 msgr2=0x55a1a15f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.766+0000 7f3f729aa740 1 -- 172.21.15.64:0/4293427683 shutdown_connections 2024-05-09T09:11:57.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.767+0000 7f3f729aa740 1 -- 172.21.15.64:0/4293427683 wait complete. 2024-05-09T09:11:57.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.768+0000 7f3f729aa740 1 -- start start 2024-05-09T09:11:57.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.768+0000 7f3f729aa740 1 Processor -- start 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a1601870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a1601db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a16091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a1a15ffc30 con 0x55a1a15fcc80 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1516b90 con 0x55a1a1606de0 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a1a15ffdb0 con 0x55a1a15f7120 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f6d15c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a1601db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f6d15c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a1601db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53244/0 (socket says 172.21.15.64:53244) 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f6d15c640 1 -- 172.21.15.64:0/3531761988 learned_addr learned my addr 172.21.15.64:0/3531761988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f729aa740 10 client.0 _ll_register_callbacks cb 0x55a1a1552da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a1601870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.769+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a16091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 msgr2=0x55a1a16091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a16091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3531761988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 msgr2=0x55a1a1601db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a1601db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3531761988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a126f3f0 con 0x55a1a15f7120 2024-05-09T09:11:57.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a1601870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a1a15f8370 tx=0x7f3f58012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd23d62482f469d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.770+0000 7f3f517fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:11:57.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.771+0000 7f3f517fa640 1 -- 172.21.15.64:0/3531761988 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58004050 con 0x55a1a15f7120 2024-05-09T09:11:57.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.771+0000 7f3f517fa640 1 -- 172.21.15.64:0/3531761988 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f580041f0 con 0x55a1a15f7120 2024-05-09T09:11:57.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.771+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f64022070 con 0x55a1a15f7120 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f517fa640 1 -- 172.21.15.64:0/3531761988 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f580043f0 con 0x55a1a15f7120 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f729aa740 10 client.26023 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1a122f0f0 con 0x55a1a15f7120 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f517fa640 1 -- 172.21.15.64:0/3531761988 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f5800fd40 con 0x55a1a15f7120 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f517fa640 1 -- 172.21.15.64:0/3531761988 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3f58003ca0 con 0x55a1a15f7120 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f729aa740 10 client.26023 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f729aa740 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f729aa740 20 client.26023 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.772+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a160da90 0x55a1a160fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.773+0000 7f3f729aa740 4 client.26023 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.773+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f64022070 con 0x55a1a160da90 2024-05-09T09:11:57.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.773+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a160da90 0x55a1a160fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.773+0000 7f3f517fa640 10 client.26023 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.773+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a160da90 0x55a1a160fef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771f982619ff2f6b server_cookie=e4eb43290f6321cd in_seq=0 out_seq=0 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.774+0000 7f3f517fa640 1 -- 172.21.15.64:0/3531761988 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3f64022070 con 0x55a1a160da90 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.774+0000 7f3f517fa640 10 client.26023 handle_command_reply: tid=0 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.774+0000 7f3f729aa740 1 client.26023 shutdown 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 msgr2=0x55a1a1601870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a1601870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a1a15f8370 tx=0x7f3f58012b60 comp rx=0 tx=0).stop 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 shutdown_connections 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a160da90 0x55a1a160fef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a16091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a1601db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3531761988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a1601870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 >> 172.21.15.64:0/3531761988 conn(0x55a1a15f2790 msgr2=0x55a1a15f1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f517fa640 0 client.26023 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 shutdown_connections 2024-05-09T09:11:57.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 1 -- 172.21.15.64:0/3531761988 wait complete. 2024-05-09T09:11:57.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.775+0000 7f3f729aa740 20 client.26023 trim_cache size 0 max 16384 2024-05-09T09:11:57.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.805+0000 7f3f729aa740 1 Processor -- start 2024-05-09T09:11:57.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.805+0000 7f3f729aa740 1 -- start start 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.805+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 0x55a1a15f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a1600e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 0x55a1a15f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a1606de0 0x55a1a16013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 0x55a1a15f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60020/0 (socket says 172.21.15.64:60020) 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f729aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681bb970 con 0x55a1a1606de0 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f729aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b91b0 con 0x55a1a15f7120 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f729aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b7000 con 0x55a1a15fcc80 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3274550158 learned_addr learned my addr 172.21.15.64:0/3274550158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a1600e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a1606de0 0x55a1a16013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3274550158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 msgr2=0x55a1a1600e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a1600e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3274550158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a1606de0 msgr2=0x55a1a16013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a1606de0 0x55a1a16013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.806+0000 7f3f6d95d640 1 -- 172.21.15.64:0/3274550158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a15f7ef0 con 0x55a1a15f7120 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.807+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 0x55a1a15f5d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f580058c0 tx=0x7f3f580056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c781016b2f07870f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.807+0000 7f3f537fe640 1 -- 172.21.15.64:0/3274550158 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58005020 con 0x55a1a15f7120 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.807+0000 7f3f537fe640 1 -- 172.21.15.64:0/3274550158 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f58010c20 con 0x55a1a15f7120 2024-05-09T09:11:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.807+0000 7f3f537fe640 1 -- 172.21.15.64:0/3274550158 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800fb30 con 0x55a1a15f7120 2024-05-09T09:11:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 -- 172.21.15.64:0/3274550158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 msgr2=0x55a1a15f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 0x55a1a15f5d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f580058c0 tx=0x7f3f580056e0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 -- 172.21.15.64:0/3274550158 shutdown_connections 2024-05-09T09:11:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a1606de0 0x55a1a16013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15fcc80 0x55a1a1600e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 --2- 172.21.15.64:0/3274550158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a15f7120 0x55a1a15f5d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 -- 172.21.15.64:0/3274550158 >> 172.21.15.64:0/3274550158 conn(0x55a1a15f2790 msgr2=0x55a1a15f5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 -- 172.21.15.64:0/3274550158 shutdown_connections 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.808+0000 7f3f729aa740 1 -- 172.21.15.64:0/3274550158 wait complete. 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.809+0000 7f3f729aa740 1 -- start start 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.809+0000 7f3f729aa740 1 Processor -- start 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a161ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a161f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a1620380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b7670 con 0x55a1a15fcc80 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b8ae0 con 0x55a1a1606de0 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681b7d40 con 0x55a1a15f7120 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a161f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a161f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53274/0 (socket says 172.21.15.64:53274) 2024-05-09T09:11:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 -- 172.21.15.64:0/1818790932 learned_addr learned my addr 172.21.15.64:0/1818790932 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f729aa740 10 client.0 _ll_register_callbacks cb 0x7f3f58009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a1620380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a161ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 -- 172.21.15.64:0/1818790932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 msgr2=0x55a1a161ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a161ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 -- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 msgr2=0x55a1a1620380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a1620380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.810+0000 7f3f6d15c640 1 -- 172.21.15.64:0/1818790932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a1620970 con 0x55a1a15fcc80 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.811+0000 7f3f6e15e640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a1620380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.811+0000 7f3f6d15c640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a161f370 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3f640137d0 tx=0x7f3f64014b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cad575cddd12d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.811+0000 7f3f517fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.811+0000 7f3f517fa640 1 -- 172.21.15.64:0/1818790932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6402f3c0 con 0x55a1a15fcc80 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.811+0000 7f3f517fa640 1 -- 172.21.15.64:0/1818790932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f6402f560 con 0x55a1a15fcc80 2024-05-09T09:11:57.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.812+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f58010c20 con 0x55a1a15fcc80 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.812+0000 7f3f517fa640 1 -- 172.21.15.64:0/1818790932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f6403d050 con 0x55a1a15fcc80 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.812+0000 7f3f729aa740 10 client.16359 fetch_fsmap learned FSMap version 30 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.812+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1a1673f20 con 0x55a1a15fcc80 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f517fa640 1 -- 172.21.15.64:0/1818790932 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f58010c20 con 0x55a1a15fcc80 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f517fa640 1 -- 172.21.15.64:0/1818790932 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3f6401c030 con 0x55a1a15fcc80 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f729aa740 10 client.16359 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f729aa740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f729aa740 20 client.16359 populate_metadata read hostname 'smithi064' 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f729aa740 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a1674640 0x55a1a1676a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f729aa740 4 client.16359 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.813+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1a1569430 con 0x55a1a1674640 2024-05-09T09:11:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.814+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a1674640 0x55a1a1676a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:11:57.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.814+0000 7f3f517fa640 10 client.16359 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.814+0000 7f3f6d95d640 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a1674640 0x55a1a1676a60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672080d971563d66 server_cookie=1c5ac10e2306986e in_seq=0 out_seq=0 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.815+0000 7f3f517fa640 1 -- 172.21.15.64:0/1818790932 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a1a1569430 con 0x55a1a1674640 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.815+0000 7f3f517fa640 10 client.16359 handle_command_reply: tid=0 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.815+0000 7f3f729aa740 1 client.16359 shutdown 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.815+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 msgr2=0x55a1a161f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.815+0000 7f3f729aa740 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a161f370 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3f640137d0 tx=0x7f3f64014b60 comp rx=0 tx=0).stop 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 shutdown_connections 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1a1674640 0x55a1a1676a60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f517fa640 0 client.16359 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1a1606de0 0x55a1a1620380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1a15fcc80 0x55a1a161f370 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 --2- 172.21.15.64:0/1818790932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1a15f7120 0x55a1a161ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 >> 172.21.15.64:0/1818790932 conn(0x55a1a15f2790 msgr2=0x55a1a15f5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 shutdown_connections 2024-05-09T09:11:57.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 1 -- 172.21.15.64:0/1818790932 wait complete. 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.816+0000 7f3f729aa740 20 client.16359 trim_cache size 0 max 16384 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stdout: "7bf9ef14-4e50-41c5-92ef-4aefa1022887": { 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:11:57.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "7bf9ef14-4e50-41c5-92ef-4aefa1022887", 2024-05-09T09:11:57.819 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:11:57.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:11:57.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:11:57.819 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.818+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3f540a2c90 msgr2=0x7f3f540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.818+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3f540a2c90 0x7f3f540a5150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f680729b0 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.818+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 msgr2=0x7f3f68072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.818+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 0x7f3f68072a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f6002ac10 tx=0x7f3f60030fc0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.818+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 shutdown_connections 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3f540a2c90 0x7f3f540a5150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f68101150 0x7f3f68072a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f68100790 0x7f3f68071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 --2- 172.21.15.64:0/2552398711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f680ffa30 0x7f3f680714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:11:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 >> 172.21.15.64:0/2552398711 conn(0x7f3f680fb360 msgr2=0x7f3f681019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:11:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 shutdown_connections 2024-05-09T09:11:57.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:11:57.819+0000 7f3f6e95f640 1 -- 172.21.15.64:0/2552398711 wait complete. 2024-05-09T09:11:57.843 INFO:tasks.cephfs.filesystem:scrub status for tag:7bf9ef14-4e50-41c5-92ef-4aefa1022887 - {'path': '/', 'tag': '7bf9ef14-4e50-41c5-92ef-4aefa1022887', 'options': 'recursive,force'} 2024-05-09T09:11:58.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:57 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:58.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:57 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:58.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:57 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:58.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:57 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:58.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:57 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:58.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:57 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:58.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:57 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:58.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:57 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:58.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:57 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:59.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: pgmap v455: 129 pgs: 129 active+clean; 980 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:11:58 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: pgmap v455: 129 pgs: 129 active+clean; 980 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.505 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:11:58 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: pgmap v455: 129 pgs: 129 active+clean; 980 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:11:59.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:11:59.506 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:11:59.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:11:58 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:01.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:00 smithi179 ceph-mon[34086]: pgmap v456: 129 pgs: 129 active+clean; 988 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-09T09:12:01.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:00 smithi139 ceph-mon[33599]: pgmap v456: 129 pgs: 129 active+clean; 988 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-09T09:12:01.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 100.0 s, 1063.2 tps, lat 93.837 ms stddev 28.764 2024-05-09T09:12:01.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:00 smithi064 ceph-mon[30149]: pgmap v456: 129 pgs: 129 active+clean; 988 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-09T09:12:03.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:02 smithi179 ceph-mon[34086]: pgmap v457: 129 pgs: 129 active+clean; 992 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-05-09T09:12:03.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:02 smithi139 ceph-mon[33599]: pgmap v457: 129 pgs: 129 active+clean; 992 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-05-09T09:12:03.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:02 smithi064 ceph-mon[30149]: pgmap v457: 129 pgs: 129 active+clean; 992 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-05-09T09:12:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 -- 172.21.15.64:0/323605231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc105420 msgr2=0x7f35bc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 --2- 172.21.15.64:0/323605231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc105420 0x7f35bc1058a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009ca0 tx=0x7f35b402a8d0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 -- 172.21.15.64:0/323605231 shutdown_connections 2024-05-09T09:12:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 --2- 172.21.15.64:0/323605231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc105420 0x7f35bc1058a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 --2- 172.21.15.64:0/323605231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35bc102a70 0x7f35bc104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 --2- 172.21.15.64:0/323605231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc100100 0x7f35bc102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.854+0000 7f35c4291640 1 -- 172.21.15.64:0/323605231 >> 172.21.15.64:0/323605231 conn(0x7f35bc0fbd00 msgr2=0x7f35bc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.855+0000 7f35c4291640 1 -- 172.21.15.64:0/323605231 shutdown_connections 2024-05-09T09:12:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.856+0000 7f35c4291640 1 -- 172.21.15.64:0/323605231 wait complete. 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c4291640 1 Processor -- start 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c4291640 1 -- start start 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c4291640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc100100 0x7f35bc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c4291640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 0x7f35bc19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c4291640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35bc105420 0x7f35bc1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c182c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 0x7f35bc19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.857+0000 7f35c182c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 0x7f35bc19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35828/0 (socket says 172.21.15.64:35828) 2024-05-09T09:12:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.858+0000 7f35c182c640 1 -- 172.21.15.64:0/548206720 learned_addr learned my addr 172.21.15.64:0/548206720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.858+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc113a80 con 0x7f35bc102a70 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.858+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc113900 con 0x7f35bc100100 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.858+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc113c00 con 0x7f35bc105420 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.858+0000 7f35c282e640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35bc105420 0x7f35bc1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.859+0000 7f35c182c640 1 -- 172.21.15.64:0/548206720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35bc105420 msgr2=0x7f35bc1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.859+0000 7f35c182c640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35bc105420 0x7f35bc1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.859+0000 7f35c182c640 1 -- 172.21.15.64:0/548206720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc100100 msgr2=0x7f35bc19df10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.859+0000 7f35c182c640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc100100 0x7f35bc19df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.859+0000 7f35c182c640 1 -- 172.21.15.64:0/548206720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc1a5a90 con 0x7f35bc102a70 2024-05-09T09:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.859+0000 7f35c182c640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 0x7f35bc19e450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b450 tx=0x7f35ac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.860+0000 7f35b37fe640 1 -- 172.21.15.64:0/548206720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac015020 con 0x7f35bc102a70 2024-05-09T09:12:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.860+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc1a5d80 con 0x7f35bc102a70 2024-05-09T09:12:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.860+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc1a62c0 con 0x7f35bc102a70 2024-05-09T09:12:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.861+0000 7f35b37fe640 1 -- 172.21.15.64:0/548206720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ac011d40 con 0x7f35bc102a70 2024-05-09T09:12:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.861+0000 7f35b37fe640 1 -- 172.21.15.64:0/548206720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00f340 con 0x7f35bc102a70 2024-05-09T09:12:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.864+0000 7f35c6f95740 1 Processor -- start 2024-05-09T09:12:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 -- start start 2024-05-09T09:12:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c875670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 0x56328c87a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c87adb0 0x56328c87d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56328c7d9340 con 0x56328c875250 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56328c781290 con 0x56328c87adb0 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c6f95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56328c7d5300 con 0x56328c875c40 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c202d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 0x56328c87a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35b37fe640 1 -- 172.21.15.64:0/548206720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35ac018020 con 0x7f35bc102a70 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c202d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 0x56328c87a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37294/0 (socket says 172.21.15.64:37294) 2024-05-09T09:12:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.865+0000 7f35c202d640 1 -- 172.21.15.64:0/95508658 learned_addr learned my addr 172.21.15.64:0/95508658 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c282e640 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c875670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c202d640 1 -- 172.21.15.64:0/95508658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c87adb0 msgr2=0x56328c87d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c182c640 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c87adb0 0x56328c87d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c202d640 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c87adb0 0x56328c87d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c202d640 1 -- 172.21.15.64:0/95508658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 msgr2=0x56328c875670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c202d640 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c875670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c202d640 1 -- 172.21.15.64:0/95508658 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328c7a58e0 con 0x56328c875c40 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c182c640 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c87adb0 0x56328c87d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:12:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.866+0000 7f35c202d640 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 0x56328c87a740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009bd0 tx=0x7f35b800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e14307066a1ef4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.867+0000 7f35b0bfe640 1 -- 172.21.15.64:0/95508658 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8005020 con 0x56328c875c40 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.867+0000 7f35b0bfe640 1 -- 172.21.15.64:0/95508658 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b80047a0 con 0x56328c875c40 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.867+0000 7f35b0bfe640 1 -- 172.21.15.64:0/95508658 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8010690 con 0x56328c875c40 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35c6f95740 1 -- 172.21.15.64:0/95508658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 msgr2=0x56328c87a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35c6f95740 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 0x56328c87a740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009bd0 tx=0x7f35b800e720 comp rx=0 tx=0).stop 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35b37fe640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35a80a2d30 0x7f35a80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35c6f95740 1 -- 172.21.15.64:0/95508658 shutdown_connections 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35c6f95740 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c87adb0 0x56328c87d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35c6f95740 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875c40 0x56328c87a740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.868+0000 7f35c6f95740 1 --2- 172.21.15.64:0/95508658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c875670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.869+0000 7f35c6f95740 1 -- 172.21.15.64:0/95508658 >> 172.21.15.64:0/95508658 conn(0x56328c8708c0 msgr2=0x56328c870cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.869+0000 7f35b37fe640 1 -- 172.21.15.64:0/548206720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35ac0ceda0 con 0x7f35bc102a70 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.869+0000 7f35c282e640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35a80a2d30 0x7f35a80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.869+0000 7f35c6f95740 1 -- 172.21.15.64:0/95508658 shutdown_connections 2024-05-09T09:12:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.869+0000 7f35c6f95740 1 -- 172.21.15.64:0/95508658 wait complete. 2024-05-09T09:12:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.869+0000 7f35c6f95740 1 -- start start 2024-05-09T09:12:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 Processor -- start 2024-05-09T09:12:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c890510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c890a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c897a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56328c88ece0 con 0x56328c87adb0 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56328c794790 con 0x56328c875c40 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56328c88ee60 con 0x56328c875250 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c6f95740 10 client.0 _ll_register_callbacks cb 0x56328c7cfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.870+0000 7f35c202d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c890510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c182c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c890a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c890510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37310/0 (socket says 172.21.15.64:37310) 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 -- 172.21.15.64:0/1550549182 learned_addr learned my addr 172.21.15.64:0/1550549182 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 -- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 msgr2=0x56328c890a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c890a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 -- 172.21.15.64:0/1550549182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 msgr2=0x56328c897a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c897a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 -- 172.21.15.64:0/1550549182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328c4ed210 con 0x56328c875250 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c182c640 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c890a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c282e640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35a80a2d30 0x7f35a80a51f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005e30 tx=0x7f35b40099b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35c202d640 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c890510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f35b80058c0 tx=0x7f35b8004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df3f2dbfdcb27f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.871+0000 7f35a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.872+0000 7f35a5ffb640 1 -- 172.21.15.64:0/1550549182 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b801d040 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.872+0000 7f35a5ffb640 1 -- 172.21.15.64:0/1550549182 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35b8004230 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.872+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b80047a0 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.873+0000 7f35a5ffb640 1 -- 172.21.15.64:0/1550549182 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35b8009850 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.873+0000 7f35c6f95740 10 client.26038 fetch_fsmap learned FSMap version 30 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.873+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56328c5b9620 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.873+0000 7f35a5ffb640 1 -- 172.21.15.64:0/1550549182 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b80047a0 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35a5ffb640 1 -- 172.21.15.64:0/1550549182 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f35b8003ba0 con 0x56328c875250 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c6f95740 10 client.26038 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c6f95740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c6f95740 20 client.26038 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c89c310 0x56328c89e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c6f95740 4 client.26038 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328c7e6a70 con 0x56328c89c310 2024-05-09T09:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.874+0000 7f35c182c640 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c89c310 0x56328c89e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.875+0000 7f35a5ffb640 10 client.26038 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.875+0000 7f35c182c640 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c89c310 0x56328c89e790 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62585af5a4541da server_cookie=79c97a88cdf34bc5 in_seq=0 out_seq=0 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.875+0000 7f35a5ffb640 1 -- 172.21.15.64:0/1550549182 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56328c7e6a70 con 0x56328c89c310 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.875+0000 7f35a5ffb640 10 client.26038 handle_command_reply: tid=0 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.875+0000 7f35c6f95740 1 client.26038 shutdown 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 msgr2=0x56328c890510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c890510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f35b80058c0 tx=0x7f35b8004f00 comp rx=0 tx=0).stop 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 shutdown_connections 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c89c310 0x56328c89e790 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c897a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c890a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1550549182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c890510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 >> 172.21.15.64:0/1550549182 conn(0x56328c8708c0 msgr2=0x56328c86fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35a5ffb640 0 client.26038 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 shutdown_connections 2024-05-09T09:12:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.876+0000 7f35c6f95740 1 -- 172.21.15.64:0/1550549182 wait complete. 2024-05-09T09:12:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.877+0000 7f35c6f95740 20 client.26038 trim_cache size 0 max 16384 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.907+0000 7f35c6f95740 1 Processor -- start 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.907+0000 7f35c6f95740 1 -- start start 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c873f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c897a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c88f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c6f95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35a00125e0 con 0x56328c87adb0 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c6f95740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35a000fe20 con 0x56328c875c40 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c6f95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f35a000dc70 con 0x56328c875250 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c282e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c88f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c282e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c88f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35862/0 (socket says 172.21.15.64:35862) 2024-05-09T09:12:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c282e640 1 -- 172.21.15.64:0/1218776579 learned_addr learned my addr 172.21.15.64:0/1218776579 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.908+0000 7f35c182c640 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c897a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35c182c640 1 -- 172.21.15.64:0/1218776579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 msgr2=0x56328c873f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35c182c640 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c873f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35c182c640 1 -- 172.21.15.64:0/1218776579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 msgr2=0x56328c88f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35c182c640 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c88f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35c182c640 1 -- 172.21.15.64:0/1218776579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328c871630 con 0x56328c875c40 2024-05-09T09:12:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35c182c640 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c897a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x56328c8919f0 tx=0x7f35ac0974d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254ffb21b6a165c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35a7fff640 1 -- 172.21.15.64:0/1218776579 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0992b0 con 0x56328c875c40 2024-05-09T09:12:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35a7fff640 1 -- 172.21.15.64:0/1218776579 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ac0b9e50 con 0x56328c875c40 2024-05-09T09:12:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.909+0000 7f35a7fff640 1 -- 172.21.15.64:0/1218776579 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac081bd0 con 0x56328c875c40 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.910+0000 7f35c6f95740 1 -- 172.21.15.64:0/1218776579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 msgr2=0x56328c897a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.910+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c897a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x56328c8919f0 tx=0x7f35ac0974d0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 -- 172.21.15.64:0/1218776579 shutdown_connections 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c87adb0 0x56328c88f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c897a50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 --2- 172.21.15.64:0/1218776579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c875250 0x56328c873f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 -- 172.21.15.64:0/1218776579 >> 172.21.15.64:0/1218776579 conn(0x56328c8708c0 msgr2=0x56328c873840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 -- 172.21.15.64:0/1218776579 shutdown_connections 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 -- 172.21.15.64:0/1218776579 wait complete. 2024-05-09T09:12:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.911+0000 7f35c6f95740 1 -- start start 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.912+0000 7f35c6f95740 1 Processor -- start 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.912+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c897980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.912+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c8944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c6f95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c87adb0 0x56328c894a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c6f95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35a000e2e0 con 0x56328c875250 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c202d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c897980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c182c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c8944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c282e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c87adb0 0x56328c894a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c202d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c897980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35870/0 (socket says 172.21.15.64:35870) 2024-05-09T09:12:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c282e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c87adb0 0x56328c894a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37326/0 (socket says 172.21.15.64:37326) 2024-05-09T09:12:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c202d640 1 -- 172.21.15.64:0/842327713 learned_addr learned my addr 172.21.15.64:0/842327713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c182c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c8944e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49642/0 (socket says 172.21.15.64:49642) 2024-05-09T09:12:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35a000f750 con 0x56328c875c40 2024-05-09T09:12:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f35a000e9b0 con 0x56328c87adb0 2024-05-09T09:12:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.913+0000 7f35c6f95740 10 client.0 _ll_register_callbacks cb 0x7f35b8009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.915+0000 7f35c202d640 1 -- 172.21.15.64:0/842327713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c87adb0 msgr2=0x56328c894a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.915+0000 7f35c202d640 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c87adb0 0x56328c894a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.915+0000 7f35c202d640 1 -- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 msgr2=0x56328c8944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.915+0000 7f35c202d640 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c8944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.915+0000 7f35c202d640 1 -- 172.21.15.64:0/842327713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328c900900 con 0x56328c875250 2024-05-09T09:12:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.916+0000 7f35c202d640 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c897980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f35b80057c0 tx=0x7f35b8004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3228b738d587d83a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.916+0000 7f35a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.916+0000 7f35a5ffb640 1 -- 172.21.15.64:0/842327713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8038070 con 0x56328c875250 2024-05-09T09:12:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.916+0000 7f35a5ffb640 1 -- 172.21.15.64:0/842327713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f35b8018c90 con 0x56328c875250 2024-05-09T09:12:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.916+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ac0b9e50 con 0x56328c875250 2024-05-09T09:12:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.917+0000 7f35a5ffb640 1 -- 172.21.15.64:0/842327713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35b8041050 con 0x56328c875250 2024-05-09T09:12:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.917+0000 7f35c6f95740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-05-09T09:12:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.917+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56328c901bc0 con 0x56328c875250 2024-05-09T09:12:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.918+0000 7f35a5ffb640 1 -- 172.21.15.64:0/842327713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ac0b9e50 con 0x56328c875250 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35a5ffb640 1 -- 172.21.15.64:0/842327713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f35b803b020 con 0x56328c875250 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35c6f95740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35c6f95740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35c6f95740 20 client.16374 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35c6f95740 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c9025d0 0x56328c9049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35c6f95740 4 client.16374 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.919+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56328c7e0dc0 con 0x56328c9025d0 2024-05-09T09:12:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.920+0000 7f35c182c640 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c9025d0 0x56328c9049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.921+0000 7f35c182c640 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c9025d0 0x56328c9049f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3521262552583b2 server_cookie=1fdcf8aa491f831f in_seq=0 out_seq=0 2024-05-09T09:12:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.921+0000 7f35a5ffb640 10 client.16374 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.940+0000 7f35a5ffb640 1 -- 172.21.15.64:0/842327713 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56328c7e0dc0 con 0x56328c9025d0 2024-05-09T09:12:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.940+0000 7f35a5ffb640 10 client.16374 handle_command_reply: tid=0 2024-05-09T09:12:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.940+0000 7f35c6f95740 1 client.16374 shutdown 2024-05-09T09:12:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.940+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 msgr2=0x56328c897980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.940+0000 7f35c6f95740 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c897980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f35b80057c0 tx=0x7f35b8004e90 comp rx=0 tx=0).stop 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 shutdown_connections 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35c6f95740 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56328c9025d0 0x56328c9049f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35c6f95740 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56328c87adb0 0x56328c894a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35a5ffb640 0 client.16374 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35c6f95740 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56328c875c40 0x56328c8944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35c6f95740 1 --2- 172.21.15.64:0/842327713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56328c875250 0x56328c897980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.941+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 >> 172.21.15.64:0/842327713 conn(0x56328c8708c0 msgr2=0x56328c873840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.942+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 shutdown_connections 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.942+0000 7f35c6f95740 1 -- 172.21.15.64:0/842327713 wait complete. 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.943+0000 7f35c6f95740 20 client.16374 trim_cache size 0 max 16384 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:12:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:12:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.944+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35a80a2d30 msgr2=0x7f35a80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.944+0000 7f35c4291640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35a80a2d30 0x7f35a80a51f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005e30 tx=0x7f35b40099b0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.944+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 msgr2=0x7f35bc19e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.944+0000 7f35c4291640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 0x7f35bc19e450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b450 tx=0x7f35ac00b910 comp rx=0 tx=0).stop 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 shutdown_connections 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35a80a2d30 0x7f35a80a51f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35bc105420 0x7f35bc1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35bc102a70 0x7f35bc19e450 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 --2- 172.21.15.64:0/548206720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35bc100100 0x7f35bc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 >> 172.21.15.64:0/548206720 conn(0x7f35bc0fbd00 msgr2=0x7f35bc100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.945+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 shutdown_connections 2024-05-09T09:12:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:04.946+0000 7f35c4291640 1 -- 172.21.15.64:0/548206720 wait complete. 2024-05-09T09:12:05.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:04 smithi179 ceph-mon[34086]: pgmap v458: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-05-09T09:12:05.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:04 smithi139 ceph-mon[33599]: pgmap v458: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-05-09T09:12:05.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:04 smithi064 ceph-mon[30149]: pgmap v458: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-05-09T09:12:06.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:12:06.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:12:06.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:12:06.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:12:06.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:12:06.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:12:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 105.0 s, 1056.8 tps, lat 94.571 ms stddev 26.776 2024-05-09T09:12:06.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:12:06.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:12:06.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:12:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:06 smithi179 ceph-mon[34086]: pgmap v459: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-05-09T09:12:07.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:06 smithi139 ceph-mon[33599]: pgmap v459: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-05-09T09:12:07.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:06 smithi064 ceph-mon[30149]: pgmap v459: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-05-09T09:12:09.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:08 smithi179 ceph-mon[34086]: pgmap v460: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.6 MiB/s wr, 1.05k op/s 2024-05-09T09:12:09.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:08 smithi139 ceph-mon[33599]: pgmap v460: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.6 MiB/s wr, 1.05k op/s 2024-05-09T09:12:09.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:08 smithi064 ceph-mon[30149]: pgmap v460: 129 pgs: 129 active+clean; 992 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.6 MiB/s wr, 1.05k op/s 2024-05-09T09:12:11.228 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:10 smithi179 ceph-mon[34086]: pgmap v461: 129 pgs: 129 active+clean; 992 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-05-09T09:12:11.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:10 smithi139 ceph-mon[33599]: pgmap v461: 129 pgs: 129 active+clean; 992 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-05-09T09:12:11.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 110.0 s, 1026.2 tps, lat 97.318 ms stddev 25.976 2024-05-09T09:12:11.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:10 smithi064 ceph-mon[30149]: pgmap v461: 129 pgs: 129 active+clean; 992 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-05-09T09:12:12.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:11 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:12.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:11 smithi179 ceph-mon[34086]: pgmap v462: 129 pgs: 129 active+clean; 993 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-05-09T09:12:12.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:11 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:12.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:11 smithi139 ceph-mon[33599]: pgmap v462: 129 pgs: 129 active+clean; 993 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-05-09T09:12:12.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:11 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:12.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:11 smithi064 ceph-mon[30149]: pgmap v462: 129 pgs: 129 active+clean; 993 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-05-09T09:12:14.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:14 smithi139 ceph-mon[33599]: pgmap v463: 129 pgs: 129 active+clean; 993 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.1 MiB/s wr, 1.10k op/s 2024-05-09T09:12:14.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:14 smithi064 ceph-mon[30149]: pgmap v463: 129 pgs: 129 active+clean; 993 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.1 MiB/s wr, 1.10k op/s 2024-05-09T09:12:14.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:14 smithi179 ceph-mon[34086]: pgmap v463: 129 pgs: 129 active+clean; 993 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.1 MiB/s wr, 1.10k op/s 2024-05-09T09:12:16.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 115.0 s, 1028.0 tps, lat 97.478 ms stddev 32.419 2024-05-09T09:12:16.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:16 smithi139 ceph-mon[33599]: pgmap v464: 129 pgs: 129 active+clean; 993 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-05-09T09:12:16.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:16 smithi064 ceph-mon[30149]: pgmap v464: 129 pgs: 129 active+clean; 993 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-05-09T09:12:16.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:16 smithi179 ceph-mon[34086]: pgmap v464: 129 pgs: 129 active+clean; 993 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-05-09T09:12:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.112+0000 7ff26c683640 1 -- 172.21.15.64:0/3099107934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264104cb0 msgr2=0x7ff264105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.112+0000 7ff26c683640 1 --2- 172.21.15.64:0/3099107934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264104cb0 0x7ff264105130 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff25402a410 comp rx=0 tx=0).stop 2024-05-09T09:12:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 -- 172.21.15.64:0/3099107934 shutdown_connections 2024-05-09T09:12:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 --2- 172.21.15.64:0/3099107934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff264105670 0x7ff26410bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 --2- 172.21.15.64:0/3099107934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264104cb0 0x7ff264105130 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 --2- 172.21.15.64:0/3099107934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264103ac0 0x7ff264103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 -- 172.21.15.64:0/3099107934 >> 172.21.15.64:0/3099107934 conn(0x7ff2640ff050 msgr2=0x7ff264101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 -- 172.21.15.64:0/3099107934 shutdown_connections 2024-05-09T09:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.115+0000 7ff26c683640 1 -- 172.21.15.64:0/3099107934 wait complete. 2024-05-09T09:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.116+0000 7ff26c683640 1 Processor -- start 2024-05-09T09:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.116+0000 7ff26c683640 1 -- start start 2024-05-09T09:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.117+0000 7ff26c683640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264103ac0 0x7ff26406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.117+0000 7ff26c683640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff264104cb0 0x7ff26406de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.117+0000 7ff26c683640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 0x7ff264072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.117+0000 7ff26c683640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff264076e80 con 0x7ff264103ac0 2024-05-09T09:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.117+0000 7ff26c683640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff264076d00 con 0x7ff264105670 2024-05-09T09:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.117+0000 7ff26c683640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff264077000 con 0x7ff264104cb0 2024-05-09T09:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.118+0000 7ff26ac20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 0x7ff264072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.118+0000 7ff26ac20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 0x7ff264072f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36696/0 (socket says 172.21.15.64:36696) 2024-05-09T09:12:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.118+0000 7ff26ac20640 1 -- 172.21.15.64:0/3646462707 learned_addr learned my addr 172.21.15.64:0/3646462707 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.119+0000 7ff26a41f640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264103ac0 0x7ff26406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.119+0000 7ff26ac20640 1 -- 172.21.15.64:0/3646462707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff264104cb0 msgr2=0x7ff26406de80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.119+0000 7ff26ac20640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff264104cb0 0x7ff26406de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.119+0000 7ff26ac20640 1 -- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264103ac0 msgr2=0x7ff26406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.119+0000 7ff26ac20640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264103ac0 0x7ff26406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.119+0000 7ff26ac20640 1 -- 172.21.15.64:0/3646462707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2640735c0 con 0x7ff264105670 2024-05-09T09:12:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.120+0000 7ff26ac20640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 0x7ff264072f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff26000e970 tx=0x7ff26000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.121+0000 7ff25b7fe640 1 -- 172.21.15.64:0/3646462707 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26000ccc0 con 0x7ff264105670 2024-05-09T09:12:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.122+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26419d940 con 0x7ff264105670 2024-05-09T09:12:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.122+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26419de90 con 0x7ff264105670 2024-05-09T09:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.122+0000 7ff25b7fe640 1 -- 172.21.15.64:0/3646462707 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff26000ce60 con 0x7ff264105670 2024-05-09T09:12:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.122+0000 7ff25b7fe640 1 -- 172.21.15.64:0/3646462707 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260010430 con 0x7ff264105670 2024-05-09T09:12:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.126+0000 7ff26f38a740 1 Processor -- start 2024-05-09T09:12:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff25b7fe640 1 -- 172.21.15.64:0/3646462707 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2600105d0 con 0x7ff264105670 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 -- start start 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 0x557983d21740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d24240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557983c80340 con 0x557983d21db0 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557983c28290 con 0x557983d1c250 2024-05-09T09:12:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.127+0000 7ff26f38a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557983c7c300 con 0x557983d1cc40 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36702/0 (socket says 172.21.15.64:36702) 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 -- 172.21.15.64:0/3814464875 learned_addr learned my addr 172.21.15.64:0/3814464875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d24240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 -- 172.21.15.64:0/3814464875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 msgr2=0x557983d21740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 0x557983d21740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 -- 172.21.15.64:0/3814464875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 msgr2=0x557983d24240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d24240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.128+0000 7ff269c1e640 1 -- 172.21.15.64:0/3814464875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557983c4c8e0 con 0x557983d1c250 2024-05-09T09:12:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.129+0000 7ff269c1e640 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1c670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2540096b0 tx=0x7ff254009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d9cf510e10642 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.129+0000 7ff258ff9640 1 -- 172.21.15.64:0/3814464875 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254042020 con 0x557983d1c250 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.129+0000 7ff258ff9640 1 -- 172.21.15.64:0/3814464875 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff254031c60 con 0x557983d1c250 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.129+0000 7ff258ff9640 1 -- 172.21.15.64:0/3814464875 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254035040 con 0x557983d1c250 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.130+0000 7ff25b7fe640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2500a2df0 0x7ff2500a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.130+0000 7ff26a41f640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2500a2df0 0x7ff2500a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.130+0000 7ff25b7fe640 1 -- 172.21.15.64:0/3646462707 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2600cf6c0 con 0x7ff264105670 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 -- 172.21.15.64:0/3814464875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 msgr2=0x557983d1c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1c670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2540096b0 tx=0x7ff254009460 comp rx=0 tx=0).stop 2024-05-09T09:12:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 -- 172.21.15.64:0/3814464875 shutdown_connections 2024-05-09T09:12:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d24240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 0x557983d21740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 --2- 172.21.15.64:0/3814464875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1c670 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 -- 172.21.15.64:0/3814464875 >> 172.21.15.64:0/3814464875 conn(0x557983d178c0 msgr2=0x557983d17cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26a41f640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2500a2df0 0x7ff2500a52b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557983d1d040 tx=0x7ff25c0097d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 -- 172.21.15.64:0/3814464875 shutdown_connections 2024-05-09T09:12:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.131+0000 7ff26f38a740 1 -- 172.21.15.64:0/3814464875 wait complete. 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.132+0000 7ff26f38a740 1 -- start start 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.132+0000 7ff26f38a740 1 Processor -- start 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.132+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 0x557983d264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.132+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1cc40 0x557983d26a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26f38a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557983d24d00 con 0x557983d21db0 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26f38a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557983c3b790 con 0x557983d1cc40 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26f38a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557983d24e80 con 0x557983d1c250 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26f38a740 10 client.0 _ll_register_callbacks cb 0x557983c76a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26a41f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 0x557983d264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26a41f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 0x557983d264c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48476/0 (socket says 172.21.15.64:48476) 2024-05-09T09:12:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff26a41f640 1 -- 172.21.15.64:0/1211940142 learned_addr learned my addr 172.21.15.64:0/1211940142 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.133+0000 7ff269c1e640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1cc40 0x557983d26a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26a41f640 1 -- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1cc40 msgr2=0x557983d26a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26a41f640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1cc40 0x557983d26a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26ac20640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26a41f640 1 -- 172.21.15.64:0/1211940142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 msgr2=0x557983d2da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26a41f640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26a41f640 1 -- 172.21.15.64:0/1211940142 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557983994210 con 0x557983d1c250 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff269c1e640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1cc40 0x557983d26a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26ac20640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:12:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26a41f640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 0x557983d264c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005fd0 tx=0x7ff25c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f514b59270ea4160 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff24dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff24dffb640 1 -- 172.21.15.64:0/1211940142 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c023070 con 0x557983d1c250 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff24dffb640 1 -- 172.21.15.64:0/1211940142 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff25c0118b0 con 0x557983d1c250 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.134+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff254031c60 con 0x557983d1c250 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff24dffb640 1 -- 172.21.15.64:0/1211940142 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff25c02f050 con 0x557983d1c250 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff26f38a740 10 client.26047 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557983a60620 con 0x557983d1c250 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff24dffb640 1 -- 172.21.15.64:0/1211940142 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff254031c60 con 0x557983d1c250 2024-05-09T09:12:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff24dffb640 1 -- 172.21.15.64:0/1211940142 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff25c00f040 con 0x557983d1c250 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff26f38a740 10 client.26047 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.135+0000 7ff26f38a740 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.136+0000 7ff26f38a740 20 client.26047 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.136+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d32510 0x557983d349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.136+0000 7ff26f38a740 4 client.26047 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.136+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557983c8da70 con 0x557983d32510 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.136+0000 7ff269c1e640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d32510 0x557983d349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.137+0000 7ff269c1e640 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d32510 0x557983d349d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc1005d28fb062f server_cookie=b3b22060981182e8 in_seq=0 out_seq=0 2024-05-09T09:12:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.137+0000 7ff24dffb640 10 client.26047 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.137+0000 7ff24dffb640 1 -- 172.21.15.64:0/1211940142 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557983c8da70 con 0x557983d32510 2024-05-09T09:12:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.137+0000 7ff24dffb640 10 client.26047 handle_command_reply: tid=0 2024-05-09T09:12:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.138+0000 7ff26f38a740 1 client.26047 shutdown 2024-05-09T09:12:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.138+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 msgr2=0x557983d264c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.138+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 0x557983d264c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005fd0 tx=0x7ff25c004a40 comp rx=0 tx=0).stop 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 shutdown_connections 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d32510 0x557983d349d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1cc40 0x557983d26a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1211940142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1c250 0x557983d264c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 >> 172.21.15.64:0/1211940142 conn(0x557983d178c0 msgr2=0x557983d16a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff24dffb640 0 client.26047 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.139+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 shutdown_connections 2024-05-09T09:12:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.140+0000 7ff26f38a740 1 -- 172.21.15.64:0/1211940142 wait complete. 2024-05-09T09:12:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.140+0000 7ff26f38a740 20 client.26047 trim_cache size 0 max 16384 2024-05-09T09:12:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.174+0000 7ff26f38a740 1 Processor -- start 2024-05-09T09:12:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.174+0000 7ff26f38a740 1 -- start start 2024-05-09T09:12:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.174+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d1cc40 0x557983d1e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d21db0 0x557983d2d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26f38a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2480124f0 con 0x557983d1cc40 2024-05-09T09:12:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26f38a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff24800fd30 con 0x557983d1c250 2024-05-09T09:12:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26f38a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff24800db80 con 0x557983d21db0 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff269c1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d1cc40 0x557983d1e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26ac20640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d21db0 0x557983d2d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff269c1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d1cc40 0x557983d1e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45570/0 (socket says 172.21.15.64:45570) 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26a41f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff269c1e640 1 -- 172.21.15.64:0/1929578080 learned_addr learned my addr 172.21.15.64:0/1929578080 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26a41f640 1 -- 172.21.15.64:0/1929578080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d21db0 msgr2=0x557983d2d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26a41f640 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d21db0 0x557983d2d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26a41f640 1 -- 172.21.15.64:0/1929578080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d1cc40 msgr2=0x557983d1e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26a41f640 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d1cc40 0x557983d1e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.175+0000 7ff26a41f640 1 -- 172.21.15.64:0/1929578080 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557983d243d0 con 0x557983d1c250 2024-05-09T09:12:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.176+0000 7ff26a41f640 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1dec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009710 tx=0x7ff25c027500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20af651133a70d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.176+0000 7ff24ffff640 1 -- 172.21.15.64:0/1929578080 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c037070 con 0x557983d1c250 2024-05-09T09:12:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.176+0000 7ff24ffff640 1 -- 172.21.15.64:0/1929578080 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff25c0111d0 con 0x557983d1c250 2024-05-09T09:12:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.176+0000 7ff24ffff640 1 -- 172.21.15.64:0/1929578080 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c023050 con 0x557983d1c250 2024-05-09T09:12:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 -- 172.21.15.64:0/1929578080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 msgr2=0x557983d1dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1dec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009710 tx=0x7ff25c027500 comp rx=0 tx=0).stop 2024-05-09T09:12:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 -- 172.21.15.64:0/1929578080 shutdown_connections 2024-05-09T09:12:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d21db0 0x557983d2d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d1cc40 0x557983d1e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 --2- 172.21.15.64:0/1929578080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d1dec0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.178+0000 7ff26f38a740 1 -- 172.21.15.64:0/1929578080 >> 172.21.15.64:0/1929578080 conn(0x557983d178c0 msgr2=0x557983d214c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.179+0000 7ff26f38a740 1 -- 172.21.15.64:0/1929578080 shutdown_connections 2024-05-09T09:12:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.179+0000 7ff26f38a740 1 -- 172.21.15.64:0/1929578080 wait complete. 2024-05-09T09:12:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.180+0000 7ff26f38a740 1 -- start start 2024-05-09T09:12:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.180+0000 7ff26f38a740 1 Processor -- start 2024-05-09T09:12:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d2d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 0x557983d2a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26a41f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d2d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26a41f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d2d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36724/0 (socket says 172.21.15.64:36724) 2024-05-09T09:12:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26a41f640 1 -- 172.21.15.64:0/655501491 learned_addr learned my addr 172.21.15.64:0/655501491 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff24800e1f0 con 0x557983d21db0 2024-05-09T09:12:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff24800f660 con 0x557983d1c250 2024-05-09T09:12:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff24800e8c0 con 0x557983d1cc40 2024-05-09T09:12:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26f38a740 10 client.0 _ll_register_callbacks cb 0x7ff25c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.181+0000 7ff26ac20640 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff26a41f640 1 -- 172.21.15.64:0/655501491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 msgr2=0x557983d2a360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff26a41f640 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 0x557983d2a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff26a41f640 1 -- 172.21.15.64:0/655501491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 msgr2=0x557983d2a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff26a41f640 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff26a41f640 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557983d2af40 con 0x557983d1c250 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff26a41f640 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d2d850 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff25c01ff80 tx=0x7ff25c0118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5c78aae27cd170 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff24dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.182+0000 7ff24dffb640 1 -- 172.21.15.64:0/655501491 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c023070 con 0x557983d1c250 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.183+0000 7ff24dffb640 1 -- 172.21.15.64:0/655501491 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff25c004a40 con 0x557983d1c250 2024-05-09T09:12:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.183+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff25c0111d0 con 0x557983d1c250 2024-05-09T09:12:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.184+0000 7ff24dffb640 1 -- 172.21.15.64:0/655501491 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff25c04b050 con 0x557983d1c250 2024-05-09T09:12:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.185+0000 7ff26f38a740 10 client.25907 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.185+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557983d97c60 con 0x557983d1c250 2024-05-09T09:12:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.185+0000 7ff24dffb640 1 -- 172.21.15.64:0/655501491 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff25c040030 con 0x557983d1c250 2024-05-09T09:12:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.185+0000 7ff24dffb640 1 -- 172.21.15.64:0/655501491 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff25c01f450 con 0x557983d1c250 2024-05-09T09:12:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.186+0000 7ff26f38a740 10 client.25907 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.186+0000 7ff26f38a740 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.186+0000 7ff26f38a740 20 client.25907 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.186+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d98660 0x557983d9aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.186+0000 7ff26f38a740 4 client.25907 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:12:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.186+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff25c0111d0 con 0x557983d98660 2024-05-09T09:12:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.188+0000 7ff269c1e640 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d98660 0x557983d9aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.189+0000 7ff269c1e640 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d98660 0x557983d9aa80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fff99054b31d7a1 server_cookie=5a95378c0a9015fb in_seq=0 out_seq=0 2024-05-09T09:12:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.189+0000 7ff24dffb640 10 client.25907 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.204+0000 7ff24dffb640 1 -- 172.21.15.64:0/655501491 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff25c0111d0 con 0x557983d98660 2024-05-09T09:12:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.204+0000 7ff24dffb640 10 client.25907 handle_command_reply: tid=0 2024-05-09T09:12:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.204+0000 7ff26f38a740 1 client.25907 shutdown 2024-05-09T09:12:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 msgr2=0x557983d2d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d2d850 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff25c01ff80 tx=0x7ff25c0118e0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 shutdown_connections 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557983d98660 0x557983d9aa80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557983d21db0 0x557983d2a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557983d1cc40 0x557983d2a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 --2- 172.21.15.64:0/655501491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557983d1c250 0x557983d2d850 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff25c01ff80 tx=0x7ff25c0118e0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 >> 172.21.15.64:0/655501491 conn(0x557983d178c0 msgr2=0x557983d1dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff24dffb640 0 client.25907 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 shutdown_connections 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 1 -- 172.21.15.64:0/655501491 wait complete. 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.207+0000 7ff26f38a740 20 client.25907 trim_cache size 0 max 16384 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:12:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2500a2df0 msgr2=0x7ff2500a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2500a2df0 0x7ff2500a52b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557983d1d040 tx=0x7ff25c0097d0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 msgr2=0x7ff264072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 0x7ff264072f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff26000e970 tx=0x7ff26000ed70 comp rx=0 tx=0).stop 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 shutdown_connections 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2500a2df0 0x7ff2500a52b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff264105670 0x7ff264072f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff264104cb0 0x7ff26406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 --2- 172.21.15.64:0/3646462707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff264103ac0 0x7ff26406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 >> 172.21.15.64:0/3646462707 conn(0x7ff2640ff050 msgr2=0x7ff264079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 shutdown_connections 2024-05-09T09:12:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:18.209+0000 7ff26c683640 1 -- 172.21.15.64:0/3646462707 wait complete. 2024-05-09T09:12:18.529 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:18 smithi179 ceph-mon[34086]: pgmap v465: 129 pgs: 129 active+clean; 993 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-05-09T09:12:18.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:18 smithi139 ceph-mon[33599]: pgmap v465: 129 pgs: 129 active+clean; 993 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-05-09T09:12:18.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:18 smithi064 ceph-mon[30149]: pgmap v465: 129 pgs: 129 active+clean; 993 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-05-09T09:12:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:20 smithi139 ceph-mon[33599]: pgmap v466: 129 pgs: 129 active+clean; 993 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-05-09T09:12:20.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:20 smithi064 ceph-mon[30149]: pgmap v466: 129 pgs: 129 active+clean; 993 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-05-09T09:12:20.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:20 smithi179 ceph-mon[34086]: pgmap v466: 129 pgs: 129 active+clean; 993 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-05-09T09:12:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 120.0 s, 968.3 tps, lat 103.240 ms stddev 33.680 2024-05-09T09:12:22.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:22 smithi139 ceph-mon[33599]: pgmap v467: 129 pgs: 129 active+clean; 993 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-05-09T09:12:22.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:22 smithi064 ceph-mon[30149]: pgmap v467: 129 pgs: 129 active+clean; 993 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-05-09T09:12:22.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:22 smithi179 ceph-mon[34086]: pgmap v467: 129 pgs: 129 active+clean; 993 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-05-09T09:12:24.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:24 smithi139 ceph-mon[33599]: pgmap v468: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 1.11k op/s 2024-05-09T09:12:24.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:24 smithi064 ceph-mon[30149]: pgmap v468: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 1.11k op/s 2024-05-09T09:12:24.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:24 smithi179 ceph-mon[34086]: pgmap v468: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 1.11k op/s 2024-05-09T09:12:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 125.0 s, 1154.3 tps, lat 86.710 ms stddev 25.058 2024-05-09T09:12:26.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:26 smithi139 ceph-mon[33599]: pgmap v469: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:26.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:26 smithi064 ceph-mon[30149]: pgmap v469: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:26.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:26 smithi179 ceph-mon[34086]: pgmap v469: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:27.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:12:27.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 -- 172.21.15.64:0/705894433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc10b1a0 msgr2=0x7f1ccc10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 --2- 172.21.15.64:0/705894433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc10b1a0 0x7f1ccc10d630 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009900 tx=0x7f1cbc02a410 comp rx=0 tx=0).stop 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 -- 172.21.15.64:0/705894433 shutdown_connections 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 --2- 172.21.15.64:0/705894433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ccc10db70 0x7f1ccc10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 --2- 172.21.15.64:0/705894433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc10b1a0 0x7f1ccc10d630 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 --2- 172.21.15.64:0/705894433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc108830 0x7f1ccc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.040+0000 7f1cd3858640 1 -- 172.21.15.64:0/705894433 >> 172.21.15.64:0/705894433 conn(0x7f1ccc06e6c0 msgr2=0x7f1ccc070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.041+0000 7f1cd3858640 1 -- 172.21.15.64:0/705894433 shutdown_connections 2024-05-09T09:12:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.041+0000 7f1cd3858640 1 -- 172.21.15.64:0/705894433 wait complete. 2024-05-09T09:12:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.042+0000 7f1cd3858640 1 Processor -- start 2024-05-09T09:12:28.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 -- start start 2024-05-09T09:12:28.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc108830 0x7f1ccc115000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ccc10b1a0 0x7f1ccc115540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 0x7f1ccc1b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc11c1a0 con 0x7f1ccc108830 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc11c020 con 0x7f1ccc10db70 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.043+0000 7f1cd3858640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc11c320 con 0x7f1ccc10b1a0 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 0x7f1ccc1b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd15f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc108830 0x7f1ccc115000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 0x7f1ccc1b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35122/0 (socket says 172.21.15.64:35122) 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 -- 172.21.15.64:0/4185474330 learned_addr learned my addr 172.21.15.64:0/4185474330 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 -- 172.21.15.64:0/4185474330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ccc10b1a0 msgr2=0x7f1ccc115540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ccc10b1a0 0x7f1ccc115540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 -- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc108830 msgr2=0x7f1ccc115000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc108830 0x7f1ccc115000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 -- 172.21.15.64:0/4185474330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc1b6dd0 con 0x7f1ccc10db70 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.044+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 0x7f1ccc1b6890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400eb40 tx=0x7f1cc400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.045+0000 7f1cc27fc640 1 -- 172.21.15.64:0/4185474330 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4004b20 con 0x7f1ccc10db70 2024-05-09T09:12:28.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.045+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc1b7060 con 0x7f1ccc10db70 2024-05-09T09:12:28.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.045+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1b7580 con 0x7f1ccc10db70 2024-05-09T09:12:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.049+0000 7f1cc27fc640 1 -- 172.21.15.64:0/4185474330 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cc4004cc0 con 0x7f1ccc10db70 2024-05-09T09:12:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.049+0000 7f1cc27fc640 1 -- 172.21.15.64:0/4185474330 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40107b0 con 0x7f1ccc10db70 2024-05-09T09:12:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 Processor -- start 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 -- start start 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 0x56085ae5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56085adbad00 con 0x56085ae55f30 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56085ad624c0 con 0x56085ae5ba90 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.050+0000 7f1cd6563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56085adb6cc0 con 0x56085ae56920 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 0x56085ae5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd0df3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 0x56085ae5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43644/0 (socket says 172.21.15.64:43644) 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 -- 172.21.15.64:0/3032154226 learned_addr learned my addr 172.21.15.64:0/3032154226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd0df3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae56350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45258/0 (socket says 172.21.15.64:45258) 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae5df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 -- 172.21.15.64:0/3032154226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 msgr2=0x56085ae5df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae5df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 -- 172.21.15.64:0/3032154226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 msgr2=0x56085ae56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd15f4640 1 -- 172.21.15.64:0/3032154226 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085ab987d0 con 0x56085ae56920 2024-05-09T09:12:28.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.051+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:12:28.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.052+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 0x56085ae5b420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800bd50 tx=0x7f1cc80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5193fcf54f3a19be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.053+0000 7f1cb77fe640 1 -- 172.21.15.64:0/3032154226 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8005220 con 0x56085ae56920 2024-05-09T09:12:28.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.053+0000 7f1cb77fe640 1 -- 172.21.15.64:0/3032154226 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cc800be40 con 0x56085ae56920 2024-05-09T09:12:28.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.053+0000 7f1cb77fe640 1 -- 172.21.15.64:0/3032154226 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80117c0 con 0x56085ae56920 2024-05-09T09:12:28.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.054+0000 7f1cc27fc640 1 -- 172.21.15.64:0/4185474330 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1cc4005020 con 0x7f1ccc10db70 2024-05-09T09:12:28.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 -- 172.21.15.64:0/3032154226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 msgr2=0x56085ae5b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 0x56085ae5b420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800bd50 tx=0x7f1cc80107a0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cc27fc640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1cb80a2c90 0x7f1cb80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 -- 172.21.15.64:0/3032154226 shutdown_connections 2024-05-09T09:12:28.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae56920 0x56085ae5b420 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800bd50 tx=0x7f1cc80107a0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3032154226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.055+0000 7f1cd6563740 1 -- 172.21.15.64:0/3032154226 >> 172.21.15.64:0/3032154226 conn(0x56085ae515a0 msgr2=0x56085ae519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.056+0000 7f1cc27fc640 1 -- 172.21.15.64:0/4185474330 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1cc40cfa20 con 0x7f1ccc10db70 2024-05-09T09:12:28.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.056+0000 7f1cd6563740 1 -- 172.21.15.64:0/3032154226 shutdown_connections 2024-05-09T09:12:28.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.056+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1cb80a2c90 0x7f1cb80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.057+0000 7f1cd6563740 1 -- 172.21.15.64:0/3032154226 wait complete. 2024-05-09T09:12:28.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.057+0000 7f1cd6563740 1 -- start start 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 Processor -- start 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1cb80a2c90 0x7f1cb80a5150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009230 tx=0x7f1cbc005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 0x56085ae60560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 0x56085ae60aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae67ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56085ae5ea40 con 0x56085ae5ba90 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56085ad75b90 con 0x56085ae55f30 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56085ae5ebc0 con 0x56085ae65ad0 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.058+0000 7f1cd6563740 10 client.0 _ll_register_callbacks cb 0x56085adb1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 0x56085ae60aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 0x56085ae60aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45272/0 (socket says 172.21.15.64:45272) 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 -- 172.21.15.64:0/1224286991 learned_addr learned my addr 172.21.15.64:0/1224286991 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 -- 172.21.15.64:0/1224286991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 msgr2=0x56085ae67ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae67ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 -- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 msgr2=0x56085ae60560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 0x56085ae60560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.059+0000 7f1cd0df3640 1 -- 172.21.15.64:0/1224286991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085aace3f0 con 0x56085ae5ba90 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.060+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 0x56085ae60aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009900 tx=0x7f1cbc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7b0b19d042551e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.060+0000 7f1cb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.060+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/1224286991 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc04d070 con 0x56085ae5ba90 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.061+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/1224286991 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1cbc0316d0 con 0x56085ae5ba90 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.061+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56085adc8430 con 0x56085ae5ba90 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.061+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/1224286991 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc057050 con 0x56085ae5ba90 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.061+0000 7f1cd6563740 10 client.16401 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.061+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56085aa8e0f0 con 0x56085ae5ba90 2024-05-09T09:12:28.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/1224286991 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cbc031d80 con 0x56085ae5ba90 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/1224286991 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1cbc03a040 con 0x56085ae5ba90 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cd6563740 10 client.16401 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cd6563740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cd6563740 20 client.16401 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cd6563740 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085ae6c9a0 0x56085ae6edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cd6563740 4 client.16401 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.062+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56085adc8430 con 0x56085ae6c9a0 2024-05-09T09:12:28.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.064+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085ae6c9a0 0x56085ae6edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.066+0000 7f1cb4ff9640 10 client.16401 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:28.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.066+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085ae6c9a0 0x56085ae6edc0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f8453eef35037d server_cookie=c4c364aa5d017175 in_seq=0 out_seq=0 2024-05-09T09:12:28.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.066+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/1224286991 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56085adc8430 con 0x56085ae6c9a0 2024-05-09T09:12:28.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.066+0000 7f1cb4ff9640 10 client.16401 handle_command_reply: tid=0 2024-05-09T09:12:28.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.067+0000 7f1cd6563740 1 client.16401 shutdown 2024-05-09T09:12:28.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.067+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 msgr2=0x56085ae60aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.067+0000 7f1cd6563740 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 0x56085ae60aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009900 tx=0x7f1cbc02a410 comp rx=0 tx=0).stop 2024-05-09T09:12:28.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 shutdown_connections 2024-05-09T09:12:28.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085ae6c9a0 0x56085ae6edc0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae67ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae5ba90 0x56085ae60aa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 --2- 172.21.15.64:0/1224286991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 0x56085ae60560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 >> 172.21.15.64:0/1224286991 conn(0x56085ae515a0 msgr2=0x56085ae507e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cb4ff9640 0 client.16401 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 shutdown_connections 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 1 -- 172.21.15.64:0/1224286991 wait complete. 2024-05-09T09:12:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.068+0000 7f1cd6563740 20 client.16401 trim_cache size 0 max 16384 2024-05-09T09:12:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.108+0000 7f1cd6563740 1 Processor -- start 2024-05-09T09:12:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 -- start start 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae740e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00125f0 con 0x56085ae55f30 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb000fe30 con 0x56085ae5ba90 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.109+0000 7f1cd6563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb000dc80 con 0x56085ae65ad0 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43652/0 (socket says 172.21.15.64:43652) 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 -- 172.21.15.64:0/3731659838 learned_addr learned my addr 172.21.15.64:0/3731659838 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 -- 172.21.15.64:0/3731659838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 msgr2=0x56085ae73ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae73ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 -- 172.21.15.64:0/3731659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 msgr2=0x56085ae73660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:12:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae73660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 -- 172.21.15.64:0/3731659838 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085ae5b170 con 0x56085ae65ad0 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae740e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1cc401a040 tx=0x7f1cc409ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4a896ddbe1d327 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.110+0000 7f1cb6ffd640 1 -- 172.21.15.64:0/3731659838 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc409be90 con 0x56085ae65ad0 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.111+0000 7f1cb6ffd640 1 -- 172.21.15.64:0/3731659838 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cc40a0240 con 0x56085ae65ad0 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.111+0000 7f1cb6ffd640 1 -- 172.21.15.64:0/3731659838 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40a48b0 con 0x56085ae65ad0 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 -- 172.21.15.64:0/3731659838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 msgr2=0x56085ae740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae740e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1cc401a040 tx=0x7f1cc409ba90 comp rx=0 tx=0).stop 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 -- 172.21.15.64:0/3731659838 shutdown_connections 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae65ad0 0x56085ae740e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae5ba90 0x56085ae73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 --2- 172.21.15.64:0/3731659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae55f30 0x56085ae73660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.112+0000 7f1cd6563740 1 -- 172.21.15.64:0/3731659838 >> 172.21.15.64:0/3731659838 conn(0x56085ae515a0 msgr2=0x56085ae54e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.113+0000 7f1cd6563740 1 -- 172.21.15.64:0/3731659838 shutdown_connections 2024-05-09T09:12:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.113+0000 7f1cd6563740 1 -- 172.21.15.64:0/3731659838 wait complete. 2024-05-09T09:12:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.114+0000 7f1cd6563740 1 -- start start 2024-05-09T09:12:28.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.114+0000 7f1cd6563740 1 Processor -- start 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 0x56085ae80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae5ba90 0x56085aece9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 0x56085aed0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb000e2f0 con 0x56085ae65ad0 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb000f760 con 0x56085ae55f30 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb000e9c0 con 0x56085ae5ba90 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd6563740 10 client.0 _ll_register_callbacks cb 0x7f1cc40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 0x56085aed0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 0x56085aed0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45278/0 (socket says 172.21.15.64:45278) 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 -- 172.21.15.64:0/2559444483 learned_addr learned my addr 172.21.15.64:0/2559444483 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:28.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd0df3640 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae5ba90 0x56085aece9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 -- 172.21.15.64:0/2559444483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae5ba90 msgr2=0x56085aece9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae5ba90 0x56085aece9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 -- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 msgr2=0x56085ae80160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 0x56085ae80160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.115+0000 7f1cd1df5640 1 -- 172.21.15.64:0/2559444483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085aed17a0 con 0x56085ae65ad0 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.116+0000 7f1cd1df5640 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 0x56085aed0f70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4004af0 tx=0x7f1cc40a4b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b808323d6430aec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.116+0000 7f1cb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.117+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/2559444483 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40cd240 con 0x56085ae65ad0 2024-05-09T09:12:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.117+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/2559444483 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1cc409b2e0 con 0x56085ae65ad0 2024-05-09T09:12:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.117+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc40a0240 con 0x56085ae65ad0 2024-05-09T09:12:28.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.118+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/2559444483 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cc409b520 con 0x56085ae65ad0 2024-05-09T09:12:28.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.118+0000 7f1cd6563740 10 client.16404 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:28.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.118+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56085aed2ab0 con 0x56085ae65ad0 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/2559444483 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cc409ad50 con 0x56085ae65ad0 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/2559444483 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1cc40a14e0 con 0x56085ae65ad0 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd6563740 10 client.16404 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd6563740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd6563740 20 client.16404 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd6563740 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085aed33f0 0x56085aed5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd6563740 4 client.16404 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:12:28.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1cc40a0240 con 0x56085aed33f0 2024-05-09T09:12:28.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.119+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085aed33f0 0x56085aed5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.121+0000 7f1cd15f4640 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085aed33f0 0x56085aed5810 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b222110eece0be server_cookie=8285743b05101b7b in_seq=0 out_seq=0 2024-05-09T09:12:28.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.121+0000 7f1cb4ff9640 10 client.16404 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:28.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.124+0000 7f1cb4ff9640 1 -- 172.21.15.64:0/2559444483 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f1cc40a0240 con 0x56085aed33f0 2024-05-09T09:12:28.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.124+0000 7f1cb4ff9640 10 client.16404 handle_command_reply: tid=0 2024-05-09T09:12:28.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.124+0000 7f1cd6563740 1 client.16404 shutdown 2024-05-09T09:12:28.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.125+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 msgr2=0x56085aed0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.125+0000 7f1cd6563740 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 0x56085aed0f70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4004af0 tx=0x7f1cc40a4b00 comp rx=0 tx=0).stop 2024-05-09T09:12:28.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 shutdown_connections 2024-05-09T09:12:28.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56085aed33f0 0x56085aed5810 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56085ae65ad0 0x56085aed0f70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085ae5ba90 0x56085aece9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 --2- 172.21.15.64:0/2559444483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56085ae55f30 0x56085ae80160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 >> 172.21.15.64:0/2559444483 conn(0x56085ae515a0 msgr2=0x56085ae4ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cb4ff9640 0 client.16404 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:28.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 shutdown_connections 2024-05-09T09:12:28.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 1 -- 172.21.15.64:0/2559444483 wait complete. 2024-05-09T09:12:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.126+0000 7f1cd6563740 20 client.16404 trim_cache size 0 max 16384 2024-05-09T09:12:28.126 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:12:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:12:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:12:28.126 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:12:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1cb80a2c90 msgr2=0x7f1cb80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1cb80a2c90 0x7f1cb80a5150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009230 tx=0x7f1cbc005d50 comp rx=0 tx=0).stop 2024-05-09T09:12:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 msgr2=0x7f1ccc1b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 0x7f1ccc1b6890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400eb40 tx=0x7f1cc400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 shutdown_connections 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1cb80a2c90 0x7f1cb80a5150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ccc10db70 0x7f1ccc1b6890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ccc10b1a0 0x7f1ccc115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 --2- 172.21.15.64:0/4185474330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc108830 0x7f1ccc115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 >> 172.21.15.64:0/4185474330 conn(0x7f1ccc06e6c0 msgr2=0x7f1ccc108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.128+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 shutdown_connections 2024-05-09T09:12:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.129+0000 7f1cd3858640 1 -- 172.21.15.64:0/4185474330 wait complete. 2024-05-09T09:12:28.150 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:12:28.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 -- 172.21.15.64:0/2221240073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 msgr2=0x7fe6bc105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/2221240073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 0x7fe6bc105880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6b401acb0 tx=0x7fe6b403b8e0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 -- 172.21.15.64:0/2221240073 shutdown_connections 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/2221240073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 0x7fe6bc105880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/2221240073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc102a50 0x7fe6bc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/2221240073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 -- 172.21.15.64:0/2221240073 >> 172.21.15.64:0/2221240073 conn(0x7fe6bc0fbce0 msgr2=0x7fe6bc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 -- 172.21.15.64:0/2221240073 shutdown_connections 2024-05-09T09:12:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.340+0000 7fe6c46b3640 1 -- 172.21.15.64:0/2221240073 wait complete. 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 Processor -- start 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 -- start start 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc102a50 0x7fe6bc195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 0x7fe6bc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc1091a0 con 0x7fe6bc102a50 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc109020 con 0x7fe6bc1000e0 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c46b3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc109320 con 0x7fe6bc105400 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.341+0000 7fe6c1c4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc102a50 0x7fe6bc195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c1c4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc102a50 0x7fe6bc195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45288/0 (socket says 172.21.15.64:45288) 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35176/0 (socket says 172.21.15.64:35176) 2024-05-09T09:12:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c1c4e640 1 -- 172.21.15.64:0/1594392520 learned_addr learned my addr 172.21.15.64:0/1594392520 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 -- 172.21.15.64:0/1594392520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 msgr2=0x7fe6bc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 0x7fe6bc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 -- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc102a50 msgr2=0x7fe6bc195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc102a50 0x7fe6bc195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.342+0000 7fe6c244f640 1 -- 172.21.15.64:0/1594392520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc19d450 con 0x7fe6bc1000e0 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.343+0000 7fe6c244f640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc195880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00d6d0 tx=0x7fe6ac00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.343+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac017070 con 0x7fe6bc1000e0 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.343+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6ac009820 con 0x7fe6bc1000e0 2024-05-09T09:12:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.343+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004ba0 con 0x7fe6bc1000e0 2024-05-09T09:12:28.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.344+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6bc19d740 con 0x7fe6bc1000e0 2024-05-09T09:12:28.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.344+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6bc19dc10 con 0x7fe6bc1000e0 2024-05-09T09:12:28.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.347+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6bc19de70 con 0x7fe6bc1000e0 2024-05-09T09:12:28.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.347+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe6ac0099d0 con 0x7fe6bc1000e0 2024-05-09T09:12:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.348+0000 7fe6b37fe640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6a80a2c60 0x7fe6a80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.349+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe6ac0cf290 con 0x7fe6bc1000e0 2024-05-09T09:12:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.349+0000 7fe6c1c4e640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6a80a2c60 0x7fe6a80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.351+0000 7fe6c1c4e640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6a80a2c60 0x7fe6a80a5120 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc196d40 tx=0x7fe6b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.354+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe6ac020020 con 0x7fe6bc1000e0 2024-05-09T09:12:28.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:28 smithi064 ceph-mon[30149]: pgmap v470: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.4 MiB/s wr, 1.00k op/s 2024-05-09T09:12:28.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:28 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:12:28.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:28 smithi139 ceph-mon[33599]: pgmap v470: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.4 MiB/s wr, 1.00k op/s 2024-05-09T09:12:28.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:28 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:12:28.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:28 smithi179 ceph-mon[34086]: pgmap v470: 129 pgs: 129 active+clean; 993 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.4 MiB/s wr, 1.00k op/s 2024-05-09T09:12:28.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:28 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:12:28.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.829+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6bc19e040 con 0x7fe6bc1000e0 2024-05-09T09:12:28.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.833+0000 7fe6b37fe640 1 -- 172.21.15.64:0/1594392520 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11884 (secure 0 0 0) 0x7fe6ac09be00 con 0x7fe6bc1000e0 2024-05-09T09:12:28.835 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:12:28.835 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:12:10.080639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:12:28.835 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 31 2024-05-09T09:12:28.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.841+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6a80a2c60 msgr2=0x7fe6a80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6a80a2c60 0x7fe6a80a5120 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc196d40 tx=0x7fe6b8005f90 comp rx=0 tx=0).stop 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 msgr2=0x7fe6bc195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc195880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00d6d0 tx=0x7fe6ac00db90 comp rx=0 tx=0).stop 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 shutdown_connections 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6a80a2c60 0x7fe6a80a5120 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc105400 0x7fe6bc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.842+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6bc102a50 0x7fe6bc195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.843+0000 7fe6c46b3640 1 --2- 172.21.15.64:0/1594392520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6bc1000e0 0x7fe6bc195880 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.843+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 >> 172.21.15.64:0/1594392520 conn(0x7fe6bc0fbce0 msgr2=0x7fe6bc0fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.843+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 shutdown_connections 2024-05-09T09:12:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:28.843+0000 7fe6c46b3640 1 -- 172.21.15.64:0/1594392520 wait complete. 2024-05-09T09:12:28.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:12:29.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2538897200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b80968a0 msgr2=0x7fc8b8098c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2538897200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b80968a0 0x7fc8b8098c90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009860 tx=0x7fc8b402a280 comp rx=0 tx=0).stop 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2538897200 shutdown_connections 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2538897200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b809bb00 0x7fc8b809def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2538897200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80991d0 0x7fc8b809b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2538897200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b80968a0 0x7fc8b8098c90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.063+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2538897200 >> 172.21.15.64:0/2538897200 conn(0x7fc8b80901d0 msgr2=0x7fc8b8092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.064+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2538897200 shutdown_connections 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.064+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2538897200 wait complete. 2024-05-09T09:12:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 Processor -- start 2024-05-09T09:12:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 -- start start 2024-05-09T09:12:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80968a0 0x7fc8b8132600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 0x7fc8b8132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b809bb00 0x7fc8b8139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b80a3ae0 con 0x7fc8b80991d0 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b80a3960 con 0x7fc8b80968a0 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8c67a9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b80a3c60 con 0x7fc8b809bb00 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 0x7fc8b8132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 0x7fc8b8132b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45306/0 (socket says 172.21.15.64:45306) 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.065+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/2659634077 learned_addr learned my addr 172.21.15.64:0/2659634077 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8c4d46640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b809bb00 0x7fc8b8139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8bffff640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80968a0 0x7fc8b8132600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8c4d46640 1 -- 172.21.15.64:0/2659634077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80968a0 msgr2=0x7fc8b8132600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8c4d46640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80968a0 0x7fc8b8132600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8c4d46640 1 -- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 msgr2=0x7fc8b8132b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8c4d46640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 0x7fc8b8132b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8c4d46640 1 -- 172.21.15.64:0/2659634077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b813a220 con 0x7fc8b809bb00 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.066+0000 7fc8bffff640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80968a0 0x7fc8b8132600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.067+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 0x7fc8b8132b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.067+0000 7fc8c4d46640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b809bb00 0x7fc8b8139bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8c004f8c0 tx=0x7fc8c007ae70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.068+0000 7fc8bd7fa640 1 -- 172.21.15.64:0/2659634077 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0085070 con 0x7fc8b809bb00 2024-05-09T09:12:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.068+0000 7fc8bd7fa640 1 -- 172.21.15.64:0/2659634077 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8c0080460 con 0x7fc8b809bb00 2024-05-09T09:12:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.068+0000 7fc8bd7fa640 1 -- 172.21.15.64:0/2659634077 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0069330 con 0x7fc8b809bb00 2024-05-09T09:12:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.068+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b813a510 con 0x7fc8b809bb00 2024-05-09T09:12:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.068+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b813a9e0 con 0x7fc8b809bb00 2024-05-09T09:12:29.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.072+0000 7fc8c959a740 1 Processor -- start 2024-05-09T09:12:29.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.073+0000 7fc8bd7fa640 1 -- 172.21.15.64:0/2659634077 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8c0088070 con 0x7fc8b809bb00 2024-05-09T09:12:29.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.073+0000 7fc8c959a740 1 -- start start 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.073+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 0x55bbea1b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.073+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.073+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1bbc60 0x55bbea1be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.073+0000 7fc8c959a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bbea11ac40 con 0x55bbea1b6af0 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.074+0000 7fc8c959a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bbea0c2580 con 0x55bbea1b6100 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.074+0000 7fc8c959a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bbea116c00 con 0x55bbea1bbc60 2024-05-09T09:12:29.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.074+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 0x55bbea1b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.074+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 0x55bbea1b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35206/0 (socket says 172.21.15.64:35206) 2024-05-09T09:12:29.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.074+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3066088692 learned_addr learned my addr 172.21.15.64:0/3066088692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1bbc60 0x55bbea1be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bd7fa640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8b00a2c90 0x7fc8b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3066088692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1bbc60 msgr2=0x55bbea1be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1bbc60 0x55bbea1be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3066088692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 msgr2=0x55bbea1bb5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:12:29.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1bb5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3066088692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe9e2e3f0 con 0x55bbea1b6100 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.075+0000 7fc8bd7fa640 1 -- 172.21.15.64:0/2659634077 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8c013a990 con 0x7fc8b809bb00 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.076+0000 7fc8c4d46640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8b00a2c90 0x7fc8b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.076+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 0x55bbea1b6520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac013880 tx=0x7fc8ac009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4ef6200cc64562 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.076+0000 7fc8aa7fc640 1 -- 172.21.15.64:0/3066088692 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac013e00 con 0x55bbea1b6100 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.076+0000 7fc8aa7fc640 1 -- 172.21.15.64:0/3066088692 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8ac0130d0 con 0x55bbea1b6100 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.076+0000 7fc8aa7fc640 1 -- 172.21.15.64:0/3066088692 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac011890 con 0x55bbea1b6100 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c4d46640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8b00a2c90 0x7fc8b00a5150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc8c010f540 tx=0x7fc8c0075040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 -- 172.21.15.64:0/3066088692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 msgr2=0x55bbea1b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 0x55bbea1b6520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac013880 tx=0x7fc8ac009760 comp rx=0 tx=0).stop 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 -- 172.21.15.64:0/3066088692 shutdown_connections 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1bbc60 0x55bbea1be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3066088692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1b6100 0x55bbea1b6520 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac013880 tx=0x7fc8ac009760 comp rx=0 tx=0).stop 2024-05-09T09:12:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 -- 172.21.15.64:0/3066088692 >> 172.21.15.64:0/3066088692 conn(0x55bbea1b1770 msgr2=0x55bbea1b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 -- 172.21.15.64:0/3066088692 shutdown_connections 2024-05-09T09:12:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.078+0000 7fc8c959a740 1 -- 172.21.15.64:0/3066088692 wait complete. 2024-05-09T09:12:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.079+0000 7fc8c959a740 1 -- start start 2024-05-09T09:12:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.079+0000 7fc8c959a740 1 Processor -- start 2024-05-09T09:12:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 0x55bbea1d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1d6ca0 0x55bbea1d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bbea1cfbc0 con 0x55bbea1d6ca0 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bbea0d5c40 con 0x55bbea1bbc60 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bbea1cfd40 con 0x55bbea1b6af0 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8c959a740 10 client.0 _ll_register_callbacks cb 0x55bbe9e2dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 0x55bbea1d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 0x55bbea1d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43706/0 (socket says 172.21.15.64:43706) 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3191155017 learned_addr learned my addr 172.21.15.64:0/3191155017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.080+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1d6ca0 0x55bbea1d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3191155017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 msgr2=0x55bbea1d1c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:29.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1d1c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3191155017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1d6ca0 msgr2=0x55bbea1d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1d6ca0 0x55bbea1d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/3191155017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe9ef87d0 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 0x55bbea1d1730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55bbea1b7350 tx=0x7fc8ac009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=247734cd8a7b149b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.081+0000 7fc89f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.082+0000 7fc89f7fe640 1 -- 172.21.15.64:0/3191155017 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac013280 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.082+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ac0130d0 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.082+0000 7fc89f7fe640 1 -- 172.21.15.64:0/3191155017 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8ac029040 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.082+0000 7fc89f7fe640 1 -- 172.21.15.64:0/3191155017 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc8ac004030 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.082+0000 7fc8c959a740 10 client.26077 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.082+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bbea1dc400 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.083+0000 7fc89f7fe640 1 -- 172.21.15.64:0/3191155017 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8ac011820 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.083+0000 7fc89f7fe640 1 -- 172.21.15.64:0/3191155017 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc8ac0035f0 con 0x55bbea1b6af0 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.083+0000 7fc8c959a740 10 client.26077 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:29.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.084+0000 7fc8c959a740 10 client.26077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:12:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.084+0000 7fc8c959a740 20 client.26077 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.084+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea1ddde0 0x55bbea1e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.084+0000 7fc8c959a740 4 client.26077 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.084+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8ac0130d0 con 0x55bbea1ddde0 2024-05-09T09:12:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.084+0000 7fc8bffff640 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea1ddde0 0x55bbea1e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.085+0000 7fc8bffff640 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea1ddde0 0x55bbea1e0220 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aec4a241a3e12be4 server_cookie=138536bf4f862788 in_seq=0 out_seq=0 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.085+0000 7fc89f7fe640 10 client.26077 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc89f7fe640 1 -- 172.21.15.64:0/3191155017 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc8ac0130d0 con 0x55bbea1ddde0 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc89f7fe640 10 client.26077 handle_command_reply: tid=0 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 client.26077 shutdown 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 msgr2=0x55bbea1d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 0x55bbea1d1730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55bbea1b7350 tx=0x7fc8ac009fb0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 shutdown_connections 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea1ddde0 0x55bbea1e0220 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1d6ca0 0x55bbea1d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.086+0000 7fc8c959a740 1 --2- 172.21.15.64:0/3191155017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1b6af0 0x55bbea1d1730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.087+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 >> 172.21.15.64:0/3191155017 conn(0x55bbea1b1770 msgr2=0x55bbea1b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.087+0000 7fc89f7fe640 0 client.26077 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:12:29.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.087+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 shutdown_connections 2024-05-09T09:12:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.087+0000 7fc8c959a740 1 -- 172.21.15.64:0/3191155017 wait complete. 2024-05-09T09:12:29.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.088+0000 7fc8c959a740 20 client.26077 trim_cache size 0 max 16384 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.117+0000 7fc8c959a740 1 Processor -- start 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.117+0000 7fc8c959a740 1 -- start start 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.118+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.118+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.118+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.118+0000 7fc8c959a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b813f150 con 0x55bbea1b6af0 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.118+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.119+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45320/0 (socket says 172.21.15.64:45320) 2024-05-09T09:12:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.119+0000 7fc8bf7fe640 1 -- 172.21.15.64:0/1658201946 learned_addr learned my addr 172.21.15.64:0/1658201946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.119+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b813c990 con 0x55bbea1bbc60 2024-05-09T09:12:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.119+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b8131470 con 0x55bbea1d6ca0 2024-05-09T09:12:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.119+0000 7fc8bffff640 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.119+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8bffff640 1 -- 172.21.15.64:0/1658201946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 msgr2=0x55bbea1b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8bffff640 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8bffff640 1 -- 172.21.15.64:0/1658201946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 msgr2=0x55bbea1b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8bffff640 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8bffff640 1 -- 172.21.15.64:0/1658201946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbea1b7380 con 0x55bbea1bbc60 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8bffff640 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1b81a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4035040 tx=0x7fc8b4005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b354b49232390f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.120+0000 7fc8a9ffb640 1 -- 172.21.15.64:0/1658201946 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4042e60 con 0x55bbea1bbc60 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.121+0000 7fc8a9ffb640 1 -- 172.21.15.64:0/1658201946 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8b4046070 con 0x55bbea1bbc60 2024-05-09T09:12:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.121+0000 7fc8a9ffb640 1 -- 172.21.15.64:0/1658201946 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4037040 con 0x55bbea1bbc60 2024-05-09T09:12:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 msgr2=0x55bbea1b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1b81a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4035040 tx=0x7fc8b4005a90 comp rx=0 tx=0).stop 2024-05-09T09:12:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 shutdown_connections 2024-05-09T09:12:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1b81a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 --2- 172.21.15.64:0/1658201946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 >> 172.21.15.64:0/1658201946 conn(0x55bbea1b1770 msgr2=0x55bbea1ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 shutdown_connections 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 -- 172.21.15.64:0/1658201946 wait complete. 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.122+0000 7fc8c959a740 1 -- start start 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 Processor -- start 2024-05-09T09:12:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1e5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1e4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b813ae50 con 0x55bbea1b6af0 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b813c2c0 con 0x55bbea1bbc60 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b813b520 con 0x55bbea1d6ca0 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.123+0000 7fc8c959a740 10 client.0 _ll_register_callbacks cb 0x7fc8ac009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35226/0 (socket says 172.21.15.64:35226) 2024-05-09T09:12:29.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 -- 172.21.15.64:0/2347815063 learned_addr learned my addr 172.21.15.64:0/2347815063 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 -- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 msgr2=0x55bbea1e48d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:29.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1e48d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 -- 172.21.15.64:0/2347815063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 msgr2=0x55bbea1e5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1e5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.124+0000 7fc8bffff640 1 -- 172.21.15.64:0/2347815063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbea1e4f40 con 0x55bbea1bbc60 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.125+0000 7fc8bffff640 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1e4390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4000c00 tx=0x7fc8b4030a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc32f9bf1c958b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.125+0000 7fc89f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.125+0000 7fc89f7fe640 1 -- 172.21.15.64:0/2347815063 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4037050 con 0x55bbea1bbc60 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.125+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b4046070 con 0x55bbea1bbc60 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.125+0000 7fc89f7fe640 1 -- 172.21.15.64:0/2347815063 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8b4004240 con 0x55bbea1bbc60 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.126+0000 7fc89f7fe640 1 -- 172.21.15.64:0/2347815063 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc8b4042e60 con 0x55bbea1bbc60 2024-05-09T09:12:29.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.126+0000 7fc8c959a740 10 client.25937 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.126+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bbea243260 con 0x55bbea1bbc60 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc89f7fe640 1 -- 172.21.15.64:0/2347815063 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8b4046070 con 0x55bbea1bbc60 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc89f7fe640 1 -- 172.21.15.64:0/2347815063 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc8b40036f0 con 0x55bbea1bbc60 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc8c959a740 10 client.25937 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc8c959a740 10 client.25937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc8c959a740 20 client.25937 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:29.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc8c959a740 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea243b70 0x55bbea245f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc8c959a740 4 client.25937 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:12:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.127+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbea128370 con 0x55bbea243b70 2024-05-09T09:12:29.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.128+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea243b70 0x55bbea245f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.128+0000 7fc8bf7fe640 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea243b70 0x55bbea245f90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45ad9ac8fa64e72c server_cookie=7eb4549ce76e9698 in_seq=0 out_seq=0 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.128+0000 7fc89f7fe640 10 client.25937 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc89f7fe640 1 -- 172.21.15.64:0/2347815063 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbea128370 con 0x55bbea243b70 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc89f7fe640 10 client.25937 handle_command_reply: tid=0 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 client.25937 shutdown 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 msgr2=0x55bbea1e4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1e4390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4000c00 tx=0x7fc8b4030a70 comp rx=0 tx=0).stop 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 shutdown_connections 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55bbea243b70 0x55bbea245f90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbea1d6ca0 0x55bbea1e48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbea1bbc60 0x55bbea1e4390 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4000c00 tx=0x7fc8b4030a70 comp rx=0 tx=0).stop 2024-05-09T09:12:29.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 --2- 172.21.15.64:0/2347815063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbea1b6af0 0x55bbea1e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 >> 172.21.15.64:0/2347815063 conn(0x55bbea1b1770 msgr2=0x55bbea1ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc89f7fe640 0 client.25937 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:12:29.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.129+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 shutdown_connections 2024-05-09T09:12:29.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.130+0000 7fc8c959a740 1 -- 172.21.15.64:0/2347815063 wait complete. 2024-05-09T09:12:29.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.130+0000 7fc8c959a740 20 client.25937 trim_cache size 0 max 16384 2024-05-09T09:12:29.130 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.131+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8b00a2c90 msgr2=0x7fc8b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.131+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8b00a2c90 0x7fc8b00a5150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc8c010f540 tx=0x7fc8c0075040 comp rx=0 tx=0).stop 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.131+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b809bb00 msgr2=0x7fc8b8139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.131+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b809bb00 0x7fc8b8139bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8c004f8c0 tx=0x7fc8c007ae70 comp rx=0 tx=0).stop 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 shutdown_connections 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8b00a2c90 0x7fc8b00a5150 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b809bb00 0x7fc8b8139bb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8b80991d0 0x7fc8b8132b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 --2- 172.21.15.64:0/2659634077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b80968a0 0x7fc8b8132600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 >> 172.21.15.64:0/2659634077 conn(0x7fc8b80901d0 msgr2=0x7fc8b80986b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 shutdown_connections 2024-05-09T09:12:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.132+0000 7fc8c67a9640 1 -- 172.21.15.64:0/2659634077 wait complete. 2024-05-09T09:12:29.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:12:29.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.351+0000 7f286c734640 1 -- 172.21.15.64:0/3211807173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410ec80 msgr2=0x7f2864111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.351+0000 7f286c734640 1 --2- 172.21.15.64:0/3211807173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410ec80 0x7f2864111110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f285c009ca0 tx=0x7f285c02a8b0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 -- 172.21.15.64:0/3211807173 shutdown_connections 2024-05-09T09:12:29.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 --2- 172.21.15.64:0/3211807173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410ec80 0x7f2864111110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 --2- 172.21.15.64:0/3211807173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f286410c270 0x7f286410e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 --2- 172.21.15.64:0/3211807173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2864109940 0x7f286410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 -- 172.21.15.64:0/3211807173 >> 172.21.15.64:0/3211807173 conn(0x7f286406ea60 msgr2=0x7f2864070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 -- 172.21.15.64:0/3211807173 shutdown_connections 2024-05-09T09:12:29.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.352+0000 7f286c734640 1 -- 172.21.15.64:0/3211807173 wait complete. 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.353+0000 7f286c734640 1 Processor -- start 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.353+0000 7f286c734640 1 -- start start 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.353+0000 7f286c734640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2864109940 0x7f286419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.353+0000 7f286c734640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 0x7f286419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f286c734640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f286410ec80 0x7f286419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f286c734640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2864114b00 con 0x7f286410ec80 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f286c734640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2864114980 con 0x7f286410c270 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f286c734640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2864114c80 con 0x7f2864109940 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 0x7f286419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 0x7f286419e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35258/0 (socket says 172.21.15.64:35258) 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 -- 172.21.15.64:0/3780779266 learned_addr learned my addr 172.21.15.64:0/3780779266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f286acd1640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f286410ec80 0x7f286419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f286a4d0640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2864109940 0x7f286419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 -- 172.21.15.64:0/3780779266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2864109940 msgr2=0x7f286419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2864109940 0x7f286419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 -- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f286410ec80 msgr2=0x7f286419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f286410ec80 0x7f286419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.354+0000 7f2869ccf640 1 -- 172.21.15.64:0/3780779266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286419af60 con 0x7f286410c270 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.355+0000 7f2869ccf640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 0x7f286419e070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f285400d770 tx=0x7f285400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.355+0000 7f285b7fe640 1 -- 172.21.15.64:0/3780779266 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854017070 con 0x7f286410c270 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.355+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28641a2090 con 0x7f286410c270 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.355+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28641a2630 con 0x7f286410c270 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.356+0000 7f285b7fe640 1 -- 172.21.15.64:0/3780779266 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2854009c30 con 0x7f286410c270 2024-05-09T09:12:29.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.356+0000 7f285b7fe640 1 -- 172.21.15.64:0/3780779266 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400f620 con 0x7f286410c270 2024-05-09T09:12:29.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.359+0000 7f286f443740 1 Processor -- start 2024-05-09T09:12:29.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.360+0000 7f286f443740 1 -- start start 2024-05-09T09:12:29.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.361+0000 7f286f443740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.361+0000 7f286f443740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 0x561241574430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.361+0000 7f286f443740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241576f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.361+0000 7f286f443740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5612414d3c40 con 0x56124156f930 2024-05-09T09:12:29.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.361+0000 7f286f443740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56124147b580 con 0x561241574aa0 2024-05-09T09:12:29.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.361+0000 7f286f443740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5612414cfc00 con 0x56124156ef40 2024-05-09T09:12:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.362+0000 7f285b7fe640 1 -- 172.21.15.64:0/3780779266 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f285401a070 con 0x7f286410c270 2024-05-09T09:12:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.362+0000 7f286a4d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 0x561241574430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.362+0000 7f286a4d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 0x561241574430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45352/0 (socket says 172.21.15.64:45352) 2024-05-09T09:12:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.362+0000 7f286a4d0640 1 -- 172.21.15.64:0/2637586360 learned_addr learned my addr 172.21.15.64:0/2637586360 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.362+0000 7f286acd1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f2869ccf640 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241576f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f286acd1640 1 -- 172.21.15.64:0/2637586360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 msgr2=0x561241576f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f286acd1640 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241576f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f286acd1640 1 -- 172.21.15.64:0/2637586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 msgr2=0x561241574430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f286acd1640 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 0x561241574430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f286acd1640 1 -- 172.21.15.64:0/2637586360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612411e73f0 con 0x56124156ef40 2024-05-09T09:12:29.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f2869ccf640 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241576f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:12:29.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.363+0000 7f285b7fe640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28500a2c90 0x7f28500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.364+0000 7f285b7fe640 1 -- 172.21.15.64:0/3780779266 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28540cecb0 con 0x7f286410c270 2024-05-09T09:12:29.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.364+0000 7f286acd1640 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156f360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f285c006010 tx=0x7f285c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7022183ddc7b99a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.365+0000 7f286a4d0640 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 0x561241574430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:29.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.365+0000 7f286acd1640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28500a2c90 0x7f28500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.365+0000 7f2858ff9640 1 -- 172.21.15.64:0/2637586360 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c030960 con 0x56124156ef40 2024-05-09T09:12:29.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.365+0000 7f2858ff9640 1 -- 172.21.15.64:0/2637586360 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f285c036030 con 0x56124156ef40 2024-05-09T09:12:29.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.365+0000 7f2858ff9640 1 -- 172.21.15.64:0/2637586360 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c004540 con 0x56124156ef40 2024-05-09T09:12:29.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.366+0000 7f286f443740 1 -- 172.21.15.64:0/2637586360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 msgr2=0x56124156f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.366+0000 7f286f443740 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156f360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f285c006010 tx=0x7f285c009a20 comp rx=0 tx=0).stop 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286f443740 1 -- 172.21.15.64:0/2637586360 shutdown_connections 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286f443740 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241576f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286f443740 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156f930 0x561241574430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286f443740 1 --2- 172.21.15.64:0/2637586360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156f360 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286f443740 1 -- 172.21.15.64:0/2637586360 >> 172.21.15.64:0/2637586360 conn(0x56124156a5b0 msgr2=0x56124156a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286f443740 1 -- 172.21.15.64:0/2637586360 shutdown_connections 2024-05-09T09:12:29.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.367+0000 7f286acd1640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28500a2c90 0x7f28500a5150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f285c004770 tx=0x7f285c0046e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.368+0000 7f286f443740 1 -- 172.21.15.64:0/2637586360 wait complete. 2024-05-09T09:12:29.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.368+0000 7f286f443740 1 -- start start 2024-05-09T09:12:29.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 Processor -- start 2024-05-09T09:12:29.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x561241579300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 0x561241579840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241580880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5612415779f0 con 0x56124156ef40 2024-05-09T09:12:29.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56124148ec40 con 0x561241574aa0 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561241577b70 con 0x56124156f930 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.369+0000 7f286f443740 10 client.0 _ll_register_callbacks cb 0x5612411e6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 0x561241579840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f286a4d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x561241579300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f286a4d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x561241579300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45360/0 (socket says 172.21.15.64:45360) 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 0x561241579840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43770/0 (socket says 172.21.15.64:43770) 2024-05-09T09:12:29.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f286a4d0640 1 -- 172.21.15.64:0/639436952 learned_addr learned my addr 172.21.15.64:0/639436952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f286acd1640 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241580880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 -- 172.21.15.64:0/639436952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 msgr2=0x561241580880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241580880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 -- 172.21.15.64:0/639436952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 msgr2=0x561241579300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x561241579300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.370+0000 7f2869ccf640 1 -- 172.21.15.64:0/639436952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612412b17d0 con 0x56124156f930 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f2869ccf640 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 0x561241579840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f285409aed0 tx=0x7f2854097590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89cb967d0313ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f284dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f284dffb640 1 -- 172.21.15.64:0/639436952 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854021070 con 0x56124156f930 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f284dffb640 1 -- 172.21.15.64:0/639436952 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28540bb0b0 con 0x56124156f930 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f285c036030 con 0x56124156f930 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f284dffb640 1 -- 172.21.15.64:0/639436952 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f285409b1d0 con 0x56124156f930 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.371+0000 7f286f443740 10 client.26092 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:29.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.372+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561241583c60 con 0x56124156f930 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.372+0000 7f284dffb640 1 -- 172.21.15.64:0/639436952 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f285409f4d0 con 0x56124156f930 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.372+0000 7f284dffb640 1 -- 172.21.15.64:0/639436952 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2854096f20 con 0x56124156f930 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286f443740 10 client.26092 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286f443740 10 client.26092 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286f443740 20 client.26092 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286f443740 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x561241585940 0x561241587d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286f443740 4 client.26092 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f285c036030 con 0x561241585940 2024-05-09T09:12:29.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.373+0000 7f286a4d0640 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x561241585940 0x561241587d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.375+0000 7f286a4d0640 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x561241585940 0x561241587d60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3a4444dfc726e01 server_cookie=a702aff526d3f452 in_seq=0 out_seq=0 2024-05-09T09:12:29.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.375+0000 7f284dffb640 10 client.26092 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:12:29.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f284dffb640 1 -- 172.21.15.64:0/639436952 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f285c036030 con 0x561241585940 2024-05-09T09:12:29.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f284dffb640 10 client.26092 handle_command_reply: tid=0 2024-05-09T09:12:29.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f286f443740 1 client.26092 shutdown 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 msgr2=0x561241579840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f286f443740 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 0x561241579840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f285409aed0 tx=0x7f2854097590 comp rx=0 tx=0).stop 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 shutdown_connections 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f286f443740 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561241574aa0 0x561241580880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.377+0000 7f286f443740 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156f930 0x561241579840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f286f443740 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x561241585940 0x561241587d60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f286f443740 1 --2- 172.21.15.64:0/639436952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x561241579300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 >> 172.21.15.64:0/639436952 conn(0x56124156a5b0 msgr2=0x5612415697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f284dffb640 0 client.26092 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:12:29.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 shutdown_connections 2024-05-09T09:12:29.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f286f443740 1 -- 172.21.15.64:0/639436952 wait complete. 2024-05-09T09:12:29.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.378+0000 7f286f443740 20 client.26092 trim_cache size 0 max 16384 2024-05-09T09:12:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:29 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1594392520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:12:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:12:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:12:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:12:29.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.410+0000 7f286f443740 1 Processor -- start 2024-05-09T09:12:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.410+0000 7f286f443740 1 -- start start 2024-05-09T09:12:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286f443740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286f443740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x561241580840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286f443740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561241574aa0 0x56124156d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286f443740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2848012630 con 0x561241574aa0 2024-05-09T09:12:29.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286f443740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f284800fe70 con 0x56124156f930 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286f443740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f284800dcc0 con 0x56124156ef40 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x561241580840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43786/0 (socket says 172.21.15.64:43786) 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x561241580840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35276/0 (socket says 172.21.15.64:35276) 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 -- 172.21.15.64:0/2900742274 learned_addr learned my addr 172.21.15.64:0/2900742274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286acd1640 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561241574aa0 0x56124156d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 -- 172.21.15.64:0/2900742274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 msgr2=0x561241580840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x561241580840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 -- 172.21.15.64:0/2900742274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561241574aa0 msgr2=0x56124156d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561241574aa0 0x56124156d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.411+0000 7f286a4d0640 1 -- 172.21.15.64:0/2900742274 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124155bd20 con 0x56124156ef40 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.412+0000 7f286a4d0640 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2860005bc0 tx=0x7f2860005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c4c1bca6063b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.412+0000 7f284ffff640 1 -- 172.21.15.64:0/2900742274 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860005220 con 0x56124156ef40 2024-05-09T09:12:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.412+0000 7f284ffff640 1 -- 172.21.15.64:0/2900742274 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2860024030 con 0x56124156ef40 2024-05-09T09:12:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.412+0000 7f284ffff640 1 -- 172.21.15.64:0/2900742274 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860020330 con 0x56124156ef40 2024-05-09T09:12:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.413+0000 7f286f443740 1 -- 172.21.15.64:0/2900742274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 msgr2=0x56124156d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.413+0000 7f286f443740 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2860005bc0 tx=0x7f2860005f90 comp rx=0 tx=0).stop 2024-05-09T09:12:29.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.414+0000 7f286f443740 1 -- 172.21.15.64:0/2900742274 shutdown_connections 2024-05-09T09:12:29.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.414+0000 7f286f443740 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561241574aa0 0x56124156d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.414+0000 7f286f443740 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x561241580840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.414+0000 7f286f443740 1 --2- 172.21.15.64:0/2900742274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56124156ef40 0x56124156d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.414+0000 7f286f443740 1 -- 172.21.15.64:0/2900742274 >> 172.21.15.64:0/2900742274 conn(0x56124156a5b0 msgr2=0x561241570cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 -- 172.21.15.64:0/2900742274 shutdown_connections 2024-05-09T09:12:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 -- 172.21.15.64:0/2900742274 wait complete. 2024-05-09T09:12:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 -- start start 2024-05-09T09:12:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 Processor -- start 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x56124157e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x56124157e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.415+0000 7f286f443740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561241574aa0 0x56124157f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f286f443740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f284800e330 con 0x56124156ef40 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f286f443740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f284800f7a0 con 0x56124156f930 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f286f443740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f284800ea00 con 0x561241574aa0 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f286f443740 10 client.0 _ll_register_callbacks cb 0x7f2854010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x56124157e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x56124157e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35280/0 (socket says 172.21.15.64:35280) 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 -- 172.21.15.64:0/36274065 learned_addr learned my addr 172.21.15.64:0/36274065 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f286acd1640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561241574aa0 0x56124157f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f286a4d0640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x56124157e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 -- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561241574aa0 msgr2=0x56124157f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561241574aa0 0x56124157f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 -- 172.21.15.64:0/36274065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 msgr2=0x56124157e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x56124157e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.416+0000 7f2869ccf640 1 -- 172.21.15.64:0/36274065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124157fc20 con 0x56124156f930 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.417+0000 7f286acd1640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561241574aa0 0x56124157f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:29.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.417+0000 7f2869ccf640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x56124157e670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2854009610 tx=0x7f28540d2210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e453914743830fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.417+0000 7f284dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.417+0000 7f284dffb640 1 -- 172.21.15.64:0/36274065 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854021070 con 0x56124156f930 2024-05-09T09:12:29.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.417+0000 7f284dffb640 1 -- 172.21.15.64:0/36274065 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28540d2280 con 0x56124156f930 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.418+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2860024030 con 0x56124156f930 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.418+0000 7f284dffb640 1 -- 172.21.15.64:0/36274065 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f285409f110 con 0x56124156f930 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.418+0000 7f286f443740 10 client.25955 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.418+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612415ebfd0 con 0x56124156f930 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.418+0000 7f284dffb640 1 -- 172.21.15.64:0/36274065 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2860024030 con 0x56124156f930 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.418+0000 7f284dffb640 1 -- 172.21.15.64:0/36274065 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f285409a250 con 0x56124156f930 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286f443740 10 client.25955 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286f443740 10 client.25955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286f443740 20 client.25955 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286f443740 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5612415ec790 0x5612415eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286f443740 4 client.25955 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612414e1370 con 0x5612415ec790 2024-05-09T09:12:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.419+0000 7f286a4d0640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5612415ec790 0x5612415eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.420+0000 7f286a4d0640 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5612415ec790 0x5612415eebb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e94f641d113b892 server_cookie=48b32168ade172e0 in_seq=0 out_seq=0 2024-05-09T09:12:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.420+0000 7f284dffb640 10 client.25955 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:12:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.421+0000 7f284dffb640 1 -- 172.21.15.64:0/36274065 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612414e1370 con 0x5612415ec790 2024-05-09T09:12:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.421+0000 7f284dffb640 10 client.25955 handle_command_reply: tid=0 2024-05-09T09:12:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.421+0000 7f286f443740 1 client.25955 shutdown 2024-05-09T09:12:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 msgr2=0x56124157e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x56124157e670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2854009610 tx=0x7f28540d2210 comp rx=0 tx=0).stop 2024-05-09T09:12:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 shutdown_connections 2024-05-09T09:12:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561241574aa0 0x56124157f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56124156f930 0x56124157e670 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2854009610 tx=0x7f28540d2210 comp rx=0 tx=0).stop 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5612415ec790 0x5612415eebb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 --2- 172.21.15.64:0/36274065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56124156ef40 0x56124157e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 >> 172.21.15.64:0/36274065 conn(0x56124156a5b0 msgr2=0x561241568f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f284dffb640 0 client.25955 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 shutdown_connections 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.426+0000 7f286f443740 1 -- 172.21.15.64:0/36274065 wait complete. 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.427+0000 7f286f443740 20 client.25955 trim_cache size 0 max 16384 2024-05-09T09:12:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.429+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28500a2c90 msgr2=0x7f28500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.429+0000 7f286c734640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28500a2c90 0x7f28500a5150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f285c004770 tx=0x7f285c0046e0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.429+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 msgr2=0x7f286419e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.429+0000 7f286c734640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 0x7f286419e070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f285400d770 tx=0x7f285400dc30 comp rx=0 tx=0).stop 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.430+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 shutdown_connections 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.430+0000 7f286c734640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28500a2c90 0x7f28500a5150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.430+0000 7f286c734640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f286410ec80 0x7f286419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.430+0000 7f286c734640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f286410c270 0x7f286419e070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.430+0000 7f286c734640 1 --2- 172.21.15.64:0/3780779266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2864109940 0x7f286419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.430+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 >> 172.21.15.64:0/3780779266 conn(0x7f286406ea60 msgr2=0x7f2864108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.431+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 shutdown_connections 2024-05-09T09:12:29.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.431+0000 7f286c734640 1 -- 172.21.15.64:0/3780779266 wait complete. 2024-05-09T09:12:29.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:12:29.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:29 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1594392520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:12:29.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:12:29.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:12:29.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:12:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 -- 172.21.15.64:0/293821134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8409ba60 msgr2=0x7f8e8409deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/293821134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8409ba60 0x7f8e8409deb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c06b220 tx=0x7f8e8c096060 comp rx=0 tx=0).stop 2024-05-09T09:12:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 -- 172.21.15.64:0/293821134 shutdown_connections 2024-05-09T09:12:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/293821134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8409ba60 0x7f8e8409deb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/293821134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84099130 0x7f8e8409b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/293821134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 0x7f8e84098bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 -- 172.21.15.64:0/293821134 >> 172.21.15.64:0/293821134 conn(0x7f8e840901d0 msgr2=0x7f8e84092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 -- 172.21.15.64:0/293821134 shutdown_connections 2024-05-09T09:12:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.644+0000 7f8e8b59e640 1 -- 172.21.15.64:0/293821134 wait complete. 2024-05-09T09:12:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.645+0000 7f8e8b59e640 1 Processor -- start 2024-05-09T09:12:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.645+0000 7f8e8b59e640 1 -- start start 2024-05-09T09:12:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.646+0000 7f8e8b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 0x7f8e84129b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.646+0000 7f8e8b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 0x7f8e8412a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.646+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 0x7f8e84129b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.646+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 0x7f8e84129b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45408/0 (socket says 172.21.15.64:45408) 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.646+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 0x7f8e8412a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.646+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 0x7f8e8412a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35304/0 (socket says 172.21.15.64:35304) 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.647+0000 7f8e89d9b640 1 -- 172.21.15.64:0/2292925875 learned_addr learned my addr 172.21.15.64:0/2292925875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.647+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e8409ba60 0x7f8e84131140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.647+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e840a1860 con 0x7f8e84096800 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.647+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e840a16e0 con 0x7f8e84099130 2024-05-09T09:12:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.647+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e840a19e0 con 0x7f8e8409ba60 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.647+0000 7f8e8ad9d640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e8409ba60 0x7f8e84131140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e89d9b640 1 -- 172.21.15.64:0/2292925875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e8409ba60 msgr2=0x7f8e84131140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e8409ba60 0x7f8e84131140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e89d9b640 1 -- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 msgr2=0x7f8e84129b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 0x7f8e84129b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e89d9b640 1 -- 172.21.15.64:0/2292925875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e841317b0 con 0x7f8e84099130 2024-05-09T09:12:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 0x7f8e8412a0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000d770 tx=0x7f8e8000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.648+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/2292925875 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80017070 con 0x7f8e84099130 2024-05-09T09:12:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.649+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/2292925875 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e80009bb0 con 0x7f8e84099130 2024-05-09T09:12:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.649+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e84131aa0 con 0x7f8e84099130 2024-05-09T09:12:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.649+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e84131ff0 con 0x7f8e84099130 2024-05-09T09:12:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.649+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/2292925875 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000f620 con 0x7f8e84099130 2024-05-09T09:12:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.652+0000 7f8e93609740 1 Processor -- start 2024-05-09T09:12:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 -- start start 2024-05-09T09:12:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056b9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 0x55d7056c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d70561dc40 con 0x55d7056bec90 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d7055c5580 con 0x55d7056b9130 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e93609740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d705619c00 con 0x55d7056b9b20 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e8ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056b9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e8ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056b9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35318/0 (socket says 172.21.15.64:35318) 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e8ad9d640 1 -- 172.21.15.64:0/260681784 learned_addr learned my addr 172.21.15.64:0/260681784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.653+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e8a59c640 1 -- 172.21.15.64:0/260681784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 msgr2=0x55d7056b9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056b9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e8a59c640 1 -- 172.21.15.64:0/260681784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 msgr2=0x55d7056c1120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:12:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 0x55d7056c1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e8a59c640 1 -- 172.21.15.64:0/260681784 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7053313f0 con 0x55d7056b9b20 2024-05-09T09:12:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056be620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009c20 tx=0x7f8e7c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b79c208bf7e1608 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/2292925875 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e8001a070 con 0x7f8e84099130 2024-05-09T09:12:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e78ff9640 1 -- 172.21.15.64:0/260681784 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004050 con 0x55d7056b9b20 2024-05-09T09:12:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e78ff9640 1 -- 172.21.15.64:0/260681784 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c0054d0 con 0x55d7056b9b20 2024-05-09T09:12:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.654+0000 7f8e78ff9640 1 -- 172.21.15.64:0/260681784 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004d00 con 0x55d7056b9b20 2024-05-09T09:12:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.655+0000 7f8e93609740 1 -- 172.21.15.64:0/260681784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 msgr2=0x55d7056be620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.655+0000 7f8e93609740 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056be620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009c20 tx=0x7f8e7c011810 comp rx=0 tx=0).stop 2024-05-09T09:12:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 -- 172.21.15.64:0/260681784 shutdown_connections 2024-05-09T09:12:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 0x55d7056c1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056be620 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 --2- 172.21.15.64:0/260681784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056b9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 -- 172.21.15.64:0/260681784 >> 172.21.15.64:0/260681784 conn(0x55d7056b47a0 msgr2=0x55d7056b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 -- 172.21.15.64:0/260681784 shutdown_connections 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 -- 172.21.15.64:0/260681784 wait complete. 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e7b7fe640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e740a2bf0 0x7f8e740a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 -- start start 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e740a2bf0 0x7f8e740a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/2292925875 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e800cec90 con 0x7f8e84099130 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.656+0000 7f8e93609740 1 Processor -- start 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 0x55d7056cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d7056c1c40 con 0x55d7056bec90 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d7055d8c40 con 0x55d7056b9130 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d7056c1dc0 con 0x55d7056b9b20 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e93609740 10 client.0 _ll_register_callbacks cb 0x55d705330be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35334/0 (socket says 172.21.15.64:35334) 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e8a59c640 1 -- 172.21.15.64:0/1313550137 learned_addr learned my addr 172.21.15.64:0/1313550137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.657+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e740a2bf0 0x7f8e740a50b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c01b040 tx=0x7f8e7c0045e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e8a59c640 1 -- 172.21.15.64:0/1313550137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 msgr2=0x55d7056c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e8a59c640 1 -- 172.21.15.64:0/1313550137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 msgr2=0x55d7056cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 0x55d7056cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e8a59c640 1 -- 172.21.15.64:0/1313550137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7053fb7d0 con 0x55d7056b9130 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.658+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056c35a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009c20 tx=0x7f8e7c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9167429ddf3151 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1313550137 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c034020 con 0x55d7056b9130 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1313550137 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e7c005190 con 0x55d7056b9130 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e7c0054d0 con 0x55d7056b9130 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1313550137 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c0043c0 con 0x55d7056b9130 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e93609740 10 client.25967 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.659+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7056cdef0 con 0x55d7056b9130 2024-05-09T09:12:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1313550137 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c0054d0 con 0x55d7056b9130 2024-05-09T09:12:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1313550137 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8e7c019430 con 0x55d7056b9130 2024-05-09T09:12:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e93609740 10 client.25967 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e93609740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e93609740 20 client.25967 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e93609740 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d7056cf420 0x55d7056d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e93609740 4 client.25967 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d70562b370 con 0x55d7056cf420 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.660+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d7056cf420 0x55d7056d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.661+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d7056cf420 0x55d7056d18c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbabe0c733eb694 server_cookie=388669351eb1873f in_seq=0 out_seq=0 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.661+0000 7f8e71ffb640 10 client.25967 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.662+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1313550137 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d70562b370 con 0x55d7056cf420 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.662+0000 7f8e71ffb640 10 client.25967 handle_command_reply: tid=0 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.662+0000 7f8e93609740 1 client.25967 shutdown 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 msgr2=0x55d7056c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056c35a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009c20 tx=0x7f8e7c011810 comp rx=0 tx=0).stop 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 shutdown_connections 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d7056cf420 0x55d7056d18c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056bec90 0x55d7056cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056b9b20 0x55d7056c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 --2- 172.21.15.64:0/1313550137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9130 0x55d7056c35a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 >> 172.21.15.64:0/1313550137 conn(0x55d7056b47a0 msgr2=0x55d7056b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e71ffb640 0 client.25967 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 shutdown_connections 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 1 -- 172.21.15.64:0/1313550137 wait complete. 2024-05-09T09:12:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.664+0000 7f8e93609740 20 client.25967 trim_cache size 0 max 16384 2024-05-09T09:12:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.692+0000 7f8e93609740 1 Processor -- start 2024-05-09T09:12:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.692+0000 7f8e93609740 1 -- start start 2024-05-09T09:12:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e93609740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e93609740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e93609740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e93609740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c0125c0 con 0x55d7056b9130 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e93609740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c00fe00 con 0x55d7056b9b20 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e93609740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c00dc50 con 0x55d7056bec90 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45416/0 (socket says 172.21.15.64:45416) 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e8a59c640 1 -- 172.21.15.64:0/473303412 learned_addr learned my addr 172.21.15.64:0/473303412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e8ad9d640 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e89d9b640 1 -- 172.21.15.64:0/473303412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 msgr2=0x55d7056b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e89d9b640 1 -- 172.21.15.64:0/473303412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 msgr2=0x55d7056afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.693+0000 7f8e89d9b640 1 -- 172.21.15.64:0/473303412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7056b01a0 con 0x55d7056b9b20 2024-05-09T09:12:29.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.694+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056cab00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000d740 tx=0x7f8e800057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac42731dcc5bdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.694+0000 7f8e73fff640 1 -- 172.21.15.64:0/473303412 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80021040 con 0x55d7056b9b20 2024-05-09T09:12:29.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.694+0000 7f8e73fff640 1 -- 172.21.15.64:0/473303412 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e800a3d70 con 0x55d7056b9b20 2024-05-09T09:12:29.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.694+0000 7f8e73fff640 1 -- 172.21.15.64:0/473303412 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e800bb0b0 con 0x55d7056b9b20 2024-05-09T09:12:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 -- 172.21.15.64:0/473303412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 msgr2=0x55d7056cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056cab00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000d740 tx=0x7f8e800057f0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 -- 172.21.15.64:0/473303412 shutdown_connections 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056cab00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 --2- 172.21.15.64:0/473303412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 -- 172.21.15.64:0/473303412 >> 172.21.15.64:0/473303412 conn(0x55d7056b47a0 msgr2=0x55d7056b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 -- 172.21.15.64:0/473303412 shutdown_connections 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 -- 172.21.15.64:0/473303412 wait complete. 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.695+0000 7f8e93609740 1 -- start start 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 Processor -- start 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35354/0 (socket says 172.21.15.64:35354) 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1740003821 learned_addr learned my addr 172.21.15.64:0/1740003821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e8ad9d640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c00e2c0 con 0x55d7056b9130 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c00f730 con 0x55d7056b9b20 2024-05-09T09:12:29.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c00e990 con 0x55d7056bec90 2024-05-09T09:12:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.696+0000 7f8e93609740 10 client.0 _ll_register_callbacks cb 0x7f8e8c075a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:29.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1740003821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 msgr2=0x55d7056c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1740003821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 msgr2=0x55d7056c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7056c9820 con 0x55d7056b9b20 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056c8220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000df50 tx=0x7f8e80009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d93cbce4e1f760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1740003821 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80021070 con 0x55d7056b9b20 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1740003821 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e800bb0d0 con 0x55d7056b9b20 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.697+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e800a3d70 con 0x55d7056b9b20 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.698+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1740003821 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e80097820 con 0x55d7056b9b20 2024-05-09T09:12:29.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.698+0000 7f8e93609740 10 client.25973 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.698+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7056cab20 con 0x55d7056b9b20 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.698+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1740003821 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e80096f00 con 0x55d7056b9b20 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1740003821 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8e8009a150 con 0x55d7056b9b20 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e93609740 10 client.25973 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e93609740 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e93609740 20 client.25973 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e93609740 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d705735dc0 0x55d7057381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e93609740 4 client.25973 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8e800a3d70 con 0x55d705735dc0 2024-05-09T09:12:29.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.699+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d705735dc0 0x55d7057381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.700+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d705735dc0 0x55d7057381e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fd53aa290ae5c4 server_cookie=494b38982db575b7 in_seq=0 out_seq=0 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.700+0000 7f8e71ffb640 10 client.25973 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.700+0000 7f8e71ffb640 1 -- 172.21.15.64:0/1740003821 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8e800a3d70 con 0x55d705735dc0 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.700+0000 7f8e71ffb640 10 client.25973 handle_command_reply: tid=0 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 client.25973 shutdown 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 msgr2=0x55d7056c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056c8220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000df50 tx=0x7f8e80009f90 comp rx=0 tx=0).stop 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 shutdown_connections 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d705735dc0 0x55d7057381e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7056bec90 0x55d7056c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d7056b9b20 0x55d7056c8220 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 --2- 172.21.15.64:0/1740003821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7056b9130 0x55d7056c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 >> 172.21.15.64:0/1740003821 conn(0x55d7056b47a0 msgr2=0x55d7056c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e71ffb640 0 client.25973 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 shutdown_connections 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 1 -- 172.21.15.64:0/1740003821 wait complete. 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.701+0000 7f8e93609740 20 client.25973 trim_cache size 0 max 16384 2024-05-09T09:12:29.703 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:12:29.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.703+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e740a2bf0 msgr2=0x7f8e740a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.703+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e740a2bf0 0x7f8e740a50b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c01b040 tx=0x7f8e7c0045e0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.703+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 msgr2=0x7f8e8412a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:29.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.703+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 0x7f8e8412a0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000d770 tx=0x7f8e8000dc30 comp rx=0 tx=0).stop 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.704+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 shutdown_connections 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.704+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e740a2bf0 0x7f8e740a50b0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c01b040 tx=0x7f8e7c0045e0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.704+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e8409ba60 0x7f8e84131140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.704+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e84099130 0x7f8e8412a0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.704+0000 7f8e8b59e640 1 --2- 172.21.15.64:0/2292925875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e84096800 0x7f8e84129b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.704+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 >> 172.21.15.64:0/2292925875 conn(0x7f8e840901d0 msgr2=0x7f8e84095e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.705+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 shutdown_connections 2024-05-09T09:12:29.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:29.705+0000 7f8e8b59e640 1 -- 172.21.15.64:0/2292925875 wait complete. 2024-05-09T09:12:29.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:29 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1594392520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:12:29.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:12:29.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:12:29.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:12:30.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:30 smithi139 ceph-mon[33599]: pgmap v471: 129 pgs: 129 active+clean; 993 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.29k op/s 2024-05-09T09:12:30.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:30 smithi064 ceph-mon[30149]: pgmap v471: 129 pgs: 129 active+clean; 993 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.29k op/s 2024-05-09T09:12:30.728 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:12:30.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:12:30.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:12:30.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:30 smithi179 ceph-mon[34086]: pgmap v471: 129 pgs: 129 active+clean; 993 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 1.29k op/s 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 -- 172.21.15.64:0/2845008865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 msgr2=0x7f9558107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 --2- 172.21.15.64:0/2845008865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 0x7f9558107010 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9554009860 tx=0x7f955402a340 comp rx=0 tx=0).stop 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 -- 172.21.15.64:0/2845008865 shutdown_connections 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 --2- 172.21.15.64:0/2845008865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 --2- 172.21.15.64:0/2845008865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 0x7f9558107010 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 --2- 172.21.15.64:0/2845008865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95581022f0 0x7f95581046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 -- 172.21.15.64:0/2845008865 >> 172.21.15.64:0/2845008865 conn(0x7f95580fbce0 msgr2=0x7f95580fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:30.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 -- 172.21.15.64:0/2845008865 shutdown_connections 2024-05-09T09:12:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.929+0000 7f955f309640 1 -- 172.21.15.64:0/2845008865 wait complete. 2024-05-09T09:12:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.930+0000 7f955f309640 1 Processor -- start 2024-05-09T09:12:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.930+0000 7f955f309640 1 -- start start 2024-05-09T09:12:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955f309640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95581022f0 0x7f9558070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955f309640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 0x7f9558070870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955f309640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955f309640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f955810f600 con 0x7f9558104c20 2024-05-09T09:12:30.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955f309640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f955810f480 con 0x7f95581022f0 2024-05-09T09:12:30.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955f309640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f955810f780 con 0x7f9558107550 2024-05-09T09:12:30.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43866/0 (socket says 172.21.15.64:43866) 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 -- 172.21.15.64:0/3858627691 learned_addr learned my addr 172.21.15.64:0/3858627691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 0x7f9558070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d0a5640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95581022f0 0x7f9558070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 -- 172.21.15.64:0/3858627691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95581022f0 msgr2=0x7f9558070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95581022f0 0x7f9558070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 -- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 msgr2=0x7f9558070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:30.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 0x7f9558070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.931+0000 7f955d8a6640 1 -- 172.21.15.64:0/3858627691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95581aa210 con 0x7f9558107550 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.932+0000 7f955d8a6640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581a9cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f955000eda0 tx=0x7f955000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.932+0000 7f954e7fc640 1 -- 172.21.15.64:0/3858627691 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550003b70 con 0x7f9558107550 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.932+0000 7f954e7fc640 1 -- 172.21.15.64:0/3858627691 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9550003d10 con 0x7f9558107550 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.932+0000 7f954e7fc640 1 -- 172.21.15.64:0/3858627691 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550017940 con 0x7f9558107550 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.932+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95581aa500 con 0x7f9558107550 2024-05-09T09:12:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.932+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955810a090 con 0x7f9558107550 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.937+0000 7f954e7fc640 1 -- 172.21.15.64:0/3858627691 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9550017ae0 con 0x7f9558107550 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.937+0000 7f9562012740 1 Processor -- start 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 -- start start 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 0x562a1b2dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a1b241990 con 0x562a1b2ddeb0 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562a1b1e95b0 con 0x562a1b2e3020 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.938+0000 7f9562012740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a1b23d950 con 0x562a1b2dd4c0 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 0x562a1b2dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 0x562a1b2dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43874/0 (socket says 172.21.15.64:43874) 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 -- 172.21.15.64:0/3739237908 learned_addr learned my addr 172.21.15.64:0/3739237908 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955d0a5640 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f954e7fc640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95440a2d90 0x7f95440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 -- 172.21.15.64:0/3739237908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 msgr2=0x562a1b2e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2e54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 -- 172.21.15.64:0/3739237908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2ddeb0 msgr2=0x562a1b2e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955c8a4640 1 -- 172.21.15.64:0/3739237908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1b01d7e0 con 0x562a1b2dd4c0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f955d8a6640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95440a2d90 0x7f95440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.939+0000 7f954e7fc640 1 -- 172.21.15.64:0/3858627691 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f95500cf500 con 0x7f9558107550 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.940+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 0x562a1b2dd8e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9554005b60 tx=0x7f955402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d422ec99314ad1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.940+0000 7f955d8a6640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95440a2d90 0x7f95440a5250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f955009b670 tx=0x7f9550009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.940+0000 7f95437fe640 1 -- 172.21.15.64:0/3739237908 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554037030 con 0x562a1b2dd4c0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.940+0000 7f95437fe640 1 -- 172.21.15.64:0/3739237908 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9554047070 con 0x562a1b2dd4c0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.940+0000 7f95437fe640 1 -- 172.21.15.64:0/3739237908 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554037030 con 0x562a1b2dd4c0 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.941+0000 7f9562012740 1 -- 172.21.15.64:0/3739237908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 msgr2=0x562a1b2dd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.941+0000 7f9562012740 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 0x562a1b2dd8e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9554005b60 tx=0x7f955402a340 comp rx=0 tx=0).stop 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 -- 172.21.15.64:0/3739237908 shutdown_connections 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 --2- 172.21.15.64:0/3739237908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2dd4c0 0x562a1b2dd8e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 -- 172.21.15.64:0/3739237908 >> 172.21.15.64:0/3739237908 conn(0x562a1b2a50a0 msgr2=0x562a1b2d8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:30.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 -- 172.21.15.64:0/3739237908 shutdown_connections 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.942+0000 7f9562012740 1 -- 172.21.15.64:0/3739237908 wait complete. 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.943+0000 7f9562012740 1 -- start start 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.943+0000 7f9562012740 1 Processor -- start 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.943+0000 7f9562012740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.943+0000 7f9562012740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f9562012740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f9562012740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a1b2e5f70 con 0x562a1b2dd4c0 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f9562012740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562a1b1fcb40 con 0x562a1b2e3020 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f9562012740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a1b2e60f0 con 0x562a1b2ddeb0 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f955d0a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f955d0a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43890/0 (socket says 172.21.15.64:43890) 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f955d0a5640 1 -- 172.21.15.64:0/3031343596 learned_addr learned my addr 172.21.15.64:0/3031343596 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.944+0000 7f9562012740 10 client.0 _ll_register_callbacks cb 0x562a1b1f10f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955d0a5640 1 -- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 msgr2=0x562a1b2eee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955d0a5640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955d0a5640 1 -- 172.21.15.64:0/3031343596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 msgr2=0x562a1b2e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955d0a5640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2e78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955d0a5640 1 -- 172.21.15.64:0/3031343596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1b07df70 con 0x562a1b2ddeb0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f955d0a5640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e7e00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f954800bad0 tx=0x7f954800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c633477f086c0381 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.945+0000 7f9540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9540ff9640 1 -- 172.21.15.64:0/3031343596 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548003da0 con 0x562a1b2ddeb0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9540ff9640 1 -- 172.21.15.64:0/3031343596 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9548011440 con 0x562a1b2ddeb0 2024-05-09T09:12:30.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9554047070 con 0x562a1b2ddeb0 2024-05-09T09:12:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9540ff9640 1 -- 172.21.15.64:0/3031343596 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9548011680 con 0x562a1b2ddeb0 2024-05-09T09:12:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9562012740 10 client.26128 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:30.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a1af15170 con 0x562a1b2ddeb0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.946+0000 7f9540ff9640 1 -- 172.21.15.64:0/3031343596 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9548011cd0 con 0x562a1b2ddeb0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9540ff9640 1 -- 172.21.15.64:0/3031343596 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9548025b90 con 0x562a1b2ddeb0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9562012740 10 client.26128 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9562012740 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9562012740 20 client.26128 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9562012740 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b2f33f0 0x562a1b2f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9562012740 4 client.26128 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.947+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9554047070 con 0x562a1b2f33f0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.948+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b2f33f0 0x562a1b2f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.949+0000 7f955c8a4640 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b2f33f0 0x562a1b2f5870 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75ff2317aae0faf server_cookie=184209814df45e78 in_seq=0 out_seq=0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.949+0000 7f9540ff9640 10 client.26128 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.950+0000 7f9540ff9640 1 -- 172.21.15.64:0/3031343596 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f9554047070 con 0x562a1b2f33f0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.950+0000 7f9540ff9640 10 client.26128 handle_command_reply: tid=0 2024-05-09T09:12:30.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.950+0000 7f9562012740 1 client.26128 shutdown 2024-05-09T09:12:30.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 msgr2=0x562a1b2e7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e7e00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f954800bad0 tx=0x7f954800bf90 comp rx=0 tx=0).stop 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 shutdown_connections 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b2f33f0 0x562a1b2f5870 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2e7e00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 --2- 172.21.15.64:0/3031343596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 >> 172.21.15.64:0/3031343596 conn(0x562a1b2a50a0 msgr2=0x562a1b2d8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:30.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9540ff9640 0 client.26128 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:30.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 shutdown_connections 2024-05-09T09:12:30.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 1 -- 172.21.15.64:0/3031343596 wait complete. 2024-05-09T09:12:30.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.951+0000 7f9562012740 20 client.26128 trim_cache size 0 max 16384 2024-05-09T09:12:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.985+0000 7f9562012740 1 Processor -- start 2024-05-09T09:12:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.985+0000 7f9562012740 1 -- start start 2024-05-09T09:12:30.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f9562012740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f9562012740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f9562012740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f9562012740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f953c0125e0 con 0x562a1b2dd4c0 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f9562012740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f953c00fe20 con 0x562a1b2e3020 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f9562012740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f953c00dc70 con 0x562a1b2ddeb0 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2df270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45470/0 (socket says 172.21.15.64:45470) 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.986+0000 7f955c8a4640 1 -- 172.21.15.64:0/537286197 learned_addr learned my addr 172.21.15.64:0/537286197 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955d0a5640 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 -- 172.21.15.64:0/537286197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 msgr2=0x562a1b2df7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2df7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 -- 172.21.15.64:0/537286197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 msgr2=0x562a1b2eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 -- 172.21.15.64:0/537286197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1b2e2d20 con 0x562a1b2dd4c0 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955d0a5640 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.987+0000 7f955c8a4640 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2df270 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9554005c40 tx=0x7f9554005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c379e278602237af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.988+0000 7f9542ffd640 1 -- 172.21.15.64:0/537286197 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554037030 con 0x562a1b2dd4c0 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.988+0000 7f9542ffd640 1 -- 172.21.15.64:0/537286197 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9554047040 con 0x562a1b2dd4c0 2024-05-09T09:12:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.988+0000 7f9542ffd640 1 -- 172.21.15.64:0/537286197 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554032a80 con 0x562a1b2dd4c0 2024-05-09T09:12:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.989+0000 7f9562012740 1 -- 172.21.15.64:0/537286197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 msgr2=0x562a1b2df270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.989+0000 7f9562012740 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2df270 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9554005c40 tx=0x7f9554005c90 comp rx=0 tx=0).stop 2024-05-09T09:12:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.990+0000 7f9562012740 1 -- 172.21.15.64:0/537286197 shutdown_connections 2024-05-09T09:12:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.990+0000 7f9562012740 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2e3020 0x562a1b2eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.991+0000 7f9562012740 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.991+0000 7f9562012740 1 --2- 172.21.15.64:0/537286197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2dd4c0 0x562a1b2df270 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.991+0000 7f9562012740 1 -- 172.21.15.64:0/537286197 >> 172.21.15.64:0/537286197 conn(0x562a1b2a50a0 msgr2=0x562a1b2e78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.991+0000 7f9562012740 1 -- 172.21.15.64:0/537286197 shutdown_connections 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.991+0000 7f9562012740 1 -- 172.21.15.64:0/537286197 wait complete. 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.992+0000 7f9562012740 1 -- start start 2024-05-09T09:12:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.992+0000 7f9562012740 1 Processor -- start 2024-05-09T09:12:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 0x562a1b2eca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2e3020 0x562a1b2edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f953c00e2e0 con 0x562a1b2e3020 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f953c00f750 con 0x562a1b2dd4c0 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f953c00e9b0 con 0x562a1b2ddeb0 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f9562012740 10 client.0 _ll_register_callbacks cb 0x7f9548004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 0x562a1b2eca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 0x562a1b2eca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35394/0 (socket says 172.21.15.64:35394) 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 -- 172.21.15.64:0/1564040536 learned_addr learned my addr 172.21.15.64:0/1564040536 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955d0a5640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 -- 172.21.15.64:0/1564040536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 msgr2=0x562a1b2ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 -- 172.21.15.64:0/1564040536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2e3020 msgr2=0x562a1b2edf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2e3020 0x562a1b2edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.993+0000 7f955c8a4640 1 -- 172.21.15.64:0/1564040536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1b2ee560 con 0x562a1b2dd4c0 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.994+0000 7f955d0a5640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.994+0000 7f955c8a4640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 0x562a1b2eca90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9554009340 tx=0x7f9554032440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d285023ced31e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.994+0000 7f9540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.994+0000 7f9540ff9640 1 -- 172.21.15.64:0/1564040536 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554037040 con 0x562a1b2dd4c0 2024-05-09T09:12:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.994+0000 7f9540ff9640 1 -- 172.21.15.64:0/1564040536 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9554056330 con 0x562a1b2dd4c0 2024-05-09T09:12:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.994+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9554047040 con 0x562a1b2dd4c0 2024-05-09T09:12:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.995+0000 7f9540ff9640 1 -- 172.21.15.64:0/1564040536 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9554056530 con 0x562a1b2dd4c0 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.996+0000 7f9562012740 10 client.25991 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.996+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a1b359ed0 con 0x562a1b2dd4c0 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9540ff9640 1 -- 172.21.15.64:0/1564040536 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9554047040 con 0x562a1b2dd4c0 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9540ff9640 1 -- 172.21.15.64:0/1564040536 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f95540614d0 con 0x562a1b2dd4c0 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9562012740 10 client.25991 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9562012740 10 client.25991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9562012740 20 client.25991 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9562012740 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b35a7d0 0x562a1b35cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9562012740 4 client.25991 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:12:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a1b24f0c0 con 0x562a1b35a7d0 2024-05-09T09:12:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.997+0000 7f955d0a5640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b35a7d0 0x562a1b35cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.998+0000 7f955d0a5640 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b35a7d0 0x562a1b35cbf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa32d7a537248e4 server_cookie=8c9fbd7a6424d0b3 in_seq=0 out_seq=0 2024-05-09T09:12:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:30.998+0000 7f9540ff9640 10 client.25991 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:30.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9540ff9640 1 -- 172.21.15.64:0/1564040536 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a1b24f0c0 con 0x562a1b35a7d0 2024-05-09T09:12:30.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9540ff9640 10 client.25991 handle_command_reply: tid=0 2024-05-09T09:12:30.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9562012740 1 client.25991 shutdown 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 msgr2=0x562a1b2eca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9562012740 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 0x562a1b2eca90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9554009340 tx=0x7f9554032440 comp rx=0 tx=0).stop 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 shutdown_connections 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9562012740 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562a1b35a7d0 0x562a1b35cbf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9562012740 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a1b2e3020 0x562a1b2edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.000+0000 7f9540ff9640 0 client.25991 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.001+0000 7f9562012740 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a1b2ddeb0 0x562a1b2ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.001+0000 7f9562012740 1 --2- 172.21.15.64:0/1564040536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a1b2dd4c0 0x562a1b2eca90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.001+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 >> 172.21.15.64:0/1564040536 conn(0x562a1b2a50a0 msgr2=0x562a1b2e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.001+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 shutdown_connections 2024-05-09T09:12:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.001+0000 7f9562012740 1 -- 172.21.15.64:0/1564040536 wait complete. 2024-05-09T09:12:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.002+0000 7f9562012740 20 client.25991 trim_cache size 0 max 16384 2024-05-09T09:12:31.001 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:12:31.001 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:12:31.001 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "f7c9163c-02a9-45c3-8847-2660f9704a49", 2024-05-09T09:12:31.001 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:12:31.002 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95440a2d90 msgr2=0x7f95440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95440a2d90 0x7f95440a5250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f955009b670 tx=0x7f9550009480 comp rx=0 tx=0).stop 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 msgr2=0x7f95581a9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581a9cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f955000eda0 tx=0x7f955000c6e0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 shutdown_connections 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f95440a2d90 0x7f95440a5250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9558107550 0x7f95581a9cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9558104c20 0x7f9558070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 --2- 172.21.15.64:0/3858627691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95581022f0 0x7f9558070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.003+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 >> 172.21.15.64:0/3858627691 conn(0x7f95580fbce0 msgr2=0x7f95581041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.004+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 shutdown_connections 2024-05-09T09:12:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.004+0000 7f955f309640 1 -- 172.21.15.64:0/3858627691 wait complete. 2024-05-09T09:12:31.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:12:31.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 -- 172.21.15.64:0/633752723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 msgr2=0x7f2814078670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 --2- 172.21.15.64:0/633752723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 0x7f2814078670 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f280c009ca0 tx=0x7f280c02a8f0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 -- 172.21.15.64:0/633752723 shutdown_connections 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 --2- 172.21.15.64:0/633752723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 0x7f2814078670 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 --2- 172.21.15.64:0/633752723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814077430 0x7f2814077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 --2- 172.21.15.64:0/633752723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28140761f0 0x7f2814076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.243+0000 7f2818a00640 1 -- 172.21.15.64:0/633752723 >> 172.21.15.64:0/633752723 conn(0x7f28140fef80 msgr2=0x7f28141013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.246+0000 7f2818a00640 1 -- 172.21.15.64:0/633752723 shutdown_connections 2024-05-09T09:12:31.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.247+0000 7f2818a00640 1 -- 172.21.15.64:0/633752723 wait complete. 2024-05-09T09:12:31.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.248+0000 7f2818a00640 1 Processor -- start 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 -- start start 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28140761f0 0x7f2814103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2814077430 0x7f2814103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 0x7f281410add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28141139a0 con 0x7f2814077dd0 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2814113820 con 0x7f2814077430 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f2818a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2814113b20 con 0x7f28140761f0 2024-05-09T09:12:31.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f281259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28140761f0 0x7f2814103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f281259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28140761f0 0x7f2814103820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43932/0 (socket says 172.21.15.64:43932) 2024-05-09T09:12:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f281259c640 1 -- 172.21.15.64:0/1977322869 learned_addr learned my addr 172.21.15.64:0/1977322869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.249+0000 7f280bfff640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2814077430 0x7f2814103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.250+0000 7f280bfff640 1 -- 172.21.15.64:0/1977322869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28140761f0 msgr2=0x7f2814103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.250+0000 7f280bfff640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28140761f0 0x7f2814103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.250+0000 7f280bfff640 1 -- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 msgr2=0x7f281410add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.250+0000 7f280bfff640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 0x7f281410add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.250+0000 7f280bfff640 1 -- 172.21.15.64:0/1977322869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281410b440 con 0x7f2814077430 2024-05-09T09:12:31.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.250+0000 7f280bfff640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2814077430 0x7f2814103d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f280400b3f0 tx=0x7f280400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.253+0000 7f280b7fe640 1 -- 172.21.15.64:0/1977322869 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804015020 con 0x7f2814077430 2024-05-09T09:12:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.253+0000 7f280b7fe640 1 -- 172.21.15.64:0/1977322869 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2804011ce0 con 0x7f2814077430 2024-05-09T09:12:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.253+0000 7f280b7fe640 1 -- 172.21.15.64:0/1977322869 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804010d70 con 0x7f2814077430 2024-05-09T09:12:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.253+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281410b730 con 0x7f2814077430 2024-05-09T09:12:31.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.253+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281419dd70 con 0x7f2814077430 2024-05-09T09:12:31.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.256+0000 7f281b7ec740 1 Processor -- start 2024-05-09T09:12:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 -- start start 2024-05-09T09:12:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 0x565175c31540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31b10 0x565175c36610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c36c80 0x565175c39110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565175b95d00 con 0x565175c31120 2024-05-09T09:12:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565175b3d4c0 con 0x565175c36c80 2024-05-09T09:12:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.257+0000 7f281b7ec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x565175b91cc0 con 0x565175c31b10 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.258+0000 7f2812d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 0x565175c31540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.258+0000 7f2812d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 0x565175c31540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45514/0 (socket says 172.21.15.64:45514) 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.258+0000 7f2812d9d640 1 -- 172.21.15.64:0/1907808120 learned_addr learned my addr 172.21.15.64:0/1907808120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.258+0000 7f281259c640 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31b10 0x565175c36610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.259+0000 7f280b7fe640 1 -- 172.21.15.64:0/1977322869 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2804018020 con 0x7f2814077430 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.259+0000 7f280bfff640 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c36c80 0x565175c39110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.260+0000 7f2812d9d640 1 -- 172.21.15.64:0/1907808120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31b10 msgr2=0x565175c36610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.260+0000 7f2812d9d640 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31b10 0x565175c36610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.260+0000 7f2812d9d640 1 -- 172.21.15.64:0/1907808120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c36c80 msgr2=0x565175c39110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.260+0000 7f2812d9d640 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c36c80 0x565175c39110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.260+0000 7f2812d9d640 1 -- 172.21.15.64:0/1907808120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651759737d0 con 0x565175c31120 2024-05-09T09:12:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.261+0000 7f280b7fe640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2d10 0x7f28000a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.262+0000 7f2812d9d640 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 0x565175c31540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f280c009a50 tx=0x7f280c0305f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d8d3ca6511544a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.262+0000 7f281259c640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2d10 0x7f28000a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.263+0000 7f2808ff9640 1 -- 172.21.15.64:0/1907808120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c031b50 con 0x565175c31120 2024-05-09T09:12:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.263+0000 7f2808ff9640 1 -- 172.21.15.64:0/1907808120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c03f070 con 0x565175c31120 2024-05-09T09:12:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.263+0000 7f2808ff9640 1 -- 172.21.15.64:0/1907808120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c003bd0 con 0x565175c31120 2024-05-09T09:12:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.263+0000 7f280b7fe640 1 -- 172.21.15.64:0/1977322869 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28040ceb10 con 0x7f2814077430 2024-05-09T09:12:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.264+0000 7f281b7ec740 1 -- 172.21.15.64:0/1907808120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 msgr2=0x565175c31540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.264+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 0x565175c31540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f280c009a50 tx=0x7f280c0305f0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.265+0000 7f281259c640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2d10 0x7f28000a51d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x565175c31f10 tx=0x7f27fc02b1d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.267+0000 7f281b7ec740 1 -- 172.21.15.64:0/1907808120 shutdown_connections 2024-05-09T09:12:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.267+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c36c80 0x565175c39110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.267+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31b10 0x565175c36610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.267+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1907808120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c31120 0x565175c31540 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.267+0000 7f281b7ec740 1 -- 172.21.15.64:0/1907808120 >> 172.21.15.64:0/1907808120 conn(0x565175c2c790 msgr2=0x565175c2cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.269+0000 7f281b7ec740 1 -- 172.21.15.64:0/1907808120 shutdown_connections 2024-05-09T09:12:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.271+0000 7f281b7ec740 1 -- 172.21.15.64:0/1907808120 wait complete. 2024-05-09T09:12:31.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.271+0000 7f281b7ec740 1 -- start start 2024-05-09T09:12:31.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.271+0000 7f281b7ec740 1 Processor -- start 2024-05-09T09:12:31.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.272+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.272+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.272+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c42b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.272+0000 7f281259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.272+0000 7f281259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43960/0 (socket says 172.21.15.64:43960) 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35422/0 (socket says 172.21.15.64:35422) 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f281259c640 1 -- 172.21.15.64:0/1905337839 learned_addr learned my addr 172.21.15.64:0/1905337839 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565175c39bd0 con 0x565175c36c80 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565175b50b90 con 0x565175c31b10 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x565175c39d50 con 0x565175c31120 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f281b7ec740 10 client.0 _ll_register_callbacks cb 0x565175b8cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 -- 172.21.15.64:0/1905337839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 msgr2=0x565175c3b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 -- 172.21.15.64:0/1905337839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 msgr2=0x565175c42b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c42b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f280bfff640 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651758a93f0 con 0x565175c31b10 2024-05-09T09:12:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.273+0000 7f281259c640 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:12:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.277+0000 7f280bfff640 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3bb50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28040cc2d0 tx=0x7f2804097340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dea6eb93c6e6577 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.277+0000 7f27f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.277+0000 7f27f9ffb640 1 -- 172.21.15.64:0/1905337839 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040d4070 con 0x565175c31b10 2024-05-09T09:12:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.277+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c03f070 con 0x565175c31b10 2024-05-09T09:12:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.278+0000 7f27f9ffb640 1 -- 172.21.15.64:0/1905337839 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2804097650 con 0x565175c31b10 2024-05-09T09:12:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.278+0000 7f27f9ffb640 1 -- 172.21.15.64:0/1905337839 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f28040bb0e0 con 0x565175c31b10 2024-05-09T09:12:31.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.281+0000 7f281b7ec740 10 client.26000 fetch_fsmap learned FSMap version 31 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.281+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5651758690f0 con 0x565175c31b10 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f27f9ffb640 1 -- 172.21.15.64:0/1905337839 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c03f070 con 0x565175c31b10 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f27f9ffb640 1 -- 172.21.15.64:0/1905337839 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2804011830 con 0x565175c31b10 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281b7ec740 10 client.26000 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281b7ec740 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281b7ec740 20 client.26000 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175c477f0 0x565175c49c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281b7ec740 4 client.26000 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565175ba3430 con 0x565175c477f0 2024-05-09T09:12:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.282+0000 7f281259c640 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175c477f0 0x565175c49c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.283+0000 7f281259c640 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175c477f0 0x565175c49c70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e94c7a6d9635cb server_cookie=e60be02c13290ed3 in_seq=0 out_seq=0 2024-05-09T09:12:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.285+0000 7f27f9ffb640 10 client.26000 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.285+0000 7f27f9ffb640 1 -- 172.21.15.64:0/1905337839 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565175ba3430 con 0x565175c477f0 2024-05-09T09:12:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.285+0000 7f27f9ffb640 10 client.26000 handle_command_reply: tid=0 2024-05-09T09:12:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.285+0000 7f281b7ec740 1 client.26000 shutdown 2024-05-09T09:12:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.287+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 msgr2=0x565175c3bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.287+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3bb50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28040cc2d0 tx=0x7f2804097340 comp rx=0 tx=0).stop 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 shutdown_connections 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175c477f0 0x565175c49c70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c42b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3bb50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 --2- 172.21.15.64:0/1905337839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 >> 172.21.15.64:0/1905337839 conn(0x565175c2c790 msgr2=0x565175c2ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f27f9ffb640 0 client.26000 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 shutdown_connections 2024-05-09T09:12:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 1 -- 172.21.15.64:0/1905337839 wait complete. 2024-05-09T09:12:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.288+0000 7f281b7ec740 20 client.26000 trim_cache size 0 max 16384 2024-05-09T09:12:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.337+0000 7f281b7ec740 1 Processor -- start 2024-05-09T09:12:31.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.339+0000 7f281b7ec740 1 -- start start 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.339+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c32510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.339+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c42ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.339+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c3b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f280bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c42ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f280bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c42ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35432/0 (socket says 172.21.15.64:35432) 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f280bfff640 1 -- 172.21.15.64:0/2612999480 learned_addr learned my addr 172.21.15.64:0/2612999480 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28141aab10 con 0x565175c36c80 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a8350 con 0x565175c31b10 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a61a0 con 0x565175c31120 2024-05-09T09:12:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.340+0000 7f2812d9d640 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c3b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.341+0000 7f2812d9d640 1 -- 172.21.15.64:0/2612999480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 msgr2=0x565175c32510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.341+0000 7f2812d9d640 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c32510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.341+0000 7f2812d9d640 1 -- 172.21.15.64:0/2612999480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 msgr2=0x565175c42ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.341+0000 7f2812d9d640 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c42ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.341+0000 7f2812d9d640 1 -- 172.21.15.64:0/2612999480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565175c36300 con 0x565175c36c80 2024-05-09T09:12:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.341+0000 7f2812d9d640 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c3b610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x565175c32370 tx=0x7f280c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f4ea73bac7401e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.342+0000 7f27fbfff640 1 -- 172.21.15.64:0/2612999480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c031b50 con 0x565175c36c80 2024-05-09T09:12:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.342+0000 7f27fbfff640 1 -- 172.21.15.64:0/2612999480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c048070 con 0x565175c36c80 2024-05-09T09:12:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.343+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 msgr2=0x565175c3b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.343+0000 7f281b7ec740 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c3b610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x565175c32370 tx=0x7f280c031000 comp rx=0 tx=0).stop 2024-05-09T09:12:31.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 shutdown_connections 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175c3b610 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c42ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 --2- 172.21.15.64:0/2612999480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c32510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 >> 172.21.15.64:0/2612999480 conn(0x565175c2c790 msgr2=0x565175c3a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 shutdown_connections 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 -- 172.21.15.64:0/2612999480 wait complete. 2024-05-09T09:12:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 -- start start 2024-05-09T09:12:31.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.344+0000 7f281b7ec740 1 Processor -- start 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175cabde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a6810 con 0x565175c36c80 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a7c80 con 0x565175c31b10 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f280bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a6ee0 con 0x565175c31120 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f280bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35436/0 (socket says 172.21.15.64:35436) 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f280bfff640 1 -- 172.21.15.64:0/3156035680 learned_addr learned my addr 172.21.15.64:0/3156035680 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281b7ec740 10 client.0 _ll_register_callbacks cb 0x7f2804010f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f281259c640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.345+0000 7f2812d9d640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175cabde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.346+0000 7f280bfff640 1 -- 172.21.15.64:0/3156035680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 msgr2=0x565175c3f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.346+0000 7f280bfff640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.346+0000 7f280bfff640 1 -- 172.21.15.64:0/3156035680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 msgr2=0x565175cabde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.346+0000 7f280bfff640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175cabde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.346+0000 7f280bfff640 1 -- 172.21.15.64:0/3156035680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565175cac320 con 0x565175c31b10 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.347+0000 7f280bfff640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3fe80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f280400bdb0 tx=0x7f28040047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e8588be67c0233 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.347+0000 7f27f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.347+0000 7f27f9ffb640 1 -- 172.21.15.64:0/3156035680 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040d4040 con 0x565175c31b10 2024-05-09T09:12:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.347+0000 7f27f9ffb640 1 -- 172.21.15.64:0/3156035680 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2804095b50 con 0x565175c31b10 2024-05-09T09:12:31.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.349+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c048070 con 0x565175c31b10 2024-05-09T09:12:31.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.349+0000 7f27f9ffb640 1 -- 172.21.15.64:0/3156035680 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f280409ab20 con 0x565175c31b10 2024-05-09T09:12:31.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.351+0000 7f281b7ec740 10 client.26006 fetch_fsmap learned FSMap version 32 2024-05-09T09:12:31.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.351+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565175cad4d0 con 0x565175c31b10 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 130.0 s, 989.1 tps, lat 94.337 ms stddev 28.453 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f27f9ffb640 1 -- 172.21.15.64:0/3156035680 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c048070 con 0x565175c31b10 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f27f9ffb640 1 -- 172.21.15.64:0/3156035680 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f2804099ea0 con 0x565175c31b10 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281b7ec740 10 client.26006 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281b7ec740 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281b7ec740 20 client.26006 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281b7ec740 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175cadd20 0x565175cb0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281b7ec740 4 client.26006 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565175b9d780 con 0x565175cadd20 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.352+0000 7f281259c640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175cadd20 0x565175cb0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.353+0000 7f27f9ffb640 10 client.26006 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.353+0000 7f281259c640 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175cadd20 0x565175cb0140 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d03c62e7182cd02 server_cookie=bbe7cfb6f04f7dd5 in_seq=0 out_seq=0 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.353+0000 7f27f9ffb640 1 -- 172.21.15.64:0/3156035680 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565175b9d780 con 0x565175cadd20 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.353+0000 7f27f9ffb640 10 client.26006 handle_command_reply: tid=0 2024-05-09T09:12:31.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.353+0000 7f281b7ec740 1 client.26006 shutdown 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.354+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 msgr2=0x565175c3fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.354+0000 7f281b7ec740 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3fe80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f280400bdb0 tx=0x7f28040047d0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.355+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 shutdown_connections 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.355+0000 7f281b7ec740 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565175cadd20 0x565175cb0140 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.355+0000 7f281b7ec740 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565175c36c80 0x565175cabde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.355+0000 7f281b7ec740 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565175c31b10 0x565175c3fe80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.355+0000 7f281b7ec740 1 --2- 172.21.15.64:0/3156035680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565175c31120 0x565175c3f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.355+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 >> 172.21.15.64:0/3156035680 conn(0x565175c2c790 msgr2=0x565175c3a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.356+0000 7f27f9ffb640 0 client.26006 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.356+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 shutdown_connections 2024-05-09T09:12:31.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.357+0000 7f281b7ec740 1 -- 172.21.15.64:0/3156035680 wait complete. 2024-05-09T09:12:31.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.358+0000 7f281b7ec740 20 client.26006 trim_cache size 0 max 16384 2024-05-09T09:12:31.358 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:12:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:12:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "f7c9163c-02a9-45c3-8847-2660f9704a49": { 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "f7c9163c-02a9-45c3-8847-2660f9704a49", 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:12:31.359 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:12:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.362+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2d10 msgr2=0x7f28000a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.362+0000 7f2818a00640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2d10 0x7f28000a51d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x565175c31f10 tx=0x7f27fc02b1d0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.362+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2814077430 msgr2=0x7f2814103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.362+0000 7f2818a00640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2814077430 0x7f2814103d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f280400b3f0 tx=0x7f280400b8b0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.364+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 shutdown_connections 2024-05-09T09:12:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.365+0000 7f2818a00640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2d10 0x7f28000a51d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.365+0000 7f2818a00640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814077dd0 0x7f281410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.365+0000 7f2818a00640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2814077430 0x7f2814103d60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.365+0000 7f2818a00640 1 --2- 172.21.15.64:0/1977322869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28140761f0 0x7f2814103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.365+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 >> 172.21.15.64:0/1977322869 conn(0x7f28140fef80 msgr2=0x7f2814100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.366+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 shutdown_connections 2024-05-09T09:12:31.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:12:31.366+0000 7f2818a00640 1 -- 172.21.15.64:0/1977322869 wait complete. 2024-05-09T09:12:31.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:12:31.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:31 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.398 INFO:tasks.cephfs.filesystem:scrub status for tag:f7c9163c-02a9-45c3-8847-2660f9704a49 - {'path': '/', 'tag': 'f7c9163c-02a9-45c3-8847-2660f9704a49', 'options': 'recursive,force'} 2024-05-09T09:12:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.397+0000 7f352a8f8640 1 -- 172.21.15.64:0/542939152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524072b60 msgr2=0x7f3524072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.397+0000 7f352a8f8640 1 --2- 172.21.15.64:0/542939152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524072b60 0x7f3524072f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f351c009ca0 tx=0x7f351c02a8d0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.405+0000 7f352a8f8640 1 -- 172.21.15.64:0/542939152 shutdown_connections 2024-05-09T09:12:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.405+0000 7f352a8f8640 1 --2- 172.21.15.64:0/542939152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524109800 0x7f352410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.405+0000 7f352a8f8640 1 --2- 172.21.15.64:0/542939152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524108e40 0x7f35241092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.405+0000 7f352a8f8640 1 --2- 172.21.15.64:0/542939152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524072b60 0x7f3524072f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.405+0000 7f352a8f8640 1 -- 172.21.15.64:0/542939152 >> 172.21.15.64:0/542939152 conn(0x7f352406e6d0 msgr2=0x7f3524070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.411+0000 7f352a8f8640 1 -- 172.21.15.64:0/542939152 shutdown_connections 2024-05-09T09:12:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.413+0000 7f352a8f8640 1 -- 172.21.15.64:0/542939152 wait complete. 2024-05-09T09:12:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 Processor -- start 2024-05-09T09:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 -- start start 2024-05-09T09:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524072b60 0x7f3524114fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108e40 0x7f3524115510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524109800 0x7f35241b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f352411c140 con 0x7f3524108e40 2024-05-09T09:12:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f352411bfc0 con 0x7f3524072b60 2024-05-09T09:12:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.414+0000 7f352a8f8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f352411c2c0 con 0x7f3524109800 2024-05-09T09:12:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.416+0000 7f352a0f7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524109800 0x7f35241b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.416+0000 7f352a0f7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524109800 0x7f35241b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43992/0 (socket says 172.21.15.64:43992) 2024-05-09T09:12:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.416+0000 7f352a0f7640 1 -- 172.21.15.64:0/790375944 learned_addr learned my addr 172.21.15.64:0/790375944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.418+0000 7f35298f6640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524072b60 0x7f3524114fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.419+0000 7f35290f5640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108e40 0x7f3524115510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.419+0000 7f35298f6640 1 -- 172.21.15.64:0/790375944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524109800 msgr2=0x7f35241b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.419+0000 7f35298f6640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524109800 0x7f35241b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.419+0000 7f35298f6640 1 -- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108e40 msgr2=0x7f3524115510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.419+0000 7f35298f6640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108e40 0x7f3524115510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.419+0000 7f35298f6640 1 -- 172.21.15.64:0/790375944 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35241b6d90 con 0x7f3524072b60 2024-05-09T09:12:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.422+0000 7f35298f6640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524072b60 0x7f3524114fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f351c02aec0 tx=0x7f351c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.424+0000 7f351affd640 1 -- 172.21.15.64:0/790375944 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c004040 con 0x7f3524072b60 2024-05-09T09:12:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.424+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35241b6f60 con 0x7f3524072b60 2024-05-09T09:12:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.424+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35241b73f0 con 0x7f3524072b60 2024-05-09T09:12:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.424+0000 7f351affd640 1 -- 172.21.15.64:0/790375944 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351c004690 con 0x7f3524072b60 2024-05-09T09:12:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.424+0000 7f351affd640 1 -- 172.21.15.64:0/790375944 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c030ec0 con 0x7f3524072b60 2024-05-09T09:12:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.428+0000 7f352e866740 1 Processor -- start 2024-05-09T09:12:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.431+0000 7f351affd640 1 -- 172.21.15.64:0/790375944 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f351c0305d0 con 0x7f3524072b60 2024-05-09T09:12:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.432+0000 7f351affd640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35100a2f30 0x7f35100a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.432+0000 7f351affd640 1 -- 172.21.15.64:0/790375944 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f351c0f0aa0 con 0x7f3524072b60 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.434+0000 7f352e866740 1 -- start start 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352e866740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19400 0x563736b19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352e866740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 0x563736b1e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352e866740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b1ef60 0x563736b213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352e866740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563736a7d340 con 0x563736b1ef60 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352e866740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563736a25290 con 0x563736b19df0 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352e866740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563736a79300 con 0x563736b19400 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f352a0f7640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35100a2f30 0x7f35100a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.435+0000 7f35298f6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19400 0x563736b19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 0x563736b1e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 0x563736b1e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35456/0 (socket says 172.21.15.64:35456) 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 -- 172.21.15.64:0/529059356 learned_addr learned my addr 172.21.15.64:0/529059356 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f352a0f7640 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b1ef60 0x563736b213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 -- 172.21.15.64:0/529059356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19400 msgr2=0x563736b19820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19400 0x563736b19820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 -- 172.21.15.64:0/529059356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b1ef60 msgr2=0x563736b213f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b1ef60 0x563736b213f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.436+0000 7f35290f5640 1 -- 172.21.15.64:0/529059356 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563736a498e0 con 0x563736b19df0 2024-05-09T09:12:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.437+0000 7f352a0f7640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35100a2f30 0x7f35100a53f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f35140156d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.437+0000 7f35290f5640 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 0x563736b1e8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3520006a70 tx=0x7f352000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8169b2d07f10b20f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.439+0000 7f350ffff640 1 -- 172.21.15.64:0/529059356 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352000d950 con 0x563736b19df0 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.439+0000 7f350ffff640 1 -- 172.21.15.64:0/529059356 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3520018070 con 0x563736b19df0 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.439+0000 7f350ffff640 1 -- 172.21.15.64:0/529059356 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35200123e0 con 0x563736b19df0 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.440+0000 7f352e866740 1 -- 172.21.15.64:0/529059356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 msgr2=0x563736b1e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.440+0000 7f352e866740 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 0x563736b1e8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3520006a70 tx=0x7f352000bc80 comp rx=0 tx=0).stop 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.441+0000 7f352e866740 1 -- 172.21.15.64:0/529059356 shutdown_connections 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.441+0000 7f352e866740 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b1ef60 0x563736b213f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.441+0000 7f352e866740 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19df0 0x563736b1e8f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.441+0000 7f352e866740 1 --2- 172.21.15.64:0/529059356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19400 0x563736b19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.441+0000 7f352e866740 1 -- 172.21.15.64:0/529059356 >> 172.21.15.64:0/529059356 conn(0x563736b14a70 msgr2=0x563736b14e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.442+0000 7f352e866740 1 -- 172.21.15.64:0/529059356 shutdown_connections 2024-05-09T09:12:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.443+0000 7f352e866740 1 -- 172.21.15.64:0/529059356 wait complete. 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.443+0000 7f352e866740 1 -- start start 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.443+0000 7f352e866740 1 Processor -- start 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f352e866740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19400 0x563736b23700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f352e866740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b23c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f352e866740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b2ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f352e866740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563736b21f10 con 0x563736b19400 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f352e866740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563736a38790 con 0x563736b1ef60 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f352e866740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563736b22090 con 0x563736b19df0 2024-05-09T09:12:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.444+0000 7f35290f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b23c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b2ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352e866740 10 client.0 _ll_register_callbacks cb 0x563736a73a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b2ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35466/0 (socket says 172.21.15.64:35466) 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 -- 172.21.15.64:0/800260169 learned_addr learned my addr 172.21.15.64:0/800260169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f35298f6640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19400 0x563736b23700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 -- 172.21.15.64:0/800260169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 msgr2=0x563736b23c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b23c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 -- 172.21.15.64:0/800260169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19400 msgr2=0x563736b23700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19400 0x563736b23700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f352a0f7640 1 -- 172.21.15.64:0/800260169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563736791210 con 0x563736b1ef60 2024-05-09T09:12:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.445+0000 7f35298f6640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19400 0x563736b23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.446+0000 7f352a0f7640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b2ac80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35140098b0 tx=0x7f35140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e17bf02d9de09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.446+0000 7f350d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.447+0000 7f350d7fa640 1 -- 172.21.15.64:0/800260169 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514018e40 con 0x563736b1ef60 2024-05-09T09:12:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.447+0000 7f350d7fa640 1 -- 172.21.15.64:0/800260169 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3514017b70 con 0x563736b1ef60 2024-05-09T09:12:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.447+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3520018070 con 0x563736b1ef60 2024-05-09T09:12:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.451+0000 7f350d7fa640 1 -- 172.21.15.64:0/800260169 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3514017d70 con 0x563736b1ef60 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.452+0000 7f352e866740 10 client.26015 fetch_fsmap learned FSMap version 32 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.452+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56373685d620 con 0x563736b1ef60 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f350d7fa640 1 -- 172.21.15.64:0/800260169 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3520018070 con 0x563736b1ef60 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f350d7fa640 1 -- 172.21.15.64:0/800260169 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f35140244a0 con 0x563736b1ef60 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f352e866740 10 client.26015 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f352e866740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f352e866740 20 client.26015 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f352e866740 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b2f2d0 0x563736b31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f352e866740 4 client.26015 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.453+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563736a8aa70 con 0x563736b2f2d0 2024-05-09T09:12:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.454+0000 7f35298f6640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b2f2d0 0x563736b31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.454+0000 7f35298f6640 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b2f2d0 0x563736b31770 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62824fb728775d30 server_cookie=73b93875bb70730f in_seq=0 out_seq=0 2024-05-09T09:12:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.454+0000 7f350d7fa640 10 client.26015 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.456+0000 7f350d7fa640 1 -- 172.21.15.64:0/800260169 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563736a8aa70 con 0x563736b2f2d0 2024-05-09T09:12:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.456+0000 7f350d7fa640 10 client.26015 handle_command_reply: tid=0 2024-05-09T09:12:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.456+0000 7f352e866740 1 client.26015 shutdown 2024-05-09T09:12:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.457+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 msgr2=0x563736b2ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.457+0000 7f352e866740 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b2ac80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35140098b0 tx=0x7f35140046c0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.457+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 shutdown_connections 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b2f2d0 0x563736b31770 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b2ac80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b23c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 --2- 172.21.15.64:0/800260169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19400 0x563736b23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 >> 172.21.15.64:0/800260169 conn(0x563736b14a70 msgr2=0x563736b13bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f350d7fa640 0 client.26015 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 shutdown_connections 2024-05-09T09:12:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 1 -- 172.21.15.64:0/800260169 wait complete. 2024-05-09T09:12:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.458+0000 7f352e866740 20 client.26015 trim_cache size 0 max 16384 2024-05-09T09:12:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.491+0000 7f352e866740 1 Processor -- start 2024-05-09T09:12:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.491+0000 7f352e866740 1 -- start start 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.491+0000 7f352e866740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 0x563736b14e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f352e866740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19df0 0x563736b2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f352e866740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b1ef60 0x563736b37040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f352e866740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35080125c0 con 0x563736b19df0 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f352e866740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f350800fe30 con 0x563736b19400 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f352e866740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f350800dc80 con 0x563736b1ef60 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f35298f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 0x563736b14e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f35298f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 0x563736b14e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35468/0 (socket says 172.21.15.64:35468) 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f35298f6640 1 -- 172.21.15.64:0/2178037098 learned_addr learned my addr 172.21.15.64:0/2178037098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.492+0000 7f352a0f7640 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b1ef60 0x563736b37040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f35298f6640 1 -- 172.21.15.64:0/2178037098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b1ef60 msgr2=0x563736b37040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f35298f6640 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b1ef60 0x563736b37040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f35298f6640 1 -- 172.21.15.64:0/2178037098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19df0 msgr2=0x563736b2aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f35298f6640 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19df0 0x563736b2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f35298f6640 1 -- 172.21.15.64:0/2178037098 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563736b189e0 con 0x563736b19400 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f35298f6640 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 0x563736b14e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f351c0b9370 tx=0x7f351c03e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb9d07d0f667f93b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f352a0f7640 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b1ef60 0x563736b37040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.493+0000 7f350f7fe640 1 -- 172.21.15.64:0/2178037098 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0ca070 con 0x563736b19400 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.495+0000 7f350f7fe640 1 -- 172.21.15.64:0/2178037098 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351c036030 con 0x563736b19400 2024-05-09T09:12:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.495+0000 7f350f7fe640 1 -- 172.21.15.64:0/2178037098 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0c5b80 con 0x563736b19400 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 -- 172.21.15.64:0/2178037098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 msgr2=0x563736b14e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 0x563736b14e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f351c0b9370 tx=0x7f351c03e040 comp rx=0 tx=0).stop 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 -- 172.21.15.64:0/2178037098 shutdown_connections 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b1ef60 0x563736b37040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b19df0 0x563736b2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 --2- 172.21.15.64:0/2178037098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b19400 0x563736b14e50 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f351c0b9370 tx=0x7f351c03e040 comp rx=0 tx=0).stop 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.496+0000 7f352e866740 1 -- 172.21.15.64:0/2178037098 >> 172.21.15.64:0/2178037098 conn(0x563736b14a70 msgr2=0x563736b224c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.497+0000 7f352e866740 1 -- 172.21.15.64:0/2178037098 shutdown_connections 2024-05-09T09:12:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.497+0000 7f352e866740 1 -- 172.21.15.64:0/2178037098 wait complete. 2024-05-09T09:12:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.497+0000 7f352e866740 1 -- start start 2024-05-09T09:12:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.497+0000 7f352e866740 1 Processor -- start 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b28710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b29720 0x563736b29ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f350800e2f0 con 0x563736b29720 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f350800f760 con 0x563736b1ef60 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f350800e9c0 con 0x563736b19df0 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.498+0000 7f352e866740 10 client.0 _ll_register_callbacks cb 0x7f3514009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35298f6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b28710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b28710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:35478/0 (socket says 172.21.15.64:35478) 2024-05-09T09:12:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35298f6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44020/0 (socket says 172.21.15.64:44020) 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 -- 172.21.15.64:0/480900530 learned_addr learned my addr 172.21.15.64:0/480900530 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f352a0f7640 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b29720 0x563736b29ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 -- 172.21.15.64:0/480900530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 msgr2=0x563736b281d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b281d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 -- 172.21.15.64:0/480900530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b29720 msgr2=0x563736b29ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b29720 0x563736b29ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 -- 172.21.15.64:0/480900530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563736b2a0e0 con 0x563736b1ef60 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.499+0000 7f35290f5640 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b28710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35200047b0 tx=0x7f352000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a39c107a29f34e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f350d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f350d7fa640 1 -- 172.21.15.64:0/480900530 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520013680 con 0x563736b1ef60 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f350d7fa640 1 -- 172.21.15.64:0/480900530 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3520013820 con 0x563736b1ef60 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f351c036030 con 0x563736b1ef60 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f350d7fa640 1 -- 172.21.15.64:0/480900530 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3520013a20 con 0x563736b1ef60 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f352e866740 10 client.26021 fetch_fsmap learned FSMap version 32 2024-05-09T09:12:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.500+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563736b95ca0 con 0x563736b1ef60 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f350d7fa640 1 -- 172.21.15.64:0/480900530 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351c036030 con 0x563736b1ef60 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f350d7fa640 1 -- 172.21.15.64:0/480900530 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f352002ece0 con 0x563736b1ef60 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f352e866740 10 client.26021 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f352e866740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f352e866740 20 client.26021 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f352e866740 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b96600 0x563736b98a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f352e866740 4 client.26021 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.501+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563736a84dc0 con 0x563736b96600 2024-05-09T09:12:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.502+0000 7f35298f6640 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b96600 0x563736b98a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.503+0000 7f35298f6640 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b96600 0x563736b98a20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce7ebe748c47661 server_cookie=ba24a97c11563df8 in_seq=0 out_seq=0 2024-05-09T09:12:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.504+0000 7f350d7fa640 10 client.26021 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.549+0000 7f350d7fa640 1 -- 172.21.15.64:0/480900530 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563736a84dc0 con 0x563736b96600 2024-05-09T09:12:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.549+0000 7f350d7fa640 10 client.26021 handle_command_reply: tid=0 2024-05-09T09:12:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.550+0000 7f352e866740 1 client.26021 shutdown 2024-05-09T09:12:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 msgr2=0x563736b28710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b28710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35200047b0 tx=0x7f352000dcc0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 shutdown_connections 2024-05-09T09:12:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563736b96600 0x563736b98a20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563736b29720 0x563736b29ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563736b1ef60 0x563736b28710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 --2- 172.21.15.64:0/480900530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563736b19df0 0x563736b281d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.551+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 >> 172.21.15.64:0/480900530 conn(0x563736b14a70 msgr2=0x563736b14e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.552+0000 7f350d7fa640 0 client.26021 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.552+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 shutdown_connections 2024-05-09T09:12:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.552+0000 7f352e866740 1 -- 172.21.15.64:0/480900530 wait complete. 2024-05-09T09:12:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.552+0000 7f352e866740 20 client.26021 trim_cache size 0 max 16384 2024-05-09T09:12:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:12:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:12:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:12:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:12:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35100a2f30 msgr2=0x7f35100a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35100a2f30 0x7f35100a53f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f35140156d0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524072b60 msgr2=0x7f3524114fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524072b60 0x7f3524114fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f351c02aec0 tx=0x7f351c003c10 comp rx=0 tx=0).stop 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 shutdown_connections 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f35100a2f30 0x7f35100a53f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3524109800 0x7f35241b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108e40 0x7f3524115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 --2- 172.21.15.64:0/790375944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3524072b60 0x7f3524114fd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.554+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 >> 172.21.15.64:0/790375944 conn(0x7f352406e6d0 msgr2=0x7f3524070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.555+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 shutdown_connections 2024-05-09T09:12:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:31.555+0000 7f352a8f8640 1 -- 172.21.15.64:0/790375944 wait complete. 2024-05-09T09:12:31.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:12:31.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:31 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:12:31.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:31.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:31 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:32 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:12:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:32 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:32 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:12:32.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:32 smithi139 ceph-mon[33599]: pgmap v472: 129 pgs: 129 active+clean; 993 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 1.06k op/s 2024-05-09T09:12:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:32 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:32 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:32 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:12:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:32 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:32 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:12:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:32 smithi064 ceph-mon[30149]: pgmap v472: 129 pgs: 129 active+clean; 993 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 1.06k op/s 2024-05-09T09:12:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:32 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:32 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:32.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:32 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:12:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:32 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:32 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:12:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:32 smithi179 ceph-mon[34086]: pgmap v472: 129 pgs: 129 active+clean; 993 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 1.06k op/s 2024-05-09T09:12:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:32 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:12:32.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:32 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:34.572 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:34 smithi139 ceph-mon[33599]: pgmap v473: 129 pgs: 129 active+clean; 1001 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:34.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:34 smithi064 ceph-mon[30149]: pgmap v473: 129 pgs: 129 active+clean; 1001 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:34.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:34 smithi179 ceph-mon[34086]: pgmap v473: 129 pgs: 129 active+clean; 1001 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 135.0 s, 985.4 tps, lat 107.947 ms stddev 140.062 2024-05-09T09:12:36.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:36 smithi064 ceph-mon[30149]: pgmap v474: 129 pgs: 129 active+clean; 1009 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:12:36.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:36 smithi179 ceph-mon[34086]: pgmap v474: 129 pgs: 129 active+clean; 1009 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:12:36.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:36 smithi139 ceph-mon[33599]: pgmap v474: 129 pgs: 129 active+clean; 1009 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-05-09T09:12:37.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:37 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:37.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:37 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:37.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:37 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:38.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:38 smithi064 ceph-mon[30149]: pgmap v475: 129 pgs: 129 active+clean; 1009 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.3 MiB/s wr, 1.04k op/s 2024-05-09T09:12:38.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:38 smithi179 ceph-mon[34086]: pgmap v475: 129 pgs: 129 active+clean; 1009 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.3 MiB/s wr, 1.04k op/s 2024-05-09T09:12:38.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:38 smithi139 ceph-mon[33599]: pgmap v475: 129 pgs: 129 active+clean; 1009 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.3 MiB/s wr, 1.04k op/s 2024-05-09T09:12:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:40 smithi064 ceph-mon[30149]: pgmap v476: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-05-09T09:12:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:12:40.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:12:40.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:40 smithi179 ceph-mon[34086]: pgmap v476: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-05-09T09:12:40.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:12:40.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:12:40.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:40 smithi139 ceph-mon[33599]: pgmap v476: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-05-09T09:12:40.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:12:40.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:12:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 140.0 s, 951.7 tps, lat 105.358 ms stddev 44.588 2024-05-09T09:12:41.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:41 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:12:41.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:41 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:12:41.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:41 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:12:42.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:42 smithi064 ceph-mon[30149]: pgmap v477: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 9.9 MiB/s wr, 1.09k op/s 2024-05-09T09:12:42.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:42 smithi179 ceph-mon[34086]: pgmap v477: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 9.9 MiB/s wr, 1.09k op/s 2024-05-09T09:12:42.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:42 smithi139 ceph-mon[33599]: pgmap v477: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 9.9 MiB/s wr, 1.09k op/s 2024-05-09T09:12:44.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:44 smithi064 ceph-mon[30149]: pgmap v478: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-05-09T09:12:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.719+0000 7f3c35770640 1 -- 172.21.15.64:0/4066695467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c301005e0 msgr2=0x7f3c30100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.719+0000 7f3c35770640 1 --2- 172.21.15.64:0/4066695467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c301005e0 0x7f3c30100a00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009ca0 tx=0x7f3c2802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 -- 172.21.15.64:0/4066695467 shutdown_connections 2024-05-09T09:12:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 --2- 172.21.15.64:0/4066695467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c30102190 0x7f3c30108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 --2- 172.21.15.64:0/4066695467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c301017d0 0x7f3c30101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 --2- 172.21.15.64:0/4066695467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c301005e0 0x7f3c30100a00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 -- 172.21.15.64:0/4066695467 >> 172.21.15.64:0/4066695467 conn(0x7f3c300fbb50 msgr2=0x7f3c300fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 -- 172.21.15.64:0/4066695467 shutdown_connections 2024-05-09T09:12:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.720+0000 7f3c35770640 1 -- 172.21.15.64:0/4066695467 wait complete. 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.721+0000 7f3c35770640 1 Processor -- start 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.721+0000 7f3c35770640 1 -- start start 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c35770640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c301005e0 0x7f3c3010ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c35770640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 0x7f3c301092a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c35770640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c30102190 0x7f3c301097e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c35770640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3010d230 con 0x7f3c30102190 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c35770640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3010d0b0 con 0x7f3c301005e0 2024-05-09T09:12:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c35770640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3010d3b0 con 0x7f3c301017d0 2024-05-09T09:12:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c2effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c301005e0 0x7f3c3010ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c2e7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 0x7f3c301092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c2e7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 0x7f3c301092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43156/0 (socket says 172.21.15.64:43156) 2024-05-09T09:12:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/3364620564 learned_addr learned my addr 172.21.15.64:0/3364620564 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.722+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c30102190 0x7f3c301097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/3364620564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c301005e0 msgr2=0x7f3c3010ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c301005e0 0x7f3c3010ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c30102190 msgr2=0x7f3c301097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c30102190 0x7f3c301097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/3364620564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c30109d20 con 0x7f3c301017d0 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c30102190 0x7f3c301097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 0x7f3c301092a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c890 tx=0x7f3c2000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c17fff640 1 -- 172.21.15.64:0/3364620564 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20004220 con 0x7f3c301017d0 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c17fff640 1 -- 172.21.15.64:0/3364620564 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c200043c0 con 0x7f3c301017d0 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.723+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3010a010 con 0x7f3c301017d0 2024-05-09T09:12:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.724+0000 7f3c17fff640 1 -- 172.21.15.64:0/3364620564 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20004a00 con 0x7f3c301017d0 2024-05-09T09:12:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.724+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c301a6200 con 0x7f3c301017d0 2024-05-09T09:12:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.727+0000 7f3c38483740 1 Processor -- start 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c17fff640 1 -- 172.21.15.64:0/3364620564 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c20004ba0 con 0x7f3c301017d0 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 -- start start 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 0x55bee80f0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0df0 0x55bee80f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f5f60 0x55bee80f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bee8054340 con 0x55bee80f0400 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bee7ffc290 con 0x55bee80f0df0 2024-05-09T09:12:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.728+0000 7f3c38483740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bee8050300 con 0x55bee80f5f60 2024-05-09T09:12:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 0x55bee80f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 0x55bee80f0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48488/0 (socket says 172.21.15.64:48488) 2024-05-09T09:12:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/2385663162 learned_addr learned my addr 172.21.15.64:0/2385663162 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:44.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:44 smithi179 ceph-mon[34086]: pgmap v478: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-05-09T09:12:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c17fff640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c180a2c80 0x7f3c180a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f5f60 0x55bee80f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/2385663162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f5f60 msgr2=0x55bee80f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f5f60 0x55bee80f83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/2385663162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0df0 msgr2=0x55bee80f58f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0df0 0x55bee80f58f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.729+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/2385663162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee80208e0 con 0x55bee80f0400 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.730+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 0x55bee80f0820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400bcf0 tx=0x7f3c2400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11013c888c8a7fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.730+0000 7f3c17fff640 1 -- 172.21.15.64:0/3364620564 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3c200cf0a0 con 0x7f3c301017d0 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.730+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c180a2c80 0x7f3c180a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.730+0000 7f3c157fa640 1 -- 172.21.15.64:0/2385663162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24015070 con 0x55bee80f0400 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.730+0000 7f3c157fa640 1 -- 172.21.15.64:0/2385663162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c2401ab10 con 0x55bee80f0400 2024-05-09T09:12:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.731+0000 7f3c157fa640 1 -- 172.21.15.64:0/2385663162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24011c80 con 0x55bee80f0400 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.731+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c180a2c80 0x7f3c180a5140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3c24011fd0 tx=0x7f3c24011f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.731+0000 7f3c38483740 1 -- 172.21.15.64:0/2385663162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 msgr2=0x55bee80f0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.731+0000 7f3c38483740 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 0x55bee80f0820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400bcf0 tx=0x7f3c2400f730 comp rx=0 tx=0).stop 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 -- 172.21.15.64:0/2385663162 shutdown_connections 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f5f60 0x55bee80f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0df0 0x55bee80f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 --2- 172.21.15.64:0/2385663162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0400 0x55bee80f0820 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 -- 172.21.15.64:0/2385663162 >> 172.21.15.64:0/2385663162 conn(0x55bee80eba70 msgr2=0x55bee80ebe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 -- 172.21.15.64:0/2385663162 shutdown_connections 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.732+0000 7f3c38483740 1 -- 172.21.15.64:0/2385663162 wait complete. 2024-05-09T09:12:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.733+0000 7f3c38483740 1 -- start start 2024-05-09T09:12:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.733+0000 7f3c38483740 1 Processor -- start 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bee80f8f60 con 0x55bee80f5f60 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bee800f790 con 0x55bee80f0400 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bee80f90e0 con 0x55bee80f0df0 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c38483740 10 client.0 _ll_register_callbacks cb 0x55bee804aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c2effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.734+0000 7f3c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48494/0 (socket says 172.21.15.64:48494) 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/1418673030 learned_addr learned my addr 172.21.15.64:0/1418673030 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/1418673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 msgr2=0x55bee80fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 msgr2=0x55bee80fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/1418673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee7d68210 con 0x55bee80f5f60 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101c00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005c50 tx=0x7f3c2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6083529c193d6056 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.735+0000 7f3c12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c12ff5640 1 -- 172.21.15.64:0/1418673030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24015050 con 0x55bee80f5f60 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2401ab10 con 0x55bee80f5f60 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c12ff5640 1 -- 172.21.15.64:0/1418673030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3c24011980 con 0x55bee80f5f60 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c12ff5640 1 -- 172.21.15.64:0/1418673030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3c2403d050 con 0x55bee80f5f60 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c38483740 10 client.16503 fetch_fsmap learned FSMap version 33 2024-05-09T09:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bee7e34620 con 0x55bee80f5f60 2024-05-09T09:12:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c12ff5640 1 -- 172.21.15.64:0/1418673030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c24034070 con 0x55bee80f5f60 2024-05-09T09:12:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c12ff5640 1 -- 172.21.15.64:0/1418673030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3c2401a460 con 0x55bee80f5f60 2024-05-09T09:12:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.736+0000 7f3c38483740 10 client.16503 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T09:12:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c38483740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c38483740 20 client.16503 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c38483740 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee8105e80 0x55bee8108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c38483740 4 client.16503 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3c2401ab10 con 0x55bee8105e80 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c2effd640 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee8105e80 0x55bee8108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c2effd640 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee8105e80 0x55bee8108270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305a9fa8d9e656c0 server_cookie=621c4865553a4913 in_seq=0 out_seq=0 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.737+0000 7f3c12ff5640 10 client.16503 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c12ff5640 1 -- 172.21.15.64:0/1418673030 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3c2401ab10 con 0x55bee8105e80 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c12ff5640 10 client.16503 handle_command_reply: tid=0 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 client.16503 shutdown 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 msgr2=0x55bee8101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101c00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005c50 tx=0x7f3c2401a000 comp rx=0 tx=0).stop 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 shutdown_connections 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee8105e80 0x55bee8108270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101c00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c12ff5640 0 client.16503 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 --2- 172.21.15.64:0/1418673030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 >> 172.21.15.64:0/1418673030 conn(0x55bee80eba70 msgr2=0x55bee80eabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 shutdown_connections 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 1 -- 172.21.15.64:0/1418673030 wait complete. 2024-05-09T09:12:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.738+0000 7f3c38483740 20 client.16503 trim_cache size 0 max 16384 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 Processor -- start 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 -- start start 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80eef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b2ff0 con 0x55bee80f5f60 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b0890 con 0x55bee80f0400 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c38483740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301ae6e0 con 0x55bee80f0df0 2024-05-09T09:12:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48508/0 (socket says 172.21.15.64:48508) 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.753+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/787054522 learned_addr learned my addr 172.21.15.64:0/787054522 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2effd640 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/787054522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 msgr2=0x55bee80ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/787054522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 msgr2=0x55bee80eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2f7fe640 1 -- 172.21.15.64:0/787054522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee81085e0 con 0x55bee80f5f60 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101bc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3c24000c00 tx=0x7f3c2400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cc6a0467574ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c14ff9640 1 -- 172.21.15.64:0/787054522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24015070 con 0x55bee80f5f60 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c14ff9640 1 -- 172.21.15.64:0/787054522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c24034040 con 0x55bee80f5f60 2024-05-09T09:12:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.754+0000 7f3c14ff9640 1 -- 172.21.15.64:0/787054522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24019bc0 con 0x55bee80f5f60 2024-05-09T09:12:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- 172.21.15.64:0/787054522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 msgr2=0x55bee8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101bc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3c24000c00 tx=0x7f3c2400f730 comp rx=0 tx=0).stop 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- 172.21.15.64:0/787054522 shutdown_connections 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f5f60 0x55bee8101bc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0df0 0x55bee80ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- 172.21.15.64:0/787054522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f0400 0x55bee80eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- 172.21.15.64:0/787054522 >> 172.21.15.64:0/787054522 conn(0x55bee80eba70 msgr2=0x55bee80ee7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- 172.21.15.64:0/787054522 shutdown_connections 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- 172.21.15.64:0/787054522 wait complete. 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- start start 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 Processor -- start 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0400 0x55bee80fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 0x55bee80ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f5f60 0x55bee8100250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301aed50 con 0x55bee80f0df0 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b01c0 con 0x55bee80f5f60 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301af420 con 0x55bee80f0400 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.755+0000 7f3c38483740 10 client.0 _ll_register_callbacks cb 0x7f3c2400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 0x55bee80ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 0x55bee80ff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48518/0 (socket says 172.21.15.64:48518) 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/965868411 learned_addr learned my addr 172.21.15.64:0/965868411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f5f60 0x55bee8100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2effd640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0400 0x55bee80fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/965868411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0400 msgr2=0x55bee80fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0400 0x55bee80fed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f5f60 msgr2=0x55bee8100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f5f60 0x55bee8100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 -- 172.21.15.64:0/965868411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee8100840 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2f7fe640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f5f60 0x55bee8100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c2e7fc640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 0x55bee80ff240 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3c200860d0 tx=0x7f3c20097870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc2bb381419d321 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c12ff5640 1 -- 172.21.15.64:0/965868411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200b3060 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c12ff5640 1 -- 172.21.15.64:0/965868411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3c20099810 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c24034040 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c12ff5640 1 -- 172.21.15.64:0/965868411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3c2001d040 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 10 client.16509 fetch_fsmap learned FSMap version 33 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bee8101bf0 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c12ff5640 1 -- 172.21.15.64:0/965868411 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c24034040 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c12ff5640 1 -- 172.21.15.64:0/965868411 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3c200d10b0 con 0x55bee80f0df0 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 10 client.16509 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 20 client.16509 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.756+0000 7f3c38483740 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee816c910 0x55bee816ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.757+0000 7f3c38483740 4 client.16509 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:12:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.757+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bee8061a70 con 0x55bee816c910 2024-05-09T09:12:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.757+0000 7f3c2effd640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee816c910 0x55bee816ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.757+0000 7f3c2effd640 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee816c910 0x55bee816ed30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d355ea9c0de70e5c server_cookie=46bd68f35f75fe66 in_seq=0 out_seq=0 2024-05-09T09:12:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.757+0000 7f3c12ff5640 10 client.16509 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.823+0000 7f3c12ff5640 1 -- 172.21.15.64:0/965868411 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bee8061a70 con 0x55bee816c910 2024-05-09T09:12:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.823+0000 7f3c12ff5640 10 client.16509 handle_command_reply: tid=0 2024-05-09T09:12:44.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:44 smithi139 ceph-mon[33599]: pgmap v478: 129 pgs: 129 active+clean; 1009 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-05-09T09:12:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.826+0000 7f3c38483740 1 client.16509 shutdown 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.830+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 msgr2=0x55bee80ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 0x55bee80ff240 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3c200860d0 tx=0x7f3c20097870 comp rx=0 tx=0).stop 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 shutdown_connections 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bee816c910 0x55bee816ed30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c12ff5640 0 client.16509 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bee80f5f60 0x55bee8100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bee80f0df0 0x55bee80ff240 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 --2- 172.21.15.64:0/965868411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bee80f0400 0x55bee80fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 >> 172.21.15.64:0/965868411 conn(0x55bee80eba70 msgr2=0x55bee80ef080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.831+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 shutdown_connections 2024-05-09T09:12:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.832+0000 7f3c38483740 1 -- 172.21.15.64:0/965868411 wait complete. 2024-05-09T09:12:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.832+0000 7f3c38483740 20 client.16509 trim_cache size 0 max 16384 2024-05-09T09:12:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:12:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:12:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:12:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c180a2c80 msgr2=0x7f3c180a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c180a2c80 0x7f3c180a5140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3c24011fd0 tx=0x7f3c24011f00 comp rx=0 tx=0).stop 2024-05-09T09:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 msgr2=0x7f3c301092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 0x7f3c301092a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c890 tx=0x7f3c2000cd50 comp rx=0 tx=0).stop 2024-05-09T09:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 shutdown_connections 2024-05-09T09:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3c180a2c80 0x7f3c180a5140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c30102190 0x7f3c301097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c301017d0 0x7f3c301092a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 --2- 172.21.15.64:0/3364620564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c301005e0 0x7f3c3010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.837+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 >> 172.21.15.64:0/3364620564 conn(0x7f3c300fbb50 msgr2=0x7f3c30104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.838+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 shutdown_connections 2024-05-09T09:12:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:44.838+0000 7f3c35770640 1 -- 172.21.15.64:0/3364620564 wait complete. 2024-05-09T09:12:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 145.0 s, 1052.5 tps, lat 94.615 ms stddev 27.936 2024-05-09T09:12:46.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:46 smithi064 ceph-mon[30149]: pgmap v479: 129 pgs: 129 active+clean; 1009 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:12:46.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:46 smithi179 ceph-mon[34086]: pgmap v479: 129 pgs: 129 active+clean; 1009 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:12:46.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:46 smithi139 ceph-mon[33599]: pgmap v479: 129 pgs: 129 active+clean; 1009 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:12:48.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:48 smithi064 ceph-mon[30149]: pgmap v480: 129 pgs: 129 active+clean; 1009 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.0 MiB/s wr, 935 op/s 2024-05-09T09:12:48.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:48 smithi179 ceph-mon[34086]: pgmap v480: 129 pgs: 129 active+clean; 1009 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.0 MiB/s wr, 935 op/s 2024-05-09T09:12:48.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:48 smithi139 ceph-mon[33599]: pgmap v480: 129 pgs: 129 active+clean; 1009 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.0 MiB/s wr, 935 op/s 2024-05-09T09:12:50.624 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:50 smithi139 ceph-mon[33599]: pgmap v481: 129 pgs: 129 active+clean; 1009 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:50 smithi064 ceph-mon[30149]: pgmap v481: 129 pgs: 129 active+clean; 1009 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:50.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:50 smithi179 ceph-mon[34086]: pgmap v481: 129 pgs: 129 active+clean; 1009 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:12:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 150.0 s, 1083.7 tps, lat 92.577 ms stddev 36.888 2024-05-09T09:12:52.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:52 smithi064 ceph-mon[30149]: pgmap v482: 129 pgs: 129 active+clean; 1009 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.3 MiB/s wr, 974 op/s 2024-05-09T09:12:52.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:52 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:52.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:52 smithi179 ceph-mon[34086]: pgmap v482: 129 pgs: 129 active+clean; 1009 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.3 MiB/s wr, 974 op/s 2024-05-09T09:12:52.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:52 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:52.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:52 smithi139 ceph-mon[33599]: pgmap v482: 129 pgs: 129 active+clean; 1009 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.3 MiB/s wr, 974 op/s 2024-05-09T09:12:52.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:52 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:54.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:54 smithi064 ceph-mon[30149]: pgmap v483: 129 pgs: 129 active+clean; 1009 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.9 MiB/s wr, 1.15k op/s 2024-05-09T09:12:54.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:54 smithi179 ceph-mon[34086]: pgmap v483: 129 pgs: 129 active+clean; 1009 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.9 MiB/s wr, 1.15k op/s 2024-05-09T09:12:54.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:54 smithi139 ceph-mon[33599]: pgmap v483: 129 pgs: 129 active+clean; 1009 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.9 MiB/s wr, 1.15k op/s 2024-05-09T09:12:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 155.0 s, 1143.0 tps, lat 87.418 ms stddev 52.041 2024-05-09T09:12:56.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:56 smithi064 ceph-mon[30149]: pgmap v484: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-05-09T09:12:56.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:56 smithi179 ceph-mon[34086]: pgmap v484: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-05-09T09:12:56.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:56 smithi139 ceph-mon[33599]: pgmap v484: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-05-09T09:12:57.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:57 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:57.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:57 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:57 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:12:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.996+0000 7fb0931a9640 1 -- 172.21.15.64:0/1110947942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c100100 msgr2=0x7fb08c102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.997+0000 7fb0931a9640 1 --2- 172.21.15.64:0/1110947942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c100100 0x7fb08c102530 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb074009900 tx=0x7fb07402a410 comp rx=0 tx=0).stop 2024-05-09T09:12:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.997+0000 7fb0931a9640 1 -- 172.21.15.64:0/1110947942 shutdown_connections 2024-05-09T09:12:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.997+0000 7fb0931a9640 1 --2- 172.21.15.64:0/1110947942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c105420 0x7fb08c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.997+0000 7fb0931a9640 1 --2- 172.21.15.64:0/1110947942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08c102a70 0x7fb08c104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.997+0000 7fb0931a9640 1 --2- 172.21.15.64:0/1110947942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c100100 0x7fb08c102530 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.997+0000 7fb0931a9640 1 -- 172.21.15.64:0/1110947942 >> 172.21.15.64:0/1110947942 conn(0x7fb08c0fbd00 msgr2=0x7fb08c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:57.999+0000 7fb0931a9640 1 -- 172.21.15.64:0/1110947942 shutdown_connections 2024-05-09T09:12:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.000+0000 7fb0931a9640 1 -- 172.21.15.64:0/1110947942 wait complete. 2024-05-09T09:12:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 Processor -- start 2024-05-09T09:12:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 -- start start 2024-05-09T09:12:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 0x7fb08c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c102a70 0x7fb08c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08c105420 0x7fb08c1a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c113a70 con 0x7fb08c102a70 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1138f0 con 0x7fb08c105420 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb0931a9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c113bf0 con 0x7fb08c100100 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb090f45640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 0x7fb08c19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb090f45640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 0x7fb08c19e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33204/0 (socket says 172.21.15.64:33204) 2024-05-09T09:12:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.001+0000 7fb090f45640 1 -- 172.21.15.64:0/264542403 learned_addr learned my addr 172.21.15.64:0/264542403 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb08bfff640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c102a70 0x7fb08c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb091746640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08c105420 0x7fb08c1a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb090f45640 1 -- 172.21.15.64:0/264542403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08c105420 msgr2=0x7fb08c1a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb090f45640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08c105420 0x7fb08c1a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb090f45640 1 -- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c102a70 msgr2=0x7fb08c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb090f45640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c102a70 0x7fb08c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb090f45640 1 -- 172.21.15.64:0/264542403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c1a5ca0 con 0x7fb08c100100 2024-05-09T09:12:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.002+0000 7fb08bfff640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c102a70 0x7fb08c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:12:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.004+0000 7fb090f45640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 0x7fb08c19e080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0740314e0 tx=0x7fb074003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.004+0000 7fb089ffb640 1 -- 172.21.15.64:0/264542403 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07403f070 con 0x7fb08c100100 2024-05-09T09:12:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.004+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c1a5f30 con 0x7fb08c100100 2024-05-09T09:12:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.004+0000 7fb089ffb640 1 -- 172.21.15.64:0/264542403 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb074031c40 con 0x7fb08c100100 2024-05-09T09:12:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.004+0000 7fb089ffb640 1 -- 172.21.15.64:0/264542403 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074035030 con 0x7fb08c100100 2024-05-09T09:12:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.005+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a6410 con 0x7fb08c100100 2024-05-09T09:12:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 Processor -- start 2024-05-09T09:12:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 -- start start 2024-05-09T09:12:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef17670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 0x5599cef1c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef1f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5599cee7b340 con 0x5599cef17250 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5599cee23290 con 0x5599cef17c40 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.010+0000 7fb095f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5599cee77300 con 0x5599cef1cdb0 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef17670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef17670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33066/0 (socket says 172.21.15.64:33066) 2024-05-09T09:12:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 -- 172.21.15.64:0/604849532 learned_addr learned my addr 172.21.15.64:0/604849532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 -- 172.21.15.64:0/604849532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 msgr2=0x5599cef1f240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef1f240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 -- 172.21.15.64:0/604849532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 msgr2=0x5599cef1c740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb089ffb640 1 -- 172.21.15.64:0/264542403 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb074042070 con 0x7fb08c100100 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 0x5599cef1c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.011+0000 7fb091746640 1 -- 172.21.15.64:0/604849532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cee478e0 con 0x5599cef17250 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.012+0000 7fb091746640 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef17670 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb084009c80 tx=0x7fb084011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed23073378ee638 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.012+0000 7fb082ffd640 1 -- 172.21.15.64:0/604849532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004220 con 0x5599cef17250 2024-05-09T09:12:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.012+0000 7fb082ffd640 1 -- 172.21.15.64:0/604849532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb084009e30 con 0x5599cef17250 2024-05-09T09:12:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.012+0000 7fb082ffd640 1 -- 172.21.15.64:0/604849532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084019760 con 0x5599cef17250 2024-05-09T09:12:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.013+0000 7fb095f96740 1 -- 172.21.15.64:0/604849532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 msgr2=0x5599cef17670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.013+0000 7fb095f96740 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef17670 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb084009c80 tx=0x7fb084011810 comp rx=0 tx=0).stop 2024-05-09T09:12:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.014+0000 7fb089ffb640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0780a2d30 0x7fb0780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.014+0000 7fb089ffb640 1 -- 172.21.15.64:0/264542403 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0740ef910 con 0x7fb08c100100 2024-05-09T09:12:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb091746640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0780a2d30 0x7fb0780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb095f96740 1 -- 172.21.15.64:0/604849532 shutdown_connections 2024-05-09T09:12:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb095f96740 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef1f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb095f96740 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 0x5599cef1c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb095f96740 1 --2- 172.21.15.64:0/604849532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef17670 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb095f96740 1 -- 172.21.15.64:0/604849532 >> 172.21.15.64:0/604849532 conn(0x5599cef128c0 msgr2=0x5599cef12cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.015+0000 7fb095f96740 1 -- 172.21.15.64:0/604849532 shutdown_connections 2024-05-09T09:12:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.016+0000 7fb095f96740 1 -- 172.21.15.64:0/604849532 wait complete. 2024-05-09T09:12:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.016+0000 7fb095f96740 1 -- start start 2024-05-09T09:12:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 Processor -- start 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef21530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 0x5599cef21a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef28ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5599cef1fd60 con 0x5599cef17250 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5599cee36790 con 0x5599cef17c40 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.017+0000 7fb095f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5599cef1fee0 con 0x5599cef1cdb0 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb095f96740 10 client.0 _ll_register_callbacks cb 0x5599cee71a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb090f45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef21530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb090f45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef21530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33076/0 (socket says 172.21.15.64:33076) 2024-05-09T09:12:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb090f45640 1 -- 172.21.15.64:0/3711600046 learned_addr learned my addr 172.21.15.64:0/3711600046 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb091746640 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef28ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb091746640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0780a2d30 0x7fb0780a51f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb084009a90 tx=0x7fb084000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb090f45640 1 -- 172.21.15.64:0/3711600046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 msgr2=0x5599cef28ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.018+0000 7fb090f45640 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef28ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb090f45640 1 -- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 msgr2=0x5599cef21a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb090f45640 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 0x5599cef21a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb090f45640 1 -- 172.21.15.64:0/3711600046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ceb8f210 con 0x5599cef17250 2024-05-09T09:12:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb090f45640 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef21530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb07402a9d0 tx=0x7fb0740bb790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ccf7e3d019bc55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb06bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb06bfff640 1 -- 172.21.15.64:0/3711600046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0740c49f0 con 0x5599cef17250 2024-05-09T09:12:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.019+0000 7fb06bfff640 1 -- 172.21.15.64:0/3711600046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0740b84a0 con 0x5599cef17250 2024-05-09T09:12:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.020+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb084009e30 con 0x5599cef17250 2024-05-09T09:12:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.021+0000 7fb06bfff640 1 -- 172.21.15.64:0/3711600046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0740bc090 con 0x5599cef17250 2024-05-09T09:12:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.021+0000 7fb095f96740 10 client.16521 fetch_fsmap learned FSMap version 35 2024-05-09T09:12:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.021+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599cec5b620 con 0x5599cef17250 2024-05-09T09:12:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.024+0000 7fb06bfff640 1 -- 172.21.15.64:0/3711600046 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb084009e30 con 0x5599cef17250 2024-05-09T09:12:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb06bfff640 1 -- 172.21.15.64:0/3711600046 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fb0740bb280 con 0x5599cef17250 2024-05-09T09:12:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb095f96740 10 client.16521 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:12:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb095f96740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb095f96740 20 client.16521 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb095f96740 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cef3e280 0x5599cef40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb095f96740 4 client.16521 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:12:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599cee88a70 con 0x5599cef3e280 2024-05-09T09:12:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.025+0000 7fb08bfff640 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cef3e280 0x5599cef40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.026+0000 7fb06bfff640 10 client.16521 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.026+0000 7fb08bfff640 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cef3e280 0x5599cef40720 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9680aac7d8981bbc server_cookie=55a924b4d0ae99c9 in_seq=0 out_seq=0 2024-05-09T09:12:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.027+0000 7fb06bfff640 1 -- 172.21.15.64:0/3711600046 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599cee88a70 con 0x5599cef3e280 2024-05-09T09:12:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.027+0000 7fb06bfff640 10 client.16521 handle_command_reply: tid=0 2024-05-09T09:12:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.029+0000 7fb095f96740 1 client.16521 shutdown 2024-05-09T09:12:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 msgr2=0x5599cef21530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef21530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb07402a9d0 tx=0x7fb0740bb790 comp rx=0 tx=0).stop 2024-05-09T09:12:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 shutdown_connections 2024-05-09T09:12:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cef3e280 0x5599cef40720 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef28ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17c40 0x5599cef21a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 --2- 172.21.15.64:0/3711600046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef21530 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 >> 172.21.15.64:0/3711600046 conn(0x5599cef128c0 msgr2=0x5599cef11a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.030+0000 7fb06bfff640 0 client.16521 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.031+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 shutdown_connections 2024-05-09T09:12:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.031+0000 7fb095f96740 1 -- 172.21.15.64:0/3711600046 wait complete. 2024-05-09T09:12:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.032+0000 7fb095f96740 20 client.16521 trim_cache size 0 max 16384 2024-05-09T09:12:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.061+0000 7fb095f96740 1 Processor -- start 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.061+0000 7fb095f96740 1 -- start start 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb095f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 0x5599cef15f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb095f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17c40 0x5599cef28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb095f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef46030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb095f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1aaaf0 con 0x5599cef17c40 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb095f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1a8360 con 0x5599cef17250 2024-05-09T09:12:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb095f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c0735c0 con 0x5599cef1cdb0 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb08bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17c40 0x5599cef28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 0x5599cef15f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 0x5599cef15f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41682/0 (socket says 172.21.15.64:41682) 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 -- 172.21.15.64:0/2754900077 learned_addr learned my addr 172.21.15.64:0/2754900077 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb08bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17c40 0x5599cef28a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33088/0 (socket says 172.21.15.64:33088) 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb091746640 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef46030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 -- 172.21.15.64:0/2754900077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 msgr2=0x5599cef46030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef46030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 -- 172.21.15.64:0/2754900077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17c40 msgr2=0x5599cef28a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17c40 0x5599cef28a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.062+0000 7fb090f45640 1 -- 172.21.15.64:0/2754900077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cef10f00 con 0x5599cef17250 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.063+0000 7fb090f45640 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 0x5599cef15f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb0740bb6a0 tx=0x7fb0740ba340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8dc1bee71eedf6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.063+0000 7fb0827fc640 1 -- 172.21.15.64:0/2754900077 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0740c49f0 con 0x5599cef17250 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.063+0000 7fb0827fc640 1 -- 172.21.15.64:0/2754900077 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb07403b760 con 0x5599cef17250 2024-05-09T09:12:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.063+0000 7fb0827fc640 1 -- 172.21.15.64:0/2754900077 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0740b85b0 con 0x5599cef17250 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.064+0000 7fb095f96740 1 -- 172.21.15.64:0/2754900077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 msgr2=0x5599cef15f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.064+0000 7fb095f96740 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 0x5599cef15f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb0740bb6a0 tx=0x7fb0740ba340 comp rx=0 tx=0).stop 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 -- 172.21.15.64:0/2754900077 shutdown_connections 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef1cdb0 0x5599cef46030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17c40 0x5599cef28a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 --2- 172.21.15.64:0/2754900077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef17250 0x5599cef15f80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 -- 172.21.15.64:0/2754900077 >> 172.21.15.64:0/2754900077 conn(0x5599cef128c0 msgr2=0x5599cef15870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 -- 172.21.15.64:0/2754900077 shutdown_connections 2024-05-09T09:12:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 -- 172.21.15.64:0/2754900077 wait complete. 2024-05-09T09:12:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 -- start start 2024-05-09T09:12:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.065+0000 7fb095f96740 1 Processor -- start 2024-05-09T09:12:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef17c40 0x5599cef26ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 0x5599cef27ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1a6820 con 0x5599cef17250 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1a7c90 con 0x5599cef1cdb0 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1a6ef0 con 0x5599cef17c40 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb095f96740 10 client.0 _ll_register_callbacks cb 0x7fb0740bbc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb091746640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 0x5599cef27ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb090f45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb091746640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 0x5599cef27ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41694/0 (socket says 172.21.15.64:41694) 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb091746640 1 -- 172.21.15.64:0/3786174549 learned_addr learned my addr 172.21.15.64:0/3786174549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:12:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.066+0000 7fb08bfff640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef17c40 0x5599cef26ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb090f45640 1 -- 172.21.15.64:0/3786174549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef17c40 msgr2=0x5599cef26ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb090f45640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef17c40 0x5599cef26ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb090f45640 1 -- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 msgr2=0x5599cef27ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:12:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb090f45640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 0x5599cef27ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb090f45640 1 -- 172.21.15.64:0/3786174549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cef280c0 con 0x5599cef17250 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb091746640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 0x5599cef27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb090f45640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef26580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb0740b8290 tx=0x7fb0740c48a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb7ded3f82f53a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.067+0000 7fb06bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.068+0000 7fb06bfff640 1 -- 172.21.15.64:0/3786174549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0740c49f0 con 0x5599cef17250 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.068+0000 7fb06bfff640 1 -- 172.21.15.64:0/3786174549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb074031790 con 0x5599cef17250 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.068+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07403b760 con 0x5599cef17250 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.068+0000 7fb06bfff640 1 -- 172.21.15.64:0/3786174549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0740bbe30 con 0x5599cef17250 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.068+0000 7fb095f96740 10 client.16527 fetch_fsmap learned FSMap version 35 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.068+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599cef280c0 con 0x5599cef17250 2024-05-09T09:12:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb06bfff640 1 -- 172.21.15.64:0/3786174549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0740cb080 con 0x5599cef17250 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb06bfff640 1 -- 172.21.15.64:0/3786174549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fb0740bacf0 con 0x5599cef17250 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb095f96740 10 client.16527 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb095f96740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb095f96740 20 client.16527 populate_metadata read hostname 'smithi064' 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb095f96740 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cefa5240 0x5599cefa7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb095f96740 4 client.16527 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:12:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb07403b760 con 0x5599cefa5240 2024-05-09T09:12:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.069+0000 7fb08bfff640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cefa5240 0x5599cefa7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:12:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.070+0000 7fb06bfff640 10 client.16527 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.070+0000 7fb08bfff640 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cefa5240 0x5599cefa7660 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65d3301480cf920 server_cookie=a095da2299f36762 in_seq=0 out_seq=0 2024-05-09T09:12:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.109+0000 7fb06bfff640 1 -- 172.21.15.64:0/3786174549 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb07403b760 con 0x5599cefa5240 2024-05-09T09:12:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.109+0000 7fb06bfff640 10 client.16527 handle_command_reply: tid=0 2024-05-09T09:12:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.109+0000 7fb095f96740 1 client.16527 shutdown 2024-05-09T09:12:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.109+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 msgr2=0x5599cef26580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.109+0000 7fb095f96740 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef26580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb0740b8290 tx=0x7fb0740c48a0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 shutdown_connections 2024-05-09T09:12:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5599cefa5240 0x5599cefa7660 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599cef1cdb0 0x5599cef27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5599cef17c40 0x5599cef26ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 --2- 172.21.15.64:0/3786174549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599cef17250 0x5599cef26580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 >> 172.21.15.64:0/3786174549 conn(0x5599cef128c0 msgr2=0x5599cef21120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb06bfff640 0 client.16527 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 shutdown_connections 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.110+0000 7fb095f96740 1 -- 172.21.15.64:0/3786174549 wait complete. 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.111+0000 7fb095f96740 20 client.16527 trim_cache size 0 max 16384 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:12:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:12:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.112+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0780a2d30 msgr2=0x7fb0780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.112+0000 7fb0931a9640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0780a2d30 0x7fb0780a51f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb084009a90 tx=0x7fb084000bc0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.112+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 msgr2=0x7fb08c19e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:12:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.112+0000 7fb0931a9640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 0x7fb08c19e080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0740314e0 tx=0x7fb074003c10 comp rx=0 tx=0).stop 2024-05-09T09:12:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 shutdown_connections 2024-05-09T09:12:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0780a2d30 0x7fb0780a51f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08c105420 0x7fb08c1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb08c102a70 0x7fb08c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 --2- 172.21.15.64:0/264542403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb08c100100 0x7fb08c19e080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:12:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 >> 172.21.15.64:0/264542403 conn(0x7fb08c0fbd00 msgr2=0x7fb08c1034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:12:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 shutdown_connections 2024-05-09T09:12:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:12:58.114+0000 7fb0931a9640 1 -- 172.21.15.64:0/264542403 wait complete. 2024-05-09T09:12:58.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:12:58 smithi179 ceph-mon[34086]: pgmap v485: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 981 op/s 2024-05-09T09:12:58.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:12:58 smithi139 ceph-mon[33599]: pgmap v485: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 981 op/s 2024-05-09T09:12:58.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:12:58 smithi064 ceph-mon[30149]: pgmap v485: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 981 op/s 2024-05-09T09:13:00.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:00 smithi179 ceph-mon[34086]: pgmap v486: 129 pgs: 129 active+clean; 1013 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-05-09T09:13:00.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:00 smithi139 ceph-mon[33599]: pgmap v486: 129 pgs: 129 active+clean; 1013 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-05-09T09:13:00.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:00 smithi064 ceph-mon[30149]: pgmap v486: 129 pgs: 129 active+clean; 1013 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-05-09T09:13:01.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 160.0 s, 1104.4 tps, lat 90.472 ms stddev 39.113 2024-05-09T09:13:01.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:13:01.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:13:01.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.589+0000 7f3d82ffd640 1 -- 172.21.15.64:0/607582185 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74031880 con 0x7f3d84109900 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 -- 172.21.15.64:0/607582185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 msgr2=0x7f3d84110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 --2- 172.21.15.64:0/607582185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d84110190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009900 tx=0x7f3d7402a3f0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 -- 172.21.15.64:0/607582185 shutdown_connections 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 --2- 172.21.15.64:0/607582185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d84110190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 --2- 172.21.15.64:0/607582185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d84108f40 0x7f3d841093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 --2- 172.21.15.64:0/607582185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d84107d50 0x7f3d84108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 -- 172.21.15.64:0/607582185 >> 172.21.15.64:0/607582185 conn(0x7f3d84076cf0 msgr2=0x7f3d84079130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 -- 172.21.15.64:0/607582185 shutdown_connections 2024-05-09T09:13:01.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.590+0000 7f3d8910e640 1 -- 172.21.15.64:0/607582185 wait complete. 2024-05-09T09:13:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.591+0000 7f3d8910e640 1 Processor -- start 2024-05-09T09:13:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.591+0000 7f3d8910e640 1 -- start start 2024-05-09T09:13:01.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8910e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d84107d50 0x7f3d84070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8910e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d84108f40 0x7f3d84070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8910e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d841adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8910e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841138b0 con 0x7f3d84107d50 2024-05-09T09:13:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8910e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84113730 con 0x7f3d84109900 2024-05-09T09:13:01.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8910e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84113a30 con 0x7f3d84108f40 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d84107d50 0x7f3d84070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d841adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d841adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41706/0 (socket says 172.21.15.64:41706) 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 -- 172.21.15.64:0/877020275 learned_addr learned my addr 172.21.15.64:0/877020275 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d837fe640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d84108f40 0x7f3d84070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 -- 172.21.15.64:0/877020275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d84108f40 msgr2=0x7f3d84070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d84108f40 0x7f3d84070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 -- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d84107d50 msgr2=0x7f3d84070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d84107d50 0x7f3d84070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.592+0000 7f3d8890d640 1 -- 172.21.15.64:0/877020275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d840737f0 con 0x7f3d84109900 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.593+0000 7f3d8890d640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d841adf80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009850 tx=0x7f3d74005160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.593+0000 7f3d817fa640 1 -- 172.21.15.64:0/877020275 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7403f070 con 0x7f3d84109900 2024-05-09T09:13:01.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.593+0000 7f3d817fa640 1 -- 172.21.15.64:0/877020275 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d74003aa0 con 0x7f3d84109900 2024-05-09T09:13:01.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.593+0000 7f3d817fa640 1 -- 172.21.15.64:0/877020275 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74035030 con 0x7f3d84109900 2024-05-09T09:13:01.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.593+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d841ae520 con 0x7f3d84109900 2024-05-09T09:13:01.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.593+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841ae990 con 0x7f3d84109900 2024-05-09T09:13:01.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.597+0000 7f3d8d083740 1 Processor -- start 2024-05-09T09:13:01.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.597+0000 7f3d817fa640 1 -- 172.21.15.64:0/877020275 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d74042070 con 0x7f3d84109900 2024-05-09T09:13:01.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d817fa640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3d6c0a2c60 0x7f3d6c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d817fa640 1 -- 172.21.15.64:0/877020275 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3d740efca0 con 0x7f3d84109900 2024-05-09T09:13:01.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 -- start start 2024-05-09T09:13:01.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d837fe640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3d6c0a2c60 0x7f3d6c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d9c40 0x5622cc2dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5622cc238d00 con 0x5622cc2d40e0 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5622cc1e04c0 con 0x5622cc2d9c40 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.599+0000 7f3d8d083740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5622cc234cc0 con 0x5622cc2d4ad0 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2d4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33118/0 (socket says 172.21.15.64:33118) 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d83fff640 1 -- 172.21.15.64:0/2898005603 learned_addr learned my addr 172.21.15.64:0/2898005603 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d837fe640 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d9c40 0x5622cc2dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d8890d640 1 -- 172.21.15.64:0/2898005603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 msgr2=0x5622cc2d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d8890d640 1 -- 172.21.15.64:0/2898005603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 msgr2=0x5622cc2d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.600+0000 7f3d8890d640 1 -- 172.21.15.64:0/2898005603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cc0167d0 con 0x5622cc2d9c40 2024-05-09T09:13:01.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.601+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d9c40 0x5622cc2dc0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d740ed410 tx=0x7f3d74009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f07fd4e0ef0d2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.601+0000 7f3d6a7fc640 1 -- 172.21.15.64:0/2898005603 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74049040 con 0x5622cc2d9c40 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.601+0000 7f3d6a7fc640 1 -- 172.21.15.64:0/2898005603 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d740bc870 con 0x5622cc2d9c40 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.601+0000 7f3d6a7fc640 1 -- 172.21.15.64:0/2898005603 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740b87e0 con 0x5622cc2d9c40 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.602+0000 7f3d8d083740 1 -- 172.21.15.64:0/2898005603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d9c40 msgr2=0x5622cc2dc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.602+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d9c40 0x5622cc2dc0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d740ed410 tx=0x7f3d74009260 comp rx=0 tx=0).stop 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.602+0000 7f3d8d083740 1 -- 172.21.15.64:0/2898005603 shutdown_connections 2024-05-09T09:13:01.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d9c40 0x5622cc2dc0d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2898005603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d8d083740 1 -- 172.21.15.64:0/2898005603 >> 172.21.15.64:0/2898005603 conn(0x5622cc2cf750 msgr2=0x5622cc2cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d837fe640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3d6c0a2c60 0x7f3d6c0a5120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d84071640 tx=0x7f3d7800a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d8d083740 1 -- 172.21.15.64:0/2898005603 shutdown_connections 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.603+0000 7f3d8d083740 1 -- 172.21.15.64:0/2898005603 wait complete. 2024-05-09T09:13:01.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 -- start start 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 Processor -- start 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d40e0 0x5622cc2de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 0x5622cc2e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5622cc2dc9b0 con 0x5622cc2d4ad0 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5622cc1f3b90 con 0x5622cc2d40e0 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5622cc2dcb30 con 0x5622cc2d9c40 2024-05-09T09:13:01.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.604+0000 7f3d8d083740 10 client.0 _ll_register_callbacks cb 0x5622cc22fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 0x5622cc2e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d40e0 0x5622cc2de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d40e0 0x5622cc2de2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41730/0 (socket says 172.21.15.64:41730) 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 0x5622cc2e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33268/0 (socket says 172.21.15.64:33268) 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d83fff640 1 -- 172.21.15.64:0/2289578370 learned_addr learned my addr 172.21.15.64:0/2289578370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d837fe640 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 -- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d40e0 msgr2=0x5622cc2de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d40e0 0x5622cc2de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 -- 172.21.15.64:0/2289578370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 msgr2=0x5622cc2de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.605+0000 7f3d8890d640 1 -- 172.21.15.64:0/2289578370 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cbf4c3f0 con 0x5622cc2d9c40 2024-05-09T09:13:01.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.606+0000 7f3d8890d640 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 0x5622cc2e5860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d740bca20 tx=0x7f3d740b8a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24252843f6591808 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.606+0000 7f3d637fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.607+0000 7f3d637fe640 1 -- 172.21.15.64:0/2289578370 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740d4020 con 0x5622cc2d9c40 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.607+0000 7f3d637fe640 1 -- 172.21.15.64:0/2289578370 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3d740dc0c0 con 0x5622cc2d9c40 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.607+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d740bc870 con 0x5622cc2d9c40 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.607+0000 7f3d637fe640 1 -- 172.21.15.64:0/2289578370 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3d7409f050 con 0x5622cc2d9c40 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.608+0000 7f3d8d083740 10 client.26194 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.608+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5622cbf0c0f0 con 0x5622cc2d9c40 2024-05-09T09:13:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.608+0000 7f3d637fe640 1 -- 172.21.15.64:0/2289578370 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d740bc870 con 0x5622cc2d9c40 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d637fe640 1 -- 172.21.15.64:0/2289578370 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f3d740b6ce0 con 0x5622cc2d9c40 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d8d083740 10 client.26194 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d8d083740 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d8d083740 20 client.26194 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2ea1c0 0x5622cc2ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d8d083740 4 client.26194 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622cc246430 con 0x5622cc2ea1c0 2024-05-09T09:13:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.609+0000 7f3d83fff640 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2ea1c0 0x5622cc2ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.611+0000 7f3d83fff640 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2ea1c0 0x5622cc2ec620 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376b3727e23825b5 server_cookie=bbf6c8831df40e82 in_seq=0 out_seq=0 2024-05-09T09:13:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.611+0000 7f3d637fe640 10 client.26194 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.612+0000 7f3d637fe640 1 -- 172.21.15.64:0/2289578370 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622cc246430 con 0x5622cc2ea1c0 2024-05-09T09:13:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.612+0000 7f3d637fe640 10 client.26194 handle_command_reply: tid=0 2024-05-09T09:13:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.612+0000 7f3d8d083740 1 client.26194 shutdown 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 msgr2=0x5622cc2e5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 0x5622cc2e5860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d740bca20 tx=0x7f3d740b8a10 comp rx=0 tx=0).stop 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 shutdown_connections 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2ea1c0 0x5622cc2ec620 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d9c40 0x5622cc2e5860 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 --2- 172.21.15.64:0/2289578370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2d40e0 0x5622cc2de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.613+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 >> 172.21.15.64:0/2289578370 conn(0x5622cc2cf750 msgr2=0x5622cc2cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.614+0000 7f3d637fe640 0 client.26194 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.614+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 shutdown_connections 2024-05-09T09:13:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.614+0000 7f3d8d083740 1 -- 172.21.15.64:0/2289578370 wait complete. 2024-05-09T09:13:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.614+0000 7f3d8d083740 20 client.26194 trim_cache size 0 max 16384 2024-05-09T09:13:01.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.655+0000 7f3d8d083740 1 Processor -- start 2024-05-09T09:13:01.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.657+0000 7f3d8d083740 1 -- start start 2024-05-09T09:13:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41736/0 (socket says 172.21.15.64:41736) 2024-05-09T09:13:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d83fff640 1 -- 172.21.15.64:0/1494401772 learned_addr learned my addr 172.21.15.64:0/1494401772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d8890d640 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.658+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841bb910 con 0x5622cc2d40e0 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b9180 con 0x5622cc2cf750 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b6fd0 con 0x5622cc2d4ad0 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d837fe640 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d83fff640 1 -- 172.21.15.64:0/1494401772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 msgr2=0x5622cc2d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d83fff640 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d83fff640 1 -- 172.21.15.64:0/1494401772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 msgr2=0x5622cc2e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d83fff640 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d83fff640 1 -- 172.21.15.64:0/1494401772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cc2cf580 con 0x5622cc2cf750 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d837fe640 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.659+0000 7f3d83fff640 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2d2630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005fd0 tx=0x7f3d7c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd46dd55f9db711 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.660+0000 7f3d69ffb640 1 -- 172.21.15.64:0/1494401772 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c026070 con 0x5622cc2cf750 2024-05-09T09:13:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.660+0000 7f3d69ffb640 1 -- 172.21.15.64:0/1494401772 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d7c004d20 con 0x5622cc2cf750 2024-05-09T09:13:01.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.660+0000 7f3d69ffb640 1 -- 172.21.15.64:0/1494401772 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c005610 con 0x5622cc2cf750 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.661+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 msgr2=0x5622cc2d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.661+0000 7f3d8d083740 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2d2630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005fd0 tx=0x7f3d7c009500 comp rx=0 tx=0).stop 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.661+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 shutdown_connections 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.661+0000 7f3d8d083740 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d4ad0 0x5622cc2d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.661+0000 7f3d8d083740 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d40e0 0x5622cc2e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.662+0000 7f3d8d083740 1 --2- 172.21.15.64:0/1494401772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2d2630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.662+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 >> 172.21.15.64:0/1494401772 conn(0x5622cc2dd190 msgr2=0x5622cc2de2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.662+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 shutdown_connections 2024-05-09T09:13:01.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.662+0000 7f3d8d083740 1 -- 172.21.15.64:0/1494401772 wait complete. 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.662+0000 7f3d8d083740 1 -- start start 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.663+0000 7f3d8d083740 1 Processor -- start 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.663+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d40e0 0x5622cc2e3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2e33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41752/0 (socket says 172.21.15.64:41752) 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8d083740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 -- 172.21.15.64:0/679672611 learned_addr learned my addr 172.21.15.64:0/679672611 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b7640 con 0x5622cc2d4ad0 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b8ab0 con 0x5622cc2cf750 2024-05-09T09:13:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841b7d10 con 0x5622cc2d40e0 2024-05-09T09:13:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8890d640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d837fe640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d40e0 0x5622cc2e3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d8d083740 10 client.0 _ll_register_callbacks cb 0x7f3d7c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 -- 172.21.15.64:0/679672611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d40e0 msgr2=0x5622cc2e3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d40e0 0x5622cc2e3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 -- 172.21.15.64:0/679672611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 msgr2=0x5622cc2e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.664+0000 7f3d83fff640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.665+0000 7f3d83fff640 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cc2e4e10 con 0x5622cc2cf750 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.665+0000 7f3d83fff640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2e33e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009fa0 tx=0x7f3d7c009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100d40946be0fa8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.665+0000 7f3d637fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.665+0000 7f3d637fe640 1 -- 172.21.15.64:0/679672611 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c02f020 con 0x5622cc2cf750 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.665+0000 7f3d637fe640 1 -- 172.21.15.64:0/679672611 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3d7c005390 con 0x5622cc2cf750 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.666+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c004d20 con 0x5622cc2cf750 2024-05-09T09:13:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.666+0000 7f3d637fe640 1 -- 172.21.15.64:0/679672611 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c026040 con 0x5622cc2cf750 2024-05-09T09:13:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.666+0000 7f3d8d083740 10 client.26060 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.666+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5622cc350af0 con 0x5622cc2cf750 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.666+0000 7f3d637fe640 1 -- 172.21.15.64:0/679672611 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d7c004d20 con 0x5622cc2cf750 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d637fe640 1 -- 172.21.15.64:0/679672611 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f3d7c02b410 con 0x5622cc2cf750 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d8d083740 10 client.26060 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d8d083740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d8d083740 20 client.26060 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d8d083740 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2d9c40 0x5622cc353360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d8d083740 4 client.26060 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622cc240780 con 0x5622cc2d9c40 2024-05-09T09:13:01.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.667+0000 7f3d837fe640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2d9c40 0x5622cc353360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d837fe640 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2d9c40 0x5622cc353360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21cd3455c72ed1 server_cookie=59ce53e48a939044 in_seq=0 out_seq=0 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d637fe640 10 client.26060 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d637fe640 1 -- 172.21.15.64:0/679672611 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622cc240780 con 0x5622cc2d9c40 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d637fe640 10 client.26060 handle_command_reply: tid=0 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d8d083740 1 client.26060 shutdown 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 msgr2=0x5622cc2e33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.669+0000 7f3d8d083740 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2e33e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009fa0 tx=0x7f3d7c009d80 comp rx=0 tx=0).stop 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 shutdown_connections 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5622cc2d9c40 0x5622cc353360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622cc2d4ad0 0x5622cc2e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622cc2d40e0 0x5622cc2e3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 --2- 172.21.15.64:0/679672611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5622cc2cf750 0x5622cc2e33e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 >> 172.21.15.64:0/679672611 conn(0x5622cc2dd190 msgr2=0x5622cc2ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d637fe640 0 client.26060 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 shutdown_connections 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 1 -- 172.21.15.64:0/679672611 wait complete. 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.670+0000 7f3d8d083740 20 client.26060 trim_cache size 0 max 16384 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:13:01.671 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.671+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3d6c0a2c60 msgr2=0x7f3d6c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.671+0000 7f3d8910e640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3d6c0a2c60 0x7f3d6c0a5120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d84071640 tx=0x7f3d7800a8c0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.671+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 msgr2=0x7f3d841adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.671+0000 7f3d8910e640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d841adf80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009850 tx=0x7f3d74005160 comp rx=0 tx=0).stop 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.672+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 shutdown_connections 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.672+0000 7f3d8910e640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3d6c0a2c60 0x7f3d6c0a5120 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.672+0000 7f3d8910e640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d84109900 0x7f3d841adf80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.672+0000 7f3d8910e640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d84108f40 0x7f3d84070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.672+0000 7f3d8910e640 1 --2- 172.21.15.64:0/877020275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d84107d50 0x7f3d84070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.672+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 >> 172.21.15.64:0/877020275 conn(0x7f3d84076cf0 msgr2=0x7f3d8410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.673+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 shutdown_connections 2024-05-09T09:13:01.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.673+0000 7f3d8910e640 1 -- 172.21.15.64:0/877020275 wait complete. 2024-05-09T09:13:01.696 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:13:01.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:13:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 -- 172.21.15.64:0/3324519922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80077430 msgr2=0x7f3e80077890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 --2- 172.21.15.64:0/3324519922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80077430 0x7f3e80077890 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009900 tx=0x7f3e6802a410 comp rx=0 tx=0).stop 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 -- 172.21.15.64:0/3324519922 shutdown_connections 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 --2- 172.21.15.64:0/3324519922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e80077dd0 0x7f3e80078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 --2- 172.21.15.64:0/3324519922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80077430 0x7f3e80077890 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 --2- 172.21.15.64:0/3324519922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e800761f0 0x7f3e80076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 -- 172.21.15.64:0/3324519922 >> 172.21.15.64:0/3324519922 conn(0x7f3e800ff020 msgr2=0x7f3e80101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 -- 172.21.15.64:0/3324519922 shutdown_connections 2024-05-09T09:13:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.879+0000 7f3e856a5640 1 -- 172.21.15.64:0/3324519922 wait complete. 2024-05-09T09:13:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.880+0000 7f3e856a5640 1 Processor -- start 2024-05-09T09:13:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 -- start start 2024-05-09T09:13:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e800761f0 0x7f3e8019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e80077430 0x7f3e8019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e80077dd0 0x7f3e801a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801139f0 con 0x7f3e800761f0 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80113870 con 0x7f3e80077430 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e856a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80113b70 con 0x7f3e80077dd0 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e7effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e800761f0 0x7f3e8019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e7effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e800761f0 0x7f3e8019e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33168/0 (socket says 172.21.15.64:33168) 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e7effd640 1 -- 172.21.15.64:0/1111063978 learned_addr learned my addr 172.21.15.64:0/1111063978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e7e7fc640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e80077430 0x7f3e8019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.881+0000 7f3e7f7fe640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e80077dd0 0x7f3e801a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.882+0000 7f3e7e7fc640 1 -- 172.21.15.64:0/1111063978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e80077dd0 msgr2=0x7f3e801a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.882+0000 7f3e7e7fc640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e80077dd0 0x7f3e801a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.882+0000 7f3e7e7fc640 1 -- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e800761f0 msgr2=0x7f3e8019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.882+0000 7f3e7e7fc640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e800761f0 0x7f3e8019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.882+0000 7f3e7e7fc640 1 -- 172.21.15.64:0/1111063978 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e801a5ce0 con 0x7f3e80077430 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.882+0000 7f3e7e7fc640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e80077430 0x7f3e8019e600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009850 tx=0x7f3e68003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.883+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6803f070 con 0x7f3e80077430 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.883+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e68031c30 con 0x7f3e80077430 2024-05-09T09:13:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.883+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68035030 con 0x7f3e80077430 2024-05-09T09:13:01.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.883+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e801a5f70 con 0x7f3e80077430 2024-05-09T09:13:01.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.883+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e801a6380 con 0x7f3e80077430 2024-05-09T09:13:01.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.884+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e80110620 con 0x7f3e80077430 2024-05-09T09:13:01.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.887+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e68042070 con 0x7f3e80077430 2024-05-09T09:13:01.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.888+0000 7f3e6ffff640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e740a2d10 0x7f3e740a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:01.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.889+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e680eff70 con 0x7f3e80077430 2024-05-09T09:13:01.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.889+0000 7f3e7effd640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e740a2d10 0x7f3e740a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:01.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.890+0000 7f3e7effd640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e740a2d10 0x7f3e740a51d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009c90 tx=0x7f3e70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:01.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:01.892+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e680bc510 con 0x7f3e80077430 2024-05-09T09:13:02.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.338+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e8002d2e0 con 0x7f3e80077430 2024-05-09T09:13:02.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.339+0000 7f3e6ffff640 1 -- 172.21.15.64:0/1111063978 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11883 (secure 0 0 0) 0x7f3e680c0810 con 0x7f3e80077430 2024-05-09T09:13:02.339 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:13:02.339 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:12:56.241385+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:13:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-05-09T09:13:02.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.343+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e740a2d10 msgr2=0x7f3e740a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.343+0000 7f3e856a5640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e740a2d10 0x7f3e740a51d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009c90 tx=0x7f3e70005f90 comp rx=0 tx=0).stop 2024-05-09T09:13:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e80077430 msgr2=0x7f3e8019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e80077430 0x7f3e8019e600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009850 tx=0x7f3e68003b90 comp rx=0 tx=0).stop 2024-05-09T09:13:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 shutdown_connections 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e740a2d10 0x7f3e740a51d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e80077dd0 0x7f3e801a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e80077430 0x7f3e8019e600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 --2- 172.21.15.64:0/1111063978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e800761f0 0x7f3e8019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.344+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 >> 172.21.15.64:0/1111063978 conn(0x7f3e800ff020 msgr2=0x7f3e80100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.345+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 shutdown_connections 2024-05-09T09:13:02.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.345+0000 7f3e856a5640 1 -- 172.21.15.64:0/1111063978 wait complete. 2024-05-09T09:13:02.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:13:02.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.553+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2012725673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a5d70 msgr2=0x7fbe380a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.553+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2012725673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a5d70 0x7fbe380a61d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009900 tx=0x7fbe2c02a420 comp rx=0 tx=0).stop 2024-05-09T09:13:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.557+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2012725673 shutdown_connections 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.557+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2012725673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a6710 0x7fbe380aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.557+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2012725673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a5d70 0x7fbe380a61d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.558+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2012725673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe380a4c40 0x7fbe380a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.558+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2012725673 >> 172.21.15.64:0/2012725673 conn(0x7fbe380a00d0 msgr2=0x7fbe380a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.558+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2012725673 shutdown_connections 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.559+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2012725673 wait complete. 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 Processor -- start 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 -- start start 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a4c40 0x7fbe38143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 0x7fbe38143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe380a6710 0x7fbe3814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380b9010 con 0x7fbe380a5d70 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380b8e90 con 0x7fbe380a4c40 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.560+0000 7fbe45e8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380b9190 con 0x7fbe380a6710 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.561+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 0x7fbe38143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.561+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 0x7fbe38143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33186/0 (socket says 172.21.15.64:33186) 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.561+0000 7fbe3ffff640 1 -- 172.21.15.64:0/2177586545 learned_addr learned my addr 172.21.15.64:0/2177586545 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.561+0000 7fbe4568b640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe380a6710 0x7fbe3814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a4c40 0x7fbe38143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe3ffff640 1 -- 172.21.15.64:0/2177586545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe380a6710 msgr2=0x7fbe3814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe380a6710 0x7fbe3814ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe3ffff640 1 -- 172.21.15.64:0/2177586545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a4c40 msgr2=0x7fbe38143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a4c40 0x7fbe38143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe3ffff640 1 -- 172.21.15.64:0/2177586545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3814b2e0 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a4c40 0x7fbe38143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.562+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 0x7fbe38143c50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0314e0 tx=0x7fbe2c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.563+0000 7fbe3dffb640 1 -- 172.21.15.64:0/2177586545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c03f070 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.563+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3814b570 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.563+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3814bae0 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.563+0000 7fbe3dffb640 1 -- 172.21.15.64:0/2177586545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe2c031e40 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.563+0000 7fbe3dffb640 1 -- 172.21.15.64:0/2177586545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c035030 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.567+0000 7fbe3dffb640 1 -- 172.21.15.64:0/2177586545 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe2c042070 con 0x7fbe380a5d70 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.570+0000 7fbe49df9740 1 Processor -- start 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.570+0000 7fbe49df9740 1 -- start start 2024-05-09T09:13:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 0x55ad9b3ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe49df9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad9b353c40 con 0x55ad9b3f4c90 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe49df9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad9b2fb580 con 0x55ad9b3efb20 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe49df9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad9b34fc00 con 0x55ad9b3ef130 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe3dffb640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe300a2d90 0x7fbe300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe4568b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 0x55ad9b3ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe4568b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 0x55ad9b3ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33322/0 (socket says 172.21.15.64:33322) 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe4568b640 1 -- 172.21.15.64:0/1664980968 learned_addr learned my addr 172.21.15.64:0/1664980968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe3dffb640 1 -- 172.21.15.64:0/2177586545 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe2c0efd00 con 0x7fbe380a5d70 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.571+0000 7fbe4568b640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe300a2d90 0x7fbe300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.572+0000 7fbe4568b640 1 -- 172.21.15.64:0/1664980968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3efb20 msgr2=0x55ad9b3f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.572+0000 7fbe4568b640 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.572+0000 7fbe4568b640 1 -- 172.21.15.64:0/1664980968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 msgr2=0x55ad9b3f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.572+0000 7fbe4568b640 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.572+0000 7fbe4568b640 1 -- 172.21.15.64:0/1664980968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9b0673f0 con 0x55ad9b3ef130 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.573+0000 7fbe4568b640 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 0x55ad9b3ef550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe400503a0 tx=0x7fbe40075830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed06096e1a95c90f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.573+0000 7fbe2affd640 1 -- 172.21.15.64:0/1664980968 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40086070 con 0x55ad9b3ef130 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.573+0000 7fbe4568b640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe300a2d90 0x7fbe300a5250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbe4004f8c0 tx=0x7fbe4007e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.574+0000 7fbe2affd640 1 -- 172.21.15.64:0/1664980968 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe400818b0 con 0x55ad9b3ef130 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe2affd640 1 -- 172.21.15.64:0/1664980968 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40072610 con 0x55ad9b3ef130 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 -- 172.21.15.64:0/1664980968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 msgr2=0x55ad9b3ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 0x55ad9b3ef550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe400503a0 tx=0x7fbe40075830 comp rx=0 tx=0).stop 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 -- 172.21.15.64:0/1664980968 shutdown_connections 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1664980968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3ef130 0x55ad9b3ef550 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 -- 172.21.15.64:0/1664980968 >> 172.21.15.64:0/1664980968 conn(0x55ad9b3ea7a0 msgr2=0x55ad9b3eaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 -- 172.21.15.64:0/1664980968 shutdown_connections 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.575+0000 7fbe49df9740 1 -- 172.21.15.64:0/1664980968 wait complete. 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.576+0000 7fbe49df9740 1 -- start start 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.576+0000 7fbe49df9740 1 Processor -- start 2024-05-09T09:13:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 0x55ad9b3f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b400b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad9b3f7be0 con 0x55ad9b3ef130 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad9b30ec40 con 0x55ad9b3f4c90 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad9b3f7d60 con 0x55ad9b3efb20 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe49df9740 10 client.0 _ll_register_callbacks cb 0x55ad9b066be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe44e8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 0x55ad9b3f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe44e8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 0x55ad9b3f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33202/0 (socket says 172.21.15.64:33202) 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe44e8a640 1 -- 172.21.15.64:0/1041531841 learned_addr learned my addr 172.21.15.64:0/1041531841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.577+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.578+0000 7fbe4568b640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b400b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.578+0000 7fbe44e8a640 1 -- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 msgr2=0x55ad9b3f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.578+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.578+0000 7fbe44e8a640 1 -- 172.21.15.64:0/1041531841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 msgr2=0x55ad9b400b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.579+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b400b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.579+0000 7fbe44e8a640 1 -- 172.21.15.64:0/1041531841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9b1317d0 con 0x55ad9b3ef130 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.579+0000 7fbe4568b640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b400b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.579+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.579+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 0x55ad9b3f9630 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400e2c0 tx=0x7fbe3400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b39db2efac808 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.579+0000 7fbe1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.580+0000 7fbe1ffff640 1 -- 172.21.15.64:0/1041531841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34018020 con 0x55ad9b3ef130 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.580+0000 7fbe1ffff640 1 -- 172.21.15.64:0/1041531841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe34013660 con 0x55ad9b3ef130 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.580+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe400818b0 con 0x55ad9b3ef130 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.581+0000 7fbe1ffff640 1 -- 172.21.15.64:0/1041531841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbe34023050 con 0x55ad9b3ef130 2024-05-09T09:13:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.581+0000 7fbe49df9740 10 client.16554 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.581+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad9b403f80 con 0x55ad9b3ef130 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe1ffff640 1 -- 172.21.15.64:0/1041531841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe400818b0 con 0x55ad9b3ef130 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe1ffff640 1 -- 172.21.15.64:0/1041531841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fbe34014960 con 0x55ad9b3ef130 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe49df9740 10 client.16554 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe49df9740 10 client.16554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe49df9740 20 client.16554 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b405af0 0x55ad9b407f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe49df9740 4 client.16554 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad9b361370 con 0x55ad9b405af0 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.582+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b405af0 0x55ad9b407f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.583+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b405af0 0x55ad9b407f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2936a3fd5fb6cd5a server_cookie=fd78758e68fdbcd3 in_seq=0 out_seq=0 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.583+0000 7fbe1ffff640 10 client.16554 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe1ffff640 1 -- 172.21.15.64:0/1041531841 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad9b361370 con 0x55ad9b405af0 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe1ffff640 10 client.16554 handle_command_reply: tid=0 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 client.16554 shutdown 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 msgr2=0x55ad9b3f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 0x55ad9b3f9630 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400e2c0 tx=0x7fbe3400e780 comp rx=0 tx=0).stop 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 shutdown_connections 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b405af0 0x55ad9b407f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b400b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 --2- 172.21.15.64:0/1041531841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3ef130 0x55ad9b3f9630 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 >> 172.21.15.64:0/1041531841 conn(0x55ad9b3ea7a0 msgr2=0x55ad9b3e99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe1ffff640 0 client.16554 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 shutdown_connections 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.584+0000 7fbe49df9740 1 -- 172.21.15.64:0/1041531841 wait complete. 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.585+0000 7fbe49df9740 20 client.16554 trim_cache size 0 max 16384 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.609+0000 7fbe49df9740 1 Processor -- start 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.609+0000 7fbe49df9740 1 -- start start 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 0x55ad9b3e9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe49df9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240125c0 con 0x55ad9b3efb20 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe49df9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2400fe30 con 0x55ad9b3f4c90 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe49df9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2400dc80 con 0x55ad9b400a60 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 0x55ad9b3e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 0x55ad9b3e9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33350/0 (socket says 172.21.15.64:33350) 2024-05-09T09:13:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 -- 172.21.15.64:0/265068591 learned_addr learned my addr 172.21.15.64:0/265068591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 -- 172.21.15.64:0/265068591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 msgr2=0x55ad9b3f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 -- 172.21.15.64:0/265068591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3efb20 msgr2=0x55ad9b3f0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe4568b640 1 -- 172.21.15.64:0/265068591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9b3e4da0 con 0x55ad9b400a60 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.610+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.611+0000 7fbe4568b640 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 0x55ad9b3e9130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe4002d750 tx=0x7fbe40089000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce597cd770dc1d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.611+0000 7fbe2a7fc640 1 -- 172.21.15.64:0/265068591 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40086070 con 0x55ad9b400a60 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.612+0000 7fbe2a7fc640 1 -- 172.21.15.64:0/265068591 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe40081710 con 0x55ad9b400a60 2024-05-09T09:13:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.612+0000 7fbe2a7fc640 1 -- 172.21.15.64:0/265068591 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400766b0 con 0x55ad9b400a60 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 -- 172.21.15.64:0/265068591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 msgr2=0x55ad9b3e9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 0x55ad9b3e9130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe4002d750 tx=0x7fbe40089000 comp rx=0 tx=0).stop 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 -- 172.21.15.64:0/265068591 shutdown_connections 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b400a60 0x55ad9b3e9130 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 --2- 172.21.15.64:0/265068591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3efb20 0x55ad9b3f0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 -- 172.21.15.64:0/265068591 >> 172.21.15.64:0/265068591 conn(0x55ad9b3ea7a0 msgr2=0x55ad9b3f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 -- 172.21.15.64:0/265068591 shutdown_connections 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 -- 172.21.15.64:0/265068591 wait complete. 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.613+0000 7fbe49df9740 1 -- start start 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 Processor -- start 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b400a60 0x55ad9b46a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2400e2f0 con 0x55ad9b3f4c90 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2400f760 con 0x55ad9b400a60 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2400e9c0 con 0x55ad9b3efb20 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.614+0000 7fbe49df9740 10 client.0 _ll_register_callbacks cb 0x7fbe3400c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3fd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33354/0 (socket says 172.21.15.64:33354) 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 -- 172.21.15.64:0/3281062909 learned_addr learned my addr 172.21.15.64:0/3281062909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe4568b640 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b400a60 0x55ad9b46a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 -- 172.21.15.64:0/3281062909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b400a60 msgr2=0x55ad9b46a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b400a60 0x55ad9b46a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 -- 172.21.15.64:0/3281062909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 msgr2=0x55ad9b3fd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 -- 172.21.15.64:0/3281062909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9b400650 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.615+0000 7fbe44e8a640 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3fd040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbe34014d80 tx=0x7fbe3400c2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5f01aa515326ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe1ffff640 1 -- 172.21.15.64:0/3281062909 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34018040 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe1ffff640 1 -- 172.21.15.64:0/3281062909 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbe34005b90 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe40081710 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe1ffff640 1 -- 172.21.15.64:0/3281062909 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbe34005d90 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe49df9740 10 client.26221 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad9b46b580 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.616+0000 7fbe1ffff640 1 -- 172.21.15.64:0/3281062909 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe40081710 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe1ffff640 1 -- 172.21.15.64:0/3281062909 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fbe34014960 con 0x55ad9b3efb20 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe49df9740 10 client.26221 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:02.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe49df9740 10 client.26221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe49df9740 20 client.26221 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe49df9740 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b46bfb0 0x55ad9b46e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe49df9740 4 client.26221 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad9b35b6c0 con 0x55ad9b46bfb0 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.618+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b46bfb0 0x55ad9b46e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.620+0000 7fbe3ffff640 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b46bfb0 0x55ad9b46e3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=715400b13ea4c44d server_cookie=566d2b2158a1fb6f in_seq=0 out_seq=0 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.620+0000 7fbe1ffff640 10 client.26221 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.620+0000 7fbe1ffff640 1 -- 172.21.15.64:0/3281062909 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad9b35b6c0 con 0x55ad9b46bfb0 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.620+0000 7fbe1ffff640 10 client.26221 handle_command_reply: tid=0 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.620+0000 7fbe49df9740 1 client.26221 shutdown 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.621+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 msgr2=0x55ad9b3fd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.621+0000 7fbe49df9740 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3fd040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbe34014d80 tx=0x7fbe3400c2f0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 shutdown_connections 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe49df9740 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55ad9b46bfb0 0x55ad9b46e3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe49df9740 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad9b400a60 0x55ad9b46a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe49df9740 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad9b3f4c90 0x55ad9b3fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe49df9740 1 --2- 172.21.15.64:0/3281062909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad9b3efb20 0x55ad9b3fd040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 >> 172.21.15.64:0/3281062909 conn(0x55ad9b3ea7a0 msgr2=0x55ad9b3f0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.622+0000 7fbe1ffff640 0 client.26221 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.623+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 shutdown_connections 2024-05-09T09:13:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.624+0000 7fbe49df9740 1 -- 172.21.15.64:0/3281062909 wait complete. 2024-05-09T09:13:02.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.624+0000 7fbe49df9740 20 client.26221 trim_cache size 0 max 16384 2024-05-09T09:13:02.714 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe300a2d90 msgr2=0x7fbe300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe300a2d90 0x7fbe300a5250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbe4004f8c0 tx=0x7fbe4007e000 comp rx=0 tx=0).stop 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 msgr2=0x7fbe38143c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 0x7fbe38143c50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0314e0 tx=0x7fbe2c003d40 comp rx=0 tx=0).stop 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 shutdown_connections 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbe300a2d90 0x7fbe300a5250 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe380a6710 0x7fbe3814ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe380a5d70 0x7fbe38143c50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 --2- 172.21.15.64:0/2177586545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe380a4c40 0x7fbe38143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 >> 172.21.15.64:0/2177586545 conn(0x7fbe380a00d0 msgr2=0x7fbe380a6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 shutdown_connections 2024-05-09T09:13:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.628+0000 7fbe45e8c640 1 -- 172.21.15.64:0/2177586545 wait complete. 2024-05-09T09:13:02.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:02 smithi064 ceph-mon[30149]: pgmap v487: 129 pgs: 129 active+clean; 1013 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:13:02.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:02 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1111063978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:13:02.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:13:02.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:02 smithi179 ceph-mon[34086]: pgmap v487: 129 pgs: 129 active+clean; 1013 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:13:02.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:02 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1111063978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:13:02.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:02 smithi139 ceph-mon[33599]: pgmap v487: 129 pgs: 129 active+clean; 1013 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-05-09T09:13:02.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:02 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1111063978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:13:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 -- 172.21.15.64:0/583028078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009c370 msgr2=0x7ff89009c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/583028078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009c370 0x7ff89009c790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff894009900 tx=0x7ff89402a400 comp rx=0 tx=0).stop 2024-05-09T09:13:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 -- 172.21.15.64:0/583028078 shutdown_connections 2024-05-09T09:13:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/583028078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff89009df20 0x7ff8900a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/583028078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff89009d560 0x7ff89009d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/583028078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009c370 0x7ff89009c790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.908+0000 7ff89ef2e640 1 -- 172.21.15.64:0/583028078 >> 172.21.15.64:0/583028078 conn(0x7ff89000b310 msgr2=0x7ff89000b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.909+0000 7ff89ef2e640 1 -- 172.21.15.64:0/583028078 shutdown_connections 2024-05-09T09:13:02.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.909+0000 7ff89ef2e640 1 -- 172.21.15.64:0/583028078 wait complete. 2024-05-09T09:13:02.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.910+0000 7ff89ef2e640 1 Processor -- start 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.910+0000 7ff89ef2e640 1 -- start start 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89ef2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff89009c370 0x7ff890143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89ef2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 0x7ff890143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89ef2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff89009df20 0x7ff89014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89ef2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8900a7f60 con 0x7ff89009c370 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89ef2e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff8900a7de0 con 0x7ff89009df20 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89ef2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff8900a80e0 con 0x7ff89009d560 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 0x7ff890143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 0x7ff890143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33368/0 (socket says 172.21.15.64:33368) 2024-05-09T09:13:02.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff89d72b640 1 -- 172.21.15.64:0/4183114229 learned_addr learned my addr 172.21.15.64:0/4183114229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.911+0000 7ff8a0190640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff89009df20 0x7ff89014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89d72b640 1 -- 172.21.15.64:0/4183114229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff89009df20 msgr2=0x7ff89014abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89d72b640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff89009df20 0x7ff89014abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89d72b640 1 -- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff89009c370 msgr2=0x7ff890143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89d72b640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff89009c370 0x7ff890143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89d72b640 1 -- 172.21.15.64:0/4183114229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89014b250 con 0x7ff89009d560 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89d72b640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 0x7ff890143b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff88800e9f0 tx=0x7ff88800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff88effd640 1 -- 172.21.15.64:0/4183114229 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88800ccc0 con 0x7ff89009d560 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff88effd640 1 -- 172.21.15.64:0/4183114229 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8880108e0 con 0x7ff89009d560 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff88effd640 1 -- 172.21.15.64:0/4183114229 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880187e0 con 0x7ff89009d560 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.912+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89014b540 con 0x7ff89009d560 2024-05-09T09:13:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.913+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89014ba00 con 0x7ff89009d560 2024-05-09T09:13:02.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.916+0000 7ff88effd640 1 -- 172.21.15.64:0/4183114229 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff888018980 con 0x7ff89009d560 2024-05-09T09:13:02.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.917+0000 7ff8a2f7e740 1 Processor -- start 2024-05-09T09:13:02.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.917+0000 7ff8a2f7e740 1 -- start start 2024-05-09T09:13:02.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.917+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c97100 0x55f889c97520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889c9c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c9cc60 0x55f889c9f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f889bfbc40 con 0x55f889c97100 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f889ba3580 con 0x55f889c9cc60 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f889bf7c00 con 0x55f889c97af0 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff88effd640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8840a2c10 0x7ff8840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.918+0000 7ff88effd640 1 -- 172.21.15.64:0/4183114229 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff8880cf7d0 con 0x7ff89009d560 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.919+0000 7ff8a0190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c97100 0x55f889c97520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.919+0000 7ff8a0190640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8840a2c10 0x7ff8840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.919+0000 7ff8a0190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c97100 0x55f889c97520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33242/0 (socket says 172.21.15.64:33242) 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.919+0000 7ff8a0190640 1 -- 172.21.15.64:0/1157444790 learned_addr learned my addr 172.21.15.64:0/1157444790 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.919+0000 7ff89df2c640 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889c9c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.919+0000 7ff89d72b640 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c9cc60 0x55f889c9f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.920+0000 7ff89df2c640 1 -- 172.21.15.64:0/1157444790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c9cc60 msgr2=0x55f889c9f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.920+0000 7ff89df2c640 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c9cc60 0x55f889c9f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.920+0000 7ff89df2c640 1 -- 172.21.15.64:0/1157444790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c97100 msgr2=0x55f889c97520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.920+0000 7ff89df2c640 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c97100 0x55f889c97520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.920+0000 7ff89df2c640 1 -- 172.21.15.64:0/1157444790 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88990f3f0 con 0x55f889c97af0 2024-05-09T09:13:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.920+0000 7ff8a0190640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8840a2c10 0x7ff8840a50d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff89804f8c0 tx=0x7ff89804fc50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.922+0000 7ff89df2c640 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889c9c5f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff89402a920 tx=0x7ff894000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad385153e3f385b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.922+0000 7ff883fff640 1 -- 172.21.15.64:0/1157444790 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89403e030 con 0x55f889c97af0 2024-05-09T09:13:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.922+0000 7ff883fff640 1 -- 172.21.15.64:0/1157444790 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8940035f0 con 0x55f889c97af0 2024-05-09T09:13:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.922+0000 7ff883fff640 1 -- 172.21.15.64:0/1157444790 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894035040 con 0x55f889c97af0 2024-05-09T09:13:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.923+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/1157444790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 msgr2=0x55f889c9c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.923+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889c9c5f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff89402a920 tx=0x7ff894000bc0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/1157444790 shutdown_connections 2024-05-09T09:13:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c9cc60 0x55f889c9f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889c9c5f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/1157444790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c97100 0x55f889c97520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/1157444790 >> 172.21.15.64:0/1157444790 conn(0x55f889c92770 msgr2=0x55f889c92b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/1157444790 shutdown_connections 2024-05-09T09:13:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/1157444790 wait complete. 2024-05-09T09:13:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.924+0000 7ff8a2f7e740 1 -- start start 2024-05-09T09:13:02.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 Processor -- start 2024-05-09T09:13:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889ca8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f889c9fbb0 con 0x55f889c9cc60 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f889bb6c40 con 0x55f889c97100 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f889c9fd30 con 0x55f889c97af0 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a2f7e740 10 client.0 _ll_register_callbacks cb 0x55f88990ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff8a0190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889ca8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff89df2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33394/0 (socket says 172.21.15.64:33394) 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff89d72b640 1 -- 172.21.15.64:0/390144776 learned_addr learned my addr 172.21.15.64:0/390144776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.925+0000 7ff89df2c640 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41836/0 (socket says 172.21.15.64:41836) 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff89d72b640 1 -- 172.21.15.64:0/390144776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 msgr2=0x55f889ca1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff89d72b640 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff89d72b640 1 -- 172.21.15.64:0/390144776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 msgr2=0x55f889ca8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff89d72b640 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889ca8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff89d72b640 1 -- 172.21.15.64:0/390144776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8899d97d0 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff89d72b640 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca1a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ccf90 tx=0x7ff888010f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2630bad4f9b1ab0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff8817fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff8817fa640 1 -- 172.21.15.64:0/390144776 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880a4910 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff8817fa640 1 -- 172.21.15.64:0/390144776 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff888010430 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.926+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8940035f0 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8817fa640 1 -- 172.21.15.64:0/390144776 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff888010630 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 10 client.26233 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f889cabdf0 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8817fa640 1 -- 172.21.15.64:0/390144776 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8940035f0 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8817fa640 1 -- 172.21.15.64:0/390144776 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff88809ab60 con 0x55f889c97af0 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 10 client.26233 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 10 client.26233 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 20 client.26233 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889cad5c0 0x55f889cafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 4 client.26233 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.927+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f889c09370 con 0x55f889cad5c0 2024-05-09T09:13:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.928+0000 7ff89df2c640 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889cad5c0 0x55f889cafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.929+0000 7ff89df2c640 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889cad5c0 0x55f889cafa40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a63262eb53319098 server_cookie=c45b0c9659c5bf42 in_seq=0 out_seq=0 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.929+0000 7ff8817fa640 10 client.26233 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8817fa640 1 -- 172.21.15.64:0/390144776 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f889c09370 con 0x55f889cad5c0 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8817fa640 10 client.26233 handle_command_reply: tid=0 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 client.26233 shutdown 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 msgr2=0x55f889ca1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca1a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ccf90 tx=0x7ff888010f40 comp rx=0 tx=0).stop 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 shutdown_connections 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889cad5c0 0x55f889cafa40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889ca8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca1a60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/390144776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.930+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 >> 172.21.15.64:0/390144776 conn(0x55f889c92770 msgr2=0x55f889c91990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.931+0000 7ff8817fa640 0 client.26233 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.931+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 shutdown_connections 2024-05-09T09:13:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.931+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/390144776 wait complete. 2024-05-09T09:13:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.931+0000 7ff8a2f7e740 20 client.26233 trim_cache size 0 max 16384 2024-05-09T09:13:02.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.958+0000 7ff8a2f7e740 1 Processor -- start 2024-05-09T09:13:02.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.958+0000 7ff8a2f7e740 1 -- start start 2024-05-09T09:13:02.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.958+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.958+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.958+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889c95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff890150090 con 0x55f889c9cc60 2024-05-09T09:13:02.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff89014d900 con 0x55f889c97100 2024-05-09T09:13:02.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff8a0190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889c95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff8a0190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889c95720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33252/0 (socket says 172.21.15.64:33252) 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff8a0190640 1 -- 172.21.15.64:0/4091757590 learned_addr learned my addr 172.21.15.64:0/4091757590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/4091757590 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff890015710 con 0x55f889c97af0 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.959+0000 7ff89df2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8a0190640 1 -- 172.21.15.64:0/4091757590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 msgr2=0x55f889ca8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8a0190640 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8a0190640 1 -- 172.21.15.64:0/4091757590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 msgr2=0x55f889ca9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8a0190640 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8a0190640 1 -- 172.21.15.64:0/4091757590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f889c934e0 con 0x55f889c9cc60 2024-05-09T09:13:02.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8a0190640 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889c95720 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff89807c010 tx=0x7ff89807e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c792b58bc4f99c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.960+0000 7ff8837fe640 1 -- 172.21.15.64:0/4091757590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89808a070 con 0x55f889c9cc60 2024-05-09T09:13:02.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.961+0000 7ff8837fe640 1 -- 172.21.15.64:0/4091757590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff898080040 con 0x55f889c9cc60 2024-05-09T09:13:02.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.961+0000 7ff8837fe640 1 -- 172.21.15.64:0/4091757590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980855c0 con 0x55f889c9cc60 2024-05-09T09:13:02.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.962+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/4091757590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 msgr2=0x55f889c95720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.962+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889c95720 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff89807c010 tx=0x7ff89807e000 comp rx=0 tx=0).stop 2024-05-09T09:13:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.963+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/4091757590 shutdown_connections 2024-05-09T09:13:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.963+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889c95720 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.963+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.963+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/4091757590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.963+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/4091757590 >> 172.21.15.64:0/4091757590 conn(0x55f889c92770 msgr2=0x55f889ca94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.964+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/4091757590 shutdown_connections 2024-05-09T09:13:02.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.964+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/4091757590 wait complete. 2024-05-09T09:13:02.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.965+0000 7ff8a2f7e740 1 -- start start 2024-05-09T09:13:02.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.965+0000 7ff8a2f7e740 1 Processor -- start 2024-05-09T09:13:02.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.966+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.966+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.966+0000 7ff8a2f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889d11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff89df2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff89014bdc0 con 0x55f889c9cc60 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff89014d230 con 0x55f889c97100 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a2f7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff89014c490 con 0x55f889c97af0 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff89d72b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33420/0 (socket says 172.21.15.64:33420) 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff89d72b640 1 -- 172.21.15.64:0/3994962911 learned_addr learned my addr 172.21.15.64:0/3994962911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a2f7e740 10 client.0 _ll_register_callbacks cb 0x7ff888097f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:02.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a0190640 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889d11bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a0190640 1 -- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 msgr2=0x55f889ca5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a0190640 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a0190640 1 -- 172.21.15.64:0/3994962911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 msgr2=0x55f889ca5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a0190640 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.967+0000 7ff8a0190640 1 -- 172.21.15.64:0/3994962911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f889ca62e0 con 0x55f889c9cc60 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.968+0000 7ff8a0190640 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889d11bb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff89807c010 tx=0x7ff898086c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dc4021f27c45da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:02.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.968+0000 7ff8817fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:13:02.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.968+0000 7ff8817fa640 1 -- 172.21.15.64:0/3994962911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89808a050 con 0x55f889c9cc60 2024-05-09T09:13:02.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.969+0000 7ff8817fa640 1 -- 172.21.15.64:0/3994962911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff89808f860 con 0x55f889c9cc60 2024-05-09T09:13:02.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.969+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff898080040 con 0x55f889c9cc60 2024-05-09T09:13:02.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.969+0000 7ff8817fa640 1 -- 172.21.15.64:0/3994962911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff8980a0050 con 0x55f889c9cc60 2024-05-09T09:13:02.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.970+0000 7ff8a2f7e740 10 client.16572 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:02.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.970+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f889d131a0 con 0x55f889c9cc60 2024-05-09T09:13:02.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8817fa640 1 -- 172.21.15.64:0/3994962911 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff898097400 con 0x55f889c9cc60 2024-05-09T09:13:02.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8817fa640 1 -- 172.21.15.64:0/3994962911 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7ff898085040 con 0x55f889c9cc60 2024-05-09T09:13:02.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8a2f7e740 10 client.16572 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:02.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8a2f7e740 10 client.16572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:13:02.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8a2f7e740 20 client.16572 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:02.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889d13b30 0x55f889d15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:02.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8a2f7e740 4 client.16572 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:13:02.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.971+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff898080040 con 0x55f889d13b30 2024-05-09T09:13:02.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.972+0000 7ff89df2c640 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889d13b30 0x55f889d15f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.973+0000 7ff89df2c640 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889d13b30 0x55f889d15f50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d99abcc415564c5a server_cookie=54d0e028e7172473 in_seq=0 out_seq=0 2024-05-09T09:13:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.973+0000 7ff8817fa640 10 client.16572 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.973+0000 7ff8817fa640 1 -- 172.21.15.64:0/3994962911 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff898080040 con 0x55f889d13b30 2024-05-09T09:13:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.973+0000 7ff8817fa640 10 client.16572 handle_command_reply: tid=0 2024-05-09T09:13:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.973+0000 7ff8a2f7e740 1 client.16572 shutdown 2024-05-09T09:13:02.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.974+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 msgr2=0x55f889d11bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.974+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889d11bb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff89807c010 tx=0x7ff898086c20 comp rx=0 tx=0).stop 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 shutdown_connections 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f889d13b30 0x55f889d15f50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f889c9cc60 0x55f889d11bb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8817fa640 0 client.16572 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f889c97af0 0x55f889ca5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8a2f7e740 1 --2- 172.21.15.64:0/3994962911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f889c97100 0x55f889ca5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.976+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 >> 172.21.15.64:0/3994962911 conn(0x55f889c92770 msgr2=0x55f889c95720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.977+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 shutdown_connections 2024-05-09T09:13:02.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.977+0000 7ff8a2f7e740 1 -- 172.21.15.64:0/3994962911 wait complete. 2024-05-09T09:13:02.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.977+0000 7ff8a2f7e740 20 client.16572 trim_cache size 0 max 16384 2024-05-09T09:13:02.976 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.978+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8840a2c10 msgr2=0x7ff8840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.978+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8840a2c10 0x7ff8840a50d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff89804f8c0 tx=0x7ff89804fc50 comp rx=0 tx=0).stop 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.978+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 msgr2=0x7ff890143b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.978+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 0x7ff890143b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff88800e9f0 tx=0x7ff88800edf0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.978+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 shutdown_connections 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff8840a2c10 0x7ff8840a50d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff89009df20 0x7ff89014abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff89009d560 0x7ff890143b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 --2- 172.21.15.64:0/4183114229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff89009c370 0x7ff890143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:02.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 >> 172.21.15.64:0/4183114229 conn(0x7ff89000b310 msgr2=0x7ff8900a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 shutdown_connections 2024-05-09T09:13:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:02.979+0000 7ff89ef2e640 1 -- 172.21.15.64:0/4183114229 wait complete. 2024-05-09T09:13:03.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.194+0000 7f2343b5a640 1 -- 172.21.15.64:0/376060572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c108fe0 msgr2=0x7f233c109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.194+0000 7f2343b5a640 1 --2- 172.21.15.64:0/376060572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c108fe0 0x7f233c109460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2334009ca0 tx=0x7f233402a8e0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 -- 172.21.15.64:0/376060572 shutdown_connections 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 --2- 172.21.15.64:0/376060572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c1099a0 0x7f233c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 --2- 172.21.15.64:0/376060572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c108fe0 0x7f233c109460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 --2- 172.21.15.64:0/376060572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f233c107e90 0x7f233c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 -- 172.21.15.64:0/376060572 >> 172.21.15.64:0/376060572 conn(0x7f233c076e30 msgr2=0x7f233c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 -- 172.21.15.64:0/376060572 shutdown_connections 2024-05-09T09:13:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.195+0000 7f2343b5a640 1 -- 172.21.15.64:0/376060572 wait complete. 2024-05-09T09:13:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 Processor -- start 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 -- start start 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f233c107e90 0x7f233c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 0x7f233c071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c1099a0 0x7f233c072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f233c06b9b0 con 0x7f233c107e90 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f233c06b830 con 0x7f233c108fe0 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f2343b5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f233c06bb30 con 0x7f233c1099a0 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f23410f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 0x7f233c071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f23410f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 0x7f233c071820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41874/0 (socket says 172.21.15.64:41874) 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f23410f5640 1 -- 172.21.15.64:0/294799164 learned_addr learned my addr 172.21.15.64:0/294799164 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f23418f6640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f233c107e90 0x7f233c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.196+0000 7f23420f7640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c1099a0 0x7f233c072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.197+0000 7f23410f5640 1 -- 172.21.15.64:0/294799164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c1099a0 msgr2=0x7f233c072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.197+0000 7f23410f5640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c1099a0 0x7f233c072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.197+0000 7f23410f5640 1 -- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f233c107e90 msgr2=0x7f233c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.197+0000 7f23410f5640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f233c107e90 0x7f233c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.197+0000 7f23410f5640 1 -- 172.21.15.64:0/294799164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c072ed0 con 0x7f233c108fe0 2024-05-09T09:13:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.199+0000 7f23410f5640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 0x7f233c071820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2334009bf0 tx=0x7f2334032fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.199+0000 7f2332ffd640 1 -- 172.21.15.64:0/294799164 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334004240 con 0x7f233c108fe0 2024-05-09T09:13:03.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.199+0000 7f2332ffd640 1 -- 172.21.15.64:0/294799164 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23340305e0 con 0x7f233c108fe0 2024-05-09T09:13:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.200+0000 7f2332ffd640 1 -- 172.21.15.64:0/294799164 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233403a660 con 0x7f233c108fe0 2024-05-09T09:13:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.200+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c073160 con 0x7f233c108fe0 2024-05-09T09:13:03.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.200+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c073630 con 0x7f233c108fe0 2024-05-09T09:13:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.204+0000 7f2346867740 1 Processor -- start 2024-05-09T09:13:03.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.205+0000 7f2332ffd640 1 -- 172.21.15.64:0/294799164 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23340043e0 con 0x7f233c108fe0 2024-05-09T09:13:03.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.205+0000 7f2346867740 1 -- start start 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f2346867740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f2346867740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc83930 0x56348fc88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f2346867740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 0x56348fc8af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f2346867740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56348fbe7c40 con 0x56348fc88aa0 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f2346867740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56348fb8f580 con 0x56348fc82f40 2024-05-09T09:13:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f2346867740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56348fbe3c00 con 0x56348fc83930 2024-05-09T09:13:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f23410f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 0x56348fc8af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f23410f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 0x56348fc8af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33282/0 (socket says 172.21.15.64:33282) 2024-05-09T09:13:03.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f23410f5640 1 -- 172.21.15.64:0/848910635 learned_addr learned my addr 172.21.15.64:0/848910635 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.206+0000 7f23420f7640 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23418f6640 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc83930 0x56348fc88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f2332ffd640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23280a2d60 0x7f23280a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 -- 172.21.15.64:0/848910635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc83930 msgr2=0x56348fc88430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc83930 0x56348fc88430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 -- 172.21.15.64:0/848910635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 msgr2=0x56348fc8af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 0x56348fc8af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 -- 172.21.15.64:0/848910635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348f8fb3f0 con 0x56348fc82f40 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23410f5640 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 0x56348fc8af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f2332ffd640 1 -- 172.21.15.64:0/294799164 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23340f0b80 con 0x7f233c108fe0 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc83360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2338009900 tx=0x7f2338009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c03b994c10ba4ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f23420f7640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23280a2d60 0x7f23280a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.207+0000 7f2327fff640 1 -- 172.21.15.64:0/848910635 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338017070 con 0x56348fc82f40 2024-05-09T09:13:03.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2327fff640 1 -- 172.21.15.64:0/848910635 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2338013920 con 0x56348fc82f40 2024-05-09T09:13:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2327fff640 1 -- 172.21.15.64:0/848910635 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233801ca80 con 0x56348fc82f40 2024-05-09T09:13:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 -- 172.21.15.64:0/848910635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 msgr2=0x56348fc83360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc83360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2338009900 tx=0x7f2338009dc0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f23420f7640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23280a2d60 0x7f23280a5220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2338009fd0 tx=0x7f23380127a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 -- 172.21.15.64:0/848910635 shutdown_connections 2024-05-09T09:13:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc88aa0 0x56348fc8af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc83930 0x56348fc88430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 --2- 172.21.15.64:0/848910635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc83360 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.208+0000 7f2346867740 1 -- 172.21.15.64:0/848910635 >> 172.21.15.64:0/848910635 conn(0x56348fc7e5b0 msgr2=0x56348fc7e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:03.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.210+0000 7f2346867740 1 -- 172.21.15.64:0/848910635 shutdown_connections 2024-05-09T09:13:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.210+0000 7f2346867740 1 -- 172.21.15.64:0/848910635 wait complete. 2024-05-09T09:13:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.210+0000 7f2346867740 1 -- start start 2024-05-09T09:13:03.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.211+0000 7f2346867740 1 Processor -- start 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.211+0000 7f2346867740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 0x56348fc9e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f2346867740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fc9e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f2346867740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc88aa0 0x56348fca5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 0x56348fc9e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 0x56348fc9e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33458/0 (socket says 172.21.15.64:33458) 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 -- 172.21.15.64:0/2257211611 learned_addr learned my addr 172.21.15.64:0/2257211611 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56348fc9cb00 con 0x56348fc83930 2024-05-09T09:13:03.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56348fba2c40 con 0x56348fc88aa0 2024-05-09T09:13:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56348fc9cc80 con 0x56348fc82f40 2024-05-09T09:13:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23410f5640 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fc9e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f2346867740 10 client.0 _ll_register_callbacks cb 0x56348f8fabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:03.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 -- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc88aa0 msgr2=0x56348fca5910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc88aa0 0x56348fca5910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 -- 172.21.15.64:0/2257211611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 msgr2=0x56348fc9e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fc9e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.212+0000 7f23418f6640 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348f9c57d0 con 0x56348fc82f40 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.213+0000 7f23418f6640 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 0x56348fc9e400 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f232c010270 tx=0x7f232c010670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ca873239bb4bb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.213+0000 7f23257fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.213+0000 7f23257fa640 1 -- 172.21.15.64:0/2257211611 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c018070 con 0x56348fc82f40 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.213+0000 7f23257fa640 1 -- 172.21.15.64:0/2257211611 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f232c014600 con 0x56348fc82f40 2024-05-09T09:13:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.213+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2338013920 con 0x56348fc82f40 2024-05-09T09:13:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.215+0000 7f23257fa640 1 -- 172.21.15.64:0/2257211611 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f232c024050 con 0x56348fc82f40 2024-05-09T09:13:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 10 client.26248 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:03.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56348fca8d10 con 0x56348fc82f40 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f23257fa640 1 -- 172.21.15.64:0/2257211611 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f232c014cb0 con 0x56348fc82f40 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f23257fa640 1 -- 172.21.15.64:0/2257211611 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f232c01cb40 con 0x56348fc82f40 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 10 client.26248 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 20 client.26248 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fcaa3f0 0x56348fcac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.216+0000 7f2346867740 4 client.26248 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.217+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2338013920 con 0x56348fcaa3f0 2024-05-09T09:13:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.217+0000 7f23410f5640 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fcaa3f0 0x56348fcac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.218+0000 7f23410f5640 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fcaa3f0 0x56348fcac890 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0602be212450d9e server_cookie=ade5198c004ec608 in_seq=0 out_seq=0 2024-05-09T09:13:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.218+0000 7f23257fa640 10 client.26248 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.219+0000 7f23257fa640 1 -- 172.21.15.64:0/2257211611 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f2338013920 con 0x56348fcaa3f0 2024-05-09T09:13:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.219+0000 7f23257fa640 10 client.26248 handle_command_reply: tid=0 2024-05-09T09:13:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.219+0000 7f2346867740 1 client.26248 shutdown 2024-05-09T09:13:03.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.220+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 msgr2=0x56348fc9e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.220+0000 7f2346867740 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 0x56348fc9e400 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f232c010270 tx=0x7f232c010670 comp rx=0 tx=0).stop 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 shutdown_connections 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fcaa3f0 0x56348fcac890 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc88aa0 0x56348fca5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fc9e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 --2- 172.21.15.64:0/2257211611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc82f40 0x56348fc9e400 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 >> 172.21.15.64:0/2257211611 conn(0x56348fc7e5b0 msgr2=0x56348fc7d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f23257fa640 0 client.26248 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 shutdown_connections 2024-05-09T09:13:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 1 -- 172.21.15.64:0/2257211611 wait complete. 2024-05-09T09:13:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.221+0000 7f2346867740 20 client.26248 trim_cache size 0 max 16384 2024-05-09T09:13:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 Processor -- start 2024-05-09T09:13:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 -- start start 2024-05-09T09:13:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc7cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fca58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fc7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1b3300 con 0x56348fc83930 2024-05-09T09:13:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1b0b70 con 0x56348fc82f40 2024-05-09T09:13:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.252+0000 7f2346867740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f233c082130 con 0x56348fc88aa0 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23420f7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fc7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fca58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fca58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33306/0 (socket says 172.21.15.64:33306) 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 -- 172.21.15.64:0/125398176 learned_addr learned my addr 172.21.15.64:0/125398176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23418f6640 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc7cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 -- 172.21.15.64:0/125398176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 msgr2=0x56348fc7d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fc7d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 -- 172.21.15.64:0/125398176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 msgr2=0x56348fc7cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc7cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 -- 172.21.15.64:0/125398176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348fc6fd20 con 0x56348fc83930 2024-05-09T09:13:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23410f5640 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fca58d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f23340b99c0 tx=0x7f23340f2f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b137d780cd8732d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.253+0000 7f23277fe640 1 -- 172.21.15.64:0/125398176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23340f9040 con 0x56348fc83930 2024-05-09T09:13:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.254+0000 7f23277fe640 1 -- 172.21.15.64:0/125398176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23340fa040 con 0x56348fc83930 2024-05-09T09:13:03.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.254+0000 7f23277fe640 1 -- 172.21.15.64:0/125398176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23340c13a0 con 0x56348fc83930 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 -- 172.21.15.64:0/125398176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 msgr2=0x56348fca58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fca58d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f23340b99c0 tx=0x7f23340f2f50 comp rx=0 tx=0).stop 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 -- 172.21.15.64:0/125398176 shutdown_connections 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fc7d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc83930 0x56348fca58d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 --2- 172.21.15.64:0/125398176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc82f40 0x56348fc7cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 -- 172.21.15.64:0/125398176 >> 172.21.15.64:0/125398176 conn(0x56348fc7e5b0 msgr2=0x56348fc84cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:03.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 -- 172.21.15.64:0/125398176 shutdown_connections 2024-05-09T09:13:03.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.255+0000 7f2346867740 1 -- 172.21.15.64:0/125398176 wait complete. 2024-05-09T09:13:03.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.256+0000 7f2346867740 1 -- start start 2024-05-09T09:13:03.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.256+0000 7f2346867740 1 Processor -- start 2024-05-09T09:13:03.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.256+0000 7f2346867740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc82f40 0x56348fca29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f2346867740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc83930 0x56348fca2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23418f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc82f40 0x56348fca29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23418f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc82f40 0x56348fca29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33310/0 (socket says 172.21.15.64:33310) 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f2346867740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fca3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f2346867740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1af120 con 0x56348fc82f40 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f2346867740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1b04a0 con 0x56348fc83930 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f2346867740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1af700 con 0x56348fc88aa0 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f2346867740 10 client.0 _ll_register_callbacks cb 0x7f232c0093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23410f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc83930 0x56348fca2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23420f7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fca3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23410f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc83930 0x56348fca2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41900/0 (socket says 172.21.15.64:41900) 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23410f5640 1 -- 172.21.15.64:0/3008888462 learned_addr learned my addr 172.21.15.64:0/3008888462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23420f7640 1 -- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc83930 msgr2=0x56348fca2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23420f7640 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc83930 0x56348fca2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23420f7640 1 -- 172.21.15.64:0/3008888462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc82f40 msgr2=0x56348fca29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.257+0000 7f23420f7640 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc82f40 0x56348fca29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.258+0000 7f23420f7640 1 -- 172.21.15.64:0/3008888462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348fca4500 con 0x56348fc88aa0 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.258+0000 7f23420f7640 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fca3f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2338009850 tx=0x7f233801e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9ac084bf4222a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.258+0000 7f23257fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.258+0000 7f23257fa640 1 -- 172.21.15.64:0/3008888462 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338009540 con 0x56348fc88aa0 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.259+0000 7f23257fa640 1 -- 172.21.15.64:0/3008888462 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2338013a70 con 0x56348fc88aa0 2024-05-09T09:13:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.259+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23340fa040 con 0x56348fc88aa0 2024-05-09T09:13:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.260+0000 7f23257fa640 1 -- 172.21.15.64:0/3008888462 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2338013c70 con 0x56348fc88aa0 2024-05-09T09:13:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.260+0000 7f2346867740 10 client.26254 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.260+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56348fca5850 con 0x56348fc88aa0 2024-05-09T09:13:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f23257fa640 1 -- 172.21.15.64:0/3008888462 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2338005e20 con 0x56348fc88aa0 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f23257fa640 1 -- 172.21.15.64:0/3008888462 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f2338006560 con 0x56348fc88aa0 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f2346867740 10 client.26254 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f2346867740 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f2346867740 20 client.26254 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f2346867740 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fd10ba0 0x56348fd12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f2346867740 4 client.26254 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f23340fa040 con 0x56348fd10ba0 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f23418f6640 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fd10ba0 0x56348fd12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.261+0000 7f23257fa640 10 client.26254 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.262+0000 7f23418f6640 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fd10ba0 0x56348fd12f90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9fd1a14e7c1273 server_cookie=5db4c69717e25503 in_seq=0 out_seq=0 2024-05-09T09:13:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.266+0000 7f23257fa640 1 -- 172.21.15.64:0/3008888462 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f23340fa040 con 0x56348fd10ba0 2024-05-09T09:13:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.266+0000 7f23257fa640 10 client.26254 handle_command_reply: tid=0 2024-05-09T09:13:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.266+0000 7f2346867740 1 client.26254 shutdown 2024-05-09T09:13:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.266+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 msgr2=0x56348fca3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.266+0000 7f2346867740 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fca3f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2338009850 tx=0x7f233801e050 comp rx=0 tx=0).stop 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 shutdown_connections 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f2346867740 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56348fd10ba0 0x56348fd12f90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f2346867740 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56348fc88aa0 0x56348fca3f10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f2346867740 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56348fc83930 0x56348fca2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f2346867740 1 --2- 172.21.15.64:0/3008888462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56348fc82f40 0x56348fca29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 >> 172.21.15.64:0/3008888462 conn(0x56348fc7e5b0 msgr2=0x56348fc81330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:03.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.267+0000 7f23257fa640 0 client.26254 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:03.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.268+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 shutdown_connections 2024-05-09T09:13:03.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.268+0000 7f2346867740 1 -- 172.21.15.64:0/3008888462 wait complete. 2024-05-09T09:13:03.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.268+0000 7f2346867740 20 client.26254 trim_cache size 0 max 16384 2024-05-09T09:13:03.267 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:13:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23280a2d60 msgr2=0x7f23280a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23280a2d60 0x7f23280a5220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2338009fd0 tx=0x7f23380127a0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 msgr2=0x7f233c071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 0x7f233c071820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2334009bf0 tx=0x7f2334032fd0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 shutdown_connections 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23280a2d60 0x7f23280a5220 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c1099a0 0x7f233c072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c108fe0 0x7f233c071820 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.269+0000 7f2343b5a640 1 --2- 172.21.15.64:0/294799164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f233c107e90 0x7f233c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.270+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 >> 172.21.15.64:0/294799164 conn(0x7f233c076e30 msgr2=0x7f233c10c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.270+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 shutdown_connections 2024-05-09T09:13:03.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:03.270+0000 7f2343b5a640 1 -- 172.21.15.64:0/294799164 wait complete. 2024-05-09T09:13:04.295 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:13:04.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:13:04.296 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 -- 172.21.15.64:0/1765881735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 msgr2=0x7fb3c410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/1765881735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c410e020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009be0 tx=0x7fb3bc02a6c0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 -- 172.21.15.64:0/1765881735 shutdown_connections 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/1765881735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c410e020 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/1765881735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3c410b1a0 0x7fb3c410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/1765881735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 0x7fb3c410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.491+0000 7fb3cc24d640 1 -- 172.21.15.64:0/1765881735 >> 172.21.15.64:0/1765881735 conn(0x7fb3c406e6c0 msgr2=0x7fb3c4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.493+0000 7fb3cc24d640 1 -- 172.21.15.64:0/1765881735 shutdown_connections 2024-05-09T09:13:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.495+0000 7fb3cc24d640 1 -- 172.21.15.64:0/1765881735 wait complete. 2024-05-09T09:13:04.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 Processor -- start 2024-05-09T09:13:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 -- start start 2024-05-09T09:13:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 0x7fb3c41a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3c410b1a0 0x7fb3c41a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c41ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c411c200 con 0x7fb3c410b1a0 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c411c080 con 0x7fb3c410db70 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3cc24d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c411c380 con 0x7fb3c4108830 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3c9fe9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 0x7fb3c41a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3ca7ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c41ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3c9fe9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 0x7fb3c41a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33504/0 (socket says 172.21.15.64:33504) 2024-05-09T09:13:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.496+0000 7fb3c9fe9640 1 -- 172.21.15.64:0/3864111972 learned_addr learned my addr 172.21.15.64:0/3864111972 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.497+0000 7fb3ca7ea640 1 -- 172.21.15.64:0/3864111972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 msgr2=0x7fb3c41a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.497+0000 7fb3ca7ea640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 0x7fb3c41a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.497+0000 7fb3ca7ea640 1 -- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3c410b1a0 msgr2=0x7fb3c41a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.497+0000 7fb3ca7ea640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3c410b1a0 0x7fb3c41a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.497+0000 7fb3ca7ea640 1 -- 172.21.15.64:0/3864111972 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c41ae4f0 con 0x7fb3c410db70 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.498+0000 7fb3ca7ea640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c41ade80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc030850 tx=0x7fb3bc030640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.498+0000 7fb3baffd640 1 -- 172.21.15.64:0/3864111972 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc041020 con 0x7fb3c410db70 2024-05-09T09:13:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.498+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c41ae780 con 0x7fb3c410db70 2024-05-09T09:13:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.498+0000 7fb3baffd640 1 -- 172.21.15.64:0/3864111972 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3bc037040 con 0x7fb3c410db70 2024-05-09T09:13:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.498+0000 7fb3baffd640 1 -- 172.21.15.64:0/3864111972 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc036070 con 0x7fb3c410db70 2024-05-09T09:13:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.498+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c41aec90 con 0x7fb3c410db70 2024-05-09T09:13:04.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.502+0000 7fb3baffd640 1 -- 172.21.15.64:0/3864111972 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb3bc044020 con 0x7fb3c410db70 2024-05-09T09:13:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.503+0000 7fb3cf037740 1 Processor -- start 2024-05-09T09:13:04.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.503+0000 7fb3cf037740 1 -- start start 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 0x559ed40788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed407d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3cf037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559ed3fdc990 con 0x559ed407e020 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3cf037740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559ed3f845b0 con 0x559ed4078eb0 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3cf037740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559ed3fd8950 con 0x559ed40784c0 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 0x559ed40788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 0x559ed40788e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33514/0 (socket says 172.21.15.64:33514) 2024-05-09T09:13:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 -- 172.21.15.64:0/313288326 learned_addr learned my addr 172.21.15.64:0/313288326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed407d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3baffd640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb3b00a2c90 0x7fb3b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 -- 172.21.15.64:0/313288326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 msgr2=0x559ed407d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed407d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 -- 172.21.15.64:0/313288326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 msgr2=0x559ed40804b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.504+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40804b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3c97e8640 1 -- 172.21.15.64:0/313288326 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed3db87e0 con 0x559ed40784c0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 0x559ed40788e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005c80 tx=0x7fb3b40095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ebfefb25b4dae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3baffd640 1 -- 172.21.15.64:0/3864111972 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3bc0efee0 con 0x7fb3c410db70 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3ca7ea640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb3b00a2c90 0x7fb3b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3affff640 1 -- 172.21.15.64:0/313288326 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4013110 con 0x559ed40784c0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3affff640 1 -- 172.21.15.64:0/313288326 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b40132b0 con 0x559ed40784c0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.505+0000 7fb3affff640 1 -- 172.21.15.64:0/313288326 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40047a0 con 0x559ed40784c0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3ca7ea640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb3b00a2c90 0x7fb3b00a5150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0bd0c0 tx=0x7fb3bc02a6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 -- 172.21.15.64:0/313288326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 msgr2=0x559ed40788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 0x559ed40788e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005c80 tx=0x7fb3b40095e0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 -- 172.21.15.64:0/313288326 shutdown_connections 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed407d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 --2- 172.21.15.64:0/313288326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed40784c0 0x559ed40788e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 -- 172.21.15.64:0/313288326 >> 172.21.15.64:0/313288326 conn(0x559ed40400a0 msgr2=0x559ed4073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.506+0000 7fb3cf037740 1 -- 172.21.15.64:0/313288326 shutdown_connections 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.507+0000 7fb3cf037740 1 -- 172.21.15.64:0/313288326 wait complete. 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.507+0000 7fb3cf037740 1 -- start start 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.507+0000 7fb3cf037740 1 Processor -- start 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 0x559ed40828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed407e020 0x559ed4089e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559ed4080f70 con 0x559ed4078eb0 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559ed3f97b40 con 0x559ed40784c0 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559ed40810f0 con 0x559ed407e020 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 0x559ed40828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3cf037740 10 client.0 _ll_register_callbacks cb 0x559ed3f8c0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:04.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 0x559ed40828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41924/0 (socket says 172.21.15.64:41924) 2024-05-09T09:13:04.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 -- 172.21.15.64:0/4189996977 learned_addr learned my addr 172.21.15.64:0/4189996977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed407e020 0x559ed4089e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 -- 172.21.15.64:0/4189996977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed407e020 msgr2=0x559ed4089e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed407e020 0x559ed4089e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 -- 172.21.15.64:0/4189996977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 msgr2=0x559ed4082e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.509+0000 7fb3c97e8640 1 -- 172.21.15.64:0/4189996977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed3e18f70 con 0x559ed40784c0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.510+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 0x559ed40828f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4004700 tx=0x7fb3b4005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8b21a0686fa775 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.510+0000 7fb3ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.510+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/4189996977 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4027070 con 0x559ed40784c0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.510+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/4189996977 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3b40049a0 con 0x559ed40784c0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.511+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b40132b0 con 0x559ed40784c0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.511+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/4189996977 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb3b4004ba0 con 0x559ed40784c0 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.511+0000 7fb3cf037740 10 client.26117 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:04.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.511+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ed3cb0170 con 0x559ed40784c0 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.512+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/4189996977 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b4004d40 con 0x559ed40784c0 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.513+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/4189996977 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fb3b40114d0 con 0x559ed40784c0 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.513+0000 7fb3cf037740 10 client.26117 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.513+0000 7fb3cf037740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.513+0000 7fb3cf037740 20 client.26117 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.513+0000 7fb3cf037740 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed408e050 0x559ed4090470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.514+0000 7fb3cf037740 4 client.26117 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.514+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb3b40132b0 con 0x559ed408e050 2024-05-09T09:13:04.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.514+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed408e050 0x559ed4090470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.515+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed408e050 0x559ed4090470 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97064bb8e093f4ef server_cookie=7234060ba2235f36 in_seq=0 out_seq=0 2024-05-09T09:13:04.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.515+0000 7fb3ad7fa640 10 client.26117 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.516+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/4189996977 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb3b40132b0 con 0x559ed408e050 2024-05-09T09:13:04.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.516+0000 7fb3ad7fa640 10 client.26117 handle_command_reply: tid=0 2024-05-09T09:13:04.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.516+0000 7fb3cf037740 1 client.26117 shutdown 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.516+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 msgr2=0x559ed40828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.516+0000 7fb3cf037740 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 0x559ed40828f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4004700 tx=0x7fb3b4005a70 comp rx=0 tx=0).stop 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 shutdown_connections 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed408e050 0x559ed4090470 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed407e020 0x559ed4089e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 --2- 172.21.15.64:0/4189996977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed40784c0 0x559ed40828f0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4004700 tx=0x7fb3b4005a70 comp rx=0 tx=0).stop 2024-05-09T09:13:04.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 >> 172.21.15.64:0/4189996977 conn(0x559ed40400a0 msgr2=0x559ed4073080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3ad7fa640 0 client.26117 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 shutdown_connections 2024-05-09T09:13:04.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 1 -- 172.21.15.64:0/4189996977 wait complete. 2024-05-09T09:13:04.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.517+0000 7fb3cf037740 20 client.26117 trim_cache size 0 max 16384 2024-05-09T09:13:04.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 Processor -- start 2024-05-09T09:13:04.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 -- start start 2024-05-09T09:13:04.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed407e020 0x559ed4089e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4095d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8012600 con 0x559ed4078eb0 2024-05-09T09:13:04.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800fe70 con 0x559ed407e020 2024-05-09T09:13:04.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3cf037740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800dcc0 con 0x559ed4077540 2024-05-09T09:13:04.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33348/0 (socket says 172.21.15.64:33348) 2024-05-09T09:13:04.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3c97e8640 1 -- 172.21.15.64:0/3774111568 learned_addr learned my addr 172.21.15.64:0/3774111568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4095d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.556+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed407e020 0x559ed4089e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.557+0000 7fb3c97e8640 1 -- 172.21.15.64:0/3774111568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed407e020 msgr2=0x559ed4089e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.557+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed407e020 0x559ed4089e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.557+0000 7fb3c97e8640 1 -- 172.21.15.64:0/3774111568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 msgr2=0x559ed4082d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.557+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.557+0000 7fb3c97e8640 1 -- 172.21.15.64:0/3774111568 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed4074bf0 con 0x559ed4077540 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.557+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4095d20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4001eb0 tx=0x7fb3b4005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805300ec11795764 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.558+0000 7fb3af7fe640 1 -- 172.21.15.64:0/3774111568 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4027070 con 0x559ed4077540 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.558+0000 7fb3af7fe640 1 -- 172.21.15.64:0/3774111568 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b4033070 con 0x559ed4077540 2024-05-09T09:13:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.558+0000 7fb3af7fe640 1 -- 172.21.15.64:0/3774111568 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4011440 con 0x559ed4077540 2024-05-09T09:13:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 -- 172.21.15.64:0/3774111568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 msgr2=0x559ed4095d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4095d20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4001eb0 tx=0x7fb3b4005a00 comp rx=0 tx=0).stop 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 -- 172.21.15.64:0/3774111568 shutdown_connections 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4095d20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed407e020 0x559ed4089e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 --2- 172.21.15.64:0/3774111568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed4078eb0 0x559ed4082d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 -- 172.21.15.64:0/3774111568 >> 172.21.15.64:0/3774111568 conn(0x559ed40400a0 msgr2=0x559ed4076e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.559+0000 7fb3cf037740 1 -- 172.21.15.64:0/3774111568 shutdown_connections 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.560+0000 7fb3cf037740 1 -- 172.21.15.64:0/3774111568 wait complete. 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.560+0000 7fb3cf037740 1 -- start start 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.560+0000 7fb3cf037740 1 Processor -- start 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4087b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed40880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800e330 con 0x559ed407e020 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800f7a0 con 0x559ed4078eb0 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800ea00 con 0x559ed4077540 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3cf037740 10 client.0 _ll_register_callbacks cb 0x7fb3b4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4087b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4087b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33528/0 (socket says 172.21.15.64:33528) 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 -- 172.21.15.64:0/84395214 learned_addr learned my addr 172.21.15.64:0/84395214 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed40880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 -- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 msgr2=0x559ed40880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed40880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 -- 172.21.15.64:0/84395214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 msgr2=0x559ed40890b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40890b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.561+0000 7fb3c97e8640 1 -- 172.21.15.64:0/84395214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed40896a0 con 0x559ed4077540 2024-05-09T09:13:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3c97e8640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4087b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4013680 tx=0x7fb3b4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826748c10c181c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/84395214 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4027070 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/84395214 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb3b4011440 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b4033070 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/84395214 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb3b4044040 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3cf037740 10 client.26269 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.562+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ed40f4bf0 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/84395214 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b40099d0 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/84395214 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7fb3b400ebd0 con 0x559ed4077540 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3cf037740 10 client.26269 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3cf037740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3cf037740 20 client.26269 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3cf037740 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed40f5480 0x559ed40f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3cf037740 4 client.26269 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:13:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb3b4033070 con 0x559ed40f5480 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.563+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed40f5480 0x559ed40f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.564+0000 7fb3c9fe9640 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed40f5480 0x559ed40f78a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12166fbeb5d8c3 server_cookie=2c776efff853c6e9 in_seq=0 out_seq=0 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.564+0000 7fb3ad7fa640 10 client.26269 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.565+0000 7fb3ad7fa640 1 -- 172.21.15.64:0/84395214 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb3b4033070 con 0x559ed40f5480 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.565+0000 7fb3ad7fa640 10 client.26269 handle_command_reply: tid=0 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.565+0000 7fb3cf037740 1 client.26269 shutdown 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 msgr2=0x559ed4087b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4087b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4013680 tx=0x7fb3b4005e80 comp rx=0 tx=0).stop 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 shutdown_connections 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559ed40f5480 0x559ed40f78a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ed407e020 0x559ed40890b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559ed4078eb0 0x559ed40880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3ad7fa640 0 client.26269 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 --2- 172.21.15.64:0/84395214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ed4077540 0x559ed4087b60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 >> 172.21.15.64:0/84395214 conn(0x559ed40400a0 msgr2=0x559ed4094d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.566+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 shutdown_connections 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.567+0000 7fb3cf037740 1 -- 172.21.15.64:0/84395214 wait complete. 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.567+0000 7fb3cf037740 20 client.26269 trim_cache size 0 max 16384 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "c349956a-ecca-407d-9a37-398bb2cb1e84", 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:13:04.573 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb3b00a2c90 msgr2=0x7fb3b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb3b00a2c90 0x7fb3b00a5150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0bd0c0 tx=0x7fb3bc02a6c0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 msgr2=0x7fb3c41ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c41ade80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc030850 tx=0x7fb3bc030640 comp rx=0 tx=0).stop 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 shutdown_connections 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb3b00a2c90 0x7fb3b00a5150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.568+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c410db70 0x7fb3c41ade80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.569+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3c410b1a0 0x7fb3c41a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.569+0000 7fb3cc24d640 1 --2- 172.21.15.64:0/3864111972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3c4108830 0x7fb3c41a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.569+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 >> 172.21.15.64:0/3864111972 conn(0x7fb3c406e6c0 msgr2=0x7fb3c410c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.570+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 shutdown_connections 2024-05-09T09:13:04.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.571+0000 7fb3cc24d640 1 -- 172.21.15.64:0/3864111972 wait complete. 2024-05-09T09:13:04.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:13:04.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:13:04.593 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:04 smithi064 ceph-mon[30149]: pgmap v488: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-05-09T09:13:04.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:04 smithi179 ceph-mon[34086]: pgmap v488: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-05-09T09:13:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.789+0000 7f3e6b200640 1 -- 172.21.15.64:0/2925016452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640ffeb0 msgr2=0x7f3e641002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.789+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2925016452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640ffeb0 0x7f3e641002d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00aa10 tx=0x7f3e4c02b720 comp rx=0 tx=0).stop 2024-05-09T09:13:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.790+0000 7f3e6b200640 1 -- 172.21.15.64:0/2925016452 shutdown_connections 2024-05-09T09:13:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.790+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2925016452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e64101a50 0x7f3e64108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.790+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2925016452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e64101090 0x7f3e64101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.790+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2925016452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640ffeb0 0x7f3e641002d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.790+0000 7f3e6b200640 1 -- 172.21.15.64:0/2925016452 >> 172.21.15.64:0/2925016452 conn(0x7f3e640fb470 msgr2=0x7f3e640fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.792+0000 7f3e6b200640 1 -- 172.21.15.64:0/2925016452 shutdown_connections 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.793+0000 7f3e6b200640 1 -- 172.21.15.64:0/2925016452 wait complete. 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.794+0000 7f3e6b200640 1 Processor -- start 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.794+0000 7f3e6b200640 1 -- start start 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.794+0000 7f3e6b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e640ffeb0 0x7f3e641958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.794+0000 7f3e6b200640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e64101090 0x7f3e64195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e6b200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e64101a50 0x7f3e641ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e6b200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410b1e0 con 0x7f3e64101a50 2024-05-09T09:13:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e6b200640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410b060 con 0x7f3e64101090 2024-05-09T09:13:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e6b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410b360 con 0x7f3e640ffeb0 2024-05-09T09:13:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e64101090 0x7f3e64195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e640ffeb0 0x7f3e641958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e640ffeb0 0x7f3e641958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38514/0 (socket says 172.21.15.64:38514) 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e68f9c640 1 -- 172.21.15.64:0/2899093669 learned_addr learned my addr 172.21.15.64:0/2899093669 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.795+0000 7f3e6979d640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e64101a50 0x7f3e641ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.796+0000 7f3e63fff640 1 -- 172.21.15.64:0/2899093669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e640ffeb0 msgr2=0x7f3e641958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.796+0000 7f3e63fff640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e640ffeb0 0x7f3e641958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.796+0000 7f3e63fff640 1 -- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e64101a50 msgr2=0x7f3e641ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.796+0000 7f3e63fff640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e64101a50 0x7f3e641ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.796+0000 7f3e63fff640 1 -- 172.21.15.64:0/2899093669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6406b250 con 0x7f3e64101090 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.797+0000 7f3e63fff640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e64101090 0x7f3e64195e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00cc40 tx=0x7f3e5c00a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.797+0000 7f3e61ffb640 1 -- 172.21.15.64:0/2899093669 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003b70 con 0x7f3e64101090 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.797+0000 7f3e61ffb640 1 -- 172.21.15.64:0/2899093669 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e5c00ce50 con 0x7f3e64101090 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.797+0000 7f3e61ffb640 1 -- 172.21.15.64:0/2899093669 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00f7f0 con 0x7f3e64101090 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.797+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6406b540 con 0x7f3e64101090 2024-05-09T09:13:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.797+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6406ba40 con 0x7f3e64101090 2024-05-09T09:13:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.801+0000 7f3e6dfef740 1 Processor -- start 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 -- start start 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523e0920 0x561d523e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e5a90 0x561d523e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d52344d00 con 0x561d523e0920 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d522ec4c0 con 0x561d523e5a90 2024-05-09T09:13:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561d52340cc0 con 0x561d523dff30 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6979d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6979d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38520/0 (socket says 172.21.15.64:38520) 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.803+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523e0920 0x561d523e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.802+0000 7f3e6979d640 1 -- 172.21.15.64:0/264465097 learned_addr learned my addr 172.21.15.64:0/264465097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.803+0000 7f3e6979d640 1 -- 172.21.15.64:0/264465097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e5a90 msgr2=0x561d523e7f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.803+0000 7f3e6979d640 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e5a90 0x561d523e7f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.803+0000 7f3e6979d640 1 -- 172.21.15.64:0/264465097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523e0920 msgr2=0x561d523e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.803+0000 7f3e6979d640 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523e0920 0x561d523e5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.803+0000 7f3e6979d640 1 -- 172.21.15.64:0/264465097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d521227d0 con 0x561d523dff30 2024-05-09T09:13:04.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.804+0000 7f3e6979d640 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e0350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b6b0 tx=0x7f3e5400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad11d579a767a056 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.804+0000 7f3e5affd640 1 -- 172.21.15.64:0/264465097 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54003bd0 con 0x561d523dff30 2024-05-09T09:13:04.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.804+0000 7f3e5affd640 1 -- 172.21.15.64:0/264465097 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e54015070 con 0x561d523dff30 2024-05-09T09:13:04.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.804+0000 7f3e5affd640 1 -- 172.21.15.64:0/264465097 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540116a0 con 0x561d523dff30 2024-05-09T09:13:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 -- 172.21.15.64:0/264465097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 msgr2=0x561d523e0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e0350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b6b0 tx=0x7f3e5400bb70 comp rx=0 tx=0).stop 2024-05-09T09:13:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e61ffb640 1 -- 172.21.15.64:0/2899093669 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e5c00f990 con 0x7f3e64101090 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 -- 172.21.15.64:0/264465097 shutdown_connections 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e5a90 0x561d523e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523e0920 0x561d523e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/264465097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e0350 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.805+0000 7f3e6dfef740 1 -- 172.21.15.64:0/264465097 >> 172.21.15.64:0/264465097 conn(0x561d523db5a0 msgr2=0x561d523db9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.806+0000 7f3e6dfef740 1 -- 172.21.15.64:0/264465097 shutdown_connections 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.806+0000 7f3e6dfef740 1 -- 172.21.15.64:0/264465097 wait complete. 2024-05-09T09:13:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.806+0000 7f3e6dfef740 1 -- start start 2024-05-09T09:13:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.806+0000 7f3e6dfef740 1 Processor -- start 2024-05-09T09:13:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.807+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d523ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.807+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d523eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.807+0000 7f3e61ffb640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e500a2c20 0x7f3e500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d523eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e61ffb640 1 -- 172.21.15.64:0/2899093669 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e5c0cf3c0 con 0x7f3e64101090 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d523eaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38524/0 (socket says 172.21.15.64:38524) 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d523f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d523e8ae0 con 0x561d523dff30 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d522ffb90 con 0x561d523efb20 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561d523e8c60 con 0x561d523e0920 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e6dfef740 10 client.0 _ll_register_callbacks cb 0x561d5233bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 -- 172.21.15.64:0/59222913 learned_addr learned my addr 172.21.15.64:0/59222913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 -- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 msgr2=0x561d523f1f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:13:04.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d523f1f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 -- 172.21.15.64:0/59222913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 msgr2=0x561d523ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d523ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.808+0000 7f3e68f9c640 1 -- 172.21.15.64:0/59222913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d520583f0 con 0x561d523e0920 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d523eaaf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00a960 tx=0x7f3e4c0044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d31202a93a9ad07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e63fff640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e500a2c20 0x7f3e500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e43fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e43fff640 1 -- 172.21.15.64:0/59222913 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c036040 con 0x561d523e0920 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e43fff640 1 -- 172.21.15.64:0/59222913 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e4c030b60 con 0x561d523e0920 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e54015070 con 0x561d523e0920 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e43fff640 1 -- 172.21.15.64:0/59222913 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c030d60 con 0x561d523e0920 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.809+0000 7f3e6dfef740 10 client.26281 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561d520180f0 con 0x561d523e0920 2024-05-09T09:13:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e43fff640 1 -- 172.21.15.64:0/59222913 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e54015070 con 0x561d523e0920 2024-05-09T09:13:04.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e43fff640 1 -- 172.21.15.64:0/59222913 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f3e4c00b040 con 0x561d523e0920 2024-05-09T09:13:04.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 10 client.26281 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:04.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:04.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 20 client.26281 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:04.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d523f6d00 0x561d523f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 4 client.26281 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.810+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d52352430 con 0x561d523f6d00 2024-05-09T09:13:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.811+0000 7f3e6979d640 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d523f6d00 0x561d523f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.811+0000 7f3e63fff640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e500a2c20 0x7f3e500a50e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c09b8d0 tx=0x7f3e5c0047a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.811+0000 7f3e6979d640 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d523f6d00 0x561d523f9180 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0a82b059ec8733 server_cookie=b2225e63143f38e in_seq=0 out_seq=0 2024-05-09T09:13:04.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.812+0000 7f3e43fff640 10 client.26281 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.812+0000 7f3e43fff640 1 -- 172.21.15.64:0/59222913 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d52352430 con 0x561d523f6d00 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.812+0000 7f3e43fff640 10 client.26281 handle_command_reply: tid=0 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.812+0000 7f3e6dfef740 1 client.26281 shutdown 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.812+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 msgr2=0x561d523eaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.812+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d523eaaf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00a960 tx=0x7f3e4c0044f0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 shutdown_connections 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d523f6d00 0x561d523f9180 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d523f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d523eaaf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/59222913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d523ea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 >> 172.21.15.64:0/59222913 conn(0x561d523db5a0 msgr2=0x561d523da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e43fff640 0 client.26281 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 shutdown_connections 2024-05-09T09:13:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.813+0000 7f3e6dfef740 1 -- 172.21.15.64:0/59222913 wait complete. 2024-05-09T09:13:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.814+0000 7f3e6dfef740 20 client.26281 trim_cache size 0 max 16384 2024-05-09T09:13:04.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:04 smithi139 ceph-mon[33599]: pgmap v488: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-05-09T09:13:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.847+0000 7f3e6dfef740 1 Processor -- start 2024-05-09T09:13:04.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.847+0000 7f3e6dfef740 1 -- start start 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e0920 0x561d523e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523efb20 0x561d523ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38528/0 (socket says 172.21.15.64:38528) 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 -- 172.21.15.64:0/1032224131 learned_addr learned my addr 172.21.15.64:0/1032224131 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641c42e0 con 0x561d523efb20 2024-05-09T09:13:04.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641c1b50 con 0x561d523e0920 2024-05-09T09:13:04.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641bf9a0 con 0x561d523dff30 2024-05-09T09:13:04.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e0920 0x561d523e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523efb20 0x561d523ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 -- 172.21.15.64:0/1032224131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e0920 msgr2=0x561d523e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e0920 0x561d523e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 -- 172.21.15.64:0/1032224131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523efb20 msgr2=0x561d523ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523efb20 0x561d523ea5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.848+0000 7f3e6979d640 1 -- 172.21.15.64:0/1032224131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d523e51a0 con 0x561d523dff30 2024-05-09T09:13:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.849+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e1ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3e540045c0 tx=0x7f3e54004120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d68e35854a8235a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.850+0000 7f3e5a7fc640 1 -- 172.21.15.64:0/1032224131 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54005020 con 0x561d523dff30 2024-05-09T09:13:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.850+0000 7f3e5a7fc640 1 -- 172.21.15.64:0/1032224131 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e5401f030 con 0x561d523dff30 2024-05-09T09:13:04.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.850+0000 7f3e5a7fc640 1 -- 172.21.15.64:0/1032224131 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540048e0 con 0x561d523dff30 2024-05-09T09:13:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.851+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 msgr2=0x561d523e1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.851+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e1ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3e540045c0 tx=0x7f3e54004120 comp rx=0 tx=0).stop 2024-05-09T09:13:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 shutdown_connections 2024-05-09T09:13:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523efb20 0x561d523ea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523e0920 0x561d523e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1032224131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523dff30 0x561d523e1ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 >> 172.21.15.64:0/1032224131 conn(0x561d523db5a0 msgr2=0x561d523e11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 shutdown_connections 2024-05-09T09:13:04.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1032224131 wait complete. 2024-05-09T09:13:04.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.852+0000 7f3e6dfef740 1 -- start start 2024-05-09T09:13:04.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.853+0000 7f3e6dfef740 1 Processor -- start 2024-05-09T09:13:04.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.853+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d524072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.853+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d524077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.853+0000 7f3e6dfef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d5245add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641c0010 con 0x561d523dff30 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641c1480 con 0x561d523efb20 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e6dfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641c06e0 con 0x561d523e0920 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e6dfef740 10 client.0 _ll_register_callbacks cb 0x7f3e4c00a750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d524077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d524077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38542/0 (socket says 172.21.15.64:38542) 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 -- 172.21.15.64:0/1605594837 learned_addr learned my addr 172.21.15.64:0/1605594837 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d5245add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d524072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 -- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 msgr2=0x561d5245add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d5245add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 -- 172.21.15.64:0/1605594837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 msgr2=0x561d524072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d524072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.854+0000 7f3e68f9c640 1 -- 172.21.15.64:0/1605594837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5240a780 con 0x561d523e0920 2024-05-09T09:13:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.855+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d5245add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.855+0000 7f3e68f9c640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d524077e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c02b760 tx=0x7f3e4c030f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbee6ee4d5ae0b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.856+0000 7f3e43fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.856+0000 7f3e43fff640 1 -- 172.21.15.64:0/1605594837 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c057020 con 0x561d523e0920 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.856+0000 7f3e43fff640 1 -- 172.21.15.64:0/1605594837 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e4c03ebc0 con 0x561d523e0920 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.856+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5401f030 con 0x561d523e0920 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.857+0000 7f3e43fff640 1 -- 172.21.15.64:0/1605594837 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c062050 con 0x561d523e0920 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.857+0000 7f3e6dfef740 10 client.26287 fetch_fsmap learned FSMap version 35 2024-05-09T09:13:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.857+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561d5245c290 con 0x561d523e0920 2024-05-09T09:13:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e43fff640 1 -- 172.21.15.64:0/1605594837 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e4c042070 con 0x561d523e0920 2024-05-09T09:13:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e43fff640 1 -- 172.21.15.64:0/1605594837 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f3e4c05e4b0 con 0x561d523e0920 2024-05-09T09:13:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e6dfef740 10 client.26287 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e6dfef740 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e6dfef740 20 client.26287 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d5245ccc0 0x561d5245f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e6dfef740 4 client.26287 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.858+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3e5401f030 con 0x561d5245ccc0 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.859+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d5245ccc0 0x561d5245f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.860+0000 7f3e43fff640 10 client.26287 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.860+0000 7f3e6979d640 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d5245ccc0 0x561d5245f0e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7c7b58c31bfcb7 server_cookie=719814a728892a09 in_seq=0 out_seq=0 2024-05-09T09:13:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.861+0000 7f3e43fff640 1 -- 172.21.15.64:0/1605594837 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f3e5401f030 con 0x561d5245ccc0 2024-05-09T09:13:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.861+0000 7f3e43fff640 10 client.26287 handle_command_reply: tid=0 2024-05-09T09:13:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.861+0000 7f3e6dfef740 1 client.26287 shutdown 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.861+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 msgr2=0x561d524077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.861+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d524077e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c02b760 tx=0x7f3e4c030f80 comp rx=0 tx=0).stop 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 shutdown_connections 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d5245ccc0 0x561d5245f0e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d523efb20 0x561d5245add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e43fff640 0 client.26287 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d523e0920 0x561d524077e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e6dfef740 1 --2- 172.21.15.64:0/1605594837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d523dff30 0x561d524072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.862+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 >> 172.21.15.64:0/1605594837 conn(0x561d523db5a0 msgr2=0x561d523e1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.863+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 shutdown_connections 2024-05-09T09:13:04.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.863+0000 7f3e6dfef740 1 -- 172.21.15.64:0/1605594837 wait complete. 2024-05-09T09:13:04.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.864+0000 7f3e6dfef740 20 client.26287 trim_cache size 0 max 16384 2024-05-09T09:13:04.863 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:13:04.863 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: "c349956a-ecca-407d-9a37-398bb2cb1e84": { 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "c349956a-ecca-407d-9a37-398bb2cb1e84", 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:13:04.864 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:13:04.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e500a2c20 msgr2=0x7f3e500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e500a2c20 0x7f3e500a50e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c09b8d0 tx=0x7f3e5c0047a0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e64101090 msgr2=0x7f3e64195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e64101090 0x7f3e64195e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00cc40 tx=0x7f3e5c00a620 comp rx=0 tx=0).stop 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 shutdown_connections 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e500a2c20 0x7f3e500a50e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e64101a50 0x7f3e641ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e64101090 0x7f3e64195e00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.866+0000 7f3e6b200640 1 --2- 172.21.15.64:0/2899093669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e640ffeb0 0x7f3e641958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.867+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 >> 172.21.15.64:0/2899093669 conn(0x7f3e640fb470 msgr2=0x7f3e641043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:04.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.867+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 shutdown_connections 2024-05-09T09:13:04.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:04.867+0000 7f3e6b200640 1 -- 172.21.15.64:0/2899093669 wait complete. 2024-05-09T09:13:04.891 INFO:tasks.cephfs.filesystem:scrub status for tag:c349956a-ecca-407d-9a37-398bb2cb1e84 - {'path': '/', 'tag': 'c349956a-ecca-407d-9a37-398bb2cb1e84', 'options': 'recursive,force'} 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:05 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:13:05.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:05 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:13:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:05.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:05.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:05.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:05 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 165.0 s, 981.8 tps, lat 101.837 ms stddev 33.516 2024-05-09T09:13:06.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:06 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:06.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:06 smithi179 ceph-mon[34086]: pgmap v489: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-05-09T09:13:06.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:06 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:06.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:06 smithi139 ceph-mon[33599]: pgmap v489: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-05-09T09:13:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:06 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:06 smithi064 ceph-mon[30149]: pgmap v489: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-05-09T09:13:08.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:08 smithi179 ceph-mon[34086]: pgmap v490: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:13:08.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:08 smithi139 ceph-mon[33599]: pgmap v490: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:13:08.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:08 smithi064 ceph-mon[30149]: pgmap v490: 129 pgs: 129 active+clean; 1013 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:13:10.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:10 smithi179 ceph-mon[34086]: pgmap v491: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-05-09T09:13:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:13:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:13:10.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:10 smithi139 ceph-mon[33599]: pgmap v491: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-05-09T09:13:10.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:13:10.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:13:10.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:10 smithi064 ceph-mon[30149]: pgmap v491: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-05-09T09:13:10.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:13:10.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:13:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.277+0000 7f1fe1130640 1 -- 172.21.15.64:0/1208237369 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4031880 con 0x7f1fdc10db50 2024-05-09T09:13:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 -- 172.21.15.64:0/1208237369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 msgr2=0x7f1fdc10dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 --2- 172.21.15.64:0/1208237369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 0x7f1fdc10dfd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009900 tx=0x7f1fd402a3f0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 -- 172.21.15.64:0/1208237369 shutdown_connections 2024-05-09T09:13:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 --2- 172.21.15.64:0/1208237369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 0x7f1fdc10dfd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 --2- 172.21.15.64:0/1208237369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 --2- 172.21.15.64:0/1208237369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc108830 0x7f1fdc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 -- 172.21.15.64:0/1208237369 >> 172.21.15.64:0/1208237369 conn(0x7f1fdc06e8a0 msgr2=0x7f1fdc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 -- 172.21.15.64:0/1208237369 shutdown_connections 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.278+0000 7f1fe4396640 1 -- 172.21.15.64:0/1208237369 wait complete. 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 Processor -- start 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 -- start start 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc108830 0x7f1fdc1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc1a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 0x7f1fdc1adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc11c220 con 0x7f1fdc108830 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc11c0a0 con 0x7f1fdc10db50 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.279+0000 7f1fe4396640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc11c3a0 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc1a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc1a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38562/0 (socket says 172.21.15.64:38562) 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 -- 172.21.15.64:0/50840139 learned_addr learned my addr 172.21.15.64:0/50840139 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc108830 0x7f1fdc1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 -- 172.21.15.64:0/50840139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 msgr2=0x7f1fdc1adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 0x7f1fdc1adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 -- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc108830 msgr2=0x7f1fdc1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc108830 0x7f1fdc1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.280+0000 7f1fe1931640 1 -- 172.21.15.64:0/50840139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc1ae2f0 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.281+0000 7f1fe1931640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc1a6c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000cd00 tx=0x7f1fd000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.281+0000 7f1fdb7fe640 1 -- 172.21.15.64:0/50840139 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0003b70 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.282+0000 7f1fdb7fe640 1 -- 172.21.15.64:0/50840139 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd0003d10 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.282+0000 7f1fdb7fe640 1 -- 172.21.15.64:0/50840139 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0003b70 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.282+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc1ae5e0 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.282+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc1aeae0 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.286+0000 7f1fe7176740 1 Processor -- start 2024-05-09T09:13:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.289+0000 7f1fdb7fe640 1 -- 172.21.15.64:0/50840139 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1fd0003ec0 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.289+0000 7f1fe7176740 1 -- start start 2024-05-09T09:13:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.290+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34250 0x564b3fd34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.290+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd39740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.290+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd3c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.290+0000 7f1fe7176740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564b3fc98340 con 0x564b3fd34250 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.290+0000 7f1fe7176740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b3fc40290 con 0x564b3fd34c40 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.290+0000 7f1fe7176740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564b3fc94300 con 0x564b3fd39db0 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.291+0000 7f1fe1931640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd3c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.291+0000 7f1fdb7fe640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fcc0a2f10 0x7f1fcc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.291+0000 7f1fe1931640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd3c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38574/0 (socket says 172.21.15.64:38574) 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.291+0000 7f1fe1931640 1 -- 172.21.15.64:0/2216860419 learned_addr learned my addr 172.21.15.64:0/2216860419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.291+0000 7f1fdb7fe640 1 -- 172.21.15.64:0/50840139 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1fd00cf560 con 0x7f1fdc10b1a0 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.292+0000 7f1fe1931640 1 -- 172.21.15.64:0/2216860419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 msgr2=0x564b3fd39740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.292+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd39740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.292+0000 7f1fe1931640 1 -- 172.21.15.64:0/2216860419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34250 msgr2=0x564b3fd34670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.292+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34250 0x564b3fd34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.292+0000 7f1fe1931640 1 -- 172.21.15.64:0/2216860419 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3fc648e0 con 0x564b3fd39db0 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.292+0000 7f1fe2933640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fcc0a2f10 0x7f1fcc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.293+0000 7f1fe2933640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fcc0a2f10 0x7f1fcc0a53d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009850 tx=0x7f1fd4009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.295+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd3c240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0007a80 tx=0x7f1fd009b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3191ce1f70a069c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.297+0000 7f1fd8bfe640 1 -- 172.21.15.64:0/2216860419 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd001d030 con 0x564b3fd39db0 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.297+0000 7f1fd8bfe640 1 -- 172.21.15.64:0/2216860419 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd009b9d0 con 0x564b3fd39db0 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.297+0000 7f1fd8bfe640 1 -- 172.21.15.64:0/2216860419 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0099d60 con 0x564b3fd39db0 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 -- 172.21.15.64:0/2216860419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 msgr2=0x564b3fd3c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd3c240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0007a80 tx=0x7f1fd009b860 comp rx=0 tx=0).stop 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 -- 172.21.15.64:0/2216860419 shutdown_connections 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd3c240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd39740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2216860419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34250 0x564b3fd34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.298+0000 7f1fe7176740 1 -- 172.21.15.64:0/2216860419 >> 172.21.15.64:0/2216860419 conn(0x564b3fd2f8c0 msgr2=0x564b3fd2fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.301+0000 7f1fe7176740 1 -- 172.21.15.64:0/2216860419 shutdown_connections 2024-05-09T09:13:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.302+0000 7f1fe7176740 1 -- 172.21.15.64:0/2216860419 wait complete. 2024-05-09T09:13:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.302+0000 7f1fe7176740 1 -- start start 2024-05-09T09:13:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.302+0000 7f1fe7176740 1 Processor -- start 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 0x564b3fd3e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fd3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564b3fd3cd60 con 0x564b3fd34c40 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b3fc53790 con 0x564b3fd34250 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564b3fd3cee0 con 0x564b3fd39db0 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.304+0000 7f1fe7176740 10 client.0 _ll_register_callbacks cb 0x564b3fc8ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 0x564b3fd3e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 0x564b3fd3e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:44576/0 (socket says 172.21.15.64:44576) 2024-05-09T09:13:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 -- 172.21.15.64:0/3540557832 learned_addr learned my addr 172.21.15.64:0/3540557832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 -- 172.21.15.64:0/3540557832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 msgr2=0x564b3fd45a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:13:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd45a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 -- 172.21.15.64:0/3540557832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 msgr2=0x564b3fd3ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fd3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.305+0000 7f1fe2132640 1 -- 172.21.15.64:0/3540557832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3f9ac210 con 0x564b3fd34250 2024-05-09T09:13:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.306+0000 7f1fe2132640 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 0x564b3fd3e4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800be00 tx=0x7f1fc80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f690ee0defb9c919 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.306+0000 7f1fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.307+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3540557832 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8011850 con 0x564b3fd34250 2024-05-09T09:13:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.307+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd009b9d0 con 0x564b3fd34250 2024-05-09T09:13:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.307+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3540557832 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1fc80119f0 con 0x564b3fd34250 2024-05-09T09:13:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.308+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3540557832 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1fc8025050 con 0x564b3fd34250 2024-05-09T09:13:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.308+0000 7f1fe7176740 10 client.26138 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.308+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b3fa78620 con 0x564b3fd34250 2024-05-09T09:13:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.308+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3540557832 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd009b9d0 con 0x564b3fd34250 2024-05-09T09:13:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3540557832 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f1fc801cc10 con 0x564b3fd34250 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fe7176740 10 client.26138 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fe7176740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fe7176740 20 client.26138 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd4a4a0 0x564b3fd4c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fe7176740 4 client.26138 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.309+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b3fca5a70 con 0x564b3fd4a4a0 2024-05-09T09:13:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.310+0000 7f1fe1931640 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd4a4a0 0x564b3fd4c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.310+0000 7f1fe1931640 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd4a4a0 0x564b3fd4c940 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c58b86a981dd1e server_cookie=62caf5fcea8cfe11 in_seq=0 out_seq=0 2024-05-09T09:13:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.311+0000 7f1fc5ffb640 10 client.26138 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.312+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3540557832 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b3fca5a70 con 0x564b3fd4a4a0 2024-05-09T09:13:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.312+0000 7f1fc5ffb640 10 client.26138 handle_command_reply: tid=0 2024-05-09T09:13:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.312+0000 7f1fe7176740 1 client.26138 shutdown 2024-05-09T09:13:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 msgr2=0x564b3fd3e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 0x564b3fd3e4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800be00 tx=0x7f1fc80107a0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 shutdown_connections 2024-05-09T09:13:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd4a4a0 0x564b3fd4c940 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd39db0 0x564b3fd45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fd3ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3540557832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34250 0x564b3fd3e4f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.315+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 >> 172.21.15.64:0/3540557832 conn(0x564b3fd2f8c0 msgr2=0x564b3fd2ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.316+0000 7f1fc5ffb640 0 client.26138 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.316+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 shutdown_connections 2024-05-09T09:13:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.317+0000 7f1fe7176740 1 -- 172.21.15.64:0/3540557832 wait complete. 2024-05-09T09:13:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.319+0000 7f1fe7176740 20 client.26138 trim_cache size 0 max 16384 2024-05-09T09:13:11.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 170.0 s, 958.8 tps, lat 103.997 ms stddev 116.926 2024-05-09T09:13:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.355+0000 7f1fe7176740 1 Processor -- start 2024-05-09T09:13:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.355+0000 7f1fe7176740 1 -- start start 2024-05-09T09:13:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd2f8c0 0x564b3fd2a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd32950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd2f8c0 0x564b3fd2a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd2f8c0 0x564b3fd2a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40326/0 (socket says 172.21.15.64:40326) 2024-05-09T09:13:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe2132640 1 -- 172.21.15.64:0/2382141482 learned_addr learned my addr 172.21.15.64:0/2382141482 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1bbaf0 con 0x564b3fd2f8c0 2024-05-09T09:13:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b9360 con 0x564b3fd34c40 2024-05-09T09:13:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b71b0 con 0x564b3fd34250 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe2933640 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd32950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.356+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.358+0000 7f1fe1931640 1 -- 172.21.15.64:0/2382141482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 msgr2=0x564b3fd32950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.358+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd32950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.358+0000 7f1fe1931640 1 -- 172.21.15.64:0/2382141482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd2f8c0 msgr2=0x564b3fd2a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.358+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd2f8c0 0x564b3fd2a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.358+0000 7f1fe1931640 1 -- 172.21.15.64:0/2382141482 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3fd35020 con 0x564b3fd34250 2024-05-09T09:13:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.358+0000 7f1fe1931640 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd45a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fd009c030 tx=0x7f1fd000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343405bdb5fb1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.359+0000 7f1fc7fff640 1 -- 172.21.15.64:0/2382141482 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd001d070 con 0x564b3fd34250 2024-05-09T09:13:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.359+0000 7f1fc7fff640 1 -- 172.21.15.64:0/2382141482 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd00a4640 con 0x564b3fd34250 2024-05-09T09:13:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.359+0000 7f1fc7fff640 1 -- 172.21.15.64:0/2382141482 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00a41b0 con 0x564b3fd34250 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.359+0000 7f1fe2933640 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd32950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 msgr2=0x564b3fd45a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd45a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fd009c030 tx=0x7f1fd000cc80 comp rx=0 tx=0).stop 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 shutdown_connections 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd34c40 0x564b3fd32950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd45a30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 --2- 172.21.15.64:0/2382141482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd2f8c0 0x564b3fd2a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.360+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 >> 172.21.15.64:0/2382141482 conn(0x564b3fd39880 msgr2=0x564b3fd2fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.361+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 shutdown_connections 2024-05-09T09:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.361+0000 7f1fe7176740 1 -- 172.21.15.64:0/2382141482 wait complete. 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.361+0000 7f1fe7176740 1 -- start start 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.361+0000 7f1fe7176740 1 Processor -- start 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 0x564b3fd426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe7176740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 0x564b3fd426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe2132640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 0x564b3fd426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:44594/0 (socket says 172.21.15.64:44594) 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe2132640 1 -- 172.21.15.64:0/3241741030 learned_addr learned my addr 172.21.15.64:0/3241741030 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fdaeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b7820 con 0x564b3fd34c40 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b8c90 con 0x564b3fd2f8c0 2024-05-09T09:13:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.362+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b7ef0 con 0x564b3fd34250 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.363+0000 7f1fe2933640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fdaeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.363+0000 7f1fe7176740 10 client.0 _ll_register_callbacks cb 0x7f1fc800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.363+0000 7f1fe1931640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fe2132640 1 -- 172.21.15.64:0/3241741030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 msgr2=0x564b3fd42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fe2132640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd42be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fe2132640 1 -- 172.21.15.64:0/3241741030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 msgr2=0x564b3fdaeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fe2132640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fdaeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fe2132640 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3fd45cb0 con 0x564b3fd2f8c0 2024-05-09T09:13:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fe2132640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 0x564b3fd426a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x564b3fd33a00 tx=0x7f1fc80052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78688288490358b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.364+0000 7f1fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.365+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3241741030 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc802f070 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.365+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3241741030 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1fc8011850 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd00a4640 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3241741030 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1fc8039050 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fe7176740 10 client.26141 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b3fdaff90 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3241741030 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fc8004640 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3241741030 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f1fc8028050 con 0x564b3fd2f8c0 2024-05-09T09:13:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fe7176740 10 client.26141 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fe7176740 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.366+0000 7f1fe7176740 20 client.26141 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.367+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd39db0 0x564b3fdb2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.367+0000 7f1fe7176740 4 client.26141 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.367+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1fd00a4640 con 0x564b3fd39db0 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.367+0000 7f1fe1931640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd39db0 0x564b3fdb2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.369+0000 7f1fe1931640 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd39db0 0x564b3fdb2950 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eabc66a04305c39 server_cookie=9f40d535b5dab282 in_seq=0 out_seq=0 2024-05-09T09:13:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.369+0000 7f1fc5ffb640 10 client.26141 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.388+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/3241741030 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1fd00a4640 con 0x564b3fd39db0 2024-05-09T09:13:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.388+0000 7f1fc5ffb640 10 client.26141 handle_command_reply: tid=0 2024-05-09T09:13:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.388+0000 7f1fe7176740 1 client.26141 shutdown 2024-05-09T09:13:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.389+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 msgr2=0x564b3fd426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.389+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 0x564b3fd426a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x564b3fd33a00 tx=0x7f1fc80052f0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 shutdown_connections 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564b3fd39db0 0x564b3fdb2950 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b3fd34c40 0x564b3fdaeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b3fd34250 0x564b3fd42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fe7176740 1 --2- 172.21.15.64:0/3241741030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b3fd2f8c0 0x564b3fd426a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 >> 172.21.15.64:0/3241741030 conn(0x564b3fd39880 msgr2=0x564b3fd390e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.391+0000 7f1fc5ffb640 0 client.26141 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.392+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 shutdown_connections 2024-05-09T09:13:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.392+0000 7f1fe7176740 1 -- 172.21.15.64:0/3241741030 wait complete. 2024-05-09T09:13:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.393+0000 7f1fe7176740 20 client.26141 trim_cache size 0 max 16384 2024-05-09T09:13:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:13:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:13:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:13:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:13:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.394+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fcc0a2f10 msgr2=0x7f1fcc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.394+0000 7f1fe4396640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fcc0a2f10 0x7f1fcc0a53d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009850 tx=0x7f1fd4009560 comp rx=0 tx=0).stop 2024-05-09T09:13:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.394+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 msgr2=0x7f1fdc1a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.394+0000 7f1fe4396640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc1a6c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000cd00 tx=0x7f1fd000a620 comp rx=0 tx=0).stop 2024-05-09T09:13:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 shutdown_connections 2024-05-09T09:13:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1fcc0a2f10 0x7f1fcc0a53d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fdc10db50 0x7f1fdc1adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fdc10b1a0 0x7f1fdc1a6c60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 --2- 172.21.15.64:0/50840139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc108830 0x7f1fdc1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 >> 172.21.15.64:0/50840139 conn(0x7f1fdc06e8a0 msgr2=0x7f1fdc109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 shutdown_connections 2024-05-09T09:13:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:11.395+0000 7f1fe4396640 1 -- 172.21.15.64:0/50840139 wait complete. 2024-05-09T09:13:11.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:11 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:13:11.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:11 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:11.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:11 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:13:11.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:11 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:11.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:11 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:13:11.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:11 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:12.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:12 smithi179 ceph-mon[34086]: pgmap v492: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.8 MiB/s wr, 1.01k op/s 2024-05-09T09:13:12.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:12 smithi139 ceph-mon[33599]: pgmap v492: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.8 MiB/s wr, 1.01k op/s 2024-05-09T09:13:12.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:12 smithi064 ceph-mon[30149]: pgmap v492: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.8 MiB/s wr, 1.01k op/s 2024-05-09T09:13:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:14 smithi179 ceph-mon[34086]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 1.11k op/s 2024-05-09T09:13:14.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:14 smithi139 ceph-mon[33599]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 1.11k op/s 2024-05-09T09:13:14.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:14 smithi064 ceph-mon[30149]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 1.11k op/s 2024-05-09T09:13:16.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 175.0 s, 1001.8 tps, lat 100.203 ms stddev 34.078 2024-05-09T09:13:16.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:16 smithi179 ceph-mon[34086]: pgmap v494: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.4 MiB/s wr, 1.03k op/s 2024-05-09T09:13:16.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:16 smithi139 ceph-mon[33599]: pgmap v494: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.4 MiB/s wr, 1.03k op/s 2024-05-09T09:13:16.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:16 smithi064 ceph-mon[30149]: pgmap v494: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.4 MiB/s wr, 1.03k op/s 2024-05-09T09:13:18.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:18 smithi179 ceph-mon[34086]: pgmap v495: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.5 MiB/s wr, 919 op/s 2024-05-09T09:13:18.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:18 smithi139 ceph-mon[33599]: pgmap v495: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.5 MiB/s wr, 919 op/s 2024-05-09T09:13:18.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:18 smithi064 ceph-mon[30149]: pgmap v495: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.5 MiB/s wr, 919 op/s 2024-05-09T09:13:20.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:20 smithi139 ceph-mon[33599]: pgmap v496: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:13:20.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:20 smithi064 ceph-mon[30149]: pgmap v496: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:13:20.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:20 smithi179 ceph-mon[34086]: pgmap v496: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:13:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 180.0 s, 985.1 tps, lat 101.422 ms stddev 35.166 2024-05-09T09:13:22.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:22 smithi139 ceph-mon[33599]: pgmap v497: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.6 MiB/s wr, 870 op/s 2024-05-09T09:13:22.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:22 smithi064 ceph-mon[30149]: pgmap v497: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.6 MiB/s wr, 870 op/s 2024-05-09T09:13:22.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:22 smithi179 ceph-mon[34086]: pgmap v497: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.6 MiB/s wr, 870 op/s 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 -- 172.21.15.64:0/1574441364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8109050 msgr2=0x7fb0b81094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1574441364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8109050 0x7fb0b81094d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009900 tx=0x7fb0a802a400 comp rx=0 tx=0).stop 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 -- 172.21.15.64:0/1574441364 shutdown_connections 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1574441364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 0x7fb0b8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1574441364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8109050 0x7fb0b81094d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1574441364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8107e70 0x7fb0b8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.555+0000 7fb0c0272640 1 -- 172.21.15.64:0/1574441364 >> 172.21.15.64:0/1574441364 conn(0x7fb0b8076e80 msgr2=0x7fb0b80792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.556+0000 7fb0c0272640 1 -- 172.21.15.64:0/1574441364 shutdown_connections 2024-05-09T09:13:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.556+0000 7fb0c0272640 1 -- 172.21.15.64:0/1574441364 wait complete. 2024-05-09T09:13:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 Processor -- start 2024-05-09T09:13:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 -- start start 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8107e70 0x7fb0b819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 0x7fb0b819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0be00e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8107e70 0x7fb0b819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0bd80d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 0x7fb0b819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0bd80d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 0x7fb0b819e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35640/0 (socket says 172.21.15.64:35640) 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 0x7fb0b81a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1837369030 learned_addr learned my addr 172.21.15.64:0/1837369030 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8113a50 con 0x7fb0b8109a10 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0be00e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8107e70 0x7fb0b819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:55482/0 (socket says 172.21.15.64:55482) 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81138d0 con 0x7fb0b8107e70 2024-05-09T09:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.557+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8113bd0 con 0x7fb0b8109050 2024-05-09T09:13:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0be80f640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 0x7fb0b81a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1837369030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8107e70 msgr2=0x7fb0b819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8107e70 0x7fb0b819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 msgr2=0x7fb0b81a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 0x7fb0b81a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1837369030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b81a5a30 con 0x7fb0b8109050 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0be80f640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 0x7fb0b81a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 0x7fb0b819e350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009850 tx=0x7fb0a8003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.558+0000 7fb0af7fe640 1 -- 172.21.15.64:0/1837369030 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a803f030 con 0x7fb0b8109050 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.559+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b81a5cc0 con 0x7fb0b8109050 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.559+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a61b0 con 0x7fb0b8109050 2024-05-09T09:13:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.559+0000 7fb0af7fe640 1 -- 172.21.15.64:0/1837369030 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a8032c60 con 0x7fb0b8109050 2024-05-09T09:13:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.559+0000 7fb0af7fe640 1 -- 172.21.15.64:0/1837369030 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8035070 con 0x7fb0b8109050 2024-05-09T09:13:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.563+0000 7fb0c2f7f740 1 Processor -- start 2024-05-09T09:13:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.564+0000 7fb0af7fe640 1 -- 172.21.15.64:0/1837369030 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0a8042030 con 0x7fb0b8109050 2024-05-09T09:13:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.564+0000 7fb0c2f7f740 1 -- start start 2024-05-09T09:13:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.564+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45158620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.564+0000 7fb0be80f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a4515d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4515dd60 0x560a451601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a450bc340 con 0x560a45158200 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a45064290 con 0x560a4515dd60 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a450b8300 con 0x560a45158bf0 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0be80f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56936/0 (socket says 172.21.15.64:56936) 2024-05-09T09:13:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0be80f640 1 -- 172.21.15.64:0/2845549521 learned_addr learned my addr 172.21.15.64:0/2845549521 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.565+0000 7fb0be00e640 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a4515d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be80f640 1 -- 172.21.15.64:0/2845549521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 msgr2=0x560a4515d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4515dd60 0x560a451601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be80f640 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a4515d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be80f640 1 -- 172.21.15.64:0/2845549521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4515dd60 msgr2=0x560a451601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be80f640 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4515dd60 0x560a451601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be80f640 1 -- 172.21.15.64:0/2845549521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a450888e0 con 0x560a45158200 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4515dd60 0x560a451601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0af7fe640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2d30 0x7fb0a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be00e640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2d30 0x7fb0a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0af7fe640 1 -- 172.21.15.64:0/1837369030 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0a80efb80 con 0x7fb0b8109050 2024-05-09T09:13:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.566+0000 7fb0be80f640 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45158620 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009ea0 tx=0x7fb0b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d482360611f8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.567+0000 7fb0be00e640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2d30 0x7fb0a40a51f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x560a45158ff0 tx=0x7fb0b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.567+0000 7fb0acbfe640 1 -- 172.21.15.64:0/2845549521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0004d20 con 0x560a45158200 2024-05-09T09:13:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.567+0000 7fb0acbfe640 1 -- 172.21.15.64:0/2845549521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b00054d0 con 0x560a45158200 2024-05-09T09:13:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.567+0000 7fb0acbfe640 1 -- 172.21.15.64:0/2845549521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00118a0 con 0x560a45158200 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2845549521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 msgr2=0x560a45158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45158620 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009ea0 tx=0x7fb0b0009f90 comp rx=0 tx=0).stop 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2845549521 shutdown_connections 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a4515dd60 0x560a451601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a4515d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2845549521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45158620 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.568+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2845549521 >> 172.21.15.64:0/2845549521 conn(0x560a45153870 msgr2=0x560a45153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.569+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2845549521 shutdown_connections 2024-05-09T09:13:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.569+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2845549521 wait complete. 2024-05-09T09:13:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.569+0000 7fb0c2f7f740 1 -- start start 2024-05-09T09:13:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.569+0000 7fb0c2f7f740 1 Processor -- start 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.570+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.570+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 0x560a45162a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4515dd60 0x560a45169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0bd80d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 0x560a45162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0bd80d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 0x560a45162a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:55508/0 (socket says 172.21.15.64:55508) 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0bd80d640 1 -- 172.21.15.64:0/2798273503 learned_addr learned my addr 172.21.15.64:0/2798273503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0be80f640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4515dd60 0x560a45169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a45160d60 con 0x560a45158200 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0be00e640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a45077790 con 0x560a45158bf0 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a45160ee0 con 0x560a4515dd60 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.571+0000 7fb0c2f7f740 10 client.0 _ll_register_callbacks cb 0x560a450b2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.572+0000 7fb0bd80d640 1 -- 172.21.15.64:0/2798273503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4515dd60 msgr2=0x560a45169ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.572+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4515dd60 0x560a45169ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.572+0000 7fb0bd80d640 1 -- 172.21.15.64:0/2798273503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 msgr2=0x560a45162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.572+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45162550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.572+0000 7fb0bd80d640 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a44dd0210 con 0x560a45158bf0 2024-05-09T09:13:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.573+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 0x560a45162a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b89c0 tx=0x7fb0a80bbbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23dcd9646148717 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.573+0000 7fb0a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.573+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/2798273503 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8049050 con 0x560a45158bf0 2024-05-09T09:13:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.573+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/2798273503 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0a80b8660 con 0x560a45158bf0 2024-05-09T09:13:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.573+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b00054d0 con 0x560a45158bf0 2024-05-09T09:13:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.576+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/2798273503 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb0a80bbff0 con 0x560a45158bf0 2024-05-09T09:13:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.576+0000 7fb0c2f7f740 10 client.26150 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.576+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560a44e9c620 con 0x560a45158bf0 2024-05-09T09:13:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.577+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/2798273503 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b00054d0 con 0x560a45158bf0 2024-05-09T09:13:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.577+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/2798273503 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fb0a80bc250 con 0x560a45158bf0 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.578+0000 7fb0c2f7f740 10 client.26150 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.578+0000 7fb0c2f7f740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.578+0000 7fb0c2f7f740 20 client.26150 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.578+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4516dd70 0x560a45170190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.578+0000 7fb0c2f7f740 4 client.26150 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.578+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a450c9a70 con 0x560a4516dd70 2024-05-09T09:13:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.579+0000 7fb0be00e640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4516dd70 0x560a45170190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.580+0000 7fb0be00e640 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4516dd70 0x560a45170190 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743ccbadb5cea9c6 server_cookie=1a70567720b881e in_seq=0 out_seq=0 2024-05-09T09:13:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.580+0000 7fb0a1ffb640 10 client.26150 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.581+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/2798273503 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a450c9a70 con 0x560a4516dd70 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.581+0000 7fb0a1ffb640 10 client.26150 handle_command_reply: tid=0 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.581+0000 7fb0c2f7f740 1 client.26150 shutdown 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.581+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 msgr2=0x560a45162a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.581+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 0x560a45162a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b89c0 tx=0x7fb0a80bbbb0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 shutdown_connections 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4516dd70 0x560a45170190 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a4515dd60 0x560a45169ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158bf0 0x560a45162a90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0a1ffb640 0 client.26150 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/2798273503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158200 0x560a45162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 >> 172.21.15.64:0/2798273503 conn(0x560a45153870 msgr2=0x560a451529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 shutdown_connections 2024-05-09T09:13:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.582+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/2798273503 wait complete. 2024-05-09T09:13:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.583+0000 7fb0c2f7f740 20 client.26150 trim_cache size 0 max 16384 2024-05-09T09:13:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 Processor -- start 2024-05-09T09:13:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 -- start start 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45153870 0x560a45159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45169a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158bf0 0x560a4515a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81b3140 con 0x560a45158bf0 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81b09b0 con 0x560a45158200 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.615+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ae800 con 0x560a45153870 2024-05-09T09:13:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.616+0000 7fb0bd80d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.616+0000 7fb0bd80d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45169a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:55632/0 (socket says 172.21.15.64:55632) 2024-05-09T09:13:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.616+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1052394651 learned_addr learned my addr 172.21.15.64:0/1052394651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.616+0000 7fb0be80f640 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158bf0 0x560a4515a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1052394651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45153870 msgr2=0x560a45159eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:13:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45153870 0x560a45159eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1052394651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158bf0 msgr2=0x560a4515a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158bf0 0x560a4515a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0bd80d640 1 -- 172.21.15.64:0/1052394651 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a45162de0 con 0x560a45158200 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0be80f640 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158bf0 0x560a4515a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45169a90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80bbf80 tx=0x7fb0a8009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7108aa5c1bd7ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.617+0000 7fb0a3fff640 1 -- 172.21.15.64:0/1052394651 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8049070 con 0x560a45158200 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.618+0000 7fb0a3fff640 1 -- 172.21.15.64:0/1052394651 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a80c03a0 con 0x560a45158200 2024-05-09T09:13:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.618+0000 7fb0a3fff640 1 -- 172.21.15.64:0/1052394651 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80b8660 con 0x560a45158200 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/1052394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 msgr2=0x560a45169a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45169a90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80bbf80 tx=0x7fb0a8009680 comp rx=0 tx=0).stop 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/1052394651 shutdown_connections 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45158bf0 0x560a4515a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45169a90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/1052394651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45153870 0x560a45159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/1052394651 >> 172.21.15.64:0/1052394651 conn(0x560a4515d860 msgr2=0x560a45153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/1052394651 shutdown_connections 2024-05-09T09:13:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.619+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/1052394651 wait complete. 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 -- start start 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 Processor -- start 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45153870 0x560a451673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45167920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a45168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81aee70 con 0x560a45153870 2024-05-09T09:13:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81b02e0 con 0x560a45158200 2024-05-09T09:13:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81af540 con 0x560a45158bf0 2024-05-09T09:13:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.620+0000 7fb0c2f7f740 10 client.0 _ll_register_callbacks cb 0x7fb0a8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a45168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a45168930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35036/0 (socket says 172.21.15.64:35036) 2024-05-09T09:13:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 -- 172.21.15.64:0/120145907 learned_addr learned my addr 172.21.15.64:0/120145907 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be00e640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45153870 0x560a451673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0bd80d640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45167920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 -- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 msgr2=0x560a45167920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45167920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 -- 172.21.15.64:0/120145907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45153870 msgr2=0x560a451673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45153870 0x560a451673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be80f640 1 -- 172.21.15.64:0/120145907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a45168f20 con 0x560a45158bf0 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.621+0000 7fb0be00e640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45153870 0x560a451673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.622+0000 7fb0be80f640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a45168930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0010010 tx=0x7fb0b0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263b7a8c7ce7fca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.622+0000 7fb0a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.622+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/120145907 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b002b020 con 0x560a45158bf0 2024-05-09T09:13:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.622+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a80c03a0 con 0x560a45158bf0 2024-05-09T09:13:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.622+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/120145907 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0b0005710 con 0x560a45158bf0 2024-05-09T09:13:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.622+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/120145907 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb0b0036050 con 0x560a45158bf0 2024-05-09T09:13:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.624+0000 7fb0c2f7f740 10 client.26314 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.624+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560a451d45c0 con 0x560a45158bf0 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/120145907 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a80c03a0 con 0x560a45158bf0 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/120145907 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fb0b0011440 con 0x560a45158bf0 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0c2f7f740 10 client.26314 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0c2f7f740 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0c2f7f740 20 client.26314 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4515dd60 0x560a451d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0c2f7f740 4 client.26314 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.625+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a45161990 con 0x560a4515dd60 2024-05-09T09:13:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.626+0000 7fb0be00e640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4515dd60 0x560a451d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.627+0000 7fb0be00e640 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4515dd60 0x560a451d6c40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4289f4f7543c7f3f server_cookie=8c7141c98d697c59 in_seq=0 out_seq=0 2024-05-09T09:13:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.627+0000 7fb0a1ffb640 10 client.26314 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.646+0000 7fb0a1ffb640 1 -- 172.21.15.64:0/120145907 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a45161990 con 0x560a4515dd60 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.646+0000 7fb0a1ffb640 10 client.26314 handle_command_reply: tid=0 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.646+0000 7fb0c2f7f740 1 client.26314 shutdown 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.646+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 msgr2=0x560a45168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.646+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a45168930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0010010 tx=0x7fb0b0000f40 comp rx=0 tx=0).stop 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 shutdown_connections 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a4515dd60 0x560a451d6c40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a45158bf0 0x560a45168930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a45158200 0x560a45167920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 --2- 172.21.15.64:0/120145907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a45153870 0x560a451673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 >> 172.21.15.64:0/120145907 conn(0x560a4515d860 msgr2=0x560a45159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0a1ffb640 0 client.26314 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 shutdown_connections 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 1 -- 172.21.15.64:0/120145907 wait complete. 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.647+0000 7fb0c2f7f740 20 client.26314 trim_cache size 0 max 16384 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:13:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.648+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2d30 msgr2=0x7fb0a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.648+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2d30 0x7fb0a40a51f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x560a45158ff0 tx=0x7fb0b400b390 comp rx=0 tx=0).stop 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.648+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 msgr2=0x7fb0b819e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.648+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 0x7fb0b819e350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009850 tx=0x7fb0a8003f80 comp rx=0 tx=0).stop 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.648+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 shutdown_connections 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.649+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0a40a2d30 0x7fb0a40a51f0 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x560a45158ff0 tx=0x7fb0b400b390 comp rx=0 tx=0).stop 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.649+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0b8109a10 0x7fb0b81a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.649+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0b8109050 0x7fb0b819e350 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.649+0000 7fb0c0272640 1 --2- 172.21.15.64:0/1837369030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0b8107e70 0x7fb0b819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.649+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 >> 172.21.15.64:0/1837369030 conn(0x7fb0b8076e80 msgr2=0x7fb0b810c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.649+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 shutdown_connections 2024-05-09T09:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:24.650+0000 7fb0c0272640 1 -- 172.21.15.64:0/1837369030 wait complete. 2024-05-09T09:13:24.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:24 smithi139 ceph-mon[33599]: pgmap v498: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 997 op/s 2024-05-09T09:13:24.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:24 smithi064 ceph-mon[30149]: pgmap v498: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 997 op/s 2024-05-09T09:13:24.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:24 smithi179 ceph-mon[34086]: pgmap v498: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 997 op/s 2024-05-09T09:13:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 185.0 s, 979.1 tps, lat 102.268 ms stddev 35.187 2024-05-09T09:13:26.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:26 smithi139 ceph-mon[33599]: pgmap v499: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 955 op/s 2024-05-09T09:13:26.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:26 smithi064 ceph-mon[30149]: pgmap v499: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 955 op/s 2024-05-09T09:13:26.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:26 smithi179 ceph-mon[34086]: pgmap v499: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 955 op/s 2024-05-09T09:13:28.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:28 smithi139 ceph-mon[33599]: pgmap v500: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 846 op/s 2024-05-09T09:13:28.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:28 smithi064 ceph-mon[30149]: pgmap v500: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 846 op/s 2024-05-09T09:13:28.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:28 smithi179 ceph-mon[34086]: pgmap v500: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 846 op/s 2024-05-09T09:13:29.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:29 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:13:29.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:29 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:13:29.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:29 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:13:30.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:30 smithi139 ceph-mon[33599]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 1.16k op/s 2024-05-09T09:13:30.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:13:30.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:13:30.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:13:30.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:30 smithi064 ceph-mon[30149]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 1.16k op/s 2024-05-09T09:13:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:13:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:13:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:13:30.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:30 smithi179 ceph-mon[34086]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 1.16k op/s 2024-05-09T09:13:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:13:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:13:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:13:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 190.0 s, 1014.2 tps, lat 98.517 ms stddev 36.805 2024-05-09T09:13:32.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:32 smithi139 ceph-mon[33599]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 906 op/s 2024-05-09T09:13:32.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:32 smithi064 ceph-mon[30149]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 906 op/s 2024-05-09T09:13:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:32 smithi179 ceph-mon[34086]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 906 op/s 2024-05-09T09:13:34.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:34 smithi139 ceph-mon[33599]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 1k op/s 2024-05-09T09:13:34.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:13:34.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:13:34.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:34 smithi064 ceph-mon[30149]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 1k op/s 2024-05-09T09:13:34.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:34 smithi179 ceph-mon[34086]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 1k op/s 2024-05-09T09:13:35.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 -- 172.21.15.64:0/1738875225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34107d70 msgr2=0x7f2e34108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1738875225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34107d70 0x7f2e34108190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009ca0 tx=0x7f2e2c02a8d0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 -- 172.21.15.64:0/1738875225 shutdown_connections 2024-05-09T09:13:35.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1738875225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1738875225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34108f60 0x7f2e341093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1738875225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34107d70 0x7f2e34108190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 -- 172.21.15.64:0/1738875225 >> 172.21.15.64:0/1738875225 conn(0x7f2e34076d00 msgr2=0x7f2e34079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 -- 172.21.15.64:0/1738875225 shutdown_connections 2024-05-09T09:13:35.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.082+0000 7f2e39aae640 1 -- 172.21.15.64:0/1738875225 wait complete. 2024-05-09T09:13:35.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 Processor -- start 2024-05-09T09:13:35.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 -- start start 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34107d70 0x7f2e3407f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34108f60 0x7f2e3407f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34113960 con 0x7f2e34108f60 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341137e0 con 0x7f2e34109920 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e39aae640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34113ae0 con 0x7f2e34107d70 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e392ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e392ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37074/0 (socket says 172.21.15.64:37074) 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e392ad640 1 -- 172.21.15.64:0/1636768449 learned_addr learned my addr 172.21.15.64:0/1636768449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.084+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34107d70 0x7f2e3407f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e33fff640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34108f60 0x7f2e3407f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e38aac640 1 -- 172.21.15.64:0/1636768449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 msgr2=0x7f2e341ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e38aac640 1 -- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34108f60 msgr2=0x7f2e3407f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34108f60 0x7f2e3407f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e38aac640 1 -- 172.21.15.64:0/1636768449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e341ae5b0 con 0x7f2e34107d70 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e392ad640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.085+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34107d70 0x7f2e3407f0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009bf0 tx=0x7f2e2c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.086+0000 7f2e31ffb640 1 -- 172.21.15.64:0/1636768449 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c004040 con 0x7f2e34107d70 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.086+0000 7f2e31ffb640 1 -- 172.21.15.64:0/1636768449 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e2c004690 con 0x7f2e34107d70 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.086+0000 7f2e31ffb640 1 -- 172.21.15.64:0/1636768449 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c030d60 con 0x7f2e34107d70 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.086+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e341ae780 con 0x7f2e34107d70 2024-05-09T09:13:35.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.087+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341aec70 con 0x7f2e34107d70 2024-05-09T09:13:35.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.091+0000 7f2e31ffb640 1 -- 172.21.15.64:0/1636768449 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e2c04e440 con 0x7f2e34107d70 2024-05-09T09:13:35.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.091+0000 7f2e3da1d740 1 Processor -- start 2024-05-09T09:13:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 -- start start 2024-05-09T09:13:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 0x564ecb347540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347b10 0x564ecb34c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb34cc80 0x564ecb34f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ecb2abd00 con 0x564ecb34cc80 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564ecb2534c0 con 0x564ecb347120 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.092+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ecb2a7cc0 con 0x564ecb347b10 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.093+0000 7f2e392ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 0x564ecb347540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.093+0000 7f2e392ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 0x564ecb347540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37086/0 (socket says 172.21.15.64:37086) 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.093+0000 7f2e392ad640 1 -- 172.21.15.64:0/3281692803 learned_addr learned my addr 172.21.15.64:0/3281692803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.093+0000 7f2e38aac640 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347b10 0x564ecb34c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.093+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb34cc80 0x564ecb34f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e33fff640 1 -- 172.21.15.64:0/3281692803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347b10 msgr2=0x564ecb34c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347b10 0x564ecb34c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e33fff640 1 -- 172.21.15.64:0/3281692803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 msgr2=0x564ecb347540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 0x564ecb347540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e33fff640 1 -- 172.21.15.64:0/3281692803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ecb0897d0 con 0x564ecb34cc80 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e392ad640 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 0x564ecb347540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e31ffb640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e1c0a2d90 0x7f2e1c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.094+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb34cc80 0x564ecb34f110 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006fd0 tx=0x7f2e28009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a754c9aec1c3511 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.095+0000 7f2e392ad640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e1c0a2d90 0x7f2e1c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.095+0000 7f2e31ffb640 1 -- 172.21.15.64:0/1636768449 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e2c0f15c0 con 0x7f2e34107d70 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.095+0000 7f2e1affd640 1 -- 172.21.15.64:0/3281692803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28017070 con 0x564ecb34cc80 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.096+0000 7f2e392ad640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e1c0a2d90 0x7f2e1c0a5250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x564ecb348370 tx=0x7f2e2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.096+0000 7f2e1affd640 1 -- 172.21.15.64:0/3281692803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e28013470 con 0x564ecb34cc80 2024-05-09T09:13:35.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.096+0000 7f2e1affd640 1 -- 172.21.15.64:0/3281692803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28017070 con 0x564ecb34cc80 2024-05-09T09:13:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3281692803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb34cc80 msgr2=0x564ecb34f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb34cc80 0x564ecb34f110 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006fd0 tx=0x7f2e28009e10 comp rx=0 tx=0).stop 2024-05-09T09:13:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3281692803 shutdown_connections 2024-05-09T09:13:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb34cc80 0x564ecb34f110 secure :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e28006fd0 tx=0x7f2e28009e10 comp rx=0 tx=0).stop 2024-05-09T09:13:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347b10 0x564ecb34c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3281692803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347120 0x564ecb347540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3281692803 >> 172.21.15.64:0/3281692803 conn(0x564ecb342790 msgr2=0x564ecb342b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3281692803 shutdown_connections 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.097+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3281692803 wait complete. 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 -- start start 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 Processor -- start 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347120 0x564ecb3516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 0x564ecb351c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb356c30 0x564ecb359020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ecb34fc30 con 0x564ecb347b10 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564ecb266b90 con 0x564ecb356c30 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ecb34fdb0 con 0x564ecb347120 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.098+0000 7f2e3da1d740 10 client.0 _ll_register_callbacks cb 0x564ecb2a2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 0x564ecb351c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 0x564ecb351c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33754/0 (socket says 172.21.15.64:33754) 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 -- 172.21.15.64:0/1002229447 learned_addr learned my addr 172.21.15.64:0/1002229447 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347120 0x564ecb3516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e392ad640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb356c30 0x564ecb359020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 -- 172.21.15.64:0/1002229447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347120 msgr2=0x564ecb3516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347120 0x564ecb3516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 -- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb356c30 msgr2=0x564ecb359020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb356c30 0x564ecb359020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 -- 172.21.15.64:0/1002229447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ecafbf3f0 con 0x564ecb347b10 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347120 0x564ecb3516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e33fff640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 0x564ecb351c00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000c00 tx=0x7f2e2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37557c0b3d3079e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.099+0000 7f2e13fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e13fff640 1 -- 172.21.15.64:0/1002229447 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28017070 con 0x564ecb347b10 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e13fff640 1 -- 172.21.15.64:0/1002229447 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e2800f040 con 0x564ecb347b10 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e28013470 con 0x564ecb347b10 2024-05-09T09:13:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e13fff640 1 -- 172.21.15.64:0/1002229447 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2e28033050 con 0x564ecb347b10 2024-05-09T09:13:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e3da1d740 10 client.16653 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ecaf7f0f0 con 0x564ecb347b10 2024-05-09T09:13:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.100+0000 7f2e13fff640 1 -- 172.21.15.64:0/1002229447 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e28013470 con 0x564ecb347b10 2024-05-09T09:13:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e13fff640 1 -- 172.21.15.64:0/1002229447 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f2e2802e030 con 0x564ecb347b10 2024-05-09T09:13:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e3da1d740 10 client.16653 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e3da1d740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e3da1d740 20 client.16653 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb35d5f0 0x564ecb35fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e3da1d740 4 client.16653 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.101+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ecb2b9430 con 0x564ecb35d5f0 2024-05-09T09:13:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.102+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb35d5f0 0x564ecb35fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.105+0000 7f2e13fff640 10 client.16653 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.105+0000 7f2e38aac640 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb35d5f0 0x564ecb35fa00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48d5a4c0613c45c server_cookie=37dffbb0448acf22 in_seq=0 out_seq=0 2024-05-09T09:13:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.106+0000 7f2e13fff640 1 -- 172.21.15.64:0/1002229447 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ecb2b9430 con 0x564ecb35d5f0 2024-05-09T09:13:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.106+0000 7f2e13fff640 10 client.16653 handle_command_reply: tid=0 2024-05-09T09:13:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.106+0000 7f2e3da1d740 1 client.16653 shutdown 2024-05-09T09:13:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 msgr2=0x564ecb351c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 0x564ecb351c00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000c00 tx=0x7f2e2800ecb0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 shutdown_connections 2024-05-09T09:13:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb35d5f0 0x564ecb35fa00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb356c30 0x564ecb359020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347b10 0x564ecb351c00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/1002229447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb347120 0x564ecb3516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 >> 172.21.15.64:0/1002229447 conn(0x564ecb342790 msgr2=0x564ecb341a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.107+0000 7f2e13fff640 0 client.16653 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.108+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 shutdown_connections 2024-05-09T09:13:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.109+0000 7f2e3da1d740 1 -- 172.21.15.64:0/1002229447 wait complete. 2024-05-09T09:13:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.109+0000 7f2e3da1d740 20 client.16653 trim_cache size 0 max 16384 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.145+0000 7f2e3da1d740 1 Processor -- start 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.145+0000 7f2e3da1d740 1 -- start start 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.145+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb33d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.145+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb3516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.145+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb351c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200124f0 con 0x564ecb347120 2024-05-09T09:13:35.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000fd60 con 0x564ecb347b10 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e3da1d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000dbb0 con 0x564ecb356c30 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e38aac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb33d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb3516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e38aac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb33d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33768/0 (socket says 172.21.15.64:33768) 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e38aac640 1 -- 172.21.15.64:0/3876942400 learned_addr learned my addr 172.21.15.64:0/3876942400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e392ad640 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb351c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 -- 172.21.15.64:0/3876942400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 msgr2=0x564ecb351c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb351c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 -- 172.21.15.64:0/3876942400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 msgr2=0x564ecb33d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb33d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 -- 172.21.15.64:0/3876942400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ecb333930 con 0x564ecb347b10 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e38aac640 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb33d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.146+0000 7f2e33fff640 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb3516c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000ea0 tx=0x7f2e280139b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e0e27826e59940 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.147+0000 7f2e1a7fc640 1 -- 172.21.15.64:0/3876942400 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28017040 con 0x564ecb347b10 2024-05-09T09:13:35.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.147+0000 7f2e1a7fc640 1 -- 172.21.15.64:0/3876942400 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e2800e5a0 con 0x564ecb347b10 2024-05-09T09:13:35.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.147+0000 7f2e1a7fc640 1 -- 172.21.15.64:0/3876942400 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28004040 con 0x564ecb347b10 2024-05-09T09:13:35.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3876942400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 msgr2=0x564ecb3516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb3516c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000ea0 tx=0x7f2e280139b0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3876942400 shutdown_connections 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb351c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb3516c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/3876942400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb33d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.148+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3876942400 >> 172.21.15.64:0/3876942400 conn(0x564ecb342790 msgr2=0x564ecb350180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.149+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3876942400 shutdown_connections 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.149+0000 7f2e3da1d740 1 -- 172.21.15.64:0/3876942400 wait complete. 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.149+0000 7f2e3da1d740 1 -- start start 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.149+0000 7f2e3da1d740 1 Processor -- start 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.150+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb36e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.150+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb36e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.150+0000 7f2e3da1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb36f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb36e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb36e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37110/0 (socket says 172.21.15.64:37110) 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e392ad640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb36f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e392ad640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb36f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53800/0 (socket says 172.21.15.64:53800) 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e38aac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb36e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e38aac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb36e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33770/0 (socket says 172.21.15.64:33770) 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e33fff640 1 -- 172.21.15.64:0/4186531315 learned_addr learned my addr 172.21.15.64:0/4186531315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000e220 con 0x564ecb347120 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000f690 con 0x564ecb347b10 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.151+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2000e8f0 con 0x564ecb356c30 2024-05-09T09:13:35.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e33fff640 1 -- 172.21.15.64:0/4186531315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 msgr2=0x564ecb36f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e33fff640 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb36f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e33fff640 1 -- 172.21.15.64:0/4186531315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 msgr2=0x564ecb36e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e33fff640 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb36e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e33fff640 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ecb36fe30 con 0x564ecb347b10 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e33fff640 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb36e830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e28010010 tx=0x7f2e280267f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0e1edbf3e3575a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e3da1d740 10 client.0 _ll_register_callbacks cb 0x7f2e2c0305d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2800e5a0 con 0x564ecb347b10 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e13fff640 10 client.26174 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.152+0000 7f2e13fff640 1 -- 172.21.15.64:0/4186531315 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28017070 con 0x564ecb347b10 2024-05-09T09:13:35.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.153+0000 7f2e13fff640 1 -- 172.21.15.64:0/4186531315 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2e28004040 con 0x564ecb347b10 2024-05-09T09:13:35.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.157+0000 7f2e13fff640 1 -- 172.21.15.64:0/4186531315 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2e28048050 con 0x564ecb347b10 2024-05-09T09:13:35.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.157+0000 7f2e3da1d740 10 client.26174 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:35.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.157+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=36}) -- 0x564ecb370db0 con 0x564ecb347b10 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e13fff640 1 -- 172.21.15.64:0/4186531315 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f2e2802e070 con 0x564ecb347b10 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e3da1d740 10 client.26174 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e3da1d740 10 client.26174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e3da1d740 20 client.26174 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb3c3960 0x564ecb3c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e3da1d740 4 client.26174 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ecb2b3780 con 0x564ecb3c3960 2024-05-09T09:13:35.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.158+0000 7f2e38aac640 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb3c3960 0x564ecb3c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.159+0000 7f2e38aac640 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb3c3960 0x564ecb3c5e20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2bf45778bd8ff95 server_cookie=801245e45f0e2ed1 in_seq=0 out_seq=0 2024-05-09T09:13:35.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.159+0000 7f2e13fff640 10 client.26174 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:35.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e13fff640 1 -- 172.21.15.64:0/4186531315 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564ecb2b3780 con 0x564ecb3c3960 2024-05-09T09:13:35.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e13fff640 10 client.26174 handle_command_reply: tid=0 2024-05-09T09:13:35.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 client.26174 shutdown 2024-05-09T09:13:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 msgr2=0x564ecb36e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb36e830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e28010010 tx=0x7f2e280267f0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 shutdown_connections 2024-05-09T09:13:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564ecb3c3960 0x564ecb3c5e20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ecb356c30 0x564ecb36f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ecb347b10 0x564ecb36e830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 --2- 172.21.15.64:0/4186531315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ecb347120 0x564ecb36e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.160+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 >> 172.21.15.64:0/4186531315 conn(0x564ecb342790 msgr2=0x564ecb33d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.161+0000 7f2e13fff640 0 client.26174 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.161+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 shutdown_connections 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.161+0000 7f2e3da1d740 1 -- 172.21.15.64:0/4186531315 wait complete. 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.162+0000 7f2e3da1d740 20 client.26174 trim_cache size 0 max 16384 2024-05-09T09:13:35.161 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:13:35.162 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:13:35.162 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:13:35.162 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:13:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e1c0a2d90 msgr2=0x7f2e1c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e1c0a2d90 0x7f2e1c0a5250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x564ecb348370 tx=0x7f2e2400b390 comp rx=0 tx=0).stop 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34107d70 msgr2=0x7f2e3407f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34107d70 0x7f2e3407f0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009bf0 tx=0x7f2e2c003ba0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 shutdown_connections 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2e1c0a2d90 0x7f2e1c0a5250 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e34109920 0x7f2e341ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e34108f60 0x7f2e3407f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 --2- 172.21.15.64:0/1636768449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e34107d70 0x7f2e3407f0a0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009bf0 tx=0x7f2e2c003ba0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.164+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 >> 172.21.15.64:0/1636768449 conn(0x7f2e34076d00 msgr2=0x7f2e3410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.168+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 shutdown_connections 2024-05-09T09:13:35.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.168+0000 7f2e39aae640 1 -- 172.21.15.64:0/1636768449 wait complete. 2024-05-09T09:13:35.189 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:13:35.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:13:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.428+0000 7f72b7361640 1 -- 172.21.15.64:0/1079257726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b010b1a0 msgr2=0x7f72b010d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.428+0000 7f72b7361640 1 --2- 172.21.15.64:0/1079257726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b010b1a0 0x7f72b010d630 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009900 tx=0x7f72a002a410 comp rx=0 tx=0).stop 2024-05-09T09:13:35.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.430+0000 7f72b7361640 1 -- 172.21.15.64:0/1079257726 shutdown_connections 2024-05-09T09:13:35.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.430+0000 7f72b7361640 1 --2- 172.21.15.64:0/1079257726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 0x7f72b010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.430+0000 7f72b7361640 1 --2- 172.21.15.64:0/1079257726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b010b1a0 0x7f72b010d630 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.430+0000 7f72b7361640 1 --2- 172.21.15.64:0/1079257726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72b0108830 0x7f72b010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.430+0000 7f72b7361640 1 -- 172.21.15.64:0/1079257726 >> 172.21.15.64:0/1079257726 conn(0x7f72b006e6d0 msgr2=0x7f72b0070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.430+0000 7f72b7361640 1 -- 172.21.15.64:0/1079257726 shutdown_connections 2024-05-09T09:13:35.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.431+0000 7f72b7361640 1 -- 172.21.15.64:0/1079257726 wait complete. 2024-05-09T09:13:35.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.432+0000 7f72b7361640 1 Processor -- start 2024-05-09T09:13:35.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.432+0000 7f72b7361640 1 -- start start 2024-05-09T09:13:35.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.432+0000 7f72b7361640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b0108830 0x7f72b01a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.432+0000 7f72b7361640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72b010b1a0 0x7f72b01a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b7361640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 0x7f72b01adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b50fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b0108830 0x7f72b01a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b50fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b0108830 0x7f72b01a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33788/0 (socket says 172.21.15.64:33788) 2024-05-09T09:13:35.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b50fd640 1 -- 172.21.15.64:0/2736964141 learned_addr learned my addr 172.21.15.64:0/2736964141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:35.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b48fc640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72b010b1a0 0x7f72b01a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72b011c230 con 0x7f72b0108830 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72b011c0b0 con 0x7f72b010db70 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72b011c3b0 con 0x7f72b010b1a0 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b58fe640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 0x7f72b01adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b58fe640 1 -- 172.21.15.64:0/2736964141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72b010b1a0 msgr2=0x7f72b01a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b58fe640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72b010b1a0 0x7f72b01a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b58fe640 1 -- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b0108830 msgr2=0x7f72b01a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b58fe640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b0108830 0x7f72b01a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.433+0000 7f72b58fe640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b01ae520 con 0x7f72b010db70 2024-05-09T09:13:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.434+0000 7f72b58fe640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 0x7f72b01adeb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72a800bdf0 tx=0x7f72a8009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.434+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8019070 con 0x7f72b010db70 2024-05-09T09:13:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.434+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72a80148f0 con 0x7f72b010db70 2024-05-09T09:13:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.434+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8012880 con 0x7f72b010db70 2024-05-09T09:13:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.435+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b01ae810 con 0x7f72b010db70 2024-05-09T09:13:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.435+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01aed60 con 0x7f72b010db70 2024-05-09T09:13:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.437+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b00711d0 con 0x7f72b010db70 2024-05-09T09:13:35.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.438+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72a801c070 con 0x7f72b010db70 2024-05-09T09:13:35.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.440+0000 7f72a67fc640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f729c0a2ce0 0x7f729c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:35.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.440+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72a80ceb80 con 0x7f72b010db70 2024-05-09T09:13:35.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.440+0000 7f72b50fd640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f729c0a2ce0 0x7f729c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:35.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.441+0000 7f72b50fd640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f729c0a2ce0 0x7f729c0a51a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00ac90 tx=0x7f72ac00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:35.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.445+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f72a809b100 con 0x7f72b010db70 2024-05-09T09:13:35.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.891+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72b0062430 con 0x7f72b010db70 2024-05-09T09:13:35.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.893+0000 7f72a67fc640 1 -- 172.21.15.64:0/2736964141 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11882 (secure 0 0 0) 0x7f72a809f400 con 0x7f72b010db70 2024-05-09T09:13:35.892 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:13:35.892 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:13:10.080621+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:13:35.893 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 36 2024-05-09T09:13:35.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.897+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f729c0a2ce0 msgr2=0x7f729c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.897+0000 7f72b7361640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f729c0a2ce0 0x7f729c0a51a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00ac90 tx=0x7f72ac00a380 comp rx=0 tx=0).stop 2024-05-09T09:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.897+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 msgr2=0x7f72b01adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.897+0000 7f72b7361640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 0x7f72b01adeb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72a800bdf0 tx=0x7f72a8009ef0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.898+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 shutdown_connections 2024-05-09T09:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.898+0000 7f72b7361640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f729c0a2ce0 0x7f729c0a51a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.898+0000 7f72b7361640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b010db70 0x7f72b01adeb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.898+0000 7f72b7361640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72b010b1a0 0x7f72b01a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.898+0000 7f72b7361640 1 --2- 172.21.15.64:0/2736964141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b0108830 0x7f72b01a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.898+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 >> 172.21.15.64:0/2736964141 conn(0x7f72b006e6d0 msgr2=0x7f72b0108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.899+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 shutdown_connections 2024-05-09T09:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:35.899+0000 7f72b7361640 1 -- 172.21.15.64:0/2736964141 wait complete. 2024-05-09T09:13:35.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:13:36.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.116+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3183196982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 msgr2=0x7f2cb410e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.116+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3183196982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb410e170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009900 tx=0x7f2ca402a420 comp rx=0 tx=0).stop 2024-05-09T09:13:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3183196982 shutdown_connections 2024-05-09T09:13:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3183196982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb410e170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3183196982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb4108f10 0x7f2cb4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3183196982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4072c70 0x7f2cb4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3183196982 >> 172.21.15.64:0/3183196982 conn(0x7f2cb406e740 msgr2=0x7f2cb4070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3183196982 shutdown_connections 2024-05-09T09:13:36.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.117+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3183196982 wait complete. 2024-05-09T09:13:36.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 Processor -- start 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 -- start start 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb4072c70 0x7f2cb4114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4108f10 0x7f2cb4115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb41b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb411c120 con 0x7f2cb4108f10 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb411bfa0 con 0x7f2cb41098b0 2024-05-09T09:13:36.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.118+0000 7f2cb9d4d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb411c2a0 con 0x7f2cb4072c70 2024-05-09T09:13:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.120+0000 7f2cb954c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb41b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb8d4b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb4072c70 0x7f2cb4114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb41b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37158/0 (socket says 172.21.15.64:37158) 2024-05-09T09:13:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 -- 172.21.15.64:0/3470103532 learned_addr learned my addr 172.21.15.64:0/3470103532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4108f10 0x7f2cb4115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 -- 172.21.15.64:0/3470103532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb4072c70 msgr2=0x7f2cb4114f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb4072c70 0x7f2cb4114f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 -- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4108f10 msgr2=0x7f2cb4115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4108f10 0x7f2cb4115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb954c640 1 -- 172.21.15.64:0/3470103532 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb41b6d70 con 0x7f2cb41098b0 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.121+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4108f10 0x7f2cb4115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.122+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb41b6830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4005ab0 tx=0x7f2ca4003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.122+0000 7f2cb1ffb640 1 -- 172.21.15.64:0/3470103532 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca403f070 con 0x7f2cb41098b0 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.123+0000 7f2cb1ffb640 1 -- 172.21.15.64:0/3470103532 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca4003e40 con 0x7f2cb41098b0 2024-05-09T09:13:36.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.123+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb41b6f40 con 0x7f2cb41098b0 2024-05-09T09:13:36.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.123+0000 7f2cb1ffb640 1 -- 172.21.15.64:0/3470103532 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4035030 con 0x7f2cb41098b0 2024-05-09T09:13:36.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.123+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb41b7370 con 0x7f2cb41098b0 2024-05-09T09:13:36.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.127+0000 7f2cb1ffb640 1 -- 172.21.15.64:0/3470103532 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ca4042070 con 0x7f2cb41098b0 2024-05-09T09:13:36.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.127+0000 7f2cbdd95740 1 Processor -- start 2024-05-09T09:13:36.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.127+0000 7f2cbdd95740 1 -- start start 2024-05-09T09:13:36.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.127+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a616520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a616520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a616520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53834/0 (socket says 172.21.15.64:53834) 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616af0 0x55a61a61b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a61e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a61a57ac40 con 0x55a61a616af0 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a61a522580 con 0x55a61a61bc60 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a61a576c00 con 0x55a61a616100 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 -- 172.21.15.64:0/2159536452 learned_addr learned my addr 172.21.15.64:0/2159536452 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 -- 172.21.15.64:0/2159536452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 msgr2=0x55a61a61e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a61e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 -- 172.21.15.64:0/2159536452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616af0 msgr2=0x55a61a61b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616af0 0x55a61a61b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 -- 172.21.15.64:0/2159536452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a61a28e3f0 con 0x55a61a616100 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.128+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a616520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c620 tx=0x7f2cac00cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84374bc603c48c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.129+0000 7f2c9affd640 1 -- 172.21.15.64:0/2159536452 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac01a070 con 0x55a61a616100 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.129+0000 7f2c9affd640 1 -- 172.21.15.64:0/2159536452 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cac00c0d0 con 0x55a61a616100 2024-05-09T09:13:36.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.129+0000 7f2c9affd640 1 -- 172.21.15.64:0/2159536452 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac015810 con 0x55a61a616100 2024-05-09T09:13:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.129+0000 7f2cb1ffb640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c9c0a2d60 0x7f2c9c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cb1ffb640 1 -- 172.21.15.64:0/3470103532 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ca40efc20 con 0x7f2cb41098b0 2024-05-09T09:13:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 -- 172.21.15.64:0/2159536452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 msgr2=0x55a61a616520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a616520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c620 tx=0x7f2cac00cae0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c9c0a2d60 0x7f2c9c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 -- 172.21.15.64:0/2159536452 shutdown_connections 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a61e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616af0 0x55a61a61b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/2159536452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a616520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 -- 172.21.15.64:0/2159536452 >> 172.21.15.64:0/2159536452 conn(0x55a61a611770 msgr2=0x55a61a611b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.130+0000 7f2cbdd95740 1 -- 172.21.15.64:0/2159536452 shutdown_connections 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.131+0000 7f2cbdd95740 1 -- 172.21.15.64:0/2159536452 wait complete. 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.131+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c9c0a2d60 0x7f2c9c0a5220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55a61a6126e0 tx=0x7f2ca80046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.131+0000 7f2cbdd95740 1 -- start start 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.131+0000 7f2cbdd95740 1 Processor -- start 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a620b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a61bc60 0x55a61a627b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a61a61ec10 con 0x55a61a616100 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a61a535c40 con 0x55a61a616af0 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a61a61ed90 con 0x55a61a61bc60 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cbdd95740 10 client.0 _ll_register_callbacks cb 0x55a61a28dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a620b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cb954c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a61bc60 0x55a61a627b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a620b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37178/0 (socket says 172.21.15.64:37178) 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cb3fff640 1 -- 172.21.15.64:0/630759687 learned_addr learned my addr 172.21.15.64:0/630759687 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.132+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cb954c640 1 -- 172.21.15.64:0/630759687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 msgr2=0x55a61a620b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cb954c640 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a620b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cb954c640 1 -- 172.21.15.64:0/630759687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 msgr2=0x55a61a6205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cb954c640 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cb954c640 1 -- 172.21.15.64:0/630759687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a61a3587d0 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cb954c640 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a61bc60 0x55a61a627b60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40b84f0 tx=0x7f2ca40bbc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e116e045c57e45c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2c93fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2c93fff640 1 -- 172.21.15.64:0/630759687 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4049040 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2c93fff640 1 -- 172.21.15.64:0/630759687 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ca40b8700 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.133+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cac00c0d0 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.134+0000 7f2c93fff640 1 -- 172.21.15.64:0/630759687 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2ca40c0440 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.134+0000 7f2cbdd95740 10 client.26341 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.134+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a61a61ec80 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.134+0000 7f2c93fff640 1 -- 172.21.15.64:0/630759687 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cac00c0d0 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.134+0000 7f2c93fff640 1 -- 172.21.15.64:0/630759687 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f2ca403b560 con 0x55a61a61bc60 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cbdd95740 10 client.26341 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:36.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cbdd95740 10 client.26341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:13:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cbdd95740 20 client.26341 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a62c680 0x55a61a62eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cbdd95740 4 client.26341 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a61a588370 con 0x55a61a62c680 2024-05-09T09:13:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.135+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a62c680 0x55a61a62eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.136+0000 7f2c93fff640 10 client.26341 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.136+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a62c680 0x55a61a62eb20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db7806f15c4307eb server_cookie=c333d0357088a709 in_seq=0 out_seq=0 2024-05-09T09:13:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2c93fff640 1 -- 172.21.15.64:0/630759687 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a61a588370 con 0x55a61a62c680 2024-05-09T09:13:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2c93fff640 10 client.26341 handle_command_reply: tid=0 2024-05-09T09:13:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 client.26341 shutdown 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a61bc60 msgr2=0x55a61a627b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a61bc60 0x55a61a627b60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40b84f0 tx=0x7f2ca40bbc90 comp rx=0 tx=0).stop 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 shutdown_connections 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a62c680 0x55a61a62eb20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a61bc60 0x55a61a627b60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a620b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.137+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/630759687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.138+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 >> 172.21.15.64:0/630759687 conn(0x55a61a611770 msgr2=0x55a61a610990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.138+0000 7f2c93fff640 0 client.26341 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.138+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 shutdown_connections 2024-05-09T09:13:36.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.139+0000 7f2cbdd95740 1 -- 172.21.15.64:0/630759687 wait complete. 2024-05-09T09:13:36.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.142+0000 7f2cbdd95740 20 client.26341 trim_cache size 0 max 16384 2024-05-09T09:13:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 Processor -- start 2024-05-09T09:13:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 -- start start 2024-05-09T09:13:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a617f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a627b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 0x55a61a618490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41c40d0 con 0x55a61a61bc60 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41c1940 con 0x55a61a616af0 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.172+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41bf790 con 0x55a61a616100 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 0x55a61a618490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 0x55a61a618490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33828/0 (socket says 172.21.15.64:33828) 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 -- 172.21.15.64:0/3406515034 learned_addr learned my addr 172.21.15.64:0/3406515034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a627b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 -- 172.21.15.64:0/3406515034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 msgr2=0x55a61a617f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a617f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 -- 172.21.15.64:0/3406515034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 msgr2=0x55a61a627b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a627b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.173+0000 7f2cb954c640 1 -- 172.21.15.64:0/3406515034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a61a62f1a0 con 0x55a61a61bc60 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.174+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 0x55a61a618490 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2ca402a9e0 tx=0x7f2ca40bbd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9147bcf1b4a1595 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.174+0000 7f2c9a7fc640 1 -- 172.21.15.64:0/3406515034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4049070 con 0x55a61a61bc60 2024-05-09T09:13:36.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.174+0000 7f2c9a7fc640 1 -- 172.21.15.64:0/3406515034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca40c4d00 con 0x55a61a61bc60 2024-05-09T09:13:36.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.174+0000 7f2c9a7fc640 1 -- 172.21.15.64:0/3406515034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40b7fc0 con 0x55a61a61bc60 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.175+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3406515034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 msgr2=0x55a61a618490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.175+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 0x55a61a618490 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2ca402a9e0 tx=0x7f2ca40bbd00 comp rx=0 tx=0).stop 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.175+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3406515034 shutdown_connections 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.175+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a61bc60 0x55a61a618490 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.175+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a616af0 0x55a61a627b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.176+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3406515034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616100 0x55a61a617f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.176+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3406515034 >> 172.21.15.64:0/3406515034 conn(0x55a61a611770 msgr2=0x55a61a6205e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.176+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3406515034 shutdown_connections 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.176+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3406515034 wait complete. 2024-05-09T09:13:36.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.177+0000 7f2cbdd95740 1 -- start start 2024-05-09T09:13:36.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.177+0000 7f2cbdd95740 1 Processor -- start 2024-05-09T09:13:36.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 0x55a61a6258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cbdd95740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a626890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cbdd95740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41bfe00 con 0x55a61a616100 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 0x55a61a6258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 0x55a61a6258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53854/0 (socket says 172.21.15.64:53854) 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cb3fff640 1 -- 172.21.15.64:0/3304026674 learned_addr learned my addr 172.21.15.64:0/3304026674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cb954c640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a626890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.178+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41c1270 con 0x55a61a61bc60 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41c04d0 con 0x55a61a616af0 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cbdd95740 10 client.0 _ll_register_callbacks cb 0x7f2ca4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cb3fff640 1 -- 172.21.15.64:0/3304026674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 msgr2=0x55a61a626890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a626890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cb3fff640 1 -- 172.21.15.64:0/3304026674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 msgr2=0x55a61a6253b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6253b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cb3fff640 1 -- 172.21.15.64:0/3304026674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a61a626e80 con 0x55a61a616af0 2024-05-09T09:13:36.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2cb3fff640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 0x55a61a6258f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009770 tx=0x7f2cac009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad48c1eed2481b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2c93fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.179+0000 7f2c93fff640 1 -- 172.21.15.64:0/3304026674 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac024070 con 0x55a61a616af0 2024-05-09T09:13:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca40c4d00 con 0x55a61a616af0 2024-05-09T09:13:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2c93fff640 1 -- 172.21.15.64:0/3304026674 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2cac003a10 con 0x55a61a616af0 2024-05-09T09:13:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2c93fff640 1 -- 172.21.15.64:0/3304026674 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2cac02f050 con 0x55a61a616af0 2024-05-09T09:13:36.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 10 client.26344 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a61a692cf0 con 0x55a61a616af0 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2c93fff640 1 -- 172.21.15.64:0/3304026674 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca40c4d00 con 0x55a61a616af0 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2c93fff640 1 -- 172.21.15.64:0/3304026674 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f2cac020a90 con 0x55a61a616af0 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 10 client.26344 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 10 client.26344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 20 client.26344 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a6933c0 0x55a61a6957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 4 client.26344 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.180+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a61a5826c0 con 0x55a61a6933c0 2024-05-09T09:13:36.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.181+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a6933c0 0x55a61a6957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.183+0000 7f2cb8d4b640 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a6933c0 0x55a61a6957e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b885bd87c6b1672 server_cookie=33c77101867b2758 in_seq=0 out_seq=0 2024-05-09T09:13:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.184+0000 7f2c93fff640 10 client.26344 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.184+0000 7f2c93fff640 1 -- 172.21.15.64:0/3304026674 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a61a5826c0 con 0x55a61a6933c0 2024-05-09T09:13:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.184+0000 7f2c93fff640 10 client.26344 handle_command_reply: tid=0 2024-05-09T09:13:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.184+0000 7f2cbdd95740 1 client.26344 shutdown 2024-05-09T09:13:36.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 msgr2=0x55a61a6258f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 0x55a61a6258f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009770 tx=0x7f2cac009d20 comp rx=0 tx=0).stop 2024-05-09T09:13:36.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 shutdown_connections 2024-05-09T09:13:36.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55a61a6933c0 0x55a61a6957e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a61a61bc60 0x55a61a626890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a61a616af0 0x55a61a6258f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.186+0000 7f2cbdd95740 1 --2- 172.21.15.64:0/3304026674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a61a616100 0x55a61a6253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.187+0000 7f2c93fff640 0 client.26344 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:13:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.187+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 >> 172.21.15.64:0/3304026674 conn(0x55a61a611770 msgr2=0x55a61a620580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.189+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 shutdown_connections 2024-05-09T09:13:36.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.189+0000 7f2cbdd95740 1 -- 172.21.15.64:0/3304026674 wait complete. 2024-05-09T09:13:36.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.189+0000 7f2cbdd95740 20 client.26344 trim_cache size 0 max 16384 2024-05-09T09:13:36.190 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c9c0a2d60 msgr2=0x7f2c9c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c9c0a2d60 0x7f2c9c0a5220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55a61a6126e0 tx=0x7f2ca80046c0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 msgr2=0x7f2cb41b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb41b6830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4005ab0 tx=0x7f2ca4003cd0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 shutdown_connections 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c9c0a2d60 0x7f2c9c0a5220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb41098b0 0x7f2cb41b6830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cb4108f10 0x7f2cb4115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 --2- 172.21.15.64:0/3470103532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cb4072c70 0x7f2cb4114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.193+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 >> 172.21.15.64:0/3470103532 conn(0x7f2cb406e740 msgr2=0x7f2cb4070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.195+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 shutdown_connections 2024-05-09T09:13:36.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.196+0000 7f2cb9d4d640 1 -- 172.21.15.64:0/3470103532 wait complete. 2024-05-09T09:13:36.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:13:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 195.0 s, 1056.0 tps, lat 94.428 ms stddev 30.416 2024-05-09T09:13:36.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.406+0000 7f6f1759e640 1 -- 172.21.15.64:0/3782405808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 msgr2=0x7f6f100950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.406+0000 7f6f1759e640 1 --2- 172.21.15.64:0/3782405808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f100950a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009860 tx=0x7f6f0c02a340 comp rx=0 tx=0).stop 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.407+0000 7f6f1759e640 1 -- 172.21.15.64:0/3782405808 shutdown_connections 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.407+0000 7f6f1759e640 1 --2- 172.21.15.64:0/3782405808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f10096830 0x7f6f1009d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.407+0000 7f6f1759e640 1 --2- 172.21.15.64:0/3782405808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 0x7f6f100962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.407+0000 7f6f1759e640 1 --2- 172.21.15.64:0/3782405808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f100950a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.407+0000 7f6f1759e640 1 -- 172.21.15.64:0/3782405808 >> 172.21.15.64:0/3782405808 conn(0x7f6f100901d0 msgr2=0x7f6f10092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.407+0000 7f6f1759e640 1 -- 172.21.15.64:0/3782405808 shutdown_connections 2024-05-09T09:13:36.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.408+0000 7f6f1759e640 1 -- 172.21.15.64:0/3782405808 wait complete. 2024-05-09T09:13:36.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.409+0000 7f6f1759e640 1 Processor -- start 2024-05-09T09:13:36.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.409+0000 7f6f1759e640 1 -- start start 2024-05-09T09:13:36.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f10129ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 0x7f6f1012a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f10096830 0x7f6f10131150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1009f6b0 con 0x7f6f10096830 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1759e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1009f530 con 0x7f6f10094c80 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1759e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1009f830 con 0x7f6f10095e70 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 0x7f6f1012a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f15d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 0x7f6f1012a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53868/0 (socket says 172.21.15.64:53868) 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f15d9b640 1 -- 172.21.15.64:0/2060272235 learned_addr learned my addr 172.21.15.64:0/2060272235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.410+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f10129ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.411+0000 7f6f1659c640 1 -- 172.21.15.64:0/2060272235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 msgr2=0x7f6f1012a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.411+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 0x7f6f1012a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.411+0000 7f6f1659c640 1 -- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f10096830 msgr2=0x7f6f10131150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:13:36.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.411+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f10096830 0x7f6f10131150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.411+0000 7f6f1659c640 1 -- 172.21.15.64:0/2060272235 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f10131690 con 0x7f6f10094c80 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.412+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f10129ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009780 tx=0x7f6f0c030f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.412+0000 7f6f077fe640 1 -- 172.21.15.64:0/2060272235 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c040030 con 0x7f6f10094c80 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.412+0000 7f6f077fe640 1 -- 172.21.15.64:0/2060272235 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f0c038040 con 0x7f6f10094c80 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.412+0000 7f6f077fe640 1 -- 172.21.15.64:0/2060272235 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c037070 con 0x7f6f10094c80 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.412+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f10131920 con 0x7f6f10094c80 2024-05-09T09:13:36.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.412+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f10131cd0 con 0x7f6f10094c80 2024-05-09T09:13:36.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.418+0000 7f6f077fe640 1 -- 172.21.15.64:0/2060272235 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f0c043070 con 0x7f6f10094c80 2024-05-09T09:13:36.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f077fe640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f000a2c70 0x7f6f000a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 Processor -- start 2024-05-09T09:13:36.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f077fe640 1 -- 172.21.15.64:0/2060272235 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f0c0ef630 con 0x7f6f10094c80 2024-05-09T09:13:36.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 -- start start 2024-05-09T09:13:36.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eeee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eef35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55984ee52c40 con 0x55984eeeeaf0 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55984edfa580 con 0x55984eef3c60 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.419+0000 7f6f1f985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55984ee4ec00 con 0x55984eeee100 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37246/0 (socket says 172.21.15.64:37246) 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f16d9d640 1 -- 172.21.15.64:0/2254055248 learned_addr learned my addr 172.21.15.64:0/2254055248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eeee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f1659c640 1 -- 172.21.15.64:0/2254055248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 msgr2=0x55984eef60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f1659c640 1 -- 172.21.15.64:0/2254055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 msgr2=0x55984eef35f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eef35f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f1659c640 1 -- 172.21.15.64:0/2254055248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984eb663f0 con 0x55984eeee100 2024-05-09T09:13:36.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.420+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f000a2c70 0x7f6f000a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.421+0000 7f6f1659c640 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eeee520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0ecd40 tx=0x7f6f0c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6670c406d0e9aeb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.421+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f000a2c70 0x7f6f000a5130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55984eeeeef0 tx=0x7f6f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.424+0000 7f6f04ff9640 1 -- 172.21.15.64:0/2254055248 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0d4070 con 0x55984eeee100 2024-05-09T09:13:36.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.424+0000 7f6f04ff9640 1 -- 172.21.15.64:0/2254055248 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f0c0328f0 con 0x55984eeee100 2024-05-09T09:13:36.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f04ff9640 1 -- 172.21.15.64:0/2254055248 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0ef300 con 0x55984eeee100 2024-05-09T09:13:36.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 -- 172.21.15.64:0/2254055248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 msgr2=0x55984eeee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eeee520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0ecd40 tx=0x7f6f0c02a340 comp rx=0 tx=0).stop 2024-05-09T09:13:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 -- 172.21.15.64:0/2254055248 shutdown_connections 2024-05-09T09:13:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eef35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 --2- 172.21.15.64:0/2254055248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eeee520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.425+0000 7f6f1f985740 1 -- 172.21.15.64:0/2254055248 >> 172.21.15.64:0/2254055248 conn(0x55984eee9770 msgr2=0x55984eee9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.430+0000 7f6f1f985740 1 -- 172.21.15.64:0/2254055248 shutdown_connections 2024-05-09T09:13:36.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.431+0000 7f6f1f985740 1 -- 172.21.15.64:0/2254055248 wait complete. 2024-05-09T09:13:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.431+0000 7f6f1f985740 1 -- start start 2024-05-09T09:13:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 Processor -- start 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eef8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eef8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eeff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55984ee4ec00 con 0x55984eeeeaf0 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55984ee0dc40 con 0x55984eef3c60 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55984eef69d0 con 0x55984eeee100 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.432+0000 7f6f1f985740 10 client.0 _ll_register_callbacks cb 0x55984eb65be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eef8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eef8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53892/0 (socket says 172.21.15.64:53892) 2024-05-09T09:13:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 -- 172.21.15.64:0/809201038 learned_addr learned my addr 172.21.15.64:0/809201038 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 -- 172.21.15.64:0/809201038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 msgr2=0x55984eeff6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:13:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eeff6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 -- 172.21.15.64:0/809201038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 msgr2=0x55984eef8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eef8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 -- 172.21.15.64:0/809201038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984ec307d0 con 0x55984eeee100 2024-05-09T09:13:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6f1659c640 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eef8140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0bba30 tx=0x7f6f0c009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ee3623dc92469 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.433+0000 7f6efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.434+0000 7f6efdffb640 1 -- 172.21.15.64:0/809201038 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0d8030 con 0x55984eeee100 2024-05-09T09:13:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.434+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c0328f0 con 0x55984eeee100 2024-05-09T09:13:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.434+0000 7f6efdffb640 1 -- 172.21.15.64:0/809201038 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f0c0b9da0 con 0x55984eeee100 2024-05-09T09:13:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.435+0000 7f6efdffb640 1 -- 172.21.15.64:0/809201038 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f0c0d4040 con 0x55984eeee100 2024-05-09T09:13:36.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.438+0000 7f6f1f985740 10 client.26356 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:36.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.438+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55984ef02a30 con 0x55984eeee100 2024-05-09T09:13:36.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.438+0000 7f6efdffb640 1 -- 172.21.15.64:0/809201038 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f0c0328f0 con 0x55984eeee100 2024-05-09T09:13:36.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.438+0000 7f6efdffb640 1 -- 172.21.15.64:0/809201038 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6f0c04d060 con 0x55984eeee100 2024-05-09T09:13:36.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.440+0000 7f6f1f985740 10 client.26356 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:36.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.440+0000 7f6f1f985740 10 client.26356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:13:36.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.440+0000 7f6f1f985740 20 client.26356 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:36.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.440+0000 7f6f1f985740 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef03ed0 0x55984ef06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.440+0000 7f6f1f985740 4 client.26356 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:36.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.440+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55984ee60370 con 0x55984ef03ed0 2024-05-09T09:13:36.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.442+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef03ed0 0x55984ef06350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.443+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef03ed0 0x55984ef06350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88ff93efe296eb72 server_cookie=1a7e45bbc62fb92c in_seq=0 out_seq=0 2024-05-09T09:13:36.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.443+0000 7f6efdffb640 10 client.26356 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.444+0000 7f6efdffb640 1 -- 172.21.15.64:0/809201038 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55984ee60370 con 0x55984ef03ed0 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.444+0000 7f6efdffb640 10 client.26356 handle_command_reply: tid=0 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.444+0000 7f6f1f985740 1 client.26356 shutdown 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 msgr2=0x55984eef8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eef8140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0bba30 tx=0x7f6f0c009820 comp rx=0 tx=0).stop 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 shutdown_connections 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eeff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef03ed0 0x55984ef06350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eef8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 --2- 172.21.15.64:0/809201038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeee100 0x55984eef8140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 >> 172.21.15.64:0/809201038 conn(0x55984eee9770 msgr2=0x55984eee8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6efdffb640 0 client.26356 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:36.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 shutdown_connections 2024-05-09T09:13:36.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 1 -- 172.21.15.64:0/809201038 wait complete. 2024-05-09T09:13:36.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.445+0000 7f6f1f985740 20 client.26356 trim_cache size 0 max 16384 2024-05-09T09:13:36.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.473+0000 7f6f1f985740 1 Processor -- start 2024-05-09T09:13:36.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.474+0000 7f6f1f985740 1 -- start start 2024-05-09T09:13:36.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeee100 0x55984eef8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeeeaf0 0x55984eeff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeee100 0x55984eef8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeee100 0x55984eef8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33854/0 (socket says 172.21.15.64:33854) 2024-05-09T09:13:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f1659c640 1 -- 172.21.15.64:0/3708461139 learned_addr learned my addr 172.21.15.64:0/3708461139 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.475+0000 7f6f16d9d640 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.476+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10147660 con 0x55984eeee100 2024-05-09T09:13:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.476+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10144ed0 con 0x55984eef3c60 2024-05-09T09:13:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.476+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10142d20 con 0x55984eeeeaf0 2024-05-09T09:13:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.476+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeeeaf0 0x55984eeff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.477+0000 7f6f15d9b640 1 -- 172.21.15.64:0/3708461139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 msgr2=0x55984eef6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.477+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.477+0000 7f6f15d9b640 1 -- 172.21.15.64:0/3708461139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeee100 msgr2=0x55984eef8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.477+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeee100 0x55984eef8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.477+0000 7f6f15d9b640 1 -- 172.21.15.64:0/3708461139 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984eef3160 con 0x55984eeeeaf0 2024-05-09T09:13:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.477+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeeeaf0 0x55984eeff680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55984ef06bc0 tx=0x7f6f08014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635d2fbae85de993 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.478+0000 7f6efffff640 1 -- 172.21.15.64:0/3708461139 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004380 con 0x55984eeeeaf0 2024-05-09T09:13:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.478+0000 7f6efffff640 1 -- 172.21.15.64:0/3708461139 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f08026850 con 0x55984eeeeaf0 2024-05-09T09:13:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.478+0000 7f6efffff640 1 -- 172.21.15.64:0/3708461139 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004a00 con 0x55984eeeeaf0 2024-05-09T09:13:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeeeaf0 msgr2=0x55984eeff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeeeaf0 0x55984eeff680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55984ef06bc0 tx=0x7f6f08014000 comp rx=0 tx=0).stop 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 shutdown_connections 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eef3c60 0x55984eef6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eeeeaf0 0x55984eeff680 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 --2- 172.21.15.64:0/3708461139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeee100 0x55984eef8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.479+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 >> 172.21.15.64:0/3708461139 conn(0x55984eee9770 msgr2=0x55984eef7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.480+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 shutdown_connections 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.480+0000 7f6f1f985740 1 -- 172.21.15.64:0/3708461139 wait complete. 2024-05-09T09:13:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.480+0000 7f6f1f985740 1 -- start start 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.480+0000 7f6f1f985740 1 Processor -- start 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 0x55984eefc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eefca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 0x55984ef68480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10143390 con 0x55984eeeeaf0 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10144800 con 0x55984eeee100 2024-05-09T09:13:36.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10143a60 con 0x55984eef3c60 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1f985740 10 client.0 _ll_register_callbacks cb 0x7f6f0c0d2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 0x55984ef68480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 0x55984eefc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 0x55984eefc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37280/0 (socket says 172.21.15.64:37280) 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f16d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 0x55984ef68480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53920/0 (socket says 172.21.15.64:53920) 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f1659c640 1 -- 172.21.15.64:0/961589648 learned_addr learned my addr 172.21.15.64:0/961589648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.481+0000 7f6f15d9b640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eefca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.482+0000 7f6f16d9d640 1 -- 172.21.15.64:0/961589648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 msgr2=0x55984eefc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.482+0000 7f6f16d9d640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 0x55984eefc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.482+0000 7f6f16d9d640 1 -- 172.21.15.64:0/961589648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 msgr2=0x55984eefca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.482+0000 7f6f16d9d640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eefca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.482+0000 7f6f16d9d640 1 -- 172.21.15.64:0/961589648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984ef689c0 con 0x55984eef3c60 2024-05-09T09:13:36.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.482+0000 7f6f1659c640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 0x55984eefc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:36.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.483+0000 7f6f16d9d640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 0x55984ef68480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f180503a0 tx=0x7f6f18069650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e02153cbd7ac841 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.483+0000 7f6efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.484+0000 7f6efdffb640 1 -- 172.21.15.64:0/961589648 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f180806b0 con 0x55984eef3c60 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.484+0000 7f6efdffb640 1 -- 172.21.15.64:0/961589648 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6f18080850 con 0x55984eef3c60 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.484+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f08026850 con 0x55984eef3c60 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6efdffb640 1 -- 172.21.15.64:0/961589648 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f18080a70 con 0x55984eef3c60 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6f1f985740 10 client.26362 fetch_fsmap learned FSMap version 36 2024-05-09T09:13:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55984ef69cb0 con 0x55984eef3c60 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6efdffb640 1 -- 172.21.15.64:0/961589648 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f08026850 con 0x55984eef3c60 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6efdffb640 1 -- 172.21.15.64:0/961589648 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6f18067ce0 con 0x55984eef3c60 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6f1f985740 10 client.26362 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6f1f985740 10 client.26362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.485+0000 7f6f1f985740 20 client.26362 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.486+0000 7f6f1f985740 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef6a660 0x55984ef6ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.486+0000 7f6f1f985740 4 client.26362 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:13:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.486+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55984ee5a6c0 con 0x55984ef6a660 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.486+0000 7f6f1659c640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef6a660 0x55984ef6ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.486+0000 7f6f1659c640 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef6a660 0x55984ef6ca80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5393ca30c96ad61 server_cookie=a9a7fb5f003301ad in_seq=0 out_seq=0 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.486+0000 7f6efdffb640 10 client.26362 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6efdffb640 1 -- 172.21.15.64:0/961589648 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55984ee5a6c0 con 0x55984ef6a660 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6efdffb640 10 client.26362 handle_command_reply: tid=0 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 client.26362 shutdown 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 msgr2=0x55984ef68480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 0x55984ef68480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f180503a0 tx=0x7f6f18069650 comp rx=0 tx=0).stop 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 shutdown_connections 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55984eef3c60 0x55984ef68480 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55984ef6a660 0x55984ef6ca80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55984eeeeaf0 0x55984eefca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 --2- 172.21.15.64:0/961589648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55984eeee100 0x55984eefc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.487+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 >> 172.21.15.64:0/961589648 conn(0x55984eee9770 msgr2=0x55984eeed090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.488+0000 7f6efdffb640 0 client.26362 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:13:36.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.488+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 shutdown_connections 2024-05-09T09:13:36.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.488+0000 7f6f1f985740 1 -- 172.21.15.64:0/961589648 wait complete. 2024-05-09T09:13:36.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.488+0000 7f6f1f985740 20 client.26362 trim_cache size 0 max 16384 2024-05-09T09:13:36.489 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:13:36.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f000a2c70 msgr2=0x7f6f000a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f000a2c70 0x7f6f000a5130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55984eeeeef0 tx=0x7f6f08009260 comp rx=0 tx=0).stop 2024-05-09T09:13:36.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 msgr2=0x7f6f10129ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f10129ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009780 tx=0x7f6f0c030f40 comp rx=0 tx=0).stop 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 shutdown_connections 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6f000a2c70 0x7f6f000a5130 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f10096830 0x7f6f10131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f10095e70 0x7f6f1012a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 --2- 172.21.15.64:0/2060272235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f10094c80 0x7f6f10129ba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 >> 172.21.15.64:0/2060272235 conn(0x7f6f100901d0 msgr2=0x7f6f100990a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 shutdown_connections 2024-05-09T09:13:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.489+0000 7f6f1759e640 1 -- 172.21.15.64:0/2060272235 wait complete. 2024-05-09T09:13:36.507 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.706+0000 7f03c4676640 1 -- 172.21.15.64:0/695953824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 msgr2=0x7f03bc105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.706+0000 7f03c4676640 1 --2- 172.21.15.64:0/695953824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc105af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009ca0 tx=0x7f03b402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.707+0000 7f03c4676640 1 -- 172.21.15.64:0/695953824 shutdown_connections 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.707+0000 7f03c4676640 1 --2- 172.21.15.64:0/695953824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc105af0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.707+0000 7f03c4676640 1 --2- 172.21.15.64:0/695953824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc100890 0x7f03bc100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.707+0000 7f03c4676640 1 --2- 172.21.15.64:0/695953824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03bc0ffb30 0x7f03bc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.707+0000 7f03c4676640 1 -- 172.21.15.64:0/695953824 >> 172.21.15.64:0/695953824 conn(0x7f03bc0fb460 msgr2=0x7f03bc0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.708+0000 7f03c4676640 1 -- 172.21.15.64:0/695953824 shutdown_connections 2024-05-09T09:13:36.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.708+0000 7f03c4676640 1 -- 172.21.15.64:0/695953824 wait complete. 2024-05-09T09:13:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.708+0000 7f03c4676640 1 Processor -- start 2024-05-09T09:13:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 -- start start 2024-05-09T09:13:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03bc0ffb30 0x7f03bc070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc100890 0x7f03bc070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc1ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113a20 con 0x7f03bc100890 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1138a0 con 0x7f03bc0ffb30 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c4676640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113ba0 con 0x7f03bc101250 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c2c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc1ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c2c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc1ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53938/0 (socket says 172.21.15.64:53938) 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c2c13640 1 -- 172.21.15.64:0/1322715828 learned_addr learned my addr 172.21.15.64:0/1322715828 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c1c11640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc100890 0x7f03bc070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c2412640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03bc0ffb30 0x7f03bc070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.709+0000 7f03c2c13640 1 -- 172.21.15.64:0/1322715828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03bc0ffb30 msgr2=0x7f03bc070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03bc0ffb30 0x7f03bc070210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03c2c13640 1 -- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc100890 msgr2=0x7f03bc070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc100890 0x7f03bc070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03c2c13640 1 -- 172.21.15.64:0/1322715828 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc106f10 con 0x7f03bc101250 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc1ae0f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03b4031040 tx=0x7f03b4032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03b37fe640 1 -- 172.21.15.64:0/1322715828 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4030c00 con 0x7f03bc101250 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.710+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc1071a0 con 0x7f03bc101250 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.711+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc107680 con 0x7f03bc101250 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.711+0000 7f03b37fe640 1 -- 172.21.15.64:0/1322715828 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b4030da0 con 0x7f03bc101250 2024-05-09T09:13:36.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.711+0000 7f03b37fe640 1 -- 172.21.15.64:0/1322715828 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b403b410 con 0x7f03bc101250 2024-05-09T09:13:36.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.714+0000 7f03b37fe640 1 -- 172.21.15.64:0/1322715828 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f03b4003970 con 0x7f03bc101250 2024-05-09T09:13:36.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.714+0000 7f03c7385740 1 Processor -- start 2024-05-09T09:13:36.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.714+0000 7f03c7385740 1 -- start start 2024-05-09T09:13:36.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c7385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c7385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 0x5648282ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c7385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282abaa0 0x5648282adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c7385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56482820ac40 con 0x5648282a5f40 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c7385740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5648281b2580 con 0x5648282a6930 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c7385740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564828206c00 con 0x5648282abaa0 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c1c11640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282abaa0 0x5648282adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c1c11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c1c11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33884/0 (socket says 172.21.15.64:33884) 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.715+0000 7f03c1c11640 1 -- 172.21.15.64:0/3333968656 learned_addr learned my addr 172.21.15.64:0/3333968656 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03c1c11640 1 -- 172.21.15.64:0/3333968656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 msgr2=0x5648282ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03c1c11640 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 0x5648282ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03c1c11640 1 -- 172.21.15.64:0/3333968656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 msgr2=0x5648282a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03c1c11640 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03b37fe640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03a80a2c90 0x7f03a80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03b37fe640 1 -- 172.21.15.64:0/1322715828 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03b40f0970 con 0x7f03bc101250 2024-05-09T09:13:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.716+0000 7f03c1c11640 1 -- 172.21.15.64:0/3333968656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564827f1e3f0 con 0x5648282abaa0 2024-05-09T09:13:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.717+0000 7f03c1c11640 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282abaa0 0x5648282adf30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03ac012730 tx=0x7f03ac005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818f88d266835324 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.717+0000 7f03b0ff9640 1 -- 172.21.15.64:0/3333968656 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac012cb0 con 0x5648282abaa0 2024-05-09T09:13:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.717+0000 7f03b0ff9640 1 -- 172.21.15.64:0/3333968656 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac012e50 con 0x5648282abaa0 2024-05-09T09:13:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.717+0000 7f03b0ff9640 1 -- 172.21.15.64:0/3333968656 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00d6e0 con 0x5648282abaa0 2024-05-09T09:13:36.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c2412640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03a80a2c90 0x7f03a80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 -- 172.21.15.64:0/3333968656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282abaa0 msgr2=0x5648282adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282abaa0 0x5648282adf30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03ac012730 tx=0x7f03ac005850 comp rx=0 tx=0).stop 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 -- 172.21.15.64:0/3333968656 shutdown_connections 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282abaa0 0x5648282adf30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 0x5648282ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 --2- 172.21.15.64:0/3333968656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.718+0000 7f03c7385740 1 -- 172.21.15.64:0/3333968656 >> 172.21.15.64:0/3333968656 conn(0x5648282a15b0 msgr2=0x5648282a19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.719+0000 7f03c7385740 1 -- 172.21.15.64:0/3333968656 shutdown_connections 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.721+0000 7f03c7385740 1 -- 172.21.15.64:0/3333968656 wait complete. 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.721+0000 7f03c7385740 1 -- start start 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.721+0000 7f03c7385740 1 Processor -- start 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a5f40 0x5648282b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 0x5648282b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282b5b50 0x5648282b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5648282aea50 con 0x5648282b5b50 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5648281c5c40 con 0x5648282a6930 2024-05-09T09:13:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5648282aebd0 con 0x5648282a5f40 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.722+0000 7f03c7385740 10 client.0 _ll_register_callbacks cb 0x564827f1dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2412640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03a80a2c90 0x7f03a80a5150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5648282a2520 tx=0x7f03b80046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2412640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a5f40 0x5648282b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2412640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a5f40 0x5648282b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53962/0 (socket says 172.21.15.64:53962) 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2412640 1 -- 172.21.15.64:0/4032074555 learned_addr learned my addr 172.21.15.64:0/4032074555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2c13640 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282b5b50 0x5648282b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2c13640 1 -- 172.21.15.64:0/4032074555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a5f40 msgr2=0x5648282b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2c13640 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a5f40 0x5648282b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2c13640 1 -- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 msgr2=0x5648282b0b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.723+0000 7f03c2c13640 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 0x5648282b0b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.724+0000 7f03c2c13640 1 -- 172.21.15.64:0/4032074555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564827fe87d0 con 0x5648282b5b50 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.724+0000 7f03c2c13640 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282b5b50 0x5648282b7f40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f03b40b9240 tx=0x7f03b40bc740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf485996d0c4fc0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.724+0000 7f03a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:13:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.724+0000 7f03a5ffb640 1 -- 172.21.15.64:0/4032074555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40bcde0 con 0x5648282b5b50 2024-05-09T09:13:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.724+0000 7f03a5ffb640 1 -- 172.21.15.64:0/4032074555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03b4003c60 con 0x5648282b5b50 2024-05-09T09:13:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.725+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ac012e50 con 0x5648282b5b50 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.725+0000 7f03a5ffb640 1 -- 172.21.15.64:0/4032074555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f03b40c1190 con 0x5648282b5b50 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.725+0000 7f03c7385740 10 client.16701 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.725+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5648282bb2b0 con 0x5648282b5b50 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03a5ffb640 1 -- 172.21.15.64:0/4032074555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac012e50 con 0x5648282b5b50 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03a5ffb640 1 -- 172.21.15.64:0/4032074555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f03b4004150 con 0x5648282b5b50 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03c7385740 10 client.16701 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03c7385740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03c7385740 20 client.16701 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03c7385740 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5648282bcbe0 0x5648282bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03c7385740 4 client.16701 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.726+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564828218370 con 0x5648282bcbe0 2024-05-09T09:13:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.728+0000 7f03c2412640 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5648282bcbe0 0x5648282bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.730+0000 7f03c2412640 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5648282bcbe0 0x5648282bf060 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb05eec698ed805 server_cookie=ffb6c6b3f2876477 in_seq=0 out_seq=0 2024-05-09T09:13:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.730+0000 7f03a5ffb640 10 client.16701 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03a5ffb640 1 -- 172.21.15.64:0/4032074555 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564828218370 con 0x5648282bcbe0 2024-05-09T09:13:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03a5ffb640 10 client.16701 handle_command_reply: tid=0 2024-05-09T09:13:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 client.16701 shutdown 2024-05-09T09:13:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282b5b50 msgr2=0x5648282b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282b5b50 0x5648282b7f40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f03b40b9240 tx=0x7f03b40bc740 comp rx=0 tx=0).stop 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 shutdown_connections 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5648282bcbe0 0x5648282bf060 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282b5b50 0x5648282b7f40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a6930 0x5648282b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 --2- 172.21.15.64:0/4032074555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a5f40 0x5648282b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.731+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 >> 172.21.15.64:0/4032074555 conn(0x5648282a15b0 msgr2=0x5648282a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.732+0000 7f03a5ffb640 0 client.16701 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.732+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 shutdown_connections 2024-05-09T09:13:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.732+0000 7f03c7385740 1 -- 172.21.15.64:0/4032074555 wait complete. 2024-05-09T09:13:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.732+0000 7f03c7385740 20 client.16701 trim_cache size 0 max 16384 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.759+0000 7f03c7385740 1 Processor -- start 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.759+0000 7f03c7385740 1 -- start start 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.759+0000 7f03c7385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a5f40 0x5648282a0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.759+0000 7f03c7385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a6930 0x5648282a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.759+0000 7f03c7385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282b5b50 0x5648282af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c7385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0012470 con 0x5648282a6930 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c1c11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a6930 0x5648282a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c1c11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a6930 0x5648282a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33916/0 (socket says 172.21.15.64:33916) 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c1c11640 1 -- 172.21.15.64:0/1431718145 learned_addr learned my addr 172.21.15.64:0/1431718145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282b5b50 0x5648282af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000fcb0 con 0x5648282a5f40 2024-05-09T09:13:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000db00 con 0x5648282b5b50 2024-05-09T09:13:36.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c2412640 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a5f40 0x5648282a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c2c13640 1 -- 172.21.15.64:0/1431718145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a5f40 msgr2=0x5648282a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a5f40 0x5648282a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c2c13640 1 -- 172.21.15.64:0/1431718145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a6930 msgr2=0x5648282a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.760+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a6930 0x5648282a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.761+0000 7f03c2c13640 1 -- 172.21.15.64:0/1431718145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648282a2020 con 0x5648282b5b50 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.761+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282b5b50 0x5648282af000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03b40bcd70 tx=0x7f03b40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44581106522c4a6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.761+0000 7f03a7fff640 1 -- 172.21.15.64:0/1431718145 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40bcde0 con 0x5648282b5b50 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.761+0000 7f03a7fff640 1 -- 172.21.15.64:0/1431718145 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b4036070 con 0x5648282b5b50 2024-05-09T09:13:36.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.761+0000 7f03a7fff640 1 -- 172.21.15.64:0/1431718145 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40c5820 con 0x5648282b5b50 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.762+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282b5b50 msgr2=0x5648282af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.762+0000 7f03c7385740 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282b5b50 0x5648282af000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03b40bcd70 tx=0x7f03b40095c0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.762+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 shutdown_connections 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.762+0000 7f03c7385740 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282b5b50 0x5648282af000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.762+0000 7f03c7385740 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a6930 0x5648282a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.763+0000 7f03c7385740 1 --2- 172.21.15.64:0/1431718145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282a5f40 0x5648282a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.763+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 >> 172.21.15.64:0/1431718145 conn(0x5648282a15b0 msgr2=0x56482829ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.763+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 shutdown_connections 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.763+0000 7f03c7385740 1 -- 172.21.15.64:0/1431718145 wait complete. 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.763+0000 7f03c7385740 1 -- start start 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.764+0000 7f03c7385740 1 Processor -- start 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.764+0000 7f03c7385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c7385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a6930 0x5648282cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c7385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282b5b50 0x5648282cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c2412640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c2412640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33932/0 (socket says 172.21.15.64:33932) 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c7385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000e170 con 0x5648282a5f40 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c2412640 1 -- 172.21.15.64:0/1528081434 learned_addr learned my addr 172.21.15.64:0/1528081434 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000f5e0 con 0x5648282b5b50 2024-05-09T09:13:36.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000e840 con 0x5648282a6930 2024-05-09T09:13:36.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c1c11640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a6930 0x5648282cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c7385740 10 client.0 _ll_register_callbacks cb 0x7f03b4009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:36.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c2c13640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282b5b50 0x5648282cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c1c11640 1 -- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282b5b50 msgr2=0x5648282cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c1c11640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282b5b50 0x5648282cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c1c11640 1 -- 172.21.15.64:0/1528081434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 msgr2=0x5648282cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c1c11640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.765+0000 7f03c1c11640 1 -- 172.21.15.64:0/1528081434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648282cf2a0 con 0x5648282a6930 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.766+0000 7f03c1c11640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a6930 0x5648282cdd60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005f00 tx=0x7f03ac0052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765b18a153f184d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.766+0000 7f03a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.766+0000 7f03a5ffb640 1 -- 172.21.15.64:0/1528081434 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00bc90 con 0x5648282a6930 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.766+0000 7f03a5ffb640 1 -- 172.21.15.64:0/1528081434 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03ac012220 con 0x5648282a6930 2024-05-09T09:13:36.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.766+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b4036070 con 0x5648282a6930 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.767+0000 7f03a5ffb640 1 -- 172.21.15.64:0/1528081434 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f03ac02f050 con 0x5648282a6930 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.767+0000 7f03c7385740 10 client.26380 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.767+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5648282d0560 con 0x5648282a6930 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.767+0000 7f03a5ffb640 1 -- 172.21.15.64:0/1528081434 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b4036070 con 0x5648282a6930 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.767+0000 7f03a5ffb640 1 -- 172.21.15.64:0/1528081434 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f03ac026070 con 0x5648282a6930 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.768+0000 7f03c7385740 10 client.26380 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.768+0000 7f03c7385740 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.768+0000 7f03c7385740 20 client.26380 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.768+0000 7f03c7385740 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564828322fc0 0x5648283253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.768+0000 7f03c7385740 4 client.26380 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.768+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648282126c0 con 0x564828322fc0 2024-05-09T09:13:36.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.769+0000 7f03c2412640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564828322fc0 0x5648283253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.770+0000 7f03c2412640 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564828322fc0 0x5648283253b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8110c36ae2da4c37 server_cookie=1a9d70d8c24206ae in_seq=0 out_seq=0 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.770+0000 7f03a5ffb640 10 client.26380 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.770+0000 7f03a5ffb640 1 -- 172.21.15.64:0/1528081434 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648282126c0 con 0x564828322fc0 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.770+0000 7f03a5ffb640 10 client.26380 handle_command_reply: tid=0 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.770+0000 7f03c7385740 1 client.26380 shutdown 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.771+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a6930 msgr2=0x5648282cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.771+0000 7f03c7385740 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a6930 0x5648282cdd60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005f00 tx=0x7f03ac0052e0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.771+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 shutdown_connections 2024-05-09T09:13:36.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.771+0000 7f03c7385740 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564828322fc0 0x5648283253b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.771+0000 7f03c7385740 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5648282b5b50 0x5648282cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.771+0000 7f03a5ffb640 0 client.26380 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:36.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.772+0000 7f03c7385740 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648282a6930 0x5648282cdd60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.772+0000 7f03c7385740 1 --2- 172.21.15.64:0/1528081434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648282a5f40 0x5648282cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.772+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 >> 172.21.15.64:0/1528081434 conn(0x5648282a15b0 msgr2=0x5648282a4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.772+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 shutdown_connections 2024-05-09T09:13:36.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.772+0000 7f03c7385740 1 -- 172.21.15.64:0/1528081434 wait complete. 2024-05-09T09:13:36.774 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.773+0000 7f03c7385740 20 client.26380 trim_cache size 0 max 16384 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03a80a2c90 msgr2=0x7f03a80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03a80a2c90 0x7f03a80a5150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5648282a2520 tx=0x7f03b80046c0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 msgr2=0x7f03bc1ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc1ae0f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03b4031040 tx=0x7f03b4032ec0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 shutdown_connections 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03a80a2c90 0x7f03a80a5150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03bc101250 0x7f03bc1ae0f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc100890 0x7f03bc070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 --2- 172.21.15.64:0/1322715828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03bc0ffb30 0x7f03bc070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:36.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 >> 172.21.15.64:0/1322715828 conn(0x7f03bc0fb460 msgr2=0x7f03bc101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:36.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 shutdown_connections 2024-05-09T09:13:36.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:36.774+0000 7f03c4676640 1 -- 172.21.15.64:0/1322715828 wait complete. 2024-05-09T09:13:36.792 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:36 smithi064 ceph-mon[30149]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-05-09T09:13:36.792 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:36 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2736964141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:13:36.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:36 smithi139 ceph-mon[33599]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-05-09T09:13:36.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:36 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2736964141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:13:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:36 smithi179 ceph-mon[34086]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-05-09T09:13:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:36 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2736964141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:13:37.794 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:13:37.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:13:37.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:13:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.801+0000 7f1095ffb640 1 -- 172.21.15.64:0/956654482 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080031880 con 0x7f1090094030 2024-05-09T09:13:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 -- 172.21.15.64:0/956654482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090094030 msgr2=0x7f1090094450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 --2- 172.21.15.64:0/956654482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090094030 0x7f1090094450 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1080009930 tx=0x7f108002a460 comp rx=0 tx=0).stop 2024-05-09T09:13:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 -- 172.21.15.64:0/956654482 shutdown_connections 2024-05-09T09:13:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 --2- 172.21.15.64:0/956654482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 0x7f109009e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 --2- 172.21.15.64:0/956654482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090097000 0x7f1090097480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 --2- 172.21.15.64:0/956654482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090094030 0x7f1090094450 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 -- 172.21.15.64:0/956654482 >> 172.21.15.64:0/956654482 conn(0x7f109008f940 msgr2=0x7f1090091da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 -- 172.21.15.64:0/956654482 shutdown_connections 2024-05-09T09:13:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.802+0000 7f1097fff640 1 -- 172.21.15.64:0/956654482 wait complete. 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.803+0000 7f1097fff640 1 Processor -- start 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.803+0000 7f1097fff640 1 -- start start 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1097fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090094030 0x7f1090132660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1097fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090097000 0x7f1090132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1097fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 0x7f1090139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1097fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1090096920 con 0x7f1090097000 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1097fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10900967a0 con 0x7f1090094030 2024-05-09T09:13:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1097fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1090096aa0 con 0x7f10900979c0 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f10967fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090097000 0x7f1090132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f10967fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090097000 0x7f1090132ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33954/0 (socket says 172.21.15.64:33954) 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f10967fc640 1 -- 172.21.15.64:0/1218268943 learned_addr learned my addr 172.21.15.64:0/1218268943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.804+0000 7f1096ffd640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090094030 0x7f1090132660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f109d9a5640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 0x7f1090139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f1096ffd640 1 -- 172.21.15.64:0/1218268943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 msgr2=0x7f1090139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f1096ffd640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 0x7f1090139c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f1096ffd640 1 -- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090097000 msgr2=0x7f1090132ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f1096ffd640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090097000 0x7f1090132ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f1096ffd640 1 -- 172.21.15.64:0/1218268943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109013a280 con 0x7f1090094030 2024-05-09T09:13:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f109d9a5640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 0x7f1090139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.805+0000 7f1096ffd640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090094030 0x7f1090132660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f108002aa50 tx=0x7f1080003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.806+0000 7f109c9a3640 1 -- 172.21.15.64:0/1218268943 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108003f070 con 0x7f1090094030 2024-05-09T09:13:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.806+0000 7f109c9a3640 1 -- 172.21.15.64:0/1218268943 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1080031e40 con 0x7f1090094030 2024-05-09T09:13:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.806+0000 7f109c9a3640 1 -- 172.21.15.64:0/1218268943 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080035030 con 0x7f1090094030 2024-05-09T09:13:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.806+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109013a510 con 0x7f1090094030 2024-05-09T09:13:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.806+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f109013aa00 con 0x7f1090094030 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.810+0000 7f10a078d740 1 Processor -- start 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 -- start start 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f109c9a3640 1 -- 172.21.15.64:0/1218268943 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1080042070 con 0x7f1090094030 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddcf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 0x56020ddd48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddd73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56020dd33340 con 0x56020ddcf3d0 2024-05-09T09:13:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56020dcdb290 con 0x56020ddcfdc0 2024-05-09T09:13:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10a078d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56020dd2f300 con 0x56020ddd4f30 2024-05-09T09:13:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 0x56020ddd48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f10967fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddd73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 0x56020ddd48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37358/0 (socket says 172.21.15.64:37358) 2024-05-09T09:13:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 -- 172.21.15.64:0/2577669114 learned_addr learned my addr 172.21.15.64:0/2577669114 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 -- 172.21.15.64:0/2577669114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 msgr2=0x56020ddd73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddd73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 -- 172.21.15.64:0/2577669114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 msgr2=0x56020ddcf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddcf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.811+0000 7f1096ffd640 1 -- 172.21.15.64:0/2577669114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020dcff8e0 con 0x56020ddcfdc0 2024-05-09T09:13:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.812+0000 7f1096ffd640 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 0x56020ddd48c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f10800f3d50 tx=0x7f10800f3e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9482b526d07db627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.813+0000 7f109c9a3640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10840a2d10 0x7f10840a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.813+0000 7f109c9a3640 1 -- 172.21.15.64:0/1218268943 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f10800efbb0 con 0x7f1090094030 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.813+0000 7f109d9a5640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10840a2d10 0x7f10840a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.813+0000 7f108dffb640 1 -- 172.21.15.64:0/2577669114 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800ff070 con 0x56020ddcfdc0 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.813+0000 7f108dffb640 1 -- 172.21.15.64:0/2577669114 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10800f8d60 con 0x56020ddcfdc0 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.814+0000 7f108dffb640 1 -- 172.21.15.64:0/2577669114 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800fa690 con 0x56020ddcfdc0 2024-05-09T09:13:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.814+0000 7f109d9a5640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10840a2d10 0x7f10840a51d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f10980674e0 tx=0x7f1098066fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 -- 172.21.15.64:0/2577669114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 msgr2=0x56020ddd48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 0x56020ddd48c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f10800f3d50 tx=0x7f10800f3e40 comp rx=0 tx=0).stop 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 -- 172.21.15.64:0/2577669114 shutdown_connections 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddd73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcfdc0 0x56020ddd48c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 --2- 172.21.15.64:0/2577669114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddcf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.815+0000 7f10a078d740 1 -- 172.21.15.64:0/2577669114 >> 172.21.15.64:0/2577669114 conn(0x56020ddcaa40 msgr2=0x56020ddcae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.816+0000 7f10a078d740 1 -- 172.21.15.64:0/2577669114 shutdown_connections 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.816+0000 7f10a078d740 1 -- 172.21.15.64:0/2577669114 wait complete. 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.816+0000 7f10a078d740 1 -- start start 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 Processor -- start 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddd9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddcfdc0 0x56020ddd98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddd4f30 0x56020dde0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56020ddd7ca0 con 0x56020ddcf3d0 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56020dcee790 con 0x56020ddd4f30 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56020ddd7e20 con 0x56020ddcfdc0 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f10a078d740 10 client.0 _ll_register_callbacks cb 0x56020dd29a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.817+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddd9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddcfdc0 0x56020ddd98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddd9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33982/0 (socket says 172.21.15.64:33982) 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f1096ffd640 1 -- 172.21.15.64:0/229605912 learned_addr learned my addr 172.21.15.64:0/229605912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f109d9a5640 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddd4f30 0x56020dde0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 -- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddd4f30 msgr2=0x56020dde0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddd4f30 0x56020dde0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 -- 172.21.15.64:0/229605912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 msgr2=0x56020ddd9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddd9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 -- 172.21.15.64:0/229605912 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020da47210 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f10967fc640 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddcfdc0 0x56020ddd98d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1090133b70 tx=0x7f1088011630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6594f12c94a0aab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.818+0000 7f1076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.819+0000 7f1076ffd640 1 -- 172.21.15.64:0/229605912 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088019070 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.819+0000 7f1076ffd640 1 -- 172.21.15.64:0/229605912 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1088015660 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.819+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10800f8d60 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.819+0000 7f1076ffd640 1 -- 172.21.15.64:0/229605912 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1088025050 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.819+0000 7f10a078d740 10 client.26392 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.819+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56020db13620 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f1076ffd640 1 -- 172.21.15.64:0/229605912 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10800f8d60 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f1076ffd640 1 -- 172.21.15.64:0/229605912 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f1088014b20 con 0x56020ddcfdc0 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f10a078d740 10 client.26392 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f10a078d740 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f10a078d740 20 client.26392 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f10a078d740 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020dde5170 0x56020dde75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f10a078d740 4 client.26392 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.820+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56020dd40a70 con 0x56020dde5170 2024-05-09T09:13:37.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:37 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.821+0000 7f1096ffd640 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020dde5170 0x56020dde75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.824+0000 7f1096ffd640 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020dde5170 0x56020dde75b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e745c7d591ed39 server_cookie=e6dd90019cc023e1 in_seq=0 out_seq=0 2024-05-09T09:13:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.824+0000 7f1076ffd640 10 client.26392 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.826+0000 7f1076ffd640 1 -- 172.21.15.64:0/229605912 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56020dd40a70 con 0x56020dde5170 2024-05-09T09:13:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.826+0000 7f1076ffd640 10 client.26392 handle_command_reply: tid=0 2024-05-09T09:13:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.826+0000 7f10a078d740 1 client.26392 shutdown 2024-05-09T09:13:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.827+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddcfdc0 msgr2=0x56020ddd98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.827+0000 7f10a078d740 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddcfdc0 0x56020ddd98d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1090133b70 tx=0x7f1088011630 comp rx=0 tx=0).stop 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 shutdown_connections 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020dde5170 0x56020dde75b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddd4f30 0x56020dde0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddcfdc0 0x56020ddd98d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 --2- 172.21.15.64:0/229605912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcf3d0 0x56020ddd9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 >> 172.21.15.64:0/229605912 conn(0x56020ddcaa40 msgr2=0x56020ddc9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f1076ffd640 0 client.26392 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.829+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 shutdown_connections 2024-05-09T09:13:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.830+0000 7f10a078d740 1 -- 172.21.15.64:0/229605912 wait complete. 2024-05-09T09:13:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.830+0000 7f10a078d740 20 client.26392 trim_cache size 0 max 16384 2024-05-09T09:13:37.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:37 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.857+0000 7f10a078d740 1 Processor -- start 2024-05-09T09:13:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.857+0000 7f10a078d740 1 -- start start 2024-05-09T09:13:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10a078d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcf3d0 0x56020ddcda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10a078d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dde08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10a078d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddcdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10a078d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10901478d0 con 0x56020ddcfdc0 2024-05-09T09:13:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10a078d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1090145110 con 0x56020ddcf3d0 2024-05-09T09:13:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10a078d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1090142f60 con 0x56020ddd4f30 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dde08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f109d9a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddcdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dde08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33998/0 (socket says 172.21.15.64:33998) 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 -- 172.21.15.64:0/2774066248 learned_addr learned my addr 172.21.15.64:0/2774066248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f1096ffd640 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcf3d0 0x56020ddcda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 -- 172.21.15.64:0/2774066248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 msgr2=0x56020ddcdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddcdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 -- 172.21.15.64:0/2774066248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcf3d0 msgr2=0x56020ddcda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcf3d0 0x56020ddcda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.858+0000 7f10967fc640 1 -- 172.21.15.64:0/2774066248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020ddda230 con 0x56020ddcfdc0 2024-05-09T09:13:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.859+0000 7f10967fc640 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dde08d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1088008010 tx=0x7f1088006f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381264639eb0ac13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.859+0000 7f108d7fa640 1 -- 172.21.15.64:0/2774066248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108802e070 con 0x56020ddcfdc0 2024-05-09T09:13:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.859+0000 7f108d7fa640 1 -- 172.21.15.64:0/2774066248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1088005b70 con 0x56020ddcfdc0 2024-05-09T09:13:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.859+0000 7f108d7fa640 1 -- 172.21.15.64:0/2774066248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088019040 con 0x56020ddcfdc0 2024-05-09T09:13:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.860+0000 7f10a078d740 1 -- 172.21.15.64:0/2774066248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 msgr2=0x56020dde08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.860+0000 7f10a078d740 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dde08d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1088008010 tx=0x7f1088006f30 comp rx=0 tx=0).stop 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.861+0000 7f10a078d740 1 -- 172.21.15.64:0/2774066248 shutdown_connections 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.861+0000 7f10a078d740 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddcdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.861+0000 7f10a078d740 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dde08d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.861+0000 7f10a078d740 1 --2- 172.21.15.64:0/2774066248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020ddcf3d0 0x56020ddcda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.861+0000 7f10a078d740 1 -- 172.21.15.64:0/2774066248 >> 172.21.15.64:0/2774066248 conn(0x56020ddcaa40 msgr2=0x56020ddd0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.861+0000 7f10a078d740 1 -- 172.21.15.64:0/2774066248 shutdown_connections 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.862+0000 7f10a078d740 1 -- 172.21.15.64:0/2774066248 wait complete. 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.863+0000 7f10a078d740 1 -- start start 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.863+0000 7f10a078d740 1 Processor -- start 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.864+0000 7f10a078d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dddd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f10a078d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f10a078d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020dddee70 0x56020dddf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f10a078d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10901435d0 con 0x56020ddcfdc0 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f10967fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f10967fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54028/0 (socket says 172.21.15.64:54028) 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f10967fc640 1 -- 172.21.15.64:0/3910932146 learned_addr learned my addr 172.21.15.64:0/3910932146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.865+0000 7f109d9a5640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020dddee70 0x56020dddf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1090144a40 con 0x56020dddee70 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f1096ffd640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dddd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1090143ca0 con 0x56020ddd4f30 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f10a078d740 10 client.0 _ll_register_callbacks cb 0x7f1088009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f109d9a5640 1 -- 172.21.15.64:0/3910932146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 msgr2=0x56020ddddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f109d9a5640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f109d9a5640 1 -- 172.21.15.64:0/3910932146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 msgr2=0x56020dddd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.866+0000 7f109d9a5640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dddd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.867+0000 7f109d9a5640 1 -- 172.21.15.64:0/3910932146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020dddf830 con 0x56020dddee70 2024-05-09T09:13:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.867+0000 7f109d9a5640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020dddee70 0x56020dddf2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f109804f8c0 tx=0x7f1098062db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd16d8fed9a805dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.867+0000 7f1076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.867+0000 7f1076ffd640 1 -- 172.21.15.64:0/3910932146 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109807fb60 con 0x56020dddee70 2024-05-09T09:13:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.868+0000 7f1076ffd640 1 -- 172.21.15.64:0/3910932146 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1098077040 con 0x56020dddee70 2024-05-09T09:13:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.868+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1088005b70 con 0x56020dddee70 2024-05-09T09:13:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.868+0000 7f1076ffd640 1 -- 172.21.15.64:0/3910932146 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f109807fd00 con 0x56020dddee70 2024-05-09T09:13:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f10a078d740 10 client.26234 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56020dde0bf0 con 0x56020dddee70 2024-05-09T09:13:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f1076ffd640 1 -- 172.21.15.64:0/3910932146 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1098083070 con 0x56020dddee70 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f1076ffd640 1 -- 172.21.15.64:0/3910932146 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f109808b560 con 0x56020dddee70 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f10a078d740 10 client.26234 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f10a078d740 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f10a078d740 20 client.26234 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.869+0000 7f10a078d740 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020de4be30 0x56020de4e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.870+0000 7f10a078d740 4 client.26234 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.870+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1088005b70 con 0x56020de4be30 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.870+0000 7f1096ffd640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020de4be30 0x56020de4e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.870+0000 7f1096ffd640 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020de4be30 0x56020de4e250 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496fae12605d1ac6 server_cookie=46ce84dca5b08c77 in_seq=0 out_seq=0 2024-05-09T09:13:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.870+0000 7f1076ffd640 10 client.26234 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.900+0000 7f1076ffd640 1 -- 172.21.15.64:0/3910932146 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1088005b70 con 0x56020de4be30 2024-05-09T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.901+0000 7f1076ffd640 10 client.26234 handle_command_reply: tid=0 2024-05-09T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.901+0000 7f10a078d740 1 client.26234 shutdown 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.902+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020dddee70 msgr2=0x56020dddf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.902+0000 7f10a078d740 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020dddee70 0x56020dddf2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f109804f8c0 tx=0x7f1098062db0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 shutdown_connections 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f10a078d740 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56020de4be30 0x56020de4e250 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f10a078d740 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56020dddee70 0x56020dddf2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f10a078d740 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56020ddd4f30 0x56020ddddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f10a078d740 1 --2- 172.21.15.64:0/3910932146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56020ddcfdc0 0x56020dddd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 >> 172.21.15.64:0/3910932146 conn(0x56020ddcaa40 msgr2=0x56020ddcda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.903+0000 7f1076ffd640 0 client.26234 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.904+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 shutdown_connections 2024-05-09T09:13:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.904+0000 7f10a078d740 1 -- 172.21.15.64:0/3910932146 wait complete. 2024-05-09T09:13:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.904+0000 7f10a078d740 20 client.26234 trim_cache size 0 max 16384 2024-05-09T09:13:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:13:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:13:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:13:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10840a2d10 msgr2=0x7f10840a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10840a2d10 0x7f10840a51d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f10980674e0 tx=0x7f1098066fd0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090094030 msgr2=0x7f1090132660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090094030 0x7f1090132660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f108002aa50 tx=0x7f1080003cd0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 shutdown_connections 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f10840a2d10 0x7f10840a51d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10900979c0 0x7f1090139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.907+0000 7f1097fff640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1090097000 0x7f1090132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.908+0000 7f1097fff640 1 --2- 172.21.15.64:0/1218268943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1090094030 0x7f1090132660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.908+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 >> 172.21.15.64:0/1218268943 conn(0x7f109008f940 msgr2=0x7f10900a22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.909+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 shutdown_connections 2024-05-09T09:13:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:37.909+0000 7f1097fff640 1 -- 172.21.15.64:0/1218268943 wait complete. 2024-05-09T09:13:37.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:37 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 -- 172.21.15.64:0/1966538948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 msgr2=0x7fca70007040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 --2- 172.21.15.64:0/1966538948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 0x7fca70007040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fca78067530 tx=0x7fca78096080 comp rx=0 tx=0).stop 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 -- 172.21.15.64:0/1966538948 shutdown_connections 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 --2- 172.21.15.64:0/1966538948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca700a2350 0x7fca700a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 --2- 172.21.15.64:0/1966538948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca70007610 0x7fca70007a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 --2- 172.21.15.64:0/1966538948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 0x7fca70007040 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.005+0000 7fca7f451640 1 -- 172.21.15.64:0/1966538948 >> 172.21.15.64:0/1966538948 conn(0x7fca700912f0 msgr2=0x7fca70093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.006+0000 7fca7f451640 1 -- 172.21.15.64:0/1966538948 shutdown_connections 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.006+0000 7fca7f451640 1 -- 172.21.15.64:0/1966538948 wait complete. 2024-05-09T09:13:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 Processor -- start 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 -- start start 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 0x7fca70143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 0x7fca70143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca700a2350 0x7fca7014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca700b8e40 con 0x7fca70008bf0 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca700b8cc0 con 0x7fca700a2350 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7f451640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca700b8fc0 con 0x7fca70007610 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7e44f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 0x7fca70143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7e44f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 0x7fca70143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54046/0 (socket says 172.21.15.64:54046) 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7e44f640 1 -- 172.21.15.64:0/3938801097 learned_addr learned my addr 172.21.15.64:0/3938801097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 0x7fca70143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.008+0000 7fca7ec50640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca700a2350 0x7fca7014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca7e44f640 1 -- 172.21.15.64:0/3938801097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca700a2350 msgr2=0x7fca7014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca7e44f640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca700a2350 0x7fca7014ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca7e44f640 1 -- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 msgr2=0x7fca70143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca7e44f640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 0x7fca70143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca7e44f640 1 -- 172.21.15.64:0/3938801097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7014b270 con 0x7fca70007610 2024-05-09T09:13:38.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca7e44f640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 0x7fca70143650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca78000b60 tx=0x7fca78098bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.009+0000 7fca6f7fe640 1 -- 172.21.15.64:0/3938801097 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78098ed0 con 0x7fca70007610 2024-05-09T09:13:38.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.010+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7014b500 con 0x7fca70007610 2024-05-09T09:13:38.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.010+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7014ba10 con 0x7fca70007610 2024-05-09T09:13:38.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.010+0000 7fca6f7fe640 1 -- 172.21.15.64:0/3938801097 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca780985c0 con 0x7fca70007610 2024-05-09T09:13:38.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.010+0000 7fca6f7fe640 1 -- 172.21.15.64:0/3938801097 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78075530 con 0x7fca70007610 2024-05-09T09:13:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.013+0000 7fca833c1740 1 Processor -- start 2024-05-09T09:13:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.013+0000 7fca833c1740 1 -- start start 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca833c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a91ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca833c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4a96ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca833c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca833c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591c49f5990 con 0x5591c4a916b0 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca833c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5591c499d5b0 con 0x5591c4a920a0 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca833c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5591c49f1950 con 0x5591c4a97210 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca6f7fe640 1 -- 172.21.15.64:0/3938801097 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca780a6060 con 0x7fca70007610 2024-05-09T09:13:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca7dc4e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca7dc4e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54048/0 (socket says 172.21.15.64:54048) 2024-05-09T09:13:38.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2941966146 learned_addr learned my addr 172.21.15.64:0/2941966146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca7e44f640 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4a96ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.014+0000 7fca7ec50640 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a91ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2941966146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 msgr2=0x5591c4a96ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4a96ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2941966146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 msgr2=0x5591c4a91ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a91ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2941966146 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c47d17e0 con 0x5591c4a97210 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7ec50640 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a91ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a996a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca7400bc80 tx=0x7fca7400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39aa515a52dd921c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca6f7fe640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca640a2ce0 0x7fca640a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.015+0000 7fca6cff9640 1 -- 172.21.15.64:0/2941966146 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74015070 con 0x5591c4a97210 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.016+0000 7fca6cff9640 1 -- 172.21.15.64:0/2941966146 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca7401a910 con 0x5591c4a97210 2024-05-09T09:13:38.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.016+0000 7fca6cff9640 1 -- 172.21.15.64:0/2941966146 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca740118d0 con 0x5591c4a97210 2024-05-09T09:13:38.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.016+0000 7fca6f7fe640 1 -- 172.21.15.64:0/3938801097 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca78152c60 con 0x7fca70007610 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.016+0000 7fca7ec50640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca640a2ce0 0x7fca640a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 -- 172.21.15.64:0/2941966146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 msgr2=0x5591c4a996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a996a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca7400bc80 tx=0x7fca7400f730 comp rx=0 tx=0).stop 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 -- 172.21.15.64:0/2941966146 shutdown_connections 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a996a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4a96ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca7ec50640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca640a2ce0 0x7fca640a51a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5591c4a92900 tx=0x7fca6800b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 --2- 172.21.15.64:0/2941966146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a91ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.017+0000 7fca833c1740 1 -- 172.21.15.64:0/2941966146 >> 172.21.15.64:0/2941966146 conn(0x5591c4a590a0 msgr2=0x5591c4a8d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.018+0000 7fca833c1740 1 -- 172.21.15.64:0/2941966146 shutdown_connections 2024-05-09T09:13:38.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.018+0000 7fca833c1740 1 -- 172.21.15.64:0/2941966146 wait complete. 2024-05-09T09:13:38.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.019+0000 7fca833c1740 1 -- start start 2024-05-09T09:13:38.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.019+0000 7fca833c1740 1 Processor -- start 2024-05-09T09:13:38.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a9b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca7e44f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a9b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca7e44f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a9b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34028/0 (socket says 172.21.15.64:34028) 2024-05-09T09:13:38.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a920a0 0x5591c4a9be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca7e44f640 1 -- 172.21.15.64:0/742971173 learned_addr learned my addr 172.21.15.64:0/742971173 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a97210 0x5591c4aa2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591c4a9a1c0 con 0x5591c4a916b0 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5591c49b0b40 con 0x5591c4a97210 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5591c4a9a340 con 0x5591c4a920a0 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.020+0000 7fca833c1740 10 client.0 _ll_register_callbacks cb 0x5591c49a50f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a920a0 0x5591c4a9be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7e44f640 1 -- 172.21.15.64:0/742971173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a920a0 msgr2=0x5591c4a9be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7e44f640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a920a0 0x5591c4a9be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7e44f640 1 -- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a97210 msgr2=0x5591c4aa2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7ec50640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a97210 0x5591c4aa2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7e44f640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a97210 0x5591c4aa2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7e44f640 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c4831f70 con 0x5591c4a916b0 2024-05-09T09:13:38.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a920a0 0x5591c4a9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:38.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7ec50640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a97210 0x5591c4aa2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:38.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca7e44f640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a9b930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fca78050180 tx=0x7fca78156160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8b211bf02cadf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.021+0000 7fca61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.022+0000 7fca61ffb640 1 -- 172.21.15.64:0/742971173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7811b3c0 con 0x5591c4a916b0 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.023+0000 7fca61ffb640 1 -- 172.21.15.64:0/742971173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca780756d0 con 0x5591c4a916b0 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.023+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7401a910 con 0x5591c4a916b0 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.023+0000 7fca61ffb640 1 -- 172.21.15.64:0/742971173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fca7811d2f0 con 0x5591c4a916b0 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.023+0000 7fca833c1740 10 client.16734 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.023+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5591c46c9170 con 0x5591c4a916b0 2024-05-09T09:13:38.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.024+0000 7fca61ffb640 1 -- 172.21.15.64:0/742971173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca7401a910 con 0x5591c4a916b0 2024-05-09T09:13:38.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.025+0000 7fca61ffb640 1 -- 172.21.15.64:0/742971173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fca78067ce0 con 0x5591c4a916b0 2024-05-09T09:13:38.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.026+0000 7fca833c1740 10 client.16734 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:38.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.026+0000 7fca833c1740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:38.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.026+0000 7fca833c1740 20 client.16734 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:38.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.026+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4aa7460 0x5591c4aa9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.026+0000 7fca833c1740 4 client.16734 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:38.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.026+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c4a030c0 con 0x5591c4aa7460 2024-05-09T09:13:38.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.027+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4aa7460 0x5591c4aa9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.028+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4aa7460 0x5591c4aa9900 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4036cedc9be0ca3 server_cookie=42a3da551c1869e in_seq=0 out_seq=0 2024-05-09T09:13:38.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.028+0000 7fca61ffb640 10 client.16734 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.030+0000 7fca61ffb640 1 -- 172.21.15.64:0/742971173 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591c4a030c0 con 0x5591c4aa7460 2024-05-09T09:13:38.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.030+0000 7fca61ffb640 10 client.16734 handle_command_reply: tid=0 2024-05-09T09:13:38.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.030+0000 7fca833c1740 1 client.16734 shutdown 2024-05-09T09:13:38.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 msgr2=0x5591c4a9b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a9b930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fca78050180 tx=0x7fca78156160 comp rx=0 tx=0).stop 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 shutdown_connections 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4aa7460 0x5591c4aa9900 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a97210 0x5591c4aa2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a920a0 0x5591c4a9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca61ffb640 0 client.16734 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 --2- 172.21.15.64:0/742971173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a916b0 0x5591c4a9b930 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.031+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 >> 172.21.15.64:0/742971173 conn(0x5591c4a590a0 msgr2=0x5591c4a8c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.032+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 shutdown_connections 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.032+0000 7fca833c1740 1 -- 172.21.15.64:0/742971173 wait complete. 2024-05-09T09:13:38.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.032+0000 7fca833c1740 20 client.16734 trim_cache size 0 max 16384 2024-05-09T09:13:38.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 Processor -- start 2024-05-09T09:13:38.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 -- start start 2024-05-09T09:13:38.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a916b0 0x5591c4a90020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 0x5591c4aa2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a90560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c0123b0 con 0x5591c4a920a0 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c00fbf0 con 0x5591c4a916b0 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.067+0000 7fca833c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c00daf0 con 0x5591c4a97210 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 0x5591c4aa2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7ec50640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a90560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 0x5591c4aa2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34036/0 (socket says 172.21.15.64:34036) 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2122608225 learned_addr learned my addr 172.21.15.64:0/2122608225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7ec50640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a90560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54072/0 (socket says 172.21.15.64:54072) 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7e44f640 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a916b0 0x5591c4a90020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2122608225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 msgr2=0x5591c4a90560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a90560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2122608225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a916b0 msgr2=0x5591c4a90020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a916b0 0x5591c4a90020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.068+0000 7fca7dc4e640 1 -- 172.21.15.64:0/2122608225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c4a92480 con 0x5591c4a920a0 2024-05-09T09:13:38.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.069+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 0x5591c4aa2e70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5591c4a8dfe0 tx=0x7fca740041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8994b7d2e1f1b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.069+0000 7fca63fff640 1 -- 172.21.15.64:0/2122608225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74015040 con 0x5591c4a920a0 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.069+0000 7fca63fff640 1 -- 172.21.15.64:0/2122608225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca7400b520 con 0x5591c4a920a0 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.069+0000 7fca63fff640 1 -- 172.21.15.64:0/2122608225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74004970 con 0x5591c4a920a0 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 -- 172.21.15.64:0/2122608225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 msgr2=0x5591c4aa2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 0x5591c4aa2e70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5591c4a8dfe0 tx=0x7fca740041d0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 -- 172.21.15.64:0/2122608225 shutdown_connections 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a97210 0x5591c4a90560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a920a0 0x5591c4aa2e70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 --2- 172.21.15.64:0/2122608225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a916b0 0x5591c4a90020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.070+0000 7fca833c1740 1 -- 172.21.15.64:0/2122608225 >> 172.21.15.64:0/2122608225 conn(0x5591c4a590a0 msgr2=0x5591c4a8cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.071+0000 7fca833c1740 1 -- 172.21.15.64:0/2122608225 shutdown_connections 2024-05-09T09:13:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.071+0000 7fca833c1740 1 -- 172.21.15.64:0/2122608225 wait complete. 2024-05-09T09:13:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.071+0000 7fca833c1740 1 -- start start 2024-05-09T09:13:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.071+0000 7fca833c1740 1 Processor -- start 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 0x5591c4aa0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4aa0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a97210 0x5591c4aa1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c00e0b0 con 0x5591c4a97210 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c00f520 con 0x5591c4a920a0 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 0x5591c4aa0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c00e780 con 0x5591c4a916b0 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 0x5591c4aa0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54074/0 (socket says 172.21.15.64:54074) 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 -- 172.21.15.64:0/453807639 learned_addr learned my addr 172.21.15.64:0/453807639 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7ec50640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a97210 0x5591c4aa1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4aa0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca833c1740 10 client.0 _ll_register_callbacks cb 0x7fca780984d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 -- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 msgr2=0x5591c4aa0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4aa0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 -- 172.21.15.64:0/453807639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a97210 msgr2=0x5591c4aa1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a97210 0x5591c4aa1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7e44f640 1 -- 172.21.15.64:0/453807639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c4aa1f20 con 0x5591c4a916b0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.072+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4aa0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca7e44f640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 0x5591c4aa0430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca78050860 tx=0x7fca7811d6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4424ac5df6cf39ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca61ffb640 1 -- 172.21.15.64:0/453807639 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7811eb10 con 0x5591c4a916b0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca61ffb640 1 -- 172.21.15.64:0/453807639 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca780756d0 con 0x5591c4a916b0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7400b520 con 0x5591c4a916b0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca61ffb640 1 -- 172.21.15.64:0/453807639 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fca7811ecf0 con 0x5591c4a916b0 2024-05-09T09:13:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca833c1740 10 client.26413 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.073+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5591c4b0da80 con 0x5591c4a916b0 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca61ffb640 1 -- 172.21.15.64:0/453807639 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca7400b520 con 0x5591c4a916b0 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca61ffb640 1 -- 172.21.15.64:0/453807639 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fca78067ce0 con 0x5591c4a916b0 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca833c1740 10 client.26413 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca833c1740 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca833c1740 20 client.26413 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca833c1740 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4b0e0a0 0x5591c4b10560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca833c1740 4 client.26413 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:13:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.074+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5591c49fd410 con 0x5591c4b0e0a0 2024-05-09T09:13:38.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.075+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4b0e0a0 0x5591c4b10560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.076+0000 7fca7dc4e640 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4b0e0a0 0x5591c4b10560 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8d7690b892e718 server_cookie=8dad9b84cac66051 in_seq=0 out_seq=0 2024-05-09T09:13:38.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.076+0000 7fca61ffb640 10 client.26413 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.078+0000 7fca61ffb640 1 -- 172.21.15.64:0/453807639 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591c49fd410 con 0x5591c4b0e0a0 2024-05-09T09:13:38.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.078+0000 7fca61ffb640 10 client.26413 handle_command_reply: tid=0 2024-05-09T09:13:38.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.078+0000 7fca833c1740 1 client.26413 shutdown 2024-05-09T09:13:38.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.079+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 msgr2=0x5591c4aa0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.079+0000 7fca833c1740 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 0x5591c4aa0430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca78050860 tx=0x7fca7811d6c0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.079+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 shutdown_connections 2024-05-09T09:13:38.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.079+0000 7fca833c1740 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5591c4b0e0a0 0x5591c4b10560 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.080+0000 7fca833c1740 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591c4a97210 0x5591c4aa1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.080+0000 7fca833c1740 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5591c4a920a0 0x5591c4aa0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.080+0000 7fca833c1740 1 --2- 172.21.15.64:0/453807639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591c4a916b0 0x5591c4aa0430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.080+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 >> 172.21.15.64:0/453807639 conn(0x5591c4a590a0 msgr2=0x5591c4a90020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.080+0000 7fca61ffb640 0 client.26413 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.082+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 shutdown_connections 2024-05-09T09:13:38.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.082+0000 7fca833c1740 1 -- 172.21.15.64:0/453807639 wait complete. 2024-05-09T09:13:38.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.083+0000 7fca833c1740 20 client.26413 trim_cache size 0 max 16384 2024-05-09T09:13:38.082 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:13:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:13:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "c443843f-a0cb-43ba-9a7f-d2a19e7b15f9", 2024-05-09T09:13:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:13:38.082 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:13:38.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca640a2ce0 msgr2=0x7fca640a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca640a2ce0 0x7fca640a51a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5591c4a92900 tx=0x7fca6800b2e0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 msgr2=0x7fca70143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 0x7fca70143650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca78000b60 tx=0x7fca78098bd0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 shutdown_connections 2024-05-09T09:13:38.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca640a2ce0 0x7fca640a51a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca700a2350 0x7fca7014ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca70008bf0 0x7fca70143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 --2- 172.21.15.64:0/3938801097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca70007610 0x7fca70143650 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.084+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 >> 172.21.15.64:0/3938801097 conn(0x7fca700912f0 msgr2=0x7fca700956f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.085+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 shutdown_connections 2024-05-09T09:13:38.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.085+0000 7fca7f451640 1 -- 172.21.15.64:0/3938801097 wait complete. 2024-05-09T09:13:38.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:13:38.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:13:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.352+0000 7f0de1bc1640 1 -- 172.21.15.64:0/1914301025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc104c80 msgr2=0x7f0ddc105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.352+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/1914301025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc104c80 0x7f0ddc105100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009860 tx=0x7f0dc402a340 comp rx=0 tx=0).stop 2024-05-09T09:13:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 -- 172.21.15.64:0/1914301025 shutdown_connections 2024-05-09T09:13:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/1914301025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc10bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/1914301025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc104c80 0x7f0ddc105100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/1914301025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc103a90 0x7f0ddc103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 -- 172.21.15.64:0/1914301025 >> 172.21.15.64:0/1914301025 conn(0x7f0ddc0ff020 msgr2=0x7f0ddc101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 -- 172.21.15.64:0/1914301025 shutdown_connections 2024-05-09T09:13:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.353+0000 7f0de1bc1640 1 -- 172.21.15.64:0/1914301025 wait complete. 2024-05-09T09:13:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.354+0000 7f0de1bc1640 1 Processor -- start 2024-05-09T09:13:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 -- start start 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc103a90 0x7f0ddc110c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc104c80 0x7f0ddc111140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc114190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc077900 con 0x7f0ddc104c80 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc077780 con 0x7f0ddc105640 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0de1bc1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc077a80 con 0x7f0ddc103a90 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc114190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc114190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37462/0 (socket says 172.21.15.64:37462) 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2566098749 learned_addr learned my addr 172.21.15.64:0/2566098749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc104c80 0x7f0ddc111140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2566098749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc103a90 msgr2=0x7f0ddc110c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc103a90 0x7f0ddc110c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc104c80 msgr2=0x7f0ddc111140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc104c80 0x7f0ddc111140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.355+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2566098749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ddc114800 con 0x7f0ddc105640 2024-05-09T09:13:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.358+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc114190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400d730 tx=0x7f0dd400dbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.359+0000 7f0dd8ff9640 1 -- 172.21.15.64:0/2566098749 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4028070 con 0x7f0ddc105640 2024-05-09T09:13:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.359+0000 7f0dd8ff9640 1 -- 172.21.15.64:0/2566098749 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd400de40 con 0x7f0ddc105640 2024-05-09T09:13:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.359+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ddc114af0 con 0x7f0ddc105640 2024-05-09T09:13:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.359+0000 7f0dd8ff9640 1 -- 172.21.15.64:0/2566098749 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4004cf0 con 0x7f0ddc105640 2024-05-09T09:13:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.359+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ddc115010 con 0x7f0ddc105640 2024-05-09T09:13:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.363+0000 7f0de49ad740 1 Processor -- start 2024-05-09T09:13:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0dd8ff9640 1 -- 172.21.15.64:0/2566098749 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0dd4021470 con 0x7f0ddc105640 2024-05-09T09:13:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 -- start start 2024-05-09T09:13:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaf9f30 0x56192eafa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 0x56192eaff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb01f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56192ea5ed00 con 0x56192eaffa90 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56192ea064c0 con 0x56192eafa920 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0de49ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56192ea5acc0 con 0x56192eaf9f30 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0ddaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaf9f30 0x56192eafa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 0x56192eaff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 0x56192eaff420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37468/0 (socket says 172.21.15.64:37468) 2024-05-09T09:13:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.364+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/445623620 learned_addr learned my addr 172.21.15.64:0/445623620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb01f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/445623620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaf9f30 msgr2=0x56192eafa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaf9f30 0x56192eafa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/445623620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 msgr2=0x56192eb01f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb01f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/445623620 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192e83c7d0 con 0x56192eafa920 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb01f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 0x56192eaff420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009e30 tx=0x7f0dcc00e760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a02f6922909b130 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.365+0000 7f0dd8ff9640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0dc80a2c60 0x7f0dc80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.366+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/445623620 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00f810 con 0x56192eafa920 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.366+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/445623620 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dcc0173f0 con 0x56192eafa920 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.366+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/445623620 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00fe60 con 0x56192eafa920 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.366+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0dc80a2c60 0x7f0dc80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.366+0000 7f0dd8ff9640 1 -- 172.21.15.64:0/2566098749 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0dd40dfe00 con 0x7f0ddc105640 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 -- 172.21.15.64:0/445623620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 msgr2=0x56192eaff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 0x56192eaff420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009e30 tx=0x7f0dcc00e760 comp rx=0 tx=0).stop 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 -- 172.21.15.64:0/445623620 shutdown_connections 2024-05-09T09:13:38.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb01f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eafa920 0x56192eaff420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 --2- 172.21.15.64:0/445623620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaf9f30 0x56192eafa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 -- 172.21.15.64:0/445623620 >> 172.21.15.64:0/445623620 conn(0x56192eaf55a0 msgr2=0x56192eaf59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0dc80a2c60 0x7f0dc80a5120 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c00 tx=0x7f0dc402a340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.367+0000 7f0de49ad740 1 -- 172.21.15.64:0/445623620 shutdown_connections 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.368+0000 7f0de49ad740 1 -- 172.21.15.64:0/445623620 wait complete. 2024-05-09T09:13:38.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.368+0000 7f0de49ad740 1 -- start start 2024-05-09T09:13:38.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.368+0000 7f0de49ad740 1 Processor -- start 2024-05-09T09:13:38.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eafa920 0x56192eb04720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 0x56192eb0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56192eb02a40 con 0x56192eafa920 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56192ea19b90 con 0x56192eaf9f30 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56192eb02bc0 con 0x56192eaffa90 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0de49ad740 10 client.0 _ll_register_callbacks cb 0x56192ea55da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0ddbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 0x56192eb0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0ddbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 0x56192eb0b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54088/0 (socket says 172.21.15.64:54088) 2024-05-09T09:13:38.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0ddbfff640 1 -- 172.21.15.64:0/455721223 learned_addr learned my addr 172.21.15.64:0/455721223 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.369+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eafa920 0x56192eb04720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.370+0000 7f0ddbfff640 1 -- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 msgr2=0x56192eb041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.370+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.370+0000 7f0ddbfff640 1 -- 172.21.15.64:0/455721223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eafa920 msgr2=0x56192eb04720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.371+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eafa920 0x56192eb04720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.371+0000 7f0ddbfff640 1 -- 172.21.15.64:0/455721223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192e7723f0 con 0x56192eaffa90 2024-05-09T09:13:38.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.371+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:38.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.371+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eafa920 0x56192eb04720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:13:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.374+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 0x56192eb0b760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40abfd0 tx=0x7f0dd4009cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3466b2d5b5b797 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.374+0000 7f0dbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.374+0000 7f0dbaffd640 1 -- 172.21.15.64:0/455721223 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4031070 con 0x56192eaffa90 2024-05-09T09:13:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.375+0000 7f0dbaffd640 1 -- 172.21.15.64:0/455721223 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0dd40a8940 con 0x56192eaffa90 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.375+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dcc0173f0 con 0x56192eaffa90 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0dbaffd640 1 -- 172.21.15.64:0/455721223 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0dd40b0680 con 0x56192eaffa90 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 10 client.26422 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56192e7320f0 con 0x56192eaffa90 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0dbaffd640 1 -- 172.21.15.64:0/455721223 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd40b4f40 con 0x56192eaffa90 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0dbaffd640 1 -- 172.21.15.64:0/455721223 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f0dd40b0120 con 0x56192eaffa90 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 10 client.26422 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 20 client.26422 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb12450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 4 client.26422 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.376+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0dcc0173f0 con 0x56192eb10060 2024-05-09T09:13:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.377+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb12450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.378+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb12450 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74fc6c477c61a369 server_cookie=46498edef560b828 in_seq=0 out_seq=0 2024-05-09T09:13:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.378+0000 7f0dbaffd640 10 client.26422 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.379+0000 7f0dbaffd640 1 -- 172.21.15.64:0/455721223 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0dcc0173f0 con 0x56192eb10060 2024-05-09T09:13:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.379+0000 7f0dbaffd640 10 client.26422 handle_command_reply: tid=0 2024-05-09T09:13:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.379+0000 7f0de49ad740 1 client.26422 shutdown 2024-05-09T09:13:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.379+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 msgr2=0x56192eb0b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.379+0000 7f0de49ad740 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 0x56192eb0b760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40abfd0 tx=0x7f0dd4009cf0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 shutdown_connections 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb12450 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0dbaffd640 0 client.26422 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eaffa90 0x56192eb0b760 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eafa920 0x56192eb04720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 --2- 172.21.15.64:0/455721223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 >> 172.21.15.64:0/455721223 conn(0x56192eaf55a0 msgr2=0x56192eaf47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 shutdown_connections 2024-05-09T09:13:38.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.380+0000 7f0de49ad740 1 -- 172.21.15.64:0/455721223 wait complete. 2024-05-09T09:13:38.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.381+0000 7f0de49ad740 20 client.26422 trim_cache size 0 max 16384 2024-05-09T09:13:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.418+0000 7f0de49ad740 1 Processor -- start 2024-05-09T09:13:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.418+0000 7f0de49ad740 1 -- start start 2024-05-09T09:13:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaf9f30 0x56192eaff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb0b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaffa90 0x56192eaf84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0de49ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1b33c0 con 0x56192eaf9f30 2024-05-09T09:13:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0de49ad740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1b0c00 con 0x56192eaffa90 2024-05-09T09:13:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0de49ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc0725f0 con 0x56192eafa920 2024-05-09T09:13:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaf9f30 0x56192eaff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaf9f30 0x56192eaff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34066/0 (socket says 172.21.15.64:34066) 2024-05-09T09:13:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/2023510800 learned_addr learned my addr 172.21.15.64:0/2023510800 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaffa90 0x56192eaf84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb0b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2023510800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 msgr2=0x56192eb0b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb0b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2023510800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaf9f30 msgr2=0x56192eaff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.419+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaf9f30 0x56192eaff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.420+0000 7f0ddbfff640 1 -- 172.21.15.64:0/2023510800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192eaf9a90 con 0x56192eaffa90 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.420+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaffa90 0x56192eaf84b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40dd5c0 tx=0x7f0dd40a84a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18aa49ee450f48c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.420+0000 7f0dd17fa640 1 -- 172.21.15.64:0/2023510800 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4031070 con 0x56192eaffa90 2024-05-09T09:13:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.420+0000 7f0dd17fa640 1 -- 172.21.15.64:0/2023510800 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd40a8270 con 0x56192eaffa90 2024-05-09T09:13:38.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.420+0000 7f0dd17fa640 1 -- 172.21.15.64:0/2023510800 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40b0280 con 0x56192eaffa90 2024-05-09T09:13:38.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 -- 172.21.15.64:0/2023510800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaffa90 msgr2=0x56192eaf84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaffa90 0x56192eaf84b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40dd5c0 tx=0x7f0dd40a84a0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 -- 172.21.15.64:0/2023510800 shutdown_connections 2024-05-09T09:13:38.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaffa90 0x56192eaf84b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb0b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 --2- 172.21.15.64:0/2023510800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaf9f30 0x56192eaff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.421+0000 7f0de49ad740 1 -- 172.21.15.64:0/2023510800 >> 172.21.15.64:0/2023510800 conn(0x56192eaf55a0 msgr2=0x56192eafbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.422+0000 7f0de49ad740 1 -- 172.21.15.64:0/2023510800 shutdown_connections 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.422+0000 7f0de49ad740 1 -- 172.21.15.64:0/2023510800 wait complete. 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.422+0000 7f0de49ad740 1 -- start start 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 Processor -- start 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb09330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb09870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb0a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc072c60 con 0x56192eaffa90 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1b0530 con 0x56192eaf9f30 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc073330 con 0x56192eafa920 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0de49ad740 10 client.0 _ll_register_callbacks cb 0x7f0dd4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0ddaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb09870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0ddaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb09870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54100/0 (socket says 172.21.15.64:54100) 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb09330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.423+0000 7f0ddaffd640 1 -- 172.21.15.64:0/3861561958 learned_addr learned my addr 172.21.15.64:0/3861561958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:38.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddbfff640 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb0a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/3861561958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 msgr2=0x56192eb09870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb09870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/3861561958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 msgr2=0x56192eb0a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb0a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddb7fe640 1 -- 172.21.15.64:0/3861561958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192eb0ae70 con 0x56192eaf9f30 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0ddb7fe640 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb09330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc000f80 tx=0x7f0dcc009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b65843a1e03ef3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.424+0000 7f0dbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.425+0000 7f0dbaffd640 1 -- 172.21.15.64:0/3861561958 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc017e60 con 0x56192eaf9f30 2024-05-09T09:13:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.425+0000 7f0dbaffd640 1 -- 172.21.15.64:0/3861561958 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0dcc029b50 con 0x56192eaf9f30 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.425+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd40a8270 con 0x56192eaf9f30 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.425+0000 7f0dbaffd640 1 -- 172.21.15.64:0/3861561958 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0dcc03a050 con 0x56192eaf9f30 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.425+0000 7f0de49ad740 10 client.26267 fetch_fsmap learned FSMap version 37 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.425+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56192eb76a50 con 0x56192eaf9f30 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.426+0000 7f0dbaffd640 1 -- 172.21.15.64:0/3861561958 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd40a8270 con 0x56192eaf9f30 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0dbaffd640 1 -- 172.21.15.64:0/3861561958 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f0dcc00b030 con 0x56192eaf9f30 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0de49ad740 10 client.26267 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T09:13:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0de49ad740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:38.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0de49ad740 20 client.26267 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:38.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0de49ad740 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:38.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0de49ad740 4 client.26267 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:13:38.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.427+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56192ea6c430 con 0x56192eb10060 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.428+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.429+0000 7f0ddaffd640 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb791b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0b68fdbc08c00e server_cookie=b84c4ff0d2fa38e6 in_seq=0 out_seq=0 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.429+0000 7f0dbaffd640 10 client.26267 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.429+0000 7f0dbaffd640 1 -- 172.21.15.64:0/3861561958 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56192ea6c430 con 0x56192eb10060 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.429+0000 7f0dbaffd640 10 client.26267 handle_command_reply: tid=0 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.429+0000 7f0de49ad740 1 client.26267 shutdown 2024-05-09T09:13:38.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 msgr2=0x56192eb09330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb09330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc000f80 tx=0x7f0dcc009280 comp rx=0 tx=0).stop 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 shutdown_connections 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56192eb10060 0x56192eb791b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56192eaffa90 0x56192eb0a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56192eafa920 0x56192eb09870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 --2- 172.21.15.64:0/3861561958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56192eaf9f30 0x56192eb09330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0dbaffd640 0 client.26267 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 >> 172.21.15.64:0/3861561958 conn(0x56192eaf55a0 msgr2=0x56192eaf3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 shutdown_connections 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 1 -- 172.21.15.64:0/3861561958 wait complete. 2024-05-09T09:13:38.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.430+0000 7f0de49ad740 20 client.26267 trim_cache size 0 max 16384 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "c443843f-a0cb-43ba-9a7f-d2a19e7b15f9": { 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "c443843f-a0cb-43ba-9a7f-d2a19e7b15f9", 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:13:38.435 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0dc80a2c60 msgr2=0x7f0dc80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0dc80a2c60 0x7f0dc80a5120 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c00 tx=0x7f0dc402a340 comp rx=0 tx=0).stop 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 msgr2=0x7f0ddc114190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc114190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400d730 tx=0x7f0dd400dbf0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 shutdown_connections 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0dc80a2c60 0x7f0dc80a5120 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ddc105640 0x7f0ddc114190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc104c80 0x7f0ddc111140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.435+0000 7f0de1bc1640 1 --2- 172.21.15.64:0/2566098749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ddc103a90 0x7f0ddc110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.436+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 >> 172.21.15.64:0/2566098749 conn(0x7f0ddc0ff020 msgr2=0x7f0ddc0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.436+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 shutdown_connections 2024-05-09T09:13:38.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:13:38.437+0000 7f0de1bc1640 1 -- 172.21.15.64:0/2566098749 wait complete. 2024-05-09T09:13:38.468 INFO:tasks.cephfs.filesystem:scrub status for tag:c443843f-a0cb-43ba-9a7f-d2a19e7b15f9 - {'path': '/', 'tag': 'c443843f-a0cb-43ba-9a7f-d2a19e7b15f9', 'options': 'recursive,force'} 2024-05-09T09:13:38.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 897 op/s 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:38.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:38 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 897 op/s 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:38 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 897 op/s 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:38 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:39.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:39 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:13:39.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:39 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:39.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:39 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:39.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:39 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:13:39.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:39 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:39.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:39 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:39.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:39 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:13:39.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:39 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:39.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:39 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:13:40.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:40 smithi139 ceph-mon[33599]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-05-09T09:13:40.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:40 smithi064 ceph-mon[30149]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-05-09T09:13:40.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:40 smithi179 ceph-mon[34086]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-05-09T09:13:41.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 200.0 s, 943.4 tps, lat 106.320 ms stddev 49.902 2024-05-09T09:13:41.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:41 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:41.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:41 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:41.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:41 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:42.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:42 smithi064 ceph-mon[30149]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.8 MiB/s wr, 975 op/s 2024-05-09T09:13:42.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:42 smithi179 ceph-mon[34086]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.8 MiB/s wr, 975 op/s 2024-05-09T09:13:43.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:42 smithi139 ceph-mon[33599]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.8 MiB/s wr, 975 op/s 2024-05-09T09:13:44.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:44 smithi064 ceph-mon[30149]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.4 MiB/s wr, 1.09k op/s 2024-05-09T09:13:44.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:44 smithi179 ceph-mon[34086]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.4 MiB/s wr, 1.09k op/s 2024-05-09T09:13:45.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:44 smithi139 ceph-mon[33599]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.4 MiB/s wr, 1.09k op/s 2024-05-09T09:13:45.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:45 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:13:45.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:45 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:13:45.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:45 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:13:45.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:45 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:13:46.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:45 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:13:46.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:45 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:13:46.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 205.0 s, 966.8 tps, lat 103.672 ms stddev 110.637 2024-05-09T09:13:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:46 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:13:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:46 smithi064 ceph-mon[30149]: pgmap v509: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.2 MiB/s wr, 1.14k op/s 2024-05-09T09:13:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:46 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:46 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:13:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:46 smithi179 ceph-mon[34086]: pgmap v509: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.2 MiB/s wr, 1.14k op/s 2024-05-09T09:13:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:46 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:47.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:46 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:13:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:46 smithi139 ceph-mon[33599]: pgmap v509: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.2 MiB/s wr, 1.14k op/s 2024-05-09T09:13:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:46 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:13:48.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:48 smithi064 ceph-mon[30149]: pgmap v510: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.9 MiB/s wr, 966 op/s 2024-05-09T09:13:48.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:48 smithi179 ceph-mon[34086]: pgmap v510: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.9 MiB/s wr, 966 op/s 2024-05-09T09:13:49.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:48 smithi139 ceph-mon[33599]: pgmap v510: 129 pgs: 129 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.9 MiB/s wr, 966 op/s 2024-05-09T09:13:50.874 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:50 smithi139 ceph-mon[33599]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-05-09T09:13:50.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:50 smithi064 ceph-mon[30149]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-05-09T09:13:50.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:50 smithi179 ceph-mon[34086]: pgmap v511: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-05-09T09:13:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.065+0000 7f207cfa7640 1 -- 172.21.15.64:0/1969919603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20781014b0 msgr2=0x7f2078107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.065+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1969919603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20781014b0 0x7f2078107d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2068005fd0 tx=0x7f206802c750 comp rx=0 tx=0).stop 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 -- 172.21.15.64:0/1969919603 shutdown_connections 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1969919603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20781014b0 0x7f2078107d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1969919603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2078100af0 0x7f2078100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1969919603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20780ff900 0x7f20780ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 -- 172.21.15.64:0/1969919603 >> 172.21.15.64:0/1969919603 conn(0x7f20780fae90 msgr2=0x7f20780fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 -- 172.21.15.64:0/1969919603 shutdown_connections 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.066+0000 7f207cfa7640 1 -- 172.21.15.64:0/1969919603 wait complete. 2024-05-09T09:13:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.067+0000 7f207cfa7640 1 Processor -- start 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.067+0000 7f207cfa7640 1 -- start start 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.067+0000 7f207cfa7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20780ff900 0x7f207810a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f207cfa7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 0x7f207810cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f207cfa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20781014b0 0x7f207810f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f207cfa7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2078071d20 con 0x7f20781014b0 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f207cfa7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2078071ba0 con 0x7f2078100af0 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f207cfa7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2078071ea0 con 0x7f20780ff900 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 0x7f207810cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 0x7f207810cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33018/0 (socket says 172.21.15.64:33018) 2024-05-09T09:13:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 -- 172.21.15.64:0/1695427627 learned_addr learned my addr 172.21.15.64:0/1695427627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2077fff640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20781014b0 0x7f207810f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 -- 172.21.15.64:0/1695427627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20780ff900 msgr2=0x7f207810a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f20777fe640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20780ff900 0x7f207810a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20780ff900 0x7f207810a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 -- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20781014b0 msgr2=0x7f207810f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20781014b0 0x7f207810f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.068+0000 7f2076ffd640 1 -- 172.21.15.64:0/1695427627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207810fe40 con 0x7f2078100af0 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.069+0000 7f2076ffd640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 0x7f207810cef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f206c01fb10 tx=0x7f206c01a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.069+0000 7f2074ff9640 1 -- 172.21.15.64:0/1695427627 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c01dd40 con 0x7f2078100af0 2024-05-09T09:13:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.069+0000 7f2074ff9640 1 -- 172.21.15.64:0/1695427627 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f206c0054d0 con 0x7f2078100af0 2024-05-09T09:13:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.069+0000 7f2074ff9640 1 -- 172.21.15.64:0/1695427627 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c021440 con 0x7f2078100af0 2024-05-09T09:13:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.069+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2078110130 con 0x7f2078100af0 2024-05-09T09:13:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.069+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2078110630 con 0x7f2078100af0 2024-05-09T09:13:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 Processor -- start 2024-05-09T09:13:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2074ff9640 1 -- 172.21.15.64:0/1695427627 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f206c0046c0 con 0x7f2078100af0 2024-05-09T09:13:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 -- start start 2024-05-09T09:13:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b40820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40df0 0x562132b458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562132aa4340 con 0x562132b45f60 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562132a4c290 con 0x562132b40400 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2080ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562132aa0300 con 0x562132b40df0 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f20777fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40df0 0x562132b458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.073+0000 7f2077fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b40820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f20777fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40df0 0x562132b458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55068/0 (socket says 172.21.15.64:55068) 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f20777fe640 1 -- 172.21.15.64:0/1290773771 learned_addr learned my addr 172.21.15.64:0/1290773771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2077fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b40820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33032/0 (socket says 172.21.15.64:33032) 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2076ffd640 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2077fff640 1 -- 172.21.15.64:0/1290773771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40df0 msgr2=0x562132b458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2077fff640 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40df0 0x562132b458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2077fff640 1 -- 172.21.15.64:0/1290773771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 msgr2=0x562132b483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2077fff640 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.074+0000 7f2077fff640 1 -- 172.21.15.64:0/1290773771 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562132a708e0 con 0x562132b40400 2024-05-09T09:13:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.075+0000 7f2077fff640 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b40820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2068033960 tx=0x7f206800d440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0489ee89e903cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.075+0000 7f2074ff9640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20600a2cd0 0x7f20600a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.076+0000 7f205dffb640 1 -- 172.21.15.64:0/1290773771 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068043070 con 0x562132b40400 2024-05-09T09:13:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.076+0000 7f205dffb640 1 -- 172.21.15.64:0/1290773771 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2068033740 con 0x562132b40400 2024-05-09T09:13:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.076+0000 7f205dffb640 1 -- 172.21.15.64:0/1290773771 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068038040 con 0x562132b40400 2024-05-09T09:13:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.076+0000 7f2074ff9640 1 -- 172.21.15.64:0/1695427627 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f206c0e0980 con 0x7f2078100af0 2024-05-09T09:13:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.076+0000 7f20777fe640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20600a2cd0 0x7f20600a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f2080ff0740 1 -- 172.21.15.64:0/1290773771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 msgr2=0x562132b40820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f2080ff0740 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b40820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2068033960 tx=0x7f206800d440 comp rx=0 tx=0).stop 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f2080ff0740 1 -- 172.21.15.64:0/1290773771 shutdown_connections 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f2080ff0740 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f20777fe640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20600a2cd0 0x7f20600a5190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x562132b411f0 tx=0x7f2070008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f2080ff0740 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40df0 0x562132b458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.077+0000 7f2080ff0740 1 --2- 172.21.15.64:0/1290773771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b40820 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.078+0000 7f2080ff0740 1 -- 172.21.15.64:0/1290773771 >> 172.21.15.64:0/1290773771 conn(0x562132b3ba70 msgr2=0x562132b3be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.079+0000 7f2080ff0740 1 -- 172.21.15.64:0/1290773771 shutdown_connections 2024-05-09T09:13:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.079+0000 7f2080ff0740 1 -- 172.21.15.64:0/1290773771 wait complete. 2024-05-09T09:13:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.079+0000 7f2080ff0740 1 -- start start 2024-05-09T09:13:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 Processor -- start 2024-05-09T09:13:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b4abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b51bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562132b48eb0 con 0x562132b45f60 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562132a5f790 con 0x562132b40df0 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562132b49030 con 0x562132b40400 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2080ff0740 10 client.0 _ll_register_callbacks cb 0x562132a9aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f20777fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f20777fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b4a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55078/0 (socket says 172.21.15.64:55078) 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f20777fe640 1 -- 172.21.15.64:0/3304716480 learned_addr learned my addr 172.21.15.64:0/3304716480 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2076ffd640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b4abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.080+0000 7f2077fff640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b51bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2076ffd640 1 -- 172.21.15.64:0/3304716480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 msgr2=0x562132b4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2076ffd640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b4a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2076ffd640 1 -- 172.21.15.64:0/3304716480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 msgr2=0x562132b51bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2076ffd640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b51bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2076ffd640 1 -- 172.21.15.64:0/3304716480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621327b8210 con 0x562132b40df0 2024-05-09T09:13:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2076ffd640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b4abb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f206c0acd80 tx=0x7f206c0acc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5e660a4d83fcf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2056ffd640 1 -- 172.21.15.64:0/3304716480 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c0b0060 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.081+0000 7f2056ffd640 1 -- 172.21.15.64:0/3304716480 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f206c0a9460 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.082+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2068033740 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.082+0000 7f2056ffd640 1 -- 172.21.15.64:0/3304716480 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f206c0ad4f0 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.082+0000 7f2080ff0740 10 client.26279 fetch_fsmap learned FSMap version 39 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.082+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562132884620 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.082+0000 7f2056ffd640 1 -- 172.21.15.64:0/3304716480 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2068033740 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2056ffd640 1 -- 172.21.15.64:0/3304716480 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f206c004fd0 con 0x562132b40df0 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2080ff0740 10 client.26279 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2080ff0740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2080ff0740 20 client.26279 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132b56170 0x562132b58560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2080ff0740 4 client.26279 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:13:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562132ab1a70 con 0x562132b56170 2024-05-09T09:13:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.083+0000 7f20777fe640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132b56170 0x562132b58560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.084+0000 7f20777fe640 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132b56170 0x562132b58560 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbec02dcb7b30ae6 server_cookie=11cd149e96d5653a in_seq=0 out_seq=0 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.084+0000 7f2056ffd640 10 client.26279 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.084+0000 7f2056ffd640 1 -- 172.21.15.64:0/3304716480 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562132ab1a70 con 0x562132b56170 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.084+0000 7f2056ffd640 10 client.26279 handle_command_reply: tid=0 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.085+0000 7f2080ff0740 1 client.26279 shutdown 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.085+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 msgr2=0x562132b4abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.085+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b4abb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f206c0acd80 tx=0x7f206c0acc80 comp rx=0 tx=0).stop 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 shutdown_connections 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132b56170 0x562132b58560 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b51bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b4abb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3304716480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 >> 172.21.15.64:0/3304716480 conn(0x562132b3ba70 msgr2=0x562132b3abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2056ffd640 0 client.26279 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 shutdown_connections 2024-05-09T09:13:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 1 -- 172.21.15.64:0/3304716480 wait complete. 2024-05-09T09:13:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.086+0000 7f2080ff0740 20 client.26279 trim_cache size 0 max 16384 2024-05-09T09:13:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.119+0000 7f2080ff0740 1 Processor -- start 2024-05-09T09:13:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.119+0000 7f2080ff0740 1 -- start start 2024-05-09T09:13:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b3c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b51bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2080ff0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b3190 con 0x562132b45f60 2024-05-09T09:13:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2080ff0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b09d0 con 0x562132b40df0 2024-05-09T09:13:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2080ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20781ae820 con 0x562132b40400 2024-05-09T09:13:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2077fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2077fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b3eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35904/0 (socket says 172.21.15.64:35904) 2024-05-09T09:13:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.120+0000 7f2077fff640 1 -- 172.21.15.64:0/3363758721 learned_addr learned my addr 172.21.15.64:0/3363758721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f20777fe640 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b3c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f2077fff640 1 -- 172.21.15.64:0/3363758721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 msgr2=0x562132b3c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f2077fff640 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b3c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f2077fff640 1 -- 172.21.15.64:0/3363758721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 msgr2=0x562132b51bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f2077fff640 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b51bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f2077fff640 1 -- 172.21.15.64:0/3363758721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562132b3c7e0 con 0x562132b45f60 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.121+0000 7f2077fff640 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b3eb00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f206800a480 tx=0x7f206802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d50bef9eb46e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.122+0000 7f205d7fa640 1 -- 172.21.15.64:0/3363758721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206804c020 con 0x562132b45f60 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.122+0000 7f205d7fa640 1 -- 172.21.15.64:0/3363758721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2068003970 con 0x562132b45f60 2024-05-09T09:13:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.122+0000 7f205d7fa640 1 -- 172.21.15.64:0/3363758721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068043070 con 0x562132b45f60 2024-05-09T09:13:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.123+0000 7f2080ff0740 1 -- 172.21.15.64:0/3363758721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 msgr2=0x562132b3eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.123+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b3eb00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f206800a480 tx=0x7f206802cdf0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.123+0000 7f2080ff0740 1 -- 172.21.15.64:0/3363758721 shutdown_connections 2024-05-09T09:13:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.123+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b45f60 0x562132b3eb00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.123+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40df0 0x562132b51bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.124+0000 7f2080ff0740 1 --2- 172.21.15.64:0/3363758721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b40400 0x562132b3c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.124+0000 7f2080ff0740 1 -- 172.21.15.64:0/3363758721 >> 172.21.15.64:0/3363758721 conn(0x562132b3ba70 msgr2=0x562132b57ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.124+0000 7f2080ff0740 1 -- 172.21.15.64:0/3363758721 shutdown_connections 2024-05-09T09:13:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.124+0000 7f2080ff0740 1 -- 172.21.15.64:0/3363758721 wait complete. 2024-05-09T09:13:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.125+0000 7f2080ff0740 1 -- start start 2024-05-09T09:13:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.125+0000 7f2080ff0740 1 Processor -- start 2024-05-09T09:13:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.125+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b4f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2080ff0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b40df0 0x562132b4f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f20777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b4f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f20777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b4f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33050/0 (socket says 172.21.15.64:33050) 2024-05-09T09:13:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2076ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b40df0 0x562132b4f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2076ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b40df0 0x562132b4f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35914/0 (socket says 172.21.15.64:35914) 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f20777fe640 1 -- 172.21.15.64:0/4033662830 learned_addr learned my addr 172.21.15.64:0/4033662830 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b45f60 0x562132b50980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20781aee90 con 0x562132b40df0 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b0300 con 0x562132b40400 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2077fff640 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b45f60 0x562132b50980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.126+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20781af560 con 0x562132b45f60 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2080ff0740 10 client.0 _ll_register_callbacks cb 0x7f206c0a9090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2077fff640 1 -- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 msgr2=0x562132b4f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2077fff640 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b4f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2077fff640 1 -- 172.21.15.64:0/4033662830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b40df0 msgr2=0x562132b4f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2077fff640 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b40df0 0x562132b4f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2077fff640 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562132b50f70 con 0x562132b45f60 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2077fff640 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b45f60 0x562132b50980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f206800ab50 tx=0x7f206800a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7764a7cd84f86c9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.127+0000 7f2056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.128+0000 7f2056ffd640 1 -- 172.21.15.64:0/4033662830 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068043030 con 0x562132b45f60 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.128+0000 7f2056ffd640 1 -- 172.21.15.64:0/4033662830 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2068003d20 con 0x562132b45f60 2024-05-09T09:13:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.128+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2068003970 con 0x562132b45f60 2024-05-09T09:13:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.128+0000 7f2056ffd640 1 -- 172.21.15.64:0/4033662830 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f206804c040 con 0x562132b45f60 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.128+0000 7f2080ff0740 10 client.26443 fetch_fsmap learned FSMap version 39 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562132bbc860 con 0x562132b45f60 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2056ffd640 1 -- 172.21.15.64:0/4033662830 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2068003970 con 0x562132b45f60 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2056ffd640 1 -- 172.21.15.64:0/4033662830 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f206800b040 con 0x562132b45f60 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 10 client.26443 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 10 client.26443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 20 client.26443 populate_metadata read hostname 'smithi064' 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132bbcd00 0x562132bbf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 4 client.26443 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:13:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.129+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562132aabdc0 con 0x562132bbcd00 2024-05-09T09:13:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.130+0000 7f20777fe640 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132bbcd00 0x562132bbf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:13:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.132+0000 7f20777fe640 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132bbcd00 0x562132bbf0f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ececf2a2bcab0fa2 server_cookie=cbcda975272d3c5c in_seq=0 out_seq=0 2024-05-09T09:13:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.133+0000 7f2056ffd640 10 client.26443 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.156+0000 7f2056ffd640 1 -- 172.21.15.64:0/4033662830 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562132aabdc0 con 0x562132bbcd00 2024-05-09T09:13:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.156+0000 7f2056ffd640 10 client.26443 handle_command_reply: tid=0 2024-05-09T09:13:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.156+0000 7f2080ff0740 1 client.26443 shutdown 2024-05-09T09:13:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b45f60 msgr2=0x562132b50980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b45f60 0x562132b50980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f206800ab50 tx=0x7f206800a840 comp rx=0 tx=0).stop 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 shutdown_connections 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562132bbcd00 0x562132bbf0f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2056ffd640 0 client.26443 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562132b45f60 0x562132b50980 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562132b40df0 0x562132b4f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 --2- 172.21.15.64:0/4033662830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562132b40400 0x562132b4f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.157+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 >> 172.21.15.64:0/4033662830 conn(0x562132b3ba70 msgr2=0x562132b3eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.158+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 shutdown_connections 2024-05-09T09:13:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.158+0000 7f2080ff0740 1 -- 172.21.15.64:0/4033662830 wait complete. 2024-05-09T09:13:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.158+0000 7f2080ff0740 20 client.26443 trim_cache size 0 max 16384 2024-05-09T09:13:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:13:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:13:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:13:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:13:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.160+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20600a2cd0 msgr2=0x7f20600a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.160+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20600a2cd0 0x7f20600a5190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x562132b411f0 tx=0x7f2070008040 comp rx=0 tx=0).stop 2024-05-09T09:13:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.160+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 msgr2=0x7f207810cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:13:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.161+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 0x7f207810cef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f206c01fb10 tx=0x7f206c01a9e0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.161+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 shutdown_connections 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.161+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20600a2cd0 0x7f20600a5190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.161+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20781014b0 0x7f207810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.161+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2078100af0 0x7f207810cef0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.162+0000 7f207cfa7640 1 --2- 172.21.15.64:0/1695427627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20780ff900 0x7f207810a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.162+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 >> 172.21.15.64:0/1695427627 conn(0x7f20780fae90 msgr2=0x7f20780fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.162+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 shutdown_connections 2024-05-09T09:13:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:13:51.162+0000 7f207cfa7640 1 -- 172.21.15.64:0/1695427627 wait complete. 2024-05-09T09:13:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 210.0 s, 1068.2 tps, lat 93.281 ms stddev 28.427 2024-05-09T09:13:52.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:52 smithi064 ceph-mon[30149]: pgmap v512: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.0 MiB/s wr, 943 op/s 2024-05-09T09:13:52.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:52 smithi179 ceph-mon[34086]: pgmap v512: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.0 MiB/s wr, 943 op/s 2024-05-09T09:13:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:52 smithi139 ceph-mon[33599]: pgmap v512: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.0 MiB/s wr, 943 op/s 2024-05-09T09:13:54.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:54 smithi064 ceph-mon[30149]: pgmap v513: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.7 MiB/s wr, 1.01k op/s 2024-05-09T09:13:54.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:54 smithi179 ceph-mon[34086]: pgmap v513: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.7 MiB/s wr, 1.01k op/s 2024-05-09T09:13:55.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:54 smithi139 ceph-mon[33599]: pgmap v513: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.7 MiB/s wr, 1.01k op/s 2024-05-09T09:13:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 215.0 s, 990.4 tps, lat 101.172 ms stddev 32.531 2024-05-09T09:13:56.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:56 smithi064 ceph-mon[30149]: pgmap v514: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-05-09T09:13:56.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:56 smithi179 ceph-mon[34086]: pgmap v514: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-05-09T09:13:57.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:56 smithi139 ceph-mon[33599]: pgmap v514: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-05-09T09:13:58.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:13:58 smithi064 ceph-mon[30149]: pgmap v515: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 857 op/s 2024-05-09T09:13:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:13:58 smithi179 ceph-mon[34086]: pgmap v515: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 857 op/s 2024-05-09T09:13:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:13:58 smithi139 ceph-mon[33599]: pgmap v515: 129 pgs: 129 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 857 op/s 2024-05-09T09:14:00.905 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:00 smithi064 ceph-mon[30149]: pgmap v516: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-05-09T09:14:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:00 smithi179 ceph-mon[34086]: pgmap v516: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-05-09T09:14:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:00 smithi139 ceph-mon[33599]: pgmap v516: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-05-09T09:14:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 220.0 s, 965.4 tps, lat 102.886 ms stddev 28.220 2024-05-09T09:14:02.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:02 smithi064 ceph-mon[30149]: pgmap v517: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 881 op/s 2024-05-09T09:14:02.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:02 smithi179 ceph-mon[34086]: pgmap v517: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 881 op/s 2024-05-09T09:14:03.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:02 smithi139 ceph-mon[33599]: pgmap v517: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 881 op/s 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.318+0000 7f51981b0640 1 -- 172.21.15.64:0/2498611509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 msgr2=0x7f5190105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.318+0000 7f51981b0640 1 --2- 172.21.15.64:0/2498611509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 0x7f5190105ac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f518002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 -- 172.21.15.64:0/2498611509 shutdown_connections 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 --2- 172.21.15.64:0/2498611509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 0x7f5190105ac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 --2- 172.21.15.64:0/2498611509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5190100860 0x7f5190100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 --2- 172.21.15.64:0/2498611509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51900ff730 0x7f51900ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 -- 172.21.15.64:0/2498611509 >> 172.21.15.64:0/2498611509 conn(0x7f51900fb050 msgr2=0x7f51900fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 -- 172.21.15.64:0/2498611509 shutdown_connections 2024-05-09T09:14:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.319+0000 7f51981b0640 1 -- 172.21.15.64:0/2498611509 wait complete. 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.320+0000 7f51981b0640 1 Processor -- start 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.320+0000 7f51981b0640 1 -- start start 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.320+0000 7f51981b0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51900ff730 0x7f519010f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f51981b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5190100860 0x7f519010c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51900ff730 0x7f519010f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51900ff730 0x7f519010f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33414/0 (socket says 172.21.15.64:33414) 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f51981b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 0x7f519010c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f5195f4c640 1 -- 172.21.15.64:0/2763405008 learned_addr learned my addr 172.21.15.64:0/2763405008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51901139c0 con 0x7f5190100860 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5190113840 con 0x7f5190101200 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5190113b40 con 0x7f51900ff730 2024-05-09T09:14:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f519574b640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5190100860 0x7f519010c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.321+0000 7f519674d640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 0x7f519010c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.322+0000 7f519574b640 1 -- 172.21.15.64:0/2763405008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51900ff730 msgr2=0x7f519010f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.322+0000 7f519574b640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51900ff730 0x7f519010f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.322+0000 7f519574b640 1 -- 172.21.15.64:0/2763405008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 msgr2=0x7f519010c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.322+0000 7f519574b640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 0x7f519010c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.322+0000 7f519574b640 1 -- 172.21.15.64:0/2763405008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51901ae1c0 con 0x7f5190100860 2024-05-09T09:14:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.322+0000 7f519574b640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5190100860 0x7f519010c080 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d6d0 tx=0x7f518c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.324+0000 7f5186ffd640 1 -- 172.21.15.64:0/2763405008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c017070 con 0x7f5190100860 2024-05-09T09:14:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.324+0000 7f5186ffd640 1 -- 172.21.15.64:0/2763405008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f518c009820 con 0x7f5190100860 2024-05-09T09:14:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.325+0000 7f5186ffd640 1 -- 172.21.15.64:0/2763405008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c004b70 con 0x7f5190100860 2024-05-09T09:14:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.325+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51901ae4b0 con 0x7f5190100860 2024-05-09T09:14:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.325+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51901ae980 con 0x7f5190100860 2024-05-09T09:14:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 Processor -- start 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 -- start start 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 0x55a982481670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481c40 0x55a982486740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982486db0 0x55a982489240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a9823e5340 con 0x55a982481c40 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a98238d290 con 0x55a982486db0 2024-05-09T09:14:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.329+0000 7f519af9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a9823e1300 con 0x55a982481250 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 0x55a982481670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 0x55a982481670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33426/0 (socket says 172.21.15.64:33426) 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 -- 172.21.15.64:0/2754393607 learned_addr learned my addr 172.21.15.64:0/2754393607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 -- 172.21.15.64:0/2754393607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982486db0 msgr2=0x55a982489240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982486db0 0x55a982489240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 -- 172.21.15.64:0/2754393607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481c40 msgr2=0x55a982486740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481c40 0x55a982486740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 -- 172.21.15.64:0/2754393607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9823b18e0 con 0x55a982481250 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f519674d640 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 0x55a982481670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f51800040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49501e0d30b2267 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.330+0000 7f5186ffd640 1 -- 172.21.15.64:0/2763405008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f518c004d10 con 0x7f5190100860 2024-05-09T09:14:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.331+0000 7f517bfff640 1 -- 172.21.15.64:0/2754393607 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180035040 con 0x55a982481250 2024-05-09T09:14:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.331+0000 7f517bfff640 1 -- 172.21.15.64:0/2754393607 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5180004760 con 0x55a982481250 2024-05-09T09:14:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.331+0000 7f517bfff640 1 -- 172.21.15.64:0/2754393607 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180003850 con 0x55a982481250 2024-05-09T09:14:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.332+0000 7f5186ffd640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f517c0a2c40 0x7f517c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.332+0000 7f519af9d740 1 -- 172.21.15.64:0/2754393607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 msgr2=0x55a982481670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.332+0000 7f5186ffd640 1 -- 172.21.15.64:0/2763405008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f518c0cedd0 con 0x7f5190100860 2024-05-09T09:14:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.332+0000 7f519af9d740 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 0x55a982481670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f51800040f0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.332+0000 7f5195f4c640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f517c0a2c40 0x7f517c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 -- 172.21.15.64:0/2754393607 shutdown_connections 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982486db0 0x55a982489240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481c40 0x55a982486740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 --2- 172.21.15.64:0/2754393607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481250 0x55a982481670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 -- 172.21.15.64:0/2754393607 >> 172.21.15.64:0/2754393607 conn(0x55a98247c8c0 msgr2=0x55a98247ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f5195f4c640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f517c0a2c40 0x7f517c0a5100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55a982482040 tx=0x7f5188009bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 -- 172.21.15.64:0/2754393607 shutdown_connections 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.333+0000 7f519af9d740 1 -- 172.21.15.64:0/2754393607 wait complete. 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.334+0000 7f519af9d740 1 -- start start 2024-05-09T09:14:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.334+0000 7f519af9d740 1 Processor -- start 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.334+0000 7f519af9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 0x55a98249c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.334+0000 7f519af9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98249c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519af9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982486db0 0x55a9824a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 0x55a98249c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 0x55a98249c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37220/0 (socket says 172.21.15.64:37220) 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519574b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98249c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519574b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98249c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33438/0 (socket says 172.21.15.64:33438) 2024-05-09T09:14:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 -- 172.21.15.64:0/1149107538 learned_addr learned my addr 172.21.15.64:0/1149107538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a98249ad10 con 0x55a982486db0 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a9823a0790 con 0x55a982481250 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a98249ae90 con 0x55a982481c40 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519af9d740 10 client.0 _ll_register_callbacks cb 0x55a9823dba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f519674d640 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982486db0 0x55a9824a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 -- 172.21.15.64:0/1149107538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 msgr2=0x55a98249c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98249c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 -- 172.21.15.64:0/1149107538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982486db0 msgr2=0x55a9824a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982486db0 0x55a9824a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.335+0000 7f5195f4c640 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9820f9210 con 0x55a982481250 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.336+0000 7f5195f4c640 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 0x55a98249c490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f518800fb40 tx=0x7f5188009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b98bdfe43609570 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.336+0000 7f51797fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.336+0000 7f51797fa640 1 -- 172.21.15.64:0/1149107538 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518801dc70 con 0x55a982481250 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.336+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5180004760 con 0x55a982481250 2024-05-09T09:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.336+0000 7f51797fa640 1 -- 172.21.15.64:0/1149107538 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f518801e430 con 0x55a982481250 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f51797fa640 1 -- 172.21.15.64:0/1149107538 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5188031050 con 0x55a982481250 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f519af9d740 10 client.26291 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a9821c5620 con 0x55a982481250 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f51797fa640 1 -- 172.21.15.64:0/1149107538 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5180004760 con 0x55a982481250 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f51797fa640 1 -- 172.21.15.64:0/1149107538 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f518801b650 con 0x55a982481250 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f519af9d740 10 client.26291 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f519af9d740 10 client.26291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.337+0000 7f519af9d740 20 client.26291 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.338+0000 7f519af9d740 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a9824a7f70 0x55a9824aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.338+0000 7f519af9d740 4 client.26291 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.338+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9823f2a70 con 0x55a9824a7f70 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.338+0000 7f519574b640 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a9824a7f70 0x55a9824aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.338+0000 7f519574b640 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a9824a7f70 0x55a9824aa3f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8578d5ad6a47e11b server_cookie=af6752bf9338aaf7 in_seq=0 out_seq=0 2024-05-09T09:14:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.338+0000 7f51797fa640 10 client.26291 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.339+0000 7f51797fa640 1 -- 172.21.15.64:0/1149107538 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9823f2a70 con 0x55a9824a7f70 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.339+0000 7f51797fa640 10 client.26291 handle_command_reply: tid=0 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.339+0000 7f519af9d740 1 client.26291 shutdown 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.340+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 msgr2=0x55a98249c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.340+0000 7f519af9d740 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 0x55a98249c490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f518800fb40 tx=0x7f5188009500 comp rx=0 tx=0).stop 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 shutdown_connections 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a9824a7f70 0x55a9824aa3f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982486db0 0x55a9824a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98249c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 --2- 172.21.15.64:0/1149107538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a982481250 0x55a98249c490 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f51797fa640 0 client.26291 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 >> 172.21.15.64:0/1149107538 conn(0x55a98247c8c0 msgr2=0x55a98247ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 shutdown_connections 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.341+0000 7f519af9d740 1 -- 172.21.15.64:0/1149107538 wait complete. 2024-05-09T09:14:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.342+0000 7f519af9d740 20 client.26291 trim_cache size 0 max 16384 2024-05-09T09:14:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.376+0000 7f519af9d740 1 Processor -- start 2024-05-09T09:14:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 -- start start 2024-05-09T09:14:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a98249c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98247f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51901b2f70 con 0x55a982481250 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f51901b07b0 con 0x55a98247c8c0 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519af9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f51900733b0 con 0x55a982481c40 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a98249c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519574b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519574b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52220/0 (socket says 172.21.15.64:52220) 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.377+0000 7f519574b640 1 -- 172.21.15.64:0/1571155022 learned_addr learned my addr 172.21.15.64:0/1571155022 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.378+0000 7f519574b640 1 -- 172.21.15.64:0/1571155022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 msgr2=0x55a98247f840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.378+0000 7f519574b640 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98247f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.378+0000 7f519574b640 1 -- 172.21.15.64:0/1571155022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 msgr2=0x55a98249c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.378+0000 7f519574b640 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a98249c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.378+0000 7f519574b640 1 -- 172.21.15.64:0/1571155022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a982481630 con 0x55a982481250 2024-05-09T09:14:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.378+0000 7f519574b640 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a39d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f518c0cc590 tx=0x7f518c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bf40ce5b74decd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.380+0000 7f517b7fe640 1 -- 172.21.15.64:0/1571155022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0993c0 con 0x55a982481250 2024-05-09T09:14:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.380+0000 7f517b7fe640 1 -- 172.21.15.64:0/1571155022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f518c099620 con 0x55a982481250 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 -- 172.21.15.64:0/1571155022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 msgr2=0x55a9824a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a39d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f518c0cc590 tx=0x7f518c0ce040 comp rx=0 tx=0).stop 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 -- 172.21.15.64:0/1571155022 shutdown_connections 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a98247f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a39d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 --2- 172.21.15.64:0/1571155022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a98249c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 -- 172.21.15.64:0/1571155022 >> 172.21.15.64:0/1571155022 conn(0x55a982482840 msgr2=0x55a98249c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.381+0000 7f519af9d740 1 -- 172.21.15.64:0/1571155022 shutdown_connections 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.382+0000 7f519af9d740 1 -- 172.21.15.64:0/1571155022 wait complete. 2024-05-09T09:14:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.382+0000 7f519af9d740 1 -- start start 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.382+0000 7f519af9d740 1 Processor -- start 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a9824a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a9824a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51901aeca0 con 0x55a982481250 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f51901b00e0 con 0x55a98247c8c0 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f51901af340 con 0x55a982481c40 2024-05-09T09:14:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519af9d740 10 client.0 _ll_register_callbacks cb 0x7f518800f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a9824a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f5195f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a9824a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37246/0 (socket says 172.21.15.64:37246) 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f5195f4c640 1 -- 172.21.15.64:0/2802457529 learned_addr learned my addr 172.21.15.64:0/2802457529 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519674d640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a9824a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.383+0000 7f519574b640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f5195f4c640 1 -- 172.21.15.64:0/2802457529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 msgr2=0x55a9824a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f5195f4c640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a9824a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f5195f4c640 1 -- 172.21.15.64:0/2802457529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 msgr2=0x55a9824a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f5195f4c640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f5195f4c640 1 -- 172.21.15.64:0/2802457529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9824a2cb0 con 0x55a98247c8c0 2024-05-09T09:14:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f5195f4c640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a9824a1170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f518801bc60 tx=0x7f518801d8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a55474de024b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f51797fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.384+0000 7f51797fa640 1 -- 172.21.15.64:0/2802457529 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188039020 con 0x55a98247c8c0 2024-05-09T09:14:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.385+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f518c099620 con 0x55a98247c8c0 2024-05-09T09:14:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.385+0000 7f51797fa640 1 -- 172.21.15.64:0/2802457529 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5188004a40 con 0x55a98247c8c0 2024-05-09T09:14:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.389+0000 7f51797fa640 1 -- 172.21.15.64:0/2802457529 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f518801de80 con 0x55a98247c8c0 2024-05-09T09:14:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.389+0000 7f519af9d740 10 client.26294 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.389+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a98248f990 con 0x55a98247c8c0 2024-05-09T09:14:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.389+0000 7f51797fa640 1 -- 172.21.15.64:0/2802457529 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f518c099620 con 0x55a98247c8c0 2024-05-09T09:14:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f51797fa640 1 -- 172.21.15.64:0/2802457529 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f51880267b0 con 0x55a98247c8c0 2024-05-09T09:14:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519af9d740 10 client.26294 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519af9d740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519af9d740 20 client.26294 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519af9d740 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a982486db0 0x55a9824923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519af9d740 4 client.26294 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:14:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9823ecdc0 con 0x55a982486db0 2024-05-09T09:14:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.390+0000 7f519574b640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a982486db0 0x55a9824923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.391+0000 7f519574b640 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a982486db0 0x55a9824923b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4449e67d91b225 server_cookie=187a325f1b198738 in_seq=0 out_seq=0 2024-05-09T09:14:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.391+0000 7f51797fa640 10 client.26294 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.407+0000 7f51797fa640 1 -- 172.21.15.64:0/2802457529 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9823ecdc0 con 0x55a982486db0 2024-05-09T09:14:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.407+0000 7f51797fa640 10 client.26294 handle_command_reply: tid=0 2024-05-09T09:14:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.407+0000 7f519af9d740 1 client.26294 shutdown 2024-05-09T09:14:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.408+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 msgr2=0x55a9824a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.408+0000 7f519af9d740 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a9824a1170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f518801bc60 tx=0x7f518801d8b0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 shutdown_connections 2024-05-09T09:14:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a982486db0 0x55a9824923b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a982481c40 0x55a9824a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a982481250 0x55a9824a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 --2- 172.21.15.64:0/2802457529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a98247c8c0 0x55a9824a1170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 >> 172.21.15.64:0/2802457529 conn(0x55a982482840 msgr2=0x55a98247f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f51797fa640 0 client.26294 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 shutdown_connections 2024-05-09T09:14:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 1 -- 172.21.15.64:0/2802457529 wait complete. 2024-05-09T09:14:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.410+0000 7f519af9d740 20 client.26294 trim_cache size 0 max 16384 2024-05-09T09:14:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:14:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:14:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:14:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f517c0a2c40 msgr2=0x7f517c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f517c0a2c40 0x7f517c0a5100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55a982482040 tx=0x7f5188009bb0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5190100860 msgr2=0x7f519010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5190100860 0x7f519010c080 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d6d0 tx=0x7f518c00db90 comp rx=0 tx=0).stop 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 shutdown_connections 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f517c0a2c40 0x7f517c0a5100 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55a982482040 tx=0x7f5188009bb0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5190101200 0x7f519010c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5190100860 0x7f519010c080 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.415+0000 7f51981b0640 1 --2- 172.21.15.64:0/2763405008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51900ff730 0x7f519010f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.416+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 >> 172.21.15.64:0/2763405008 conn(0x7f51900fb050 msgr2=0x7f5190101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.416+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 shutdown_connections 2024-05-09T09:14:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:04.417+0000 7f51981b0640 1 -- 172.21.15.64:0/2763405008 wait complete. 2024-05-09T09:14:04.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:04 smithi064 ceph-mon[30149]: pgmap v518: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 1.04k op/s 2024-05-09T09:14:04.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:04 smithi179 ceph-mon[34086]: pgmap v518: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 1.04k op/s 2024-05-09T09:14:05.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:04 smithi139 ceph-mon[33599]: pgmap v518: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 1.04k op/s 2024-05-09T09:14:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 225.0 s, 880.4 tps, lat 114.034 ms stddev 85.382 2024-05-09T09:14:06.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:06 smithi064 ceph-mon[30149]: pgmap v519: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 1.04k op/s 2024-05-09T09:14:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:06 smithi179 ceph-mon[34086]: pgmap v519: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 1.04k op/s 2024-05-09T09:14:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:06 smithi139 ceph-mon[33599]: pgmap v519: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 1.04k op/s 2024-05-09T09:14:08.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:14:08.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:14:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.663+0000 7fe8e492e640 1 -- 172.21.15.64:0/532039174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 msgr2=0x7fe8e0108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.663+0000 7fe8e492e640 1 --2- 172.21.15.64:0/532039174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e0108190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009ca0 tx=0x7fe8d802a8d0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 -- 172.21.15.64:0/532039174 shutdown_connections 2024-05-09T09:14:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 --2- 172.21.15.64:0/532039174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 0x7fe8e01101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 --2- 172.21.15.64:0/532039174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8e0108f60 0x7fe8e01093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 --2- 172.21.15.64:0/532039174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e0108190 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 -- 172.21.15.64:0/532039174 >> 172.21.15.64:0/532039174 conn(0x7fe8e0076d10 msgr2=0x7fe8e0079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 -- 172.21.15.64:0/532039174 shutdown_connections 2024-05-09T09:14:08.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.664+0000 7fe8e492e640 1 -- 172.21.15.64:0/532039174 wait complete. 2024-05-09T09:14:08.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.665+0000 7fe8e492e640 1 Processor -- start 2024-05-09T09:14:08.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.666+0000 7fe8e492e640 1 -- start start 2024-05-09T09:14:08.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.666+0000 7fe8e492e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.666+0000 7fe8e492e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8e0108f60 0x7fe8e019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8e492e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 0x7fe8e01a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8e492e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0113910 con 0x7fe8e0107d70 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8e492e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0113790 con 0x7fe8e0109920 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8e492e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0113a90 con 0x7fe8e0108f60 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 0x7fe8e01a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8deffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8e0108f60 0x7fe8e019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 0x7fe8e01a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53342/0 (socket says 172.21.15.64:53342) 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8dffff640 1 -- 172.21.15.64:0/1434046976 learned_addr learned my addr 172.21.15.64:0/1434046976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:08.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.667+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.668+0000 7fe8df7fe640 1 -- 172.21.15.64:0/1434046976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8e0108f60 msgr2=0x7fe8e019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.668+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8e0108f60 0x7fe8e019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.668+0000 7fe8df7fe640 1 -- 172.21.15.64:0/1434046976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 msgr2=0x7fe8e01a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.668+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 0x7fe8e01a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.668+0000 7fe8df7fe640 1 -- 172.21.15.64:0/1434046976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e01a5cb0 con 0x7fe8e0107d70 2024-05-09T09:14:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.668+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e019e090 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009ca0 tx=0x7fe8d8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.669+0000 7fe8dcff9640 1 -- 172.21.15.64:0/1434046976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80307d0 con 0x7fe8e0107d70 2024-05-09T09:14:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.669+0000 7fe8dcff9640 1 -- 172.21.15.64:0/1434046976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d8030e20 con 0x7fe8e0107d70 2024-05-09T09:14:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.670+0000 7fe8dcff9640 1 -- 172.21.15.64:0/1434046976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d803b410 con 0x7fe8e0107d70 2024-05-09T09:14:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.670+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e01a5f40 con 0x7fe8e0107d70 2024-05-09T09:14:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.670+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01a63d0 con 0x7fe8e0107d70 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.674+0000 7fe8e8979740 1 Processor -- start 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 -- start start 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8dcff9640 1 -- 172.21.15.64:0/1434046976 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8d8004260 con 0x7fe8e0107d70 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41e0e0 0x56377c41e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c4235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56377c382d00 con 0x56377c41e0e0 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56377c32a4c0 con 0x56377c41ead0 2024-05-09T09:14:08.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8e8979740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56377c37ecc0 con 0x56377c423c40 2024-05-09T09:14:08.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8deffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8deffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4260d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56830/0 (socket says 172.21.15.64:56830) 2024-05-09T09:14:08.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8deffd640 1 -- 172.21.15.64:0/3514297064 learned_addr learned my addr 172.21.15.64:0/3514297064 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:08.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.675+0000 7fe8dffff640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41e0e0 0x56377c41e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.676+0000 7fe8deffd640 1 -- 172.21.15.64:0/3514297064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 msgr2=0x56377c4235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.676+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c4235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8dcff9640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8c80a2d60 0x7fe8c80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8dcff9640 1 -- 172.21.15.64:0/1434046976 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe8d80f0b80 con 0x7fe8e0107d70 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c4235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8deffd640 1 -- 172.21.15.64:0/3514297064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41e0e0 msgr2=0x56377c41e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41e0e0 0x56377c41e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8deffd640 1 -- 172.21.15.64:0/3514297064 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56377c1607d0 con 0x56377c423c40 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c4235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.677+0000 7fe8dffff640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41e0e0 0x56377c41e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.678+0000 7fe8dffff640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8c80a2d60 0x7fe8c80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.678+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4260d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000b7f0 tx=0x7fe8d000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9223db2e97c3004 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.679+0000 7fe8c5ffb640 1 -- 172.21.15.64:0/3514297064 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000d950 con 0x56377c423c40 2024-05-09T09:14:08.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.679+0000 7fe8c5ffb640 1 -- 172.21.15.64:0/3514297064 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d0018070 con 0x56377c423c40 2024-05-09T09:14:08.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.679+0000 7fe8c5ffb640 1 -- 172.21.15.64:0/3514297064 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00123e0 con 0x56377c423c40 2024-05-09T09:14:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.680+0000 7fe8e8979740 1 -- 172.21.15.64:0/3514297064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 msgr2=0x56377c4260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.680+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4260d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000b7f0 tx=0x7fe8d000bcb0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.680+0000 7fe8dffff640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8c80a2d60 0x7fe8c80a5220 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56377c41f330 tx=0x7fe8d40046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.681+0000 7fe8e8979740 1 -- 172.21.15.64:0/3514297064 shutdown_connections 2024-05-09T09:14:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.681+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4260d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.681+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c4235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.681+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3514297064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41e0e0 0x56377c41e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.681+0000 7fe8e8979740 1 -- 172.21.15.64:0/3514297064 >> 172.21.15.64:0/3514297064 conn(0x56377c419750 msgr2=0x56377c419b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.682+0000 7fe8e8979740 1 -- 172.21.15.64:0/3514297064 shutdown_connections 2024-05-09T09:14:08.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.682+0000 7fe8e8979740 1 -- 172.21.15.64:0/3514297064 wait complete. 2024-05-09T09:14:08.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.683+0000 7fe8e8979740 1 -- start start 2024-05-09T09:14:08.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.683+0000 7fe8e8979740 1 Processor -- start 2024-05-09T09:14:08.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.683+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 0x56377c428560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.683+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c428aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c423c40 0x56377c42fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8e8979740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56377c426c40 con 0x56377c423c40 2024-05-09T09:14:08.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8e8979740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56377c33db90 con 0x56377c41ead0 2024-05-09T09:14:08.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8e8979740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56377c426dc0 con 0x56377c41e0e0 2024-05-09T09:14:08.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 0x56377c428560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 0x56377c428560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56844/0 (socket says 172.21.15.64:56844) 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8e8979740 10 client.0 _ll_register_callbacks cb 0x56377c379da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3767340862 learned_addr learned my addr 172.21.15.64:0/3767340862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8dffff640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c423c40 0x56377c42fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 msgr2=0x56377c428aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c428aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c428aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3767340862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c423c40 msgr2=0x56377c42fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c423c40 0x56377c42fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3767340862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56377c0963f0 con 0x56377c41e0e0 2024-05-09T09:14:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.684+0000 7fe8dffff640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c423c40 0x56377c42fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 0x56377c428560 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80b99c0 tx=0x7fe8d80bcbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52b2265991203dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8beffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8beffd640 1 -- 172.21.15.64:0/3767340862 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80c13a0 con 0x56377c41e0e0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8beffd640 1 -- 172.21.15.64:0/3767340862 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe8d80bb2f0 con 0x56377c41e0e0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d0018070 con 0x56377c41e0e0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8beffd640 1 -- 172.21.15.64:0/3767340862 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe8d80c5c60 con 0x56377c41e0e0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8e8979740 10 client.26467 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56377c0560f0 con 0x56377c41e0e0 2024-05-09T09:14:08.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.685+0000 7fe8beffd640 1 -- 172.21.15.64:0/3767340862 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d8036020 con 0x56377c41e0e0 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8beffd640 1 -- 172.21.15.64:0/3767340862 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fe8d80b7bf0 con 0x56377c41e0e0 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8e8979740 10 client.26467 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8e8979740 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8e8979740 20 client.26467 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c434440 0x56377c4368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8e8979740 4 client.26467 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.686+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe8d0018070 con 0x56377c434440 2024-05-09T09:14:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.687+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c434440 0x56377c4368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.688+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c434440 0x56377c4368e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b2a50e101c34bb server_cookie=7e3022791b8fed4e in_seq=0 out_seq=0 2024-05-09T09:14:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.688+0000 7fe8beffd640 10 client.26467 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8beffd640 1 -- 172.21.15.64:0/3767340862 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fe8d0018070 con 0x56377c434440 2024-05-09T09:14:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8beffd640 10 client.26467 handle_command_reply: tid=0 2024-05-09T09:14:08.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 client.26467 shutdown 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 msgr2=0x56377c428560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 0x56377c428560 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80b99c0 tx=0x7fe8d80bcbf0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 shutdown_connections 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c434440 0x56377c4368e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c423c40 0x56377c42fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41ead0 0x56377c428aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3767340862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c41e0e0 0x56377c428560 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 >> 172.21.15.64:0/3767340862 conn(0x56377c419750 msgr2=0x56377c418a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.689+0000 7fe8beffd640 0 client.26467 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.690+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 shutdown_connections 2024-05-09T09:14:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.690+0000 7fe8e8979740 1 -- 172.21.15.64:0/3767340862 wait complete. 2024-05-09T09:14:08.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.690+0000 7fe8e8979740 20 client.26467 trim_cache size 0 max 16384 2024-05-09T09:14:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.727+0000 7fe8e8979740 1 Processor -- start 2024-05-09T09:14:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.727+0000 7fe8e8979740 1 -- start start 2024-05-09T09:14:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.727+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c428170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.727+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c428170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c428170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53364/0 (socket says 172.21.15.64:53364) 2024-05-09T09:14:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3399913827 learned_addr learned my addr 172.21.15.64:0/3399913827 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01b3270 con 0x56377c41ead0 2024-05-09T09:14:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01b0ab0 con 0x56377c41e0e0 2024-05-09T09:14:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.728+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01ae900 con 0x56377c423c40 2024-05-09T09:14:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3399913827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 msgr2=0x56377c4286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8dffff640 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4286b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3399913827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 msgr2=0x56377c42faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8df7fe640 1 -- 172.21.15.64:0/3399913827 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56377c419580 con 0x56377c41e0e0 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8deffd640 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.729+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c428170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80b9450 tx=0x7fe8d8003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca18488bd827fc42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.730+0000 7fe8c57fa640 1 -- 172.21.15.64:0/3399913827 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80c13a0 con 0x56377c41e0e0 2024-05-09T09:14:08.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.730+0000 7fe8c57fa640 1 -- 172.21.15.64:0/3399913827 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d8036040 con 0x56377c41e0e0 2024-05-09T09:14:08.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.730+0000 7fe8c57fa640 1 -- 172.21.15.64:0/3399913827 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80b7dc0 con 0x56377c41e0e0 2024-05-09T09:14:08.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.731+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 msgr2=0x56377c428170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.731+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c428170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80b9450 tx=0x7fe8d8003f90 comp rx=0 tx=0).stop 2024-05-09T09:14:08.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 shutdown_connections 2024-05-09T09:14:08.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c4286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 --2- 172.21.15.64:0/3399913827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c428170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 >> 172.21.15.64:0/3399913827 conn(0x56377c419750 msgr2=0x56377c4148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 shutdown_connections 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.732+0000 7fe8e8979740 1 -- 172.21.15.64:0/3399913827 wait complete. 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.733+0000 7fe8e8979740 1 -- start start 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.733+0000 7fe8e8979740 1 Processor -- start 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c42d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c42e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01aef70 con 0x56377c41ead0 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01b03e0 con 0x56377c41e0e0 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01af640 con 0x56377c423c40 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8e8979740 10 client.0 _ll_register_callbacks cb 0x7fe8d80bcdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c42d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8deffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8deffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51554/0 (socket says 172.21.15.64:51554) 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8deffd640 1 -- 172.21.15.64:0/230229478 learned_addr learned my addr 172.21.15.64:0/230229478 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.734+0000 7fe8dffff640 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c42e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8df7fe640 1 -- 172.21.15.64:0/230229478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 msgr2=0x56377c42e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c42e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8df7fe640 1 -- 172.21.15.64:0/230229478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 msgr2=0x56377c42d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8df7fe640 1 -- 172.21.15.64:0/230229478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56377c42ed40 con 0x56377c41e0e0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8df7fe640 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c42d270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006010 tx=0x7fe8d80b7df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6a4db5fc7fed97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8beffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8beffd640 1 -- 172.21.15.64:0/230229478 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80c13a0 con 0x56377c41e0e0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d8036040 con 0x56377c41e0e0 2024-05-09T09:14:08.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.735+0000 7fe8beffd640 1 -- 172.21.15.64:0/230229478 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe8d8030b80 con 0x56377c41e0e0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8beffd640 1 -- 172.21.15.64:0/230229478 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe8d80c57d0 con 0x56377c41e0e0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 10 client.26303 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56377c42ed40 con 0x56377c41e0e0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8beffd640 1 -- 172.21.15.64:0/230229478 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d80d1020 con 0x56377c41e0e0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8beffd640 1 -- 172.21.15.64:0/230229478 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fe8d80bbeb0 con 0x56377c41e0e0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 10 client.26303 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 20 client.26303 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c49b030 0x56377c49d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 4 client.26303 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.736+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe8d8036040 con 0x56377c49b030 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.737+0000 7fe8deffd640 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c49b030 0x56377c49d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.738+0000 7fe8deffd640 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c49b030 0x56377c49d450 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffecc9f9a5d128eb server_cookie=53f5b82f0248a74e in_seq=0 out_seq=0 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.738+0000 7fe8beffd640 10 client.26303 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:08.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8beffd640 1 -- 172.21.15.64:0/230229478 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe8d8036040 con 0x56377c49b030 2024-05-09T09:14:08.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8beffd640 10 client.26303 handle_command_reply: tid=0 2024-05-09T09:14:08.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8e8979740 1 client.26303 shutdown 2024-05-09T09:14:08.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 msgr2=0x56377c42d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8e8979740 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c42d270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006010 tx=0x7fe8d80b7df0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 shutdown_connections 2024-05-09T09:14:08.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.739+0000 7fe8e8979740 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56377c49b030 0x56377c49d450 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8beffd640 0 client.26303 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:08.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56377c423c40 0x56377c42e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56377c41ead0 0x56377c42d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 1 --2- 172.21.15.64:0/230229478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56377c41e0e0 0x56377c42d270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 >> 172.21.15.64:0/230229478 conn(0x56377c419750 msgr2=0x56377c428150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 shutdown_connections 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 1 -- 172.21.15.64:0/230229478 wait complete. 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.740+0000 7fe8e8979740 20 client.26303 trim_cache size 0 max 16384 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:14:08.743 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:14:08.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8c80a2d60 msgr2=0x7fe8c80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8c80a2d60 0x7fe8c80a5220 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56377c41f330 tx=0x7fe8d40046c0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 msgr2=0x7fe8e019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e019e090 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009ca0 tx=0x7fe8d8003ce0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 shutdown_connections 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe8c80a2d60 0x7fe8c80a5220 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e0109920 0x7fe8e01a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8e0108f60 0x7fe8e019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 --2- 172.21.15.64:0/1434046976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8e0107d70 0x7fe8e019e090 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 >> 172.21.15.64:0/1434046976 conn(0x7fe8e0076d10 msgr2=0x7fe8e010c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.742+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 shutdown_connections 2024-05-09T09:14:08.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.743+0000 7fe8e492e640 1 -- 172.21.15.64:0/1434046976 wait complete. 2024-05-09T09:14:08.762 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:14:08.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:14:08.809 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:08 smithi064 ceph-mon[30149]: pgmap v520: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.7 MiB/s wr, 878 op/s 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 -- 172.21.15.64:0/37547540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 msgr2=0x7fe484118770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 --2- 172.21.15.64:0/37547540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe484118770 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe478009860 tx=0x7fe47802a360 comp rx=0 tx=0).stop 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 -- 172.21.15.64:0/37547540 shutdown_connections 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 --2- 172.21.15.64:0/37547540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe484118770 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 --2- 172.21.15.64:0/37547540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48410ae10 0x7fe48410d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 --2- 172.21.15.64:0/37547540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe484073660 0x7fe48410a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 -- 172.21.15.64:0/37547540 >> 172.21.15.64:0/37547540 conn(0x7fe48406e590 msgr2=0x7fe4840709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 -- 172.21.15.64:0/37547540 shutdown_connections 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.961+0000 7fe4896e0640 1 -- 172.21.15.64:0/37547540 wait complete. 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.962+0000 7fe4896e0640 1 Processor -- start 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.962+0000 7fe4896e0640 1 -- start start 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe4896e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe484073660 0x7fe484114f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe4896e0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48410ae10 0x7fe484115460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe4896e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe4841b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe4896e0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe48411c030 con 0x7fe48410d910 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe4896e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe48411beb0 con 0x7fe484073660 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe4896e0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe48411c1b0 con 0x7fe48410ae10 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe488edf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe4841b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe488edf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe4841b6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51570/0 (socket says 172.21.15.64:51570) 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.963+0000 7fe488edf640 1 -- 172.21.15.64:0/555947014 learned_addr learned my addr 172.21.15.64:0/555947014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.964+0000 7fe488edf640 1 -- 172.21.15.64:0/555947014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48410ae10 msgr2=0x7fe484115460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.964+0000 7fe488edf640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48410ae10 0x7fe484115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.964+0000 7fe488edf640 1 -- 172.21.15.64:0/555947014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe484073660 msgr2=0x7fe484114f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.964+0000 7fe488edf640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe484073660 0x7fe484114f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.964+0000 7fe488edf640 1 -- 172.21.15.64:0/555947014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4841b6c80 con 0x7fe48410d910 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.964+0000 7fe488edf640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe4841b6740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe47802a880 tx=0x7fe4780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.965+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47803f070 con 0x7fe48410d910 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.965+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe478033e20 con 0x7fe48410d910 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.965+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478037070 con 0x7fe48410d910 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.965+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4841b6e50 con 0x7fe48410d910 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.965+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4841b7200 con 0x7fe48410d910 2024-05-09T09:14:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.967+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe484068570 con 0x7fe48410d910 2024-05-09T09:14:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.970+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe478042070 con 0x7fe48410d910 2024-05-09T09:14:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.971+0000 7fe4817fa640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe46c0a2d60 0x7fe46c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.971+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe4780ef4d0 con 0x7fe48410d910 2024-05-09T09:14:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.972+0000 7fe483fff640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe46c0a2d60 0x7fe46c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.972+0000 7fe483fff640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe46c0a2d60 0x7fe46c0a5220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00abb0 tx=0x7fe47c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:08.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:08.977+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe4780bc0c0 con 0x7fe48410d910 2024-05-09T09:14:08.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:08 smithi179 ceph-mon[34086]: pgmap v520: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.7 MiB/s wr, 878 op/s 2024-05-09T09:14:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:08 smithi139 ceph-mon[33599]: pgmap v520: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.7 MiB/s wr, 878 op/s 2024-05-09T09:14:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.443+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe48402d2e0 con 0x7fe48410d910 2024-05-09T09:14:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.445+0000 7fe4817fa640 1 -- 172.21.15.64:0/555947014 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11886 (secure 0 0 0) 0x7fe48402d2e0 con 0x7fe48410d910 2024-05-09T09:14:09.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:14:09.445 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:13:45.084962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:14:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.450+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe46c0a2d60 msgr2=0x7fe46c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.450+0000 7fe4896e0640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe46c0a2d60 0x7fe46c0a5220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00abb0 tx=0x7fe47c009250 comp rx=0 tx=0).stop 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.450+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 msgr2=0x7fe4841b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.450+0000 7fe4896e0640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe4841b6740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe47802a880 tx=0x7fe4780042f0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.451+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 shutdown_connections 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.451+0000 7fe4896e0640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe46c0a2d60 0x7fe46c0a5220 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.451+0000 7fe4896e0640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe48410d910 0x7fe4841b6740 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.451+0000 7fe4896e0640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48410ae10 0x7fe484115460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.451+0000 7fe4896e0640 1 --2- 172.21.15.64:0/555947014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe484073660 0x7fe484114f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.451+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 >> 172.21.15.64:0/555947014 conn(0x7fe48406e590 msgr2=0x7fe484108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.453+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 shutdown_connections 2024-05-09T09:14:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.453+0000 7fe4896e0640 1 -- 172.21.15.64:0/555947014 wait complete. 2024-05-09T09:14:09.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 -- 172.21.15.64:0/4124888877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c108f70 msgr2=0x7fa75c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 --2- 172.21.15.64:0/4124888877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c108f70 0x7fa75c1093d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa754009ca0 tx=0x7fa75402a8d0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 -- 172.21.15.64:0/4124888877 shutdown_connections 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 --2- 172.21.15.64:0/4124888877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c109910 0x7fa75c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 --2- 172.21.15.64:0/4124888877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c108f70 0x7fa75c1093d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 --2- 172.21.15.64:0/4124888877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c072d20 0x7fa75c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.675+0000 7fa763363640 1 -- 172.21.15.64:0/4124888877 >> 172.21.15.64:0/4124888877 conn(0x7fa75c06e740 msgr2=0x7fa75c070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.676+0000 7fa763363640 1 -- 172.21.15.64:0/4124888877 shutdown_connections 2024-05-09T09:14:09.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.676+0000 7fa763363640 1 -- 172.21.15.64:0/4124888877 wait complete. 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 Processor -- start 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 -- start start 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c072d20 0x7fa75c10e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 0x7fa75c10e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c109910 0x7fa75c1158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c117ca0 con 0x7fa75c108f70 2024-05-09T09:14:09.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c117b20 con 0x7fa75c109910 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa763363640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c117e20 con 0x7fa75c072d20 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa7608fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 0x7fa75c10e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa7608fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 0x7fa75c10e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51588/0 (socket says 172.21.15.64:51588) 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa7608fe640 1 -- 172.21.15.64:0/4036090733 learned_addr learned my addr 172.21.15.64:0/4036090733 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa7610ff640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c072d20 0x7fa75c10e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.677+0000 7fa761900640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c109910 0x7fa75c1158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.678+0000 7fa7610ff640 1 -- 172.21.15.64:0/4036090733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c109910 msgr2=0x7fa75c1158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.678+0000 7fa7610ff640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c109910 0x7fa75c1158f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.678+0000 7fa7610ff640 1 -- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 msgr2=0x7fa75c10e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.678+0000 7fa7610ff640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 0x7fa75c10e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.678+0000 7fa7610ff640 1 -- 172.21.15.64:0/4036090733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c1b6ba0 con 0x7fa75c072d20 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.678+0000 7fa7610ff640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c072d20 0x7fa75c10e340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00c970 tx=0x7fa74c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.679+0000 7fa7527fc640 1 -- 172.21.15.64:0/4036090733 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c007bb0 con 0x7fa75c072d20 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.679+0000 7fa7527fc640 1 -- 172.21.15.64:0/4036090733 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa74c007d50 con 0x7fa75c072d20 2024-05-09T09:14:09.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.679+0000 7fa7527fc640 1 -- 172.21.15.64:0/4036090733 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00f450 con 0x7fa75c072d20 2024-05-09T09:14:09.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.679+0000 7fa761900640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c109910 0x7fa75c1158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:09.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.679+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c1b6e90 con 0x7fa75c072d20 2024-05-09T09:14:09.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.679+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c1b72a0 con 0x7fa75c072d20 2024-05-09T09:14:09.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.680+0000 7fa7608fe640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 0x7fa75c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:09.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.683+0000 7fa76607b740 1 Processor -- start 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 -- start start 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651437550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437b20 0x55b65143c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b65143f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b65139bc40 con 0x55b65143cc90 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b651343580 con 0x55b651437b20 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa76607b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b651397c00 con 0x55b651437130 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa7527fc640 1 -- 172.21.15.64:0/4036090733 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa74c004e70 con 0x7fa75c072d20 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa761900640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651437550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa7608fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b65143f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa761900640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651437550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56920/0 (socket says 172.21.15.64:56920) 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa761900640 1 -- 172.21.15.64:0/814517346 learned_addr learned my addr 172.21.15.64:0/814517346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.684+0000 7fa7610ff640 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437b20 0x55b65143c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.685+0000 7fa7608fe640 1 -- 172.21.15.64:0/814517346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 msgr2=0x55b651437550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.685+0000 7fa7527fc640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7480a2ce0 0x7fa7480a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.686+0000 7fa7527fc640 1 -- 172.21.15.64:0/4036090733 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa74c0ceeb0 con 0x7fa75c072d20 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.686+0000 7fa7608fe640 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651437550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.686+0000 7fa7608fe640 1 -- 172.21.15.64:0/814517346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437b20 msgr2=0x55b65143c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.686+0000 7fa7608fe640 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437b20 0x55b65143c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.686+0000 7fa7608fe640 1 -- 172.21.15.64:0/814517346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6510af3f0 con 0x55b65143cc90 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.686+0000 7fa7610ff640 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437b20 0x55b65143c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.687+0000 7fa7608fe640 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b65143f120 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa754031a60 tx=0x7fa754009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0305332a88eac9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.687+0000 7fa761900640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7480a2ce0 0x7fa7480a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.687+0000 7fa7477fe640 1 -- 172.21.15.64:0/814517346 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7540307d0 con 0x55b65143cc90 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.687+0000 7fa7477fe640 1 -- 172.21.15.64:0/814517346 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa754036030 con 0x55b65143cc90 2024-05-09T09:14:09.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.687+0000 7fa7477fe640 1 -- 172.21.15.64:0/814517346 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754004330 con 0x55b65143cc90 2024-05-09T09:14:09.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.688+0000 7fa76607b740 1 -- 172.21.15.64:0/814517346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 msgr2=0x55b65143f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.688+0000 7fa76607b740 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b65143f120 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa754031a60 tx=0x7fa754009930 comp rx=0 tx=0).stop 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.688+0000 7fa761900640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7480a2ce0 0x7fa7480a51a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55b651438380 tx=0x7fa758004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 -- 172.21.15.64:0/814517346 shutdown_connections 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b65143f120 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437b20 0x55b65143c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 --2- 172.21.15.64:0/814517346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651437550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 -- 172.21.15.64:0/814517346 >> 172.21.15.64:0/814517346 conn(0x55b6514327a0 msgr2=0x55b651432ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 -- 172.21.15.64:0/814517346 shutdown_connections 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.689+0000 7fa76607b740 1 -- 172.21.15.64:0/814517346 wait complete. 2024-05-09T09:14:09.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.690+0000 7fa76607b740 1 -- start start 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.690+0000 7fa76607b740 1 Processor -- start 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.690+0000 7fa76607b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.690+0000 7fa76607b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651441aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.690+0000 7fa76607b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b651448ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa76607b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b65143fbe0 con 0x55b651437b20 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa76607b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b651356c40 con 0x55b65143cc90 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa76607b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b65143fd60 con 0x55b651437130 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7608fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651441aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56934/0 (socket says 172.21.15.64:56934) 2024-05-09T09:14:09.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 -- 172.21.15.64:0/2987058267 learned_addr learned my addr 172.21.15.64:0/2987058267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa76607b740 10 client.0 _ll_register_callbacks cb 0x55b6510aebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa761900640 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b651448ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 -- 172.21.15.64:0/2987058267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 msgr2=0x55b651448ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b651448ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 -- 172.21.15.64:0/2987058267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 msgr2=0x55b651441aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651441aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 -- 172.21.15.64:0/2987058267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6511797d0 con 0x55b651437130 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa7610ff640 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa74c007590 tx=0x7fa74c007f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c368ab170e84ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.691+0000 7fa744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.692+0000 7fa744ff9640 1 -- 172.21.15.64:0/2987058267 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c01d040 con 0x55b651437130 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.692+0000 7fa744ff9640 1 -- 172.21.15.64:0/2987058267 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa74c0a3d60 con 0x55b651437130 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.692+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa754036030 con 0x55b651437130 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.692+0000 7fa744ff9640 1 -- 172.21.15.64:0/2987058267 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa74c09aec0 con 0x55b651437130 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.692+0000 7fa76607b740 10 client.26482 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.692+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b65144bf50 con 0x55b651437130 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa744ff9640 1 -- 172.21.15.64:0/2987058267 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa754036030 con 0x55b651437130 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa744ff9640 1 -- 172.21.15.64:0/2987058267 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fa74c005160 con 0x55b651437130 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa76607b740 10 client.26482 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa76607b740 10 client.26482 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa76607b740 20 client.26482 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa76607b740 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b65144d6c0 0x55b65144fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa76607b740 4 client.26482 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.693+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6513a9370 con 0x55b65144d6c0 2024-05-09T09:14:09.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.694+0000 7fa7608fe640 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b65144d6c0 0x55b65144fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.694+0000 7fa7608fe640 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b65144d6c0 0x55b65144fb80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=548622bf928f6727 server_cookie=43e5daedbf7a1d65 in_seq=0 out_seq=0 2024-05-09T09:14:09.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.694+0000 7fa744ff9640 10 client.26482 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:09.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa744ff9640 1 -- 172.21.15.64:0/2987058267 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6513a9370 con 0x55b65144d6c0 2024-05-09T09:14:09.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa744ff9640 10 client.26482 handle_command_reply: tid=0 2024-05-09T09:14:09.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 client.26482 shutdown 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 msgr2=0x55b651441560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa74c007590 tx=0x7fa74c007f00 comp rx=0 tx=0).stop 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 shutdown_connections 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b65144d6c0 0x55b65144fb80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b651448ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651441aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 --2- 172.21.15.64:0/2987058267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441560 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa744ff9640 0 client.26482 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 >> 172.21.15.64:0/2987058267 conn(0x55b6514327a0 msgr2=0x55b6514319c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.695+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 shutdown_connections 2024-05-09T09:14:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.696+0000 7fa76607b740 1 -- 172.21.15.64:0/2987058267 wait complete. 2024-05-09T09:14:09.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.696+0000 7fa76607b740 20 client.26482 trim_cache size 0 max 16384 2024-05-09T09:14:09.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.724+0000 7fa76607b740 1 Processor -- start 2024-05-09T09:14:09.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.724+0000 7fa76607b740 1 -- start start 2024-05-09T09:14:09.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa76607b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa76607b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651448aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa76607b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b6514416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa76607b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1c40a0 con 0x55b651437b20 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa76607b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1c18e0 con 0x55b65143cc90 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa76607b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1bf730 con 0x55b651437130 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa7610ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa7610ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56942/0 (socket says 172.21.15.64:56942) 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa7610ff640 1 -- 172.21.15.64:0/3593595019 learned_addr learned my addr 172.21.15.64:0/3593595019 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa7608fe640 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651448aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.725+0000 7fa761900640 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b6514416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa7608fe640 1 -- 172.21.15.64:0/3593595019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 msgr2=0x55b651441170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa7608fe640 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa7608fe640 1 -- 172.21.15.64:0/3593595019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 msgr2=0x55b6514416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa7608fe640 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b6514416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa7608fe640 1 -- 172.21.15.64:0/3593595019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b651437f00 con 0x55b651437b20 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa761900640 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b6514416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:09.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.726+0000 7fa7608fe640 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651448aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa754000f80 tx=0x7fa754004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ecc8828b108cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.727+0000 7fa746ffd640 1 -- 172.21.15.64:0/3593595019 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75404a040 con 0x55b651437b20 2024-05-09T09:14:09.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.727+0000 7fa746ffd640 1 -- 172.21.15.64:0/3593595019 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa754036070 con 0x55b651437b20 2024-05-09T09:14:09.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.727+0000 7fa746ffd640 1 -- 172.21.15.64:0/3593595019 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754030b40 con 0x55b651437b20 2024-05-09T09:14:09.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.728+0000 7fa76607b740 1 -- 172.21.15.64:0/3593595019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 msgr2=0x55b651448aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.728+0000 7fa76607b740 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651448aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa754000f80 tx=0x7fa754004a60 comp rx=0 tx=0).stop 2024-05-09T09:14:09.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.729+0000 7fa76607b740 1 -- 172.21.15.64:0/3593595019 shutdown_connections 2024-05-09T09:14:09.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.729+0000 7fa76607b740 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b65143cc90 0x55b6514416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.729+0000 7fa76607b740 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b651437b20 0x55b651448aa0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.729+0000 7fa76607b740 1 --2- 172.21.15.64:0/3593595019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437130 0x55b651441170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.729+0000 7fa76607b740 1 -- 172.21.15.64:0/3593595019 >> 172.21.15.64:0/3593595019 conn(0x55b6514327a0 msgr2=0x55b651438fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.731+0000 7fa76607b740 1 -- 172.21.15.64:0/3593595019 shutdown_connections 2024-05-09T09:14:09.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.732+0000 7fa76607b740 1 -- 172.21.15.64:0/3593595019 wait complete. 2024-05-09T09:14:09.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.732+0000 7fa76607b740 1 -- start start 2024-05-09T09:14:09.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.732+0000 7fa76607b740 1 Processor -- start 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.733+0000 7fa76607b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437130 0x55b651445900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.733+0000 7fa76607b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 0x55b651445e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.733+0000 7fa76607b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b6514b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.733+0000 7fa7608fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 0x55b651445e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 0x55b651445e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56958/0 (socket says 172.21.15.64:56958) 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 -- 172.21.15.64:0/1270208758 learned_addr learned my addr 172.21.15.64:0/1270208758 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1bfda0 con 0x55b65143cc90 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1c1210 con 0x55b651437130 2024-05-09T09:14:09.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1c0470 con 0x55b651437b20 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa76607b740 10 client.0 _ll_register_callbacks cb 0x7fa74c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 -- 172.21.15.64:0/1270208758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437130 msgr2=0x55b651445900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437130 0x55b651445900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 -- 172.21.15.64:0/1270208758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 msgr2=0x55b6514b1cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b6514b1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.734+0000 7fa7608fe640 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6514b21f0 con 0x55b651437b20 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.735+0000 7fa7608fe640 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 0x55b651445e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa754031930 tx=0x7fa754009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b44ac296628a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.735+0000 7fa744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:09.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.735+0000 7fa744ff9640 1 -- 172.21.15.64:0/1270208758 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754030c90 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.735+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa754036070 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.736+0000 7fa744ff9640 1 -- 172.21.15.64:0/1270208758 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa754054180 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.736+0000 7fa744ff9640 1 -- 172.21.15.64:0/1270208758 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa754065050 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.736+0000 7fa76607b740 10 client.26488 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.736+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b6514b33b0 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa744ff9640 1 -- 172.21.15.64:0/1270208758 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa754036070 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa744ff9640 1 -- 172.21.15.64:0/1270208758 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fa754043070 con 0x55b651437b20 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa76607b740 10 client.26488 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa76607b740 10 client.26488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa76607b740 20 client.26488 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa76607b740 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b6514b3910 0x55b6514b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa76607b740 4 client.26488 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:14:09.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6513a36c0 con 0x55b6514b3910 2024-05-09T09:14:09.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.737+0000 7fa7610ff640 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b6514b3910 0x55b6514b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.738+0000 7fa7610ff640 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b6514b3910 0x55b6514b5d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bd64455d0e9382 server_cookie=171d186291e65de in_seq=0 out_seq=0 2024-05-09T09:14:09.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.738+0000 7fa744ff9640 10 client.26488 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:09.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.738+0000 7fa744ff9640 1 -- 172.21.15.64:0/1270208758 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6513a36c0 con 0x55b6514b3910 2024-05-09T09:14:09.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.738+0000 7fa744ff9640 10 client.26488 handle_command_reply: tid=0 2024-05-09T09:14:09.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.739+0000 7fa76607b740 1 client.26488 shutdown 2024-05-09T09:14:09.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 msgr2=0x55b651445e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 0x55b651445e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa754031930 tx=0x7fa754009930 comp rx=0 tx=0).stop 2024-05-09T09:14:09.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 shutdown_connections 2024-05-09T09:14:09.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55b6514b3910 0x55b6514b5d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b65143cc90 0x55b6514b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b651437b20 0x55b651445e40 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa754031930 tx=0x7fa754009930 comp rx=0 tx=0).stop 2024-05-09T09:14:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 --2- 172.21.15.64:0/1270208758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b651437130 0x55b651445900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 >> 172.21.15.64:0/1270208758 conn(0x55b6514327a0 msgr2=0x55b651441170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.740+0000 7fa744ff9640 0 client.26488 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.741+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 shutdown_connections 2024-05-09T09:14:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.741+0000 7fa76607b740 1 -- 172.21.15.64:0/1270208758 wait complete. 2024-05-09T09:14:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.741+0000 7fa76607b740 20 client.26488 trim_cache size 0 max 16384 2024-05-09T09:14:09.741 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:14:09.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.742+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7480a2ce0 msgr2=0x7fa7480a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.742+0000 7fa763363640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7480a2ce0 0x7fa7480a51a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55b651438380 tx=0x7fa758004730 comp rx=0 tx=0).stop 2024-05-09T09:14:09.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.742+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c072d20 msgr2=0x7fa75c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.742+0000 7fa763363640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c072d20 0x7fa75c10e340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00c970 tx=0x7fa74c00ce30 comp rx=0 tx=0).stop 2024-05-09T09:14:09.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.743+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 shutdown_connections 2024-05-09T09:14:09.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.743+0000 7fa763363640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7480a2ce0 0x7fa7480a51a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.743+0000 7fa763363640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa75c109910 0x7fa75c1158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.743+0000 7fa763363640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa75c108f70 0x7fa75c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.743+0000 7fa763363640 1 --2- 172.21.15.64:0/4036090733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75c072d20 0x7fa75c10e340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.743+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 >> 172.21.15.64:0/4036090733 conn(0x7fa75c06e740 msgr2=0x7fa75c070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.744+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 shutdown_connections 2024-05-09T09:14:09.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.745+0000 7fa763363640 1 -- 172.21.15.64:0/4036090733 wait complete. 2024-05-09T09:14:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:09 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/555947014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:14:09.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 -- 172.21.15.64:0/1899347362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 msgr2=0x7f729810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 --2- 172.21.15.64:0/1899347362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f729810e170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7290009be0 tx=0x7f729002a6f0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 -- 172.21.15.64:0/1899347362 shutdown_connections 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 --2- 172.21.15.64:0/1899347362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f729810e170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 --2- 172.21.15.64:0/1899347362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7298108f10 0x7f7298109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 --2- 172.21.15.64:0/1899347362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7298072c70 0x7f7298073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 -- 172.21.15.64:0/1899347362 >> 172.21.15.64:0/1899347362 conn(0x7f729806e740 msgr2=0x7f7298070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.953+0000 7f729db8c640 1 -- 172.21.15.64:0/1899347362 shutdown_connections 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.954+0000 7f729db8c640 1 -- 172.21.15.64:0/1899347362 wait complete. 2024-05-09T09:14:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 Processor -- start 2024-05-09T09:14:09.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 -- start start 2024-05-09T09:14:09.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7298072c70 0x7f72981a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7298108f10 0x7f72981a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f72981adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f729811c070 con 0x7f7298072c70 2024-05-09T09:14:09.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f729811bef0 con 0x7f7298108f10 2024-05-09T09:14:09.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.955+0000 7f729db8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f729811c1f0 con 0x7f72981098b0 2024-05-09T09:14:09.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f7297fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7298108f10 0x7f72981a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f72981adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f72981adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56980/0 (socket says 172.21.15.64:56980) 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 -- 172.21.15.64:0/2485670329 learned_addr learned my addr 172.21.15.64:0/2485670329 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 -- 172.21.15.64:0/2485670329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7298108f10 msgr2=0x7f72981a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7298108f10 0x7f72981a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 -- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7298072c70 msgr2=0x7f72981a6720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:14:09.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7298072c70 0x7f72981a6720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.956+0000 7f729d38b640 1 -- 172.21.15.64:0/2485670329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72981ae340 con 0x7f72981098b0 2024-05-09T09:14:09.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.957+0000 7f729d38b640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f72981adcd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7290009b30 tx=0x7f7290030ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.957+0000 7f7295ffb640 1 -- 172.21.15.64:0/2485670329 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290041020 con 0x7f72981098b0 2024-05-09T09:14:09.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.957+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72981ae5d0 con 0x7f72981098b0 2024-05-09T09:14:09.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.958+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981aeac0 con 0x7f72981098b0 2024-05-09T09:14:09.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.958+0000 7f7295ffb640 1 -- 172.21.15.64:0/2485670329 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7290037040 con 0x7f72981098b0 2024-05-09T09:14:09.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.958+0000 7f7295ffb640 1 -- 172.21.15.64:0/2485670329 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290036070 con 0x7f72981098b0 2024-05-09T09:14:09.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.962+0000 7f72a1bd8740 1 Processor -- start 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f7295ffb640 1 -- 172.21.15.64:0/2485670329 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7290044020 con 0x7f72981098b0 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 -- start start 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa35520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 0x555fcaa3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa3ac60 0x555fcaa3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fca999c40 con 0x555fcaa35af0 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555fca941580 con 0x555fcaa3ac60 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.964+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555fca995c00 con 0x555fcaa35100 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f7297fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa35520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 0x555fcaa3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 0x555fcaa3a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51618/0 (socket says 172.21.15.64:51618) 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 -- 172.21.15.64:0/2288446258 learned_addr learned my addr 172.21.15.64:0/2288446258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f7297fff640 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa3ac60 0x555fcaa3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 -- 172.21.15.64:0/2288446258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 msgr2=0x555fcaa35520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa35520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 -- 172.21.15.64:0/2288446258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa3ac60 msgr2=0x555fcaa3d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa3ac60 0x555fcaa3d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f729cb8a640 1 -- 172.21.15.64:0/2288446258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fca6ad3f0 con 0x555fcaa35af0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.965+0000 7f7297fff640 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa3ac60 0x555fcaa3d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.966+0000 7f7295ffb640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72800a2d90 0x7f72800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.966+0000 7f729cb8a640 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 0x555fcaa3a5f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7288009bd0 tx=0x7f728800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3b94969c460c7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.966+0000 7f7295ffb640 1 -- 172.21.15.64:0/2485670329 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72900efc70 con 0x7f72981098b0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.966+0000 7f7297fff640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72800a2d90 0x7f72800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.966+0000 7f727effd640 1 -- 172.21.15.64:0/2288446258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288005020 con 0x555fcaa35af0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.967+0000 7f727effd640 1 -- 172.21.15.64:0/2288446258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72880047a0 con 0x555fcaa35af0 2024-05-09T09:14:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.967+0000 7f727effd640 1 -- 172.21.15.64:0/2288446258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288010690 con 0x555fcaa35af0 2024-05-09T09:14:09.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.967+0000 7f72a1bd8740 1 -- 172.21.15.64:0/2288446258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 msgr2=0x555fcaa3a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.967+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 0x555fcaa3a5f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7288009bd0 tx=0x7f728800e720 comp rx=0 tx=0).stop 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 -- 172.21.15.64:0/2288446258 shutdown_connections 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa3ac60 0x555fcaa3d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35af0 0x555fcaa3a5f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/2288446258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa35520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 -- 172.21.15.64:0/2288446258 >> 172.21.15.64:0/2288446258 conn(0x555fcaa30770 msgr2=0x555fcaa30b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f7297fff640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72800a2d90 0x7f72800a5250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x555fcaa36350 tx=0x7f728c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 -- 172.21.15.64:0/2288446258 shutdown_connections 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 -- 172.21.15.64:0/2288446258 wait complete. 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.968+0000 7f72a1bd8740 1 -- start start 2024-05-09T09:14:09.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 Processor -- start 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa46ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fcaa3dc10 con 0x555fcaa3ac60 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555fca954c40 con 0x555fcaa35af0 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555fcaa3dd90 con 0x555fcaa35100 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f729d38b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa46ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f729d38b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa46ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51622/0 (socket says 172.21.15.64:51622) 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f729d38b640 1 -- 172.21.15.64:0/1649328228 learned_addr learned my addr 172.21.15.64:0/1649328228 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:09.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f7297fff640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f729cb8a640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.969+0000 7f72a1bd8740 10 client.0 _ll_register_callbacks cb 0x555fca6acbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7297fff640 1 -- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 msgr2=0x555fcaa3f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7297fff640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7297fff640 1 -- 172.21.15.64:0/1649328228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 msgr2=0x555fcaa46ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7297fff640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa46ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7297fff640 1 -- 172.21.15.64:0/1649328228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fca7777d0 con 0x555fcaa35af0 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f729d38b640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa46ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f729cb8a640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7297fff640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa3fa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f728c009690 tx=0x7f728c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b8ec88a67f8a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:09.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.970+0000 7f7277fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:09.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f7277fff640 1 -- 172.21.15.64:0/1649328228 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c003e50 con 0x555fcaa35af0 2024-05-09T09:14:09.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f7277fff640 1 -- 172.21.15.64:0/1649328228 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f728c00f9d0 con 0x555fcaa35af0 2024-05-09T09:14:09.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72880047a0 con 0x555fcaa35af0 2024-05-09T09:14:09.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f7277fff640 1 -- 172.21.15.64:0/1649328228 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f728c034050 con 0x555fcaa35af0 2024-05-09T09:14:09.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f72a1bd8740 10 client.26330 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:09.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555fcaa49ec0 con 0x555fcaa35af0 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.971+0000 7f7277fff640 1 -- 172.21.15.64:0/1649328228 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f728c00fbd0 con 0x555fcaa35af0 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f7277fff640 1 -- 172.21.15.64:0/1649328228 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f728c028030 con 0x555fcaa35af0 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f72a1bd8740 10 client.26330 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f72a1bd8740 10 client.26330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f72a1bd8740 20 client.26330 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaa4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f72a1bd8740 4 client.26330 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f72880047a0 con 0x555fcaa4b630 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.972+0000 7f729cb8a640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaa4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.974+0000 7f729cb8a640 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaa4da90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ae307d1d2a2d110 server_cookie=c0410ab2db49c4ca in_seq=0 out_seq=0 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.974+0000 7f7277fff640 10 client.26330 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.974+0000 7f7277fff640 1 -- 172.21.15.64:0/1649328228 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f72880047a0 con 0x555fcaa4b630 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.974+0000 7f7277fff640 10 client.26330 handle_command_reply: tid=0 2024-05-09T09:14:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.974+0000 7f72a1bd8740 1 client.26330 shutdown 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 msgr2=0x555fcaa3fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa3fa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f728c009690 tx=0x7f728c0046c0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 shutdown_connections 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaa4da90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa46ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f7277fff640 0 client.26330 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa3fa70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/1649328228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 >> 172.21.15.64:0/1649328228 conn(0x555fcaa30770 msgr2=0x555fcaa2f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 shutdown_connections 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.975+0000 7f72a1bd8740 1 -- 172.21.15.64:0/1649328228 wait complete. 2024-05-09T09:14:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:09.976+0000 7f72a1bd8740 20 client.26330 trim_cache size 0 max 16384 2024-05-09T09:14:09.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:09 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/555947014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:14:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.005+0000 7f72a1bd8740 1 Processor -- start 2024-05-09T09:14:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.005+0000 7f72a1bd8740 1 -- start start 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.005+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.005+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa46a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa53360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7284012520 con 0x555fcaa3ac60 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f728400fd60 con 0x555fcaa35af0 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f72a1bd8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f728400dbb0 con 0x555fcaa35100 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729d38b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa53360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57006/0 (socket says 172.21.15.64:57006) 2024-05-09T09:14:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 -- 172.21.15.64:0/693326528 learned_addr learned my addr 172.21.15.64:0/693326528 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 -- 172.21.15.64:0/693326528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 msgr2=0x555fcaa46a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa46a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 -- 172.21.15.64:0/693326528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 msgr2=0x555fcaa53360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa53360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.006+0000 7f729cb8a640 1 -- 172.21.15.64:0/693326528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fcaa300c0 con 0x555fcaa35100 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.007+0000 7f729cb8a640 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3ef30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7288009960 tx=0x7f72880093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef709a1f556e7359 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.007+0000 7f727e7fc640 1 -- 172.21.15.64:0/693326528 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728801d040 con 0x555fcaa35100 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.007+0000 7f727e7fc640 1 -- 172.21.15.64:0/693326528 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72880044f0 con 0x555fcaa35100 2024-05-09T09:14:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.008+0000 7f727e7fc640 1 -- 172.21.15.64:0/693326528 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288003b70 con 0x555fcaa35100 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 -- 172.21.15.64:0/693326528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 msgr2=0x555fcaa3ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3ef30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7288009960 tx=0x7f72880093c0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 -- 172.21.15.64:0/693326528 shutdown_connections 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa3ac60 0x555fcaa53360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa46a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/693326528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa35100 0x555fcaa3ef30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.009+0000 7f72a1bd8740 1 -- 172.21.15.64:0/693326528 >> 172.21.15.64:0/693326528 conn(0x555fcaa30770 msgr2=0x555fcaa3e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.010+0000 7f72a1bd8740 1 -- 172.21.15.64:0/693326528 shutdown_connections 2024-05-09T09:14:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.010+0000 7f72a1bd8740 1 -- 172.21.15.64:0/693326528 wait complete. 2024-05-09T09:14:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.010+0000 7f72a1bd8740 1 -- start start 2024-05-09T09:14:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.011+0000 7f72a1bd8740 1 Processor -- start 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.011+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35100 0x555fcaa43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.011+0000 7f72a1bd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f729cb8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35100 0x555fcaa43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f729cb8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35100 0x555fcaa43a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51638/0 (socket says 172.21.15.64:51638) 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f729cb8a640 1 -- 172.21.15.64:0/3908846310 learned_addr learned my addr 172.21.15.64:0/3908846310 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa3ac60 0x555fcaa44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f7297fff640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f728400e220 con 0x555fcaa35100 2024-05-09T09:14:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f728400f690 con 0x555fcaa35af0 2024-05-09T09:14:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.012+0000 7f729d38b640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa3ac60 0x555fcaa44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f728400e8f0 con 0x555fcaa3ac60 2024-05-09T09:14:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7297fff640 1 -- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa3ac60 msgr2=0x555fcaa44fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7297fff640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa3ac60 0x555fcaa44fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7297fff640 1 -- 172.21.15.64:0/3908846310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35100 msgr2=0x555fcaa43a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7297fff640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35100 0x555fcaa43a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7297fff640 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fcaa455a0 con 0x555fcaa35af0 2024-05-09T09:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f72a1bd8740 10 client.0 _ll_register_callbacks cb 0x7f728c009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7297fff640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa43fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f728c020490 tx=0x7f728c0209b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1215e774711f05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.013+0000 7f7277fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.015+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72880044f0 con 0x555fcaa35af0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f7277fff640 1 -- 172.21.15.64:0/3908846310 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c03d060 con 0x555fcaa35af0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f7277fff640 1 -- 172.21.15.64:0/3908846310 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f728c020ad0 con 0x555fcaa35af0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f7277fff640 1 -- 172.21.15.64:0/3908846310 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f728c044050 con 0x555fcaa35af0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f72a1bd8740 10 client.26333 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=39}) -- 0x555fcaa46740 con 0x555fcaa35af0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f7277fff640 1 -- 172.21.15.64:0/3908846310 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f728c028070 con 0x555fcaa35af0 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f72a1bd8740 10 client.26333 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f72a1bd8740 10 client.26333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.016+0000 7f72a1bd8740 20 client.26333 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.017+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaab3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.017+0000 7f72a1bd8740 4 client.26333 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.017+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fca9a7370 con 0x555fcaa4b630 2024-05-09T09:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.019+0000 7f729cb8a640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaab3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.020+0000 7f729cb8a640 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaab3be0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fac9576585f00738 server_cookie=34c6882f591465aa in_seq=0 out_seq=0 2024-05-09T09:14:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.020+0000 7f7277fff640 10 client.26333 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:10.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.021+0000 7f7277fff640 1 -- 172.21.15.64:0/3908846310 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fca9a7370 con 0x555fcaa4b630 2024-05-09T09:14:10.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.021+0000 7f7277fff640 10 client.26333 handle_command_reply: tid=0 2024-05-09T09:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.022+0000 7f72a1bd8740 1 client.26333 shutdown 2024-05-09T09:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.023+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 msgr2=0x555fcaa43fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.023+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa43fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f728c020490 tx=0x7f728c0209b0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.023+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 shutdown_connections 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.023+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fcaa3ac60 0x555fcaa44fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fcaa35af0 0x555fcaa43fa0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f728c020490 tx=0x7f728c0209b0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fcaa4b630 0x555fcaab3be0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f7277fff640 0 client.26333 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f72a1bd8740 1 --2- 172.21.15.64:0/3908846310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fcaa35100 0x555fcaa43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 >> 172.21.15.64:0/3908846310 conn(0x555fcaa30770 msgr2=0x555fcaa33720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 shutdown_connections 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.024+0000 7f72a1bd8740 1 -- 172.21.15.64:0/3908846310 wait complete. 2024-05-09T09:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.025+0000 7f72a1bd8740 20 client.26333 trim_cache size 0 max 16384 2024-05-09T09:14:10.025 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:14:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.026+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72800a2d90 msgr2=0x7f72800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.026+0000 7f729db8c640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72800a2d90 0x7f72800a5250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x555fcaa36350 tx=0x7f728c013040 comp rx=0 tx=0).stop 2024-05-09T09:14:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.026+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 msgr2=0x7f72981adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.026+0000 7f729db8c640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f72981adcd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7290009b30 tx=0x7f7290030ef0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 shutdown_connections 2024-05-09T09:14:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72800a2d90 0x7f72800a5250 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72981098b0 0x7f72981adcd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7298108f10 0x7f72981a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 --2- 172.21.15.64:0/2485670329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7298072c70 0x7f72981a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 >> 172.21.15.64:0/2485670329 conn(0x7f729806e740 msgr2=0x7f7298070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 shutdown_connections 2024-05-09T09:14:10.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.027+0000 7f729db8c640 1 -- 172.21.15.64:0/2485670329 wait complete. 2024-05-09T09:14:10.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:14:10.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:09 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/555947014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:14:10.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.238+0000 7f8ea5048640 1 -- 172.21.15.64:0/3236503401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90031880 con 0x7f8ea00fff40 2024-05-09T09:14:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3236503401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 msgr2=0x7f8ea0106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3236503401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 0x7f8ea0106f40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009900 tx=0x7f8e9002a410 comp rx=0 tx=0).stop 2024-05-09T09:14:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3236503401 shutdown_connections 2024-05-09T09:14:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3236503401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 0x7f8ea0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3236503401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 0x7f8ea0106f40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3236503401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 0x7f8ea00ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.239+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3236503401 >> 172.21.15.64:0/3236503401 conn(0x7f8ea00fb030 msgr2=0x7f8ea00fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.242+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3236503401 shutdown_connections 2024-05-09T09:14:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.242+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3236503401 wait complete. 2024-05-09T09:14:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.243+0000 7f8ea82ae640 1 Processor -- start 2024-05-09T09:14:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 -- start start 2024-05-09T09:14:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 0x7f8ea0102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 0x7f8ea0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 0x7f8ea0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea010d450 con 0x7f8ea00fff40 2024-05-09T09:14:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea010d2d0 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.244+0000 7f8ea82ae640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea010d5d0 con 0x7f8ea01074d0 2024-05-09T09:14:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.245+0000 7f8ea684b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 0x7f8ea0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.245+0000 7f8ea684b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 0x7f8ea0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57042/0 (socket says 172.21.15.64:57042) 2024-05-09T09:14:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.245+0000 7f8ea684b640 1 -- 172.21.15.64:0/3787830431 learned_addr learned my addr 172.21.15.64:0/3787830431 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.245+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 0x7f8ea0102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.246+0000 7f8ea5849640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 0x7f8ea0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.246+0000 7f8ea604a640 1 -- 172.21.15.64:0/3787830431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 msgr2=0x7f8ea0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.246+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 0x7f8ea0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.246+0000 7f8ea604a640 1 -- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 msgr2=0x7f8ea0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.246+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 0x7f8ea0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.246+0000 7f8ea604a640 1 -- 172.21.15.64:0/3787830431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea01a7e90 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.247+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 0x7f8ea0102840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.247+0000 7f8e977fe640 1 -- 172.21.15.64:0/3787830431 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c009c30 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.247+0000 7f8e977fe640 1 -- 172.21.15.64:0/3787830431 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e9c009dd0 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.247+0000 7f8e977fe640 1 -- 172.21.15.64:0/3787830431 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c010460 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.247+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea01a8180 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.247+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea01a8650 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.252+0000 7f8e977fe640 1 -- 172.21.15.64:0/3787830431 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e9c0035f0 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.253+0000 7f8eaafb7740 1 Processor -- start 2024-05-09T09:14:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.254+0000 7f8eaafb7740 1 -- start start 2024-05-09T09:14:10.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.254+0000 7f8e977fe640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e8c0a2ce0 0x7f8e8c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.255+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c3f40 0x555d567c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.255+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c4930 0x555d567c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.254+0000 7f8e977fe640 1 -- 172.21.15.64:0/3787830431 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e9c0cedb0 con 0x7f8ea00ff5e0 2024-05-09T09:14:10.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.256+0000 7f8ea684b640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e8c0a2ce0 0x7f8e8c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c3f40 0x555d567c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c3f40 0x555d567c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57050/0 (socket says 172.21.15.64:57050) 2024-05-09T09:14:10.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8ea684b640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e8c0a2ce0 0x7f8e8c0a51a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800e250 tx=0x7f8e9800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8ea5849640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c4930 0x555d567c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d56728c40 con 0x555d567c9aa0 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555d566d0580 con 0x555d567c4930 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.258+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555d56724c00 con 0x555d567c3f40 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.259+0000 7f8ea5849640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c4930 0x555d567c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53534/0 (socket says 172.21.15.64:53534) 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.259+0000 7f8ea5849640 1 -- 172.21.15.64:0/3519615062 learned_addr learned my addr 172.21.15.64:0/3519615062 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.259+0000 7f8ea684b640 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.260+0000 7f8ea684b640 1 -- 172.21.15.64:0/3519615062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c3f40 msgr2=0x555d567c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.260+0000 7f8ea684b640 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c3f40 0x555d567c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.260+0000 7f8ea684b640 1 -- 172.21.15.64:0/3519615062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c4930 msgr2=0x555d567c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.260+0000 7f8ea684b640 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c4930 0x555d567c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.260+0000 7f8ea684b640 1 -- 172.21.15.64:0/3519615062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5643c3f0 con 0x555d567c9aa0 2024-05-09T09:14:10.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.261+0000 7f8ea684b640 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567cbf30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800eae0 tx=0x7f8e98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab72324cfee2fe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.261+0000 7f8e94bfe640 1 -- 172.21.15.64:0/3519615062 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801b040 con 0x555d567c9aa0 2024-05-09T09:14:10.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.261+0000 7f8e94bfe640 1 -- 172.21.15.64:0/3519615062 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e98024070 con 0x555d567c9aa0 2024-05-09T09:14:10.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.261+0000 7f8e94bfe640 1 -- 172.21.15.64:0/3519615062 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98013990 con 0x555d567c9aa0 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.262+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3519615062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 msgr2=0x555d567cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567cbf30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800eae0 tx=0x7f8e98011000 comp rx=0 tx=0).stop 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3519615062 shutdown_connections 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567cbf30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c4930 0x555d567c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3519615062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c3f40 0x555d567c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3519615062 >> 172.21.15.64:0/3519615062 conn(0x555d567bf5b0 msgr2=0x555d567bf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.263+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3519615062 shutdown_connections 2024-05-09T09:14:10.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.264+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3519615062 wait complete. 2024-05-09T09:14:10.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.264+0000 7f8eaafb7740 1 -- start start 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 Processor -- start 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d567cca50 con 0x555d567c9aa0 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555d566e3c40 con 0x555d567c3f40 2024-05-09T09:14:10.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555d567ccbd0 con 0x555d567c4930 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8eaafb7740 10 client.0 _ll_register_callbacks cb 0x555d5643bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.265+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53536/0 (socket says 172.21.15.64:53536) 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 -- 172.21.15.64:0/3512777717 learned_addr learned my addr 172.21.15.64:0/3512777717 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea5849640 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 -- 172.21.15.64:0/3512777717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 msgr2=0x555d567ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 -- 172.21.15.64:0/3512777717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 msgr2=0x555d567d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 -- 172.21.15.64:0/3512777717 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d565067d0 con 0x555d567c3f40 2024-05-09T09:14:10.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8ea604a640 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567ce360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c097630 tx=0x7f8e9c09ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42bda03dd3d11555 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:10.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.266+0000 7f8e89ffb640 1 -- 172.21.15.64:0/3512777717 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0b4060 con 0x555d567c3f40 2024-05-09T09:14:10.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.267+0000 7f8e89ffb640 1 -- 172.21.15.64:0/3512777717 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e9c09f170 con 0x555d567c3f40 2024-05-09T09:14:10.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.267+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e98024070 con 0x555d567c3f40 2024-05-09T09:14:10.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.267+0000 7f8e89ffb640 1 -- 172.21.15.64:0/3512777717 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8e9c0b9e60 con 0x555d567c3f40 2024-05-09T09:14:10.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.268+0000 7f8eaafb7740 10 client.26345 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:10.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.268+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d567d8cc0 con 0x555d567c3f40 2024-05-09T09:14:10.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.268+0000 7f8e89ffb640 1 -- 172.21.15.64:0/3512777717 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e98024070 con 0x555d567c3f40 2024-05-09T09:14:10.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8e89ffb640 1 -- 172.21.15.64:0/3512777717 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f8e9c096f70 con 0x555d567c3f40 2024-05-09T09:14:10.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8eaafb7740 10 client.26345 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8eaafb7740 10 client.26345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8eaafb7740 20 client.26345 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d567da2f0 0x555d567dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8eaafb7740 4 client.26345 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.269+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d56736370 con 0x555d567da2f0 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.270+0000 7f8ea5849640 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d567da2f0 0x555d567dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.270+0000 7f8ea5849640 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d567da2f0 0x555d567dc7b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8457e219ba1cbb2a server_cookie=d81d25b562bf075 in_seq=0 out_seq=0 2024-05-09T09:14:10.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.270+0000 7f8e89ffb640 10 client.26345 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:10.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.271+0000 7f8e89ffb640 1 -- 172.21.15.64:0/3512777717 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d56736370 con 0x555d567da2f0 2024-05-09T09:14:10.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.271+0000 7f8e89ffb640 10 client.26345 handle_command_reply: tid=0 2024-05-09T09:14:10.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.271+0000 7f8eaafb7740 1 client.26345 shutdown 2024-05-09T09:14:10.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 msgr2=0x555d567ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567ce360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c097630 tx=0x7f8e9c09ba70 comp rx=0 tx=0).stop 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 shutdown_connections 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d567da2f0 0x555d567dc7b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/3512777717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567ce360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 >> 172.21.15.64:0/3512777717 conn(0x555d567bf5b0 msgr2=0x555d567be7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.272+0000 7f8e89ffb640 0 client.26345 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.273+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 shutdown_connections 2024-05-09T09:14:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.273+0000 7f8eaafb7740 1 -- 172.21.15.64:0/3512777717 wait complete. 2024-05-09T09:14:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.273+0000 7f8eaafb7740 20 client.26345 trim_cache size 0 max 16384 2024-05-09T09:14:10.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.300+0000 7f8eaafb7740 1 Processor -- start 2024-05-09T09:14:10.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.301+0000 7f8eaafb7740 1 -- start start 2024-05-09T09:14:10.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.301+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 0x555d567cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.301+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.301+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c9aa0 0x555d567c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.301+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84012590 con 0x555d567c3f40 2024-05-09T09:14:10.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 0x555d567cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 0x555d567cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51674/0 (socket says 172.21.15.64:51674) 2024-05-09T09:14:10.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 -- 172.21.15.64:0/1713794674 learned_addr learned my addr 172.21.15.64:0/1713794674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8400fdd0 con 0x555d567c9aa0 2024-05-09T09:14:10.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8400dc20 con 0x555d567c4930 2024-05-09T09:14:10.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 -- 172.21.15.64:0/1713794674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 msgr2=0x555d567d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 -- 172.21.15.64:0/1713794674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c9aa0 msgr2=0x555d567c2330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:14:10.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.302+0000 7f8ea604a640 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c9aa0 0x555d567c2330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.303+0000 7f8ea604a640 1 -- 172.21.15.64:0/1713794674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d567c0020 con 0x555d567c3f40 2024-05-09T09:14:10.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.303+0000 7f8ea604a640 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 0x555d567cd860 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c003a30 tx=0x7f8e9c099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b5529bbdb2257c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.303+0000 7f8e8bfff640 1 -- 172.21.15.64:0/1713794674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0b4040 con 0x555d567c3f40 2024-05-09T09:14:10.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.303+0000 7f8e8bfff640 1 -- 172.21.15.64:0/1713794674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e9c0cc400 con 0x555d567c3f40 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 msgr2=0x555d567cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 0x555d567cd860 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c003a30 tx=0x7f8e9c099730 comp rx=0 tx=0).stop 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 shutdown_connections 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c9aa0 0x555d567c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1713794674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c3f40 0x555d567cd860 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.305+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 >> 172.21.15.64:0/1713794674 conn(0x555d567bf5b0 msgr2=0x555d567ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.307+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 shutdown_connections 2024-05-09T09:14:10.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.308+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1713794674 wait complete. 2024-05-09T09:14:10.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.308+0000 7f8eaafb7740 1 -- start start 2024-05-09T09:14:10.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.309+0000 7f8eaafb7740 1 Processor -- start 2024-05-09T09:14:10.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.309+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8eaafb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8400e290 con 0x555d567c9aa0 2024-05-09T09:14:10.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8400f700 con 0x555d567c3f40 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8ea5849640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8ea604a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8eaafb7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8400e960 con 0x555d567c4930 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8ea5849640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57076/0 (socket says 172.21.15.64:57076) 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8ea5849640 1 -- 172.21.15.64:0/1196333015 learned_addr learned my addr 172.21.15.64:0/1196333015 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8eaafb7740 10 client.0 _ll_register_callbacks cb 0x7f8e9c09b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.310+0000 7f8ea684b640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea5849640 1 -- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 msgr2=0x555d567d2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea5849640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567d2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea5849640 1 -- 172.21.15.64:0/1196333015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 msgr2=0x555d567d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:10.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea5849640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea5849640 1 -- 172.21.15.64:0/1196333015 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d567d4970 con 0x555d567c4930 2024-05-09T09:14:10.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea684b640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:10.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8ea5849640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d3370 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e9002a930 tx=0x7f8e90032740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10dda870cd44d69e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.311+0000 7f8e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.312+0000 7f8e89ffb640 1 -- 172.21.15.64:0/1196333015 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90035040 con 0x555d567c4930 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.312+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c0cc400 con 0x555d567c4930 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.312+0000 7f8e89ffb640 1 -- 172.21.15.64:0/1196333015 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e90045bc0 con 0x555d567c4930 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.312+0000 7f8e89ffb640 1 -- 172.21.15.64:0/1196333015 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8e90051050 con 0x555d567c4930 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.312+0000 7f8eaafb7740 10 client.26518 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.312+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d567d5cc0 con 0x555d567c4930 2024-05-09T09:14:10.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.313+0000 7f8e89ffb640 1 -- 172.21.15.64:0/1196333015 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e9c0cc400 con 0x555d567c4930 2024-05-09T09:14:10.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8e89ffb640 1 -- 172.21.15.64:0/1196333015 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f8e90004040 con 0x555d567c4930 2024-05-09T09:14:10.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8eaafb7740 10 client.26518 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:10.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8eaafb7740 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:10.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8eaafb7740 20 client.26518 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:10.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d56841170 0x555d56843590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:10.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8eaafb7740 4 client.26518 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:14:10.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.314+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d567306c0 con 0x555d56841170 2024-05-09T09:14:10.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.315+0000 7f8ea604a640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d56841170 0x555d56843590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:10.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.315+0000 7f8ea604a640 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d56841170 0x555d56843590 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4643fc3919ed2f3 server_cookie=e41acd81a6ec680c in_seq=0 out_seq=0 2024-05-09T09:14:10.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.315+0000 7f8e89ffb640 10 client.26518 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:10.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.315+0000 7f8e89ffb640 1 -- 172.21.15.64:0/1196333015 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d567306c0 con 0x555d56841170 2024-05-09T09:14:10.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.316+0000 7f8e89ffb640 10 client.26518 handle_command_reply: tid=0 2024-05-09T09:14:10.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.316+0000 7f8eaafb7740 1 client.26518 shutdown 2024-05-09T09:14:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.317+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 msgr2=0x555d567d3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.317+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d3370 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e9002a930 tx=0x7f8e90032740 comp rx=0 tx=0).stop 2024-05-09T09:14:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 shutdown_connections 2024-05-09T09:14:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555d56841170 0x555d56843590 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d567c9aa0 0x555d567d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d567c4930 0x555d567d3370 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 --2- 172.21.15.64:0/1196333015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555d567c3f40 0x555d567d2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 >> 172.21.15.64:0/1196333015 conn(0x555d567bf5b0 msgr2=0x555d567cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8e89ffb640 0 client.26518 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 shutdown_connections 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.318+0000 7f8eaafb7740 1 -- 172.21.15.64:0/1196333015 wait complete. 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.319+0000 7f8eaafb7740 20 client.26518 trim_cache size 0 max 16384 2024-05-09T09:14:10.318 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.321+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e8c0a2ce0 msgr2=0x7f8e8c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.321+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e8c0a2ce0 0x7f8e8c0a51a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800e250 tx=0x7f8e9800b040 comp rx=0 tx=0).stop 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.321+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 msgr2=0x7f8ea0102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.321+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 0x7f8ea0102840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c00dde0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.322+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 shutdown_connections 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.322+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e8c0a2ce0 0x7f8e8c0a51a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.322+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea01074d0 0x7f8ea0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.322+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ea00fff40 0x7f8ea0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.322+0000 7f8ea82ae640 1 --2- 172.21.15.64:0/3787830431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea00ff5e0 0x7f8ea0102840 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.322+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 >> 172.21.15.64:0/3787830431 conn(0x7f8ea00fb030 msgr2=0x7f8ea0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.323+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 shutdown_connections 2024-05-09T09:14:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:10.324+0000 7f8ea82ae640 1 -- 172.21.15.64:0/3787830431 wait complete. 2024-05-09T09:14:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:10 smithi179 ceph-mon[34086]: pgmap v521: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 1.11k op/s 2024-05-09T09:14:11.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:10 smithi139 ceph-mon[33599]: pgmap v521: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 1.11k op/s 2024-05-09T09:14:11.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:10 smithi064 ceph-mon[30149]: pgmap v521: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 1.11k op/s 2024-05-09T09:14:11.346 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:14:11.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:14:11.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:14:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 230.0 s, 915.8 tps, lat 108.960 ms stddev 38.296 2024-05-09T09:14:11.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 -- 172.21.15.64:0/2667696892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0096890 msgr2=0x7f72a0098c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 --2- 172.21.15.64:0/2667696892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0096890 0x7f72a0098c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7298009900 tx=0x7f729802a340 comp rx=0 tx=0).stop 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 -- 172.21.15.64:0/2667696892 shutdown_connections 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 --2- 172.21.15.64:0/2667696892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a009baf0 0x7f72a009dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 --2- 172.21.15.64:0/2667696892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a00991c0 0x7f72a009b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 --2- 172.21.15.64:0/2667696892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0096890 0x7f72a0098c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.542+0000 7f72a7fff640 1 -- 172.21.15.64:0/2667696892 >> 172.21.15.64:0/2667696892 conn(0x7f72a00901c0 msgr2=0x7f72a0092620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.543+0000 7f72a7fff640 1 -- 172.21.15.64:0/2667696892 shutdown_connections 2024-05-09T09:14:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.543+0000 7f72a7fff640 1 -- 172.21.15.64:0/2667696892 wait complete. 2024-05-09T09:14:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.543+0000 7f72a7fff640 1 Processor -- start 2024-05-09T09:14:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 -- start start 2024-05-09T09:14:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 0x7f72a012c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a00991c0 0x7f72a012eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a009baf0 0x7f72a0131120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0094c90 con 0x7f72a009baf0 2024-05-09T09:14:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0094b10 con 0x7f72a00991c0 2024-05-09T09:14:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a7fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0094e10 con 0x7f72a0096890 2024-05-09T09:14:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 0x7f72a012c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 0x7f72a012c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57092/0 (socket says 172.21.15.64:57092) 2024-05-09T09:14:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.544+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3033023006 learned_addr learned my addr 172.21.15.64:0/3033023006 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a77fe640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a009baf0 0x7f72a0131120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a67fc640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a00991c0 0x7f72a012eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3033023006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a00991c0 msgr2=0x7f72a012eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a00991c0 0x7f72a012eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a009baf0 msgr2=0x7f72a0131120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a009baf0 0x7f72a0131120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.545+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3033023006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a01319e0 con 0x7f72a0096890 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.546+0000 7f72a77fe640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a009baf0 0x7f72a0131120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.546+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 0x7f72a012c620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7298005ab0 tx=0x7f729802ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.546+0000 7f72a67fc640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a00991c0 0x7f72a012eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:11.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.549+0000 7f72aca10640 1 -- 172.21.15.64:0/3033023006 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298040020 con 0x7f72a0096890 2024-05-09T09:14:11.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.549+0000 7f72aca10640 1 -- 172.21.15.64:0/3033023006 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7298003aa0 con 0x7f72a0096890 2024-05-09T09:14:11.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.549+0000 7f72aca10640 1 -- 172.21.15.64:0/3033023006 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729803cad0 con 0x7f72a0096890 2024-05-09T09:14:11.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.549+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a0131c70 con 0x7f72a0096890 2024-05-09T09:14:11.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.552+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a0132190 con 0x7f72a0096890 2024-05-09T09:14:11.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.557+0000 7f72aca10640 1 -- 172.21.15.64:0/3033023006 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7298043020 con 0x7f72a0096890 2024-05-09T09:14:11.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.558+0000 7f72b07fc740 1 Processor -- start 2024-05-09T09:14:11.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.558+0000 7f72b07fc740 1 -- start start 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.558+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c3daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.558+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c3e0c0 0x5577f0c42bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.558+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.558+0000 7f72b07fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5577f0ba1990 con 0x5577f0c3d6d0 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.559+0000 7f72b07fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5577f0b495b0 con 0x5577f0c43230 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.559+0000 7f72b07fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5577f0b9d950 con 0x5577f0c3e0c0 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.559+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c3daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.559+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c3e0c0 0x5577f0c42bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.559+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c3daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51706/0 (socket says 172.21.15.64:51706) 2024-05-09T09:14:11.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.559+0000 7f72a77fe640 1 -- 172.21.15.64:0/3791780816 learned_addr learned my addr 172.21.15.64:0/3791780816 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.560+0000 7f72a67fc640 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.560+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3791780816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 msgr2=0x5577f0c456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.560+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.560+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3791780816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 msgr2=0x5577f0c3daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.560+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c3daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.560+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3791780816 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f097d7e0 con 0x5577f0c3e0c0 2024-05-09T09:14:11.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.561+0000 7f72a77fe640 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c3daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:11.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.561+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c3e0c0 0x5577f0c42bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f72980ece30 tx=0x7f72980f5b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2331b73c9d24dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.561+0000 7f72a67fc640 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:11.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.561+0000 7f72aca10640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72900a2d60 0x7f72900a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.561+0000 7f728dffb640 1 -- 172.21.15.64:0/3791780816 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980efd40 con 0x5577f0c3e0c0 2024-05-09T09:14:11.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.562+0000 7f728dffb640 1 -- 172.21.15.64:0/3791780816 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72980ff020 con 0x5577f0c3e0c0 2024-05-09T09:14:11.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.562+0000 7f728dffb640 1 -- 172.21.15.64:0/3791780816 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980fa880 con 0x5577f0c3e0c0 2024-05-09T09:14:11.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.562+0000 7f72a77fe640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72900a2d60 0x7f72900a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.562+0000 7f72aca10640 1 -- 172.21.15.64:0/3033023006 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72980ef780 con 0x7f72a0096890 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 -- 172.21.15.64:0/3791780816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c3e0c0 msgr2=0x5577f0c42bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c3e0c0 0x5577f0c42bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f72980ece30 tx=0x7f72980f5b20 comp rx=0 tx=0).stop 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72a77fe640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72900a2d60 0x7f72900a5220 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f72a8064d50 tx=0x7f72a8077040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 -- 172.21.15.64:0/3791780816 shutdown_connections 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c3e0c0 0x5577f0c42bc0 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f72980ece30 tx=0x7f72980f5b20 comp rx=0 tx=0).stop 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3791780816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c3daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.564+0000 7f72b07fc740 1 -- 172.21.15.64:0/3791780816 >> 172.21.15.64:0/3791780816 conn(0x5577f0c050a0 msgr2=0x5577f0c39150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.566+0000 7f72b07fc740 1 -- 172.21.15.64:0/3791780816 shutdown_connections 2024-05-09T09:14:11.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.566+0000 7f72b07fc740 1 -- 172.21.15.64:0/3791780816 wait complete. 2024-05-09T09:14:11.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.566+0000 7f72b07fc740 1 -- start start 2024-05-09T09:14:11.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 Processor -- start 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c47af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c48030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c4f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5577f0c45fa0 con 0x5577f0c3d6d0 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5577f0b5cb40 con 0x5577f0c43230 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5577f0c46120 con 0x5577f0c4d060 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72b07fc740 10 client.0 _ll_register_callbacks cb 0x5577f0b510f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c4f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c4f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57110/0 (socket says 172.21.15.64:57110) 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.567+0000 7f72a77fe640 1 -- 172.21.15.64:0/1884177990 learned_addr learned my addr 172.21.15.64:0/1884177990 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a67fc640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c48030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c47af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a77fe640 1 -- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 msgr2=0x5577f0c48030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a77fe640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c48030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a77fe640 1 -- 172.21.15.64:0/1884177990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 msgr2=0x5577f0c47af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a77fe640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c47af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a77fe640 1 -- 172.21.15.64:0/1884177990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f09ddf70 con 0x5577f0c4d060 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f72a77fe640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c4f450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f72a804f8c0 tx=0x7f72a8067600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86b30a368e3283e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.568+0000 7f7286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f7286ffd640 1 -- 172.21.15.64:0/1884177990 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8089050 con 0x5577f0c4d060 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f7286ffd640 1 -- 172.21.15.64:0/1884177990 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72a808d3a0 con 0x5577f0c4d060 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72980ff020 con 0x5577f0c4d060 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f7286ffd640 1 -- 172.21.15.64:0/1884177990 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f72a8083e20 con 0x5577f0c4d060 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f72b07fc740 10 client.26530 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5577f0875170 con 0x5577f0c4d060 2024-05-09T09:14:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.569+0000 7f7286ffd640 1 -- 172.21.15.64:0/1884177990 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72980ff020 con 0x5577f0c4d060 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f7286ffd640 1 -- 172.21.15.64:0/1884177990 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f72a8067ce0 con 0x5577f0c4d060 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72b07fc740 10 client.26530 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72b07fc740 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72b07fc740 20 client.26530 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72b07fc740 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0c53b20 0x5577f0c55f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72b07fc740 4 client.26530 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577f0baf0c0 con 0x5577f0c53b20 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.570+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0c53b20 0x5577f0c55f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.571+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0c53b20 0x5577f0c55f60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9df0278b8cf928 server_cookie=d4ab57e7a7864a0d in_seq=0 out_seq=0 2024-05-09T09:14:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.571+0000 7f7286ffd640 10 client.26530 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.572+0000 7f7286ffd640 1 -- 172.21.15.64:0/1884177990 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577f0baf0c0 con 0x5577f0c53b20 2024-05-09T09:14:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.572+0000 7f7286ffd640 10 client.26530 handle_command_reply: tid=0 2024-05-09T09:14:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.572+0000 7f72b07fc740 1 client.26530 shutdown 2024-05-09T09:14:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.572+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 msgr2=0x5577f0c4f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.572+0000 7f72b07fc740 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c4f450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f72a804f8c0 tx=0x7f72a8067600 comp rx=0 tx=0).stop 2024-05-09T09:14:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 shutdown_connections 2024-05-09T09:14:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f72b07fc740 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0c53b20 0x5577f0c55f60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f72b07fc740 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c4f450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f72b07fc740 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c43230 0x5577f0c48030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f72b07fc740 1 --2- 172.21.15.64:0/1884177990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c3d6d0 0x5577f0c47af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f7286ffd640 0 client.26530 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.573+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 >> 172.21.15.64:0/1884177990 conn(0x5577f0c050a0 msgr2=0x5577f0c38290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.574+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 shutdown_connections 2024-05-09T09:14:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.575+0000 7f72b07fc740 1 -- 172.21.15.64:0/1884177990 wait complete. 2024-05-09T09:14:11.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.575+0000 7f72b07fc740 20 client.26530 trim_cache size 0 max 16384 2024-05-09T09:14:11.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.611+0000 7f72b07fc740 1 Processor -- start 2024-05-09T09:14:11.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.611+0000 7f72b07fc740 1 -- start start 2024-05-09T09:14:11.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c39320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c43230 0x5577f0c5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c39320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c39320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53616/0 (socket says 172.21.15.64:53616) 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c43230 0x5577f0c5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c43230 0x5577f0c5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51712/0 (socket says 172.21.15.64:51712) 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3592972426 learned_addr learned my addr 172.21.15.64:0/3592972426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0150070 con 0x5577f0c43230 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72a014d8b0 con 0x5577f0c3d6d0 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72a014b700 con 0x5577f0c4d060 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.612+0000 7f72a77fe640 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3592972426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 msgr2=0x5577f0c5bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c5bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3592972426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c43230 msgr2=0x5577f0c5b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c43230 0x5577f0c5b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f72a6ffd640 1 -- 172.21.15.64:0/3592972426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f0c389c0 con 0x5577f0c3d6d0 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c39320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7298032f90 tx=0x7f72980f5f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a408b7b9e2c445 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f728d7fa640 1 -- 172.21.15.64:0/3592972426 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980fe040 con 0x5577f0c3d6d0 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f728d7fa640 1 -- 172.21.15.64:0/3592972426 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f729803c460 con 0x5577f0c3d6d0 2024-05-09T09:14:11.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.613+0000 7f728d7fa640 1 -- 172.21.15.64:0/3592972426 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980bb7b0 con 0x5577f0c3d6d0 2024-05-09T09:14:11.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.614+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 msgr2=0x5577f0c39320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.614+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c39320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7298032f90 tx=0x7f72980f5f50 comp rx=0 tx=0).stop 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.614+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 shutdown_connections 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.614+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c4d060 0x5577f0c5bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.615+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c43230 0x5577f0c5b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.615+0000 7f72b07fc740 1 --2- 172.21.15.64:0/3592972426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c39320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.615+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 >> 172.21.15.64:0/3592972426 conn(0x5577f0c050a0 msgr2=0x5577f0c38c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.615+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 shutdown_connections 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.615+0000 7f72b07fc740 1 -- 172.21.15.64:0/3592972426 wait complete. 2024-05-09T09:14:11.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.616+0000 7f72b07fc740 1 -- start start 2024-05-09T09:14:11.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.616+0000 7f72b07fc740 1 Processor -- start 2024-05-09T09:14:11.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 0x5577f0c65650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c4d060 0x5577f0c663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72a014bd70 con 0x5577f0c4d060 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72a014d1e0 con 0x5577f0c3d6d0 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72a014c440 con 0x5577f0c43230 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72b07fc740 10 client.0 _ll_register_callbacks cb 0x7f72a8083b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c4d060 0x5577f0c663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 0x5577f0c65650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 0x5577f0c65650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57142/0 (socket says 172.21.15.64:57142) 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 -- 172.21.15.64:0/2735768839 learned_addr learned my addr 172.21.15.64:0/2735768839 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 -- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 msgr2=0x5577f0c65090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c65090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 -- 172.21.15.64:0/2735768839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c4d060 msgr2=0x5577f0c663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c4d060 0x5577f0c663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.617+0000 7f72a67fc640 1 -- 172.21.15.64:0/2735768839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f0c669a0 con 0x5577f0c43230 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.618+0000 7f72a67fc640 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 0x5577f0c65650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f72a00051f0 tx=0x7f729c012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4946d922739b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.618+0000 7f7286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.618+0000 7f7286ffd640 1 -- 172.21.15.64:0/2735768839 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c022060 con 0x5577f0c43230 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.618+0000 7f7286ffd640 1 -- 172.21.15.64:0/2735768839 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f729c004460 con 0x5577f0c43230 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.618+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577f0c34420 con 0x5577f0c43230 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.619+0000 7f7286ffd640 1 -- 172.21.15.64:0/2735768839 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f729c012e40 con 0x5577f0c43230 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.619+0000 7f72b07fc740 10 client.26536 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:11.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.619+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5577f0cba070 con 0x5577f0c43230 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.619+0000 7f7286ffd640 1 -- 172.21.15.64:0/2735768839 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5577f0c34420 con 0x5577f0c43230 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.619+0000 7f7286ffd640 1 -- 172.21.15.64:0/2735768839 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f729c01c030 con 0x5577f0c43230 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.619+0000 7f72b07fc740 10 client.26536 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.620+0000 7f72b07fc740 10 client.26536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.620+0000 7f72b07fc740 20 client.26536 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.620+0000 7f72b07fc740 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0cbaa90 0x5577f0cbceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.620+0000 7f72b07fc740 4 client.26536 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.620+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5577f0ba9410 con 0x5577f0cbaa90 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.620+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0cbaa90 0x5577f0cbceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.621+0000 7f72a6ffd640 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0cbaa90 0x5577f0cbceb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1b50f26e52409a server_cookie=cf119b8addb27c12 in_seq=0 out_seq=0 2024-05-09T09:14:11.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.621+0000 7f7286ffd640 10 client.26536 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.622+0000 7f7286ffd640 1 -- 172.21.15.64:0/2735768839 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577f0ba9410 con 0x5577f0cbaa90 2024-05-09T09:14:11.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.622+0000 7f7286ffd640 10 client.26536 handle_command_reply: tid=0 2024-05-09T09:14:11.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.622+0000 7f72b07fc740 1 client.26536 shutdown 2024-05-09T09:14:11.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.624+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 msgr2=0x5577f0c65650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.624+0000 7f72b07fc740 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 0x5577f0c65650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f72a00051f0 tx=0x7f729c012bf0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 shutdown_connections 2024-05-09T09:14:11.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5577f0cbaa90 0x5577f0cbceb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577f0c4d060 0x5577f0c663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577f0c43230 0x5577f0c65650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 --2- 172.21.15.64:0/2735768839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5577f0c3d6d0 0x5577f0c65090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 >> 172.21.15.64:0/2735768839 conn(0x5577f0c050a0 msgr2=0x5577f0c3c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f7286ffd640 0 client.26536 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 shutdown_connections 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 1 -- 172.21.15.64:0/2735768839 wait complete. 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.625+0000 7f72b07fc740 20 client.26536 trim_cache size 0 max 16384 2024-05-09T09:14:11.625 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "5a6c5392-5176-4dda-a7cc-137219e8971a", 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72900a2d60 msgr2=0x7f72900a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72900a2d60 0x7f72900a5220 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f72a8064d50 tx=0x7f72a8077040 comp rx=0 tx=0).stop 2024-05-09T09:14:11.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 msgr2=0x7f72a012c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 0x7f72a012c620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7298005ab0 tx=0x7f729802ad50 comp rx=0 tx=0).stop 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 shutdown_connections 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72900a2d60 0x7f72900a5220 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a009baf0 0x7f72a0131120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a00991c0 0x7f72a012eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 --2- 172.21.15.64:0/3033023006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a0096890 0x7f72a012c620 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.627+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 >> 172.21.15.64:0/3033023006 conn(0x7f72a00901c0 msgr2=0x7f72a009ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.629+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 shutdown_connections 2024-05-09T09:14:11.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.630+0000 7f72a7fff640 1 -- 172.21.15.64:0/3033023006 wait complete. 2024-05-09T09:14:11.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:14:11.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:14:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.880+0000 7fb1cd910640 1 -- 172.21.15.64:0/35392759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c810db70 msgr2=0x7fb1c810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.880+0000 7fb1cd910640 1 --2- 172.21.15.64:0/35392759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c810db70 0x7fb1c810e020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009ca0 tx=0x7fb1c002a8d0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.881+0000 7fb1cd910640 1 -- 172.21.15.64:0/35392759 shutdown_connections 2024-05-09T09:14:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.881+0000 7fb1cd910640 1 --2- 172.21.15.64:0/35392759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c810db70 0x7fb1c810e020 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.881+0000 7fb1cd910640 1 --2- 172.21.15.64:0/35392759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.881+0000 7fb1cd910640 1 --2- 172.21.15.64:0/35392759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c8108830 0x7fb1c810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.881+0000 7fb1cd910640 1 -- 172.21.15.64:0/35392759 >> 172.21.15.64:0/35392759 conn(0x7fb1c806e6c0 msgr2=0x7fb1c8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.882+0000 7fb1cd910640 1 -- 172.21.15.64:0/35392759 shutdown_connections 2024-05-09T09:14:11.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.884+0000 7fb1cd910640 1 -- 172.21.15.64:0/35392759 wait complete. 2024-05-09T09:14:11.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.885+0000 7fb1cd910640 1 Processor -- start 2024-05-09T09:14:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 -- start start 2024-05-09T09:14:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c8108830 0x7fb1c81a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c81a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c810db70 0x7fb1c81ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1c67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c81a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c811c270 con 0x7fb1c810db70 2024-05-09T09:14:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1c67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c81a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53646/0 (socket says 172.21.15.64:53646) 2024-05-09T09:14:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1c67fc640 1 -- 172.21.15.64:0/1194764379 learned_addr learned my addr 172.21.15.64:0/1194764379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c811c0f0 con 0x7fb1c810b1a0 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.886+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c811c3f0 con 0x7fb1c8108830 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c810db70 0x7fb1c81ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c67fc640 1 -- 172.21.15.64:0/1194764379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c8108830 msgr2=0x7fb1c81a68c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c8108830 0x7fb1c81a68c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c67fc640 1 -- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c810db70 msgr2=0x7fb1c81ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c810db70 0x7fb1c81ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c67fc640 1 -- 172.21.15.64:0/1194764379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c81ae4e0 con 0x7fb1c810b1a0 2024-05-09T09:14:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.887+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c810db70 0x7fb1c81ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.888+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c81a6e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d770 tx=0x7fb1b800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.889+0000 7fb1cc90e640 1 -- 172.21.15.64:0/1194764379 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8017070 con 0x7fb1c810b1a0 2024-05-09T09:14:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.889+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c81ae7d0 con 0x7fb1c810b1a0 2024-05-09T09:14:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.889+0000 7fb1cc90e640 1 -- 172.21.15.64:0/1194764379 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b8009c30 con 0x7fb1c810b1a0 2024-05-09T09:14:11.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.889+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81aed20 con 0x7fb1c810b1a0 2024-05-09T09:14:11.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.891+0000 7fb1cc90e640 1 -- 172.21.15.64:0/1194764379 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003c90 con 0x7fb1c810b1a0 2024-05-09T09:14:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.892+0000 7fb1cc90e640 1 -- 172.21.15.64:0/1194764379 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1b8010460 con 0x7fb1c810b1a0 2024-05-09T09:14:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.895+0000 7fb1d061e740 1 Processor -- start 2024-05-09T09:14:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.895+0000 7fb1d061e740 1 -- start start 2024-05-09T09:14:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1d061e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed7f82ad00 con 0x55ed7f8cba90 2024-05-09T09:14:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1d061e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ed7f7d24c0 con 0x55ed7f8c5f30 2024-05-09T09:14:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1d061e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ed7f826cc0 con 0x55ed7f8c6920 2024-05-09T09:14:11.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53654/0 (socket says 172.21.15.64:53654) 2024-05-09T09:14:11.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1c77fe640 1 -- 172.21.15.64:0/760780130 learned_addr learned my addr 172.21.15.64:0/760780130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.896+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c77fe640 1 -- 172.21.15.64:0/760780130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 msgr2=0x55ed7f8cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c77fe640 1 -- 172.21.15.64:0/760780130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 msgr2=0x55ed7f8cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c77fe640 1 -- 172.21.15.64:0/760780130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7f6087d0 con 0x55ed7f8c5f30 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1cc90e640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1b00a2d20 0x7fb1b00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.897+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c6350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009ca0 tx=0x7fb1c00098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d5bf69b8bc98a0d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.898+0000 7fb1cc90e640 1 -- 172.21.15.64:0/1194764379 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb1b80cfc60 con 0x7fb1c810b1a0 2024-05-09T09:14:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.898+0000 7fb1adffb640 1 -- 172.21.15.64:0/760780130 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0049070 con 0x55ed7f8c5f30 2024-05-09T09:14:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.898+0000 7fb1adffb640 1 -- 172.21.15.64:0/760780130 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c0036040 con 0x55ed7f8c5f30 2024-05-09T09:14:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.898+0000 7fb1adffb640 1 -- 172.21.15.64:0/760780130 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0044670 con 0x55ed7f8c5f30 2024-05-09T09:14:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.898+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1b00a2d20 0x7fb1b00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.899+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1b00a2d20 0x7fb1b00a51e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55ed7f8c6d20 tx=0x7fb1bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.900+0000 7fb1d061e740 1 -- 172.21.15.64:0/760780130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 msgr2=0x55ed7f8c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.900+0000 7fb1d061e740 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c6350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009ca0 tx=0x7fb1c00098b0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 -- 172.21.15.64:0/760780130 shutdown_connections 2024-05-09T09:14:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 --2- 172.21.15.64:0/760780130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c6350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 -- 172.21.15.64:0/760780130 >> 172.21.15.64:0/760780130 conn(0x55ed7f8c15a0 msgr2=0x55ed7f8c19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 -- 172.21.15.64:0/760780130 shutdown_connections 2024-05-09T09:14:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.901+0000 7fb1d061e740 1 -- 172.21.15.64:0/760780130 wait complete. 2024-05-09T09:14:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.902+0000 7fb1d061e740 1 -- start start 2024-05-09T09:14:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.902+0000 7fb1d061e740 1 Processor -- start 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed7f8ce9e0 con 0x55ed7f8cba90 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ed7f7e5b90 con 0x55ed7f8c5f30 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ed7f8ceb60 con 0x55ed7f8c6920 2024-05-09T09:14:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1d061e740 10 client.0 _ll_register_callbacks cb 0x55ed7f821da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1c67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1c67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57160/0 (socket says 172.21.15.64:57160) 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.903+0000 7fb1c67fc640 1 -- 172.21.15.64:0/3730219083 learned_addr learned my addr 172.21.15.64:0/3730219083 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/3730219083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 msgr2=0x55ed7f8d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/3730219083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 msgr2=0x55ed7f8d7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/3730219083 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7f53e3f0 con 0x55ed7f8c5f30 2024-05-09T09:14:11.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d0430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55ed7f8c7390 tx=0x7fb1bc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854f25f8095135f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.904+0000 7fb1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:11.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.905+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/3730219083 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc023060 con 0x55ed7f8c5f30 2024-05-09T09:14:11.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.905+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/3730219083 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb1bc01e400 con 0x55ed7f8c5f30 2024-05-09T09:14:11.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.905+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c0036040 con 0x55ed7f8c5f30 2024-05-09T09:14:11.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.905+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/3730219083 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1bc02a050 con 0x55ed7f8c5f30 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.906+0000 7fb1d061e740 10 client.26375 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.906+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed7f4fe0f0 con 0x55ed7f8c5f30 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.906+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/3730219083 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c0036040 con 0x55ed7f8c5f30 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/3730219083 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fb1bc01d3a0 con 0x55ed7f8c5f30 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1d061e740 10 client.26375 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1d061e740 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1d061e740 20 client.26375 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1d061e740 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f8dc240 0x55ed7f8de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1d061e740 4 client.26375 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7f838430 con 0x55ed7f8dc240 2024-05-09T09:14:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.907+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f8dc240 0x55ed7f8de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.908+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f8dc240 0x55ed7f8de6a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b332feac6512a85e server_cookie=6ad7f484e48e7349 in_seq=0 out_seq=0 2024-05-09T09:14:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.908+0000 7fb1a6ffd640 10 client.26375 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/3730219083 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed7f838430 con 0x55ed7f8dc240 2024-05-09T09:14:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1a6ffd640 10 client.26375 handle_command_reply: tid=0 2024-05-09T09:14:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 client.26375 shutdown 2024-05-09T09:14:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 msgr2=0x55ed7f8d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d0430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55ed7f8c7390 tx=0x7fb1bc004770 comp rx=0 tx=0).stop 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 shutdown_connections 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f8dc240 0x55ed7f8de6a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 --2- 172.21.15.64:0/3730219083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d0430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 >> 172.21.15.64:0/3730219083 conn(0x55ed7f8c15a0 msgr2=0x55ed7f8c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.909+0000 7fb1a6ffd640 0 client.26375 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.910+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 shutdown_connections 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.910+0000 7fb1d061e740 1 -- 172.21.15.64:0/3730219083 wait complete. 2024-05-09T09:14:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.910+0000 7fb1d061e740 20 client.26375 trim_cache size 0 max 16384 2024-05-09T09:14:11.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 Processor -- start 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 -- start start 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8cba90 0x55ed7f8c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b40125f0 con 0x55ed7f8c6920 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b400fe30 con 0x55ed7f8cba90 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.954+0000 7fb1d061e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b400dc80 con 0x55ed7f8c5f30 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57162/0 (socket says 172.21.15.64:57162) 2024-05-09T09:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/1721123171 learned_addr learned my addr 172.21.15.64:0/1721123171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8cba90 0x55ed7f8c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/1721123171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8cba90 msgr2=0x55ed7f8c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8cba90 0x55ed7f8c1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/1721123171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8c6920 msgr2=0x55ed7f8d78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/1721123171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7f8cf530 con 0x55ed7f8c5f30 2024-05-09T09:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc008010 tx=0x7fb1bc00c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e56be8fa8972666 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.955+0000 7fb1ad7fa640 1 -- 172.21.15.64:0/1721123171 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc032020 con 0x55ed7f8c5f30 2024-05-09T09:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.956+0000 7fb1ad7fa640 1 -- 172.21.15.64:0/1721123171 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1bc01d990 con 0x55ed7f8c5f30 2024-05-09T09:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.956+0000 7fb1ad7fa640 1 -- 172.21.15.64:0/1721123171 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc01e400 con 0x55ed7f8c5f30 2024-05-09T09:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 -- 172.21.15.64:0/1721123171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 msgr2=0x55ed7f8c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc008010 tx=0x7fb1bc00c9b0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 -- 172.21.15.64:0/1721123171 shutdown_connections 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8cba90 0x55ed7f8c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1721123171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8c4e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 -- 172.21.15.64:0/1721123171 >> 172.21.15.64:0/1721123171 conn(0x55ed7f8c15a0 msgr2=0x55ed7f8d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 -- 172.21.15.64:0/1721123171 shutdown_connections 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.957+0000 7fb1d061e740 1 -- 172.21.15.64:0/1721123171 wait complete. 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.958+0000 7fb1d061e740 1 -- start start 2024-05-09T09:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.958+0000 7fb1d061e740 1 Processor -- start 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.958+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1d061e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53686/0 (socket says 172.21.15.64:53686) 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1c6ffd640 1 -- 172.21.15.64:0/1250517864 learned_addr learned my addr 172.21.15.64:0/1250517864 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b400e2f0 con 0x55ed7f8cba90 2024-05-09T09:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b400f760 con 0x55ed7f8c5f30 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b400e9c0 con 0x55ed7f8c6920 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.959+0000 7fb1d061e740 10 client.0 _ll_register_callbacks cb 0x7fb1bc00a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c77fe640 1 -- 172.21.15.64:0/1250517864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 msgr2=0x55ed7f8d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c77fe640 1 -- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 msgr2=0x55ed7f8d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c77fe640 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7f8d6ad0 con 0x55ed7f8cba90 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c67fc640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.960+0000 7fb1c77fe640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d6490 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0031980 tx=0x7fb1c0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e7ad0618847a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.961+0000 7fb1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.961+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/1250517864 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0049070 con 0x55ed7f8cba90 2024-05-09T09:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.961+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/1250517864 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb1c0030a90 con 0x55ed7f8cba90 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.961+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1bc01d990 con 0x55ed7f8cba90 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.961+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/1250517864 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1c0030c90 con 0x55ed7f8cba90 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.961+0000 7fb1d061e740 10 client.16884 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed7f9427e0 con 0x55ed7f8cba90 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/1250517864 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1bc01d990 con 0x55ed7f8cba90 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/1250517864 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fb1c0044410 con 0x55ed7f8cba90 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 10 client.16884 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 20 client.16884 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f942ce0 0x55ed7f9450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 4 client.16884 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.962+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed7f832780 con 0x55ed7f942ce0 2024-05-09T09:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.963+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f942ce0 0x55ed7f9450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.963+0000 7fb1c6ffd640 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f942ce0 0x55ed7f9450d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e13877caffb883b server_cookie=d6f1194201a1800b in_seq=0 out_seq=0 2024-05-09T09:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.963+0000 7fb1a6ffd640 10 client.16884 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.964+0000 7fb1a6ffd640 1 -- 172.21.15.64:0/1250517864 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ed7f832780 con 0x55ed7f942ce0 2024-05-09T09:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.964+0000 7fb1a6ffd640 10 client.16884 handle_command_reply: tid=0 2024-05-09T09:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.964+0000 7fb1d061e740 1 client.16884 shutdown 2024-05-09T09:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 msgr2=0x55ed7f8d6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d6490 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0031980 tx=0x7fb1c0009a20 comp rx=0 tx=0).stop 2024-05-09T09:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 shutdown_connections 2024-05-09T09:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ed7f942ce0 0x55ed7f9450d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed7f8cba90 0x55ed7f8d6490 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed7f8c6920 0x55ed7f8d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 --2- 172.21.15.64:0/1250517864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed7f8c5f30 0x55ed7f8d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 >> 172.21.15.64:0/1250517864 conn(0x55ed7f8c15a0 msgr2=0x55ed7f8bff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.965+0000 7fb1a6ffd640 0 client.16884 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.966+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 shutdown_connections 2024-05-09T09:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.966+0000 7fb1d061e740 1 -- 172.21.15.64:0/1250517864 wait complete. 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.966+0000 7fb1d061e740 20 client.16884 trim_cache size 0 max 16384 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "5a6c5392-5176-4dda-a7cc-137219e8971a": { 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "5a6c5392-5176-4dda-a7cc-137219e8971a", 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:14:11.966 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.967+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1b00a2d20 msgr2=0x7fb1b00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.967+0000 7fb1cd910640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1b00a2d20 0x7fb1b00a51e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55ed7f8c6d20 tx=0x7fb1bc008040 comp rx=0 tx=0).stop 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.967+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 msgr2=0x7fb1c81a6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.967+0000 7fb1cd910640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c81a6e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d770 tx=0x7fb1b800dc30 comp rx=0 tx=0).stop 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.967+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 shutdown_connections 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.968+0000 7fb1cd910640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb1b00a2d20 0x7fb1b00a51e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.968+0000 7fb1cd910640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1c810db70 0x7fb1c81ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.968+0000 7fb1cd910640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c810b1a0 0x7fb1c81a6e00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.968+0000 7fb1cd910640 1 --2- 172.21.15.64:0/1194764379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c8108830 0x7fb1c81a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:11.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.968+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 >> 172.21.15.64:0/1194764379 conn(0x7fb1c806e6c0 msgr2=0x7fb1c810a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.970+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 shutdown_connections 2024-05-09T09:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:11.970+0000 7fb1cd910640 1 -- 172.21.15.64:0/1194764379 wait complete. 2024-05-09T09:14:11.990 INFO:tasks.cephfs.filesystem:scrub status for tag:5a6c5392-5176-4dda-a7cc-137219e8971a - {'path': '/', 'tag': '5a6c5392-5176-4dda-a7cc-137219e8971a', 'options': 'recursive,force'} 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: pgmap v522: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 MiB/s wr, 880 op/s 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:12 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: pgmap v522: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 MiB/s wr, 880 op/s 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:12 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: pgmap v522: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 MiB/s wr, 880 op/s 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:12 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:14 smithi179 ceph-mon[34086]: pgmap v523: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.3 MiB/s wr, 978 op/s 2024-05-09T09:14:15.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:14 smithi139 ceph-mon[33599]: pgmap v523: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.3 MiB/s wr, 978 op/s 2024-05-09T09:14:15.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:14 smithi064 ceph-mon[30149]: pgmap v523: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.3 MiB/s wr, 978 op/s 2024-05-09T09:14:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 235.0 s, 964.4 tps, lat 104.117 ms stddev 36.012 2024-05-09T09:14:16.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:16 smithi179 ceph-mon[34086]: pgmap v524: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.4 MiB/s wr, 921 op/s 2024-05-09T09:14:17.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:16 smithi139 ceph-mon[33599]: pgmap v524: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.4 MiB/s wr, 921 op/s 2024-05-09T09:14:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:16 smithi064 ceph-mon[30149]: pgmap v524: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.4 MiB/s wr, 921 op/s 2024-05-09T09:14:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.578+0000 7f9d83fff640 1 -- 172.21.15.64:0/1584010136 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c031880 con 0x7f9d8c10b280 2024-05-09T09:14:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 -- 172.21.15.64:0/1584010136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10b280 msgr2=0x7f9d8c10d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 --2- 172.21.15.64:0/1584010136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10b280 0x7f9d8c10d6f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c02a420 comp rx=0 tx=0).stop 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 -- 172.21.15.64:0/1584010136 shutdown_connections 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 --2- 172.21.15.64:0/1584010136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c10dc30 0x7f9d8c10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 --2- 172.21.15.64:0/1584010136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10b280 0x7f9d8c10d6f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 --2- 172.21.15.64:0/1584010136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108910 0x7f9d8c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.579+0000 7f9d93521640 1 -- 172.21.15.64:0/1584010136 >> 172.21.15.64:0/1584010136 conn(0x7f9d8c06f7d0 msgr2=0x7f9d8c071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.580+0000 7f9d93521640 1 -- 172.21.15.64:0/1584010136 shutdown_connections 2024-05-09T09:14:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.580+0000 7f9d93521640 1 -- 172.21.15.64:0/1584010136 wait complete. 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 Processor -- start 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 -- start start 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c108910 0x7f9d8c1a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 0x7f9d8c1a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10dc30 0x7f9d8c1ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c11c230 con 0x7f9d8c10dc30 2024-05-09T09:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c11c0b0 con 0x7f9d8c108910 2024-05-09T09:14:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.581+0000 7f9d93521640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c11c3b0 con 0x7f9d8c10b280 2024-05-09T09:14:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 0x7f9d8c1a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 0x7f9d8c1a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44248/0 (socket says 172.21.15.64:44248) 2024-05-09T09:14:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 -- 172.21.15.64:0/3996080215 learned_addr learned my addr 172.21.15.64:0/3996080215 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d912bd640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c108910 0x7f9d8c1a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d91abe640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10dc30 0x7f9d8c1ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 -- 172.21.15.64:0/3996080215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c108910 msgr2=0x7f9d8c1a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c108910 0x7f9d8c1a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 -- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10dc30 msgr2=0x7f9d8c1ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10dc30 0x7f9d8c1ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 -- 172.21.15.64:0/3996080215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c1ae4b0 con 0x7f9d8c10b280 2024-05-09T09:14:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.582+0000 7f9d90abc640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 0x7f9d8c1a6dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005940 tx=0x7f9d7c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.583+0000 7f9d827fc640 1 -- 172.21.15.64:0/3996080215 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c03f030 con 0x7f9d8c10b280 2024-05-09T09:14:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.583+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c1ae740 con 0x7f9d8c10b280 2024-05-09T09:14:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.583+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1aecb0 con 0x7f9d8c10b280 2024-05-09T09:14:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.583+0000 7f9d827fc640 1 -- 172.21.15.64:0/3996080215 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c003cb0 con 0x7f9d8c10b280 2024-05-09T09:14:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.583+0000 7f9d827fc640 1 -- 172.21.15.64:0/3996080215 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035070 con 0x7f9d8c10b280 2024-05-09T09:14:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 Processor -- start 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 -- start start 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56412142a340 con 0x5641214c6c40 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5641213d2290 con 0x5641214c6250 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.587+0000 7f9d9622a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564121426300 con 0x5641214cbdb0 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d827fc640 1 -- 172.21.15.64:0/3996080215 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d7c042030 con 0x7f9d8c10b280 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d91abe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d91abe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:59888/0 (socket says 172.21.15.64:59888) 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d91abe640 1 -- 172.21.15.64:0/1464783 learned_addr learned my addr 172.21.15.64:0/1464783 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d90abc640 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d912bd640 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d91abe640 1 -- 172.21.15.64:0/1464783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 msgr2=0x5641214ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d91abe640 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.588+0000 7f9d91abe640 1 -- 172.21.15.64:0/1464783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 msgr2=0x5641214cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.589+0000 7f9d91abe640 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.589+0000 7f9d91abe640 1 -- 172.21.15.64:0/1464783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641213f68e0 con 0x5641214c6250 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.589+0000 7f9d912bd640 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.589+0000 7f9d91abe640 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214c6670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009f70 tx=0x7f9d84011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6b89053234817b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.590+0000 7f9d827fc640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2e20 0x7f9d780a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.590+0000 7f9d777fe640 1 -- 172.21.15.64:0/1464783 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84003970 con 0x5641214c6250 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.590+0000 7f9d777fe640 1 -- 172.21.15.64:0/1464783 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d84004030 con 0x5641214c6250 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.590+0000 7f9d777fe640 1 -- 172.21.15.64:0/1464783 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84012970 con 0x5641214c6250 2024-05-09T09:14:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.590+0000 7f9d827fc640 1 -- 172.21.15.64:0/3996080215 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d7c0efb70 con 0x7f9d8c10b280 2024-05-09T09:14:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.590+0000 7f9d912bd640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2e20 0x7f9d780a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.591+0000 7f9d9622a740 1 -- 172.21.15.64:0/1464783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 msgr2=0x5641214c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.591+0000 7f9d9622a740 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214c6670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009f70 tx=0x7f9d84011870 comp rx=0 tx=0).stop 2024-05-09T09:14:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.592+0000 7f9d912bd640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2e20 0x7f9d780a52e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5641214c7040 tx=0x7f9d88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.593+0000 7f9d9622a740 1 -- 172.21.15.64:0/1464783 shutdown_connections 2024-05-09T09:14:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.593+0000 7f9d9622a740 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.593+0000 7f9d9622a740 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.593+0000 7f9d9622a740 1 --2- 172.21.15.64:0/1464783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214c6670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.593+0000 7f9d9622a740 1 -- 172.21.15.64:0/1464783 >> 172.21.15.64:0/1464783 conn(0x5641214c18c0 msgr2=0x5641214c1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.594+0000 7f9d9622a740 1 -- 172.21.15.64:0/1464783 shutdown_connections 2024-05-09T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.594+0000 7f9d9622a740 1 -- 172.21.15.64:0/1464783 wait complete. 2024-05-09T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.595+0000 7f9d9622a740 1 -- start start 2024-05-09T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.595+0000 7f9d9622a740 1 Processor -- start 2024-05-09T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641214ced60 con 0x5641214c6c40 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d90abc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d90abc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43500/0 (socket says 172.21.15.64:43500) 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5641213e5790 con 0x5641214c6250 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d90abc640 1 -- 172.21.15.64:0/16732383 learned_addr learned my addr 172.21.15.64:0/16732383 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5641214ceee0 con 0x5641214cbdb0 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.596+0000 7f9d9622a740 10 client.0 _ll_register_callbacks cb 0x564121420a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:17.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d912bd640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d91abe640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d90abc640 1 -- 172.21.15.64:0/16732383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 msgr2=0x5641214d7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d90abc640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214d7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d90abc640 1 -- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 msgr2=0x5641214d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d90abc640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.597+0000 7f9d90abc640 1 -- 172.21.15.64:0/16732383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412113e210 con 0x5641214c6c40 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.598+0000 7f9d90abc640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214d0a60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b8440 tx=0x7f9d7c0b82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8bc86905f90ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.598+0000 7f9d74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.598+0000 7f9d74ff9640 1 -- 172.21.15.64:0/16732383 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c049040 con 0x5641214c6c40 2024-05-09T09:14:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.598+0000 7f9d74ff9640 1 -- 172.21.15.64:0/16732383 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d7c03b460 con 0x5641214c6c40 2024-05-09T09:14:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.599+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d84004030 con 0x5641214c6c40 2024-05-09T09:14:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.599+0000 7f9d74ff9640 1 -- 172.21.15.64:0/16732383 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c0bb940 con 0x5641214c6c40 2024-05-09T09:14:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.599+0000 7f9d9622a740 10 client.16893 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:17.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.599+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56412120a620 con 0x5641214c6c40 2024-05-09T09:14:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.600+0000 7f9d74ff9640 1 -- 172.21.15.64:0/16732383 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d84004030 con 0x5641214c6c40 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d74ff9640 1 -- 172.21.15.64:0/16732383 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f9d7c0b8650 con 0x5641214c6c40 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d9622a740 10 client.16893 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d9622a740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d9622a740 20 client.16893 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d9622a740 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x5641214de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d9622a740 4 client.16893 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.601+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564121437a70 con 0x5641214dc170 2024-05-09T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.602+0000 7f9d912bd640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x5641214de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.603+0000 7f9d912bd640 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x5641214de5b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6731b5fa4540d7 server_cookie=83846313c2f9f095 in_seq=0 out_seq=0 2024-05-09T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.603+0000 7f9d74ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.603+0000 7f9d74ff9640 1 -- 172.21.15.64:0/16732383 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564121437a70 con 0x5641214dc170 2024-05-09T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.603+0000 7f9d74ff9640 10 client.16893 handle_command_reply: tid=0 2024-05-09T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.604+0000 7f9d9622a740 1 client.16893 shutdown 2024-05-09T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.605+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 msgr2=0x5641214d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.605+0000 7f9d9622a740 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214d0a60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b8440 tx=0x7f9d7c0b82f0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 shutdown_connections 2024-05-09T09:14:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x5641214de5b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d74ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214cbdb0 0x5641214d7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6c40 0x5641214d0a60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 --2- 172.21.15.64:0/16732383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6250 0x5641214d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 >> 172.21.15.64:0/16732383 conn(0x5641214c18c0 msgr2=0x5641214c0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.606+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 shutdown_connections 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.607+0000 7f9d9622a740 1 -- 172.21.15.64:0/16732383 wait complete. 2024-05-09T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.607+0000 7f9d9622a740 20 client.16893 trim_cache size 0 max 16384 2024-05-09T09:14:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.645+0000 7f9d9622a740 1 Processor -- start 2024-05-09T09:14:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.645+0000 7f9d9622a740 1 -- start start 2024-05-09T09:14:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 0x5641214cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6c40 0x5641214c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214cbdb0 0x5641214c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d9622a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d700124d0 con 0x5641214c6250 2024-05-09T09:14:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d9622a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000fd10 con 0x5641214cbdb0 2024-05-09T09:14:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d9622a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000db60 con 0x5641214c6c40 2024-05-09T09:14:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d912bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 0x5641214cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d912bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 0x5641214cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43508/0 (socket says 172.21.15.64:43508) 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d912bd640 1 -- 172.21.15.64:0/3967170675 learned_addr learned my addr 172.21.15.64:0/3967170675 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d91abe640 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214cbdb0 0x5641214c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.646+0000 7f9d90abc640 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6c40 0x5641214c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.647+0000 7f9d912bd640 1 -- 172.21.15.64:0/3967170675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6c40 msgr2=0x5641214c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.647+0000 7f9d912bd640 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6c40 0x5641214c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.647+0000 7f9d912bd640 1 -- 172.21.15.64:0/3967170675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214cbdb0 msgr2=0x5641214c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.647+0000 7f9d912bd640 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214cbdb0 0x5641214c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.647+0000 7f9d912bd640 1 -- 172.21.15.64:0/3967170675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641214c5e90 con 0x5641214c6250 2024-05-09T09:14:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.647+0000 7f9d912bd640 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 0x5641214cfa20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006fd0 tx=0x7f9d88013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4406e30b3060550 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.648+0000 7f9d76ffd640 1 -- 172.21.15.64:0/3967170675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8801c040 con 0x5641214c6250 2024-05-09T09:14:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.648+0000 7f9d76ffd640 1 -- 172.21.15.64:0/3967170675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8801d040 con 0x5641214c6250 2024-05-09T09:14:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.648+0000 7f9d76ffd640 1 -- 172.21.15.64:0/3967170675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800f040 con 0x5641214c6250 2024-05-09T09:14:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 -- 172.21.15.64:0/3967170675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 msgr2=0x5641214cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 0x5641214cfa20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006fd0 tx=0x7f9d88013000 comp rx=0 tx=0).stop 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 -- 172.21.15.64:0/3967170675 shutdown_connections 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214cbdb0 0x5641214c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6c40 0x5641214c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 --2- 172.21.15.64:0/3967170675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214c6250 0x5641214cfa20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 -- 172.21.15.64:0/3967170675 >> 172.21.15.64:0/3967170675 conn(0x5641214c18c0 msgr2=0x5641214cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.649+0000 7f9d9622a740 1 -- 172.21.15.64:0/3967170675 shutdown_connections 2024-05-09T09:14:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.650+0000 7f9d9622a740 1 -- 172.21.15.64:0/3967170675 wait complete. 2024-05-09T09:14:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.651+0000 7f9d9622a740 1 -- start start 2024-05-09T09:14:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.651+0000 7f9d9622a740 1 Processor -- start 2024-05-09T09:14:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.652+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 0x5641214d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.652+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6c40 0x5641214d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d9622a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 0x5641214d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d912bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 0x5641214d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d912bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 0x5641214d4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44284/0 (socket says 172.21.15.64:44284) 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d9622a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000e1d0 con 0x5641214cbdb0 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d9622a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000f640 con 0x5641214c6c40 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d9622a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000e8a0 con 0x5641214c6250 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d9622a740 10 client.0 _ll_register_callbacks cb 0x7f9d7c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 0x5641214d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 0x5641214d5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43518/0 (socket says 172.21.15.64:43518) 2024-05-09T09:14:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 -- 172.21.15.64:0/2139703957 learned_addr learned my addr 172.21.15.64:0/2139703957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d90abc640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6c40 0x5641214d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 -- 172.21.15.64:0/2139703957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 msgr2=0x5641214d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 0x5641214d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 -- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6c40 msgr2=0x5641214d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.653+0000 7f9d91abe640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6c40 0x5641214d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.654+0000 7f9d91abe640 1 -- 172.21.15.64:0/2139703957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641214d6260 con 0x5641214cbdb0 2024-05-09T09:14:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.654+0000 7f9d912bd640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 0x5641214d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.654+0000 7f9d90abc640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6c40 0x5641214d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.654+0000 7f9d91abe640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 0x5641214d5c70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009f10 tx=0x7f9d84012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e2869214594bee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.654+0000 7f9d74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.655+0000 7f9d74ff9640 1 -- 172.21.15.64:0/2139703957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8402b070 con 0x5641214cbdb0 2024-05-09T09:14:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.655+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8801d040 con 0x5641214cbdb0 2024-05-09T09:14:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.655+0000 7f9d74ff9640 1 -- 172.21.15.64:0/2139703957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d84012bf0 con 0x5641214cbdb0 2024-05-09T09:14:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.656+0000 7f9d74ff9640 1 -- 172.21.15.64:0/2139703957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9d84034050 con 0x5641214cbdb0 2024-05-09T09:14:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.656+0000 7f9d9622a740 10 client.16899 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.657+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5641215427d0 con 0x5641214cbdb0 2024-05-09T09:14:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.657+0000 7f9d74ff9640 1 -- 172.21.15.64:0/2139703957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d84012e30 con 0x5641214cbdb0 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d74ff9640 1 -- 172.21.15.64:0/2139703957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f9d84037020 con 0x5641214cbdb0 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d9622a740 10 client.16899 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d9622a740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d9622a740 20 client.16899 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d9622a740 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x564121544f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d9622a740 4 client.16899 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9d8801d040 con 0x5641214dc170 2024-05-09T09:14:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.658+0000 7f9d912bd640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x564121544f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.659+0000 7f9d912bd640 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x564121544f30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca694356dfe65c43 server_cookie=869c75aaf972da0d in_seq=0 out_seq=0 2024-05-09T09:14:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.659+0000 7f9d74ff9640 10 client.16899 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.672+0000 7f9d74ff9640 1 -- 172.21.15.64:0/2139703957 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9d8801d040 con 0x5641214dc170 2024-05-09T09:14:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.672+0000 7f9d74ff9640 10 client.16899 handle_command_reply: tid=0 2024-05-09T09:14:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.674+0000 7f9d9622a740 1 client.16899 shutdown 2024-05-09T09:14:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 msgr2=0x5641214d5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 0x5641214d5c70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009f10 tx=0x7f9d84012670 comp rx=0 tx=0).stop 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 shutdown_connections 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5641214dc170 0x564121544f30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641214cbdb0 0x5641214d5c70 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5641214c6c40 0x5641214d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 --2- 172.21.15.64:0/2139703957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641214c6250 0x5641214d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d74ff9640 0 client.16899 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.677+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 >> 172.21.15.64:0/2139703957 conn(0x5641214c18c0 msgr2=0x5641214c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.678+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 shutdown_connections 2024-05-09T09:14:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.682+0000 7f9d9622a740 1 -- 172.21.15.64:0/2139703957 wait complete. 2024-05-09T09:14:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.682+0000 7f9d9622a740 20 client.16899 trim_cache size 0 max 16384 2024-05-09T09:14:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:14:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:14:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:14:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:14:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2e20 msgr2=0x7f9d780a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2e20 0x7f9d780a52e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5641214c7040 tx=0x7f9d88008040 comp rx=0 tx=0).stop 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 msgr2=0x7f9d8c1a6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 0x7f9d8c1a6dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005940 tx=0x7f9d7c0042f0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 shutdown_connections 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2e20 0x7f9d780a52e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c10dc30 0x7f9d8c1ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c10b280 0x7f9d8c1a6dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 --2- 172.21.15.64:0/3996080215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c108910 0x7f9d8c1a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.683+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 >> 172.21.15.64:0/3996080215 conn(0x7f9d8c06f7d0 msgr2=0x7f9d8c1092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.684+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 shutdown_connections 2024-05-09T09:14:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:17.684+0000 7f9d93521640 1 -- 172.21.15.64:0/3996080215 wait complete. 2024-05-09T09:14:18.956 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:18 smithi064 ceph-mon[30149]: pgmap v525: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.7 MiB/s wr, 833 op/s 2024-05-09T09:14:18.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:18 smithi179 ceph-mon[34086]: pgmap v525: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.7 MiB/s wr, 833 op/s 2024-05-09T09:14:19.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:18 smithi139 ceph-mon[33599]: pgmap v525: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.7 MiB/s wr, 833 op/s 2024-05-09T09:14:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:20 smithi179 ceph-mon[34086]: pgmap v526: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.7 MiB/s wr, 1.09k op/s 2024-05-09T09:14:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:20 smithi139 ceph-mon[33599]: pgmap v526: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.7 MiB/s wr, 1.09k op/s 2024-05-09T09:14:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:20 smithi064 ceph-mon[30149]: pgmap v526: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.7 MiB/s wr, 1.09k op/s 2024-05-09T09:14:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 240.0 s, 1008.0 tps, lat 99.229 ms stddev 30.092 2024-05-09T09:14:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:22 smithi179 ceph-mon[34086]: pgmap v527: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 895 op/s 2024-05-09T09:14:23.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:22 smithi139 ceph-mon[33599]: pgmap v527: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 895 op/s 2024-05-09T09:14:23.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:22 smithi064 ceph-mon[30149]: pgmap v527: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 895 op/s 2024-05-09T09:14:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:24 smithi179 ceph-mon[34086]: pgmap v528: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.0 MiB/s wr, 1.01k op/s 2024-05-09T09:14:25.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:24 smithi139 ceph-mon[33599]: pgmap v528: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.0 MiB/s wr, 1.01k op/s 2024-05-09T09:14:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:24 smithi064 ceph-mon[30149]: pgmap v528: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.0 MiB/s wr, 1.01k op/s 2024-05-09T09:14:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:14:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:14:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:14:26.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:14:26.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:14:26.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:14:26.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:14:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:14:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:14:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 245.0 s, 968.8 tps, lat 103.038 ms stddev 127.039 2024-05-09T09:14:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:26 smithi179 ceph-mon[34086]: pgmap v529: 129 pgs: 129 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.2 MiB/s wr, 1.00k op/s 2024-05-09T09:14:27.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:26 smithi139 ceph-mon[33599]: pgmap v529: 129 pgs: 129 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.2 MiB/s wr, 1.00k op/s 2024-05-09T09:14:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:26 smithi064 ceph-mon[30149]: pgmap v529: 129 pgs: 129 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.2 MiB/s wr, 1.00k op/s 2024-05-09T09:14:28.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:28 smithi179 ceph-mon[34086]: pgmap v530: 129 pgs: 129 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 875 op/s 2024-05-09T09:14:29.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:28 smithi139 ceph-mon[33599]: pgmap v530: 129 pgs: 129 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 875 op/s 2024-05-09T09:14:29.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:28 smithi064 ceph-mon[30149]: pgmap v530: 129 pgs: 129 active+clean; 1.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 875 op/s 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.848+0000 7f53f8577640 1 -- 172.21.15.64:0/523025150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f010dd50 msgr2=0x7f53f0110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.848+0000 7f53f8577640 1 --2- 172.21.15.64:0/523025150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f010dd50 0x7f53f0110140 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009ca0 tx=0x7f53e802a8d0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.849+0000 7f53f8577640 1 -- 172.21.15.64:0/523025150 shutdown_connections 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.849+0000 7f53f8577640 1 --2- 172.21.15.64:0/523025150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f010dd50 0x7f53f0110140 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.849+0000 7f53f8577640 1 --2- 172.21.15.64:0/523025150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f0074290 0x7f53f0074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.849+0000 7f53f8577640 1 --2- 172.21.15.64:0/523025150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f0075870 0x7f53f0073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.849+0000 7f53f8577640 1 -- 172.21.15.64:0/523025150 >> 172.21.15.64:0/523025150 conn(0x7f53f00fccf0 msgr2=0x7f53f00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.850+0000 7f53f8577640 1 -- 172.21.15.64:0/523025150 shutdown_connections 2024-05-09T09:14:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.850+0000 7f53f8577640 1 -- 172.21.15.64:0/523025150 wait complete. 2024-05-09T09:14:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 Processor -- start 2024-05-09T09:14:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 -- start start 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f0074290 0x7f53f0071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f0075870 0x7f53f0071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 0x7f53f00726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f53f006b720 con 0x7f53f0074290 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53f006b5a0 con 0x7f53f010dd50 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.851+0000 7f53f8577640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f53f006b8a0 con 0x7f53f0075870 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 0x7f53f00726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 0x7f53f00726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34036/0 (socket says 172.21.15.64:34036) 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 -- 172.21.15.64:0/3683019545 learned_addr learned my addr 172.21.15.64:0/3683019545 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6313640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f0074290 0x7f53f0071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 -- 172.21.15.64:0/3683019545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f0075870 msgr2=0x7f53f0071680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:14:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f0075870 0x7f53f0071680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 -- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f0074290 msgr2=0x7f53f0071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f0074290 0x7f53f0071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.852+0000 7f53f6b14640 1 -- 172.21.15.64:0/3683019545 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0072d30 con 0x7f53f010dd50 2024-05-09T09:14:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.853+0000 7f53f6b14640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 0x7f53f00726c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f53e802ae40 tx=0x7f53e8030be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.853+0000 7f53e77fe640 1 -- 172.21.15.64:0/3683019545 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8003970 con 0x7f53f010dd50 2024-05-09T09:14:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.853+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f0072fc0 con 0x7f53f010dd50 2024-05-09T09:14:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.853+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f0073510 con 0x7f53f010dd50 2024-05-09T09:14:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.854+0000 7f53e77fe640 1 -- 172.21.15.64:0/3683019545 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8004050 con 0x7f53f010dd50 2024-05-09T09:14:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.854+0000 7f53e77fe640 1 -- 172.21.15.64:0/3683019545 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e802f750 con 0x7f53f010dd50 2024-05-09T09:14:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.857+0000 7f53fb363740 1 Processor -- start 2024-05-09T09:14:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53e77fe640 1 -- 172.21.15.64:0/3683019545 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f53e802f8f0 con 0x7f53f010dd50 2024-05-09T09:14:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53fb363740 1 -- start start 2024-05-09T09:14:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53fb363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc2fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53fb363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 0x55abdc3028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53fb363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc3053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53fb363740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55abdc261340 con 0x55abdc2fd400 2024-05-09T09:14:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.858+0000 7f53fb363740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55abdc209290 con 0x55abdc2fddf0 2024-05-09T09:14:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53fb363740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55abdc25d300 con 0x55abdc302f60 2024-05-09T09:14:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 0x55abdc3028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 0x55abdc3028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34042/0 (socket says 172.21.15.64:34042) 2024-05-09T09:14:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 -- 172.21.15.64:0/2903125458 learned_addr learned my addr 172.21.15.64:0/2903125458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f5b12640 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc2fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f5b12640 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc3053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 -- 172.21.15.64:0/2903125458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 msgr2=0x55abdc3053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc3053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 -- 172.21.15.64:0/2903125458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 msgr2=0x55abdc2fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc2fd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53f6313640 1 -- 172.21.15.64:0/2903125458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abdc22d8e0 con 0x55abdc2fddf0 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.859+0000 7f53e77fe640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53dc0a2da0 0x7f53dc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.860+0000 7f53e77fe640 1 -- 172.21.15.64:0/3683019545 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f53e80f0830 con 0x7f53f010dd50 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.860+0000 7f53f6313640 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 0x55abdc3028f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bcb0 tx=0x7f53e000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dead951db17ee993 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.860+0000 7f53f5b12640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53dc0a2da0 0x7f53dc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.860+0000 7f53e4bfe640 1 -- 172.21.15.64:0/2903125458 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0015070 con 0x55abdc2fddf0 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.860+0000 7f53e4bfe640 1 -- 172.21.15.64:0/2903125458 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e001a910 con 0x55abdc2fddf0 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.860+0000 7f53e4bfe640 1 -- 172.21.15.64:0/2903125458 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011990 con 0x55abdc2fddf0 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.861+0000 7f53f5b12640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53dc0a2da0 0x7f53dc0a5260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55abdc2fe650 tx=0x7f53ec00b060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.861+0000 7f53fb363740 1 -- 172.21.15.64:0/2903125458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 msgr2=0x55abdc3028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.861+0000 7f53fb363740 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 0x55abdc3028f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bcb0 tx=0x7f53e000f730 comp rx=0 tx=0).stop 2024-05-09T09:14:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.862+0000 7f53fb363740 1 -- 172.21.15.64:0/2903125458 shutdown_connections 2024-05-09T09:14:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.862+0000 7f53fb363740 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc3053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.862+0000 7f53fb363740 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fddf0 0x55abdc3028f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.862+0000 7f53fb363740 1 --2- 172.21.15.64:0/2903125458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc2fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.862+0000 7f53fb363740 1 -- 172.21.15.64:0/2903125458 >> 172.21.15.64:0/2903125458 conn(0x55abdc2f8a70 msgr2=0x55abdc2f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.865+0000 7f53fb363740 1 -- 172.21.15.64:0/2903125458 shutdown_connections 2024-05-09T09:14:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.865+0000 7f53fb363740 1 -- 172.21.15.64:0/2903125458 wait complete. 2024-05-09T09:14:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.865+0000 7f53fb363740 1 -- start start 2024-05-09T09:14:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.866+0000 7f53fb363740 1 Processor -- start 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.866+0000 7f53fb363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc3076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.866+0000 7f53fb363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 0x55abdc307bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.866+0000 7f53fb363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc30ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.866+0000 7f53f5b12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 0x55abdc307bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc3076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f5b12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 0x55abdc307bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39362/0 (socket says 172.21.15.64:39362) 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc3076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34052/0 (socket says 172.21.15.64:34052) 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f5b12640 1 -- 172.21.15.64:0/2422554706 learned_addr learned my addr 172.21.15.64:0/2422554706 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55abdc305f10 con 0x55abdc2fddf0 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55abdc21c790 con 0x55abdc2fd400 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55abdc306090 con 0x55abdc302f60 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc30ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53fb363740 10 client.0 _ll_register_callbacks cb 0x55abdc257a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 -- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 msgr2=0x55abdc3076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc3076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 -- 172.21.15.64:0/2422554706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 msgr2=0x55abdc307bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 0x55abdc307bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abdbf75210 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6313640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc3076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f5b12640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 0x55abdc307bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.867+0000 7f53f6b14640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc30ec30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53e80edf40 tx=0x7f53e80098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d4f3331403b883 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53d9ffb640 1 -- 172.21.15.64:0/2422554706 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8031e20 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53d9ffb640 1 -- 172.21.15.64:0/2422554706 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f53e80b9310 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e001a910 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53d9ffb640 1 -- 172.21.15.64:0/2422554706 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f53e80bcca0 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53fb363740 10 client.26572 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.868+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abdc041620 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53d9ffb640 1 -- 172.21.15.64:0/2422554706 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e001a910 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53d9ffb640 1 -- 172.21.15.64:0/2422554706 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f53e8004b40 con 0x55abdc302f60 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53fb363740 10 client.26572 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53fb363740 10 client.26572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53fb363740 20 client.26572 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53fb363740 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc313070 0x55abdc3154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53fb363740 4 client.26572 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abdc26ea70 con 0x55abdc313070 2024-05-09T09:14:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.869+0000 7f53f6313640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc313070 0x55abdc3154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.870+0000 7f53f6313640 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc313070 0x55abdc3154d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf247d6eecf2ad6 server_cookie=2f2500a390068d6b in_seq=0 out_seq=0 2024-05-09T09:14:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.870+0000 7f53d9ffb640 10 client.26572 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.871+0000 7f53d9ffb640 1 -- 172.21.15.64:0/2422554706 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abdc26ea70 con 0x55abdc313070 2024-05-09T09:14:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.871+0000 7f53d9ffb640 10 client.26572 handle_command_reply: tid=0 2024-05-09T09:14:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.871+0000 7f53fb363740 1 client.26572 shutdown 2024-05-09T09:14:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.872+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 msgr2=0x55abdc30ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.872+0000 7f53fb363740 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc30ec30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53e80edf40 tx=0x7f53e80098d0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.872+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 shutdown_connections 2024-05-09T09:14:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.872+0000 7f53fb363740 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc313070 0x55abdc3154d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.872+0000 7f53fb363740 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc302f60 0x55abdc30ec30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.873+0000 7f53fb363740 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fddf0 0x55abdc307bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.873+0000 7f53fb363740 1 --2- 172.21.15.64:0/2422554706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc3076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.873+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 >> 172.21.15.64:0/2422554706 conn(0x55abdc2f8a70 msgr2=0x55abdc2f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.874+0000 7f53d9ffb640 0 client.26572 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.875+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 shutdown_connections 2024-05-09T09:14:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.876+0000 7f53fb363740 1 -- 172.21.15.64:0/2422554706 wait complete. 2024-05-09T09:14:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.876+0000 7f53fb363740 20 client.26572 trim_cache size 0 max 16384 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.910+0000 7f53fb363740 1 Processor -- start 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.910+0000 7f53fb363740 1 -- start start 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53fb363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc31ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53fb363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc31b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53fb363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc302f60 0x55abdc30ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53fb363740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f53d40124b0 con 0x55abdc2fd400 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53fb363740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400fcf0 con 0x55abdc302f60 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53fb363740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400db40 con 0x55abdc2fddf0 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53f5b12640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc31b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53f5b12640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc31b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33638/0 (socket says 172.21.15.64:33638) 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53f5b12640 1 -- 172.21.15.64:0/4070357748 learned_addr learned my addr 172.21.15.64:0/4070357748 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.911+0000 7f53f6313640 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc31ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f5b12640 1 -- 172.21.15.64:0/4070357748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc302f60 msgr2=0x55abdc30ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f5b12640 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc302f60 0x55abdc30ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f5b12640 1 -- 172.21.15.64:0/4070357748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 msgr2=0x55abdc31ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f5b12640 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc31ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f5b12640 1 -- 172.21.15.64:0/4070357748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abdc307f40 con 0x55abdc2fddf0 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f5b12640 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc31b320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00bad0 tx=0x7f53ec012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335157a8f55f89ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53f6313640 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc31ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53dbfff640 1 -- 172.21.15.64:0/4070357748 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec011040 con 0x55abdc2fddf0 2024-05-09T09:14:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53dbfff640 1 -- 172.21.15.64:0/4070357748 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53ec028070 con 0x55abdc2fddf0 2024-05-09T09:14:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.912+0000 7f53dbfff640 1 -- 172.21.15.64:0/4070357748 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0163b0 con 0x55abdc2fddf0 2024-05-09T09:14:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.914+0000 7f53fb363740 1 -- 172.21.15.64:0/4070357748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 msgr2=0x55abdc31b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.914+0000 7f53fb363740 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc31b320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00bad0 tx=0x7f53ec012000 comp rx=0 tx=0).stop 2024-05-09T09:14:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 -- 172.21.15.64:0/4070357748 shutdown_connections 2024-05-09T09:14:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc302f60 0x55abdc30ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc31b320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 --2- 172.21.15.64:0/4070357748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc2fd400 0x55abdc31ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 -- 172.21.15.64:0/4070357748 >> 172.21.15.64:0/4070357748 conn(0x55abdc2f8a70 msgr2=0x55abdc2f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 -- 172.21.15.64:0/4070357748 shutdown_connections 2024-05-09T09:14:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.916+0000 7f53fb363740 1 -- 172.21.15.64:0/4070357748 wait complete. 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.917+0000 7f53fb363740 1 -- start start 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.917+0000 7f53fb363740 1 Processor -- start 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.917+0000 7f53fb363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc30bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53fb363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc30c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53f6313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc30bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53f6313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc30bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34076/0 (socket says 172.21.15.64:34076) 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53f5b12640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc30c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53f5b12640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc30c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33644/0 (socket says 172.21.15.64:33644) 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53f6313640 1 -- 172.21.15.64:0/1297442145 learned_addr learned my addr 172.21.15.64:0/1297442145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc302f60 0x55abdc377640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400e1b0 con 0x55abdc302f60 2024-05-09T09:14:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400f620 con 0x55abdc2fd400 2024-05-09T09:14:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.918+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f53d400e880 con 0x55abdc2fddf0 2024-05-09T09:14:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53fb363740 10 client.0 _ll_register_callbacks cb 0x7f53e8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53f6313640 1 -- 172.21.15.64:0/1297442145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 msgr2=0x55abdc30c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53f6313640 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc30c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53f6313640 1 -- 172.21.15.64:0/1297442145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc302f60 msgr2=0x55abdc377640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53f6313640 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc302f60 0x55abdc377640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53f6313640 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abdc377bb0 con 0x55abdc2fd400 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53ec028070 con 0x55abdc2fd400 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53f6313640 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc30bc20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f53e000ba40 tx=0x7f53e000b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92496786593421fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.919+0000 7f53d9ffb640 10 client.26408 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.920+0000 7f53d9ffb640 1 -- 172.21.15.64:0/1297442145 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0015040 con 0x55abdc2fd400 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.920+0000 7f53d9ffb640 1 -- 172.21.15.64:0/1297442145 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f53e0004020 con 0x55abdc2fd400 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.920+0000 7f53d9ffb640 1 -- 172.21.15.64:0/1297442145 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f53e0004260 con 0x55abdc2fd400 2024-05-09T09:14:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.920+0000 7f53fb363740 10 client.26408 fetch_fsmap learned FSMap version 39 2024-05-09T09:14:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.920+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55abdc378dc0 con 0x55abdc2fd400 2024-05-09T09:14:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.920+0000 7f53d9ffb640 1 -- 172.21.15.64:0/1297442145 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f53e0004680 con 0x55abdc2fd400 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53fb363740 10 client.26408 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53fb363740 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53fb363740 20 client.26408 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc379480 0x55abdc37b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53fb363740 4 client.26408 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f53ec028070 con 0x55abdc379480 2024-05-09T09:14:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.921+0000 7f53f5b12640 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc379480 0x55abdc37b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.922+0000 7f53f5b12640 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc379480 0x55abdc37b8a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b26f15626edf27 server_cookie=9c02f8779e03a643 in_seq=0 out_seq=0 2024-05-09T09:14:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.922+0000 7f53d9ffb640 10 client.26408 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53d9ffb640 1 -- 172.21.15.64:0/1297442145 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f53ec028070 con 0x55abdc379480 2024-05-09T09:14:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53d9ffb640 10 client.26408 handle_command_reply: tid=0 2024-05-09T09:14:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53fb363740 1 client.26408 shutdown 2024-05-09T09:14:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 msgr2=0x55abdc30bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc30bc20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f53e000ba40 tx=0x7f53e000b640 comp rx=0 tx=0).stop 2024-05-09T09:14:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 shutdown_connections 2024-05-09T09:14:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.951+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55abdc379480 0x55abdc37b8a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abdc302f60 0x55abdc377640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53d9ffb640 0 client.26408 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55abdc2fddf0 0x55abdc30c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 1 --2- 172.21.15.64:0/1297442145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55abdc2fd400 0x55abdc30bc20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 >> 172.21.15.64:0/1297442145 conn(0x55abdc2f8a70 msgr2=0x55abdc2ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 shutdown_connections 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 1 -- 172.21.15.64:0/1297442145 wait complete. 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.952+0000 7f53fb363740 20 client.26408 trim_cache size 0 max 16384 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:14:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:14:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.954+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53dc0a2da0 msgr2=0x7f53dc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.954+0000 7f53f8577640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53dc0a2da0 0x7f53dc0a5260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55abdc2fe650 tx=0x7f53ec00b060 comp rx=0 tx=0).stop 2024-05-09T09:14:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.954+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 msgr2=0x7f53f00726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.954+0000 7f53f8577640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 0x7f53f00726c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f53e802ae40 tx=0x7f53e8030be0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 shutdown_connections 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f53dc0a2da0 0x7f53dc0a5260 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f010dd50 0x7f53f00726c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53f0075870 0x7f53f0071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 --2- 172.21.15.64:0/3683019545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f0074290 0x7f53f0071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 >> 172.21.15.64:0/3683019545 conn(0x7f53f00fccf0 msgr2=0x7f53f01010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 shutdown_connections 2024-05-09T09:14:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:30.955+0000 7f53f8577640 1 -- 172.21.15.64:0/3683019545 wait complete. 2024-05-09T09:14:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:30 smithi179 ceph-mon[34086]: pgmap v531: 129 pgs: 129 active+clean; 1.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:14:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:30 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:14:31.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:30 smithi139 ceph-mon[33599]: pgmap v531: 129 pgs: 129 active+clean; 1.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:14:31.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:30 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:14:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:30 smithi064 ceph-mon[30149]: pgmap v531: 129 pgs: 129 active+clean; 1.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:14:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:30 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:14:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 250.0 s, 992.6 tps, lat 100.809 ms stddev 37.747 2024-05-09T09:14:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:32 smithi179 ceph-mon[34086]: pgmap v532: 129 pgs: 129 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 883 op/s 2024-05-09T09:14:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:14:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:14:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:14:33.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:32 smithi139 ceph-mon[33599]: pgmap v532: 129 pgs: 129 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 883 op/s 2024-05-09T09:14:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:14:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:14:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:14:33.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:32 smithi064 ceph-mon[30149]: pgmap v532: 129 pgs: 129 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 883 op/s 2024-05-09T09:14:33.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:14:33.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:14:33.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:14:34.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:34 smithi179 ceph-mon[34086]: pgmap v533: 129 pgs: 129 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-05-09T09:14:35.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:34 smithi139 ceph-mon[33599]: pgmap v533: 129 pgs: 129 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-05-09T09:14:35.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:34 smithi064 ceph-mon[30149]: pgmap v533: 129 pgs: 129 active+clean; 1.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-05-09T09:14:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 255.0 s, 804.4 tps, lat 124.139 ms stddev 204.768 2024-05-09T09:14:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:36 smithi179 ceph-mon[34086]: pgmap v534: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 988 op/s 2024-05-09T09:14:37.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:36 smithi139 ceph-mon[33599]: pgmap v534: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 988 op/s 2024-05-09T09:14:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:36 smithi064 ceph-mon[30149]: pgmap v534: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 988 op/s 2024-05-09T09:14:39.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:38 smithi139 ceph-mon[33599]: pgmap v535: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 876 op/s 2024-05-09T09:14:39.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:38 smithi064 ceph-mon[30149]: pgmap v535: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 876 op/s 2024-05-09T09:14:39.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:38 smithi179 ceph-mon[34086]: pgmap v535: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 876 op/s 2024-05-09T09:14:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:40 smithi139 ceph-mon[33599]: pgmap v536: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-09T09:14:41.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:40 smithi064 ceph-mon[30149]: pgmap v536: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-09T09:14:41.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:40 smithi179 ceph-mon[34086]: pgmap v536: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-09T09:14:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 260.0 s, 772.6 tps, lat 129.953 ms stddev 200.200 2024-05-09T09:14:41.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:14:41.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:14:42.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.309+0000 7fe539161640 1 -- 172.21.15.64:0/3616341536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe53410db70 msgr2=0x7fe53410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.309+0000 7fe539161640 1 --2- 172.21.15.64:0/3616341536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe53410db70 0x7fe53410e020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009be0 tx=0x7fe52c02a6e0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.310+0000 7fe539161640 1 -- 172.21.15.64:0/3616341536 shutdown_connections 2024-05-09T09:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.310+0000 7fe539161640 1 --2- 172.21.15.64:0/3616341536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe53410db70 0x7fe53410e020 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.310+0000 7fe539161640 1 --2- 172.21.15.64:0/3616341536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe53410b1a0 0x7fe53410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.310+0000 7fe539161640 1 --2- 172.21.15.64:0/3616341536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe534108830 0x7fe53410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.310+0000 7fe539161640 1 -- 172.21.15.64:0/3616341536 >> 172.21.15.64:0/3616341536 conn(0x7fe53406e6c0 msgr2=0x7fe534070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.311+0000 7fe539161640 1 -- 172.21.15.64:0/3616341536 shutdown_connections 2024-05-09T09:14:42.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.311+0000 7fe539161640 1 -- 172.21.15.64:0/3616341536 wait complete. 2024-05-09T09:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.312+0000 7fe539161640 1 Processor -- start 2024-05-09T09:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 -- start start 2024-05-09T09:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 0x7fe53419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53410b1a0 0x7fe53419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe53410db70 0x7fe5341a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341119e0 con 0x7fe53410b1a0 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe534111860 con 0x7fe534108830 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.313+0000 7fe539161640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe534111b60 con 0x7fe53410db70 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.314+0000 7fe53359e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe53410db70 0x7fe5341a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.314+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 0x7fe53419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.314+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 0x7fe53419e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50810/0 (socket says 172.21.15.64:50810) 2024-05-09T09:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.314+0000 7fe532d9d640 1 -- 172.21.15.64:0/3122742022 learned_addr learned my addr 172.21.15.64:0/3122742022 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.314+0000 7fe53259c640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53410b1a0 0x7fe53419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.317+0000 7fe53259c640 1 -- 172.21.15.64:0/3122742022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe53410db70 msgr2=0x7fe5341a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.317+0000 7fe53259c640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe53410db70 0x7fe5341a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.317+0000 7fe53259c640 1 -- 172.21.15.64:0/3122742022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 msgr2=0x7fe53419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.317+0000 7fe53259c640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 0x7fe53419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.317+0000 7fe53259c640 1 -- 172.21.15.64:0/3122742022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341a5d10 con 0x7fe53410b1a0 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.317+0000 7fe532d9d640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 0x7fe53419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.318+0000 7fe53259c640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53410b1a0 0x7fe53419e630 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe524005fd0 tx=0x7fe52401fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.319+0000 7fe51bfff640 1 -- 172.21.15.64:0/3122742022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52401db90 con 0x7fe53410b1a0 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.319+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341a5fa0 con 0x7fe53410b1a0 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.319+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341a6510 con 0x7fe53410b1a0 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.320+0000 7fe51bfff640 1 -- 172.21.15.64:0/3122742022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe52401dd30 con 0x7fe53410b1a0 2024-05-09T09:14:42.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.320+0000 7fe51bfff640 1 -- 172.21.15.64:0/3122742022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52401db90 con 0x7fe53410b1a0 2024-05-09T09:14:42.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.327+0000 7fe53be6c740 1 Processor -- start 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 -- start start 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f29350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 0x561d97f2e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f2ea90 0x561d97f30f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d97e8dd00 con 0x561d97f28f30 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d97e354c0 con 0x561d97f29920 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe53be6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561d97e89cc0 con 0x561d97f2ea90 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.331+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 0x561d97f2e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.332+0000 7fe53259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f2ea90 0x561d97f30f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.332+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 0x561d97f2e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50812/0 (socket says 172.21.15.64:50812) 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.332+0000 7fe532d9d640 1 -- 172.21.15.64:0/4033835038 learned_addr learned my addr 172.21.15.64:0/4033835038 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.332+0000 7fe51bfff640 1 -- 172.21.15.64:0/3122742022 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe5240035f0 con 0x7fe53410b1a0 2024-05-09T09:14:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.334+0000 7fe51bfff640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe51c0a2e10 0x7fe51c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.335+0000 7fe51bfff640 1 -- 172.21.15.64:0/3122742022 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5240e0180 con 0x7fe53410b1a0 2024-05-09T09:14:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.336+0000 7fe53259c640 1 -- 172.21.15.64:0/4033835038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 msgr2=0x561d97f2e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.336+0000 7fe53259c640 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 0x561d97f2e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.336+0000 7fe53259c640 1 -- 172.21.15.64:0/4033835038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 msgr2=0x561d97f29350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.336+0000 7fe53259c640 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f29350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.336+0000 7fe53259c640 1 -- 172.21.15.64:0/4033835038 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d97c6b7d0 con 0x561d97f2ea90 2024-05-09T09:14:42.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.336+0000 7fe532d9d640 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 0x561d97f2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:42.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.337+0000 7fe53259c640 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f2ea90 0x561d97f30f20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe5240a8a50 tx=0x7fe5240ac3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b8e435b03baa9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.338+0000 7fe53359e640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe51c0a2e10 0x7fe51c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.339+0000 7fe5197fa640 1 -- 172.21.15.64:0/4033835038 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240accf0 con 0x561d97f2ea90 2024-05-09T09:14:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.339+0000 7fe5197fa640 1 -- 172.21.15.64:0/4033835038 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5240ac5f0 con 0x561d97f2ea90 2024-05-09T09:14:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.339+0000 7fe5197fa640 1 -- 172.21.15.64:0/4033835038 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524095de0 con 0x561d97f2ea90 2024-05-09T09:14:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.341+0000 7fe53be6c740 1 -- 172.21.15.64:0/4033835038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f2ea90 msgr2=0x561d97f30f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.341+0000 7fe53be6c740 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f2ea90 0x561d97f30f20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe5240a8a50 tx=0x7fe5240ac3c0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.345+0000 7fe53be6c740 1 -- 172.21.15.64:0/4033835038 shutdown_connections 2024-05-09T09:14:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.345+0000 7fe53be6c740 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f2ea90 0x561d97f30f20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.345+0000 7fe53be6c740 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f29920 0x561d97f2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.345+0000 7fe53be6c740 1 --2- 172.21.15.64:0/4033835038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f29350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.345+0000 7fe53be6c740 1 -- 172.21.15.64:0/4033835038 >> 172.21.15.64:0/4033835038 conn(0x561d97f245a0 msgr2=0x561d97f249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.346+0000 7fe53359e640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe51c0a2e10 0x7fe51c0a52d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004af0 tx=0x7fe52c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.349+0000 7fe53be6c740 1 -- 172.21.15.64:0/4033835038 shutdown_connections 2024-05-09T09:14:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.349+0000 7fe53be6c740 1 -- 172.21.15.64:0/4033835038 wait complete. 2024-05-09T09:14:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.349+0000 7fe53be6c740 1 -- start start 2024-05-09T09:14:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 Processor -- start 2024-05-09T09:14:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f33810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f3a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d97f319e0 con 0x561d97f28f30 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d97e48b90 con 0x561d97f2ea90 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561d97f31b60 con 0x561d97f29920 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.350+0000 7fe53be6c740 10 client.0 _ll_register_callbacks cb 0x561d97e84da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.351+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.351+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f332d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35194/0 (socket says 172.21.15.64:35194) 2024-05-09T09:14:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.351+0000 7fe532d9d640 1 -- 172.21.15.64:0/235224090 learned_addr learned my addr 172.21.15.64:0/235224090 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:42.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.353+0000 7fe53259c640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f33810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.354+0000 7fe53359e640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f3a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.354+0000 7fe532d9d640 1 -- 172.21.15.64:0/235224090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 msgr2=0x561d97f33810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.354+0000 7fe532d9d640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f33810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.354+0000 7fe532d9d640 1 -- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 msgr2=0x561d97f3a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.354+0000 7fe532d9d640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f3a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.354+0000 7fe532d9d640 1 -- 172.21.15.64:0/235224090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d97ba13f0 con 0x561d97f28f30 2024-05-09T09:14:42.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.355+0000 7fe532d9d640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f332d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe528011590 tx=0x7fe528011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3652178b35c7ebd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.357+0000 7fe516ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.357+0000 7fe516ff5640 1 -- 172.21.15.64:0/235224090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528015870 con 0x561d97f28f30 2024-05-09T09:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.357+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5240ac5f0 con 0x561d97f28f30 2024-05-09T09:14:42.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.357+0000 7fe516ff5640 1 -- 172.21.15.64:0/235224090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe528015a10 con 0x561d97f28f30 2024-05-09T09:14:42.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.359+0000 7fe516ff5640 1 -- 172.21.15.64:0/235224090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe528015c10 con 0x561d97f28f30 2024-05-09T09:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.362+0000 7fe53be6c740 10 client.16920 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:42.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.362+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561d97b610f0 con 0x561d97f28f30 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.363+0000 7fe516ff5640 1 -- 172.21.15.64:0/235224090 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5240ac5f0 con 0x561d97f28f30 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.363+0000 7fe516ff5640 1 -- 172.21.15.64:0/235224090 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fe528013400 con 0x561d97f28f30 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.364+0000 7fe53be6c740 10 client.16920 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.364+0000 7fe53be6c740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.364+0000 7fe53be6c740 20 client.16920 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.364+0000 7fe53be6c740 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97f3f230 0x561d97f41690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.364+0000 7fe53be6c740 4 client.16920 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:42.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.364+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d97e9b430 con 0x561d97f3f230 2024-05-09T09:14:42.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.375+0000 7fe53259c640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97f3f230 0x561d97f41690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.388+0000 7fe53259c640 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97f3f230 0x561d97f41690 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf8563d53edc904 server_cookie=63a233f5cd7b53e8 in_seq=0 out_seq=0 2024-05-09T09:14:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.388+0000 7fe516ff5640 10 client.16920 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:42.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.398+0000 7fe516ff5640 1 -- 172.21.15.64:0/235224090 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d97e9b430 con 0x561d97f3f230 2024-05-09T09:14:42.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.398+0000 7fe516ff5640 10 client.16920 handle_command_reply: tid=0 2024-05-09T09:14:42.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.399+0000 7fe53be6c740 1 client.16920 shutdown 2024-05-09T09:14:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.414+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 msgr2=0x561d97f332d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.414+0000 7fe53be6c740 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f332d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe528011590 tx=0x7fe528011a50 comp rx=0 tx=0).stop 2024-05-09T09:14:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.419+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 shutdown_connections 2024-05-09T09:14:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.419+0000 7fe53be6c740 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97f3f230 0x561d97f41690 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.419+0000 7fe53be6c740 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f3a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.419+0000 7fe53be6c740 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f33810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.419+0000 7fe53be6c740 1 --2- 172.21.15.64:0/235224090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f332d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.419+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 >> 172.21.15.64:0/235224090 conn(0x561d97f245a0 msgr2=0x561d97f237e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.424+0000 7fe516ff5640 0 client.16920 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.425+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 shutdown_connections 2024-05-09T09:14:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.428+0000 7fe53be6c740 1 -- 172.21.15.64:0/235224090 wait complete. 2024-05-09T09:14:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.428+0000 7fe53be6c740 20 client.16920 trim_cache size 0 max 16384 2024-05-09T09:14:42.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.467+0000 7fe53be6c740 1 Processor -- start 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 -- start start 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f3a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f27590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe520012580 con 0x561d97f28f30 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe52000fd90 con 0x561d97f2ea90 2024-05-09T09:14:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.468+0000 7fe53be6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe52000dbe0 con 0x561d97f29920 2024-05-09T09:14:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.469+0000 7fe53359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f27590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.469+0000 7fe53359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f27590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50834/0 (socket says 172.21.15.64:50834) 2024-05-09T09:14:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.469+0000 7fe53359e640 1 -- 172.21.15.64:0/1147737426 learned_addr learned my addr 172.21.15.64:0/1147737426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.469+0000 7fe532d9d640 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.469+0000 7fe53259c640 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f3a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.470+0000 7fe532d9d640 1 -- 172.21.15.64:0/1147737426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 msgr2=0x561d97f3a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.470+0000 7fe532d9d640 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f3a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.470+0000 7fe532d9d640 1 -- 172.21.15.64:0/1147737426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 msgr2=0x561d97f27590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.470+0000 7fe532d9d640 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f27590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.470+0000 7fe532d9d640 1 -- 172.21.15.64:0/1147737426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d97f34090 con 0x561d97f28f30 2024-05-09T09:14:42.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.471+0000 7fe532d9d640 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f41240 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe528008010 tx=0x7fe528009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d628d034ccfeb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.471+0000 7fe518ff9640 1 -- 172.21.15.64:0/1147737426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528013400 con 0x561d97f28f30 2024-05-09T09:14:42.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.471+0000 7fe518ff9640 1 -- 172.21.15.64:0/1147737426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5280135a0 con 0x561d97f28f30 2024-05-09T09:14:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.472+0000 7fe53be6c740 1 -- 172.21.15.64:0/1147737426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 msgr2=0x561d97f41240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.472+0000 7fe53be6c740 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f41240 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe528008010 tx=0x7fe528009e30 comp rx=0 tx=0).stop 2024-05-09T09:14:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.474+0000 7fe53be6c740 1 -- 172.21.15.64:0/1147737426 shutdown_connections 2024-05-09T09:14:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.474+0000 7fe53be6c740 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97f27590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.474+0000 7fe53be6c740 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f3a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.474+0000 7fe53be6c740 1 --2- 172.21.15.64:0/1147737426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f41240 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.474+0000 7fe53be6c740 1 -- 172.21.15.64:0/1147737426 >> 172.21.15.64:0/1147737426 conn(0x561d97f245a0 msgr2=0x561d97f24980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.477+0000 7fe53be6c740 1 -- 172.21.15.64:0/1147737426 shutdown_connections 2024-05-09T09:14:42.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.477+0000 7fe53be6c740 1 -- 172.21.15.64:0/1147737426 wait complete. 2024-05-09T09:14:42.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.477+0000 7fe53be6c740 1 -- start start 2024-05-09T09:14:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.477+0000 7fe53be6c740 1 Processor -- start 2024-05-09T09:14:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97fa3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe52000e250 con 0x561d97f28f30 2024-05-09T09:14:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe52000f6c0 con 0x561d97f2ea90 2024-05-09T09:14:42.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe52000e920 con 0x561d97f29920 2024-05-09T09:14:42.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe53be6c740 10 client.0 _ll_register_callbacks cb 0x7fe528009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:42.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35224/0 (socket says 172.21.15.64:35224) 2024-05-09T09:14:42.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.478+0000 7fe532d9d640 1 -- 172.21.15.64:0/550549448 learned_addr learned my addr 172.21.15.64:0/550549448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe53259c640 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe532d9d640 1 -- 172.21.15.64:0/550549448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 msgr2=0x561d97f37b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe532d9d640 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f37b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe532d9d640 1 -- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 msgr2=0x561d97fa3800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe532d9d640 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97fa3800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe532d9d640 1 -- 172.21.15.64:0/550549448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d97fa3d40 con 0x561d97f28f30 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe532d9d640 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f375f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe528013a10 tx=0x7fe528011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319c46b8cf5dd2aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.479+0000 7fe516ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.480+0000 7fe516ff5640 1 -- 172.21.15.64:0/550549448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528039060 con 0x561d97f28f30 2024-05-09T09:14:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.480+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5280135a0 con 0x561d97f28f30 2024-05-09T09:14:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.481+0000 7fe516ff5640 1 -- 172.21.15.64:0/550549448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe528034400 con 0x561d97f28f30 2024-05-09T09:14:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.481+0000 7fe516ff5640 1 -- 172.21.15.64:0/550549448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe528040050 con 0x561d97f28f30 2024-05-09T09:14:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.481+0000 7fe53be6c740 10 client.16926 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:42.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.481+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561d97fa4f90 con 0x561d97f28f30 2024-05-09T09:14:42.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe516ff5640 1 -- 172.21.15.64:0/550549448 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5280135a0 con 0x561d97f28f30 2024-05-09T09:14:42.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe516ff5640 1 -- 172.21.15.64:0/550549448 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fe52801d5c0 con 0x561d97f28f30 2024-05-09T09:14:42.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe53be6c740 10 client.16926 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:42.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe53be6c740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:42.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe53be6c740 20 client.16926 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:42.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe53be6c740 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97fa5870 0x561d97fa7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe53be6c740 4 client.16926 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:14:42.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.482+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d97e95780 con 0x561d97fa5870 2024-05-09T09:14:42.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.483+0000 7fe53259c640 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97fa5870 0x561d97fa7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.485+0000 7fe53259c640 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97fa5870 0x561d97fa7c90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cdface3ae21b9b6 server_cookie=20f94ca29a47f398 in_seq=0 out_seq=0 2024-05-09T09:14:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.486+0000 7fe516ff5640 10 client.16926 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.486+0000 7fe516ff5640 1 -- 172.21.15.64:0/550549448 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561d97e95780 con 0x561d97fa5870 2024-05-09T09:14:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.486+0000 7fe516ff5640 10 client.16926 handle_command_reply: tid=0 2024-05-09T09:14:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.486+0000 7fe53be6c740 1 client.16926 shutdown 2024-05-09T09:14:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.486+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 msgr2=0x561d97f375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.486+0000 7fe53be6c740 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f375f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe528013a10 tx=0x7fe528011a50 comp rx=0 tx=0).stop 2024-05-09T09:14:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 shutdown_connections 2024-05-09T09:14:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d97fa5870 0x561d97fa7c90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d97f2ea90 0x561d97fa3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d97f29920 0x561d97f37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 --2- 172.21.15.64:0/550549448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d97f28f30 0x561d97f375f0 secure :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe528013a10 tx=0x7fe528011a50 comp rx=0 tx=0).stop 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 >> 172.21.15.64:0/550549448 conn(0x561d97f245a0 msgr2=0x561d97f41240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe516ff5640 0 client.16926 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 shutdown_connections 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 1 -- 172.21.15.64:0/550549448 wait complete. 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.487+0000 7fe53be6c740 20 client.16926 trim_cache size 0 max 16384 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:14:42.488 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:14:42.489 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:14:42.489 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:14:42.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.490+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe51c0a2e10 msgr2=0x7fe51c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.490+0000 7fe539161640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe51c0a2e10 0x7fe51c0a52d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004af0 tx=0x7fe52c005fd0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.490+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53410b1a0 msgr2=0x7fe53419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.490+0000 7fe539161640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53410b1a0 0x7fe53419e630 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe524005fd0 tx=0x7fe52401fe40 comp rx=0 tx=0).stop 2024-05-09T09:14:42.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 shutdown_connections 2024-05-09T09:14:42.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe51c0a2e10 0x7fe51c0a52d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe53410db70 0x7fe5341a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53410b1a0 0x7fe53419e630 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 --2- 172.21.15.64:0/3122742022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe534108830 0x7fe53419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 >> 172.21.15.64:0/3122742022 conn(0x7fe53406e6c0 msgr2=0x7fe5341090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.491+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 shutdown_connections 2024-05-09T09:14:42.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.492+0000 7fe539161640 1 -- 172.21.15.64:0/3122742022 wait complete. 2024-05-09T09:14:42.512 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:14:42.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:14:42.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.729+0000 7f2c96109640 1 -- 172.21.15.64:0/854922128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c90108830 msgr2=0x7f2c9010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.729+0000 7f2c96109640 1 --2- 172.21.15.64:0/854922128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c90108830 0x7f2c9010ac60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a400 comp rx=0 tx=0).stop 2024-05-09T09:14:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.731+0000 7f2c96109640 1 -- 172.21.15.64:0/854922128 shutdown_connections 2024-05-09T09:14:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.731+0000 7f2c96109640 1 --2- 172.21.15.64:0/854922128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c9010db70 0x7f2c9010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.731+0000 7f2c96109640 1 --2- 172.21.15.64:0/854922128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c9010b1a0 0x7f2c9010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.731+0000 7f2c96109640 1 --2- 172.21.15.64:0/854922128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c90108830 0x7f2c9010ac60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.731+0000 7f2c96109640 1 -- 172.21.15.64:0/854922128 >> 172.21.15.64:0/854922128 conn(0x7f2c9006e6d0 msgr2=0x7f2c90070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:42.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.735+0000 7f2c96109640 1 -- 172.21.15.64:0/854922128 shutdown_connections 2024-05-09T09:14:42.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.735+0000 7f2c96109640 1 -- 172.21.15.64:0/854922128 wait complete. 2024-05-09T09:14:42.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.736+0000 7f2c96109640 1 Processor -- start 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 -- start start 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c90108830 0x7f2c90118230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c9010b1a0 0x7f2c90118770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 0x7f2c90119760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90110860 con 0x7f2c9010b1a0 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901106e0 con 0x7f2c9010db70 2024-05-09T09:14:42.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c96109640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901109e0 con 0x7f2c90108830 2024-05-09T09:14:42.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c8ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 0x7f2c90119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c8ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 0x7f2c90119760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50848/0 (socket says 172.21.15.64:50848) 2024-05-09T09:14:42.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.738+0000 7f2c8ffff640 1 -- 172.21.15.64:0/1529625701 learned_addr learned my addr 172.21.15.64:0/1529625701 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:42.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.739+0000 7f2c8effd640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c9010b1a0 0x7f2c90118770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.739+0000 7f2c8ffff640 1 -- 172.21.15.64:0/1529625701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c90108830 msgr2=0x7f2c90118230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.739+0000 7f2c8ffff640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c90108830 0x7f2c90118230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.739+0000 7f2c8ffff640 1 -- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c9010b1a0 msgr2=0x7f2c90118770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.739+0000 7f2c8ffff640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c9010b1a0 0x7f2c90118770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.739+0000 7f2c8ffff640 1 -- 172.21.15.64:0/1529625701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c90119dd0 con 0x7f2c9010db70 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.740+0000 7f2c8ffff640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 0x7f2c90119760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800ccb0 tx=0x7f2c8800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.740+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800f450 con 0x7f2c9010db70 2024-05-09T09:14:42.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.740+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9011a0c0 con 0x7f2c9010db70 2024-05-09T09:14:42.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.740+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901b7680 con 0x7f2c9010db70 2024-05-09T09:14:42.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.740+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c8800faa0 con 0x7f2c9010db70 2024-05-09T09:14:42.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.740+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88018ba0 con 0x7f2c9010db70 2024-05-09T09:14:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.743+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c0052c0 con 0x7f2c9010db70 2024-05-09T09:14:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.744+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c8800f5f0 con 0x7f2c9010db70 2024-05-09T09:14:42.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.746+0000 7f2c8cff9640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c780a2d60 0x7f2c780a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.746+0000 7f2c8f7fe640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c780a2d60 0x7f2c780a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.746+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2c880cf700 con 0x7f2c9010db70 2024-05-09T09:14:42.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.747+0000 7f2c8f7fe640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c780a2d60 0x7f2c780a5220 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005ab0 tx=0x7f2c84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:42.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:42.752+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c88014030 con 0x7f2c9010db70 2024-05-09T09:14:42.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:42 smithi064 ceph-mon[30149]: pgmap v537: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 867 op/s 2024-05-09T09:14:42.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:42 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:14:43.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:42 smithi139 ceph-mon[33599]: pgmap v537: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 867 op/s 2024-05-09T09:14:43.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:42 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:14:43.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.117+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c7c0055b0 con 0x7f2c9010db70 2024-05-09T09:14:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.118+0000 7f2c8cff9640 1 -- 172.21.15.64:0/1529625701 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11888 (secure 0 0 0) 0x7f2c8809c270 con 0x7f2c9010db70 2024-05-09T09:14:43.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:14:43.119 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:14:41.243062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:14:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 40 2024-05-09T09:14:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.121+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c780a2d60 msgr2=0x7f2c780a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.121+0000 7f2c96109640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c780a2d60 0x7f2c780a5220 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005ab0 tx=0x7f2c84005fd0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.122+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 msgr2=0x7f2c90119760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.122+0000 7f2c96109640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 0x7f2c90119760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800ccb0 tx=0x7f2c8800a620 comp rx=0 tx=0).stop 2024-05-09T09:14:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.122+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 shutdown_connections 2024-05-09T09:14:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.122+0000 7f2c96109640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2c780a2d60 0x7f2c780a5220 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.122+0000 7f2c96109640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9010db70 0x7f2c90119760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.130+0000 7f2c96109640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c9010b1a0 0x7f2c90118770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.130+0000 7f2c96109640 1 --2- 172.21.15.64:0/1529625701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c90108830 0x7f2c90118230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.130+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 >> 172.21.15.64:0/1529625701 conn(0x7f2c9006e6d0 msgr2=0x7f2c90108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.136+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 shutdown_connections 2024-05-09T09:14:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.136+0000 7f2c96109640 1 -- 172.21.15.64:0/1529625701 wait complete. 2024-05-09T09:14:43.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:14:43.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:42 smithi179 ceph-mon[34086]: pgmap v537: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 867 op/s 2024-05-09T09:14:43.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:42 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:14:43.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.328+0000 7f2254918640 1 -- 172.21.15.64:0/2638669093 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c031880 con 0x7f2250108fe0 2024-05-09T09:14:43.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 -- 172.21.15.64:0/2638669093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 msgr2=0x7f2250109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2638669093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 0x7f2250109460 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f224c009900 tx=0x7f224c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 -- 172.21.15.64:0/2638669093 shutdown_connections 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2638669093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2638669093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 0x7f2250109460 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2638669093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f22501082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.329+0000 7f2257b7e640 1 -- 172.21.15.64:0/2638669093 >> 172.21.15.64:0/2638669093 conn(0x7f2250076e30 msgr2=0x7f2250079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.330+0000 7f2257b7e640 1 -- 172.21.15.64:0/2638669093 shutdown_connections 2024-05-09T09:14:43.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.330+0000 7f2257b7e640 1 -- 172.21.15.64:0/2638669093 wait complete. 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 Processor -- start 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 -- start start 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f225019e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 0x7f225019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2250113a20 con 0x7f2250107e90 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22501138a0 con 0x7f2250108fe0 2024-05-09T09:14:43.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f2257b7e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2250113ba0 con 0x7f22501099a0 2024-05-09T09:14:43.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f225591a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f225019e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f225611b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f225611b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32798/0 (socket says 172.21.15.64:32798) 2024-05-09T09:14:43.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f225591a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f225019e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35266/0 (socket says 172.21.15.64:35266) 2024-05-09T09:14:43.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.331+0000 7f225611b640 1 -- 172.21.15.64:0/2346460772 learned_addr learned my addr 172.21.15.64:0/2346460772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.332+0000 7f225611b640 1 -- 172.21.15.64:0/2346460772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 msgr2=0x7f225019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.332+0000 7f225611b640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 0x7f225019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.332+0000 7f225611b640 1 -- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 msgr2=0x7f225019e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.332+0000 7f225611b640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f225019e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.332+0000 7f225611b640 1 -- 172.21.15.64:0/2346460772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22501a5d50 con 0x7f22501099a0 2024-05-09T09:14:43.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.332+0000 7f225591a640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f225019e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:43.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.333+0000 7f225611b640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501a56e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f224000d950 tx=0x7f224000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.333+0000 7f2246ffd640 1 -- 172.21.15.64:0/2346460772 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240017030 con 0x7f22501099a0 2024-05-09T09:14:43.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.333+0000 7f2246ffd640 1 -- 172.21.15.64:0/2346460772 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2240004020 con 0x7f22501099a0 2024-05-09T09:14:43.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.334+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22501a6040 con 0x7f22501099a0 2024-05-09T09:14:43.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.334+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501a6560 con 0x7f22501099a0 2024-05-09T09:14:43.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.334+0000 7f2246ffd640 1 -- 172.21.15.64:0/2346460772 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240003b20 con 0x7f22501099a0 2024-05-09T09:14:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.338+0000 7f2246ffd640 1 -- 172.21.15.64:0/2346460772 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f224001a070 con 0x7f22501099a0 2024-05-09T09:14:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.339+0000 7f225a96b740 1 Processor -- start 2024-05-09T09:14:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.339+0000 7f225a96b740 1 -- start start 2024-05-09T09:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f2246ffd640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f223c0a2ce0 0x7f223c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f225a96b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f225a96b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323ab930 0x55fe323b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f225a96b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f225a96b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fe3230fc40 con 0x55fe323aaf40 2024-05-09T09:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f225a96b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fe322b7580 con 0x55fe323b0aa0 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f225a96b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fe3230bc00 con 0x55fe323ab930 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.340+0000 7f2246ffd640 1 -- 172.21.15.64:0/2346460772 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f22400cec70 con 0x7f22501099a0 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.341+0000 7f2255119640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323ab930 0x55fe323b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.341+0000 7f225591a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.341+0000 7f2255119640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f223c0a2ce0 0x7f223c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.341+0000 7f225591a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35268/0 (socket says 172.21.15.64:35268) 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.341+0000 7f225591a640 1 -- 172.21.15.64:0/1479700747 learned_addr learned my addr 172.21.15.64:0/1479700747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.341+0000 7f225611b640 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225591a640 1 -- 172.21.15.64:0/1479700747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323ab930 msgr2=0x55fe323b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225591a640 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323ab930 0x55fe323b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225591a640 1 -- 172.21.15.64:0/1479700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 msgr2=0x55fe323b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225591a640 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225591a640 1 -- 172.21.15.64:0/1479700747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe320233f0 con 0x55fe323aaf40 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225611b640 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f225591a640 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323ab360 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f22480108c0 tx=0x7f2248010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5267be98ead71f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f2255119640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f223c0a2ce0 0x7f223c0a51a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f224c005730 tx=0x7f224c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.342+0000 7f223bfff640 1 -- 172.21.15.64:0/1479700747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248004d20 con 0x55fe323aaf40 2024-05-09T09:14:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.343+0000 7f223bfff640 1 -- 172.21.15.64:0/1479700747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f224800ee00 con 0x55fe323aaf40 2024-05-09T09:14:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.343+0000 7f223bfff640 1 -- 172.21.15.64:0/1479700747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248005940 con 0x55fe323aaf40 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.343+0000 7f225a96b740 1 -- 172.21.15.64:0/1479700747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 msgr2=0x55fe323ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.343+0000 7f225a96b740 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323ab360 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f22480108c0 tx=0x7f2248010d80 comp rx=0 tx=0).stop 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 -- 172.21.15.64:0/1479700747 shutdown_connections 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323ab930 0x55fe323b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 --2- 172.21.15.64:0/1479700747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323ab360 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 -- 172.21.15.64:0/1479700747 >> 172.21.15.64:0/1479700747 conn(0x55fe323a65b0 msgr2=0x55fe323a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 -- 172.21.15.64:0/1479700747 shutdown_connections 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 -- 172.21.15.64:0/1479700747 wait complete. 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.344+0000 7f225a96b740 1 -- start start 2024-05-09T09:14:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 Processor -- start 2024-05-09T09:14:43.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323aaf40 0x55fe323b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fe323b3a50 con 0x55fe323ab930 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fe322cac40 con 0x55fe323b0aa0 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fe323b3bd0 con 0x55fe323aaf40 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.345+0000 7f225a96b740 10 client.0 _ll_register_callbacks cb 0x55fe32022be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f2255119640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f2255119640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35272/0 (socket says 172.21.15.64:35272) 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f2255119640 1 -- 172.21.15.64:0/1540114755 learned_addr learned my addr 172.21.15.64:0/1540114755 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323aaf40 0x55fe323b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225611b640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 -- 172.21.15.64:0/1540114755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 msgr2=0x55fe323bc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323bc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 -- 172.21.15.64:0/1540114755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 msgr2=0x55fe323b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 -- 172.21.15.64:0/1540114755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe320ed7d0 con 0x55fe323aaf40 2024-05-09T09:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225611b640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.346+0000 7f225591a640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323aaf40 0x55fe323b5390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2248010890 tx=0x7f2248000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa9f712ce929f18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.347+0000 7f22397fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.347+0000 7f22397fa640 1 -- 172.21.15.64:0/1540114755 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248029060 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.347+0000 7f22397fa640 1 -- 172.21.15.64:0/1540114755 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2248005680 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.347+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224800ee00 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.347+0000 7f22397fa640 1 -- 172.21.15.64:0/1540114755 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2248005880 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 10 client.26605 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fe323bfbf0 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f22397fa640 1 -- 172.21.15.64:0/1540114755 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2248003e20 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f22397fa640 1 -- 172.21.15.64:0/1540114755 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f2248004030 con 0x55fe323aaf40 2024-05-09T09:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 10 client.26605 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 10 client.26605 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 20 client.26605 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe323c0ff0 0x55fe323c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 4 client.26605 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.348+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f224800ee00 con 0x55fe323c0ff0 2024-05-09T09:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.349+0000 7f2255119640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe323c0ff0 0x55fe323c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.349+0000 7f2255119640 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe323c0ff0 0x55fe323c33e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e52aa21f0f6f15c server_cookie=ee1712b9a1441c7a in_seq=0 out_seq=0 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.349+0000 7f22397fa640 10 client.26605 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.350+0000 7f22397fa640 1 -- 172.21.15.64:0/1540114755 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f224800ee00 con 0x55fe323c0ff0 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.350+0000 7f22397fa640 10 client.26605 handle_command_reply: tid=0 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.350+0000 7f225a96b740 1 client.26605 shutdown 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.351+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323aaf40 msgr2=0x55fe323b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.351+0000 7f225a96b740 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323aaf40 0x55fe323b5390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2248010890 tx=0x7f2248000f40 comp rx=0 tx=0).stop 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 shutdown_connections 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe323c0ff0 0x55fe323c33e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323b0aa0 0x55fe323bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 --2- 172.21.15.64:0/1540114755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323aaf40 0x55fe323b5390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 >> 172.21.15.64:0/1540114755 conn(0x55fe323a65b0 msgr2=0x55fe323a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f22397fa640 0 client.26605 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 shutdown_connections 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 1 -- 172.21.15.64:0/1540114755 wait complete. 2024-05-09T09:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.352+0000 7f225a96b740 20 client.26605 trim_cache size 0 max 16384 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 Processor -- start 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 -- start start 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225591a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323ab930 0x55fe323a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2234012610 con 0x55fe323aaf40 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f223400fe20 con 0x55fe323ab930 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225a96b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f223400dc70 con 0x55fe323b0aa0 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225591a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35288/0 (socket says 172.21.15.64:35288) 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.380+0000 7f225591a640 1 -- 172.21.15.64:0/544463051 learned_addr learned my addr 172.21.15.64:0/544463051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323ab930 0x55fe323a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f225611b640 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 -- 172.21.15.64:0/544463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 msgr2=0x55fe323bc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 -- 172.21.15.64:0/544463051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 msgr2=0x55fe323a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 -- 172.21.15.64:0/544463051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe323a60f0 con 0x55fe323ab930 2024-05-09T09:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f225611b640 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f2255119640 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323ab930 0x55fe323a5b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f224c02a9e0 tx=0x7f224c035000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb8390a1270d92c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.381+0000 7f223b7fe640 1 -- 172.21.15.64:0/544463051 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c035070 con 0x55fe323ab930 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.382+0000 7f223b7fe640 1 -- 172.21.15.64:0/544463051 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f224c004340 con 0x55fe323ab930 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.382+0000 7f223b7fe640 1 -- 172.21.15.64:0/544463051 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c048c90 con 0x55fe323ab930 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.382+0000 7f225a96b740 1 -- 172.21.15.64:0/544463051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323ab930 msgr2=0x55fe323a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.382+0000 7f225a96b740 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323ab930 0x55fe323a5b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f224c02a9e0 tx=0x7f224c035000 comp rx=0 tx=0).stop 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 -- 172.21.15.64:0/544463051 shutdown_connections 2024-05-09T09:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323ab930 0x55fe323a5b70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 --2- 172.21.15.64:0/544463051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323aaf40 0x55fe323a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 -- 172.21.15.64:0/544463051 >> 172.21.15.64:0/544463051 conn(0x55fe323a65b0 msgr2=0x55fe323a4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 -- 172.21.15.64:0/544463051 shutdown_connections 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.383+0000 7f225a96b740 1 -- 172.21.15.64:0/544463051 wait complete. 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.384+0000 7f225a96b740 1 -- start start 2024-05-09T09:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.384+0000 7f225a96b740 1 Processor -- start 2024-05-09T09:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.385+0000 7f225a96b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 0x55fe323b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.385+0000 7f225a96b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.385+0000 7f225591a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 0x55fe323b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.385+0000 7f225591a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 0x55fe323b9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50926/0 (socket says 172.21.15.64:50926) 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f225591a640 1 -- 172.21.15.64:0/1589279582 learned_addr learned my addr 172.21.15.64:0/1589279582 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f2255119640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f223400e2e0 con 0x55fe323ab930 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f225611b640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f223400f750 con 0x55fe323aaf40 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.386+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f223400e9b0 con 0x55fe323b0aa0 2024-05-09T09:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.387+0000 7f225a96b740 10 client.0 _ll_register_callbacks cb 0x7f224c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.387+0000 7f225591a640 1 -- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 msgr2=0x55fe323bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.387+0000 7f225591a640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.387+0000 7f225591a640 1 -- 172.21.15.64:0/1589279582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 msgr2=0x55fe323b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.387+0000 7f225591a640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.387+0000 7f225591a640 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe323a7020 con 0x55fe323aaf40 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f225591a640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 0x55fe323b9ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2248010890 tx=0x7f2248005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8242997bbe2784 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f22397fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f22397fa640 1 -- 172.21.15.64:0/1589279582 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224803c070 con 0x55fe323aaf40 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f22397fa640 1 -- 172.21.15.64:0/1589279582 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2248005140 con 0x55fe323aaf40 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224c004340 con 0x55fe323aaf40 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f22397fa640 1 -- 172.21.15.64:0/1589279582 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2248045050 con 0x55fe323aaf40 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f225a96b740 10 client.26441 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.388+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fe323bc690 con 0x55fe323aaf40 2024-05-09T09:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.389+0000 7f22397fa640 1 -- 172.21.15.64:0/1589279582 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2248009e50 con 0x55fe323aaf40 2024-05-09T09:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f22397fa640 1 -- 172.21.15.64:0/1589279582 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f224803f020 con 0x55fe323aaf40 2024-05-09T09:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f225a96b740 10 client.26441 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f225a96b740 10 client.26441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f225a96b740 20 client.26441 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe324275b0 0x55fe324299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f225a96b740 4 client.26441 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f224c004340 con 0x55fe324275b0 2024-05-09T09:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.390+0000 7f2255119640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe324275b0 0x55fe324299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f2255119640 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe324275b0 0x55fe324299d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=406a0b57627614d1 server_cookie=4c972c747f73cc56 in_seq=0 out_seq=0 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f22397fa640 10 client.26441 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f22397fa640 1 -- 172.21.15.64:0/1589279582 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f224c004340 con 0x55fe324275b0 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f22397fa640 10 client.26441 handle_command_reply: tid=0 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f225a96b740 1 client.26441 shutdown 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 msgr2=0x55fe323b9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.391+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 0x55fe323b9ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2248010890 tx=0x7f2248005c50 comp rx=0 tx=0).stop 2024-05-09T09:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.392+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 shutdown_connections 2024-05-09T09:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55fe324275b0 0x55fe324299d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe323b0aa0 0x55fe323bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe323ab930 0x55fe323b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f22397fa640 0 client.26441 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 --2- 172.21.15.64:0/1589279582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe323aaf40 0x55fe323b9ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 >> 172.21.15.64:0/1589279582 conn(0x55fe323a65b0 msgr2=0x55fe323a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 shutdown_connections 2024-05-09T09:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 1 -- 172.21.15.64:0/1589279582 wait complete. 2024-05-09T09:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.393+0000 7f225a96b740 20 client.26441 trim_cache size 0 max 16384 2024-05-09T09:14:43.394 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.395+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f223c0a2ce0 msgr2=0x7f223c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.396+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f223c0a2ce0 0x7f223c0a51a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f224c005730 tx=0x7f224c005a20 comp rx=0 tx=0).stop 2024-05-09T09:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.396+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 msgr2=0x7f22501a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.396+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501a56e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f224000d950 tx=0x7f224000de10 comp rx=0 tx=0).stop 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.396+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 shutdown_connections 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f223c0a2ce0 0x7f223c0a51a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22501099a0 0x7f22501a56e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250108fe0 0x7f225019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 --2- 172.21.15.64:0/2346460772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250107e90 0x7f225019e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 >> 172.21.15.64:0/2346460772 conn(0x7f2250076e30 msgr2=0x7f225010c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 shutdown_connections 2024-05-09T09:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.397+0000 7f2257b7e640 1 -- 172.21.15.64:0/2346460772 wait complete. 2024-05-09T09:14:43.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:14:43.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.598+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/3764604000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed41000e0 msgr2=0x7f6ed4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.598+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/3764604000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed41000e0 0x7f6ed4102510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009ca0 tx=0x7f6ec402a8d0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.598+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/3764604000 shutdown_connections 2024-05-09T09:14:43.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.598+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/3764604000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed4105400 0x7f6ed4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.598+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/3764604000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 0x7f6ed4104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.598+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/3764604000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed41000e0 0x7f6ed4102510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.599+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/3764604000 >> 172.21.15.64:0/3764604000 conn(0x7f6ed40fbd20 msgr2=0x7f6ed40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.599+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/3764604000 shutdown_connections 2024-05-09T09:14:43.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.599+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/3764604000 wait complete. 2024-05-09T09:14:43.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.600+0000 7f6ed9a7d640 1 Processor -- start 2024-05-09T09:14:43.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 -- start start 2024-05-09T09:14:43.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 0x7f6ed419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 0x7f6ed419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed4105400 0x7f6ed41a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4113970 con 0x7f6ed4102a50 2024-05-09T09:14:43.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41137f0 con 0x7f6ed41000e0 2024-05-09T09:14:43.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed9a7d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4113af0 con 0x7f6ed4105400 2024-05-09T09:14:43.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 0x7f6ed419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 0x7f6ed419e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50938/0 (socket says 172.21.15.64:50938) 2024-05-09T09:14:43.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed37fe640 1 -- 172.21.15.64:0/96749128 learned_addr learned my addr 172.21.15.64:0/96749128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 0x7f6ed419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.601+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed4105400 0x7f6ed41a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed37fe640 1 -- 172.21.15.64:0/96749128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed4105400 msgr2=0x7f6ed41a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed4105400 0x7f6ed41a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed37fe640 1 -- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 msgr2=0x7f6ed419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 0x7f6ed419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed37fe640 1 -- 172.21.15.64:0/96749128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed41a5cc0 con 0x7f6ed41000e0 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 0x7f6ed419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:43.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.602+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 0x7f6ed419e0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4031940 tx=0x7f6ec4031d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.603+0000 7f6ed0ff9640 1 -- 172.21.15.64:0/96749128 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4030e20 con 0x7f6ed41000e0 2024-05-09T09:14:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.603+0000 7f6ed0ff9640 1 -- 172.21.15.64:0/96749128 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ec4046930 con 0x7f6ed41000e0 2024-05-09T09:14:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.603+0000 7f6ed0ff9640 1 -- 172.21.15.64:0/96749128 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec403b9d0 con 0x7f6ed41000e0 2024-05-09T09:14:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.603+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed41a5f50 con 0x7f6ed41000e0 2024-05-09T09:14:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.603+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed41a6360 con 0x7f6ed41000e0 2024-05-09T09:14:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.606+0000 7f6edc78d740 1 Processor -- start 2024-05-09T09:14:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.606+0000 7f6ed0ff9640 1 -- 172.21.15.64:0/96749128 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ec403bb70 con 0x7f6ed41000e0 2024-05-09T09:14:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 -- start start 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c5930 0x563cf81ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf81ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563cf8129c40 con 0x563cf81caaa0 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563cf80d1580 con 0x563cf81c5930 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6edc78d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563cf8125c00 con 0x563cf81c4f40 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32868/0 (socket says 172.21.15.64:32868) 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2853071237 learned_addr learned my addr 172.21.15.64:0/2853071237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf81ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.607+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c5930 0x563cf81ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2853071237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c5930 msgr2=0x563cf81ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c5930 0x563cf81ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2853071237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 msgr2=0x563cf81ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf81ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2853071237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf7e3d3f0 con 0x563cf81c4f40 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c5930 0x563cf81ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81c5360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00ab80 tx=0x7f6ecc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae18352fb6396dff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6eb9ffb640 1 -- 172.21.15.64:0/2853071237 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc01d070 con 0x563cf81c4f40 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.608+0000 7f6eb9ffb640 1 -- 172.21.15.64:0/2853071237 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ecc018af0 con 0x563cf81c4f40 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.609+0000 7f6ed0ff9640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.609+0000 7f6ed0ff9640 1 -- 172.21.15.64:0/96749128 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ec40f07e0 con 0x7f6ed41000e0 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.609+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.609+0000 7f6eb9ffb640 1 -- 172.21.15.64:0/2853071237 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc010a40 con 0x563cf81c4f40 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.609+0000 7f6edc78d740 1 -- 172.21.15.64:0/2853071237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 msgr2=0x563cf81c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.609+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81c5360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00ab80 tx=0x7f6ecc00f790 comp rx=0 tx=0).stop 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 -- 172.21.15.64:0/2853071237 shutdown_connections 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf81ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c5930 0x563cf81ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2853071237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81c5360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 -- 172.21.15.64:0/2853071237 >> 172.21.15.64:0/2853071237 conn(0x563cf81c05b0 msgr2=0x563cf81c09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6ed37fe640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40b90b0 tx=0x7f6ec40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 -- 172.21.15.64:0/2853071237 shutdown_connections 2024-05-09T09:14:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 -- 172.21.15.64:0/2853071237 wait complete. 2024-05-09T09:14:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.610+0000 7f6edc78d740 1 -- start start 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 Processor -- start 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81caaa0 0x563cf81d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563cf81cdaf0 con 0x563cf81c5930 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563cf80e4c40 con 0x563cf81caaa0 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563cf81cdc70 con 0x563cf81c4f40 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6edc78d740 10 client.0 _ll_register_callbacks cb 0x563cf7e3cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6ed2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6ed2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35340/0 (socket says 172.21.15.64:35340) 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/2028700255 learned_addr learned my addr 172.21.15.64:0/2028700255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.611+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81caaa0 0x563cf81d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2028700255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81caaa0 msgr2=0x563cf81d6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81caaa0 0x563cf81d6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2028700255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 msgr2=0x563cf81cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed3fff640 1 -- 172.21.15.64:0/2028700255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf7f077d0 con 0x563cf81c4f40 2024-05-09T09:14:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.612+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.613+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81cf390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005fd0 tx=0x7f6ecc00af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c53fe6f0b7a92dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.613+0000 7f6eb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.613+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/2028700255 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc02d060 con 0x563cf81c4f40 2024-05-09T09:14:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.614+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ecc018af0 con 0x563cf81c4f40 2024-05-09T09:14:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.614+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/2028700255 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ecc005470 con 0x563cf81c4f40 2024-05-09T09:14:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.615+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/2028700255 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6ecc034050 con 0x563cf81c4f40 2024-05-09T09:14:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.615+0000 7f6edc78d740 10 client.26620 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.615+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563cf81d9c80 con 0x563cf81c4f40 2024-05-09T09:14:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/2028700255 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ecc018af0 con 0x563cf81c4f40 2024-05-09T09:14:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/2028700255 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f6ecc004750 con 0x563cf81c4f40 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6edc78d740 10 client.26620 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6edc78d740 10 client.26620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6edc78d740 20 client.26620 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf81ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6edc78d740 4 client.26620 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.616+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cf8137370 con 0x563cf81db700 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.617+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf81ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.617+0000 7f6eb2ffd640 10 client.26620 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.618+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf81ddb40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=721008dfad2eac59 server_cookie=52562859638ed16d in_seq=0 out_seq=0 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.619+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/2028700255 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563cf8137370 con 0x563cf81db700 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.619+0000 7f6eb2ffd640 10 client.26620 handle_command_reply: tid=0 2024-05-09T09:14:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.619+0000 7f6edc78d740 1 client.26620 shutdown 2024-05-09T09:14:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.619+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 msgr2=0x563cf81cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.619+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81cf390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005fd0 tx=0x7f6ecc00af30 comp rx=0 tx=0).stop 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.619+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 shutdown_connections 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81caaa0 0x563cf81d6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf81ddb40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6eb2ffd640 0 client.26620 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2028700255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c4f40 0x563cf81cf390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 >> 172.21.15.64:0/2028700255 conn(0x563cf81c05b0 msgr2=0x563cf81bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 shutdown_connections 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 1 -- 172.21.15.64:0/2028700255 wait complete. 2024-05-09T09:14:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.620+0000 7f6edc78d740 20 client.26620 trim_cache size 0 max 16384 2024-05-09T09:14:43.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.648+0000 7f6edc78d740 1 Processor -- start 2024-05-09T09:14:43.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.648+0000 7f6edc78d740 1 -- start start 2024-05-09T09:14:43.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81caaa0 0x563cf81c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6edc78d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41b33a0 con 0x563cf81c5930 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6edc78d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41b0bb0 con 0x563cf81c4f40 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6edc78d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41aea00 con 0x563cf81caaa0 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81d68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35352/0 (socket says 172.21.15.64:35352) 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/2802918050 learned_addr learned my addr 172.21.15.64:0/2802918050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81caaa0 0x563cf81c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/2802918050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81caaa0 msgr2=0x563cf81c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81caaa0 0x563cf81c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/2802918050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 msgr2=0x563cf81cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.649+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.650+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/2802918050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf81b1d20 con 0x563cf81c5930 2024-05-09T09:14:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.650+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81caaa0 0x563cf81c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.650+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81d68d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8005d00 tx=0x7f6ec8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5d023317051536 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.650+0000 7f6eb97fa640 1 -- 172.21.15.64:0/2802918050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800fc50 con 0x563cf81c5930 2024-05-09T09:14:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.651+0000 7f6eb97fa640 1 -- 172.21.15.64:0/2802918050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ec8026030 con 0x563cf81c5930 2024-05-09T09:14:43.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.651+0000 7f6eb97fa640 1 -- 172.21.15.64:0/2802918050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8004b20 con 0x563cf81c5930 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.652+0000 7f6edc78d740 1 -- 172.21.15.64:0/2802918050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 msgr2=0x563cf81d68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.652+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81d68d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8005d00 tx=0x7f6ec8005f90 comp rx=0 tx=0).stop 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.652+0000 7f6edc78d740 1 -- 172.21.15.64:0/2802918050 shutdown_connections 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.653+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81caaa0 0x563cf81c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.653+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81c5930 0x563cf81d68d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.653+0000 7f6edc78d740 1 --2- 172.21.15.64:0/2802918050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.653+0000 7f6edc78d740 1 -- 172.21.15.64:0/2802918050 >> 172.21.15.64:0/2802918050 conn(0x563cf81c05b0 msgr2=0x563cf81cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.653+0000 7f6edc78d740 1 -- 172.21.15.64:0/2802918050 shutdown_connections 2024-05-09T09:14:43.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.653+0000 7f6edc78d740 1 -- 172.21.15.64:0/2802918050 wait complete. 2024-05-09T09:14:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.654+0000 7f6edc78d740 1 -- start start 2024-05-09T09:14:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.654+0000 7f6edc78d740 1 Processor -- start 2024-05-09T09:14:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 0x563cf81d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf823fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6ed2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 0x563cf81d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6ed2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 0x563cf81d3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32902/0 (socket says 172.21.15.64:32902) 2024-05-09T09:14:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/1504250094 learned_addr learned my addr 172.21.15.64:0/1504250094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41af070 con 0x563cf81caaa0 2024-05-09T09:14:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41b04e0 con 0x563cf81c4f40 2024-05-09T09:14:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41af740 con 0x563cf81c5930 2024-05-09T09:14:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.655+0000 7f6edc78d740 10 client.0 _ll_register_callbacks cb 0x7f6ecc01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:43.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/1504250094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 msgr2=0x563cf81d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/1504250094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 msgr2=0x563cf823fcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf823fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf823fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed2ffd640 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf82401f0 con 0x563cf81c5930 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf823fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.656+0000 7f6ed2ffd640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 0x563cf81d3c50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8014010 tx=0x7f6ec8005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee1ee4e3a4125aa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6eb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/1504250094 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80220f0 con 0x563cf81c5930 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/1504250094 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ec8022290 con 0x563cf81c5930 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec8026030 con 0x563cf81c5930 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/1504250094 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6ec803d050 con 0x563cf81c5930 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6edc78d740 10 client.26623 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.657+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563cf8241330 con 0x563cf81c5930 2024-05-09T09:14:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/1504250094 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ec802f070 con 0x563cf81c5930 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/1504250094 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f6ec8034a20 con 0x563cf81c5930 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6edc78d740 10 client.26623 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6edc78d740 10 client.26623 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6edc78d740 20 client.26623 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6edc78d740 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf8243c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6edc78d740 4 client.26623 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.658+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6ec8026030 con 0x563cf81db700 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.659+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf8243c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.659+0000 7f6ed3fff640 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf8243c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0e131043d30eff6 server_cookie=15c4f43113e40125 in_seq=0 out_seq=0 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.659+0000 7f6eb2ffd640 10 client.26623 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.660+0000 7f6eb2ffd640 1 -- 172.21.15.64:0/1504250094 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6ec8026030 con 0x563cf81db700 2024-05-09T09:14:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.660+0000 7f6eb2ffd640 10 client.26623 handle_command_reply: tid=0 2024-05-09T09:14:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.660+0000 7f6edc78d740 1 client.26623 shutdown 2024-05-09T09:14:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.660+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 msgr2=0x563cf81d3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.660+0000 7f6edc78d740 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 0x563cf81d3c50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8014010 tx=0x7f6ec8005670 comp rx=0 tx=0).stop 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 shutdown_connections 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563cf81db700 0x563cf8243c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563cf81caaa0 0x563cf823fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563cf81c5930 0x563cf81d3c50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 --2- 172.21.15.64:0/1504250094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563cf81c4f40 0x563cf81d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 >> 172.21.15.64:0/1504250094 conn(0x563cf81c05b0 msgr2=0x563cf81bef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6eb2ffd640 0 client.26623 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 shutdown_connections 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 1 -- 172.21.15.64:0/1504250094 wait complete. 2024-05-09T09:14:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.661+0000 7f6edc78d740 20 client.26623 trim_cache size 0 max 16384 2024-05-09T09:14:43.663 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:14:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ebc0a2c10 msgr2=0x7f6ebc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40b90b0 tx=0x7f6ec40095c0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 msgr2=0x7f6ed419e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 0x7f6ed419e0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4031940 tx=0x7f6ec4031d40 comp rx=0 tx=0).stop 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 shutdown_connections 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ed4105400 0x7f6ed41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ed4102a50 0x7f6ed419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 --2- 172.21.15.64:0/96749128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed41000e0 0x7f6ed419e0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 >> 172.21.15.64:0/96749128 conn(0x7f6ed40fbd20 msgr2=0x7f6ed4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 shutdown_connections 2024-05-09T09:14:43.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.662+0000 7f6ed9a7d640 1 -- 172.21.15.64:0/96749128 wait complete. 2024-05-09T09:14:43.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.849+0000 7f33d09a9640 1 -- 172.21.15.64:0/4234041318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 msgr2=0x7f33cc101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.849+0000 7f33d09a9640 1 --2- 172.21.15.64:0/4234041318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc101ca0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009900 tx=0x7f33bc02a410 comp rx=0 tx=0).stop 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.849+0000 7f33d09a9640 1 -- 172.21.15.64:0/4234041318 shutdown_connections 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.849+0000 7f33d09a9640 1 --2- 172.21.15.64:0/4234041318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 0x7f33cc109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.849+0000 7f33d09a9640 1 --2- 172.21.15.64:0/4234041318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc102a70 0x7f33cc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.849+0000 7f33d09a9640 1 --2- 172.21.15.64:0/4234041318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc101ca0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.850+0000 7f33d09a9640 1 -- 172.21.15.64:0/4234041318 >> 172.21.15.64:0/4234041318 conn(0x7f33cc0fce10 msgr2=0x7f33cc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.850+0000 7f33d09a9640 1 -- 172.21.15.64:0/4234041318 shutdown_connections 2024-05-09T09:14:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.850+0000 7f33d09a9640 1 -- 172.21.15.64:0/4234041318 wait complete. 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 Processor -- start 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 -- start start 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc102a70 0x7f33cc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 0x7f33cc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc10d3d0 con 0x7f33cc101880 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35380/0 (socket says 172.21.15.64:35380) 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33ca59c640 1 -- 172.21.15.64:0/1034095342 learned_addr learned my addr 172.21.15.64:0/1034095342 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc10d250 con 0x7f33cc103430 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.851+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc10d550 con 0x7f33cc102a70 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 0x7f33cc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc102a70 0x7f33cc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 -- 172.21.15.64:0/1034095342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc102a70 msgr2=0x7f33cc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc102a70 0x7f33cc198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 -- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 msgr2=0x7f33cc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 -- 172.21.15.64:0/1034095342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc19f760 con 0x7f33cc103430 2024-05-09T09:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 0x7f33cc19f0f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f33c400ec50 tx=0x7f33c400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33bb7fe640 1 -- 172.21.15.64:0/1034095342 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4003b70 con 0x7f33cc103430 2024-05-09T09:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33bb7fe640 1 -- 172.21.15.64:0/1034095342 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c4003d10 con 0x7f33cc103430 2024-05-09T09:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.852+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc19fa50 con 0x7f33cc103430 2024-05-09T09:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.853+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc19ff50 con 0x7f33cc103430 2024-05-09T09:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.853+0000 7f33bb7fe640 1 -- 172.21.15.64:0/1034095342 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4010760 con 0x7f33cc103430 2024-05-09T09:14:43.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.856+0000 7f33bb7fe640 1 -- 172.21.15.64:0/1034095342 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33c4010920 con 0x7f33cc103430 2024-05-09T09:14:43.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.857+0000 7f33bb7fe640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33b40a2b90 0x7f33b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.857+0000 7f33d3797740 1 Processor -- start 2024-05-09T09:14:43.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.857+0000 7f33bb7fe640 1 -- 172.21.15.64:0/1034095342 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33c40cf970 con 0x7f33cc103430 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.857+0000 7f33ca59c640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33b40a2b90 0x7f33b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 -- start start 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32928/0 (socket says 172.21.15.64:32928) 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33c9d9b640 1 -- 172.21.15.64:0/1954631061 learned_addr learned my addr 172.21.15.64:0/1954631061 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556cd8715c40 con 0x556cd87b0f40 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556cd86bd580 con 0x556cd87b6aa0 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556cd8711c00 con 0x556cd87b1930 2024-05-09T09:14:43.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.858+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33ca59c640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33b40a2b90 0x7f33b40a5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f33bc000c00 tx=0x7f33bc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33ca59c640 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33cad9d640 1 -- 172.21.15.64:0/1954631061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 msgr2=0x556cd87b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33cad9d640 1 -- 172.21.15.64:0/1954631061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 msgr2=0x556cd87b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33cad9d640 1 -- 172.21.15.64:0/1954631061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd84293f0 con 0x556cd87b6aa0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33ca59c640 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.859+0000 7f33cad9d640 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87b8f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33c40cd0e0 tx=0x7f33c400ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbcf5f9207dcb7c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.860+0000 7f33b8ff9640 1 -- 172.21.15.64:0/1954631061 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c40ba070 con 0x556cd87b6aa0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.860+0000 7f33b8ff9640 1 -- 172.21.15.64:0/1954631061 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c409c540 con 0x556cd87b6aa0 2024-05-09T09:14:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.860+0000 7f33b8ff9640 1 -- 172.21.15.64:0/1954631061 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c409e030 con 0x556cd87b6aa0 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.861+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 msgr2=0x556cd87b8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.861+0000 7f33d3797740 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87b8f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33c40cd0e0 tx=0x7f33c400ce70 comp rx=0 tx=0).stop 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.861+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 shutdown_connections 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.862+0000 7f33d3797740 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87b8f30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.862+0000 7f33d3797740 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.862+0000 7f33d3797740 1 --2- 172.21.15.64:0/1954631061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.862+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 >> 172.21.15.64:0/1954631061 conn(0x556cd87ac5b0 msgr2=0x556cd87ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.862+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 shutdown_connections 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.862+0000 7f33d3797740 1 -- 172.21.15.64:0/1954631061 wait complete. 2024-05-09T09:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.863+0000 7f33d3797740 1 -- start start 2024-05-09T09:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.863+0000 7f33d3797740 1 Processor -- start 2024-05-09T09:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.864+0000 7f33d3797740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.864+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.864+0000 7f33d3797740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.864+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35392/0 (socket says 172.21.15.64:35392) 2024-05-09T09:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33d3797740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33ca59c640 1 -- 172.21.15.64:0/87725010 learned_addr learned my addr 172.21.15.64:0/87725010 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33d3797740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556cd87b9a50 con 0x556cd87b0f40 2024-05-09T09:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556cd86d0c40 con 0x556cd87b6aa0 2024-05-09T09:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556cd87b9bd0 con 0x556cd87b1930 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33cad9d640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33c9d9b640 1 -- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 msgr2=0x556cd87c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33c9d9b640 1 -- 172.21.15.64:0/87725010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 msgr2=0x556cd87bb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33c9d9b640 1 -- 172.21.15.64:0/87725010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd84f37d0 con 0x556cd87b1930 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33d3797740 10 client.0 _ll_register_callbacks cb 0x556cd8428be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.865+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c409c540 con 0x556cd87b1930 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33ca59c640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87bb6f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x556cd87b1d30 tx=0x7f33c000bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0f753ebb7a9267 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33b1ffb640 10 client.26635 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33b1ffb640 1 -- 172.21.15.64:0/87725010 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0003cd0 con 0x556cd87b1930 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33b1ffb640 1 -- 172.21.15.64:0/87725010 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33c0011440 con 0x556cd87b1930 2024-05-09T09:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33b1ffb640 1 -- 172.21.15.64:0/87725010 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f33c0011640 con 0x556cd87b1930 2024-05-09T09:14:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33d3797740 10 client.26635 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.866+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=40}) -- 0x556cd87c5a40 con 0x556cd87b1930 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33b1ffb640 1 -- 172.21.15.64:0/87725010 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f33c00117e0 con 0x556cd87b1930 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33d3797740 10 client.26635 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33d3797740 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33d3797740 20 client.26635 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33d3797740 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd87c7070 0x556cd87c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33d3797740 4 client.26635 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cd8723370 con 0x556cd87c7070 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.867+0000 7f33ca59c640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd87c7070 0x556cd87c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.868+0000 7f33ca59c640 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd87c7070 0x556cd87c94e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf2f1d92ffea1e8 server_cookie=649a9b01b204425f in_seq=0 out_seq=0 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.868+0000 7f33b1ffb640 10 client.26635 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33b1ffb640 1 -- 172.21.15.64:0/87725010 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cd8723370 con 0x556cd87c7070 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33b1ffb640 10 client.26635 handle_command_reply: tid=0 2024-05-09T09:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 client.26635 shutdown 2024-05-09T09:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 msgr2=0x556cd87bb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87bb6f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x556cd87b1d30 tx=0x7f33c000bf30 comp rx=0 tx=0).stop 2024-05-09T09:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 shutdown_connections 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd87c7070 0x556cd87c94e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33b1ffb640 0 client.26635 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87bb6f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 --2- 172.21.15.64:0/87725010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 >> 172.21.15.64:0/87725010 conn(0x556cd87ac5b0 msgr2=0x556cd87ab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 shutdown_connections 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 1 -- 172.21.15.64:0/87725010 wait complete. 2024-05-09T09:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.869+0000 7f33d3797740 20 client.26635 trim_cache size 0 max 16384 2024-05-09T09:14:43.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.895+0000 7f33d3797740 1 Processor -- start 2024-05-09T09:14:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 -- start start 2024-05-09T09:14:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b0f40 0x556cd87af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 0x556cd87c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ace20 con 0x556cd87b1930 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1aa630 con 0x556cd87b6aa0 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33d3797740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1a8480 con 0x556cd87b0f40 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 0x556cd87c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 0x556cd87c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35394/0 (socket says 172.21.15.64:35394) 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 -- 172.21.15.64:0/3484814753 learned_addr learned my addr 172.21.15.64:0/3484814753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33ca59c640 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b0f40 0x556cd87af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33cad9d640 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 -- 172.21.15.64:0/3484814753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b0f40 msgr2=0x556cd87af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b0f40 0x556cd87af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 -- 172.21.15.64:0/3484814753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 msgr2=0x556cd87afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33c9d9b640 1 -- 172.21.15.64:0/3484814753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd87c9b60 con 0x556cd87b1930 2024-05-09T09:14:43.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.896+0000 7f33cad9d640 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.897+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 0x556cd87c26f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f33c000b5d0 tx=0x7f33c0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7ea1ebf377442f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.897+0000 7f33b3fff640 1 -- 172.21.15.64:0/3484814753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0003cd0 con 0x556cd87b1930 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.897+0000 7f33b3fff640 1 -- 172.21.15.64:0/3484814753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c409c540 con 0x556cd87b1930 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.897+0000 7f33b3fff640 1 -- 172.21.15.64:0/3484814753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c001b420 con 0x556cd87b1930 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 -- 172.21.15.64:0/3484814753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 msgr2=0x556cd87c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 0x556cd87c26f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f33c000b5d0 tx=0x7f33c0009000 comp rx=0 tx=0).stop 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 -- 172.21.15.64:0/3484814753 shutdown_connections 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b1930 0x556cd87c26f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 --2- 172.21.15.64:0/3484814753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b0f40 0x556cd87af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 -- 172.21.15.64:0/3484814753 >> 172.21.15.64:0/3484814753 conn(0x556cd87ac5b0 msgr2=0x556cd87bb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 -- 172.21.15.64:0/3484814753 shutdown_connections 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 -- 172.21.15.64:0/3484814753 wait complete. 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.898+0000 7f33d3797740 1 -- start start 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 Processor -- start 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1a8af0 con 0x556cd87b0f40 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1a9f60 con 0x556cd87b6aa0 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1a91c0 con 0x556cd87b1930 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33d3797740 10 client.0 _ll_register_callbacks cb 0x7f33c0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35410/0 (socket says 172.21.15.64:35410) 2024-05-09T09:14:43.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 -- 172.21.15.64:0/2069646328 learned_addr learned my addr 172.21.15.64:0/2069646328 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33cad9d640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 -- 172.21.15.64:0/2069646328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 msgr2=0x556cd87c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 -- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 msgr2=0x556cd87c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.899+0000 7f33ca59c640 1 -- 172.21.15.64:0/2069646328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd87c1af0 con 0x556cd87b0f40 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33ca59c640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bffb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005ab0 tx=0x7f33bc03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b61498152bd33bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33b1ffb640 1 -- 172.21.15.64:0/2069646328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc057060 con 0x556cd87b0f40 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c409c540 con 0x556cd87b0f40 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33b1ffb640 1 -- 172.21.15.64:0/2069646328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33bc003d10 con 0x556cd87b0f40 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33b1ffb640 1 -- 172.21.15.64:0/2069646328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f33bc05e050 con 0x556cd87b0f40 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33d3797740 10 client.16980 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:43.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.900+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556cd8836060 con 0x556cd87b0f40 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33b1ffb640 1 -- 172.21.15.64:0/2069646328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c409c540 con 0x556cd87b0f40 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33b1ffb640 1 -- 172.21.15.64:0/2069646328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f33bc031ab0 con 0x556cd87b0f40 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33d3797740 10 client.16980 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33d3797740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33d3797740 20 client.16980 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33d3797740 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd8836a40 0x556cd8838e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33d3797740 4 client.16980 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cd871d6c0 con 0x556cd8836a40 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.901+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd8836a40 0x556cd8838e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33c9d9b640 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd8836a40 0x556cd8838e60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa8970c1a0a904e server_cookie=8314678754e25405 in_seq=0 out_seq=0 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33b1ffb640 10 client.16980 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33b1ffb640 1 -- 172.21.15.64:0/2069646328 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cd871d6c0 con 0x556cd8836a40 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33b1ffb640 10 client.16980 handle_command_reply: tid=0 2024-05-09T09:14:43.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 client.16980 shutdown 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 msgr2=0x556cd87bffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bffb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005ab0 tx=0x7f33bc03e070 comp rx=0 tx=0).stop 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 shutdown_connections 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556cd8836a40 0x556cd8838e60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556cd87b6aa0 0x556cd87c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33b1ffb640 0 client.16980 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556cd87b1930 0x556cd87c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 --2- 172.21.15.64:0/2069646328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556cd87b0f40 0x556cd87bffb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 >> 172.21.15.64:0/2069646328 conn(0x556cd87ac5b0 msgr2=0x556cd87bb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 shutdown_connections 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.902+0000 7f33d3797740 1 -- 172.21.15.64:0/2069646328 wait complete. 2024-05-09T09:14:43.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d3797740 20 client.16980 trim_cache size 0 max 16384 2024-05-09T09:14:43.908 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33b40a2b90 msgr2=0x7f33b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33b40a2b90 0x7f33b40a5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f33bc000c00 tx=0x7f33bc005a20 comp rx=0 tx=0).stop 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 msgr2=0x7f33cc19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 0x7f33cc19f0f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f33c400ec50 tx=0x7f33c400c6a0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 shutdown_connections 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f33b40a2b90 0x7f33b40a5050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33cc103430 0x7f33cc19f0f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc102a70 0x7f33cc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 --2- 172.21.15.64:0/1034095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33cc101880 0x7f33cc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 >> 172.21.15.64:0/1034095342 conn(0x7f33cc0fce10 msgr2=0x7f33cc105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.903+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 shutdown_connections 2024-05-09T09:14:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:43.904+0000 7f33d09a9640 1 -- 172.21.15.64:0/1034095342 wait complete. 2024-05-09T09:14:44.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:43 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1529625701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:14:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.106+0000 7f1c41ffb640 1 -- 172.21.15.64:0/761550518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34031880 con 0x7f1c44072b60 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 -- 172.21.15.64:0/761550518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 msgr2=0x7f1c44072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 --2- 172.21.15.64:0/761550518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c44072f80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009900 tx=0x7f1c3402a3f0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 -- 172.21.15.64:0/761550518 shutdown_connections 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 --2- 172.21.15.64:0/761550518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 0x7f1c4410a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 --2- 172.21.15.64:0/761550518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 0x7f1c441092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 --2- 172.21.15.64:0/761550518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c44072f80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.107+0000 7f1c43fff640 1 -- 172.21.15.64:0/761550518 >> 172.21.15.64:0/761550518 conn(0x7f1c4406e6d0 msgr2=0x7f1c44070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.108+0000 7f1c43fff640 1 -- 172.21.15.64:0/761550518 shutdown_connections 2024-05-09T09:14:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.108+0000 7f1c43fff640 1 -- 172.21.15.64:0/761550518 wait complete. 2024-05-09T09:14:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 Processor -- start 2024-05-09T09:14:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 -- start start 2024-05-09T09:14:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c4410f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 0x7f1c4410fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 0x7f1c4410c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44117cc0 con 0x7f1c44072b60 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44117b40 con 0x7f1c44108e40 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.109+0000 7f1c43fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44117e40 con 0x7f1c44109800 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c4410f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c4410f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35422/0 (socket says 172.21.15.64:35422) 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c42ffd640 1 -- 172.21.15.64:0/3743402959 learned_addr learned my addr 172.21.15.64:0/3743402959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c437fe640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 0x7f1c4410c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c437fe640 1 -- 172.21.15.64:0/3743402959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 msgr2=0x7f1c4410fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 0x7f1c4410fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c437fe640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 0x7f1c4410fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c437fe640 1 -- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 msgr2=0x7f1c4410f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c437fe640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c4410f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c437fe640 1 -- 172.21.15.64:0/3743402959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4410c8f0 con 0x7f1c44109800 2024-05-09T09:14:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 0x7f1c4410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.110+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c4410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.111+0000 7f1c437fe640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 0x7f1c4410c030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800d680 tx=0x7f1c3800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.111+0000 7f1c2bfff640 1 -- 172.21.15.64:0/3743402959 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38017070 con 0x7f1c44109800 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.111+0000 7f1c2bfff640 1 -- 172.21.15.64:0/3743402959 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c38009820 con 0x7f1c44109800 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.111+0000 7f1c2bfff640 1 -- 172.21.15.64:0/3743402959 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38004cf0 con 0x7f1c44109800 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.111+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c441ae7f0 con 0x7f1c44109800 2024-05-09T09:14:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.111+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c441aec80 con 0x7f1c44109800 2024-05-09T09:14:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.116+0000 7f1c2bfff640 1 -- 172.21.15.64:0/3743402959 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c380099d0 con 0x7f1c44109800 2024-05-09T09:14:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 Processor -- start 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 -- start start 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbb3d0 0x563395bbb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bc08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 0x563395bc33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563395b1f340 con 0x563395bbbdc0 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563395ac7290 con 0x563395bc0f30 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.117+0000 7f1c4c431740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563395b1b300 con 0x563395bbb3d0 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 0x563395bc33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 0x563395bc33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51022/0 (socket says 172.21.15.64:51022) 2024-05-09T09:14:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 -- 172.21.15.64:0/3220499475 learned_addr learned my addr 172.21.15.64:0/3220499475 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bc08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbb3d0 0x563395bbb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 -- 172.21.15.64:0/3220499475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbb3d0 msgr2=0x563395bbb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbb3d0 0x563395bbb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 -- 172.21.15.64:0/3220499475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 msgr2=0x563395bc08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bc08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.118+0000 7f1c427fc640 1 -- 172.21.15.64:0/3220499475 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563395aeb8e0 con 0x563395bc0f30 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.119+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 0x563395bc33c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c014bc0 tx=0x7f1c3c00a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64330af115867bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.119+0000 7f1c297fa640 1 -- 172.21.15.64:0/3220499475 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00dd00 con 0x563395bc0f30 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.119+0000 7f1c297fa640 1 -- 172.21.15.64:0/3220499475 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c0118b0 con 0x563395bc0f30 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.119+0000 7f1c297fa640 1 -- 172.21.15.64:0/3220499475 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c0106e0 con 0x563395bc0f30 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.119+0000 7f1c2bfff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c2c0a2cc0 0x7f1c2c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.120+0000 7f1c2bfff640 1 -- 172.21.15.64:0/3743402959 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c380ced10 con 0x7f1c44109800 2024-05-09T09:14:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.120+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c2c0a2cc0 0x7f1c2c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.120+0000 7f1c4c431740 1 -- 172.21.15.64:0/3220499475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 msgr2=0x563395bc33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.120+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 0x563395bc33c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c014bc0 tx=0x7f1c3c00a6d0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.120+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c2c0a2cc0 0x7f1c2c0a5180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x563395bbc830 tx=0x7f1c34009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.120+0000 7f1c4c431740 1 -- 172.21.15.64:0/3220499475 shutdown_connections 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bc0f30 0x563395bc33c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bc08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3220499475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbb3d0 0x563395bbb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 -- 172.21.15.64:0/3220499475 >> 172.21.15.64:0/3220499475 conn(0x563395bb6a40 msgr2=0x563395bb6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 -- 172.21.15.64:0/3220499475 shutdown_connections 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 -- 172.21.15.64:0/3220499475 wait complete. 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 -- start start 2024-05-09T09:14:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.121+0000 7f1c4c431740 1 Processor -- start 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.122+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.122+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bc5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.122+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 0x563395bccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.122+0000 7f1c4c431740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563395bc3ee0 con 0x563395bc0f30 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 0x563395bccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 0x563395bccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35428/0 (socket says 172.21.15.64:35428) 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 -- 172.21.15.64:0/4037821450 learned_addr learned my addr 172.21.15.64:0/4037821450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563395ada790 con 0x563395bbb3d0 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563395bc4060 con 0x563395bbbdc0 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c427fc640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bc5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c4c431740 10 client.0 _ll_register_callbacks cb 0x563395b15a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 -- 172.21.15.64:0/4037821450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 msgr2=0x563395bc5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bc5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 -- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 msgr2=0x563395bc5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bc5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c437fe640 1 -- 172.21.15.64:0/4037821450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563395833210 con 0x563395bc0f30 2024-05-09T09:14:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.123+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c437fe640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 0x563395bccbf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009610 tx=0x7f1c38009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51d1ce120343298 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:14:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c26ff5640 1 -- 172.21.15.64:0/4037821450 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38020070 con 0x563395bc0f30 2024-05-09T09:14:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c26ff5640 1 -- 172.21.15.64:0/4037821450 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c38005220 con 0x563395bc0f30 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c3c0118b0 con 0x563395bc0f30 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c26ff5640 1 -- 172.21.15.64:0/4037821450 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1c3809f530 con 0x563395bc0f30 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c4c431740 10 client.16992 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.124+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5633958ff620 con 0x563395bc0f30 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c26ff5640 1 -- 172.21.15.64:0/4037821450 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c380a3df0 con 0x563395bc0f30 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c26ff5640 1 -- 172.21.15.64:0/4037821450 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f1c3809f160 con 0x563395bc0f30 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c4c431740 10 client.16992 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c4c431740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c4c431740 20 client.16992 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c4c431740 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bd0f60 0x563395bd33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c4c431740 4 client.16992 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.125+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1c3c0118b0 con 0x563395bd0f60 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.126+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bd0f60 0x563395bd33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.126+0000 7f1c26ff5640 10 client.16992 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.126+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bd0f60 0x563395bd33e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b27d5bc7c00f84 server_cookie=28e9b83b6cd660f7 in_seq=0 out_seq=0 2024-05-09T09:14:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.127+0000 7f1c26ff5640 1 -- 172.21.15.64:0/4037821450 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f1c3c0118b0 con 0x563395bd0f60 2024-05-09T09:14:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.127+0000 7f1c26ff5640 10 client.16992 handle_command_reply: tid=0 2024-05-09T09:14:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.127+0000 7f1c4c431740 1 client.16992 shutdown 2024-05-09T09:14:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 msgr2=0x563395bccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 0x563395bccbf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009610 tx=0x7f1c38009d60 comp rx=0 tx=0).stop 2024-05-09T09:14:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 shutdown_connections 2024-05-09T09:14:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bd0f60 0x563395bd33e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bc0f30 0x563395bccbf0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bc5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 --2- 172.21.15.64:0/4037821450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.128+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 >> 172.21.15.64:0/4037821450 conn(0x563395bb6a40 msgr2=0x563395bb5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.129+0000 7f1c26ff5640 0 client.16992 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.129+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 shutdown_connections 2024-05-09T09:14:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.129+0000 7f1c4c431740 1 -- 172.21.15.64:0/4037821450 wait complete. 2024-05-09T09:14:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.129+0000 7f1c4c431740 20 client.16992 trim_cache size 0 max 16384 2024-05-09T09:14:44.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:43 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1529625701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:14:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.157+0000 7f1c4c431740 1 Processor -- start 2024-05-09T09:14:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 -- start start 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bb6a40 0x563395bb99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441bba70 con 0x563395bb6a40 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441b9280 con 0x563395bbb3d0 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c4c431740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441b70d0 con 0x563395bbbdc0 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51044/0 (socket says 172.21.15.64:51044) 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 -- 172.21.15.64:0/3182136254 learned_addr learned my addr 172.21.15.64:0/3182136254 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bb6a40 0x563395bb99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c437fe640 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 -- 172.21.15.64:0/3182136254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 msgr2=0x563395bb9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bb9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 -- 172.21.15.64:0/3182136254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bb6a40 msgr2=0x563395bb99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bb6a40 0x563395bb99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.158+0000 7f1c427fc640 1 -- 172.21.15.64:0/3182136254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563395ba8330 con 0x563395bbb3d0 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.159+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bccbb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c01a000 tx=0x7f1c3c00dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13f0767dd04cd01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.159+0000 7f1c28ff9640 1 -- 172.21.15.64:0/3182136254 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00dad0 con 0x563395bbb3d0 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.159+0000 7f1c28ff9640 1 -- 172.21.15.64:0/3182136254 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c00a4f0 con 0x563395bbb3d0 2024-05-09T09:14:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.159+0000 7f1c28ff9640 1 -- 172.21.15.64:0/3182136254 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c01f680 con 0x563395bbb3d0 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.160+0000 7f1c4c431740 1 -- 172.21.15.64:0/3182136254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 msgr2=0x563395bccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.160+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bccbb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c01a000 tx=0x7f1c3c00dfb0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 -- 172.21.15.64:0/3182136254 shutdown_connections 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bbbdc0 0x563395bb9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bccbb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3182136254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bb6a40 0x563395bb99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 -- 172.21.15.64:0/3182136254 >> 172.21.15.64:0/3182136254 conn(0x563395bc0a00 msgr2=0x563395bbd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 -- 172.21.15.64:0/3182136254 shutdown_connections 2024-05-09T09:14:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 -- 172.21.15.64:0/3182136254 wait complete. 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.161+0000 7f1c4c431740 1 -- start start 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.162+0000 7f1c4c431740 1 Processor -- start 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.162+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bb6a40 0x563395bc9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.162+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c4c431740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bcb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c42ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bb6a40 0x563395bc9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c42ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bb6a40 0x563395bc9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33010/0 (socket says 172.21.15.64:33010) 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c42ffd640 1 -- 172.21.15.64:0/3411090989 learned_addr learned my addr 172.21.15.64:0/3411090989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c437fe640 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bcb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441b7740 con 0x563395bbbdc0 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441b8bb0 con 0x563395bbb3d0 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441b7e10 con 0x563395bb6a40 2024-05-09T09:14:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.163+0000 7f1c4c431740 10 client.0 _ll_register_callbacks cb 0x7f1c38004740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c427fc640 1 -- 172.21.15.64:0/3411090989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bb6a40 msgr2=0x563395bc9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bb6a40 0x563395bc9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c427fc640 1 -- 172.21.15.64:0/3411090989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 msgr2=0x563395bcb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bcb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c427fc640 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563395bcb8b0 con 0x563395bbb3d0 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c427fc640 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bca2b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c016010 tx=0x7f1c3c00ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2820648c2f25dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.164+0000 7f1c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.165+0000 7f1c26ff5640 1 -- 172.21.15.64:0/3411090989 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c014020 con 0x563395bbb3d0 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.165+0000 7f1c26ff5640 1 -- 172.21.15.64:0/3411090989 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c3c0141c0 con 0x563395bbb3d0 2024-05-09T09:14:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.165+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c3c00a4f0 con 0x563395bbb3d0 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.165+0000 7f1c26ff5640 1 -- 172.21.15.64:0/3411090989 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1c3c0117d0 con 0x563395bbb3d0 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.165+0000 7f1c4c431740 10 client.26486 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.165+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563395bccbc0 con 0x563395bbb3d0 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c26ff5640 1 -- 172.21.15.64:0/3411090989 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c00a4f0 con 0x563395bbb3d0 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c26ff5640 1 -- 172.21.15.64:0/3411090989 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f1c3c010040 con 0x563395bbb3d0 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c4c431740 10 client.26486 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c4c431740 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c4c431740 20 client.26486 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bc0f30 0x563395c39710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c4c431740 4 client.26486 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.166+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563395b2ca70 con 0x563395bc0f30 2024-05-09T09:14:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.167+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bc0f30 0x563395c39710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.168+0000 7f1c42ffd640 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bc0f30 0x563395c39710 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b066508958d2cfa server_cookie=380f50d98acd682c in_seq=0 out_seq=0 2024-05-09T09:14:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.168+0000 7f1c26ff5640 10 client.26486 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:43 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1529625701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c26ff5640 1 -- 172.21.15.64:0/3411090989 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563395b2ca70 con 0x563395bc0f30 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c26ff5640 10 client.26486 handle_command_reply: tid=0 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 client.26486 shutdown 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 msgr2=0x563395bca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bca2b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c016010 tx=0x7f1c3c00ddf0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 shutdown_connections 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563395bc0f30 0x563395c39710 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563395bbbdc0 0x563395bcb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c26ff5640 0 client.26486 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563395bbb3d0 0x563395bca2b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 --2- 172.21.15.64:0/3411090989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563395bb6a40 0x563395bc9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.276+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 >> 172.21.15.64:0/3411090989 conn(0x563395bc0a00 msgr2=0x563395bbd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.277+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 shutdown_connections 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.277+0000 7f1c4c431740 1 -- 172.21.15.64:0/3411090989 wait complete. 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.277+0000 7f1c4c431740 20 client.26486 trim_cache size 0 max 16384 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:14:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c2c0a2cc0 msgr2=0x7f1c2c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c2c0a2cc0 0x7f1c2c0a5180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x563395bbc830 tx=0x7f1c34009610 comp rx=0 tx=0).stop 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 msgr2=0x7f1c4410c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 0x7f1c4410c030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800d680 tx=0x7f1c3800db40 comp rx=0 tx=0).stop 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 shutdown_connections 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c2c0a2cc0 0x7f1c2c0a5180 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c44109800 0x7f1c4410c030 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c44108e40 0x7f1c4410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 --2- 172.21.15.64:0/3743402959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c44072b60 0x7f1c4410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 >> 172.21.15.64:0/3743402959 conn(0x7f1c4406e6d0 msgr2=0x7f1c44070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 shutdown_connections 2024-05-09T09:14:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:44.278+0000 7f1c43fff640 1 -- 172.21.15.64:0/3743402959 wait complete. 2024-05-09T09:14:44.921 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:14:44.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:14:44.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:14:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:44 smithi139 ceph-mon[33599]: pgmap v538: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 20 MiB/s wr, 968 op/s 2024-05-09T09:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd5d59a640 1 -- 172.21.15.64:0/3401541724 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58031880 con 0x7fcd60109980 2024-05-09T09:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 -- 172.21.15.64:0/3401541724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60109980 msgr2=0x7fcd601102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/3401541724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60109980 0x7fcd601102a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009900 tx=0x7fcd5802a420 comp rx=0 tx=0).stop 2024-05-09T09:14:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 -- 172.21.15.64:0/3401541724 shutdown_connections 2024-05-09T09:14:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/3401541724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60109980 0x7fcd601102a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/3401541724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 0x7fcd60109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/3401541724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd60107e70 0x7fcd60108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.117+0000 7fcd64ea5640 1 -- 172.21.15.64:0/3401541724 >> 172.21.15.64:0/3401541724 conn(0x7fcd60076e10 msgr2=0x7fcd60079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.120+0000 7fcd64ea5640 1 -- 172.21.15.64:0/3401541724 shutdown_connections 2024-05-09T09:14:45.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.120+0000 7fcd64ea5640 1 -- 172.21.15.64:0/3401541724 wait complete. 2024-05-09T09:14:45.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 Processor -- start 2024-05-09T09:14:45.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 -- start start 2024-05-09T09:14:45.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 0x7fcd6019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 0x7fcd6019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd60109980 0x7fcd601a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60113b30 con 0x7fcd60108fc0 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601139b0 con 0x7fcd60107e70 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.121+0000 7fcd64ea5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60113cb0 con 0x7fcd60109980 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 0x7fcd6019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 0x7fcd6019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 0x7fcd6019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55656/0 (socket says 172.21.15.64:55656) 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 0x7fcd6019e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52518/0 (socket says 172.21.15.64:52518) 2024-05-09T09:14:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/2171557477 learned_addr learned my addr 172.21.15.64:0/2171557477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd60109980 0x7fcd601a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 -- 172.21.15.64:0/2171557477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd60109980 msgr2=0x7fcd601a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd60109980 0x7fcd601a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 -- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 msgr2=0x7fcd6019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 0x7fcd6019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 -- 172.21.15.64:0/2171557477 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd601a5e30 con 0x7fcd60107e70 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.122+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 0x7fcd6019e210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009fd0 tx=0x7fcd4800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.123+0000 7fcd577fe640 1 -- 172.21.15.64:0/2171557477 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48015020 con 0x7fcd60107e70 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.123+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd601a6120 con 0x7fcd60107e70 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.123+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a6740 con 0x7fcd60107e70 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.123+0000 7fcd577fe640 1 -- 172.21.15.64:0/2171557477 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd48011ce0 con 0x7fcd60107e70 2024-05-09T09:14:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.123+0000 7fcd577fe640 1 -- 172.21.15.64:0/2171557477 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48010de0 con 0x7fcd60107e70 2024-05-09T09:14:45.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.128+0000 7fcd577fe640 1 -- 172.21.15.64:0/2171557477 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcd48018020 con 0x7fcd60107e70 2024-05-09T09:14:45.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.128+0000 7fcd67bb2740 1 Processor -- start 2024-05-09T09:14:45.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 -- start start 2024-05-09T09:14:45.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 0x560abf8d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d9eb0 0x560abf8de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560abf83d990 con 0x560abf8df020 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560abf7e55b0 con 0x560abf8d9eb0 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560abf839950 con 0x560abf8d94c0 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.129+0000 7fcd5ed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 0x560abf8d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.130+0000 7fcd5ed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 0x560abf8d98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44098/0 (socket says 172.21.15.64:44098) 2024-05-09T09:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.130+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1082537235 learned_addr learned my addr 172.21.15.64:0/1082537235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.130+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.130+0000 7fcd577fe640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd4c0a2ce0 0x7fcd4c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.130+0000 7fcd577fe640 1 -- 172.21.15.64:0/2171557477 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcd480ceab0 con 0x7fcd60107e70 2024-05-09T09:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.131+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d9eb0 0x560abf8de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.131+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd4c0a2ce0 0x7fcd4c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1082537235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d9eb0 msgr2=0x560abf8de9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d9eb0 0x560abf8de9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1082537235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 msgr2=0x560abf8e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1082537235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf6197e0 con 0x560abf8d94c0 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 0x560abf8d98e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005c40 tx=0x7fcd58003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ffc645d2689049 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd4c0a2ce0 0x7fcd4c0a51a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcd580314e0 tx=0x7fcd58038000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd54ff9640 1 -- 172.21.15.64:0/1082537235 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5804e070 con 0x560abf8d94c0 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.132+0000 7fcd54ff9640 1 -- 172.21.15.64:0/1082537235 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd5803b920 con 0x560abf8d94c0 2024-05-09T09:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.133+0000 7fcd54ff9640 1 -- 172.21.15.64:0/1082537235 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58035040 con 0x560abf8d94c0 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1082537235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 msgr2=0x560abf8d98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 0x560abf8d98e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005c40 tx=0x7fcd58003af0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1082537235 shutdown_connections 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d9eb0 0x560abf8de9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1082537235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d94c0 0x560abf8d98e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.134+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1082537235 >> 172.21.15.64:0/1082537235 conn(0x560abf8a10a0 msgr2=0x560abf8d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.136+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1082537235 shutdown_connections 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.137+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1082537235 wait complete. 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.137+0000 7fcd67bb2740 1 -- start start 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.137+0000 7fcd67bb2740 1 Processor -- start 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 0x560abf8f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560abf8e1f70 con 0x560abf8df020 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560abf7f8b40 con 0x560abf8d94c0 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560abf8e20f0 con 0x560abf8d9eb0 2024-05-09T09:14:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.138+0000 7fcd67bb2740 10 client.0 _ll_register_callbacks cb 0x560abf7ed0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.139+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 0x560abf8f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.139+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 0x560abf8f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44104/0 (socket says 172.21.15.64:44104) 2024-05-09T09:14:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.139+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/2867029037 learned_addr learned my addr 172.21.15.64:0/2867029037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.139+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.139+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 msgr2=0x560abf8f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/2867029037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 msgr2=0x560abf8fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/2867029037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf679f70 con 0x560abf8d9eb0 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 0x560abf8f4e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcd50007d80 tx=0x7fcd500115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac638603c25b4587 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.140+0000 7fcd45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.141+0000 7fcd45ffb640 1 -- 172.21.15.64:0/2867029037 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5001a020 con 0x560abf8d9eb0 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.141+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd5803b920 con 0x560abf8d9eb0 2024-05-09T09:14:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.141+0000 7fcd45ffb640 1 -- 172.21.15.64:0/2867029037 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcd50011bd0 con 0x560abf8d9eb0 2024-05-09T09:14:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.141+0000 7fcd45ffb640 1 -- 172.21.15.64:0/2867029037 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcd50025050 con 0x560abf8d9eb0 2024-05-09T09:14:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.141+0000 7fcd67bb2740 10 client.26665 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:45.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.141+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560abf511170 con 0x560abf8d9eb0 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd45ffb640 1 -- 172.21.15.64:0/2867029037 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd5803b920 con 0x560abf8d9eb0 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd45ffb640 1 -- 172.21.15.64:0/2867029037 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fcd500153d0 con 0x560abf8d9eb0 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd67bb2740 10 client.26665 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd67bb2740 10 client.26665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd67bb2740 20 client.26665 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9007c0 0x560abf902c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd67bb2740 4 client.26665 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.142+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abf84b0c0 con 0x560abf9007c0 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.143+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9007c0 0x560abf902c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.143+0000 7fcd45ffb640 10 client.26665 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.143+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9007c0 0x560abf902c60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3abef6d6b669be server_cookie=35dbc57287916524 in_seq=0 out_seq=0 2024-05-09T09:14:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.144+0000 7fcd45ffb640 1 -- 172.21.15.64:0/2867029037 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560abf84b0c0 con 0x560abf9007c0 2024-05-09T09:14:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.144+0000 7fcd45ffb640 10 client.26665 handle_command_reply: tid=0 2024-05-09T09:14:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.144+0000 7fcd67bb2740 1 client.26665 shutdown 2024-05-09T09:14:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.146+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 msgr2=0x560abf8f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.146+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 0x560abf8f4e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcd50007d80 tx=0x7fcd500115a0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.147+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 shutdown_connections 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.147+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9007c0 0x560abf902c60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.147+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.147+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8d9eb0 0x560abf8f4e30 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcd50007d80 tx=0x7fcd500115a0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.147+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/2867029037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.148+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 >> 172.21.15.64:0/2867029037 conn(0x560abf8a10a0 msgr2=0x560abf8d4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.148+0000 7fcd45ffb640 0 client.26665 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.148+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 shutdown_connections 2024-05-09T09:14:45.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.148+0000 7fcd67bb2740 1 -- 172.21.15.64:0/2867029037 wait complete. 2024-05-09T09:14:45.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.149+0000 7fcd67bb2740 20 client.26665 trim_cache size 0 max 16384 2024-05-09T09:14:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:44 smithi064 ceph-mon[30149]: pgmap v538: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 20 MiB/s wr, 968 op/s 2024-05-09T09:14:45.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.185+0000 7fcd67bb2740 1 Processor -- start 2024-05-09T09:14:45.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.185+0000 7fcd67bb2740 1 -- start start 2024-05-09T09:14:45.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8df020 0x560abf8fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8dba60 0x560abf8d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40012550 con 0x560abf8dba60 2024-05-09T09:14:45.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000fd60 con 0x560abf8d94c0 2024-05-09T09:14:45.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000dbb0 con 0x560abf8df020 2024-05-09T09:14:45.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8df020 0x560abf8fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd5ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8dba60 0x560abf8d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8df020 0x560abf8fbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44114/0 (socket says 172.21.15.64:44114) 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.186+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/1197066274 learned_addr learned my addr 172.21.15.64:0/1197066274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.187+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1197066274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8df020 msgr2=0x560abf8fbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.187+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8df020 0x560abf8fbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.187+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1197066274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 msgr2=0x560abf8d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.187+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.187+0000 7fcd5ed9d640 1 -- 172.21.15.64:0/1197066274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf8f5790 con 0x560abf8dba60 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.187+0000 7fcd5ed9d640 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8dba60 0x560abf8d3d30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009900 tx=0x7fcd5802a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bc4d2237e95aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.188+0000 7fcd47fff640 1 -- 172.21.15.64:0/1197066274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58058070 con 0x560abf8dba60 2024-05-09T09:14:45.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.188+0000 7fcd47fff640 1 -- 172.21.15.64:0/1197066274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd5804f040 con 0x560abf8dba60 2024-05-09T09:14:45.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.188+0000 7fcd47fff640 1 -- 172.21.15.64:0/1197066274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58035070 con 0x560abf8dba60 2024-05-09T09:14:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1197066274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8dba60 msgr2=0x560abf8d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8dba60 0x560abf8d3d30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009900 tx=0x7fcd5802a920 comp rx=0 tx=0).stop 2024-05-09T09:14:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1197066274 shutdown_connections 2024-05-09T09:14:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8dba60 0x560abf8d3d30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8df020 0x560abf8fbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1197066274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1197066274 >> 172.21.15.64:0/1197066274 conn(0x560abf8a10a0 msgr2=0x560abf8db350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.189+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1197066274 shutdown_connections 2024-05-09T09:14:45.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.190+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1197066274 wait complete. 2024-05-09T09:14:45.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.190+0000 7fcd67bb2740 1 -- start start 2024-05-09T09:14:45.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.190+0000 7fcd67bb2740 1 Processor -- start 2024-05-09T09:14:45.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 0x560abf8f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 0x560abf8f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000e220 con 0x560abf8df020 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000f690 con 0x560abf8d94c0 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd5dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 0x560abf8f9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44128/0 (socket says 172.21.15.64:44128) 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/1010576684 learned_addr learned my addr 172.21.15.64:0/1010576684 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000e8f0 con 0x560abf8dba60 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd67bb2740 10 client.0 _ll_register_callbacks cb 0x7fcd50009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.191+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.192+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 msgr2=0x560abf8f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.192+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.192+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/1010576684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 msgr2=0x560abf8fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.192+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.192+0000 7fcd5dd9b640 1 -- 172.21.15.64:0/1010576684 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf8fb270 con 0x560abf8dba60 2024-05-09T09:14:45.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.192+0000 7fcd5dd9b640 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 0x560abf8f9ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcd500158c0 tx=0x7fcd50007c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e05714e96c57f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.193+0000 7fcd45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.193+0000 7fcd45ffb640 1 -- 172.21.15.64:0/1010576684 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5002d070 con 0x560abf8dba60 2024-05-09T09:14:45.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.193+0000 7fcd45ffb640 1 -- 172.21.15.64:0/1010576684 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcd50011ca0 con 0x560abf8dba60 2024-05-09T09:14:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.193+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd5804f040 con 0x560abf8dba60 2024-05-09T09:14:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.194+0000 7fcd45ffb640 1 -- 172.21.15.64:0/1010576684 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcd50037050 con 0x560abf8dba60 2024-05-09T09:14:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.194+0000 7fcd67bb2740 10 client.26671 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.194+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560abf966fc0 con 0x560abf8dba60 2024-05-09T09:14:45.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.195+0000 7fcd45ffb640 1 -- 172.21.15.64:0/1010576684 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd5804f040 con 0x560abf8dba60 2024-05-09T09:14:45.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.195+0000 7fcd45ffb640 1 -- 172.21.15.64:0/1010576684 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fcd5003a020 con 0x560abf8dba60 2024-05-09T09:14:45.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.195+0000 7fcd67bb2740 10 client.26671 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:45.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.196+0000 7fcd67bb2740 10 client.26671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:45.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.196+0000 7fcd67bb2740 20 client.26671 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.196+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9679f0 0x560abf969e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.196+0000 7fcd67bb2740 4 client.26671 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:14:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.196+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9679f0 0x560abf969e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.196+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560abf845410 con 0x560abf9679f0 2024-05-09T09:14:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.197+0000 7fcd45ffb640 10 client.26671 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.197+0000 7fcd5e59c640 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9679f0 0x560abf969e10 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7758f65f85cbd8c5 server_cookie=2e3695ebca7fccec in_seq=0 out_seq=0 2024-05-09T09:14:45.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.200+0000 7fcd45ffb640 1 -- 172.21.15.64:0/1010576684 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560abf845410 con 0x560abf9679f0 2024-05-09T09:14:45.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.200+0000 7fcd45ffb640 10 client.26671 handle_command_reply: tid=0 2024-05-09T09:14:45.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.200+0000 7fcd67bb2740 1 client.26671 shutdown 2024-05-09T09:14:45.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.202+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 msgr2=0x560abf8f9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.202+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 0x560abf8f9ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcd500158c0 tx=0x7fcd50007c40 comp rx=0 tx=0).stop 2024-05-09T09:14:45.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 shutdown_connections 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560abf9679f0 0x560abf969e10 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560abf8df020 0x560abf8fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560abf8dba60 0x560abf8f9ce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 --2- 172.21.15.64:0/1010576684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560abf8d94c0 0x560abf8f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 >> 172.21.15.64:0/1010576684 conn(0x560abf8a10a0 msgr2=0x560abf8d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd45ffb640 0 client.26671 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 shutdown_connections 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.203+0000 7fcd67bb2740 1 -- 172.21.15.64:0/1010576684 wait complete. 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.204+0000 7fcd67bb2740 20 client.26671 trim_cache size 0 max 16384 2024-05-09T09:14:45.204 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:14:45.205 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:14:45.205 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "0bcad003-861a-40fb-8d1a-5e4359486969", 2024-05-09T09:14:45.205 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:14:45.205 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:14:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.206+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd4c0a2ce0 msgr2=0x7fcd4c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.206+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd4c0a2ce0 0x7fcd4c0a51a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcd580314e0 tx=0x7fcd58038000 comp rx=0 tx=0).stop 2024-05-09T09:14:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.206+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 msgr2=0x7fcd6019e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.206+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 0x7fcd6019e210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009fd0 tx=0x7fcd4800b870 comp rx=0 tx=0).stop 2024-05-09T09:14:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.207+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 shutdown_connections 2024-05-09T09:14:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.207+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcd4c0a2ce0 0x7fcd4c0a51a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.207+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd60109980 0x7fcd601a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.207+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60108fc0 0x7fcd6019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.207+0000 7fcd64ea5640 1 --2- 172.21.15.64:0/2171557477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd60107e70 0x7fcd6019e210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.207+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 >> 172.21.15.64:0/2171557477 conn(0x7fcd60076e10 msgr2=0x7fcd6010c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.211+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 shutdown_connections 2024-05-09T09:14:45.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.212+0000 7fcd64ea5640 1 -- 172.21.15.64:0/2171557477 wait complete. 2024-05-09T09:14:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:44 smithi179 ceph-mon[34086]: pgmap v538: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 20 MiB/s wr, 968 op/s 2024-05-09T09:14:45.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:14:45.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:14:45.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.464+0000 7f72f1163640 1 -- 172.21.15.64:0/2057032241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 msgr2=0x7f72ec10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.464+0000 7f72f1163640 1 --2- 172.21.15.64:0/2057032241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 0x7f72ec10ac60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009ca0 tx=0x7f72e402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 -- 172.21.15.64:0/2057032241 shutdown_connections 2024-05-09T09:14:45.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 --2- 172.21.15.64:0/2057032241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10db70 0x7f72ec10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 --2- 172.21.15.64:0/2057032241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ec10b1a0 0x7f72ec10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 --2- 172.21.15.64:0/2057032241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 0x7f72ec10ac60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 -- 172.21.15.64:0/2057032241 >> 172.21.15.64:0/2057032241 conn(0x7f72ec06e6c0 msgr2=0x7f72ec070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 -- 172.21.15.64:0/2057032241 shutdown_connections 2024-05-09T09:14:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 -- 172.21.15.64:0/2057032241 wait complete. 2024-05-09T09:14:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.465+0000 7f72f1163640 1 Processor -- start 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 -- start start 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 0x7f72ec1199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 0x7f72ec119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ec10db70 0x7f72ec118030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec110820 con 0x7f72ec108830 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1106a0 con 0x7f72ec10db70 2024-05-09T09:14:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72f1163640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1109a0 con 0x7f72ec10b1a0 2024-05-09T09:14:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.467+0000 7f72ea59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 0x7f72ec119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 0x7f72ec119f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44140/0 (socket says 172.21.15.64:44140) 2024-05-09T09:14:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 -- 172.21.15.64:0/2012490656 learned_addr learned my addr 172.21.15.64:0/2012490656 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ead9d640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 0x7f72ec1199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 -- 172.21.15.64:0/2012490656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ec10db70 msgr2=0x7f72ec118030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:14:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ec10db70 0x7f72ec118030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 -- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 msgr2=0x7f72ec1199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 0x7f72ec1199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.468+0000 7f72ea59c640 1 -- 172.21.15.64:0/2012490656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec118760 con 0x7f72ec10b1a0 2024-05-09T09:14:45.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.469+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 0x7f72ec119f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e9c0 tx=0x7f72dc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.469+0000 7f72d3fff640 1 -- 172.21.15.64:0/2012490656 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00cd80 con 0x7f72ec10b1a0 2024-05-09T09:14:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.469+0000 7f72d3fff640 1 -- 172.21.15.64:0/2012490656 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72dc0035f0 con 0x7f72ec10b1a0 2024-05-09T09:14:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.469+0000 7f72d3fff640 1 -- 172.21.15.64:0/2012490656 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc010690 con 0x7f72ec10b1a0 2024-05-09T09:14:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.469+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec1189f0 con 0x7f72ec10b1a0 2024-05-09T09:14:45.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.469+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec1b7670 con 0x7f72ec10b1a0 2024-05-09T09:14:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 Processor -- start 2024-05-09T09:14:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 -- start start 2024-05-09T09:14:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 0x563fe7e69350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e6e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563fe7dcdd00 con 0x563fe7e69920 2024-05-09T09:14:45.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563fe7d754c0 con 0x563fe7e68f30 2024-05-09T09:14:45.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.473+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fe7dc9cc0 con 0x563fe7e6ea90 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72eb59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 0x563fe7e69350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72eb59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 0x563fe7e69350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52588/0 (socket says 172.21.15.64:52588) 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72eb59e640 1 -- 172.21.15.64:0/312531329 learned_addr learned my addr 172.21.15.64:0/312531329 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72ead9d640 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e6e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72eb59e640 1 -- 172.21.15.64:0/312531329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 msgr2=0x563fe7e70f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72eb59e640 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e70f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.474+0000 7f72eb59e640 1 -- 172.21.15.64:0/312531329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 msgr2=0x563fe7e6e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.475+0000 7f72eb59e640 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e6e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.475+0000 7f72eb59e640 1 -- 172.21.15.64:0/312531329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe7bab7d0 con 0x563fe7e68f30 2024-05-09T09:14:45.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.475+0000 7f72eb59e640 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 0x563fe7e69350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009b70 tx=0x7f72e000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33907367d30c4e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.476+0000 7f72d17fa640 1 -- 172.21.15.64:0/312531329 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0009e60 con 0x563fe7e68f30 2024-05-09T09:14:45.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.476+0000 7f72d17fa640 1 -- 172.21.15.64:0/312531329 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e00054d0 con 0x563fe7e68f30 2024-05-09T09:14:45.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.476+0000 7f72d17fa640 1 -- 172.21.15.64:0/312531329 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0010450 con 0x563fe7e68f30 2024-05-09T09:14:45.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.476+0000 7f72d3fff640 1 -- 172.21.15.64:0/2012490656 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72dc0037a0 con 0x7f72ec10b1a0 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 -- 172.21.15.64:0/312531329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 msgr2=0x563fe7e69350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 0x563fe7e69350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009b70 tx=0x7f72e000e720 comp rx=0 tx=0).stop 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 -- 172.21.15.64:0/312531329 shutdown_connections 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e6e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/312531329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e68f30 0x563fe7e69350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 -- 172.21.15.64:0/312531329 >> 172.21.15.64:0/312531329 conn(0x563fe7e645a0 msgr2=0x563fe7e649a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 -- 172.21.15.64:0/312531329 shutdown_connections 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.477+0000 7f72f3e7a740 1 -- 172.21.15.64:0/312531329 wait complete. 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.478+0000 7f72d3fff640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72d40a2bf0 0x7f72d40a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.478+0000 7f72d3fff640 1 -- 172.21.15.64:0/2012490656 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72dc0cf7c0 con 0x7f72ec10b1a0 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.478+0000 7f72f3e7a740 1 -- start start 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.478+0000 7f72f3e7a740 1 Processor -- start 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.478+0000 7f72ead9d640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72d40a2bf0 0x7f72d40a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e68f30 0x563fe7e73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 0x563fe7e739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72ead9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e68f30 0x563fe7e73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72ead9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e68f30 0x563fe7e73460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55744/0 (socket says 172.21.15.64:55744) 2024-05-09T09:14:45.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72ea59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 0x563fe7e739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72ea59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 0x563fe7e739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52590/0 (socket says 172.21.15.64:52590) 2024-05-09T09:14:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563fe7e71a40 con 0x563fe7e68f30 2024-05-09T09:14:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72ead9d640 1 -- 172.21.15.64:0/2123933721 learned_addr learned my addr 172.21.15.64:0/2123933721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563fe7d88b90 con 0x563fe7e69920 2024-05-09T09:14:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.479+0000 7f72eb59e640 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fe7e71bc0 con 0x563fe7e6ea90 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72f3e7a740 10 client.0 _ll_register_callbacks cb 0x563fe7dc4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72ea59c640 1 -- 172.21.15.64:0/2123933721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 msgr2=0x563fe7e7a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72ea59c640 1 -- 172.21.15.64:0/2123933721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e68f30 msgr2=0x563fe7e73460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e68f30 0x563fe7e73460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72ea59c640 1 -- 172.21.15.64:0/2123933721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe7ae13f0 con 0x563fe7e69920 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e00054d0 con 0x563fe7e69920 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.480+0000 7f72ead9d640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72d40a2bf0 0x7f72d40a50b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f72e4031970 tx=0x7f72e4030710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.482+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 0x563fe7e739a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc09b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92d8eb41734cb4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.482+0000 7f72ceff5640 10 client.26510 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:14:45.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.482+0000 7f72ceff5640 1 -- 172.21.15.64:0/2123933721 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0a4610 con 0x563fe7e69920 2024-05-09T09:14:45.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.483+0000 7f72ceff5640 1 -- 172.21.15.64:0/2123933721 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72dc09b080 con 0x563fe7e69920 2024-05-09T09:14:45.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.483+0000 7f72ceff5640 1 -- 172.21.15.64:0/2123933721 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f72dc09b300 con 0x563fe7e69920 2024-05-09T09:14:45.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.483+0000 7f72f3e7a740 10 client.26510 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:45.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.483+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=40}) -- 0x563fe7aa10f0 con 0x563fe7e69920 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72ceff5640 1 -- 172.21.15.64:0/2123933721 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f72dc097980 con 0x563fe7e69920 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72f3e7a740 10 client.26510 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72f3e7a740 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72f3e7a740 20 client.26510 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7e7f0c0 0x563fe7e81520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72f3e7a740 4 client.26510 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.485+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe7ddb430 con 0x563fe7e7f0c0 2024-05-09T09:14:45.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.486+0000 7f72ead9d640 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7e7f0c0 0x563fe7e81520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.486+0000 7f72ead9d640 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7e7f0c0 0x563fe7e81520 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d830e145016adb31 server_cookie=85692c8b62351a9 in_seq=0 out_seq=0 2024-05-09T09:14:45.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.486+0000 7f72ceff5640 10 client.26510 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.487+0000 7f72ceff5640 1 -- 172.21.15.64:0/2123933721 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fe7ddb430 con 0x563fe7e7f0c0 2024-05-09T09:14:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.487+0000 7f72ceff5640 10 client.26510 handle_command_reply: tid=0 2024-05-09T09:14:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.487+0000 7f72f3e7a740 1 client.26510 shutdown 2024-05-09T09:14:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.487+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 msgr2=0x563fe7e739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.487+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 0x563fe7e739a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc09b650 comp rx=0 tx=0).stop 2024-05-09T09:14:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.488+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 shutdown_connections 2024-05-09T09:14:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.488+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7e7f0c0 0x563fe7e81520 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.489+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.489+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e69920 0x563fe7e739a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.489+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2123933721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e68f30 0x563fe7e73460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.489+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 >> 172.21.15.64:0/2123933721 conn(0x563fe7e645a0 msgr2=0x563fe7e637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.490+0000 7f72ceff5640 0 client.26510 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.490+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 shutdown_connections 2024-05-09T09:14:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.491+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2123933721 wait complete. 2024-05-09T09:14:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.491+0000 7f72f3e7a740 20 client.26510 trim_cache size 0 max 16384 2024-05-09T09:14:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.530+0000 7f72f3e7a740 1 Processor -- start 2024-05-09T09:14:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.530+0000 7f72f3e7a740 1 -- start start 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.530+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e67280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e7a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e6ea90 0x563fe7e677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c4500 con 0x563fe7e69920 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c1d10 con 0x563fe7e6ea90 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1bfb60 con 0x563fe7e68f30 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72ead9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e67280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72ea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e7a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72ea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e7a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55754/0 (socket says 172.21.15.64:55754) 2024-05-09T09:14:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.531+0000 7f72ea59c640 1 -- 172.21.15.64:0/2813815280 learned_addr learned my addr 172.21.15.64:0/2813815280 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ea59c640 1 -- 172.21.15.64:0/2813815280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 msgr2=0x563fe7e67280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e67280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ea59c640 1 -- 172.21.15.64:0/2813815280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e6ea90 msgr2=0x563fe7e677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e6ea90 0x563fe7e677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ea59c640 1 -- 172.21.15.64:0/2813815280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe7e6a980 con 0x563fe7e69920 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ead9d640 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e67280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.532+0000 7f72ea59c640 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e7a9a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f72dc09b830 tx=0x7f72dc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b04a316eecca97c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.533+0000 7f72d0ff9640 1 -- 172.21.15.64:0/2813815280 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0cce10 con 0x563fe7e69920 2024-05-09T09:14:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.533+0000 7f72d0ff9640 1 -- 172.21.15.64:0/2813815280 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e00054d0 con 0x563fe7e69920 2024-05-09T09:14:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.533+0000 7f72d0ff9640 1 -- 172.21.15.64:0/2813815280 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0a4410 con 0x563fe7e69920 2024-05-09T09:14:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2813815280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 msgr2=0x563fe7e7a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e7a9a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f72dc09b830 tx=0x7f72dc011000 comp rx=0 tx=0).stop 2024-05-09T09:14:45.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2813815280 shutdown_connections 2024-05-09T09:14:45.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e6ea90 0x563fe7e677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e69920 0x563fe7e7a9a0 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f72dc09b830 tx=0x7f72dc011000 comp rx=0 tx=0).stop 2024-05-09T09:14:45.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/2813815280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e67280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.534+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2813815280 >> 172.21.15.64:0/2813815280 conn(0x563fe7e645a0 msgr2=0x563fe7e71f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.538+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2813815280 shutdown_connections 2024-05-09T09:14:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.538+0000 7f72f3e7a740 1 -- 172.21.15.64:0/2813815280 wait complete. 2024-05-09T09:14:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.538+0000 7f72f3e7a740 1 -- start start 2024-05-09T09:14:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.538+0000 7f72f3e7a740 1 Processor -- start 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e7a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e78910 0x563fe7e78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c01d0 con 0x563fe7e6ea90 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c1640 con 0x563fe7e78910 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c08a0 con 0x563fe7e68f30 2024-05-09T09:14:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.539+0000 7f72f3e7a740 10 client.0 _ll_register_callbacks cb 0x7f72dc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e7a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e7a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44168/0 (socket says 172.21.15.64:44168) 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 -- 172.21.15.64:0/3668064653 learned_addr learned my addr 172.21.15.64:0/3668064653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ea59c640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 -- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e78910 msgr2=0x563fe7e78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e78910 0x563fe7e78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 -- 172.21.15.64:0/3668064653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e6ea90 msgr2=0x563fe7e7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:45.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ead9d640 1 -- 172.21.15.64:0/3668064653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe7e793d0 con 0x563fe7e68f30 2024-05-09T09:14:45.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.540+0000 7f72ea59c640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:14:45.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.541+0000 7f72ead9d640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e7a220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009bf0 tx=0x7f72e4048000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee60d116f2fefc4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:45.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.541+0000 7f72ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:45.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.541+0000 7f72ceff5640 1 -- 172.21.15.64:0/3668064653 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4045040 con 0x563fe7e68f30 2024-05-09T09:14:45.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.541+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e00054d0 con 0x563fe7e68f30 2024-05-09T09:14:45.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.541+0000 7f72ceff5640 1 -- 172.21.15.64:0/3668064653 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72e4030a00 con 0x563fe7e68f30 2024-05-09T09:14:45.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.542+0000 7f72ceff5640 1 -- 172.21.15.64:0/3668064653 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f72e4064050 con 0x563fe7e68f30 2024-05-09T09:14:45.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.542+0000 7f72f3e7a740 10 client.26686 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:45.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.542+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563fe7ee6490 con 0x563fe7e68f30 2024-05-09T09:14:45.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.543+0000 7f72ceff5640 1 -- 172.21.15.64:0/3668064653 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e00054d0 con 0x563fe7e68f30 2024-05-09T09:14:45.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.543+0000 7f72ceff5640 1 -- 172.21.15.64:0/3668064653 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f72e404c450 con 0x563fe7e68f30 2024-05-09T09:14:45.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.543+0000 7f72f3e7a740 10 client.26686 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:45.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.543+0000 7f72f3e7a740 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:45.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.543+0000 7f72f3e7a740 20 client.26686 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:45.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.544+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7ee6e40 0x563fe7ee9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:45.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.544+0000 7f72f3e7a740 4 client.26686 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:14:45.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.544+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fe7dd5780 con 0x563fe7ee6e40 2024-05-09T09:14:45.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.545+0000 7f72ea59c640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7ee6e40 0x563fe7ee9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:45.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.546+0000 7f72ea59c640 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7ee6e40 0x563fe7ee9260 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ffa5d361c129a2e server_cookie=1fce7424571ca254 in_seq=0 out_seq=0 2024-05-09T09:14:45.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.546+0000 7f72ceff5640 10 client.26686 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.548+0000 7f72ceff5640 1 -- 172.21.15.64:0/3668064653 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563fe7dd5780 con 0x563fe7ee6e40 2024-05-09T09:14:45.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.548+0000 7f72ceff5640 10 client.26686 handle_command_reply: tid=0 2024-05-09T09:14:45.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.548+0000 7f72f3e7a740 1 client.26686 shutdown 2024-05-09T09:14:45.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.549+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 msgr2=0x563fe7e7a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.549+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e7a220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009bf0 tx=0x7f72e4048000 comp rx=0 tx=0).stop 2024-05-09T09:14:45.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 shutdown_connections 2024-05-09T09:14:45.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563fe7ee6e40 0x563fe7ee9260 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563fe7e78910 0x563fe7e78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563fe7e6ea90 0x563fe7e7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72f3e7a740 1 --2- 172.21.15.64:0/3668064653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fe7e68f30 0x563fe7e7a220 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 >> 172.21.15.64:0/3668064653 conn(0x563fe7e645a0 msgr2=0x563fe7e62f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.550+0000 7f72ceff5640 0 client.26686 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.554+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 shutdown_connections 2024-05-09T09:14:45.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.558+0000 7f72f3e7a740 1 -- 172.21.15.64:0/3668064653 wait complete. 2024-05-09T09:14:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.563+0000 7f72f3e7a740 20 client.26686 trim_cache size 0 max 16384 2024-05-09T09:14:45.563 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:14:45.563 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:14:45.563 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:14:45.563 INFO:teuthology.orchestra.run.smithi064.stdout: "0bcad003-861a-40fb-8d1a-5e4359486969": { 2024-05-09T09:14:45.564 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:14:45.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "0bcad003-861a-40fb-8d1a-5e4359486969", 2024-05-09T09:14:45.564 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:14:45.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:14:45.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:14:45.564 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:14:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.567+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72d40a2bf0 msgr2=0x7f72d40a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.567+0000 7f72f1163640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72d40a2bf0 0x7f72d40a50b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f72e4031970 tx=0x7f72e4030710 comp rx=0 tx=0).stop 2024-05-09T09:14:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.567+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 msgr2=0x7f72ec119f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.567+0000 7f72f1163640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 0x7f72ec119f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e9c0 tx=0x7f72dc00ee80 comp rx=0 tx=0).stop 2024-05-09T09:14:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.568+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 shutdown_connections 2024-05-09T09:14:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.568+0000 7f72f1163640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f72d40a2bf0 0x7f72d40a50b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.568+0000 7f72f1163640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ec10db70 0x7f72ec118030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.568+0000 7f72f1163640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72ec10b1a0 0x7f72ec119f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.568+0000 7f72f1163640 1 --2- 172.21.15.64:0/2012490656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72ec108830 0x7f72ec1199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.568+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 >> 172.21.15.64:0/2012490656 conn(0x7f72ec06e6c0 msgr2=0x7f72ec1090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.570+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 shutdown_connections 2024-05-09T09:14:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:14:45.570+0000 7f72f1163640 1 -- 172.21.15.64:0/2012490656 wait complete. 2024-05-09T09:14:45.572 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:14:45 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:14:45.219+0000 7f3d1eb03700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans/ auth{2=1} v2300 ap=2 f(v1 m2024-05-09T09:14:30.242793+0000 4=4+0) n(v6 rc2024-05-09T09:14:30.278333+0000 b1032192 5=4+1) (inest sync->mix g=2) caps={15456=pAsLsXsFs/-@26} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55f5a7ce6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v501>,<0x10000000005/data v111>,<0x10000000003/postgres v28>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v39>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v45>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2300>,<0x10000000005/data v902>,<0x10000000003/postgres v459>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v461>,<0x10000000001/sv_0 v479>,<0x10000000000/qa v461>,<0x1/volumes v434>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:13:27.741710+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:14:30.278333+0000 b1032192 5=4+1)","memory_value.dirstat":"f(v1 m2024-05-09T09:14:30.242793+0000 4=4+0)","memory_value.rstat":"n(v6 rc2024-05-09T09:14:30.278333+0000 b1032192 5=4+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:14:45.593 INFO:tasks.cephfs.filesystem:scrub status for tag:0bcad003-861a-40fb-8d1a-5e4359486969 - {'path': '/', 'tag': '0bcad003-861a-40fb-8d1a-5e4359486969', 'options': 'recursive,force'} 2024-05-09T09:14:45.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:45 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:45 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:45 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:45 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:46.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:45 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:46.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:45 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:46.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:45 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:46.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:45 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:46.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:45 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:46.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:45 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:46.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:45 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:46.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:45 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 265.0 s, 796.4 tps, lat 125.509 ms stddev 222.440 2024-05-09T09:14:47.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:46 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:14:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:46 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:46 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:14:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:46 smithi139 ceph-mon[33599]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:14:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:46 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:46 smithi139 ceph-mon[33599]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 901 op/s 2024-05-09T09:14:47.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:46 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:14:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:46 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:46 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:14:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:46 smithi064 ceph-mon[30149]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:14:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:46 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:46 smithi064 ceph-mon[30149]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 901 op/s 2024-05-09T09:14:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:46 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:14:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:46 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:46 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:14:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:46 smithi179 ceph-mon[34086]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:14:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:46 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:14:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:46 smithi179 ceph-mon[34086]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 901 op/s 2024-05-09T09:14:49.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:48 smithi179 ceph-mon[34086]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 770 op/s 2024-05-09T09:14:49.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:48 smithi139 ceph-mon[33599]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 770 op/s 2024-05-09T09:14:49.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:48 smithi064 ceph-mon[30149]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 770 op/s 2024-05-09T09:14:51.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:50 smithi139 ceph-mon[33599]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2024-05-09T09:14:51.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:50 smithi064 ceph-mon[30149]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2024-05-09T09:14:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:50 smithi179 ceph-mon[34086]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2024-05-09T09:14:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 270.0 s, 917.6 tps, lat 96.246 ms stddev 125.889 2024-05-09T09:14:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:52 smithi139 ceph-mon[33599]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 17 MiB/s wr, 876 op/s 2024-05-09T09:14:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:52 smithi064 ceph-mon[30149]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 17 MiB/s wr, 876 op/s 2024-05-09T09:14:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:52 smithi179 ceph-mon[34086]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 17 MiB/s wr, 876 op/s 2024-05-09T09:14:55.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:54 smithi139 ceph-mon[33599]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 19 MiB/s wr, 950 op/s 2024-05-09T09:14:55.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:54 smithi064 ceph-mon[30149]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 19 MiB/s wr, 950 op/s 2024-05-09T09:14:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:54 smithi179 ceph-mon[34086]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 19 MiB/s wr, 950 op/s 2024-05-09T09:14:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 275.0 s, 1022.9 tps, lat 109.006 ms stddev 170.809 2024-05-09T09:14:57.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:56 smithi139 ceph-mon[33599]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 918 op/s 2024-05-09T09:14:57.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:56 smithi064 ceph-mon[30149]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 918 op/s 2024-05-09T09:14:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:56 smithi179 ceph-mon[34086]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 918 op/s 2024-05-09T09:14:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.397+0000 7efdb1714640 1 -- 172.21.15.64:0/3559762715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac1019a0 msgr2=0x7efdac101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.397+0000 7efdb1714640 1 --2- 172.21.15.64:0/3559762715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac1019a0 0x7efdac101e20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efd94009900 tx=0x7efd9402a410 comp rx=0 tx=0).stop 2024-05-09T09:14:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- 172.21.15.64:0/3559762715 shutdown_connections 2024-05-09T09:14:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 --2- 172.21.15.64:0/3559762715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdac102360 0x7efdac108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 --2- 172.21.15.64:0/3559762715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac1019a0 0x7efdac101e20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 --2- 172.21.15.64:0/3559762715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- 172.21.15.64:0/3559762715 >> 172.21.15.64:0/3559762715 conn(0x7efdac0fbd40 msgr2=0x7efdac0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- 172.21.15.64:0/3559762715 shutdown_connections 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- 172.21.15.64:0/3559762715 wait complete. 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 Processor -- start 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- start start 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdac1019a0 0x7efdac197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 0x7efdac19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10c310 con 0x7efdac102360 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10c190 con 0x7efdac1019a0 2024-05-09T09:14:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.398+0000 7efdb1714640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10c490 con 0x7efdac1007b0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 0x7efdac19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59654/0 (socket says 172.21.15.64:59654) 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 -- 172.21.15.64:0/1967533082 learned_addr learned my addr 172.21.15.64:0/1967533082 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 0x7efdac19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51432/0 (socket says 172.21.15.64:51432) 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efda3fff640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdac1019a0 0x7efdac197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 -- 172.21.15.64:0/1967533082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdac1019a0 msgr2=0x7efdac197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdac1019a0 0x7efdac197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 -- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 msgr2=0x7efdac19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 0x7efdac19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 -- 172.21.15.64:0/1967533082 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac19f680 con 0x7efdac1007b0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdab7fe640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 0x7efdac19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdaaffd640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac197ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e9c0 tx=0x7efd9c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efda8ff9640 1 -- 172.21.15.64:0/1967533082 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00cd80 con 0x7efdac1007b0 2024-05-09T09:14:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac19f970 con 0x7efdac1007b0 2024-05-09T09:14:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efda8ff9640 1 -- 172.21.15.64:0/1967533082 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd9c0035f0 con 0x7efdac1007b0 2024-05-09T09:14:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efda8ff9640 1 -- 172.21.15.64:0/1967533082 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c010690 con 0x7efdac1007b0 2024-05-09T09:14:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.399+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac19fe90 con 0x7efdac1007b0 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.402+0000 7efda8ff9640 1 -- 172.21.15.64:0/1967533082 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd9c0037a0 con 0x7efdac1007b0 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.402+0000 7efdb4426740 1 Processor -- start 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.402+0000 7efdb4426740 1 -- start start 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdb4426740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6c660 0x562641c6ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdb4426740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c6d050 0x562641c71b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdb4426740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 0x562641c74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdb4426740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562641bcc340 con 0x562641c6c660 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdb4426740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562641b74290 con 0x562641c6d050 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdb4426740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562641bc8300 con 0x562641c721c0 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6c660 0x562641c6ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c6d050 0x562641c71b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 0x562641c74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 0x562641c74650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59670/0 (socket says 172.21.15.64:59670) 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c6d050 0x562641c71b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41652/0 (socket says 172.21.15.64:41652) 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 -- 172.21.15.64:0/311777926 learned_addr learned my addr 172.21.15.64:0/311777926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda8ff9640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd980a2c30 0x7efd980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 -- 172.21.15.64:0/311777926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c6d050 msgr2=0x562641c71b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c6d050 0x562641c71b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 -- 172.21.15.64:0/311777926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6c660 msgr2=0x562641c6ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6c660 0x562641c6ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda3fff640 1 -- 172.21.15.64:0/311777926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562641b988e0 con 0x562641c721c0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdab7fe640 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6c660 0x562641c6ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efdab7fe640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd980a2c30 0x7efd980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.403+0000 7efda8ff9640 1 -- 172.21.15.64:0/1967533082 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd9c0cf820 con 0x7efdac1007b0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.404+0000 7efda3fff640 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 0x562641c74650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd9402aa00 tx=0x7efd940040b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0c41d11f3b57b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.404+0000 7efdab7fe640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd980a2c30 0x7efd980a50f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efda4009370 tx=0x7efda4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.404+0000 7efda17fa640 1 -- 172.21.15.64:0/311777926 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94035030 con 0x562641c721c0 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.404+0000 7efda17fa640 1 -- 172.21.15.64:0/311777926 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd940044a0 con 0x562641c721c0 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.404+0000 7efda17fa640 1 -- 172.21.15.64:0/311777926 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940035f0 con 0x562641c721c0 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- 172.21.15.64:0/311777926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 msgr2=0x562641c74650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 0x562641c74650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd9402aa00 tx=0x7efd940040b0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- 172.21.15.64:0/311777926 shutdown_connections 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c721c0 0x562641c74650 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c6d050 0x562641c71b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- 172.21.15.64:0/311777926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6c660 0x562641c6ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- 172.21.15.64:0/311777926 >> 172.21.15.64:0/311777926 conn(0x562641c638c0 msgr2=0x562641c63cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- 172.21.15.64:0/311777926 shutdown_connections 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- 172.21.15.64:0/311777926 wait complete. 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- start start 2024-05-09T09:14:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 Processor -- start 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c76820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c76d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.405+0000 7efdb4426740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562641c75080 con 0x562641c6d050 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdb4426740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562641b87790 con 0x562641c721c0 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdb4426740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562641c75200 con 0x562641c6c660 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efda3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c76d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdb4426740 10 client.0 _ll_register_callbacks cb 0x562641bc2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c76820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59672/0 (socket says 172.21.15.64:59672) 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 -- 172.21.15.64:0/919297665 learned_addr learned my addr 172.21.15.64:0/919297665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdab7fe640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 -- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 msgr2=0x562641c7dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 -- 172.21.15.64:0/919297665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 msgr2=0x562641c76d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c76d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 -- 172.21.15.64:0/919297665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626418e0210 con 0x562641c6c660 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efda3fff640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c76d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdaaffd640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c76820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c6a0 tx=0x7efd9c009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece4c2e709e050e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efd927fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efd927fc640 1 -- 172.21.15.64:0/919297665 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c09b650 con 0x562641c6c660 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efd927fc640 1 -- 172.21.15.64:0/919297665 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd9c09b340 con 0x562641c6c660 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd940044a0 con 0x562641c6c660 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efd927fc640 1 -- 172.21.15.64:0/919297665 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efd9c003b30 con 0x562641c6c660 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.406+0000 7efdb4426740 10 client.26698 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5626419ac620 con 0x562641c6c660 2024-05-09T09:14:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efd927fc640 1 -- 172.21.15.64:0/919297665 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd940044a0 con 0x562641c6c660 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efd927fc640 1 -- 172.21.15.64:0/919297665 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7efd9c09abb0 con 0x562641c6c660 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 10 client.26698 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 20 client.26698 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641c82330 0x562641c847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 4 client.26698 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562641bd9a70 con 0x562641c82330 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.407+0000 7efda3fff640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641c82330 0x562641c847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efd927fc640 10 client.26698 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efda3fff640 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641c82330 0x562641c847c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac7caad3a78e34 server_cookie=aa84ca913e0d7d03 in_seq=0 out_seq=0 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efd927fc640 1 -- 172.21.15.64:0/919297665 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562641bd9a70 con 0x562641c82330 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efd927fc640 10 client.26698 handle_command_reply: tid=0 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efdb4426740 1 client.26698 shutdown 2024-05-09T09:14:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 msgr2=0x562641c76820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.408+0000 7efdb4426740 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c76820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c6a0 tx=0x7efd9c009700 comp rx=0 tx=0).stop 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 shutdown_connections 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641c82330 0x562641c847c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c76d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efd927fc640 0 client.26698 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 --2- 172.21.15.64:0/919297665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c76820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 >> 172.21.15.64:0/919297665 conn(0x562641c638c0 msgr2=0x562641c62a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 shutdown_connections 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 1 -- 172.21.15.64:0/919297665 wait complete. 2024-05-09T09:14:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.409+0000 7efdb4426740 20 client.26698 trim_cache size 0 max 16384 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.426+0000 7efdb4426740 1 Processor -- start 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.426+0000 7efdb4426740 1 -- start start 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdb4426740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdb4426740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c66870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdb4426740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdb4426740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1aaa80 con 0x562641c6d050 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdb4426740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a8290 con 0x562641c721c0 2024-05-09T09:14:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdb4426740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a60e0 con 0x562641c6c660 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c66870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c66870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51448/0 (socket says 172.21.15.64:51448) 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 -- 172.21.15.64:0/2429081019 learned_addr learned my addr 172.21.15.64:0/2429081019 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdab7fe640 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdaaffd640 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 -- 172.21.15.64:0/2429081019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 msgr2=0x562641c63ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c63ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 -- 172.21.15.64:0/2429081019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 msgr2=0x562641c66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efda3fff640 1 -- 172.21.15.64:0/2429081019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562641c61f00 con 0x562641c6d050 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.427+0000 7efdab7fe640 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:14:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.428+0000 7efda3fff640 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c66870 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efd94009900 tx=0x7efd94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacf3c6debd784ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.428+0000 7efda0ff9640 1 -- 172.21.15.64:0/2429081019 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94035040 con 0x562641c6d050 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.428+0000 7efda0ff9640 1 -- 172.21.15.64:0/2429081019 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd94004bd0 con 0x562641c6d050 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.428+0000 7efda0ff9640 1 -- 172.21.15.64:0/2429081019 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940507c0 con 0x562641c6d050 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 -- 172.21.15.64:0/2429081019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 msgr2=0x562641c66870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c66870 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efd94009900 tx=0x7efd94009250 comp rx=0 tx=0).stop 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 -- 172.21.15.64:0/2429081019 shutdown_connections 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c66870 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- 172.21.15.64:0/2429081019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 -- 172.21.15.64:0/2429081019 >> 172.21.15.64:0/2429081019 conn(0x562641c638c0 msgr2=0x562641c75650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 -- 172.21.15.64:0/2429081019 shutdown_connections 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 -- 172.21.15.64:0/2429081019 wait complete. 2024-05-09T09:14:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 -- start start 2024-05-09T09:14:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 Processor -- start 2024-05-09T09:14:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.429+0000 7efdb4426740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdb4426740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a6750 con 0x562641c6d050 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdb4426740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a7bc0 con 0x562641c721c0 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdb4426740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a6e20 con 0x562641c6c660 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdb4426740 10 client.0 _ll_register_callbacks cb 0x7efd9c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c7a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59682/0 (socket says 172.21.15.64:59682) 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 -- 172.21.15.64:0/2606178151 learned_addr learned my addr 172.21.15.64:0/2606178151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdab7fe640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efda3fff640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 -- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 msgr2=0x562641c7b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 -- 172.21.15.64:0/2606178151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 msgr2=0x562641c7a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c7a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 -- 172.21.15.64:0/2606178151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562641c7bfe0 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdaaffd640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c7a4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c09ae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a11b41e984a4a5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efd927fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efd927fc640 1 -- 172.21.15.64:0/2606178151 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c09b650 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efd927fc640 1 -- 172.21.15.64:0/2606178151 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efd9c097a40 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.430+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd94004bd0 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efd927fc640 1 -- 172.21.15.64:0/2606178151 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efd9c09bac0 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 10 client.26704 fetch_fsmap learned FSMap version 40 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562641c7d360 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efd927fc640 1 -- 172.21.15.64:0/2606178151 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd94004bd0 con 0x562641c6c660 2024-05-09T09:14:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efd927fc640 1 -- 172.21.15.64:0/2606178151 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7efd9c097c40 con 0x562641c6c660 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 10 client.26704 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 20 client.26704 populate_metadata read hostname 'smithi064' 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641ce8ea0 0x562641ceb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 4 client.26704 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562641bd3dc0 con 0x562641ce8ea0 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.431+0000 7efda3fff640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641ce8ea0 0x562641ceb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.433+0000 7efda3fff640 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641ce8ea0 0x562641ceb2c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab807d7ed35f00d2 server_cookie=b4071c55ad90b711 in_seq=0 out_seq=0 2024-05-09T09:14:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.433+0000 7efd927fc640 10 client.26704 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.469+0000 7efd927fc640 1 -- 172.21.15.64:0/2606178151 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562641bd3dc0 con 0x562641ce8ea0 2024-05-09T09:14:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.469+0000 7efd927fc640 10 client.26704 handle_command_reply: tid=0 2024-05-09T09:14:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.469+0000 7efdb4426740 1 client.26704 shutdown 2024-05-09T09:14:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.470+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 msgr2=0x562641c7a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.470+0000 7efdb4426740 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c7a4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c09ae60 comp rx=0 tx=0).stop 2024-05-09T09:14:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.479+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 shutdown_connections 2024-05-09T09:14:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.479+0000 7efdb4426740 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562641ce8ea0 0x562641ceb2c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.479+0000 7efdb4426740 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562641c721c0 0x562641c7b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.479+0000 7efdb4426740 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562641c6d050 0x562641c7a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.479+0000 7efdb4426740 1 --2- 172.21.15.64:0/2606178151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562641c6c660 0x562641c7a4a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.479+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 >> 172.21.15.64:0/2606178151 conn(0x562641c638c0 msgr2=0x562641c66870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.480+0000 7efd927fc640 0 client.26704 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.480+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 shutdown_connections 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.480+0000 7efdb4426740 1 -- 172.21.15.64:0/2606178151 wait complete. 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.480+0000 7efdb4426740 20 client.26704 trim_cache size 0 max 16384 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:14:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:14:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:14:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:14:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.481+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd980a2c30 msgr2=0x7efd980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.481+0000 7efdb1714640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd980a2c30 0x7efd980a50f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efda4009370 tx=0x7efda4004a40 comp rx=0 tx=0).stop 2024-05-09T09:14:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.481+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 msgr2=0x7efdac197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:14:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.481+0000 7efdb1714640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac197ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e9c0 tx=0x7efd9c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:14:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.483+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 shutdown_connections 2024-05-09T09:14:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.483+0000 7efdb1714640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efd980a2c30 0x7efd980a50f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.483+0000 7efdb1714640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac102360 0x7efdac19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.483+0000 7efdb1714640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdac1019a0 0x7efdac197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.483+0000 7efdb1714640 1 --2- 172.21.15.64:0/1967533082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1007b0 0x7efdac197ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:14:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.483+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 >> 172.21.15.64:0/1967533082 conn(0x7efdac0fbd40 msgr2=0x7efdac0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:14:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.484+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 shutdown_connections 2024-05-09T09:14:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:14:57.484+0000 7efdb1714640 1 -- 172.21.15.64:0/1967533082 wait complete. 2024-05-09T09:14:59.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:14:58 smithi139 ceph-mon[33599]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 16 MiB/s wr, 824 op/s 2024-05-09T09:14:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:14:58 smithi064 ceph-mon[30149]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 16 MiB/s wr, 824 op/s 2024-05-09T09:14:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:14:58 smithi179 ceph-mon[34086]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 16 MiB/s wr, 824 op/s 2024-05-09T09:15:01.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:00 smithi139 ceph-mon[33599]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2024-05-09T09:15:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:15:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:15:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:15:01.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:00 smithi064 ceph-mon[30149]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2024-05-09T09:15:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:15:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:15:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:15:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:00 smithi179 ceph-mon[34086]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2024-05-09T09:15:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:15:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:15:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:15:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 280.0 s, 913.9 tps, lat 109.546 ms stddev 173.586 2024-05-09T09:15:03.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:02 smithi139 ceph-mon[33599]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 801 op/s 2024-05-09T09:15:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:02 smithi064 ceph-mon[30149]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 801 op/s 2024-05-09T09:15:03.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:02 smithi179 ceph-mon[34086]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 801 op/s 2024-05-09T09:15:05.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:04 smithi139 ceph-mon[33599]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 18 MiB/s wr, 919 op/s 2024-05-09T09:15:05.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:04 smithi064 ceph-mon[30149]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 18 MiB/s wr, 919 op/s 2024-05-09T09:15:05.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:04 smithi179 ceph-mon[34086]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 18 MiB/s wr, 919 op/s 2024-05-09T09:15:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 285.0 s, 977.1 tps, lat 102.068 ms stddev 113.035 2024-05-09T09:15:07.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:06 smithi139 ceph-mon[33599]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 931 op/s 2024-05-09T09:15:07.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:06 smithi064 ceph-mon[30149]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 931 op/s 2024-05-09T09:15:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:06 smithi179 ceph-mon[34086]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 18 MiB/s wr, 931 op/s 2024-05-09T09:15:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:08 smithi139 ceph-mon[33599]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 836 op/s 2024-05-09T09:15:09.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:08 smithi064 ceph-mon[30149]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 836 op/s 2024-05-09T09:15:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:08 smithi179 ceph-mon[34086]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 836 op/s 2024-05-09T09:15:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:10 smithi139 ceph-mon[33599]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-05-09T09:15:11.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:10 smithi064 ceph-mon[30149]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-05-09T09:15:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:10 smithi179 ceph-mon[34086]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-05-09T09:15:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 290.0 s, 833.8 tps, lat 102.121 ms stddev 128.910 2024-05-09T09:15:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.970+0000 7f0ad01c1640 1 -- 172.21.15.64:0/877690281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810db50 msgr2=0x7f0ac810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.970+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/877690281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810db50 0x7f0ac810dfd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009ca0 tx=0x7f0ac002a8d0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 -- 172.21.15.64:0/877690281 shutdown_connections 2024-05-09T09:15:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/877690281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810db50 0x7f0ac810dfd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/877690281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac810b1a0 0x7f0ac810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/877690281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8108830 0x7f0ac810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 -- 172.21.15.64:0/877690281 >> 172.21.15.64:0/877690281 conn(0x7f0ac806e8a0 msgr2=0x7f0ac8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 -- 172.21.15.64:0/877690281 shutdown_connections 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.971+0000 7f0ad01c1640 1 -- 172.21.15.64:0/877690281 wait complete. 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.972+0000 7f0ad01c1640 1 Processor -- start 2024-05-09T09:15:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.972+0000 7f0ad01c1640 1 -- start start 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0ad01c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 0x7f0ac81a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0ad01c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810b1a0 0x7f0ac81a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0ad01c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac810db50 0x7f0ac81add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0ad01c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac811c1d0 con 0x7f0ac810b1a0 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0ad01c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac811c050 con 0x7f0ac810db50 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0ad01c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac811c350 con 0x7f0ac8108830 2024-05-09T09:15:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 0x7f0ac81a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 0x7f0ac81a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35104/0 (socket says 172.21.15.64:35104) 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2734212682 learned_addr learned my addr 172.21.15.64:0/2734212682 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.973+0000 7f0acd75c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810b1a0 0x7f0ac81a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.974+0000 7f0ace75e640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac810db50 0x7f0ac81add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.974+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2734212682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac810db50 msgr2=0x7f0ac81add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.974+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac810db50 0x7f0ac81add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.974+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810b1a0 msgr2=0x7f0ac81a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.974+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810b1a0 0x7f0ac81a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.974+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2734212682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac81ae3d0 con 0x7f0ac8108830 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.975+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 0x7f0ac81a67b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400ea60 tx=0x7f0ac400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.975+0000 7f0abeffd640 1 -- 172.21.15.64:0/2734212682 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400cde0 con 0x7f0ac8108830 2024-05-09T09:15:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.976+0000 7f0abeffd640 1 -- 172.21.15.64:0/2734212682 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac40035f0 con 0x7f0ac8108830 2024-05-09T09:15:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.976+0000 7f0abeffd640 1 -- 172.21.15.64:0/2734212682 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4010690 con 0x7f0ac8108830 2024-05-09T09:15:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.976+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac81ae6c0 con 0x7f0ac8108830 2024-05-09T09:15:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.976+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac81aebc0 con 0x7f0ac8108830 2024-05-09T09:15:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.979+0000 7f0abeffd640 1 -- 172.21.15.64:0/2734212682 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ac40037a0 con 0x7f0ac8108830 2024-05-09T09:15:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.981+0000 7f0abeffd640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2e70 0x7f0ab40a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.981+0000 7f0ad2fac740 1 Processor -- start 2024-05-09T09:15:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.981+0000 7f0abeffd640 1 -- 172.21.15.64:0/2734212682 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ac40cf720 con 0x7f0ac8108830 2024-05-09T09:15:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 -- start start 2024-05-09T09:15:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243675670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x56524367a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0acd75c640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2e70 0x7f0ab40a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x56524367d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5652435d9340 con 0x565243675c40 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565243581290 con 0x565243675250 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5652435d5300 con 0x56524367adb0 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ace75e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x56524367d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ace75e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x56524367d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35112/0 (socket says 172.21.15.64:35112) 2024-05-09T09:15:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.982+0000 7f0ace75e640 1 -- 172.21.15.64:0/1158621515 learned_addr learned my addr 172.21.15.64:0/1158621515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.983+0000 7f0acd75c640 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x56524367a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.983+0000 7f0ace75e640 1 -- 172.21.15.64:0/1158621515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 msgr2=0x565243675670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.983+0000 7f0ace75e640 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243675670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.983+0000 7f0ace75e640 1 -- 172.21.15.64:0/1158621515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 msgr2=0x56524367a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.983+0000 7f0ace75e640 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x56524367a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.983+0000 7f0ace75e640 1 -- 172.21.15.64:0/1158621515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652435a58e0 con 0x56524367adb0 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.984+0000 7f0ace75e640 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x56524367d240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0005f60 tx=0x7f0ac0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc8e9f2ffcb5b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.984+0000 7f0acd75c640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2e70 0x7f0ab40a5330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005fd0 tx=0x7f0ab800b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.984+0000 7f0ab3fff640 1 -- 172.21.15.64:0/1158621515 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0048070 con 0x56524367adb0 2024-05-09T09:15:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.984+0000 7f0ab3fff640 1 -- 172.21.15.64:0/1158621515 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac0036040 con 0x56524367adb0 2024-05-09T09:15:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.984+0000 7f0ab3fff640 1 -- 172.21.15.64:0/1158621515 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0030950 con 0x56524367adb0 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 -- 172.21.15.64:0/1158621515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 msgr2=0x56524367d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x56524367d240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0005f60 tx=0x7f0ac0005f90 comp rx=0 tx=0).stop 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 -- 172.21.15.64:0/1158621515 shutdown_connections 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x56524367d240 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x56524367a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/1158621515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243675670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.986+0000 7f0ad2fac740 1 -- 172.21.15.64:0/1158621515 >> 172.21.15.64:0/1158621515 conn(0x5652436708c0 msgr2=0x565243670cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.987+0000 7f0ad2fac740 1 -- 172.21.15.64:0/1158621515 shutdown_connections 2024-05-09T09:15:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.987+0000 7f0ad2fac740 1 -- 172.21.15.64:0/1158621515 wait complete. 2024-05-09T09:15:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.987+0000 7f0ad2fac740 1 -- start start 2024-05-09T09:15:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.988+0000 7f0ad2fac740 1 Processor -- start 2024-05-09T09:15:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.988+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x56524367f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x56524367fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56524367adb0 0x565243686ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x56524367f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56524367dd00 con 0x56524367adb0 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565243594790 con 0x565243675250 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56524367de80 con 0x565243675c40 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x56524367f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53690/0 (socket says 172.21.15.64:53690) 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2884552921 learned_addr learned my addr 172.21.15.64:0/2884552921 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0ad2fac740 10 client.0 _ll_register_callbacks cb 0x5652435cfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.989+0000 7f0acd75c640 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x56524367fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2884552921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 msgr2=0x56524367fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x56524367fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2884552921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56524367adb0 msgr2=0x565243686ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56524367adb0 0x565243686ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2884552921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652432ed210 con 0x565243675250 2024-05-09T09:15:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x56524367f550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ac409bb20 tx=0x7f0ac409b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f7d77076e437d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0ab17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0ab17fa640 1 -- 172.21.15.64:0/2884552921 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac409c290 con 0x565243675250 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.990+0000 7f0ab17fa640 1 -- 172.21.15.64:0/2884552921 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ac40cf3d0 con 0x565243675250 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.991+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac0036040 con 0x565243675250 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.991+0000 7f0ab17fa640 1 -- 172.21.15.64:0/2884552921 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0ac40a9050 con 0x565243675250 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.991+0000 7f0ad2fac740 10 client.26537 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.991+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5652433b9620 con 0x565243675250 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ab17fa640 1 -- 172.21.15.64:0/2884552921 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac0036040 con 0x565243675250 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ab17fa640 1 -- 172.21.15.64:0/2884552921 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f0ac40ce040 con 0x565243675250 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ad2fac740 10 client.26537 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ad2fac740 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ad2fac740 20 client.26537 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56524368ae20 0x56524368d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ad2fac740 4 client.26537 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.992+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652435e6a70 con 0x56524368ae20 2024-05-09T09:15:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.993+0000 7f0acd75c640 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56524368ae20 0x56524368d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.994+0000 7f0acd75c640 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56524368ae20 0x56524368d290 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98bc7c7c9a3219b3 server_cookie=13eb93d44ebecf03 in_seq=0 out_seq=0 2024-05-09T09:15:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.994+0000 7f0ab17fa640 10 client.26537 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.994+0000 7f0ab17fa640 1 -- 172.21.15.64:0/2884552921 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652435e6a70 con 0x56524368ae20 2024-05-09T09:15:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.994+0000 7f0ab17fa640 10 client.26537 handle_command_reply: tid=0 2024-05-09T09:15:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 client.26537 shutdown 2024-05-09T09:15:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 msgr2=0x56524367f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x56524367f550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ac409bb20 tx=0x7f0ac409b790 comp rx=0 tx=0).stop 2024-05-09T09:15:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 shutdown_connections 2024-05-09T09:15:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56524368ae20 0x56524368d290 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56524367adb0 0x565243686ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x56524367fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ab17fa640 0 client.26537 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2884552921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x56524367f550 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.995+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 >> 172.21.15.64:0/2884552921 conn(0x5652436708c0 msgr2=0x56524366fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.996+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 shutdown_connections 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.996+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2884552921 wait complete. 2024-05-09T09:15:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:11.996+0000 7f0ad2fac740 20 client.26537 trim_cache size 0 max 16384 2024-05-09T09:15:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 Processor -- start 2024-05-09T09:15:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 -- start start 2024-05-09T09:15:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 0x565243673f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x565243692ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56524367adb0 0x5652436930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81bbb00 con 0x565243675250 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b9310 con 0x56524367adb0 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.025+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b7160 con 0x565243675c40 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 0x565243673f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acd75c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x565243692ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 0x565243673f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45496/0 (socket says 172.21.15.64:45496) 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2611311459 learned_addr learned my addr 172.21.15.64:0/2611311459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0ace75e640 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56524367adb0 0x5652436930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2611311459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 msgr2=0x565243692ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x565243692ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2611311459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56524367adb0 msgr2=0x5652436930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56524367adb0 0x5652436930e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 -- 172.21.15.64:0/2611311459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565243674e90 con 0x565243675250 2024-05-09T09:15:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.026+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 0x565243673f80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400ea30 tx=0x7f0ac4003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d1b1d1f29f998d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.027+0000 7f0ab37fe640 1 -- 172.21.15.64:0/2611311459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4097e30 con 0x565243675250 2024-05-09T09:15:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.027+0000 7f0ab37fe640 1 -- 172.21.15.64:0/2611311459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac409c290 con 0x565243675250 2024-05-09T09:15:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.027+0000 7f0ab37fe640 1 -- 172.21.15.64:0/2611311459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac409bb90 con 0x565243675250 2024-05-09T09:15:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.028+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2611311459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 msgr2=0x565243673f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.028+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 0x565243673f80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400ea30 tx=0x7f0ac4003d90 comp rx=0 tx=0).stop 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2611311459 shutdown_connections 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56524367adb0 0x5652436930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565243675c40 0x565243692ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/2611311459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675250 0x565243673f80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2611311459 >> 172.21.15.64:0/2611311459 conn(0x5652436708c0 msgr2=0x565243673870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2611311459 shutdown_connections 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.029+0000 7f0ad2fac740 1 -- 172.21.15.64:0/2611311459 wait complete. 2024-05-09T09:15:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.030+0000 7f0ad2fac740 1 -- start start 2024-05-09T09:15:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.030+0000 7f0ad2fac740 1 Processor -- start 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243683b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x565243684050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ad2fac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x565243685010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ad2fac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b77d0 con 0x565243675c40 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0acdf5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243683b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0acd75c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x565243684050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ace75e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x565243685010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0acd75c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x565243684050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45502/0 (socket says 172.21.15.64:45502) 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ace75e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x565243685010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35150/0 (socket says 172.21.15.64:35150) 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0acd75c640 1 -- 172.21.15.64:0/3383209456 learned_addr learned my addr 172.21.15.64:0/3383209456 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b8c40 con 0x565243675250 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.031+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b7ea0 con 0x56524367adb0 2024-05-09T09:15:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ad2fac740 10 client.0 _ll_register_callbacks cb 0x7f0ac409b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ace75e640 1 -- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 msgr2=0x565243683b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ace75e640 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243683b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ace75e640 1 -- 172.21.15.64:0/3383209456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 msgr2=0x565243684050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ace75e640 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x565243684050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ace75e640 1 -- 172.21.15.64:0/3383209456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565243685600 con 0x56524367adb0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ace75e640 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x565243685010 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56524368d4b0 tx=0x7f0ac0031d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f70c3646a96e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.032+0000 7f0ab17fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ab17fa640 1 -- 172.21.15.64:0/3383209456 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0048030 con 0x56524367adb0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ab17fa640 1 -- 172.21.15.64:0/3383209456 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ac0040040 con 0x56524367adb0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac409c290 con 0x56524367adb0 2024-05-09T09:15:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ab17fa640 1 -- 172.21.15.64:0/3383209456 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0ac0057050 con 0x56524367adb0 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ad2fac740 10 client.26722 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5652436f17a0 con 0x56524367adb0 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ab17fa640 1 -- 172.21.15.64:0/3383209456 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac004a070 con 0x56524367adb0 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ab17fa640 1 -- 172.21.15.64:0/3383209456 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f0ac0043610 con 0x56524367adb0 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ad2fac740 10 client.26722 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.033+0000 7f0ad2fac740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.034+0000 7f0ad2fac740 20 client.26722 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.034+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652436f1d20 0x5652436f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.034+0000 7f0ad2fac740 4 client.26722 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.034+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0ac409c290 con 0x5652436f1d20 2024-05-09T09:15:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.034+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652436f1d20 0x5652436f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.034+0000 7f0ab17fa640 10 client.26722 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.035+0000 7f0acdf5d640 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652436f1d20 0x5652436f4140 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846b932e536b68ab server_cookie=653b02027519164 in_seq=0 out_seq=0 2024-05-09T09:15:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.049+0000 7f0ab17fa640 1 -- 172.21.15.64:0/3383209456 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0ac409c290 con 0x5652436f1d20 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.049+0000 7f0ab17fa640 10 client.26722 handle_command_reply: tid=0 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.049+0000 7f0ad2fac740 1 client.26722 shutdown 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.049+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 msgr2=0x565243685010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.049+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x565243685010 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56524368d4b0 tx=0x7f0ac0031d40 comp rx=0 tx=0).stop 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 shutdown_connections 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652436f1d20 0x5652436f4140 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56524367adb0 0x565243685010 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565243675c40 0x565243684050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ab17fa640 0 client.26722 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 --2- 172.21.15.64:0/3383209456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565243675250 0x565243683b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 >> 172.21.15.64:0/3383209456 conn(0x5652436708c0 msgr2=0x565243673870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 shutdown_connections 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.050+0000 7f0ad2fac740 1 -- 172.21.15.64:0/3383209456 wait complete. 2024-05-09T09:15:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.051+0000 7f0ad2fac740 20 client.26722 trim_cache size 0 max 16384 2024-05-09T09:15:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:15:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:15:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:15:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.052+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2e70 msgr2=0x7f0ab40a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.052+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2e70 0x7f0ab40a5330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005fd0 tx=0x7f0ab800b7f0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.052+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 msgr2=0x7f0ac81a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.052+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 0x7f0ac81a67b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400ea60 tx=0x7f0ac400ef20 comp rx=0 tx=0).stop 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 shutdown_connections 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2e70 0x7f0ab40a5330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac810db50 0x7f0ac81add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac810b1a0 0x7f0ac81a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 --2- 172.21.15.64:0/2734212682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8108830 0x7f0ac81a67b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 >> 172.21.15.64:0/2734212682 conn(0x7f0ac806e8a0 msgr2=0x7f0ac810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 shutdown_connections 2024-05-09T09:15:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:12.053+0000 7f0ad01c1640 1 -- 172.21.15.64:0/2734212682 wait complete. 2024-05-09T09:15:13.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:12 smithi064 ceph-mon[30149]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 847 op/s 2024-05-09T09:15:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:12 smithi179 ceph-mon[34086]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 847 op/s 2024-05-09T09:15:13.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:12 smithi139 ceph-mon[33599]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 847 op/s 2024-05-09T09:15:15.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:14 smithi064 ceph-mon[30149]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 16 MiB/s wr, 943 op/s 2024-05-09T09:15:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:14 smithi179 ceph-mon[34086]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 16 MiB/s wr, 943 op/s 2024-05-09T09:15:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:14 smithi139 ceph-mon[33599]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 16 MiB/s wr, 943 op/s 2024-05-09T09:15:15.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:15:15.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:15:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.139+0000 7f94bb7fe640 1 -- 172.21.15.64:0/327635942 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4031e40 con 0x7f94bc102320 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.140+0000 7f94c29af640 1 -- 172.21.15.64:0/327635942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc102320 msgr2=0x7f94bc108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.140+0000 7f94c29af640 1 --2- 172.21.15.64:0/327635942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc102320 0x7f94bc108bb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009ca0 tx=0x7f94b402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.140+0000 7f94c29af640 1 -- 172.21.15.64:0/327635942 shutdown_connections 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.140+0000 7f94c29af640 1 --2- 172.21.15.64:0/327635942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc102320 0x7f94bc108bb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.140+0000 7f94c29af640 1 --2- 172.21.15.64:0/327635942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc101960 0x7f94bc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.140+0000 7f94c29af640 1 --2- 172.21.15.64:0/327635942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94bc100770 0x7f94bc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.141+0000 7f94c29af640 1 -- 172.21.15.64:0/327635942 >> 172.21.15.64:0/327635942 conn(0x7f94bc0fbce0 msgr2=0x7f94bc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.141+0000 7f94c29af640 1 -- 172.21.15.64:0/327635942 shutdown_connections 2024-05-09T09:15:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.141+0000 7f94c29af640 1 -- 172.21.15.64:0/327635942 wait complete. 2024-05-09T09:15:16.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 Processor -- start 2024-05-09T09:15:16.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 -- start start 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 0x7f94bc199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94bc101960 0x7f94bc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc102320 0x7f94bc1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc10b1b0 con 0x7f94bc101960 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc10b030 con 0x7f94bc100770 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.142+0000 7f94c29af640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc10b330 con 0x7f94bc102320 2024-05-09T09:15:16.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 0x7f94bc199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 0x7f94bc199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43306/0 (socket says 172.21.15.64:43306) 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 -- 172.21.15.64:0/2113019311 learned_addr learned my addr 172.21.15.64:0/2113019311 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94b3fff640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94bc101960 0x7f94bc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94c0f4c640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc102320 0x7f94bc1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 -- 172.21.15.64:0/2113019311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc102320 msgr2=0x7f94bc1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc102320 0x7f94bc1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 -- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94bc101960 msgr2=0x7f94bc19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94bc101960 0x7f94bc19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 -- 172.21.15.64:0/2113019311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc1a18e0 con 0x7f94bc100770 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94c0f4c640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc102320 0x7f94bc1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:16.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94bbfff640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 0x7f94bc199ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0097d0 tx=0x7f94ac009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.143+0000 7f94b9ffb640 1 -- 172.21.15.64:0/2113019311 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00eab0 con 0x7f94bc100770 2024-05-09T09:15:16.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.144+0000 7f94b9ffb640 1 -- 172.21.15.64:0/2113019311 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94ac019070 con 0x7f94bc100770 2024-05-09T09:15:16.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.144+0000 7f94b9ffb640 1 -- 172.21.15.64:0/2113019311 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac013380 con 0x7f94bc100770 2024-05-09T09:15:16.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.144+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc1a1bd0 con 0x7f94bc100770 2024-05-09T09:15:16.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.144+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc1a20d0 con 0x7f94bc100770 2024-05-09T09:15:16.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.147+0000 7f94b9ffb640 1 -- 172.21.15.64:0/2113019311 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94ac013520 con 0x7f94bc100770 2024-05-09T09:15:16.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.149+0000 7f94b9ffb640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94a80a2c10 0x7f94a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.149+0000 7f94b9ffb640 1 -- 172.21.15.64:0/2113019311 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94ac0d0200 con 0x7f94bc100770 2024-05-09T09:15:16.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.149+0000 7f94b3fff640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94a80a2c10 0x7f94a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.159+0000 7f94b3fff640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94a80a2c10 0x7f94a80a50d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f94a40095f0 tx=0x7f94a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.160+0000 7f94c5795740 1 Processor -- start 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 -- start start 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0d6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0db420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0dba90 0x56215f0ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56215f03ad00 con 0x56215f0dba90 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56215efe24c0 con 0x56215f0d6920 2024-05-09T09:15:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c5795740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56215f036cc0 con 0x56215f0d5f30 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0d6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0d6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37294/0 (socket says 172.21.15.64:37294) 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94bbfff640 1 -- 172.21.15.64:0/3021046851 learned_addr learned my addr 172.21.15.64:0/3021046851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94c0f4c640 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0dba90 0x56215f0ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.161+0000 7f94b3fff640 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94bbfff640 1 -- 172.21.15.64:0/3021046851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 msgr2=0x56215f0db420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94bbfff640 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0db420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94bbfff640 1 -- 172.21.15.64:0/3021046851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0dba90 msgr2=0x56215f0ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94bbfff640 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0dba90 0x56215f0ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94bbfff640 1 -- 172.21.15.64:0/3021046851 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215ee187d0 con 0x56215f0d5f30 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94b3fff640 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.162+0000 7f94bbfff640 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0d6350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0a4db0 tx=0x7f94ac098a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42d814ede569a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.163+0000 7f94b27fc640 1 -- 172.21.15.64:0/3021046851 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0870c0 con 0x56215f0d5f30 2024-05-09T09:15:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.163+0000 7f94b27fc640 1 -- 172.21.15.64:0/3021046851 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94ac00ede0 con 0x56215f0d5f30 2024-05-09T09:15:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.163+0000 7f94b27fc640 1 -- 172.21.15.64:0/3021046851 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac014420 con 0x56215f0d5f30 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 -- 172.21.15.64:0/3021046851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 msgr2=0x56215f0d6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0d6350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0a4db0 tx=0x7f94ac098a10 comp rx=0 tx=0).stop 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 -- 172.21.15.64:0/3021046851 shutdown_connections 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0dba90 0x56215f0ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 --2- 172.21.15.64:0/3021046851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0d6350 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 -- 172.21.15.64:0/3021046851 >> 172.21.15.64:0/3021046851 conn(0x56215f0d15a0 msgr2=0x56215f0d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 -- 172.21.15.64:0/3021046851 shutdown_connections 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 -- 172.21.15.64:0/3021046851 wait complete. 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.164+0000 7f94c5795740 1 -- start start 2024-05-09T09:15:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.165+0000 7f94c5795740 1 Processor -- start 2024-05-09T09:15:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.165+0000 7f94c5795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.165+0000 7f94c5795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d6920 0x56215f0f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94c5795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94c5795740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56215f0efa50 con 0x56215f0d5f30 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42470/0 (socket says 172.21.15.64:42470) 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56215eff5b90 con 0x56215f0dba90 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 -- 172.21.15.64:0/3516871767 learned_addr learned my addr 172.21.15.64:0/3516871767 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56215f0efbd0 con 0x56215f0d6920 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94c5795740 10 client.0 _ll_register_callbacks cb 0x56215f031da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94b3fff640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d6920 0x56215f0f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94c0f4c640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 -- 172.21.15.64:0/3516871767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d6920 msgr2=0x56215f0f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d6920 0x56215f0f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 -- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 msgr2=0x56215f0f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.166+0000 7f94bbfff640 1 -- 172.21.15.64:0/3516871767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215ed4e3f0 con 0x56215f0d5f30 2024-05-09T09:15:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f94bbfff640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f94ac09a920 tx=0x7f94ac009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580c49b33d45894d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f949b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f949b7fe640 1 -- 172.21.15.64:0/3516871767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0ba060 con 0x56215f0d5f30 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f949b7fe640 1 -- 172.21.15.64:0/3516871767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94ac098ce0 con 0x56215f0d5f30 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56215f048430 con 0x56215f0d5f30 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f949b7fe640 1 -- 172.21.15.64:0/3516871767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f94ac0b4030 con 0x56215f0d5f30 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f94c5795740 10 client.17067 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.167+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56215ed0e0f0 con 0x56215f0d5f30 2024-05-09T09:15:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f949b7fe640 1 -- 172.21.15.64:0/3516871767 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56215f048430 con 0x56215f0d5f30 2024-05-09T09:15:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f949b7fe640 1 -- 172.21.15.64:0/3516871767 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f94ac0b0b90 con 0x56215f0d5f30 2024-05-09T09:15:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94c5795740 10 client.17067 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94c5795740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94c5795740 20 client.17067 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94c5795740 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f0ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94c5795740 4 client.17067 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56215f042780 con 0x56215f0fce80 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.168+0000 7f94b3fff640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f0ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.169+0000 7f94b3fff640 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f0ff270 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca589e9956c9c4eb server_cookie=f34979ce6c95ace1 in_seq=0 out_seq=0 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.169+0000 7f949b7fe640 10 client.17067 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.170+0000 7f949b7fe640 1 -- 172.21.15.64:0/3516871767 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56215f042780 con 0x56215f0fce80 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.170+0000 7f949b7fe640 10 client.17067 handle_command_reply: tid=0 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.170+0000 7f94c5795740 1 client.17067 shutdown 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 msgr2=0x56215f0f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f94ac09a920 tx=0x7f94ac009ee0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 shutdown_connections 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f0ff270 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f949b7fe640 0 client.17067 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d6920 0x56215f0f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 --2- 172.21.15.64:0/3516871767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1430 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 >> 172.21.15.64:0/3516871767 conn(0x56215f0d15a0 msgr2=0x56215f0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.171+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 shutdown_connections 2024-05-09T09:15:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.172+0000 7f94c5795740 1 -- 172.21.15.64:0/3516871767 wait complete. 2024-05-09T09:15:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.172+0000 7f94c5795740 20 client.17067 trim_cache size 0 max 16384 2024-05-09T09:15:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.201+0000 7f94c5795740 1 Processor -- start 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.201+0000 7f94c5795740 1 -- start start 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c5795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c5795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c5795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0dba90 0x56215f0d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c5795740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1a2570 con 0x56215f0d5f30 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42478/0 (socket says 172.21.15.64:42478) 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94bbfff640 1 -- 172.21.15.64:0/1387135962 learned_addr learned my addr 172.21.15.64:0/1387135962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94b3fff640 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc071480 con 0x56215f0d6920 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c0f4c640 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0dba90 0x56215f0d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc06f2d0 con 0x56215f0dba90 2024-05-09T09:15:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94b3fff640 1 -- 172.21.15.64:0/1387135962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0dba90 msgr2=0x56215f0d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94b3fff640 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0dba90 0x56215f0d44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94b3fff640 1 -- 172.21.15.64:0/1387135962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 msgr2=0x56215f0f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94b3fff640 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94b3fff640 1 -- 172.21.15.64:0/1387135962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215f0ccf60 con 0x56215f0d6920 2024-05-09T09:15:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.202+0000 7f94bbfff640 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.203+0000 7f94b3fff640 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0f1580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x56215f0ff8c0 tx=0x7f94a4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474ddb7a73d270a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.203+0000 7f94b1ffb640 1 -- 172.21.15.64:0/1387135962 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4003b40 con 0x56215f0d6920 2024-05-09T09:15:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.203+0000 7f94b1ffb640 1 -- 172.21.15.64:0/1387135962 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94a4039070 con 0x56215f0d6920 2024-05-09T09:15:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.203+0000 7f94b1ffb640 1 -- 172.21.15.64:0/1387135962 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a402f660 con 0x56215f0d6920 2024-05-09T09:15:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 msgr2=0x56215f0f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0f1580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x56215f0ff8c0 tx=0x7f94a4014000 comp rx=0 tx=0).stop 2024-05-09T09:15:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 shutdown_connections 2024-05-09T09:15:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0dba90 0x56215f0d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0d6920 0x56215f0f1580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 --2- 172.21.15.64:0/1387135962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d5f30 0x56215f0f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.204+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 >> 172.21.15.64:0/1387135962 conn(0x56215f0d15a0 msgr2=0x56215f0d1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.205+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 shutdown_connections 2024-05-09T09:15:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.205+0000 7f94c5795740 1 -- 172.21.15.64:0/1387135962 wait complete. 2024-05-09T09:15:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.205+0000 7f94c5795740 1 -- start start 2024-05-09T09:15:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.205+0000 7f94c5795740 1 Processor -- start 2024-05-09T09:15:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 0x56215f0f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc06f940 con 0x56215f0d6920 2024-05-09T09:15:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc070db0 con 0x56215f0dba90 2024-05-09T09:15:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc070010 con 0x56215f0d5f30 2024-05-09T09:15:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c5795740 10 client.0 _ll_register_callbacks cb 0x7f94ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 0x56215f0f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 0x56215f0f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42492/0 (socket says 172.21.15.64:42492) 2024-05-09T09:15:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 -- 172.21.15.64:0/394425800 learned_addr learned my addr 172.21.15.64:0/394425800 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94c0f4c640 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37316/0 (socket says 172.21.15.64:37316) 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 -- 172.21.15.64:0/394425800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 msgr2=0x56215f0f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0f55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 -- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 msgr2=0x56215f0f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 -- 172.21.15.64:0/394425800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215f0f7110 con 0x56215f0d6920 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.206+0000 7f94b3fff640 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 0x56215f0f5b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f94a40099c0 tx=0x7f94a4021d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544357cad1659eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f949b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f949b7fe640 1 -- 172.21.15.64:0/394425800 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4042060 con 0x56215f0d6920 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a4039070 con 0x56215f0d6920 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f949b7fe640 1 -- 172.21.15.64:0/394425800 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94a4033150 con 0x56215f0d6920 2024-05-09T09:15:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f949b7fe640 1 -- 172.21.15.64:0/394425800 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f94a4049050 con 0x56215f0d6920 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f94c5795740 10 client.17073 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.207+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56215f1631c0 con 0x56215f0d6920 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f949b7fe640 1 -- 172.21.15.64:0/394425800 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94a4039070 con 0x56215f0d6920 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f949b7fe640 1 -- 172.21.15.64:0/394425800 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f94a40046c0 con 0x56215f0d6920 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94c5795740 10 client.17073 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94c5795740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94c5795740 20 client.17073 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94c5795740 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f165c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94c5795740 4 client.17073 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56215efbf6f0 con 0x56215f0fce80 2024-05-09T09:15:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.208+0000 7f94bbfff640 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94bbfff640 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f165c20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ecc70cf44e8122 server_cookie=a582eba2d66efb6f in_seq=0 out_seq=0 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f949b7fe640 10 client.17073 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f949b7fe640 1 -- 172.21.15.64:0/394425800 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56215efbf6f0 con 0x56215f0fce80 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f949b7fe640 10 client.17073 handle_command_reply: tid=0 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 client.17073 shutdown 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 msgr2=0x56215f0f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 0x56215f0f5b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f94a40099c0 tx=0x7f94a4021d30 comp rx=0 tx=0).stop 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 shutdown_connections 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56215f0fce80 0x56215f165c20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56215f0dba90 0x56215f0f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f949b7fe640 0 client.17073 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56215f0d6920 0x56215f0f5b10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 --2- 172.21.15.64:0/394425800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56215f0d5f30 0x56215f0f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 >> 172.21.15.64:0/394425800 conn(0x56215f0d15a0 msgr2=0x56215f0f0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.209+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 shutdown_connections 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.210+0000 7f94c5795740 1 -- 172.21.15.64:0/394425800 wait complete. 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.210+0000 7f94c5795740 20 client.17073 trim_cache size 0 max 16384 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:15:16.220 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.210+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94a80a2c10 msgr2=0x7f94a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.210+0000 7f94c29af640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94a80a2c10 0x7f94a80a50d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f94a40095f0 tx=0x7f94a4005f90 comp rx=0 tx=0).stop 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.210+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 msgr2=0x7f94bc199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.210+0000 7f94c29af640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 0x7f94bc199ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0097d0 tx=0x7f94ac009c90 comp rx=0 tx=0).stop 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 shutdown_connections 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94a80a2c10 0x7f94a80a50d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94bc102320 0x7f94bc1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94bc101960 0x7f94bc19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 --2- 172.21.15.64:0/2113019311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94bc100770 0x7f94bc199ca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 >> 172.21.15.64:0/2113019311 conn(0x7f94bc0fbce0 msgr2=0x7f94bc104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 shutdown_connections 2024-05-09T09:15:16.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.211+0000 7f94c29af640 1 -- 172.21.15.64:0/2113019311 wait complete. 2024-05-09T09:15:16.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:15:16.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:15:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 295.0 s, 956.4 tps, lat 120.183 ms stddev 177.499 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.411+0000 7f494affd640 1 -- 172.21.15.64:0/2999792164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c030d20 con 0x7f494c1099a0 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 -- 172.21.15.64:0/2999792164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 msgr2=0x7f494c1102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 --2- 172.21.15.64:0/2999792164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 0x7f494c1102c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f493c009be0 tx=0x7f493c02a6d0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 -- 172.21.15.64:0/2999792164 shutdown_connections 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 --2- 172.21.15.64:0/2999792164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 0x7f494c1102c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 --2- 172.21.15.64:0/2999792164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 --2- 172.21.15.64:0/2999792164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f494c107e90 0x7f494c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.412+0000 7f49525a9640 1 -- 172.21.15.64:0/2999792164 >> 172.21.15.64:0/2999792164 conn(0x7f494c076e40 msgr2=0x7f494c079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.413+0000 7f49525a9640 1 -- 172.21.15.64:0/2999792164 shutdown_connections 2024-05-09T09:15:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.413+0000 7f49525a9640 1 -- 172.21.15.64:0/2999792164 wait complete. 2024-05-09T09:15:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 Processor -- start 2024-05-09T09:15:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 -- start start 2024-05-09T09:15:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f494c107e90 0x7f494c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 0x7f494c1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f494c113a20 con 0x7f494c1099a0 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f494c1138a0 con 0x7f494c108fe0 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.414+0000 7f49525a9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f494c113ba0 con 0x7f494c107e90 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.415+0000 7f494b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.415+0000 7f494b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43368/0 (socket says 172.21.15.64:43368) 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.415+0000 7f494b7fe640 1 -- 172.21.15.64:0/2761014170 learned_addr learned my addr 172.21.15.64:0/2761014170 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.415+0000 7f4950b46640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 0x7f494c1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f494b7fe640 1 -- 172.21.15.64:0/2761014170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f494c107e90 msgr2=0x7f494c19e030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f494b7fe640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f494c107e90 0x7f494c19e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f494b7fe640 1 -- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 msgr2=0x7f494c1a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f494b7fe640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 0x7f494c1a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f494b7fe640 1 -- 172.21.15.64:0/2761014170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c1a5c50 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f494b7fe640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c19e570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f494000d950 tx=0x7f494000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940017020 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4940009c30 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.416+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c1a5f40 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.417+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c1a6400 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.417+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000f6b0 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.418+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49380052c0 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.421+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f494001a020 con 0x7f494c108fe0 2024-05-09T09:15:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.422+0000 7f49497fa640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49340a2d60 0x7f49340a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.422+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f49400cf5b0 con 0x7f494c108fe0 2024-05-09T09:15:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.422+0000 7f494bfff640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49340a2d60 0x7f49340a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.423+0000 7f494bfff640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49340a2d60 0x7f49340a5220 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4944005fd0 tx=0x7f4944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.424+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4940021020 con 0x7f494c108fe0 2024-05-09T09:15:16.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.856+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49380055b0 con 0x7f494c108fe0 2024-05-09T09:15:16.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.857+0000 7f49497fa640 1 -- 172.21.15.64:0/2761014170 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11888 (secure 0 0 0) 0x7f4940009de0 con 0x7f494c108fe0 2024-05-09T09:15:16.857 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:15:16.857 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:14:41.243062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:15:16.857 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 40 2024-05-09T09:15:16.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49340a2d60 msgr2=0x7f49340a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49340a2d60 0x7f49340a5220 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4944005fd0 tx=0x7f4944009260 comp rx=0 tx=0).stop 2024-05-09T09:15:16.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 msgr2=0x7f494c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:16.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c19e570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f494000d950 tx=0x7f494000de10 comp rx=0 tx=0).stop 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 shutdown_connections 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f49340a2d60 0x7f49340a5220 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f494c1099a0 0x7f494c1a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494c108fe0 0x7f494c19e570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 --2- 172.21.15.64:0/2761014170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f494c107e90 0x7f494c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.861+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 >> 172.21.15.64:0/2761014170 conn(0x7f494c076e40 msgr2=0x7f494c10c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:16.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.862+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 shutdown_connections 2024-05-09T09:15:16.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:16.862+0000 7f49525a9640 1 -- 172.21.15.64:0/2761014170 wait complete. 2024-05-09T09:15:16.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:15:17.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.078+0000 7ff2e63ee640 1 -- 172.21.15.64:0/870371534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 msgr2=0x7ff2e010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.078+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/870371534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e010d8d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009900 tx=0x7ff2d002a410 comp rx=0 tx=0).stop 2024-05-09T09:15:17.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.079+0000 7ff2e63ee640 1 -- 172.21.15.64:0/870371534 shutdown_connections 2024-05-09T09:15:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.079+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/870371534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2e010de10 0x7ff2e01146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.079+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/870371534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e010d8d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.080+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/870371534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e010c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.080+0000 7ff2e63ee640 1 -- 172.21.15.64:0/870371534 >> 172.21.15.64:0/870371534 conn(0x7ff2e006c2c0 msgr2=0x7ff2e006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.080+0000 7ff2e63ee640 1 -- 172.21.15.64:0/870371534 shutdown_connections 2024-05-09T09:15:17.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.080+0000 7ff2e63ee640 1 -- 172.21.15.64:0/870371534 wait complete. 2024-05-09T09:15:17.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.081+0000 7ff2e63ee640 1 Processor -- start 2024-05-09T09:15:17.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.081+0000 7ff2e63ee640 1 -- start start 2024-05-09T09:15:17.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2e63ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e01a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2e63ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e01a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2e63ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2e010de10 0x7ff2e01a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2dffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e01a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2dffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e01a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37344/0 (socket says 172.21.15.64:37344) 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2dffff640 1 -- 172.21.15.64:0/2928522474 learned_addr learned my addr 172.21.15.64:0/2928522474 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.082+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0117df0 con 0x7ff2e010de10 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.083+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0117c70 con 0x7ff2e010d450 2024-05-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.083+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0117f70 con 0x7ff2e010c260 2024-05-09T09:15:17.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.083+0000 7ff2e498b640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2e010de10 0x7ff2e01a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.083+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e01a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.083+0000 7ff2dffff640 1 -- 172.21.15.64:0/2928522474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 msgr2=0x7ff2e01a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.083+0000 7ff2dffff640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e01a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.084+0000 7ff2dffff640 1 -- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2e010de10 msgr2=0x7ff2e01a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.084+0000 7ff2dffff640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2e010de10 0x7ff2e01a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.084+0000 7ff2dffff640 1 -- 172.21.15.64:0/2928522474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e01aa120 con 0x7ff2e010c260 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.084+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e01a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.084+0000 7ff2dffff640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e01a2500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e9c0 tx=0x7ff2d400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.085+0000 7ff2dd7fa640 1 -- 172.21.15.64:0/2928522474 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400cd80 con 0x7ff2e010c260 2024-05-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.085+0000 7ff2dd7fa640 1 -- 172.21.15.64:0/2928522474 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d40035f0 con 0x7ff2e010c260 2024-05-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.085+0000 7ff2dd7fa640 1 -- 172.21.15.64:0/2928522474 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4010690 con 0x7ff2e010c260 2024-05-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.085+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e01aa410 con 0x7ff2e010c260 2024-05-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.086+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01aa8e0 con 0x7ff2e010c260 2024-05-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.090+0000 7ff2e91da740 1 Processor -- start 2024-05-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.091+0000 7ff2e91da740 1 -- start start 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.092+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 0x564302998b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.092+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302999140 0x56430299dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.092+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.092+0000 7ff2e91da740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643028f4c40 con 0x564302998750 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.092+0000 7ff2e91da740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56430289c580 con 0x564302999140 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.092+0000 7ff2e91da740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5643028f0c00 con 0x56430299e2b0 2024-05-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.093+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 0x564302998b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.093+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 0x564302998b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42540/0 (socket says 172.21.15.64:42540) 2024-05-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.093+0000 7ff2e498b640 1 -- 172.21.15.64:0/3122152176 learned_addr learned my addr 172.21.15.64:0/3122152176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.093+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.093+0000 7ff2e498b640 1 -- 172.21.15.64:0/3122152176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 msgr2=0x5643029a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.094+0000 7ff2dd7fa640 1 -- 172.21.15.64:0/2928522474 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2d40037a0 con 0x7ff2e010c260 2024-05-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.094+0000 7ff2e498b640 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.094+0000 7ff2e498b640 1 -- 172.21.15.64:0/3122152176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302999140 msgr2=0x56430299dc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.094+0000 7ff2dffff640 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302999140 0x56430299dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.095+0000 7ff2e498b640 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302999140 0x56430299dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.095+0000 7ff2e498b640 1 -- 172.21.15.64:0/3122152176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643026083f0 con 0x564302998750 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.095+0000 7ff2dffff640 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302999140 0x56430299dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.095+0000 7ff2e498b640 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 0x564302998b70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8010370 tx=0x7ff2d8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bab88d4269c6417 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.095+0000 7ff2dd7fa640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2d60 0x7ff2c80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.095+0000 7ff2c67fc640 1 -- 172.21.15.64:0/3122152176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8016070 con 0x564302998750 2024-05-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.096+0000 7ff2dd7fa640 1 -- 172.21.15.64:0/2928522474 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2d40cf910 con 0x7ff2e010c260 2024-05-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.096+0000 7ff2c67fc640 1 -- 172.21.15.64:0/3122152176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d8012bc0 con 0x564302998750 2024-05-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.096+0000 7ff2dffff640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2d60 0x7ff2c80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.097+0000 7ff2e91da740 1 -- 172.21.15.64:0/3122152176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 msgr2=0x564302998b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.097+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 0x564302998b70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8010370 tx=0x7ff2d8010830 comp rx=0 tx=0).stop 2024-05-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.097+0000 7ff2c67fc640 1 -- 172.21.15.64:0/3122152176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8011c70 con 0x564302998750 2024-05-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 -- 172.21.15.64:0/3122152176 shutdown_connections 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2dffff640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2d60 0x7ff2c80a5220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40a44c0 tx=0x7ff2d4097900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302999140 0x56430299dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3122152176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302998750 0x564302998b70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 -- 172.21.15.64:0/3122152176 >> 172.21.15.64:0/3122152176 conn(0x564302993dc0 msgr2=0x5643029941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 -- 172.21.15.64:0/3122152176 shutdown_connections 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.098+0000 7ff2e91da740 1 -- 172.21.15.64:0/3122152176 wait complete. 2024-05-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.099+0000 7ff2e91da740 1 -- start start 2024-05-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.099+0000 7ff2e91da740 1 Processor -- start 2024-05-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43404/0 (socket says 172.21.15.64:43404) 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e498b640 1 -- 172.21.15.64:0/2909351059 learned_addr learned my addr 172.21.15.64:0/2909351059 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643029a12b0 con 0x564302999140 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5643028afc40 con 0x564302998750 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5643029a1430 con 0x56430299e2b0 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.100+0000 7ff2e91da740 10 client.0 _ll_register_callbacks cb 0x564302607be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 -- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 msgr2=0x5643029aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 -- 172.21.15.64:0/2909351059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 msgr2=0x5643029a3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029a3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643026d27d0 con 0x564302998750 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2e498b640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80099d0 tx=0x7ff2d8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34737c0e41de00cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.101+0000 7ff2bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/2909351059 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8023040 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/2909351059 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2d8011bb0 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d8012bc0 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/2909351059 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff2d8016040 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2e91da740 10 client.26573 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5643029ad360 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/2909351059 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d8012bc0 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.102+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/2909351059 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7ff2d8029070 con 0x564302998750 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2e91da740 10 client.26573 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2e91da740 10 client.26573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2e91da740 20 client.26573 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2e91da740 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5643029ae840 0x5643029b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2e91da740 4 client.26573 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564302902370 con 0x5643029ae840 2024-05-09T09:15:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.103+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5643029ae840 0x5643029b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.104+0000 7ff2bf7fe640 10 client.26573 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.104+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5643029ae840 0x5643029b0c80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1ceb5a28cb55ba3 server_cookie=5b09d8f65ae03ebc in_seq=0 out_seq=0 2024-05-09T09:15:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.106+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/2909351059 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564302902370 con 0x5643029ae840 2024-05-09T09:15:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.106+0000 7ff2bf7fe640 10 client.26573 handle_command_reply: tid=0 2024-05-09T09:15:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.106+0000 7ff2e91da740 1 client.26573 shutdown 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 msgr2=0x5643029a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80099d0 tx=0x7ff2d8009920 comp rx=0 tx=0).stop 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 shutdown_connections 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5643029ae840 0x5643029b0c80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 --2- 172.21.15.64:0/2909351059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2b00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 >> 172.21.15.64:0/2909351059 conn(0x564302993dc0 msgr2=0x56430298a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2bf7fe640 0 client.26573 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 shutdown_connections 2024-05-09T09:15:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.107+0000 7ff2e91da740 1 -- 172.21.15.64:0/2909351059 wait complete. 2024-05-09T09:15:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.108+0000 7ff2e91da740 20 client.26573 trim_cache size 0 max 16384 2024-05-09T09:15:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.142+0000 7ff2e91da740 1 Processor -- start 2024-05-09T09:15:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.142+0000 7ff2e91da740 1 -- start start 2024-05-09T09:15:17.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029aa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43416/0 (socket says 172.21.15.64:43416) 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e91da740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b33a0 con 0x564302999140 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e91da740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b0bb0 con 0x564302998750 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e91da740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01aea20 con 0x56430299e2b0 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2e498b640 1 -- 172.21.15.64:0/4094945558 learned_addr learned my addr 172.21.15.64:0/4094945558 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.143+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.144+0000 7ff2df7fe640 1 -- 172.21.15.64:0/4094945558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 msgr2=0x5643029a2c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.144+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a2c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.144+0000 7ff2df7fe640 1 -- 172.21.15.64:0/4094945558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 msgr2=0x5643029a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.144+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.144+0000 7ff2df7fe640 1 -- 172.21.15.64:0/4094945558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56430297cd20 con 0x564302999140 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.145+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029aa040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff2d002a930 tx=0x7ff2d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea44002b03c1cfdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.145+0000 7ff2c5ffb640 1 -- 172.21.15.64:0/4094945558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0035040 con 0x564302999140 2024-05-09T09:15:17.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.145+0000 7ff2c5ffb640 1 -- 172.21.15.64:0/4094945558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564302902370 con 0x564302999140 2024-05-09T09:15:17.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.145+0000 7ff2c5ffb640 1 -- 172.21.15.64:0/4094945558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0044810 con 0x564302999140 2024-05-09T09:15:17.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.146+0000 7ff2e91da740 1 -- 172.21.15.64:0/4094945558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 msgr2=0x5643029aa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.146+0000 7ff2e91da740 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029aa040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff2d002a930 tx=0x7ff2d0009260 comp rx=0 tx=0).stop 2024-05-09T09:15:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.147+0000 7ff2e91da740 1 -- 172.21.15.64:0/4094945558 shutdown_connections 2024-05-09T09:15:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.147+0000 7ff2e91da740 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56430299e2b0 0x5643029a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.147+0000 7ff2e91da740 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564302999140 0x5643029aa040 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.147+0000 7ff2e91da740 1 --2- 172.21.15.64:0/4094945558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.147+0000 7ff2e91da740 1 -- 172.21.15.64:0/4094945558 >> 172.21.15.64:0/4094945558 conn(0x564302993dc0 msgr2=0x56430299a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.149+0000 7ff2e91da740 1 -- 172.21.15.64:0/4094945558 shutdown_connections 2024-05-09T09:15:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.150+0000 7ff2e91da740 1 -- 172.21.15.64:0/4094945558 wait complete. 2024-05-09T09:15:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.150+0000 7ff2e91da740 1 -- start start 2024-05-09T09:15:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.151+0000 7ff2e91da740 1 Processor -- start 2024-05-09T09:15:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.151+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.151+0000 7ff2e91da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564302999140 0x5643029a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.152+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.152+0000 7ff2e498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43422/0 (socket says 172.21.15.64:43422) 2024-05-09T09:15:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.152+0000 7ff2e498b640 1 -- 172.21.15.64:0/3080507029 learned_addr learned my addr 172.21.15.64:0/3080507029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.152+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564302999140 0x5643029a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.152+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56430299e2b0 0x5643029a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.152+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01af070 con 0x56430299e2b0 2024-05-09T09:15:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.153+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b04e0 con 0x564302998750 2024-05-09T09:15:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.153+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01af740 con 0x564302999140 2024-05-09T09:15:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.153+0000 7ff2e91da740 10 client.0 _ll_register_callbacks cb 0x7ff2d8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:17.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:16 smithi064 ceph-mon[30149]: pgmap v554: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 935 op/s 2024-05-09T09:15:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.154+0000 7ff2df7fe640 1 -- 172.21.15.64:0/3080507029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 msgr2=0x5643029a76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.154+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.154+0000 7ff2df7fe640 1 -- 172.21.15.64:0/3080507029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56430299e2b0 msgr2=0x5643029a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.154+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56430299e2b0 0x5643029a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.154+0000 7ff2df7fe640 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643029a9210 con 0x564302999140 2024-05-09T09:15:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.155+0000 7ff2df7fe640 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564302999140 0x5643029a7c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009230 tx=0x7ff2d00051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e544afa46d5d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.155+0000 7ff2bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.155+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/3080507029 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0035040 con 0x564302999140 2024-05-09T09:15:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.155+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/3080507029 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff2d00046c0 con 0x564302999140 2024-05-09T09:15:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.155+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564302902370 con 0x564302999140 2024-05-09T09:15:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.156+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/3080507029 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff2d00048c0 con 0x564302999140 2024-05-09T09:15:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.158+0000 7ff2e91da740 10 client.26758 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.158+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564302a14dc0 con 0x564302999140 2024-05-09T09:15:17.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/3080507029 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d0044590 con 0x564302999140 2024-05-09T09:15:17.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/3080507029 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7ff2d0055030 con 0x564302999140 2024-05-09T09:15:17.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2e91da740 10 client.26758 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:17.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2e91da740 10 client.26758 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:15:17.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2e91da740 20 client.26758 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:17.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564302a15700 0x564302a17b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2e91da740 4 client.26758 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:15:17.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.159+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564302902370 con 0x564302a15700 2024-05-09T09:15:17.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.160+0000 7ff2e498b640 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564302a15700 0x564302a17b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.161+0000 7ff2bf7fe640 10 client.26758 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:17.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.161+0000 7ff2e498b640 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564302a15700 0x564302a17b20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac6a94ea6105c955 server_cookie=978d665e2a4459d1 in_seq=0 out_seq=0 2024-05-09T09:15:17.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.161+0000 7ff2bf7fe640 1 -- 172.21.15.64:0/3080507029 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564302902370 con 0x564302a15700 2024-05-09T09:15:17.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.161+0000 7ff2bf7fe640 10 client.26758 handle_command_reply: tid=0 2024-05-09T09:15:17.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.161+0000 7ff2e91da740 1 client.26758 shutdown 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.163+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564302999140 msgr2=0x5643029a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.163+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564302999140 0x5643029a7c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009230 tx=0x7ff2d00051d0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 shutdown_connections 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564302a15700 0x564302a17b20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56430299e2b0 0x5643029a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2bf7fe640 0 client.26758 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564302999140 0x5643029a7c10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2e91da740 1 --2- 172.21.15.64:0/3080507029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564302998750 0x5643029a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.164+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 >> 172.21.15.64:0/3080507029 conn(0x564302993dc0 msgr2=0x5643029a2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.165+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 shutdown_connections 2024-05-09T09:15:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.165+0000 7ff2e91da740 1 -- 172.21.15.64:0/3080507029 wait complete. 2024-05-09T09:15:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.165+0000 7ff2e91da740 20 client.26758 trim_cache size 0 max 16384 2024-05-09T09:15:17.165 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:15:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2d60 msgr2=0x7ff2c80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2d60 0x7ff2c80a5220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40a44c0 tx=0x7ff2d4097900 comp rx=0 tx=0).stop 2024-05-09T09:15:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 msgr2=0x7ff2e01a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e01a2500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e9c0 tx=0x7ff2d400ee80 comp rx=0 tx=0).stop 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 shutdown_connections 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff2c80a2d60 0x7ff2c80a5220 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2e010de10 0x7ff2e01a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e010d450 0x7ff2e01a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 --2- 172.21.15.64:0/2928522474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2e010c260 0x7ff2e01a2500 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.167+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 >> 172.21.15.64:0/2928522474 conn(0x7ff2e006c2c0 msgr2=0x7ff2e0110870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.168+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 shutdown_connections 2024-05-09T09:15:17.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.168+0000 7ff2e63ee640 1 -- 172.21.15.64:0/2928522474 wait complete. 2024-05-09T09:15:17.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:15:17.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:16 smithi179 ceph-mon[34086]: pgmap v554: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 935 op/s 2024-05-09T09:15:17.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:16 smithi139 ceph-mon[33599]: pgmap v554: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 935 op/s 2024-05-09T09:15:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.395+0000 7fbd785c1640 1 -- 172.21.15.64:0/2518010472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 msgr2=0x7fbd7010b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.395+0000 7fbd785c1640 1 --2- 172.21.15.64:0/2518010472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 0x7fbd7010b600 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009900 tx=0x7fbd6c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.396+0000 7fbd785c1640 1 -- 172.21.15.64:0/2518010472 shutdown_connections 2024-05-09T09:15:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.396+0000 7fbd785c1640 1 --2- 172.21.15.64:0/2518010472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70111f70 0x7fbd701143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.396+0000 7fbd785c1640 1 --2- 172.21.15.64:0/2518010472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 0x7fbd7010b600 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.396+0000 7fbd785c1640 1 --2- 172.21.15.64:0/2518010472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd70108830 0x7fbd7010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.396+0000 7fbd785c1640 1 -- 172.21.15.64:0/2518010472 >> 172.21.15.64:0/2518010472 conn(0x7fbd7006ef60 msgr2=0x7fbd700713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.397+0000 7fbd785c1640 1 -- 172.21.15.64:0/2518010472 shutdown_connections 2024-05-09T09:15:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.398+0000 7fbd785c1640 1 -- 172.21.15.64:0/2518010472 wait complete. 2024-05-09T09:15:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.398+0000 7fbd785c1640 1 Processor -- start 2024-05-09T09:15:17.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 -- start start 2024-05-09T09:15:17.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 0x7fbd701a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 0x7fbd701a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd70111f70 0x7fbd701a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70117d70 con 0x7fbd7010b1a0 2024-05-09T09:15:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70117bf0 con 0x7fbd70111f70 2024-05-09T09:15:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.401+0000 7fbd785c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70117ef0 con 0x7fbd70108830 2024-05-09T09:15:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 0x7fbd701a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 0x7fbd701a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37390/0 (socket says 172.21.15.64:37390) 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 -- 172.21.15.64:0/3118996152 learned_addr learned my addr 172.21.15.64:0/3118996152 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd75b5c640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 0x7fbd701a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 -- 172.21.15.64:0/3118996152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd70111f70 msgr2=0x7fbd701a9aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd70111f70 0x7fbd701a9aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 -- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 msgr2=0x7fbd701a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 0x7fbd701a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.402+0000 7fbd7635d640 1 -- 172.21.15.64:0/3118996152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd701aa110 con 0x7fbd70108830 2024-05-09T09:15:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.403+0000 7fbd7635d640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 0x7fbd701a24f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800cce0 tx=0x7fbd6800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.403+0000 7fbd677fe640 1 -- 172.21.15.64:0/3118996152 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800f650 con 0x7fbd70108830 2024-05-09T09:15:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.403+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd701aa400 con 0x7fbd70108830 2024-05-09T09:15:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.403+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd701aa9a0 con 0x7fbd70108830 2024-05-09T09:15:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.403+0000 7fbd677fe640 1 -- 172.21.15.64:0/3118996152 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd6800fca0 con 0x7fbd70108830 2024-05-09T09:15:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.403+0000 7fbd677fe640 1 -- 172.21.15.64:0/3118996152 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68018bb0 con 0x7fbd70108830 2024-05-09T09:15:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.408+0000 7fbd7b3ab740 1 Processor -- start 2024-05-09T09:15:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.409+0000 7fbd677fe640 1 -- 172.21.15.64:0/3118996152 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbd680213f0 con 0x7fbd70108830 2024-05-09T09:15:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.409+0000 7fbd7b3ab740 1 -- start start 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 0x55b5d753ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7542740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7b3ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b5d7496c40 con 0x55b5d75402b0 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7b3ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b5d743e580 con 0x55b5d753a750 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7b3ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b5d7492c00 con 0x55b5d753b140 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd76b5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 0x55b5d753ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd76b5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 0x55b5d753ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43446/0 (socket says 172.21.15.64:43446) 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd76b5e640 1 -- 172.21.15.64:0/4015073114 learned_addr learned my addr 172.21.15.64:0/4015073114 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.410+0000 7fbd7635d640 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd76b5e640 1 -- 172.21.15.64:0/4015073114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 msgr2=0x55b5d753fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd76b5e640 1 -- 172.21.15.64:0/4015073114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 msgr2=0x55b5d7542740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7542740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd76b5e640 1 -- 172.21.15.64:0/4015073114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d71aa3f0 con 0x55b5d753a750 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 0x55b5d753ab70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b980 tx=0x7fbd6000be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a17d0277a7cab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.411+0000 7fbd64bfe640 1 -- 172.21.15.64:0/4015073114 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60016070 con 0x55b5d753a750 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.412+0000 7fbd64bfe640 1 -- 172.21.15.64:0/4015073114 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd600054d0 con 0x55b5d753a750 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.412+0000 7fbd64bfe640 1 -- 172.21.15.64:0/4015073114 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60004c40 con 0x55b5d753a750 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.412+0000 7fbd677fe640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd5c0a2c90 0x7fbd5c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.413+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/4015073114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 msgr2=0x55b5d753ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.413+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 0x55b5d753ab70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b980 tx=0x7fbd6000be40 comp rx=0 tx=0).stop 2024-05-09T09:15:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.413+0000 7fbd7635d640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd5c0a2c90 0x7fbd5c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.413+0000 7fbd677fe640 1 -- 172.21.15.64:0/3118996152 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbd680cf800 con 0x7fbd70108830 2024-05-09T09:15:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.413+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/4015073114 shutdown_connections 2024-05-09T09:15:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.413+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7542740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.414+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.414+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/4015073114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753a750 0x55b5d753ab70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.414+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/4015073114 >> 172.21.15.64:0/4015073114 conn(0x55b5d752d5b0 msgr2=0x55b5d752d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.414+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/4015073114 shutdown_connections 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.414+0000 7fbd7635d640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd5c0a2c90 0x7fbd5c0a5150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbd68018760 tx=0x7fbd680d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.414+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/4015073114 wait complete. 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.415+0000 7fbd7b3ab740 1 -- start start 2024-05-09T09:15:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.415+0000 7fbd7b3ab740 1 Processor -- start 2024-05-09T09:15:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.416+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d75449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.416+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7544f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d754bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd7b3ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b5d7543260 con 0x55b5d75402b0 2024-05-09T09:15:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd76b5e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d75449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd76b5e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d75449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37398/0 (socket says 172.21.15.64:37398) 2024-05-09T09:15:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1302460746 learned_addr learned my addr 172.21.15.64:0/1302460746 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd75b5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7544f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b5d7451c40 con 0x55b5d753b140 2024-05-09T09:15:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b5d75433e0 con 0x55b5d753a750 2024-05-09T09:15:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd7b3ab740 10 client.0 _ll_register_callbacks cb 0x55b5d71a9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d754bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd75b5c640 1 -- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 msgr2=0x55b5d75449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd75b5c640 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d75449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd75b5c640 1 -- 172.21.15.64:0/1302460746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 msgr2=0x55b5d754bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd75b5c640 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d754bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.417+0000 7fbd75b5c640 1 -- 172.21.15.64:0/1302460746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d72747d0 con 0x55b5d753b140 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd75b5c640 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7544f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c006010 tx=0x7fbd6c0317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c12f86bd29b71ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1302460746 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c03f070 con 0x55b5d753b140 2024-05-09T09:15:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1302460746 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbd6c004450 con 0x55b5d753b140 2024-05-09T09:15:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd600054d0 con 0x55b5d753b140 2024-05-09T09:15:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1302460746 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbd6c04d050 con 0x55b5d753b140 2024-05-09T09:15:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd7b3ab740 10 client.26585 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.418+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5d754f310 con 0x55b5d753b140 2024-05-09T09:15:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.419+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1302460746 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd600054d0 con 0x55b5d753b140 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.419+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1302460746 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fbd6c004a80 con 0x55b5d753b140 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.419+0000 7fbd7b3ab740 10 client.26585 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.419+0000 7fbd7b3ab740 10 client.26585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.419+0000 7fbd7b3ab740 20 client.26585 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.420+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d7552ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.420+0000 7fbd7b3ab740 4 client.26585 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.420+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d74a4370 con 0x55b5d7550aa0 2024-05-09T09:15:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.421+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d7552ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.422+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d7552ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=732300b4317c0e60 server_cookie=b7d9b4100c150666 in_seq=0 out_seq=0 2024-05-09T09:15:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.422+0000 7fbd59ffb640 10 client.26585 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.422+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1302460746 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5d74a4370 con 0x55b5d7550aa0 2024-05-09T09:15:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.422+0000 7fbd59ffb640 10 client.26585 handle_command_reply: tid=0 2024-05-09T09:15:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.422+0000 7fbd7b3ab740 1 client.26585 shutdown 2024-05-09T09:15:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.423+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 msgr2=0x55b5d7544f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.423+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7544f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c006010 tx=0x7fbd6c0317b0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.423+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 shutdown_connections 2024-05-09T09:15:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.423+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d7552ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd59ffb640 0 client.26585 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d754bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7544f30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1302460746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d75449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 >> 172.21.15.64:0/1302460746 conn(0x55b5d752d5b0 msgr2=0x55b5d752c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 shutdown_connections 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.424+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1302460746 wait complete. 2024-05-09T09:15:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.425+0000 7fbd7b3ab740 20 client.26585 trim_cache size 0 max 16384 2024-05-09T09:15:17.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 Processor -- start 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 -- start start 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d753a750 0x55b5d753c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d75402b0 0x55b5d7543cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd76b5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d753a750 0x55b5d753c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd76b5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d753a750 0x55b5d753c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42586/0 (socket says 172.21.15.64:42586) 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1728389643 learned_addr learned my addr 172.21.15.64:0/1728389643 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54012550 con 0x55b5d753a750 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5400fd60 con 0x55b5d75402b0 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5400dbb0 con 0x55b5d753b140 2024-05-09T09:15:17.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.454+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d75402b0 0x55b5d7543cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.455+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1728389643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 msgr2=0x55b5d753c860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.455+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753c860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.455+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1728389643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d753a750 msgr2=0x55b5d753c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.455+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d753a750 0x55b5d753c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.455+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1728389643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d7545890 con 0x55b5d75402b0 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.455+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d75402b0 0x55b5d7543cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b5e0 tx=0x7fbd6000b510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8b81e3e1cd96b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.456+0000 7fbd5bfff640 1 -- 172.21.15.64:0/1728389643 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60026040 con 0x55b5d75402b0 2024-05-09T09:15:17.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.456+0000 7fbd5bfff640 1 -- 172.21.15.64:0/1728389643 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd60029720 con 0x55b5d75402b0 2024-05-09T09:15:17.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.456+0000 7fbd5bfff640 1 -- 172.21.15.64:0/1728389643 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60016040 con 0x55b5d75402b0 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.457+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d75402b0 msgr2=0x55b5d7543cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.457+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d75402b0 0x55b5d7543cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b5e0 tx=0x7fbd6000b510 comp rx=0 tx=0).stop 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.457+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 shutdown_connections 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.457+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d75402b0 0x55b5d7543cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.457+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753b140 0x55b5d753c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.458+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1728389643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d753a750 0x55b5d753c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.458+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 >> 172.21.15.64:0/1728389643 conn(0x55b5d752d5b0 msgr2=0x55b5d7528680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.458+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 shutdown_connections 2024-05-09T09:15:17.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.458+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1728389643 wait complete. 2024-05-09T09:15:17.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.459+0000 7fbd7b3ab740 1 -- start start 2024-05-09T09:15:17.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.459+0000 7fbd7b3ab740 1 Processor -- start 2024-05-09T09:15:17.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.459+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d7548a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7548f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd7b3ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7549f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd75b5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7548f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd75b5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7548f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43476/0 (socket says 172.21.15.64:43476) 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd75b5c640 1 -- 172.21.15.64:0/1589456085 learned_addr learned my addr 172.21.15.64:0/1589456085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5400e220 con 0x55b5d75402b0 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5400f690 con 0x55b5d753b140 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5400e8f0 con 0x55b5d753a750 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.460+0000 7fbd7b3ab740 10 client.0 _ll_register_callbacks cb 0x7fbd6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:17.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d7548a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7549f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1589456085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 msgr2=0x55b5d7548f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7548f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1589456085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 msgr2=0x55b5d7549f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7549f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.461+0000 7fbd76b5e640 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d753be10 con 0x55b5d753a750 2024-05-09T09:15:17.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.462+0000 7fbd76b5e640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d7548a30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbd60000c00 tx=0x7fbd60005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e77d429c0354375 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.462+0000 7fbd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:17.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.463+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1589456085 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60016070 con 0x55b5d753a750 2024-05-09T09:15:17.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.463+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1589456085 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbd60010460 con 0x55b5d753a750 2024-05-09T09:15:17.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.463+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd60029720 con 0x55b5d753a750 2024-05-09T09:15:17.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.463+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1589456085 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbd60010660 con 0x55b5d753a750 2024-05-09T09:15:17.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.464+0000 7fbd7b3ab740 10 client.26773 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:17.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.464+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5d754b740 con 0x55b5d753a750 2024-05-09T09:15:17.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1589456085 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd60029720 con 0x55b5d753a750 2024-05-09T09:15:17.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1589456085 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fbd60035730 con 0x55b5d753a750 2024-05-09T09:15:17.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd7b3ab740 10 client.26773 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:17.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd7b3ab740 10 client.26773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:15:17.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd7b3ab740 20 client.26773 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:17.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d75b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd7b3ab740 4 client.26773 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:15:17.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.465+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5d749e6c0 con 0x55b5d7550aa0 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.466+0000 7fbd75b5c640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d75b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.466+0000 7fbd59ffb640 10 client.26773 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.466+0000 7fbd75b5c640 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d75b9340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5c37d4af3a3567f server_cookie=21be2d869245a24a in_seq=0 out_seq=0 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd59ffb640 1 -- 172.21.15.64:0/1589456085 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5d749e6c0 con 0x55b5d7550aa0 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd59ffb640 10 client.26773 handle_command_reply: tid=0 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd7b3ab740 1 client.26773 shutdown 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 msgr2=0x55b5d7548a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d7548a30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbd60000c00 tx=0x7fbd60005700 comp rx=0 tx=0).stop 2024-05-09T09:15:17.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 shutdown_connections 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55b5d7550aa0 0x55b5d75b9340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.467+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5d75402b0 0x55b5d7549f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b5d753b140 0x55b5d7548f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd59ffb640 0 client.26773 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd7b3ab740 1 --2- 172.21.15.64:0/1589456085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5d753a750 0x55b5d7548a30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 >> 172.21.15.64:0/1589456085 conn(0x55b5d752d5b0 msgr2=0x55b5d75449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 shutdown_connections 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd7b3ab740 1 -- 172.21.15.64:0/1589456085 wait complete. 2024-05-09T09:15:17.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.468+0000 7fbd7b3ab740 20 client.26773 trim_cache size 0 max 16384 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd5c0a2c90 msgr2=0x7fbd5c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd5c0a2c90 0x7fbd5c0a5150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbd68018760 tx=0x7fbd680d5530 comp rx=0 tx=0).stop 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 msgr2=0x7fbd701a24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 0x7fbd701a24f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800cce0 tx=0x7fbd6800a620 comp rx=0 tx=0).stop 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 shutdown_connections 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd5c0a2c90 0x7fbd5c0a5150 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd70111f70 0x7fbd701a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd7010b1a0 0x7fbd701a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 --2- 172.21.15.64:0/3118996152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd70108830 0x7fbd701a24f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.469+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 >> 172.21.15.64:0/3118996152 conn(0x7fbd7006ef60 msgr2=0x7fbd70109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.471+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 shutdown_connections 2024-05-09T09:15:17.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.471+0000 7fbd785c1640 1 -- 172.21.15.64:0/3118996152 wait complete. 2024-05-09T09:15:17.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:15:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 -- 172.21.15.64:0/3759536529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404101960 msgr2=0x7f2404101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 --2- 172.21.15.64:0/3759536529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404101960 0x7f2404101de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009ca0 tx=0x7f23fc02c7d0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 -- 172.21.15.64:0/3759536529 shutdown_connections 2024-05-09T09:15:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 --2- 172.21.15.64:0/3759536529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404102320 0x7f2404108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 --2- 172.21.15.64:0/3759536529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404101960 0x7f2404101de0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 --2- 172.21.15.64:0/3759536529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2404100770 0x7f2404100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.696+0000 7f240bc79640 1 -- 172.21.15.64:0/3759536529 >> 172.21.15.64:0/3759536529 conn(0x7f24040fbce0 msgr2=0x7f24040fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.700+0000 7f240bc79640 1 -- 172.21.15.64:0/3759536529 shutdown_connections 2024-05-09T09:15:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.703+0000 7f240bc79640 1 -- 172.21.15.64:0/3759536529 wait complete. 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 Processor -- start 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 -- start start 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2404100770 0x7f2404190db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404101960 0x7f24041912f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404102320 0x7f2404198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f240410b2d0 con 0x7f2404101960 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f240410b150 con 0x7f2404102320 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f240bc79640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f240410b450 con 0x7f2404100770 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f2409214640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404101960 0x7f24041912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f2409214640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404101960 0x7f24041912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42612/0 (socket says 172.21.15.64:42612) 2024-05-09T09:15:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.704+0000 7f2409214640 1 -- 172.21.15.64:0/2216688276 learned_addr learned my addr 172.21.15.64:0/2216688276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404102320 0x7f2404198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 -- 172.21.15.64:0/2216688276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2404100770 msgr2=0x7f2404190db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:15:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2404100770 0x7f2404190db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 -- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404101960 msgr2=0x7f24041912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404101960 0x7f24041912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 -- 172.21.15.64:0/2216688276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24041989d0 con 0x7f2404102320 2024-05-09T09:15:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.705+0000 7f240a216640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404102320 0x7f2404198360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b410 tx=0x7f23f400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.706+0000 7f23faffd640 1 -- 172.21.15.64:0/2216688276 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4015020 con 0x7f2404102320 2024-05-09T09:15:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.706+0000 7f23faffd640 1 -- 172.21.15.64:0/2216688276 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23f4011ce0 con 0x7f2404102320 2024-05-09T09:15:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.706+0000 7f23faffd640 1 -- 172.21.15.64:0/2216688276 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4010dc0 con 0x7f2404102320 2024-05-09T09:15:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.706+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2404198cc0 con 0x7f2404102320 2024-05-09T09:15:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.706+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041aef50 con 0x7f2404102320 2024-05-09T09:15:17.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.710+0000 7f240e983740 1 Processor -- start 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.710+0000 7f240e983740 1 -- start start 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.711+0000 7f240e983740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708543f40 0x55a708544360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.711+0000 7f240e983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708544930 0x55a708549430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.711+0000 7f240e983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708549aa0 0x55a70854bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.711+0000 7f240e983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a7084a8c40 con 0x55a708544930 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.711+0000 7f240e983740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a708450580 con 0x55a708543f40 2024-05-09T09:15:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.711+0000 7f240e983740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a7084a4c00 con 0x55a708549aa0 2024-05-09T09:15:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.713+0000 7f2409214640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708543f40 0x55a708544360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.713+0000 7f2409214640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708543f40 0x55a708544360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43522/0 (socket says 172.21.15.64:43522) 2024-05-09T09:15:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.713+0000 7f2409214640 1 -- 172.21.15.64:0/1077675503 learned_addr learned my addr 172.21.15.64:0/1077675503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.713+0000 7f2409214640 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708549aa0 0x55a70854bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.713+0000 7f2409a15640 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708544930 0x55a708549430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.713+0000 7f2409a15640 1 -- 172.21.15.64:0/1077675503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708549aa0 msgr2=0x55a70854bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f2409a15640 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708549aa0 0x55a70854bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f2409a15640 1 -- 172.21.15.64:0/1077675503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708543f40 msgr2=0x55a708544360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f2409a15640 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708543f40 0x55a708544360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f2409a15640 1 -- 172.21.15.64:0/1077675503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7081bc3f0 con 0x55a708544930 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f2409214640 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708549aa0 0x55a70854bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f2409a15640 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708544930 0x55a708549430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f24041017c0 tx=0x7f2400009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283593b70a14048 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.714+0000 7f23faffd640 1 -- 172.21.15.64:0/2216688276 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23f4018020 con 0x7f2404102320 2024-05-09T09:15:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.715+0000 7f23effff640 1 -- 172.21.15.64:0/1077675503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400015020 con 0x55a708544930 2024-05-09T09:15:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.715+0000 7f23effff640 1 -- 172.21.15.64:0/1077675503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f240000bd70 con 0x55a708544930 2024-05-09T09:15:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.715+0000 7f23faffd640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23f00a2d90 0x7f23f00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.716+0000 7f23faffd640 1 -- 172.21.15.64:0/2216688276 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23f40cebb0 con 0x7f2404102320 2024-05-09T09:15:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.716+0000 7f240e983740 1 -- 172.21.15.64:0/1077675503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708544930 msgr2=0x55a708549430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.716+0000 7f240e983740 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708544930 0x55a708549430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f24041017c0 tx=0x7f2400009b20 comp rx=0 tx=0).stop 2024-05-09T09:15:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.716+0000 7f2409214640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23f00a2d90 0x7f23f00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 -- 172.21.15.64:0/1077675503 shutdown_connections 2024-05-09T09:15:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708549aa0 0x55a70854bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708544930 0x55a708549430 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 --2- 172.21.15.64:0/1077675503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708543f40 0x55a708544360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 -- 172.21.15.64:0/1077675503 >> 172.21.15.64:0/1077675503 conn(0x55a70853f5b0 msgr2=0x55a70853f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 -- 172.21.15.64:0/1077675503 shutdown_connections 2024-05-09T09:15:17.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.717+0000 7f240e983740 1 -- 172.21.15.64:0/1077675503 wait complete. 2024-05-09T09:15:17.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.718+0000 7f2409214640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23f00a2d90 0x7f23f00a5250 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55a708545190 tx=0x7f23fc036f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.718+0000 7f240e983740 1 -- start start 2024-05-09T09:15:17.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.718+0000 7f240e983740 1 Processor -- start 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708543f40 0x55a70854e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 0x55a70854e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708549aa0 0x55a708555890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a70854caf0 con 0x55a708549aa0 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a708463c40 con 0x55a708544930 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a70854cc70 con 0x55a708543f40 2024-05-09T09:15:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.719+0000 7f240e983740 10 client.0 _ll_register_callbacks cb 0x55a7081bbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:17.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.720+0000 7f2409214640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 0x55a70854e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.721+0000 7f2409214640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 0x55a70854e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43530/0 (socket says 172.21.15.64:43530) 2024-05-09T09:15:17.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.721+0000 7f2409214640 1 -- 172.21.15.64:0/332428006 learned_addr learned my addr 172.21.15.64:0/332428006 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f240a216640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708549aa0 0x55a708555890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f2409a15640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708543f40 0x55a70854e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f2409214640 1 -- 172.21.15.64:0/332428006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708543f40 msgr2=0x55a70854e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f2409214640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708543f40 0x55a70854e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f2409214640 1 -- 172.21.15.64:0/332428006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708549aa0 msgr2=0x55a708555890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f2409214640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708549aa0 0x55a708555890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.722+0000 7f2409214640 1 -- 172.21.15.64:0/332428006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7082867d0 con 0x55a708544930 2024-05-09T09:15:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.723+0000 7f2409214640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 0x55a70854e850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00d3f0 tx=0x7f23fc036f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15d75bb18457c2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.723+0000 7f23ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f23ed7fa640 1 -- 172.21.15.64:0/332428006 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc046070 con 0x55a708544930 2024-05-09T09:15:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f23ed7fa640 1 -- 172.21.15.64:0/332428006 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23fc04d470 con 0x55a708544930 2024-05-09T09:15:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240000bd70 con 0x55a708544930 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f23ed7fa640 1 -- 172.21.15.64:0/332428006 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f23fc05b050 con 0x55a708544930 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f240e983740 10 client.26603 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a708558c30 con 0x55a708544930 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.724+0000 7f23ed7fa640 1 -- 172.21.15.64:0/332428006 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f240000bd70 con 0x55a708544930 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f23ed7fa640 1 -- 172.21.15.64:0/332428006 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f23fc00b040 con 0x55a708544930 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f240e983740 10 client.26603 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f240e983740 10 client.26603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f240e983740 20 client.26603 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f240e983740 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a70855a3d0 0x55a70855c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f240e983740 4 client.26603 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7084b6370 con 0x55a70855a3d0 2024-05-09T09:15:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.725+0000 7f2409a15640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a70855a3d0 0x55a70855c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.726+0000 7f23ed7fa640 10 client.26603 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:17.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.726+0000 7f2409a15640 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a70855a3d0 0x55a70855c810 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb93b82d703123c server_cookie=cffc069f1640acca in_seq=0 out_seq=0 2024-05-09T09:15:17.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.727+0000 7f23ed7fa640 1 -- 172.21.15.64:0/332428006 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7084b6370 con 0x55a70855a3d0 2024-05-09T09:15:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.727+0000 7f23ed7fa640 10 client.26603 handle_command_reply: tid=0 2024-05-09T09:15:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.727+0000 7f240e983740 1 client.26603 shutdown 2024-05-09T09:15:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 msgr2=0x55a70854e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 0x55a70854e850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00d3f0 tx=0x7f23fc036f20 comp rx=0 tx=0).stop 2024-05-09T09:15:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 shutdown_connections 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a70855a3d0 0x55a70855c810 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708549aa0 0x55a708555890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708544930 0x55a70854e850 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 --2- 172.21.15.64:0/332428006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708543f40 0x55a70854e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f23ed7fa640 0 client.26603 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 >> 172.21.15.64:0/332428006 conn(0x55a70853f5b0 msgr2=0x55a70853e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 shutdown_connections 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 1 -- 172.21.15.64:0/332428006 wait complete. 2024-05-09T09:15:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.728+0000 7f240e983740 20 client.26603 trim_cache size 0 max 16384 2024-05-09T09:15:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.756+0000 7f240e983740 1 Processor -- start 2024-05-09T09:15:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.756+0000 7f240e983740 1 -- start start 2024-05-09T09:15:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.757+0000 7f240e983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a708542560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.757+0000 7f240e983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708555850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.757+0000 7f2409a15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a708542560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.757+0000 7f2409a15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a708542560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42638/0 (socket says 172.21.15.64:42638) 2024-05-09T09:15:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.757+0000 7f2409a15640 1 -- 172.21.15.64:0/3918464272 learned_addr learned my addr 172.21.15.64:0/3918464272 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.758+0000 7f240e983740 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a708542aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.758+0000 7f2409214640 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708555850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.758+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24041bbcf0 con 0x55a708543f40 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.758+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b9500 con 0x55a708549aa0 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.758+0000 7f240a216640 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a708542aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.758+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b7350 con 0x55a708544930 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.759+0000 7f2409214640 1 -- 172.21.15.64:0/3918464272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 msgr2=0x55a708542aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.759+0000 7f2409214640 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a708542aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.759+0000 7f2409214640 1 -- 172.21.15.64:0/3918464272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 msgr2=0x55a708542560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.759+0000 7f2409214640 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a708542560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.759+0000 7f2409214640 1 -- 172.21.15.64:0/3918464272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7085451c0 con 0x55a708544930 2024-05-09T09:15:17.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.762+0000 7f2409214640 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708555850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0328c0 tx=0x7f23fc02c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34989482481a6e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.763+0000 7f23ef7fe640 1 -- 172.21.15.64:0/3918464272 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc046070 con 0x55a708544930 2024-05-09T09:15:17.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.763+0000 7f23ef7fe640 1 -- 172.21.15.64:0/3918464272 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23fc052070 con 0x55a708544930 2024-05-09T09:15:17.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.763+0000 7f23ef7fe640 1 -- 172.21.15.64:0/3918464272 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc03a040 con 0x55a708544930 2024-05-09T09:15:17.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.765+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 msgr2=0x55a708555850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.765+0000 7f240e983740 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708555850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0328c0 tx=0x7f23fc02c7d0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 shutdown_connections 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a708542aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708555850 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 --2- 172.21.15.64:0/3918464272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a708542560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 >> 172.21.15.64:0/3918464272 conn(0x55a70853f5b0 msgr2=0x55a708545bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 shutdown_connections 2024-05-09T09:15:17.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.766+0000 7f240e983740 1 -- 172.21.15.64:0/3918464272 wait complete. 2024-05-09T09:15:17.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.767+0000 7f240e983740 1 -- start start 2024-05-09T09:15:17.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.767+0000 7f240e983740 1 Processor -- start 2024-05-09T09:15:17.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240e983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a7085527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240e983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708552d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f2409a15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a7085527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f2409a15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a7085527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42652/0 (socket says 172.21.15.64:42652) 2024-05-09T09:15:17.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f2409a15640 1 -- 172.21.15.64:0/3194318254 learned_addr learned my addr 172.21.15.64:0/3194318254 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a7085cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b79c0 con 0x55a708543f40 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b8e30 con 0x55a708549aa0 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b8090 con 0x55a708544930 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.768+0000 7f240a216640 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a7085cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240e983740 10 client.0 _ll_register_callbacks cb 0x7f23fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240a216640 1 -- 172.21.15.64:0/3194318254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 msgr2=0x55a708552d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240a216640 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708552d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240a216640 1 -- 172.21.15.64:0/3194318254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 msgr2=0x55a7085527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240a216640 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a7085527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240a216640 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7085cff00 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f240a216640 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a7085cf990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f23f40cc2c0 tx=0x7f23f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcadd6003ec5907f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.769+0000 7f23ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.770+0000 7f23ed7fa640 1 -- 172.21.15.64:0/3194318254 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f409f3d0 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.770+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc052070 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.770+0000 7f23ed7fa640 1 -- 172.21.15.64:0/3194318254 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f23f4097690 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.770+0000 7f23ed7fa640 1 -- 172.21.15.64:0/3194318254 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f23f40ab050 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.770+0000 7f240e983740 10 client.26609 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.770+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a7085d1290 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f23ed7fa640 1 -- 172.21.15.64:0/3194318254 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23f40a3c90 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f23ed7fa640 1 -- 172.21.15.64:0/3194318254 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f23f4096d70 con 0x55a708549aa0 2024-05-09T09:15:17.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f240e983740 10 client.26609 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f240e983740 10 client.26609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f240e983740 20 client.26609 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a7085d1ca0 0x55a7085d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f240e983740 4 client.26609 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f23fc052070 con 0x55a7085d1ca0 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.771+0000 7f2409a15640 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a7085d1ca0 0x55a7085d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:17.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.772+0000 7f23ed7fa640 10 client.26609 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:17.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.772+0000 7f2409a15640 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a7085d1ca0 0x55a7085d40c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa6bd30990335b2 server_cookie=7d7b49295c152b74 in_seq=0 out_seq=0 2024-05-09T09:15:17.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.773+0000 7f23ed7fa640 1 -- 172.21.15.64:0/3194318254 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f23fc052070 con 0x55a7085d1ca0 2024-05-09T09:15:17.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.773+0000 7f23ed7fa640 10 client.26609 handle_command_reply: tid=0 2024-05-09T09:15:17.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.773+0000 7f240e983740 1 client.26609 shutdown 2024-05-09T09:15:17.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.773+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 msgr2=0x55a7085cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.773+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a7085cf990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f23f40cc2c0 tx=0x7f23f40046c0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 shutdown_connections 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a7085d1ca0 0x55a7085d40c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a708549aa0 0x55a7085cf990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f23ed7fa640 0 client.26609 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a708544930 0x55a708552d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f240e983740 1 --2- 172.21.15.64:0/3194318254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a708543f40 0x55a7085527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.774+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 >> 172.21.15.64:0/3194318254 conn(0x55a70853f5b0 msgr2=0x55a70853df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.776+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 shutdown_connections 2024-05-09T09:15:17.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.776+0000 7f240e983740 1 -- 172.21.15.64:0/3194318254 wait complete. 2024-05-09T09:15:17.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.777+0000 7f240e983740 20 client.26609 trim_cache size 0 max 16384 2024-05-09T09:15:17.776 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23f00a2d90 msgr2=0x7f23f00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23f00a2d90 0x7f23f00a5250 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55a708545190 tx=0x7f23fc036f90 comp rx=0 tx=0).stop 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404102320 msgr2=0x7f2404198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404102320 0x7f2404198360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b410 tx=0x7f23f400b8d0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 shutdown_connections 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f23f00a2d90 0x7f23f00a5250 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2404102320 0x7f2404198360 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2404101960 0x7f24041912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 --2- 172.21.15.64:0/2216688276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2404100770 0x7f2404190db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:17.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.779+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 >> 172.21.15.64:0/2216688276 conn(0x7f24040fbce0 msgr2=0x7f2404104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:17.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.780+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 shutdown_connections 2024-05-09T09:15:17.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:17.780+0000 7f240bc79640 1 -- 172.21.15.64:0/2216688276 wait complete. 2024-05-09T09:15:18.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:17 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2761014170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:15:18.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:17 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2761014170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:15:18.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:17 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2761014170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:15:18.805 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:15:18.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:15:18.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.007+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/3299025565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b0101260 msgr2=0x7fc5b0105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.007+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/3299025565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b0101260 0x7fc5b0105b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009be0 tx=0x7fc5a802a6d0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/3299025565 shutdown_connections 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/3299025565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b0101260 0x7fc5b0105b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/3299025565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5b01008a0 0x7fc5b0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/3299025565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b00ffb40 0x7fc5b00fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/3299025565 >> 172.21.15.64:0/3299025565 conn(0x7fc5b00fb470 msgr2=0x7fc5b00fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/3299025565 shutdown_connections 2024-05-09T09:15:19.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.008+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/3299025565 wait complete. 2024-05-09T09:15:19.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.009+0000 7fc5b7fd5640 1 Processor -- start 2024-05-09T09:15:19.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.009+0000 7fc5b7fd5640 1 -- start start 2024-05-09T09:15:19.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.009+0000 7fc5b7fd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 0x7fc5b019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.009+0000 7fc5b7fd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b01008a0 0x7fc5b019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.010+0000 7fc5b7fd5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5b0101260 0x7fc5b01a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.010+0000 7fc5b7fd5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01139e0 con 0x7fc5b01008a0 2024-05-09T09:15:19.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.010+0000 7fc5b7fd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0113860 con 0x7fc5b00ffb40 2024-05-09T09:15:19.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.010+0000 7fc5b7fd5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0113b60 con 0x7fc5b0101260 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.011+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 0x7fc5b019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 0x7fc5b019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43572/0 (socket says 172.21.15.64:43572) 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1513247049 learned_addr learned my addr 172.21.15.64:0/1513247049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1513247049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5b0101260 msgr2=0x7fc5b01a5630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5b0101260 0x7fc5b01a5630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b01008a0 msgr2=0x7fc5b019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b01008a0 0x7fc5b019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.012+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1513247049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b01a5ca0 con 0x7fc5b00ffb40 2024-05-09T09:15:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.013+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 0x7fc5b019e080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00e9c0 tx=0x7fc5ac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.013+0000 7fc5a6ffd640 1 -- 172.21.15.64:0/1513247049 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00ccc0 con 0x7fc5b00ffb40 2024-05-09T09:15:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.013+0000 7fc5a6ffd640 1 -- 172.21.15.64:0/1513247049 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ac0054d0 con 0x7fc5b00ffb40 2024-05-09T09:15:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.013+0000 7fc5a6ffd640 1 -- 172.21.15.64:0/1513247049 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac010430 con 0x7fc5b00ffb40 2024-05-09T09:15:19.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.013+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b01a5f90 con 0x7fc5b00ffb40 2024-05-09T09:15:19.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.013+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a6400 con 0x7fc5b00ffb40 2024-05-09T09:15:19.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.017+0000 7fc5a6ffd640 1 -- 172.21.15.64:0/1513247049 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc5ac0105d0 con 0x7fc5b00ffb40 2024-05-09T09:15:19.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.017+0000 7fc5badc2740 1 Processor -- start 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.017+0000 7fc5badc2740 1 -- start start 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5badc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf08928990 con 0x55bf089c4eb0 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5badc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf088d05b0 con 0x55bf089ca020 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5badc2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf08924950 con 0x55bf089c44c0 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42682/0 (socket says 172.21.15.64:42682) 2024-05-09T09:15:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.018+0000 7fc5b5d71640 1 -- 172.21.15.64:0/618532813 learned_addr learned my addr 172.21.15.64:0/618532813 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.019+0000 7fc5b5d71640 1 -- 172.21.15.64:0/618532813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 msgr2=0x55bf089c48e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.019+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c48e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.019+0000 7fc5b5d71640 1 -- 172.21.15.64:0/618532813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 msgr2=0x55bf089cc4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.019+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089cc4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.019+0000 7fc5b5d71640 1 -- 172.21.15.64:0/618532813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf087047e0 con 0x55bf089c4eb0 2024-05-09T09:15:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.020+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c99b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0d5770 tx=0x7fc5ac0d5c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60cad9b1c844e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.021+0000 7fc5a6ffd640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc59c0a2d90 0x7fc59c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.021+0000 7fc5a6ffd640 1 -- 172.21.15.64:0/1513247049 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc5ac0cfc70 con 0x7fc5b00ffb40 2024-05-09T09:15:19.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.021+0000 7fc59bfff640 1 -- 172.21.15.64:0/618532813 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0e3070 con 0x55bf089c4eb0 2024-05-09T09:15:19.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.022+0000 7fc59bfff640 1 -- 172.21.15.64:0/618532813 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ac09c7e0 con 0x55bf089c4eb0 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.022+0000 7fc5b6572640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc59c0a2d90 0x7fc59c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.022+0000 7fc59bfff640 1 -- 172.21.15.64:0/618532813 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac098380 con 0x55bf089c4eb0 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 -- 172.21.15.64:0/618532813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 msgr2=0x55bf089c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c99b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0d5770 tx=0x7fc5ac0d5c30 comp rx=0 tx=0).stop 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 -- 172.21.15.64:0/618532813 shutdown_connections 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c99b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 --2- 172.21.15.64:0/618532813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.023+0000 7fc5badc2740 1 -- 172.21.15.64:0/618532813 >> 172.21.15.64:0/618532813 conn(0x55bf0898c0a0 msgr2=0x55bf089bff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.024+0000 7fc5b6572640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc59c0a2d90 0x7fc59c0a5250 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8005ee0 tx=0x7fc5a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.026+0000 7fc5badc2740 1 -- 172.21.15.64:0/618532813 shutdown_connections 2024-05-09T09:15:19.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.026+0000 7fc5badc2740 1 -- 172.21.15.64:0/618532813 wait complete. 2024-05-09T09:15:19.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 -- start start 2024-05-09T09:15:19.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 Processor -- start 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c4eb0 0x55bf089cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089ca020 0x55bf089d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf089cd020 con 0x55bf089ca020 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf088e3b40 con 0x55bf089c4eb0 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf089cd1a0 con 0x55bf089c44c0 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.027+0000 7fc5badc2740 10 client.0 _ll_register_callbacks cb 0x55bf088d80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37466/0 (socket says 172.21.15.64:37466) 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1978762966 learned_addr learned my addr 172.21.15.64:0/1978762966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5570640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c4eb0 0x55bf089cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b6572640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089ca020 0x55bf089d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c4eb0 msgr2=0x55bf089cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c4eb0 0x55bf089cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1978762966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089ca020 msgr2=0x55bf089d5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089ca020 0x55bf089d5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 -- 172.21.15.64:0/1978762966 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf08764f70 con 0x55bf089c44c0 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b6572640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089ca020 0x55bf089d5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.028+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089ce930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0a4820 tx=0x7fc5ac0094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b654580bdd9853b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5997fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5997fa640 1 -- 172.21.15.64:0/1978762966 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac09ba40 con 0x55bf089c44c0 2024-05-09T09:15:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5997fa640 1 -- 172.21.15.64:0/1978762966 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc5ac0cd400 con 0x55bf089c44c0 2024-05-09T09:15:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac09c7e0 con 0x55bf089c44c0 2024-05-09T09:15:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5997fa640 1 -- 172.21.15.64:0/1978762966 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc5ac09bc20 con 0x55bf089c44c0 2024-05-09T09:15:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5badc2740 10 client.26785 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bf085fc170 con 0x55bf089c44c0 2024-05-09T09:15:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.029+0000 7fc5997fa640 1 -- 172.21.15.64:0/1978762966 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ac09c7e0 con 0x55bf089c44c0 2024-05-09T09:15:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5997fa640 1 -- 172.21.15.64:0/1978762966 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fc5ac098750 con 0x55bf089c44c0 2024-05-09T09:15:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5badc2740 10 client.26785 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5badc2740 10 client.26785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5badc2740 20 client.26785 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5badc2740 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf089da010 0x55bf089dc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5badc2740 4 client.26785 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf089360c0 con 0x55bf089da010 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.030+0000 7fc5b5570640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf089da010 0x55bf089dc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.032+0000 7fc5b5570640 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf089da010 0x55bf089dc400 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187ca060da1859cf server_cookie=1fef1ce196e661bd in_seq=0 out_seq=0 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.032+0000 7fc5997fa640 10 client.26785 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.032+0000 7fc5997fa640 1 -- 172.21.15.64:0/1978762966 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf089360c0 con 0x55bf089da010 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.032+0000 7fc5997fa640 10 client.26785 handle_command_reply: tid=0 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.032+0000 7fc5badc2740 1 client.26785 shutdown 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 msgr2=0x55bf089ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089ce930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0a4820 tx=0x7fc5ac0094c0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 shutdown_connections 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf089da010 0x55bf089dc400 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089ca020 0x55bf089d5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c4eb0 0x55bf089cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.033+0000 7fc5badc2740 1 --2- 172.21.15.64:0/1978762966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089ce930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.034+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 >> 172.21.15.64:0/1978762966 conn(0x55bf0898c0a0 msgr2=0x55bf089bf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.034+0000 7fc5997fa640 0 client.26785 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.034+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 shutdown_connections 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.034+0000 7fc5badc2740 1 -- 172.21.15.64:0/1978762966 wait complete. 2024-05-09T09:15:19.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.034+0000 7fc5badc2740 20 client.26785 trim_cache size 0 max 16384 2024-05-09T09:15:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.069+0000 7fc5badc2740 1 Processor -- start 2024-05-09T09:15:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.069+0000 7fc5badc2740 1 -- start start 2024-05-09T09:15:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5badc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc594012620 con 0x55bf089c4eb0 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5badc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc59400fe30 con 0x55bf089ca020 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5badc2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc59400dc80 con 0x55bf089c44c0 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37472/0 (socket says 172.21.15.64:37472) 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 -- 172.21.15.64:0/296244825 learned_addr learned my addr 172.21.15.64:0/296244825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b6572640 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5570640 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 -- 172.21.15.64:0/296244825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 msgr2=0x55bf089d5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089d5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 -- 172.21.15.64:0/296244825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 msgr2=0x55bf089c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.070+0000 7fc5b5d71640 1 -- 172.21.15.64:0/296244825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf089be470 con 0x55bf089c44c0 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.071+0000 7fc5b5570640 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:19.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.071+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c3540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac09bde0 tx=0x7fc5ac09a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19f13bfbf16da71b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.071+0000 7fc59b7fe640 1 -- 172.21.15.64:0/296244825 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac09beb0 con 0x55bf089c44c0 2024-05-09T09:15:19.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.071+0000 7fc59b7fe640 1 -- 172.21.15.64:0/296244825 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ac09c7e0 con 0x55bf089c44c0 2024-05-09T09:15:19.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.071+0000 7fc59b7fe640 1 -- 172.21.15.64:0/296244825 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0a0490 con 0x55bf089c44c0 2024-05-09T09:15:19.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 -- 172.21.15.64:0/296244825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 msgr2=0x55bf089c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c3540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac09bde0 tx=0x7fc5ac09a5f0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 -- 172.21.15.64:0/296244825 shutdown_connections 2024-05-09T09:15:19.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089ca020 0x55bf089d5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 --2- 172.21.15.64:0/296244825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089c44c0 0x55bf089c3540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.072+0000 7fc5badc2740 1 -- 172.21.15.64:0/296244825 >> 172.21.15.64:0/296244825 conn(0x55bf0898c0a0 msgr2=0x55bf089c2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.073+0000 7fc5badc2740 1 -- 172.21.15.64:0/296244825 shutdown_connections 2024-05-09T09:15:19.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.073+0000 7fc5badc2740 1 -- 172.21.15.64:0/296244825 wait complete. 2024-05-09T09:15:19.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.073+0000 7fc5badc2740 1 -- start start 2024-05-09T09:15:19.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.073+0000 7fc5badc2740 1 Processor -- start 2024-05-09T09:15:19.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 0x55bf089d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089ca020 0x55bf089d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc59400e2f0 con 0x55bf089c4eb0 2024-05-09T09:15:19.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc59400f760 con 0x55bf089c44c0 2024-05-09T09:15:19.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc59400e9c0 con 0x55bf089ca020 2024-05-09T09:15:19.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5badc2740 10 client.0 _ll_register_callbacks cb 0x7fc5ac0e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:19.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 0x55bf089d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 0x55bf089d3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43606/0 (socket says 172.21.15.64:43606) 2024-05-09T09:15:19.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 -- 172.21.15.64:0/3623086047 learned_addr learned my addr 172.21.15.64:0/3623086047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5570640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b6572640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089ca020 0x55bf089d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 -- 172.21.15.64:0/3623086047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089ca020 msgr2=0x55bf089d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089ca020 0x55bf089d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 -- 172.21.15.64:0/3623086047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 msgr2=0x55bf089d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.074+0000 7fc5b5d71640 1 -- 172.21.15.64:0/3623086047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf089d55d0 con 0x55bf089c44c0 2024-05-09T09:15:19.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5b5d71640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 0x55bf089d3a90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c6a0 tx=0x7fc5ac0d52c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f064f03b24451e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5997fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:19.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5997fa640 1 -- 172.21.15.64:0/3623086047 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac09beb0 con 0x55bf089c44c0 2024-05-09T09:15:19.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac09c7e0 con 0x55bf089c44c0 2024-05-09T09:15:19.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5997fa640 1 -- 172.21.15.64:0/3623086047 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc5ac098750 con 0x55bf089c44c0 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5997fa640 1 -- 172.21.15.64:0/3623086047 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc5ac09ba40 con 0x55bf089c44c0 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5badc2740 10 client.26624 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.075+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bf08a406b0 con 0x55bf089c44c0 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5997fa640 1 -- 172.21.15.64:0/3623086047 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ac09c0e0 con 0x55bf089c44c0 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5997fa640 1 -- 172.21.15.64:0/3623086047 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fc5ac09c340 con 0x55bf089c44c0 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5badc2740 10 client.26624 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:19.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5badc2740 10 client.26624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5badc2740 20 client.26624 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5badc2740 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf08a40eb0 0x55bf08a432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5badc2740 4 client.26624 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc5ac09c7e0 con 0x55bf08a40eb0 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.076+0000 7fc5b5570640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf08a40eb0 0x55bf08a432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.077+0000 7fc5b5570640 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf08a40eb0 0x55bf08a432d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d700eaa9a0481a server_cookie=672252a060357448 in_seq=0 out_seq=0 2024-05-09T09:15:19.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.077+0000 7fc5997fa640 10 client.26624 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.078+0000 7fc5997fa640 1 -- 172.21.15.64:0/3623086047 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc5ac09c7e0 con 0x55bf08a40eb0 2024-05-09T09:15:19.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.078+0000 7fc5997fa640 10 client.26624 handle_command_reply: tid=0 2024-05-09T09:15:19.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.078+0000 7fc5badc2740 1 client.26624 shutdown 2024-05-09T09:15:19.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 msgr2=0x55bf089d3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 0x55bf089d3a90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c6a0 tx=0x7fc5ac0d52c0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 shutdown_connections 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf08a40eb0 0x55bf08a432d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf089ca020 0x55bf089d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf089c4eb0 0x55bf089d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 --2- 172.21.15.64:0/3623086047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf089c44c0 0x55bf089d3a90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5997fa640 0 client.26624 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.079+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 >> 172.21.15.64:0/3623086047 conn(0x55bf0898c0a0 msgr2=0x55bf089c36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.080+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 shutdown_connections 2024-05-09T09:15:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.080+0000 7fc5badc2740 1 -- 172.21.15.64:0/3623086047 wait complete. 2024-05-09T09:15:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.081+0000 7fc5badc2740 20 client.26624 trim_cache size 0 max 16384 2024-05-09T09:15:19.096 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:15:19.096 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:15:19.097 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "2e30d8fa-e71a-4329-975b-9919ddbfb51d", 2024-05-09T09:15:19.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:15:19.097 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:15:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.083+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc59c0a2d90 msgr2=0x7fc59c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.083+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc59c0a2d90 0x7fc59c0a5250 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8005ee0 tx=0x7fc5a8005fd0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 msgr2=0x7fc5b019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 0x7fc5b019e080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00e9c0 tx=0x7fc5ac00ee80 comp rx=0 tx=0).stop 2024-05-09T09:15:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 shutdown_connections 2024-05-09T09:15:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc59c0a2d90 0x7fc59c0a5250 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5b0101260 0x7fc5b01a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b01008a0 0x7fc5b019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 --2- 172.21.15.64:0/1513247049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5b00ffb40 0x7fc5b019e080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.084+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 >> 172.21.15.64:0/1513247049 conn(0x7fc5b00fb470 msgr2=0x7fc5b00fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.087+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 shutdown_connections 2024-05-09T09:15:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.087+0000 7fc5b7fd5640 1 -- 172.21.15.64:0/1513247049 wait complete. 2024-05-09T09:15:19.102 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:18 smithi139 ceph-mon[33599]: pgmap v555: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 805 op/s 2024-05-09T09:15:19.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:18 smithi064 ceph-mon[30149]: pgmap v555: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 805 op/s 2024-05-09T09:15:19.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:15:19.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:15:19.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:18 smithi179 ceph-mon[34086]: pgmap v555: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 805 op/s 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.318+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/3673543243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10e250 msgr2=0x7f1dbc1106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.318+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/3673543243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1106f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00aa10 tx=0x7f1dac02b720 comp rx=0 tx=0).stop 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/3673543243 shutdown_connections 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/3673543243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1106f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/3673543243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc10dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/3673543243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 0x7f1dbc10b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/3673543243 >> 172.21.15.64:0/3673543243 conn(0x7f1dbc06e930 msgr2=0x7f1dbc070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/3673543243 shutdown_connections 2024-05-09T09:15:19.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.319+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/3673543243 wait complete. 2024-05-09T09:15:19.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.320+0000 7f1dc1d8c640 1 Processor -- start 2024-05-09T09:15:19.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.320+0000 7f1dc1d8c640 1 -- start start 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc1d8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 0x7f1dbc1a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc1d8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc1a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc1d8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc0d8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 0x7f1dbc1a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc0d8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 0x7f1dbc1a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42728/0 (socket says 172.21.15.64:42728) 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc1a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc1a2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37490/0 (socket says 172.21.15.64:37490) 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc1d8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc115a80 con 0x7f1dbc06a4e0 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.321+0000 7f1dc0d8a640 1 -- 172.21.15.64:0/409676115 learned_addr learned my addr 172.21.15.64:0/409676115 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc115900 con 0x7f1dbc10e250 2024-05-09T09:15:19.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dc158b640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc115c00 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dbbfff640 1 -- 172.21.15.64:0/409676115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10e250 msgr2=0x7f1dbc1a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dbbfff640 1 -- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 msgr2=0x7f1dbc1a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 0x7f1dbc1a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dbbfff640 1 -- 172.21.15.64:0/409676115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc1a9d10 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.322+0000 7f1dc158b640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.323+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc1a2630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1db000d6a0 tx=0x7f1db000db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.323+0000 7f1db9ffb640 1 -- 172.21.15.64:0/409676115 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0017070 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.323+0000 7f1db9ffb640 1 -- 172.21.15.64:0/409676115 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1db0009820 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.323+0000 7f1db9ffb640 1 -- 172.21.15.64:0/409676115 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0004c40 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.323+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc1a9fa0 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.324+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc1aa470 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.328+0000 7f1dc5dda740 1 Processor -- start 2024-05-09T09:15:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.328+0000 7f1dc5dda740 1 -- start start 2024-05-09T09:15:19.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.328+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x56519246bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.328+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246c2e0 0x565192470de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x56519246bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x56519246bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37494/0 (socket says 172.21.15.64:37494) 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565192471450 0x5651924738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5651923c7d00 con 0x56519246c2e0 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56519236f4c0 con 0x565192471450 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5651923c3cc0 con 0x56519246b8f0 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 -- 172.21.15.64:0/1422510396 learned_addr learned my addr 172.21.15.64:0/1422510396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246c2e0 0x565192470de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 -- 172.21.15.64:0/1422510396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565192471450 msgr2=0x5651924738e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565192471450 0x5651924738e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 -- 172.21.15.64:0/1422510396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246c2e0 msgr2=0x565192470de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246c2e0 0x565192470de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.329+0000 7f1dc158b640 1 -- 172.21.15.64:0/1422510396 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651921a57d0 con 0x56519246b8f0 2024-05-09T09:15:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.330+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x56519246bd10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1dac005ab0 tx=0x7f1dac00a790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61ec832e55abfbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.330+0000 7f1da2ffd640 1 -- 172.21.15.64:0/1422510396 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac03f070 con 0x56519246b8f0 2024-05-09T09:15:19.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.330+0000 7f1da2ffd640 1 -- 172.21.15.64:0/1422510396 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dac0046c0 con 0x56519246b8f0 2024-05-09T09:15:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.331+0000 7f1db9ffb640 1 -- 172.21.15.64:0/409676115 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1db00099d0 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.332+0000 7f1db9ffb640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1da40a2c60 0x7f1da40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.333+0000 7f1db9ffb640 1 -- 172.21.15.64:0/409676115 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1db0099450 con 0x7f1dbc10b7f0 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.334+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1422510396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 msgr2=0x56519246bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.334+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x56519246bd10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1dac005ab0 tx=0x7f1dac00a790 comp rx=0 tx=0).stop 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.334+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1422510396 shutdown_connections 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.334+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565192471450 0x5651924738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.334+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246c2e0 0x565192470de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.334+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1422510396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x56519246bd10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.335+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1422510396 >> 172.21.15.64:0/1422510396 conn(0x56519245e750 msgr2=0x56519245eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.335+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1da40a2c60 0x7f1da40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.336+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1da40a2c60 0x7f1da40a5120 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009fd0 tx=0x7f1db40097a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.336+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1422510396 shutdown_connections 2024-05-09T09:15:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.338+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1422510396 wait complete. 2024-05-09T09:15:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.338+0000 7f1dc5dda740 1 -- start start 2024-05-09T09:15:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.338+0000 7f1dc5dda740 1 Processor -- start 2024-05-09T09:15:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192463190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x5651924636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x56519246a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565192474440 con 0x565192471450 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565192382b90 con 0x56519246c2e0 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5651924745c0 con 0x56519246b8f0 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc5dda740 10 client.0 _ll_register_callbacks cb 0x5651923beda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x5651924636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc158b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x56519246a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x5651924636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43622/0 (socket says 172.21.15.64:43622) 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dbbfff640 1 -- 172.21.15.64:0/1110012388 learned_addr learned my addr 172.21.15.64:0/1110012388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.339+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x56519246a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42748/0 (socket says 172.21.15.64:42748) 2024-05-09T09:15:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.340+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192463190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.340+0000 7f1dc158b640 1 -- 172.21.15.64:0/1110012388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 msgr2=0x565192463190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.340+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192463190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.340+0000 7f1dc158b640 1 -- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 msgr2=0x5651924636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.340+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x5651924636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.340+0000 7f1dc158b640 1 -- 172.21.15.64:0/1110012388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651920db3f0 con 0x565192471450 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.341+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x56519246a710 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00ab00 tx=0x7f1dac004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb5bd7e02809e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.341+0000 7f1d9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.341+0000 7f1d9bfff640 1 -- 172.21.15.64:0/1110012388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac03f070 con 0x565192471450 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.341+0000 7f1d9bfff640 1 -- 172.21.15.64:0/1110012388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1dac0051f0 con 0x565192471450 2024-05-09T09:15:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.341+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac0046c0 con 0x565192471450 2024-05-09T09:15:19.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.342+0000 7f1d9bfff640 1 -- 172.21.15.64:0/1110012388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1dac0053f0 con 0x565192471450 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.342+0000 7f1dc5dda740 10 client.17148 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.342+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56519209b0f0 con 0x565192471450 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.342+0000 7f1d9bfff640 1 -- 172.21.15.64:0/1110012388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dac0046c0 con 0x565192471450 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1d9bfff640 1 -- 172.21.15.64:0/1110012388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f1dac00b040 con 0x565192471450 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1dc5dda740 10 client.17148 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1dc5dda740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1dc5dda740 20 client.17148 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565192489f60 0x56519248c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1dc5dda740 4 client.17148 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.343+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651923d5430 con 0x565192489f60 2024-05-09T09:15:19.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.344+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565192489f60 0x56519248c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.345+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565192489f60 0x56519248c350 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6806c12b06ce8a61 server_cookie=cbb47609ba945caf in_seq=0 out_seq=0 2024-05-09T09:15:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.345+0000 7f1d9bfff640 10 client.17148 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1d9bfff640 1 -- 172.21.15.64:0/1110012388 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651923d5430 con 0x565192489f60 2024-05-09T09:15:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1d9bfff640 10 client.17148 handle_command_reply: tid=0 2024-05-09T09:15:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1dc5dda740 1 client.17148 shutdown 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 msgr2=0x56519246a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x56519246a710 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00ab00 tx=0x7f1dac004c00 comp rx=0 tx=0).stop 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 shutdown_connections 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565192489f60 0x56519248c350 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x56519246a710 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.346+0000 7f1d9bfff640 0 client.17148 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.347+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x5651924636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.347+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1110012388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192463190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.347+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 >> 172.21.15.64:0/1110012388 conn(0x56519245e750 msgr2=0x56519245da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.348+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 shutdown_connections 2024-05-09T09:15:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.348+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1110012388 wait complete. 2024-05-09T09:15:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.348+0000 7f1dc5dda740 20 client.17148 trim_cache size 0 max 16384 2024-05-09T09:15:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.382+0000 7f1dc5dda740 1 Processor -- start 2024-05-09T09:15:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.383+0000 7f1dc5dda740 1 -- start start 2024-05-09T09:15:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.383+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192461630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.383+0000 7f1dc0d8a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192461630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc0d8a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192461630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37512/0 (socket says 172.21.15.64:37512) 2024-05-09T09:15:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc0d8a640 1 -- 172.21.15.64:0/1687881304 learned_addr learned my addr 172.21.15.64:0/1687881304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x56519246a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x565192461b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b2ff0 con 0x565192471450 2024-05-09T09:15:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b0800 con 0x56519246c2e0 2024-05-09T09:15:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.384+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ae650 con 0x56519246b8f0 2024-05-09T09:15:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x56519246a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dc158b640 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x565192461b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dbbfff640 1 -- 172.21.15.64:0/1687881304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 msgr2=0x565192461630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192461630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dbbfff640 1 -- 172.21.15.64:0/1687881304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 msgr2=0x565192461b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x565192461b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.385+0000 7f1dbbfff640 1 -- 172.21.15.64:0/1687881304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565192463870 con 0x56519246c2e0 2024-05-09T09:15:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.386+0000 7f1dbbfff640 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x56519246a6d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009610 tx=0x7f1db00a2a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c3a479ffbabccc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.390+0000 7f1da27fc640 1 -- 172.21.15.64:0/1687881304 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0097de0 con 0x56519246c2e0 2024-05-09T09:15:19.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.390+0000 7f1da27fc640 1 -- 172.21.15.64:0/1687881304 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1db0012e50 con 0x56519246c2e0 2024-05-09T09:15:19.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.390+0000 7f1da27fc640 1 -- 172.21.15.64:0/1687881304 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0097f80 con 0x56519246c2e0 2024-05-09T09:15:19.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.391+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 msgr2=0x56519246a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.391+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x56519246a6d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009610 tx=0x7f1db00a2a70 comp rx=0 tx=0).stop 2024-05-09T09:15:19.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.391+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 shutdown_connections 2024-05-09T09:15:19.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.391+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565192471450 0x565192461b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.392+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x56519246a6d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.392+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/1687881304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56519246b8f0 0x565192461630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.392+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 >> 172.21.15.64:0/1687881304 conn(0x56519245e750 msgr2=0x56519245eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.393+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 shutdown_connections 2024-05-09T09:15:19.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.393+0000 7f1dc5dda740 1 -- 172.21.15.64:0/1687881304 wait complete. 2024-05-09T09:15:19.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.394+0000 7f1dc5dda740 1 -- start start 2024-05-09T09:15:19.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.395+0000 7f1dc5dda740 1 Processor -- start 2024-05-09T09:15:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.395+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246b8f0 0x565192467ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x565192468410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dc5dda740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565192471450 0x5651924693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dc5dda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1aecc0 con 0x56519246b8f0 2024-05-09T09:15:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x565192468410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x565192468410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43640/0 (socket says 172.21.15.64:43640) 2024-05-09T09:15:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dbbfff640 1 -- 172.21.15.64:0/3774988962 learned_addr learned my addr 172.21.15.64:0/3774988962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b0130 con 0x56519246c2e0 2024-05-09T09:15:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.396+0000 7f1dc158b640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565192471450 0x5651924693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1af390 con 0x565192471450 2024-05-09T09:15:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc5dda740 10 client.0 _ll_register_callbacks cb 0x7f1db4009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246b8f0 0x565192467ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc158b640 1 -- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 msgr2=0x565192468410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc158b640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x565192468410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc158b640 1 -- 172.21.15.64:0/3774988962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246b8f0 msgr2=0x565192467ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc158b640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246b8f0 0x565192467ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.397+0000 7f1dc158b640 1 -- 172.21.15.64:0/3774988962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519248c6c0 con 0x565192471450 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.398+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246b8f0 0x565192467ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.398+0000 7f1dc158b640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565192471450 0x5651924693b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0309a0 tx=0x7f1dac004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db1ad28c0f8cb6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.398+0000 7f1d9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.398+0000 7f1d9bfff640 1 -- 172.21.15.64:0/3774988962 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac03f040 con 0x565192471450 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.398+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db0012e50 con 0x565192471450 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.398+0000 7f1d9bfff640 1 -- 172.21.15.64:0/3774988962 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1dac0051f0 con 0x565192471450 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.399+0000 7f1d9bfff640 1 -- 172.21.15.64:0/3774988962 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1dac004260 con 0x565192471450 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.399+0000 7f1dc5dda740 10 client.26809 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.399+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=40}) -- 0x56519246a960 con 0x565192471450 2024-05-09T09:15:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.400+0000 7f1d9bfff640 1 -- 172.21.15.64:0/3774988962 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f1dac05b060 con 0x565192471450 2024-05-09T09:15:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.400+0000 7f1dc5dda740 10 client.26809 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.400+0000 7f1dc5dda740 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.400+0000 7f1dc5dda740 20 client.26809 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.400+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5651924f0ae0 0x5651924f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.401+0000 7f1dc5dda740 4 client.26809 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:15:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.401+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1db0012e50 con 0x5651924f0ae0 2024-05-09T09:15:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.401+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5651924f0ae0 0x5651924f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.403+0000 7f1dc0d8a640 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5651924f0ae0 0x5651924f2f00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da52892b5826a8f server_cookie=9992f7e699590533 in_seq=0 out_seq=0 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.403+0000 7f1d9bfff640 10 client.26809 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.403+0000 7f1d9bfff640 1 -- 172.21.15.64:0/3774988962 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f1db0012e50 con 0x5651924f0ae0 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.403+0000 7f1d9bfff640 10 client.26809 handle_command_reply: tid=0 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.403+0000 7f1dc5dda740 1 client.26809 shutdown 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565192471450 msgr2=0x5651924693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565192471450 0x5651924693b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0309a0 tx=0x7f1dac004f40 comp rx=0 tx=0).stop 2024-05-09T09:15:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 shutdown_connections 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5651924f0ae0 0x5651924f2f00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565192471450 0x5651924693b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56519246c2e0 0x565192468410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 --2- 172.21.15.64:0/3774988962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519246b8f0 0x565192467ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.405+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 >> 172.21.15.64:0/3774988962 conn(0x56519245e750 msgr2=0x56519245d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.406+0000 7f1d9bfff640 0 client.26809 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.406+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 shutdown_connections 2024-05-09T09:15:19.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.406+0000 7f1dc5dda740 1 -- 172.21.15.64:0/3774988962 wait complete. 2024-05-09T09:15:19.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.406+0000 7f1dc5dda740 20 client.26809 trim_cache size 0 max 16384 2024-05-09T09:15:19.412 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:15:19.412 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:15:19.412 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:15:19.412 INFO:teuthology.orchestra.run.smithi064.stdout: "2e30d8fa-e71a-4329-975b-9919ddbfb51d": { 2024-05-09T09:15:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:15:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "2e30d8fa-e71a-4329-975b-9919ddbfb51d", 2024-05-09T09:15:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:15:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:15:19.413 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:15:19.413 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:15:19.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1da40a2c60 msgr2=0x7f1da40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1da40a2c60 0x7f1da40a5120 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009fd0 tx=0x7f1db40097a0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 msgr2=0x7f1dbc1a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:19.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc1a2630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1db000d6a0 tx=0x7f1db000db60 comp rx=0 tx=0).stop 2024-05-09T09:15:19.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 shutdown_connections 2024-05-09T09:15:19.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1da40a2c60 0x7f1da40a5120 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc10e250 0x7f1dbc1a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1dbc10b7f0 0x7f1dbc1a2630 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 --2- 172.21.15.64:0/409676115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dbc06a4e0 0x7f1dbc1a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:19.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.409+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 >> 172.21.15.64:0/409676115 conn(0x7f1dbc06e930 msgr2=0x7f1dbc10adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:19.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.410+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 shutdown_connections 2024-05-09T09:15:19.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:19.410+0000 7f1dc1d8c640 1 -- 172.21.15.64:0/409676115 wait complete. 2024-05-09T09:15:19.432 INFO:tasks.cephfs.filesystem:scrub status for tag:2e30d8fa-e71a-4329-975b-9919ddbfb51d - {'path': '/', 'tag': '2e30d8fa-e71a-4329-975b-9919ddbfb51d', 'options': 'recursive,force'} 2024-05-09T09:15:19.573 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:15:19.102+0000 7f3d1eb03700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans/ auth{2=1} v2300 ap=2 f(v1 m2024-05-09T09:14:30.242793+0000 4=4+0) n(v6 rc2024-05-09T09:14:30.278333+0000 b1032192 5=4+1) (inest sync->mix g=2) caps={15456=pAsLsXsFs/-@26} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55f5a7ce6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v501>,<0x10000000005/data v111>,<0x10000000003/postgres v28>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v39>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v45>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2300>,<0x10000000005/data v1004>,<0x10000000003/postgres v495>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v497>,<0x10000000001/sv_0 v511>,<0x10000000000/qa v493>,<0x1/volumes v466>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:13:27.741710+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:14:30.278333+0000 b1032192 5=4+1)","memory_value.dirstat":"f(v1 m2024-05-09T09:14:30.242793+0000 4=4+0)","memory_value.rstat":"n(v6 rc2024-05-09T09:14:30.278333+0000 b1032192 5=4+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:19 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:15:20.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:15:20.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:15:20.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:15:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:15:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:19 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:19 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:21.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:20 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:20 smithi064 ceph-mon[30149]: pgmap v556: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-09T09:15:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:20 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:20 smithi179 ceph-mon[34086]: pgmap v556: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-09T09:15:21.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:20 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:20 smithi139 ceph-mon[33599]: pgmap v556: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-05-09T09:15:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 300.0 s, 933.4 tps, lat 107.098 ms stddev 118.201 2024-05-09T09:15:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:22 smithi064 ceph-mon[30149]: pgmap v557: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 811 op/s 2024-05-09T09:15:23.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:22 smithi179 ceph-mon[34086]: pgmap v557: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 811 op/s 2024-05-09T09:15:23.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:22 smithi139 ceph-mon[33599]: pgmap v557: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 811 op/s 2024-05-09T09:15:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:24 smithi064 ceph-mon[30149]: pgmap v558: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 13 MiB/s wr, 937 op/s 2024-05-09T09:15:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 -- 172.21.15.64:0/387809832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce80ffac0 msgr2=0x7f0ce80ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 --2- 172.21.15.64:0/387809832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce80ffac0 0x7f0ce80ffee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009900 tx=0x7f0cdc02a410 comp rx=0 tx=0).stop 2024-05-09T09:15:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 -- 172.21.15.64:0/387809832 shutdown_connections 2024-05-09T09:15:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 --2- 172.21.15.64:0/387809832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 0x7f0ce8101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 --2- 172.21.15.64:0/387809832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce8068200 0x7f0ce8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 --2- 172.21.15.64:0/387809832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce80ffac0 0x7f0ce80ffee0 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009900 tx=0x7f0cdc02a410 comp rx=0 tx=0).stop 2024-05-09T09:15:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 -- 172.21.15.64:0/387809832 >> 172.21.15.64:0/387809832 conn(0x7f0ce80fb050 msgr2=0x7f0ce80fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.207+0000 7f0ceef63640 1 -- 172.21.15.64:0/387809832 shutdown_connections 2024-05-09T09:15:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.208+0000 7f0ceef63640 1 -- 172.21.15.64:0/387809832 wait complete. 2024-05-09T09:15:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.208+0000 7f0ceef63640 1 Processor -- start 2024-05-09T09:15:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 -- start start 2024-05-09T09:15:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 0x7f0ce8199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 0x7f0ce819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 0x7f0ce81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810f630 con 0x7f0ce81011f0 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8109090 con 0x7f0ce8068200 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceef63640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810f7b0 con 0x7f0ce819f400 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 0x7f0ce8199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ce7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 0x7f0ce819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ced500640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 0x7f0ce81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ced500640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 0x7f0ce81a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50008/0 (socket says 172.21.15.64:50008) 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ced500640 1 -- 172.21.15.64:0/3227041913 learned_addr learned my addr 172.21.15.64:0/3227041913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.209+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 0x7f0ce8199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54744/0 (socket says 172.21.15.64:54744) 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ceccff640 1 -- 172.21.15.64:0/3227041913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 msgr2=0x7f0ce81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ceccff640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 0x7f0ce81a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ceccff640 1 -- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 msgr2=0x7f0ce819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ceccff640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 0x7f0ce819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ceccff640 1 -- 172.21.15.64:0/3227041913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce81a1e60 con 0x7f0ce8068200 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ced500640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 0x7f0ce81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.210+0000 7f0ce7fff640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 0x7f0ce819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.211+0000 7f0ceccff640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 0x7f0ce8199e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ff600 tx=0x7f0cdc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.211+0000 7f0ce5ffb640 1 -- 172.21.15.64:0/3227041913 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc03f020 con 0x7f0ce8068200 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.211+0000 7f0ce5ffb640 1 -- 172.21.15.64:0/3227041913 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cdc031c40 con 0x7f0ce8068200 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.211+0000 7f0ce5ffb640 1 -- 172.21.15.64:0/3227041913 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc035070 con 0x7f0ce8068200 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.211+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce81a20f0 con 0x7f0ce8068200 2024-05-09T09:15:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.212+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81a25b0 con 0x7f0ce8068200 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.215+0000 7f0ce5ffb640 1 -- 172.21.15.64:0/3227041913 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0cdc042020 con 0x7f0ce8068200 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.215+0000 7f0cf1c78740 1 Processor -- start 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 -- start start 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c273670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c278740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c27b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561c0c1d7340 con 0x561c0c273250 2024-05-09T09:15:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561c0c17f290 con 0x561c0c278db0 2024-05-09T09:15:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0cf1c78740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561c0c1d3300 con 0x561c0c273c40 2024-05-09T09:15:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0ced500640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c273670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0ced500640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c273670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38104/0 (socket says 172.21.15.64:38104) 2024-05-09T09:15:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0ced500640 1 -- 172.21.15.64:0/2914734052 learned_addr learned my addr 172.21.15.64:0/2914734052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.216+0000 7f0ce7fff640 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c27b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ce5ffb640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0cd00a2d30 0x7f0cd00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ce5ffb640 1 -- 172.21.15.64:0/3227041913 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0cdc0efad0 con 0x7f0ce8068200 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ceccff640 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c278740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ced500640 1 -- 172.21.15.64:0/2914734052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 msgr2=0x561c0c278740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ced500640 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c278740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ced500640 1 -- 172.21.15.64:0/2914734052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 msgr2=0x561c0c27b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ced500640 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c27b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.217+0000 7f0ced500640 1 -- 172.21.15.64:0/2914734052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0c1a38e0 con 0x561c0c273250 2024-05-09T09:15:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.218+0000 7f0ced500640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0cd00a2d30 0x7f0cd00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.218+0000 7f0ced500640 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c273670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00118b0 tx=0x7f0ce0011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d938542e9d7248a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.219+0000 7f0cceffd640 1 -- 172.21.15.64:0/2914734052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0010b00 con 0x561c0c273250 2024-05-09T09:15:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.219+0000 7f0cceffd640 1 -- 172.21.15.64:0/2914734052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce0010ca0 con 0x561c0c273250 2024-05-09T09:15:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.219+0000 7f0cceffd640 1 -- 172.21.15.64:0/2914734052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000e3b0 con 0x561c0c273250 2024-05-09T09:15:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.220+0000 7f0cf1c78740 1 -- 172.21.15.64:0/2914734052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 msgr2=0x561c0c273670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.220+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c273670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00118b0 tx=0x7f0ce0011d70 comp rx=0 tx=0).stop 2024-05-09T09:15:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.220+0000 7f0ced500640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0cd00a2d30 0x7f0cd00a51f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0020000 tx=0x7f0ce0021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.220+0000 7f0cf1c78740 1 -- 172.21.15.64:0/2914734052 shutdown_connections 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c27b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c278740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/2914734052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c273670 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 -- 172.21.15.64:0/2914734052 >> 172.21.15.64:0/2914734052 conn(0x561c0c26e8c0 msgr2=0x561c0c26ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 -- 172.21.15.64:0/2914734052 shutdown_connections 2024-05-09T09:15:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 -- 172.21.15.64:0/2914734052 wait complete. 2024-05-09T09:15:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.221+0000 7f0cf1c78740 1 -- start start 2024-05-09T09:15:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.222+0000 7f0cf1c78740 1 Processor -- start 2024-05-09T09:15:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.222+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c27d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.222+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273c40 0x561c0c27d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c27d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c27d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38118/0 (socket says 172.21.15.64:38118) 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0ce7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273c40 0x561c0c27d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0ce7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273c40 0x561c0c27d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54760/0 (socket says 172.21.15.64:54760) 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0ceccff640 1 -- 172.21.15.64:0/4178675320 learned_addr learned my addr 172.21.15.64:0/4178675320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c278db0 0x561c0c2848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561c0c27bd00 con 0x561c0c273250 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0ced500640 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c278db0 0x561c0c2848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561c0c192790 con 0x561c0c273c40 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561c0c27be80 con 0x561c0c278db0 2024-05-09T09:15:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.223+0000 7f0cf1c78740 10 client.0 _ll_register_callbacks cb 0x561c0c1cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0ced500640 1 -- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273c40 msgr2=0x561c0c27d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0ced500640 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273c40 0x561c0c27d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0ced500640 1 -- 172.21.15.64:0/4178675320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 msgr2=0x561c0c27d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0ced500640 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c27d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0ced500640 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0beeb210 con 0x561c0c278db0 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0ced500640 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c278db0 0x561c0c2848d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011880 tx=0x7f0ce0011600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17094fb06c7c4af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.224+0000 7f0cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.225+0000 7f0cc7fff640 1 -- 172.21.15.64:0/4178675320 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0034020 con 0x561c0c278db0 2024-05-09T09:15:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.225+0000 7f0cc7fff640 1 -- 172.21.15.64:0/4178675320 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ce001b420 con 0x561c0c278db0 2024-05-09T09:15:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.225+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce0010ca0 con 0x561c0c278db0 2024-05-09T09:15:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.225+0000 7f0cc7fff640 1 -- 172.21.15.64:0/4178675320 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0ce0029070 con 0x561c0c278db0 2024-05-09T09:15:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.225+0000 7f0cf1c78740 10 client.26818 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.225+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561c0bfb7620 con 0x561c0c278db0 2024-05-09T09:15:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.226+0000 7f0cc7fff640 1 -- 172.21.15.64:0/4178675320 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce0010ca0 con 0x561c0c278db0 2024-05-09T09:15:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.226+0000 7f0cc7fff640 1 -- 172.21.15.64:0/4178675320 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f0ce000e820 con 0x561c0c278db0 2024-05-09T09:15:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.226+0000 7f0cf1c78740 10 client.26818 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.226+0000 7f0cf1c78740 10 client.26818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0cf1c78740 20 client.26818 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c28b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0cf1c78740 4 client.26818 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c0c1e4a70 con 0x561c0c288c20 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0ceccff640 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c28b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0cc7fff640 10 client.26818 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.227+0000 7f0ceccff640 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c28b010 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5567a521b9778008 server_cookie=5875f6c7ef630f26 in_seq=0 out_seq=0 2024-05-09T09:15:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.228+0000 7f0cc7fff640 1 -- 172.21.15.64:0/4178675320 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c0c1e4a70 con 0x561c0c288c20 2024-05-09T09:15:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.228+0000 7f0cc7fff640 10 client.26818 handle_command_reply: tid=0 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.228+0000 7f0cf1c78740 1 client.26818 shutdown 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.228+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c278db0 msgr2=0x561c0c2848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c278db0 0x561c0c2848d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011880 tx=0x7f0ce0011600 comp rx=0 tx=0).stop 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 shutdown_connections 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c28b010 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c278db0 0x561c0c2848d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cc7fff640 0 client.26818 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273c40 0x561c0c27d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/4178675320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c27d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 >> 172.21.15.64:0/4178675320 conn(0x561c0c26e8c0 msgr2=0x561c0c26da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 shutdown_connections 2024-05-09T09:15:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 1 -- 172.21.15.64:0/4178675320 wait complete. 2024-05-09T09:15:25.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:24 smithi179 ceph-mon[34086]: pgmap v558: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 13 MiB/s wr, 937 op/s 2024-05-09T09:15:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.229+0000 7f0cf1c78740 20 client.26818 trim_cache size 0 max 16384 2024-05-09T09:15:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 Processor -- start 2024-05-09T09:15:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 -- start start 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c290900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c290e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c284890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81af500 con 0x561c0c273250 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81acd10 con 0x561c0c278db0 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0cf1c78740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81aab60 con 0x561c0c273c40 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.260+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c290900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c290900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38130/0 (socket says 172.21.15.64:38130) 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 -- 172.21.15.64:0/441039518 learned_addr learned my addr 172.21.15.64:0/441039518 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ce7fff640 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c290e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ced500640 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c284890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 -- 172.21.15.64:0/441039518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 msgr2=0x561c0c290e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c290e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 -- 172.21.15.64:0/441039518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 msgr2=0x561c0c284890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c284890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 -- 172.21.15.64:0/441039518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0c28b380 con 0x561c0c273250 2024-05-09T09:15:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ced500640 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c284890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0ceccff640 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c290900 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc005e90 tx=0x7f0cdc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd292bfb5f92851 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0cce7fc640 1 -- 172.21.15.64:0/441039518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0d4070 con 0x561c0c273250 2024-05-09T09:15:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0cce7fc640 1 -- 172.21.15.64:0/441039518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cdc0bbf40 con 0x561c0c273250 2024-05-09T09:15:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.261+0000 7f0cce7fc640 1 -- 172.21.15.64:0/441039518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0b85b0 con 0x561c0c273250 2024-05-09T09:15:25.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 -- 172.21.15.64:0/441039518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 msgr2=0x561c0c290900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c290900 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc005e90 tx=0x7f0cdc0dc040 comp rx=0 tx=0).stop 2024-05-09T09:15:25.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 -- 172.21.15.64:0/441039518 shutdown_connections 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c278db0 0x561c0c284890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c290e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/441039518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c273250 0x561c0c290900 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.262+0000 7f0cf1c78740 1 -- 172.21.15.64:0/441039518 >> 172.21.15.64:0/441039518 conn(0x561c0c26e8c0 msgr2=0x561c0c274dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.263+0000 7f0cf1c78740 1 -- 172.21.15.64:0/441039518 shutdown_connections 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.263+0000 7f0cf1c78740 1 -- 172.21.15.64:0/441039518 wait complete. 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.263+0000 7f0cf1c78740 1 -- start start 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.263+0000 7f0cf1c78740 1 Processor -- start 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 0x561c0c281ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c2823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 0x561c0c281ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c278db0 0x561c0c2833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0ceccff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 0x561c0c281ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54770/0 (socket says 172.21.15.64:54770) 2024-05-09T09:15:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0ceccff640 1 -- 172.21.15.64:0/3410035488 learned_addr learned my addr 172.21.15.64:0/3410035488 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81ab1d0 con 0x561c0c278db0 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0ced500640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c278db0 0x561c0c2833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0ce7fff640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c2823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81ac640 con 0x561c0c273250 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81ab8a0 con 0x561c0c273c40 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.264+0000 7f0cf1c78740 10 client.0 _ll_register_callbacks cb 0x7f0ce001b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0ceccff640 1 -- 172.21.15.64:0/3410035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 msgr2=0x561c0c2823e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0ceccff640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c2823e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0ceccff640 1 -- 172.21.15.64:0/3410035488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c278db0 msgr2=0x561c0c2833f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0ceccff640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c278db0 0x561c0c2833f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0ceccff640 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0c2839e0 con 0x561c0c273250 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0ceccff640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 0x561c0c281ea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0b8910 tx=0x7f0cdc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde97513fc68777c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.265+0000 7f0cc7fff640 1 -- 172.21.15.64:0/3410035488 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0d4040 con 0x561c0c273250 2024-05-09T09:15:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cc7fff640 1 -- 172.21.15.64:0/3410035488 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0cdc03b460 con 0x561c0c273250 2024-05-09T09:15:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cdc0bbf40 con 0x561c0c273250 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cc7fff640 1 -- 172.21.15.64:0/3410035488 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0cdc0c4bb0 con 0x561c0c273250 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 10 client.26654 fetch_fsmap learned FSMap version 40 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561c0c2ef170 con 0x561c0c273250 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cc7fff640 1 -- 172.21.15.64:0/3410035488 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cdc0b85b0 con 0x561c0c273250 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cc7fff640 1 -- 172.21.15.64:0/3410035488 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f0cdc03bbb0 con 0x561c0c273250 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 10 client.26654 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 20 client.26654 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c2f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 4 client.26654 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:15:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.266+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0cdc0bbf40 con 0x561c0c288c20 2024-05-09T09:15:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.267+0000 7f0ce7fff640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c2f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.267+0000 7f0ce7fff640 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c2f1a30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e04da0fa8fd9912 server_cookie=2487a553f8d06f0d in_seq=0 out_seq=0 2024-05-09T09:15:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.267+0000 7f0cc7fff640 10 client.26654 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.278+0000 7f0cc7fff640 1 -- 172.21.15.64:0/3410035488 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0cdc0bbf40 con 0x561c0c288c20 2024-05-09T09:15:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.278+0000 7f0cc7fff640 10 client.26654 handle_command_reply: tid=0 2024-05-09T09:15:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.278+0000 7f0cf1c78740 1 client.26654 shutdown 2024-05-09T09:15:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 msgr2=0x561c0c281ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 0x561c0c281ea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0b8910 tx=0x7f0cdc005d50 comp rx=0 tx=0).stop 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 shutdown_connections 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561c0c288c20 0x561c0c2f1a30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c0c278db0 0x561c0c2833f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c0c273c40 0x561c0c2823e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 --2- 172.21.15.64:0/3410035488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561c0c273250 0x561c0c281ea0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 >> 172.21.15.64:0/3410035488 conn(0x561c0c26e8c0 msgr2=0x561c0c271870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.279+0000 7f0cc7fff640 0 client.26654 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.280+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 shutdown_connections 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.280+0000 7f0cf1c78740 1 -- 172.21.15.64:0/3410035488 wait complete. 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.280+0000 7f0cf1c78740 20 client.26654 trim_cache size 0 max 16384 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:15:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:15:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0cd00a2d30 msgr2=0x7f0cd00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0cd00a2d30 0x7f0cd00a51f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0020000 tx=0x7f0ce0021000 comp rx=0 tx=0).stop 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 msgr2=0x7f0ce8199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 0x7f0ce8199e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ff600 tx=0x7f0cdc003c10 comp rx=0 tx=0).stop 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 shutdown_connections 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0cd00a2d30 0x7f0cd00a51f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.281+0000 7f0ceef63640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ce819f400 0x7f0ce81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.282+0000 7f0ceef63640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce81011f0 0x7f0ce819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.282+0000 7f0ceef63640 1 --2- 172.21.15.64:0/3227041913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ce8068200 0x7f0ce8199e60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.282+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 >> 172.21.15.64:0/3227041913 conn(0x7f0ce80fb050 msgr2=0x7f0ce8102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.282+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 shutdown_connections 2024-05-09T09:15:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:25.282+0000 7f0ceef63640 1 -- 172.21.15.64:0/3227041913 wait complete. 2024-05-09T09:15:25.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:24 smithi139 ceph-mon[33599]: pgmap v558: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 13 MiB/s wr, 937 op/s 2024-05-09T09:15:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 305.0 s, 941.6 tps, lat 106.043 ms stddev 130.450 2024-05-09T09:15:27.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:26 smithi064 ceph-mon[30149]: pgmap v559: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-05-09T09:15:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:26 smithi179 ceph-mon[34086]: pgmap v559: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-05-09T09:15:27.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:26 smithi139 ceph-mon[33599]: pgmap v559: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-05-09T09:15:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:28 smithi064 ceph-mon[30149]: pgmap v560: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 13 MiB/s wr, 870 op/s 2024-05-09T09:15:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:28 smithi179 ceph-mon[34086]: pgmap v560: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 13 MiB/s wr, 870 op/s 2024-05-09T09:15:29.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:28 smithi139 ceph-mon[33599]: pgmap v560: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 13 MiB/s wr, 870 op/s 2024-05-09T09:15:31.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:30 smithi064 ceph-mon[30149]: pgmap v561: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.14k op/s 2024-05-09T09:15:31.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:30 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:15:31.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:30 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:15:31.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:30 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:15:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:30 smithi179 ceph-mon[34086]: pgmap v561: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.14k op/s 2024-05-09T09:15:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:30 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:15:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:30 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:15:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:30 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:15:31.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:30 smithi139 ceph-mon[33599]: pgmap v561: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 1.14k op/s 2024-05-09T09:15:31.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:30 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:15:31.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:30 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:15:31.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:30 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:15:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 310.0 s, 894.0 tps, lat 112.072 ms stddev 130.538 2024-05-09T09:15:32.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:31 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:15:32.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:31 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:15:32.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:31 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:15:33.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:32 smithi179 ceph-mon[34086]: pgmap v562: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 923 op/s 2024-05-09T09:15:33.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:32 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:33.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:32 smithi139 ceph-mon[33599]: pgmap v562: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 923 op/s 2024-05-09T09:15:33.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:32 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:33.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:32 smithi064 ceph-mon[30149]: pgmap v562: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 923 op/s 2024-05-09T09:15:33.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:32 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:15:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:33 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:33 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi139 to 176.1M 2024-05-09T09:15:34.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:33 smithi179 ceph-mon[34086]: Unable to set osd_memory_target on smithi139 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-09T09:15:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:33 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:33 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi139 to 176.1M 2024-05-09T09:15:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:33 smithi139 ceph-mon[33599]: Unable to set osd_memory_target on smithi139 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-09T09:15:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:33 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:33 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi139 to 176.1M 2024-05-09T09:15:34.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:33 smithi064 ceph-mon[30149]: Unable to set osd_memory_target on smithi139 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-09T09:15:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: pgmap v563: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 1k op/s 2024-05-09T09:15:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:15:35.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:15:35.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:34 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:15:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: pgmap v563: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 1k op/s 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:15:35.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:34 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:15:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: pgmap v563: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 1k op/s 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:15:35.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:34 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:15:36.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:36 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-09T09:15:36.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:36 smithi139 ceph-mon[33599]: Unable to set osd_memory_target on smithi064 to 184775987: error parsing value: Value '184775987' is below minimum 939524096 2024-05-09T09:15:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 315.0 s, 857.8 tps, lat 116.342 ms stddev 144.250 2024-05-09T09:15:36.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:36 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-09T09:15:36.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:36 smithi064 ceph-mon[30149]: Unable to set osd_memory_target on smithi064 to 184775987: error parsing value: Value '184775987' is below minimum 939524096 2024-05-09T09:15:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:36 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-09T09:15:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:36 smithi179 ceph-mon[34086]: Unable to set osd_memory_target on smithi064 to 184775987: error parsing value: Value '184775987' is below minimum 939524096 2024-05-09T09:15:37.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:37 smithi139 ceph-mon[33599]: pgmap v564: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 954 op/s 2024-05-09T09:15:37.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:37 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:15:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:37 smithi064 ceph-mon[30149]: pgmap v564: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 954 op/s 2024-05-09T09:15:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:37 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:15:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:37 smithi179 ceph-mon[34086]: pgmap v564: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 954 op/s 2024-05-09T09:15:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:37 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.451+0000 7f202ffff640 1 -- 172.21.15.64:0/959256162 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028031880 con 0x7f2038108ed0 2024-05-09T09:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.452+0000 7f203e101640 1 -- 172.21.15.64:0/959256162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 msgr2=0x7f2038109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.452+0000 7f203e101640 1 --2- 172.21.15.64:0/959256162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 0x7f2038109350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2028009900 tx=0x7f202802a400 comp rx=0 tx=0).stop 2024-05-09T09:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 -- 172.21.15.64:0/959256162 shutdown_connections 2024-05-09T09:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 --2- 172.21.15.64:0/959256162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 0x7f2038110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 --2- 172.21.15.64:0/959256162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 0x7f2038109350 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 --2- 172.21.15.64:0/959256162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 0x7f2038108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 -- 172.21.15.64:0/959256162 >> 172.21.15.64:0/959256162 conn(0x7f2038076c80 msgr2=0x7f20380790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 -- 172.21.15.64:0/959256162 shutdown_connections 2024-05-09T09:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.453+0000 7f203e101640 1 -- 172.21.15.64:0/959256162 wait complete. 2024-05-09T09:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.454+0000 7f203e101640 1 Processor -- start 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 -- start start 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 0x7f203807f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 0x7f203807f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 0x7f20381adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2038113840 con 0x7f2038107ce0 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20381136c0 con 0x7f2038108ed0 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203e101640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20381139c0 con 0x7f2038109890 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 0x7f203807f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 0x7f203807f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54540/0 (socket says 172.21.15.64:54540) 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203d0ff640 1 -- 172.21.15.64:0/3816307103 learned_addr learned my addr 172.21.15.64:0/3816307103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203d900640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 0x7f20381adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.455+0000 7f203c8fe640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 0x7f203807f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.456+0000 7f203d900640 1 -- 172.21.15.64:0/3816307103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 msgr2=0x7f203807f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.456+0000 7f203d900640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 0x7f203807f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.456+0000 7f203d900640 1 -- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 msgr2=0x7f203807f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.456+0000 7f203d900640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 0x7f203807f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.456+0000 7f203d900640 1 -- 172.21.15.64:0/3816307103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203806b250 con 0x7f2038109890 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.456+0000 7f203d900640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 0x7f20381adf50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f203000e940 tx=0x7f203000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.457+0000 7f202e7fc640 1 -- 172.21.15.64:0/3816307103 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030004d20 con 0x7f2038109890 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.457+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203806b540 con 0x7f2038109890 2024-05-09T09:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.457+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203806ba60 con 0x7f2038109890 2024-05-09T09:15:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.457+0000 7f202e7fc640 1 -- 172.21.15.64:0/3816307103 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2030005020 con 0x7f2038109890 2024-05-09T09:15:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.458+0000 7f202e7fc640 1 -- 172.21.15.64:0/3816307103 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300116d0 con 0x7f2038109890 2024-05-09T09:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.461+0000 7f202e7fc640 1 -- 172.21.15.64:0/3816307103 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f203000cb20 con 0x7f2038109890 2024-05-09T09:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.462+0000 7f202e7fc640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20240a2b50 0x7f20240a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.462+0000 7f202e7fc640 1 -- 172.21.15.64:0/3816307103 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20300cf3f0 con 0x7f2038109890 2024-05-09T09:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.462+0000 7f203d0ff640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20240a2b50 0x7f20240a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.463+0000 7f203d0ff640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20240a2b50 0x7f20240a5010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f20340058c0 tx=0x7f2034005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.500+0000 7f204207a740 1 Processor -- start 2024-05-09T09:15:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.500+0000 7f204207a740 1 -- start start 2024-05-09T09:15:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f204207a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f204207a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15ddc0 0x562aba1628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f204207a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba1653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f204207a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562aba0c1340 con 0x562aba162f30 2024-05-09T09:15:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f204207a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562aba069290 con 0x562aba15ddc0 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f204207a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562aba0bd300 con 0x562aba15d3d0 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42358/0 (socket says 172.21.15.64:42358) 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f203d0ff640 1 -- 172.21.15.64:0/1361523659 learned_addr learned my addr 172.21.15.64:0/1361523659 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f203c8fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15ddc0 0x562aba1628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.501+0000 7f203d900640 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba1653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f203c8fe640 1 -- 172.21.15.64:0/1361523659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 msgr2=0x562aba15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f203c8fe640 1 -- 172.21.15.64:0/1361523659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 msgr2=0x562aba1653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba1653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f203c8fe640 1 -- 172.21.15.64:0/1361523659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aba08d8e0 con 0x562aba15ddc0 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15ddc0 0x562aba1628c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f202802a920 tx=0x7f20280040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d0f31372cc53c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.502+0000 7f20237fe640 1 -- 172.21.15.64:0/1361523659 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028035040 con 0x562aba15ddc0 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.503+0000 7f20237fe640 1 -- 172.21.15.64:0/1361523659 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20280054f0 con 0x562aba15ddc0 2024-05-09T09:15:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.503+0000 7f20237fe640 1 -- 172.21.15.64:0/1361523659 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280035f0 con 0x562aba15ddc0 2024-05-09T09:15:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.504+0000 7f204207a740 1 -- 172.21.15.64:0/1361523659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15ddc0 msgr2=0x562aba1628c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.504+0000 7f204207a740 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15ddc0 0x562aba1628c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f202802a920 tx=0x7f20280040c0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 -- 172.21.15.64:0/1361523659 shutdown_connections 2024-05-09T09:15:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba1653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15ddc0 0x562aba1628c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 --2- 172.21.15.64:0/1361523659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 -- 172.21.15.64:0/1361523659 >> 172.21.15.64:0/1361523659 conn(0x562aba158a40 msgr2=0x562aba158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 -- 172.21.15.64:0/1361523659 shutdown_connections 2024-05-09T09:15:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 -- 172.21.15.64:0/1361523659 wait complete. 2024-05-09T09:15:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.505+0000 7f204207a740 1 -- start start 2024-05-09T09:15:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.506+0000 7f204207a740 1 Processor -- start 2024-05-09T09:15:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.507+0000 7f204207a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba15d3d0 0x562aba167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.507+0000 7f204207a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15ddc0 0x562aba167b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.507+0000 7f204207a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.507+0000 7f204207a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562aba165e80 con 0x562aba15d3d0 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f204207a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562aba07c790 con 0x562aba162f30 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f204207a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562aba166000 con 0x562aba15ddc0 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f204207a740 10 client.0 _ll_register_callbacks cb 0x562aba0b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba15d3d0 0x562aba167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203d900640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba15d3d0 0x562aba167610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54554/0 (socket says 172.21.15.64:54554) 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203d0ff640 1 -- 172.21.15.64:0/2650870261 learned_addr learned my addr 172.21.15.64:0/2650870261 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203c8fe640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15ddc0 0x562aba167b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203c8fe640 1 -- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 msgr2=0x562aba16eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203c8fe640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203c8fe640 1 -- 172.21.15.64:0/2650870261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba15d3d0 msgr2=0x562aba167610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203c8fe640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba15d3d0 0x562aba167610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.508+0000 7f203c8fe640 1 -- 172.21.15.64:0/2650870261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab9dd5210 con 0x562aba15ddc0 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.509+0000 7f203d900640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.509+0000 7f203c8fe640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15ddc0 0x562aba167b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x562aba15e1c0 tx=0x7f202802a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574070fcb273803b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.509+0000 7f2020ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.509+0000 7f2020ff9640 1 -- 172.21.15.64:0/2650870261 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028035040 con 0x562aba15ddc0 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.509+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20280054f0 con 0x562aba15ddc0 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.509+0000 7f2020ff9640 1 -- 172.21.15.64:0/2650870261 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2028004ee0 con 0x562aba15ddc0 2024-05-09T09:15:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.510+0000 7f2020ff9640 1 -- 172.21.15.64:0/2650870261 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f202805a050 con 0x562aba15ddc0 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.510+0000 7f204207a740 10 client.26836 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.510+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562ab9ea1620 con 0x562aba15ddc0 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.510+0000 7f2020ff9640 1 -- 172.21.15.64:0/2650870261 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20280054f0 con 0x562aba15ddc0 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.510+0000 7f2020ff9640 1 -- 172.21.15.64:0/2650870261 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f202804c070 con 0x562aba15ddc0 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f204207a740 10 client.26836 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f204207a740 10 client.26836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f204207a740 20 client.26836 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f204207a740 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba172d40 0x562aba175170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f204207a740 4 client.26836 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aba0cea70 con 0x562aba172d40 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.511+0000 7f203d0ff640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba172d40 0x562aba175170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.512+0000 7f203d0ff640 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba172d40 0x562aba175170 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e44fd954de2716 server_cookie=c56d68490e00db1a in_seq=0 out_seq=0 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.512+0000 7f2020ff9640 10 client.26836 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.513+0000 7f2020ff9640 1 -- 172.21.15.64:0/2650870261 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562aba0cea70 con 0x562aba172d40 2024-05-09T09:15:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.513+0000 7f2020ff9640 10 client.26836 handle_command_reply: tid=0 2024-05-09T09:15:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.514+0000 7f204207a740 1 client.26836 shutdown 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.514+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15ddc0 msgr2=0x562aba167b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.514+0000 7f204207a740 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15ddc0 0x562aba167b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x562aba15e1c0 tx=0x7f202802a400 comp rx=0 tx=0).stop 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 shutdown_connections 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba172d40 0x562aba175170 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15ddc0 0x562aba167b50 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x562aba15e1c0 tx=0x7f202802a400 comp rx=0 tx=0).stop 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 --2- 172.21.15.64:0/2650870261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba15d3d0 0x562aba167610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 >> 172.21.15.64:0/2650870261 conn(0x562aba158a40 msgr2=0x562aba157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f2020ff9640 0 client.26836 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.515+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 shutdown_connections 2024-05-09T09:15:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.516+0000 7f204207a740 1 -- 172.21.15.64:0/2650870261 wait complete. 2024-05-09T09:15:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.516+0000 7f204207a740 20 client.26836 trim_cache size 0 max 16384 2024-05-09T09:15:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.553+0000 7f204207a740 1 Processor -- start 2024-05-09T09:15:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.553+0000 7f204207a740 1 -- start start 2024-05-09T09:15:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.554+0000 7f204207a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 0x562aba157b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.554+0000 7f204207a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba16eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.554+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 0x562aba157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.554+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 0x562aba157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52194/0 (socket says 172.21.15.64:52194) 2024-05-09T09:15:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.554+0000 7f203c8fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba16eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f203c8fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba16eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54568/0 (socket says 172.21.15.64:54568) 2024-05-09T09:15:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f203d0ff640 1 -- 172.21.15.64:0/3372519280 learned_addr learned my addr 172.21.15.64:0/3372519280 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f204207a740 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba167c80 0x562aba179a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f201c0125b0 con 0x562aba162f30 2024-05-09T09:15:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f201c00fe20 con 0x562aba15d3d0 2024-05-09T09:15:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f203d900640 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba167c80 0x562aba179a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.555+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f201c00dc70 con 0x562aba167c80 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f203d0ff640 1 -- 172.21.15.64:0/3372519280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba167c80 msgr2=0x562aba179a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f203d0ff640 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba167c80 0x562aba179a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f203d0ff640 1 -- 172.21.15.64:0/3372519280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 msgr2=0x562aba16eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f203d0ff640 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba16eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f203d0ff640 1 -- 172.21.15.64:0/3372519280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aba1684b0 con 0x562aba15d3d0 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f203d0ff640 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 0x562aba157b20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2034009460 tx=0x7f2034014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f597af0f9e27da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.556+0000 7f2022ffd640 1 -- 172.21.15.64:0/3372519280 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203401bcb0 con 0x562aba15d3d0 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.557+0000 7f2022ffd640 1 -- 172.21.15.64:0/3372519280 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f203401be50 con 0x562aba15d3d0 2024-05-09T09:15:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.557+0000 7f2022ffd640 1 -- 172.21.15.64:0/3372519280 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034005350 con 0x562aba15d3d0 2024-05-09T09:15:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 msgr2=0x562aba157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 0x562aba157b20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2034009460 tx=0x7f2034014000 comp rx=0 tx=0).stop 2024-05-09T09:15:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 shutdown_connections 2024-05-09T09:15:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba167c80 0x562aba179a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba162f30 0x562aba16eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 --2- 172.21.15.64:0/3372519280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba15d3d0 0x562aba157b20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.559+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 >> 172.21.15.64:0/3372519280 conn(0x562aba158a40 msgr2=0x562aba157410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.560+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 shutdown_connections 2024-05-09T09:15:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.560+0000 7f204207a740 1 -- 172.21.15.64:0/3372519280 wait complete. 2024-05-09T09:15:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.560+0000 7f204207a740 1 -- start start 2024-05-09T09:15:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.561+0000 7f204207a740 1 Processor -- start 2024-05-09T09:15:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.561+0000 7f204207a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba16e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f204207a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f204207a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba167c80 0x562aba16d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f204207a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f201c00e2e0 con 0x562aba167c80 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f204207a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f201c00f750 con 0x562aba162f30 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f204207a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f201c00e9b0 con 0x562aba15d3d0 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f204207a740 10 client.0 _ll_register_callbacks cb 0x7f2030005b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba16e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203d0ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba16e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42388/0 (socket says 172.21.15.64:42388) 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203d0ff640 1 -- 172.21.15.64:0/1946791673 learned_addr learned my addr 172.21.15.64:0/1946791673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203d900640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba167c80 0x562aba16d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203c8fe640 1 -- 172.21.15.64:0/1946791673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 msgr2=0x562aba16e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba16e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203c8fe640 1 -- 172.21.15.64:0/1946791673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba167c80 msgr2=0x562aba16d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba167c80 0x562aba16d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.562+0000 7f203c8fe640 1 -- 172.21.15.64:0/1946791673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aba1597b0 con 0x562aba162f30 2024-05-09T09:15:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.563+0000 7f203d900640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba167c80 0x562aba16d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.563+0000 7f203d0ff640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba16e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.563+0000 7f203c8fe640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16cd00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2028047000 tx=0x7f202803c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ee743434658dab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.564+0000 7f2020ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.564+0000 7f2020ff9640 1 -- 172.21.15.64:0/1946791673 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028062020 con 0x562aba162f30 2024-05-09T09:15:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.565+0000 7f2020ff9640 1 -- 172.21.15.64:0/1946791673 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f202803ce00 con 0x562aba162f30 2024-05-09T09:15:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.565+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203401be50 con 0x562aba162f30 2024-05-09T09:15:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.565+0000 7f2020ff9640 1 -- 172.21.15.64:0/1946791673 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f202806a050 con 0x562aba162f30 2024-05-09T09:15:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.566+0000 7f204207a740 10 client.26672 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.566+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562aba1d9dd0 con 0x562aba162f30 2024-05-09T09:15:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f2020ff9640 1 -- 172.21.15.64:0/1946791673 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2028005850 con 0x562aba162f30 2024-05-09T09:15:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f2020ff9640 1 -- 172.21.15.64:0/1946791673 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f202805d040 con 0x562aba162f30 2024-05-09T09:15:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f204207a740 10 client.26672 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f204207a740 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f204207a740 20 client.26672 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f204207a740 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba1da780 0x562aba1dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f204207a740 4 client.26672 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:15:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f203401be50 con 0x562aba1da780 2024-05-09T09:15:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.567+0000 7f203d0ff640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba1da780 0x562aba1dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.570+0000 7f2020ff9640 10 client.26672 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.570+0000 7f203d0ff640 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba1da780 0x562aba1dcba0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e540c1ab67d639b8 server_cookie=40c12d5c120e799d in_seq=0 out_seq=0 2024-05-09T09:15:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f2020ff9640 1 -- 172.21.15.64:0/1946791673 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f203401be50 con 0x562aba1da780 2024-05-09T09:15:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f2020ff9640 10 client.26672 handle_command_reply: tid=0 2024-05-09T09:15:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f204207a740 1 client.26672 shutdown 2024-05-09T09:15:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 msgr2=0x562aba16cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f204207a740 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16cd00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2028047000 tx=0x7f202803c880 comp rx=0 tx=0).stop 2024-05-09T09:15:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 shutdown_connections 2024-05-09T09:15:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.614+0000 7f204207a740 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aba1da780 0x562aba1dcba0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562aba167c80 0x562aba16d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562aba162f30 0x562aba16cd00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 1 --2- 172.21.15.64:0/1946791673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562aba15d3d0 0x562aba16e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 >> 172.21.15.64:0/1946791673 conn(0x562aba158a40 msgr2=0x562aba157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f2020ff9640 0 client.26672 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 shutdown_connections 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 1 -- 172.21.15.64:0/1946791673 wait complete. 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.615+0000 7f204207a740 20 client.26672 trim_cache size 0 max 16384 2024-05-09T09:15:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:15:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:15:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:15:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:15:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20240a2b50 msgr2=0x7f20240a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20240a2b50 0x7f20240a5010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f20340058c0 tx=0x7f2034005f90 comp rx=0 tx=0).stop 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 msgr2=0x7f20381adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 0x7f20381adf50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f203000e940 tx=0x7f203000ee00 comp rx=0 tx=0).stop 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 shutdown_connections 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20240a2b50 0x7f20240a5010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2038109890 0x7f20381adf50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2038108ed0 0x7f203807f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 --2- 172.21.15.64:0/3816307103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2038107ce0 0x7f203807f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 >> 172.21.15.64:0/3816307103 conn(0x7f2038076c80 msgr2=0x7f203810c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 shutdown_connections 2024-05-09T09:15:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:38.617+0000 7f203e101640 1 -- 172.21.15.64:0/3816307103 wait complete. 2024-05-09T09:15:39.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:39 smithi139 ceph-mon[33599]: pgmap v565: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-05-09T09:15:39.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:39 smithi064 ceph-mon[30149]: pgmap v565: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-05-09T09:15:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:39 smithi179 ceph-mon[34086]: pgmap v565: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-05-09T09:15:41.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:41 smithi139 ceph-mon[33599]: pgmap v566: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-05-09T09:15:41.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 320.0 s, 860.0 tps, lat 112.692 ms stddev 67.521 2024-05-09T09:15:41.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:41 smithi064 ceph-mon[30149]: pgmap v566: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-05-09T09:15:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:41 smithi179 ceph-mon[34086]: pgmap v566: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-05-09T09:15:43.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:43 smithi139 ceph-mon[33599]: pgmap v567: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-09T09:15:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:43 smithi064 ceph-mon[30149]: pgmap v567: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-09T09:15:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:43 smithi179 ceph-mon[34086]: pgmap v567: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-09T09:15:45.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:45 smithi139 ceph-mon[33599]: pgmap v568: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 872 op/s 2024-05-09T09:15:45.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:45 smithi064 ceph-mon[30149]: pgmap v568: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 872 op/s 2024-05-09T09:15:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:45 smithi179 ceph-mon[34086]: pgmap v568: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 872 op/s 2024-05-09T09:15:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 325.0 s, 1022.9 tps, lat 100.428 ms stddev 120.323 2024-05-09T09:15:47.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:47 smithi139 ceph-mon[33599]: pgmap v569: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 895 op/s 2024-05-09T09:15:47.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:47 smithi064 ceph-mon[30149]: pgmap v569: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 895 op/s 2024-05-09T09:15:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:47 smithi179 ceph-mon[34086]: pgmap v569: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 895 op/s 2024-05-09T09:15:49.281 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:49 smithi179 ceph-mon[34086]: pgmap v570: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 796 op/s 2024-05-09T09:15:49.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:49 smithi139 ceph-mon[33599]: pgmap v570: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 796 op/s 2024-05-09T09:15:49.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:49 smithi064 ceph-mon[30149]: pgmap v570: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 796 op/s 2024-05-09T09:15:49.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:15:49.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:15:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 -- 172.21.15.64:0/1000537552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 msgr2=0x7fde481101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 --2- 172.21.15.64:0/1000537552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481101b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ca0 tx=0x7fde4002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 -- 172.21.15.64:0/1000537552 shutdown_connections 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 --2- 172.21.15.64:0/1000537552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481101b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 --2- 172.21.15.64:0/1000537552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde48108f60 0x7fde481093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 --2- 172.21.15.64:0/1000537552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde48108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 -- 172.21.15.64:0/1000537552 >> 172.21.15.64:0/1000537552 conn(0x7fde48076d00 msgr2=0x7fde48079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.617+0000 7fde4e729640 1 -- 172.21.15.64:0/1000537552 shutdown_connections 2024-05-09T09:15:49.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.618+0000 7fde4e729640 1 -- 172.21.15.64:0/1000537552 wait complete. 2024-05-09T09:15:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.618+0000 7fde4e729640 1 Processor -- start 2024-05-09T09:15:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 -- start start 2024-05-09T09:15:49.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde4807eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4d727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde4807eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde48108f60 0x7fde4807f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4d727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde4807eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50160/0 (socket says 172.21.15.64:50160) 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4d727640 1 -- 172.21.15.64:0/1650803553 learned_addr learned my addr 172.21.15.64:0/1650803553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fde481138d0 con 0x7fde48107d70 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde48113750 con 0x7fde48109920 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.619+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde48113a50 con 0x7fde48108f60 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4df28640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4cf26640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde48108f60 0x7fde4807f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4df28640 1 -- 172.21.15.64:0/1650803553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde48108f60 msgr2=0x7fde4807f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4df28640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde48108f60 0x7fde4807f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4df28640 1 -- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 msgr2=0x7fde4807eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4df28640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde4807eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4df28640 1 -- 172.21.15.64:0/1650803553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4807fbd0 con 0x7fde48109920 2024-05-09T09:15:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.620+0000 7fde4d727640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde4807eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.621+0000 7fde4df28640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481adfe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde4002ade0 tx=0x7fde40003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.621+0000 7fde3e7fc640 1 -- 172.21.15.64:0/1650803553 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40032ec0 con 0x7fde48109920 2024-05-09T09:15:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.621+0000 7fde3e7fc640 1 -- 172.21.15.64:0/1650803553 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde400044f0 con 0x7fde48109920 2024-05-09T09:15:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.621+0000 7fde3e7fc640 1 -- 172.21.15.64:0/1650803553 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40030ae0 con 0x7fde48109920 2024-05-09T09:15:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.621+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde481ae5e0 con 0x7fde48109920 2024-05-09T09:15:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.622+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde481aeab0 con 0x7fde48109920 2024-05-09T09:15:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.626+0000 7fde52770740 1 Processor -- start 2024-05-09T09:15:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.626+0000 7fde3e7fc640 1 -- 172.21.15.64:0/1650803553 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde40030cc0 con 0x7fde48109920 2024-05-09T09:15:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 -- start start 2024-05-09T09:15:49.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53ac120 0x5646d53ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5646d5310d00 con 0x5646d53acb10 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5646d52b84c0 con 0x5646d53b1c80 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.627+0000 7fde52770740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646d530ccc0 con 0x5646d53ac120 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4cf26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53ac120 0x5646d53ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4cf26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53ac120 0x5646d53ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36728/0 (socket says 172.21.15.64:36728) 2024-05-09T09:15:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4cf26640 1 -- 172.21.15.64:0/2065876915 learned_addr learned my addr 172.21.15.64:0/2065876915 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4cf26640 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4d727640 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4d727640 1 -- 172.21.15.64:0/2065876915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53ac120 msgr2=0x5646d53ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4d727640 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53ac120 0x5646d53ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4d727640 1 -- 172.21.15.64:0/2065876915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 msgr2=0x5646d53b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4d727640 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde4d727640 1 -- 172.21.15.64:0/2065876915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d50ee7d0 con 0x5646d53acb10 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.628+0000 7fde3e7fc640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde340a2e80 0x7fde340a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.629+0000 7fde3e7fc640 1 -- 172.21.15.64:0/1650803553 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fde400f0620 con 0x7fde48109920 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.629+0000 7fde4d727640 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b1610 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fde440114c0 tx=0x7fde44011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fca101596a657cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.629+0000 7fde4cf26640 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.630+0000 7fde4cf26640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde340a2e80 0x7fde340a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.630+0000 7fde337fe640 1 -- 172.21.15.64:0/2065876915 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44005020 con 0x5646d53acb10 2024-05-09T09:15:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.630+0000 7fde337fe640 1 -- 172.21.15.64:0/2065876915 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde440047a0 con 0x5646d53acb10 2024-05-09T09:15:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.630+0000 7fde337fe640 1 -- 172.21.15.64:0/2065876915 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4400f930 con 0x5646d53acb10 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.630+0000 7fde52770740 1 -- 172.21.15.64:0/2065876915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 msgr2=0x5646d53b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.630+0000 7fde52770740 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b1610 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fde440114c0 tx=0x7fde44011980 comp rx=0 tx=0).stop 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde4cf26640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde340a2e80 0x7fde340a5340 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5646d53ad370 tx=0x7fde38013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde52770740 1 -- 172.21.15.64:0/2065876915 shutdown_connections 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde52770740 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde52770740 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b1610 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde52770740 1 --2- 172.21.15.64:0/2065876915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53ac120 0x5646d53ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde52770740 1 -- 172.21.15.64:0/2065876915 >> 172.21.15.64:0/2065876915 conn(0x5646d53a7790 msgr2=0x5646d53a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.631+0000 7fde52770740 1 -- 172.21.15.64:0/2065876915 shutdown_connections 2024-05-09T09:15:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.632+0000 7fde52770740 1 -- 172.21.15.64:0/2065876915 wait complete. 2024-05-09T09:15:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.632+0000 7fde52770740 1 -- start start 2024-05-09T09:15:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.632+0000 7fde52770740 1 Processor -- start 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 0x5646d53b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53b1c80 0x5646d53bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5646d53b4c80 con 0x5646d53acb10 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5646d52cbb90 con 0x5646d53ac120 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646d53b4e00 con 0x5646d53b1c80 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.633+0000 7fde52770740 10 client.0 _ll_register_callbacks cb 0x5646d5307da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 0x5646d53b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 0x5646d53b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49922/0 (socket says 172.21.15.64:49922) 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 -- 172.21.15.64:0/1659124956 learned_addr learned my addr 172.21.15.64:0/1659124956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4df28640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53b1c80 0x5646d53bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4cf26640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 -- 172.21.15.64:0/1659124956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53b1c80 msgr2=0x5646d53bdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53b1c80 0x5646d53bdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 -- 172.21.15.64:0/1659124956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 msgr2=0x5646d53b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.634+0000 7fde4d727640 1 -- 172.21.15.64:0/1659124956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d50243f0 con 0x5646d53ac120 2024-05-09T09:15:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.635+0000 7fde4d727640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 0x5646d53b65a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde440057c0 tx=0x7fde44005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35280d06dc3f12d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.635+0000 7fde30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.635+0000 7fde30ff9640 1 -- 172.21.15.64:0/1659124956 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44028020 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.635+0000 7fde30ff9640 1 -- 172.21.15.64:0/1659124956 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fde440053c0 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.635+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde440047a0 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde30ff9640 1 -- 172.21.15.64:0/1659124956 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde44033050 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 10 client.26684 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5646d4fe40f0 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde30ff9640 1 -- 172.21.15.64:0/1659124956 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde440047a0 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde30ff9640 1 -- 172.21.15.64:0/1659124956 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fde44023500 con 0x5646d53ac120 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 10 client.26684 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 10 client.26684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 20 client.26684 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d53c21b0 0x5646d53c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 4 client.26684 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646d531e430 con 0x5646d53c21b0 2024-05-09T09:15:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.636+0000 7fde4cf26640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d53c21b0 0x5646d53c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.637+0000 7fde4cf26640 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d53c21b0 0x5646d53c4630 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c780b06b057d1d0f server_cookie=4d5a6a52a08c494a in_seq=0 out_seq=0 2024-05-09T09:15:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.637+0000 7fde30ff9640 10 client.26684 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde30ff9640 1 -- 172.21.15.64:0/1659124956 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646d531e430 con 0x5646d53c21b0 2024-05-09T09:15:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde30ff9640 10 client.26684 handle_command_reply: tid=0 2024-05-09T09:15:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 client.26684 shutdown 2024-05-09T09:15:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 msgr2=0x5646d53b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 0x5646d53b65a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde440057c0 tx=0x7fde44005250 comp rx=0 tx=0).stop 2024-05-09T09:15:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 shutdown_connections 2024-05-09T09:15:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d53c21b0 0x5646d53c4630 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53b1c80 0x5646d53bdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.638+0000 7fde52770740 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.639+0000 7fde52770740 1 --2- 172.21.15.64:0/1659124956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53ac120 0x5646d53b65a0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde440057c0 tx=0x7fde44005250 comp rx=0 tx=0).stop 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.639+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 >> 172.21.15.64:0/1659124956 conn(0x5646d53a7790 msgr2=0x5646d53a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.639+0000 7fde30ff9640 0 client.26684 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.639+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 shutdown_connections 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.639+0000 7fde52770740 1 -- 172.21.15.64:0/1659124956 wait complete. 2024-05-09T09:15:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.639+0000 7fde52770740 20 client.26684 trim_cache size 0 max 16384 2024-05-09T09:15:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.673+0000 7fde52770740 1 Processor -- start 2024-05-09T09:15:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.673+0000 7fde52770740 1 -- start start 2024-05-09T09:15:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.673+0000 7fde52770740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.673+0000 7fde52770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde52770740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde52770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fde481bb910 con 0x5646d53acb10 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde52770740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b9180 con 0x5646d53b1c80 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde52770740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b6fd0 con 0x5646d53a7790 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36744/0 (socket says 172.21.15.64:36744) 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 -- 172.21.15.64:0/3497266850 learned_addr learned my addr 172.21.15.64:0/3497266850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4df28640 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 -- 172.21.15.64:0/3497266850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 msgr2=0x5646d53b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 -- 172.21.15.64:0/3497266850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 msgr2=0x5646d53bdab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bdab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.674+0000 7fde4d727640 1 -- 172.21.15.64:0/3497266850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d53ae760 con 0x5646d53a7790 2024-05-09T09:15:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.675+0000 7fde4d727640 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53a68b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde44023040 tx=0x7fde44011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3699d09880d532 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.675+0000 7fde32ffd640 1 -- 172.21.15.64:0/3497266850 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4403c070 con 0x5646d53a7790 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.675+0000 7fde32ffd640 1 -- 172.21.15.64:0/3497266850 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde44004780 con 0x5646d53a7790 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.675+0000 7fde32ffd640 1 -- 172.21.15.64:0/3497266850 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44028040 con 0x5646d53a7790 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 -- 172.21.15.64:0/3497266850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 msgr2=0x5646d53a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53a68b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde44023040 tx=0x7fde44011980 comp rx=0 tx=0).stop 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 -- 172.21.15.64:0/3497266850 shutdown_connections 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 --2- 172.21.15.64:0/3497266850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53a68b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.676+0000 7fde52770740 1 -- 172.21.15.64:0/3497266850 >> 172.21.15.64:0/3497266850 conn(0x5646d53a2c30 msgr2=0x5646d53a61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.677+0000 7fde52770740 1 -- 172.21.15.64:0/3497266850 shutdown_connections 2024-05-09T09:15:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.677+0000 7fde52770740 1 -- 172.21.15.64:0/3497266850 wait complete. 2024-05-09T09:15:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.677+0000 7fde52770740 1 -- start start 2024-05-09T09:15:49.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 Processor -- start 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b7640 con 0x5646d53acb10 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b8ab0 con 0x5646d53b1c80 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b7d10 con 0x5646d53a7790 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde52770740 10 client.0 _ll_register_callbacks cb 0x7fde38009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde4d727640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde4d727640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36752/0 (socket says 172.21.15.64:36752) 2024-05-09T09:15:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde4d727640 1 -- 172.21.15.64:0/2091300441 learned_addr learned my addr 172.21.15.64:0/2091300441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:49.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.678+0000 7fde4df28640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4cf26640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4df28640 1 -- 172.21.15.64:0/2091300441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 msgr2=0x5646d53bae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4df28640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53bae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4df28640 1 -- 172.21.15.64:0/2091300441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 msgr2=0x5646d53bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4df28640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4df28640 1 -- 172.21.15.64:0/2091300441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d53bc940 con 0x5646d53b1c80 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.679+0000 7fde4df28640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53bc350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde400edd90 tx=0x7fde40004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7599cdefb2dbea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.680+0000 7fde30ff9640 1 -- 172.21.15.64:0/2091300441 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400bcaf0 con 0x5646d53b1c80 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.680+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde44004780 con 0x5646d53b1c80 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.680+0000 7fde30ff9640 1 -- 172.21.15.64:0/2091300441 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fde400046a0 con 0x5646d53b1c80 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.680+0000 7fde30ff9640 1 -- 172.21.15.64:0/2091300441 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde400ca050 con 0x5646d53b1c80 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 10 client.26690 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:49.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5646d53bdcf0 con 0x5646d53b1c80 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde30ff9640 1 -- 172.21.15.64:0/2091300441 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde44004780 con 0x5646d53b1c80 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde30ff9640 1 -- 172.21.15.64:0/2091300441 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fde400f02f0 con 0x5646d53b1c80 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 10 client.26690 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 20 client.26690 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d5428de0 0x5646d542b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 4 client.26690 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:15:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.681+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646d5318780 con 0x5646d5428de0 2024-05-09T09:15:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.682+0000 7fde4d727640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d5428de0 0x5646d542b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.682+0000 7fde30ff9640 10 client.26690 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.682+0000 7fde4d727640 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d5428de0 0x5646d542b200 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d67238a2713b0a server_cookie=3def37175057b61f in_seq=0 out_seq=0 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.683+0000 7fde30ff9640 1 -- 172.21.15.64:0/2091300441 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646d5318780 con 0x5646d5428de0 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.683+0000 7fde30ff9640 10 client.26690 handle_command_reply: tid=0 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.683+0000 7fde52770740 1 client.26690 shutdown 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 msgr2=0x5646d53bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53bc350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde400edd90 tx=0x7fde40004960 comp rx=0 tx=0).stop 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 shutdown_connections 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5646d5428de0 0x5646d542b200 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5646d53b1c80 0x5646d53bc350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5646d53acb10 0x5646d53bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 --2- 172.21.15.64:0/2091300441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646d53a7790 0x5646d53bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 >> 172.21.15.64:0/2091300441 conn(0x5646d53a2c30 msgr2=0x5646d53a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.684+0000 7fde30ff9640 0 client.26690 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.685+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 shutdown_connections 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.685+0000 7fde52770740 1 -- 172.21.15.64:0/2091300441 wait complete. 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.685+0000 7fde52770740 20 client.26690 trim_cache size 0 max 16384 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:15:49.685 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:15:49.686 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde340a2e80 msgr2=0x7fde340a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde340a2e80 0x7fde340a5340 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5646d53ad370 tx=0x7fde38013040 comp rx=0 tx=0).stop 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 msgr2=0x7fde481adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481adfe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde4002ade0 tx=0x7fde40003ba0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 shutdown_connections 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde340a2e80 0x7fde340a5340 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde48109920 0x7fde481adfe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde48108f60 0x7fde4807f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 --2- 172.21.15.64:0/1650803553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde48107d70 0x7fde4807eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 >> 172.21.15.64:0/1650803553 conn(0x7fde48076d00 msgr2=0x7fde4810c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 shutdown_connections 2024-05-09T09:15:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.686+0000 7fde4e729640 1 -- 172.21.15.64:0/1650803553 wait complete. 2024-05-09T09:15:49.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:15:49.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 -- 172.21.15.64:0/871940087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c100770 msgr2=0x7fca8c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 --2- 172.21.15.64:0/871940087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c100770 0x7fca8c100b90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fca84009900 tx=0x7fca8402a410 comp rx=0 tx=0).stop 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 -- 172.21.15.64:0/871940087 shutdown_connections 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 --2- 172.21.15.64:0/871940087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 0x7fca8c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 --2- 172.21.15.64:0/871940087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca8c101960 0x7fca8c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 --2- 172.21.15.64:0/871940087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c100770 0x7fca8c100b90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 -- 172.21.15.64:0/871940087 >> 172.21.15.64:0/871940087 conn(0x7fca8c0fbce0 msgr2=0x7fca8c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:49.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.864+0000 7fca92cbd640 1 -- 172.21.15.64:0/871940087 shutdown_connections 2024-05-09T09:15:49.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.865+0000 7fca92cbd640 1 -- 172.21.15.64:0/871940087 wait complete. 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.865+0000 7fca92cbd640 1 Processor -- start 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 -- start start 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca8c100770 0x7fca8c10b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 0x7fca8c109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 0x7fca8c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10d3c0 con 0x7fca8c101960 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10d240 con 0x7fca8c100770 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca92cbd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10d540 con 0x7fca8c102320 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 0x7fca8c109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 0x7fca8c109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50210/0 (socket says 172.21.15.64:50210) 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca8bfff640 1 -- 172.21.15.64:0/1276407794 learned_addr learned my addr 172.21.15.64:0/1276407794 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca90a59640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca8c100770 0x7fca8c10b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.866+0000 7fca9125a640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 0x7fca8c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca9125a640 1 -- 172.21.15.64:0/1276407794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca8c100770 msgr2=0x7fca8c10b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca9125a640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca8c100770 0x7fca8c10b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca9125a640 1 -- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 msgr2=0x7fca8c109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca9125a640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 0x7fca8c109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca9125a640 1 -- 172.21.15.64:0/1276407794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c19ba60 con 0x7fca8c102320 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca8bfff640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 0x7fca8c109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca9125a640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 0x7fca8c109c70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e970 tx=0x7fca7400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7400ccc0 con 0x7fca8c102320 2024-05-09T09:15:49.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca74005020 con 0x7fca8c102320 2024-05-09T09:15:49.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.867+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74004a00 con 0x7fca8c102320 2024-05-09T09:15:49.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.868+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c19bd50 con 0x7fca8c102320 2024-05-09T09:15:49.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.868+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c19c250 con 0x7fca8c102320 2024-05-09T09:15:49.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.870+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c107bb0 con 0x7fca8c102320 2024-05-09T09:15:49.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.872+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca740051d0 con 0x7fca8c102320 2024-05-09T09:15:49.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.873+0000 7fca89ffb640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca780a2ce0 0x7fca780a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:49.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.873+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fca740cf810 con 0x7fca8c102320 2024-05-09T09:15:49.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.873+0000 7fca90a59640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca780a2ce0 0x7fca780a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:49.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.874+0000 7fca90a59640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca780a2ce0 0x7fca780a51a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fca84004770 tx=0x7fca84000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:49.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:49.879+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fca7401c020 con 0x7fca8c102320 2024-05-09T09:15:50.035 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:50 smithi064 ceph-mon[30149]: pgmap v571: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-05-09T09:15:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.289+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca8c10a250 con 0x7fca8c102320 2024-05-09T09:15:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.290+0000 7fca89ffb640 1 -- 172.21.15.64:0/1276407794 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11883 (secure 0 0 0) 0x7fca7409c380 con 0x7fca8c102320 2024-05-09T09:15:50.290 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:15:50.290 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:15:35.558507+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:15:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 42 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca780a2ce0 msgr2=0x7fca780a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca780a2ce0 0x7fca780a51a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fca84004770 tx=0x7fca84000bc0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 msgr2=0x7fca8c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 0x7fca8c109c70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e970 tx=0x7fca7400ee30 comp rx=0 tx=0).stop 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 shutdown_connections 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fca780a2ce0 0x7fca780a51a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca8c102320 0x7fca8c109c70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8c101960 0x7fca8c109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 --2- 172.21.15.64:0/1276407794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca8c100770 0x7fca8c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 >> 172.21.15.64:0/1276407794 conn(0x7fca8c0fbce0 msgr2=0x7fca8c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.293+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 shutdown_connections 2024-05-09T09:15:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.294+0000 7fca92cbd640 1 -- 172.21.15.64:0/1276407794 wait complete. 2024-05-09T09:15:50.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:15:50.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:50 smithi139 ceph-mon[33599]: pgmap v571: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-05-09T09:15:50.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:50 smithi179 ceph-mon[34086]: pgmap v571: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 -- 172.21.15.64:0/3216017999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 msgr2=0x7fddf810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 --2- 172.21.15.64:0/3216017999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf810e170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009ca0 tx=0x7fddf002a8f0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 -- 172.21.15.64:0/3216017999 shutdown_connections 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 --2- 172.21.15.64:0/3216017999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf810e170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 --2- 172.21.15.64:0/3216017999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 0x7fddf8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 --2- 172.21.15.64:0/3216017999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddf8072c70 0x7fddf8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.496+0000 7fddfd658640 1 -- 172.21.15.64:0/3216017999 >> 172.21.15.64:0/3216017999 conn(0x7fddf806e740 msgr2=0x7fddf8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.497+0000 7fddfd658640 1 -- 172.21.15.64:0/3216017999 shutdown_connections 2024-05-09T09:15:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.498+0000 7fddfd658640 1 -- 172.21.15.64:0/3216017999 wait complete. 2024-05-09T09:15:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.499+0000 7fddfd658640 1 Processor -- start 2024-05-09T09:15:50.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.500+0000 7fddfd658640 1 -- start start 2024-05-09T09:15:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.500+0000 7fddfd658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddf8072c70 0x7fddf8114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.500+0000 7fddfd658640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 0x7fddf81154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddfd658640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf81b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddfd658640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fddf811c170 con 0x7fddf8108f10 2024-05-09T09:15:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddfd658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddf811bff0 con 0x7fddf8072c70 2024-05-09T09:15:50.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddfd658640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fddf811c2f0 con 0x7fddf81098b0 2024-05-09T09:15:50.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 0x7fddf81154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 0x7fddf81154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50214/0 (socket says 172.21.15.64:50214) 2024-05-09T09:15:50.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.501+0000 7fddf77fe640 1 -- 172.21.15.64:0/422497401 learned_addr learned my addr 172.21.15.64:0/422497401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddf7fff640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddf8072c70 0x7fddf8114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddfce57640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf81b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddfce57640 1 -- 172.21.15.64:0/422497401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddf8072c70 msgr2=0x7fddf8114fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddfce57640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddf8072c70 0x7fddf8114fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddfce57640 1 -- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 msgr2=0x7fddf81154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddfce57640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 0x7fddf81154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.502+0000 7fddfce57640 1 -- 172.21.15.64:0/422497401 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf81b6dc0 con 0x7fddf81098b0 2024-05-09T09:15:50.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.503+0000 7fddfce57640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf81b6880 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fddf002aee0 tx=0x7fddf0030960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.503+0000 7fddf57fa640 1 -- 172.21.15.64:0/422497401 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0030d10 con 0x7fddf81098b0 2024-05-09T09:15:50.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.503+0000 7fddf57fa640 1 -- 172.21.15.64:0/422497401 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddf002d070 con 0x7fddf81098b0 2024-05-09T09:15:50.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.503+0000 7fddf57fa640 1 -- 172.21.15.64:0/422497401 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0004780 con 0x7fddf81098b0 2024-05-09T09:15:50.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.504+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf81b6f90 con 0x7fddf81098b0 2024-05-09T09:15:50.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.504+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf81b73c0 con 0x7fddf81098b0 2024-05-09T09:15:50.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.508+0000 7fde015c8740 1 Processor -- start 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 -- start start 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 0x55d8fd8cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d8fd831c40 con 0x55d8fd8cdb20 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d8fd7d9580 con 0x55d8fd8d2c90 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.509+0000 7fde015c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8fd82dc00 con 0x55d8fd8cd130 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.510+0000 7fddf57fa640 1 -- 172.21.15.64:0/422497401 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fddf0003970 con 0x7fddf81098b0 2024-05-09T09:15:50.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.510+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 0x55d8fd8cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.510+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 0x55d8fd8cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36792/0 (socket says 172.21.15.64:36792) 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.510+0000 7fddf77fe640 1 -- 172.21.15.64:0/1378026525 learned_addr learned my addr 172.21.15.64:0/1378026525 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.510+0000 7fddf7fff640 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.510+0000 7fddf77fe640 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.511+0000 7fddf77fe640 1 -- 172.21.15.64:0/1378026525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 msgr2=0x55d8fd8d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.511+0000 7fddf77fe640 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.511+0000 7fddf77fe640 1 -- 172.21.15.64:0/1378026525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 msgr2=0x55d8fd8d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.511+0000 7fddf77fe640 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.511+0000 7fddf77fe640 1 -- 172.21.15.64:0/1378026525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8fd5453f0 con 0x55d8fd8cd130 2024-05-09T09:15:50.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.511+0000 7fddf57fa640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdde00a2d90 0x7fdde00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.512+0000 7fddf57fa640 1 -- 172.21.15.64:0/422497401 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fddf00f04d0 con 0x7fddf81098b0 2024-05-09T09:15:50.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.512+0000 7fddf7fff640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdde00a2d90 0x7fdde00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.512+0000 7fddf77fe640 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 0x55d8fd8cd550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdde8017780 tx=0x7fdde80095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b0eafe8d65ea08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.512+0000 7fddde7fc640 1 -- 172.21.15.64:0/1378026525 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde80157c0 con 0x55d8fd8cd130 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.512+0000 7fddde7fc640 1 -- 172.21.15.64:0/1378026525 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde801d070 con 0x55d8fd8cd130 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.512+0000 7fddde7fc640 1 -- 172.21.15.64:0/1378026525 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8018b60 con 0x55d8fd8cd130 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 -- 172.21.15.64:0/1378026525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 msgr2=0x55d8fd8cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 0x55d8fd8cd550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdde8017780 tx=0x7fdde80095b0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fddf7fff640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdde00a2d90 0x7fdde00a5250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55d8fd8cdf20 tx=0x7fddec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 -- 172.21.15.64:0/1378026525 shutdown_connections 2024-05-09T09:15:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 --2- 172.21.15.64:0/1378026525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cd130 0x55d8fd8cd550 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.513+0000 7fde015c8740 1 -- 172.21.15.64:0/1378026525 >> 172.21.15.64:0/1378026525 conn(0x55d8fd8c87a0 msgr2=0x55d8fd8c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.514+0000 7fde015c8740 1 -- 172.21.15.64:0/1378026525 shutdown_connections 2024-05-09T09:15:50.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.514+0000 7fde015c8740 1 -- 172.21.15.64:0/1378026525 wait complete. 2024-05-09T09:15:50.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.514+0000 7fde015c8740 1 -- start start 2024-05-09T09:15:50.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.514+0000 7fde015c8740 1 Processor -- start 2024-05-09T09:15:50.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d8fd8d5be0 con 0x55d8fd8cdb20 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d8fd7ecc40 con 0x55d8fd8cd130 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8fd8d5d60 con 0x55d8fd8d2c90 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fde015c8740 10 client.0 _ll_register_callbacks cb 0x55d8fd544be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fddf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fddf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8d7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49996/0 (socket says 172.21.15.64:49996) 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fddf7fff640 1 -- 172.21.15.64:0/2371398479 learned_addr learned my addr 172.21.15.64:0/2371398479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fddf77fe640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.515+0000 7fddfce57640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddfce57640 1 -- 172.21.15.64:0/2371398479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 msgr2=0x55d8fd8d7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddfce57640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8d7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddfce57640 1 -- 172.21.15.64:0/2371398479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 msgr2=0x55d8fd8d7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddfce57640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddfce57640 1 -- 172.21.15.64:0/2371398479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8fd60f7d0 con 0x55d8fd8d2c90 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddf7fff640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8d7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddfce57640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8deb10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006010 tx=0x7fddf00bc2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf44896a5c2409ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.516+0000 7fddd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.517+0000 7fddd77fe640 1 -- 172.21.15.64:0/2371398479 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf00d0e60 con 0x55d8fd8d2c90 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.517+0000 7fddd77fe640 1 -- 172.21.15.64:0/2371398479 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fddf00f01c0 con 0x55d8fd8d2c90 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.517+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde801d070 con 0x55d8fd8d2c90 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.517+0000 7fddd77fe640 1 -- 172.21.15.64:0/2371398479 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fddf00bc940 con 0x55d8fd8d2c90 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.517+0000 7fde015c8740 10 client.26878 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:50.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.517+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d8fd8d5ca0 con 0x55d8fd8d2c90 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fddd77fe640 1 -- 172.21.15.64:0/2371398479 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde801d070 con 0x55d8fd8d2c90 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fddd77fe640 1 -- 172.21.15.64:0/2371398479 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fddf00db170 con 0x55d8fd8d2c90 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fde015c8740 10 client.26878 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fde015c8740 10 client.26878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fde015c8740 20 client.26878 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fde015c8740 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd8e35e0 0x55d8fd8e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fde015c8740 4 client.26878 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.520+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8fd83f370 con 0x55d8fd8e35e0 2024-05-09T09:15:50.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.521+0000 7fddf7fff640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd8e35e0 0x55d8fd8e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.522+0000 7fddf7fff640 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd8e35e0 0x55d8fd8e59f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7da8614c7c780ec0 server_cookie=bb7e078a304cdb0c in_seq=0 out_seq=0 2024-05-09T09:15:50.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.522+0000 7fddd77fe640 10 client.26878 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:50.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fddd77fe640 1 -- 172.21.15.64:0/2371398479 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8fd83f370 con 0x55d8fd8e35e0 2024-05-09T09:15:50.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fddd77fe640 10 client.26878 handle_command_reply: tid=0 2024-05-09T09:15:50.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 client.26878 shutdown 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 msgr2=0x55d8fd8deb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8deb10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006010 tx=0x7fddf00bc2a0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 shutdown_connections 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd8e35e0 0x55d8fd8e59f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8deb10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.523+0000 7fde015c8740 1 --2- 172.21.15.64:0/2371398479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8d7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.524+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 >> 172.21.15.64:0/2371398479 conn(0x55d8fd8c87a0 msgr2=0x55d8fd8c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.524+0000 7fddd77fe640 0 client.26878 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:50.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.524+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 shutdown_connections 2024-05-09T09:15:50.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.524+0000 7fde015c8740 1 -- 172.21.15.64:0/2371398479 wait complete. 2024-05-09T09:15:50.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.524+0000 7fde015c8740 20 client.26878 trim_cache size 0 max 16384 2024-05-09T09:15:50.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.554+0000 7fde015c8740 1 Processor -- start 2024-05-09T09:15:50.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.554+0000 7fde015c8740 1 -- start start 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.554+0000 7fde015c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cd130 0x55d8fd8ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.554+0000 7fde015c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fde015c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fde015c8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81c41c0 con 0x55d8fd8cd130 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36804/0 (socket says 172.21.15.64:36804) 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fde015c8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81c1a30 con 0x55d8fd8d2c90 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddf77fe640 1 -- 172.21.15.64:0/2671415554 learned_addr learned my addr 172.21.15.64:0/2671415554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fde015c8740 1 -- 172.21.15.64:0/2671415554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81bf880 con 0x55d8fd8cdb20 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddfce57640 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddf7fff640 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cd130 0x55d8fd8ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddfce57640 1 -- 172.21.15.64:0/2671415554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cdb20 msgr2=0x55d8fd8dead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddfce57640 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddfce57640 1 -- 172.21.15.64:0/2671415554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cd130 msgr2=0x55d8fd8ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddfce57640 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cd130 0x55d8fd8ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.555+0000 7fddfce57640 1 -- 172.21.15.64:0/2671415554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8fd8d2270 con 0x55d8fd8d2c90 2024-05-09T09:15:50.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.556+0000 7fddfce57640 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d7590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddf002aeb0 tx=0x7fddf00bc3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6831a6118791a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.556+0000 7fddddffb640 1 -- 172.21.15.64:0/2671415554 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf00d0e60 con 0x55d8fd8d2c90 2024-05-09T09:15:50.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.556+0000 7fddddffb640 1 -- 172.21.15.64:0/2671415554 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddf0036040 con 0x55d8fd8d2c90 2024-05-09T09:15:50.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.556+0000 7fddddffb640 1 -- 172.21.15.64:0/2671415554 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0004110 con 0x55d8fd8d2c90 2024-05-09T09:15:50.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.557+0000 7fde015c8740 1 -- 172.21.15.64:0/2671415554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 msgr2=0x55d8fd8d7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.558+0000 7fde015c8740 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d7590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddf002aeb0 tx=0x7fddf00bc3d0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.559+0000 7fde015c8740 1 -- 172.21.15.64:0/2671415554 shutdown_connections 2024-05-09T09:15:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.559+0000 7fde015c8740 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8d2c90 0x55d8fd8d7590 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.559+0000 7fde015c8740 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.559+0000 7fde015c8740 1 --2- 172.21.15.64:0/2671415554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cd130 0x55d8fd8ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.559+0000 7fde015c8740 1 -- 172.21.15.64:0/2671415554 >> 172.21.15.64:0/2671415554 conn(0x55d8fd8c87a0 msgr2=0x55d8fd8cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.560+0000 7fde015c8740 1 -- 172.21.15.64:0/2671415554 shutdown_connections 2024-05-09T09:15:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.561+0000 7fde015c8740 1 -- 172.21.15.64:0/2671415554 wait complete. 2024-05-09T09:15:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.561+0000 7fde015c8740 1 -- start start 2024-05-09T09:15:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.561+0000 7fde015c8740 1 Processor -- start 2024-05-09T09:15:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd947b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8db9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50022/0 (socket says 172.21.15.64:50022) 2024-05-09T09:15:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 -- 172.21.15.64:0/4198564400 learned_addr learned my addr 172.21.15.64:0/4198564400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81bfef0 con 0x55d8fd8cdb20 2024-05-09T09:15:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81c1360 con 0x55d8fd8cd130 2024-05-09T09:15:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81c05c0 con 0x55d8fd8d2c90 2024-05-09T09:15:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddfce57640 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd947b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fde015c8740 10 client.0 _ll_register_callbacks cb 0x7fddf0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 -- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 msgr2=0x55d8fd947b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd947b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 -- 172.21.15.64:0/4198564400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 msgr2=0x55d8fd8dbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.562+0000 7fddf7fff640 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8fd948060 con 0x55d8fd8cd130 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.563+0000 7fddf7fff640 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8db9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fddec015750 tx=0x7fddec012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c6dbac8de51d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.563+0000 7fddd77fe640 10 client.26708 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.563+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf0036040 con 0x55d8fd8cd130 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.563+0000 7fddd77fe640 1 -- 172.21.15.64:0/4198564400 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec02e070 con 0x55d8fd8cd130 2024-05-09T09:15:50.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.563+0000 7fddd77fe640 1 -- 172.21.15.64:0/4198564400 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fddec0285a0 con 0x55d8fd8cd130 2024-05-09T09:15:50.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.563+0000 7fddd77fe640 1 -- 172.21.15.64:0/4198564400 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fddec03a050 con 0x55d8fd8cd130 2024-05-09T09:15:50.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.565+0000 7fde015c8740 10 client.26708 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:50.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.565+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55d8fd949360 con 0x55d8fd8cd130 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fddd77fe640 1 -- 172.21.15.64:0/4198564400 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fddec01e040 con 0x55d8fd8cd130 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fde015c8740 10 client.26708 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fde015c8740 10 client.26708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fde015c8740 20 client.26708 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fde015c8740 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd949ae0 0x55d8fd94bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fde015c8740 4 client.26708 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:15:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fddf0036040 con 0x55d8fd949ae0 2024-05-09T09:15:50.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.566+0000 7fddf77fe640 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd949ae0 0x55d8fd94bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fddd77fe640 10 client.26708 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:50.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fddf77fe640 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd949ae0 0x55d8fd94bf40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bccca13f49ace9b5 server_cookie=bf7136cec5c89666 in_seq=0 out_seq=0 2024-05-09T09:15:50.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fddd77fe640 1 -- 172.21.15.64:0/4198564400 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fddf0036040 con 0x55d8fd949ae0 2024-05-09T09:15:50.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fddd77fe640 10 client.26708 handle_command_reply: tid=0 2024-05-09T09:15:50.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fde015c8740 1 client.26708 shutdown 2024-05-09T09:15:50.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 msgr2=0x55d8fd8db9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.567+0000 7fde015c8740 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8db9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fddec015750 tx=0x7fddec012000 comp rx=0 tx=0).stop 2024-05-09T09:15:50.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.568+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 shutdown_connections 2024-05-09T09:15:50.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.568+0000 7fde015c8740 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55d8fd949ae0 0x55d8fd94bf40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.569+0000 7fde015c8740 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8fd8d2c90 0x55d8fd947b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.569+0000 7fde015c8740 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8fd8cdb20 0x55d8fd8dbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.569+0000 7fddd77fe640 0 client.26708 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:15:50.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.569+0000 7fde015c8740 1 --2- 172.21.15.64:0/4198564400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8fd8cd130 0x55d8fd8db9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.569+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 >> 172.21.15.64:0/4198564400 conn(0x55d8fd8c87a0 msgr2=0x55d8fd8cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.569+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 shutdown_connections 2024-05-09T09:15:50.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.570+0000 7fde015c8740 1 -- 172.21.15.64:0/4198564400 wait complete. 2024-05-09T09:15:50.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.570+0000 7fde015c8740 20 client.26708 trim_cache size 0 max 16384 2024-05-09T09:15:50.573 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:15:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdde00a2d90 msgr2=0x7fdde00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdde00a2d90 0x7fdde00a5250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55d8fd8cdf20 tx=0x7fddec008040 comp rx=0 tx=0).stop 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 msgr2=0x7fddf81b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf81b6880 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fddf002aee0 tx=0x7fddf0030960 comp rx=0 tx=0).stop 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 shutdown_connections 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdde00a2d90 0x7fdde00a5250 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf81098b0 0x7fddf81b6880 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8108f10 0x7fddf81154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 --2- 172.21.15.64:0/422497401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddf8072c70 0x7fddf8114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.575+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 >> 172.21.15.64:0/422497401 conn(0x7fddf806e740 msgr2=0x7fddf806f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.576+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 shutdown_connections 2024-05-09T09:15:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.576+0000 7fddfd658640 1 -- 172.21.15.64:0/422497401 wait complete. 2024-05-09T09:15:50.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.797+0000 7f6307530640 1 -- 172.21.15.64:0/869861572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300104c80 msgr2=0x7f6300105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.797+0000 7f6307530640 1 --2- 172.21.15.64:0/869861572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300104c80 0x7f6300105100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009860 tx=0x7f62f002a340 comp rx=0 tx=0).stop 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.798+0000 7f6307530640 1 -- 172.21.15.64:0/869861572 shutdown_connections 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.798+0000 7f6307530640 1 --2- 172.21.15.64:0/869861572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6300105640 0x7f630010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.798+0000 7f6307530640 1 --2- 172.21.15.64:0/869861572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6300104c80 0x7f6300105100 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009860 tx=0x7f62f002a340 comp rx=0 tx=0).stop 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.798+0000 7f6307530640 1 --2- 172.21.15.64:0/869861572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.798+0000 7f6307530640 1 -- 172.21.15.64:0/869861572 >> 172.21.15.64:0/869861572 conn(0x7f63000ff020 msgr2=0x7f6300101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.798+0000 7f6307530640 1 -- 172.21.15.64:0/869861572 shutdown_connections 2024-05-09T09:15:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.799+0000 7f6307530640 1 -- 172.21.15.64:0/869861572 wait complete. 2024-05-09T09:15:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 Processor -- start 2024-05-09T09:15:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 -- start start 2024-05-09T09:15:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6300105640 0x7f6300196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f630019b070 0x7f630019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f630010f5f0 con 0x7f6300103a90 2024-05-09T09:15:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f630010f470 con 0x7f630019b070 2024-05-09T09:15:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.802+0000 7f6307530640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f630010f770 con 0x7f6300105640 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.803+0000 7f63052cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50272/0 (socket says 172.21.15.64:50272) 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f6304acb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6300105640 0x7f6300196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 -- 172.21.15.64:0/3956751860 learned_addr learned my addr 172.21.15.64:0/3956751860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f6305acd640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f630019b070 0x7f630019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 -- 172.21.15.64:0/3956751860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6300105640 msgr2=0x7f6300196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6300105640 0x7f6300196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 -- 172.21.15.64:0/3956751860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f630019b070 msgr2=0x7f630019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f630019b070 0x7f630019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.804+0000 7f63052cc640 1 -- 172.21.15.64:0/3956751860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019bae0 con 0x7f6300103a90 2024-05-09T09:15:50.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.805+0000 7f63052cc640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300195ad0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e9a0 tx=0x7f62fc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.806+0000 7f62f67fc640 1 -- 172.21.15.64:0/3956751860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc00cd80 con 0x7f6300103a90 2024-05-09T09:15:50.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.806+0000 7f62f67fc640 1 -- 172.21.15.64:0/3956751860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62fc0108e0 con 0x7f6300103a90 2024-05-09T09:15:50.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.806+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019bdd0 con 0x7f6300103a90 2024-05-09T09:15:50.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.806+0000 7f62f67fc640 1 -- 172.21.15.64:0/3956751860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc018950 con 0x7f6300103a90 2024-05-09T09:15:50.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.806+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630019c340 con 0x7f6300103a90 2024-05-09T09:15:50.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 Processor -- start 2024-05-09T09:15:50.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 -- start start 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe997e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9983430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe9983aa0 0x55dbe9985f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dbe98e2c40 con 0x55dbe9983aa0 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dbe988a580 con 0x55dbe997df40 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.810+0000 7f630a23f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dbe98dec00 con 0x55dbe997e930 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.811+0000 7f6305acd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe997e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.811+0000 7f6304acb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe9983aa0 0x55dbe9985f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.811+0000 7f6305acd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe997e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50046/0 (socket says 172.21.15.64:50046) 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.811+0000 7f6305acd640 1 -- 172.21.15.64:0/4290494764 learned_addr learned my addr 172.21.15.64:0/4290494764 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6304acb640 1 -- 172.21.15.64:0/4290494764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 msgr2=0x55dbe9983430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6304acb640 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9983430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6304acb640 1 -- 172.21.15.64:0/4290494764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 msgr2=0x55dbe997e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6304acb640 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe997e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6304acb640 1 -- 172.21.15.64:0/4290494764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe95f63f0 con 0x55dbe9983aa0 2024-05-09T09:15:50.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6305acd640 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe997e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:50.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.812+0000 7f6304acb640 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe9983aa0 0x55dbe9985f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f62f0008000 tx=0x7f62f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7d92fa34861d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.813+0000 7f62f67fc640 1 -- 172.21.15.64:0/3956751860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62fc010a90 con 0x7f6300103a90 2024-05-09T09:15:50.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.813+0000 7f62eb7fe640 1 -- 172.21.15.64:0/4290494764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0043020 con 0x55dbe9983aa0 2024-05-09T09:15:50.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.814+0000 7f62eb7fe640 1 -- 172.21.15.64:0/4290494764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62f0038040 con 0x55dbe9983aa0 2024-05-09T09:15:50.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.814+0000 7f62eb7fe640 1 -- 172.21.15.64:0/4290494764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0037040 con 0x55dbe9983aa0 2024-05-09T09:15:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.815+0000 7f630a23f740 1 -- 172.21.15.64:0/4290494764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe9983aa0 msgr2=0x55dbe9985f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.816+0000 7f630a23f740 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe9983aa0 0x55dbe9985f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f62f0008000 tx=0x7f62f0000bc0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.816+0000 7f630a23f740 1 -- 172.21.15.64:0/4290494764 shutdown_connections 2024-05-09T09:15:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.816+0000 7f630a23f740 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe9983aa0 0x55dbe9985f30 secure :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f62f0008000 tx=0x7f62f0000bc0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.816+0000 7f630a23f740 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9983430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.816+0000 7f630a23f740 1 --2- 172.21.15.64:0/4290494764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe997e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.816+0000 7f630a23f740 1 -- 172.21.15.64:0/4290494764 >> 172.21.15.64:0/4290494764 conn(0x55dbe99795b0 msgr2=0x55dbe99799b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.817+0000 7f630a23f740 1 -- 172.21.15.64:0/4290494764 shutdown_connections 2024-05-09T09:15:50.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.817+0000 7f630a23f740 1 -- 172.21.15.64:0/4290494764 wait complete. 2024-05-09T09:15:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.817+0000 7f62f67fc640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f62ec0a2bf0 0x7f62ec0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.818+0000 7f62f67fc640 1 -- 172.21.15.64:0/3956751860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62fc0cf7a0 con 0x7f6300103a90 2024-05-09T09:15:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.818+0000 7f6304acb640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f62ec0a2bf0 0x7f62ec0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.818+0000 7f630a23f740 1 -- start start 2024-05-09T09:15:50.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.818+0000 7f630a23f740 1 Processor -- start 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f630a23f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe9988550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f630a23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9988a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f630a23f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe998dac0 0x55dbe998feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f630a23f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dbe99869f0 con 0x55dbe997df40 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f6304acb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9988a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f6304acb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9988a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36842/0 (socket says 172.21.15.64:36842) 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.819+0000 7f6304acb640 1 -- 172.21.15.64:0/417134130 learned_addr learned my addr 172.21.15.64:0/417134130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dbe989dc40 con 0x55dbe998dac0 2024-05-09T09:15:50.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dbe9986b70 con 0x55dbe997e930 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f630a23f740 10 client.0 _ll_register_callbacks cb 0x55dbe95f5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f6304acb640 1 -- 172.21.15.64:0/417134130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe998dac0 msgr2=0x55dbe998feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f6304acb640 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe998dac0 0x55dbe998feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f6304acb640 1 -- 172.21.15.64:0/417134130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 msgr2=0x55dbe9988550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f6304acb640 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe9988550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f6304acb640 1 -- 172.21.15.64:0/417134130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe96c07d0 con 0x55dbe997e930 2024-05-09T09:15:50.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.820+0000 7f6304acb640 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9988a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009780 tx=0x7f62f0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd54a3c88a4ac772 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.821+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f0038040 con 0x55dbe997e930 2024-05-09T09:15:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.821+0000 7f6304acb640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f62ec0a2bf0 0x7f62ec0a50b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f62f0035040 tx=0x7f62f0003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.822+0000 7f62e8ff9640 10 client.26890 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.822+0000 7f62e8ff9640 1 -- 172.21.15.64:0/417134130 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0037040 con 0x55dbe997e930 2024-05-09T09:15:50.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.822+0000 7f62e8ff9640 1 -- 172.21.15.64:0/417134130 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62f0005240 con 0x55dbe997e930 2024-05-09T09:15:50.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.822+0000 7f62e8ff9640 1 -- 172.21.15.64:0/417134130 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f62f0005440 con 0x55dbe997e930 2024-05-09T09:15:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.824+0000 7f630a23f740 10 client.26890 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:50.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.824+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55dbe9993030 con 0x55dbe997e930 2024-05-09T09:15:50.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f62e8ff9640 1 -- 172.21.15.64:0/417134130 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f62f005b020 con 0x55dbe997e930 2024-05-09T09:15:50.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f630a23f740 10 client.26890 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:50.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f630a23f740 10 client.26890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:15:50.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f630a23f740 20 client.26890 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:50.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f630a23f740 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99949c0 0x55dbe9996e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f630a23f740 4 client.26890 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:50.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.825+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f62f0038040 con 0x55dbe99949c0 2024-05-09T09:15:50.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.826+0000 7f63052cc640 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99949c0 0x55dbe9996e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.830+0000 7f63052cc640 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99949c0 0x55dbe9996e80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c2a7e06e2c86ec1 server_cookie=96272f72749db429 in_seq=0 out_seq=0 2024-05-09T09:15:50.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.830+0000 7f62e8ff9640 10 client.26890 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:50.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.831+0000 7f62e8ff9640 1 -- 172.21.15.64:0/417134130 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f62f0038040 con 0x55dbe99949c0 2024-05-09T09:15:50.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.831+0000 7f62e8ff9640 10 client.26890 handle_command_reply: tid=0 2024-05-09T09:15:50.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.831+0000 7f630a23f740 1 client.26890 shutdown 2024-05-09T09:15:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.834+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 msgr2=0x55dbe9988a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.834+0000 7f630a23f740 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9988a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009780 tx=0x7f62f0003820 comp rx=0 tx=0).stop 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 shutdown_connections 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe998dac0 0x55dbe998feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe997e930 0x55dbe9988a90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99949c0 0x55dbe9996e80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 --2- 172.21.15.64:0/417134130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe9988550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 >> 172.21.15.64:0/417134130 conn(0x55dbe99795b0 msgr2=0x55dbe99787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f62e8ff9640 0 client.26890 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 shutdown_connections 2024-05-09T09:15:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 1 -- 172.21.15.64:0/417134130 wait complete. 2024-05-09T09:15:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.835+0000 7f630a23f740 20 client.26890 trim_cache size 0 max 16384 2024-05-09T09:15:50.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.873+0000 7f630a23f740 1 Processor -- start 2024-05-09T09:15:50.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.874+0000 7f630a23f740 1 -- start start 2024-05-09T09:15:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.874+0000 7f630a23f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe997ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.874+0000 7f630a23f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 0x55dbe99804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.874+0000 7f630a23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe9977f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f63052cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe997ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f6304acb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 0x55dbe99804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f63052cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe997ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50304/0 (socket says 172.21.15.64:50304) 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f6304acb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 0x55dbe99804b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50068/0 (socket says 172.21.15.64:50068) 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f63052cc640 1 -- 172.21.15.64:0/10510182 learned_addr learned my addr 172.21.15.64:0/10510182 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62e40125a0 con 0x55dbe997df40 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62e400fe10 con 0x55dbe997e930 2024-05-09T09:15:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.875+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f62e400dc60 con 0x55dbe998dac0 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.876+0000 7f6304acb640 1 -- 172.21.15.64:0/10510182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 msgr2=0x55dbe9977f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.876+0000 7f6304acb640 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe9977f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.876+0000 7f6304acb640 1 -- 172.21.15.64:0/10510182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 msgr2=0x55dbe997ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.876+0000 7f6304acb640 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe997ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.876+0000 7f6304acb640 1 -- 172.21.15.64:0/10510182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe996ad20 con 0x55dbe997e930 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.877+0000 7f6304acb640 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 0x55dbe99804b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f62f002a8b0 tx=0x7f62f004e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3afcefa5fbbff7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.877+0000 7f62eaffd640 1 -- 172.21.15.64:0/10510182 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f00043c0 con 0x55dbe997e930 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.877+0000 7f62eaffd640 1 -- 172.21.15.64:0/10510182 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62f0004dd0 con 0x55dbe997e930 2024-05-09T09:15:50.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.877+0000 7f62eaffd640 1 -- 172.21.15.64:0/10510182 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0032500 con 0x55dbe997e930 2024-05-09T09:15:50.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.878+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 msgr2=0x55dbe99804b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.878+0000 7f630a23f740 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 0x55dbe99804b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f62f002a8b0 tx=0x7f62f004e000 comp rx=0 tx=0).stop 2024-05-09T09:15:50.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.878+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 shutdown_connections 2024-05-09T09:15:50.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.879+0000 7f630a23f740 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe9977f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.879+0000 7f630a23f740 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997e930 0x55dbe99804b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.879+0000 7f630a23f740 1 --2- 172.21.15.64:0/10510182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997df40 0x55dbe997ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.879+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 >> 172.21.15.64:0/10510182 conn(0x55dbe99795b0 msgr2=0x55dbe9988550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.879+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 shutdown_connections 2024-05-09T09:15:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.879+0000 7f630a23f740 1 -- 172.21.15.64:0/10510182 wait complete. 2024-05-09T09:15:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.880+0000 7f630a23f740 1 -- start start 2024-05-09T09:15:50.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.880+0000 7f630a23f740 1 Processor -- start 2024-05-09T09:15:50.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f630a23f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe99a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f630a23f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997e930 0x55dbe99a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f630a23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe99f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f630a23f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62e400e2d0 con 0x55dbe997e930 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f630a23f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62e400f740 con 0x55dbe997df40 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f63052cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe99a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f6305acd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe99f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f6305acd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe99f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36856/0 (socket says 172.21.15.64:36856) 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f63052cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe99a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50074/0 (socket says 172.21.15.64:50074) 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.881+0000 7f6305acd640 1 -- 172.21.15.64:0/206312776 learned_addr learned my addr 172.21.15.64:0/206312776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:50.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f62e400e9a0 con 0x55dbe998dac0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6304acb640 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997e930 0x55dbe99a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6305acd640 1 -- 172.21.15.64:0/206312776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 msgr2=0x55dbe99a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6305acd640 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe99a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6305acd640 1 -- 172.21.15.64:0/206312776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997e930 msgr2=0x55dbe99a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6305acd640 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997e930 0x55dbe99a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6305acd640 1 -- 172.21.15.64:0/206312776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe99f9010 con 0x55dbe998dac0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f6305acd640 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe99f8ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e6d0 tx=0x7f62f800eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669141867a8675d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.882+0000 7f62e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.883+0000 7f62e8ff9640 1 -- 172.21.15.64:0/206312776 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f801c060 con 0x55dbe998dac0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.883+0000 7f62e8ff9640 1 -- 172.21.15.64:0/206312776 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62f800cb10 con 0x55dbe998dac0 2024-05-09T09:15:50.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.883+0000 7f630a23f740 10 client.0 _ll_register_callbacks cb 0x7f62f00096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:50.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.883+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f0004dd0 con 0x55dbe998dac0 2024-05-09T09:15:50.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.883+0000 7f62e8ff9640 1 -- 172.21.15.64:0/206312776 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f62f800cd10 con 0x55dbe998dac0 2024-05-09T09:15:50.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.885+0000 7f630a23f740 10 client.26893 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:50.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.885+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55dbe99fa1e0 con 0x55dbe998dac0 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.886+0000 7f62e8ff9640 1 -- 172.21.15.64:0/206312776 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f62f8017440 con 0x55dbe998dac0 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.886+0000 7f630a23f740 10 client.26893 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.886+0000 7f630a23f740 10 client.26893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.886+0000 7f630a23f740 20 client.26893 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.886+0000 7f630a23f740 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99faa90 0x55dbe99fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.887+0000 7f630a23f740 4 client.26893 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.887+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f62f0004dd0 con 0x55dbe99faa90 2024-05-09T09:15:50.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.887+0000 7f63052cc640 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99faa90 0x55dbe99fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:50.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.889+0000 7f63052cc640 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99faa90 0x55dbe99fced0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=475be7f3a843f6ae server_cookie=b9e95d62a8417382 in_seq=0 out_seq=0 2024-05-09T09:15:50.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.889+0000 7f62e8ff9640 10 client.26893 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.891+0000 7f62e8ff9640 1 -- 172.21.15.64:0/206312776 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f62f0004dd0 con 0x55dbe99faa90 2024-05-09T09:15:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.891+0000 7f62e8ff9640 10 client.26893 handle_command_reply: tid=0 2024-05-09T09:15:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.891+0000 7f630a23f740 1 client.26893 shutdown 2024-05-09T09:15:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.892+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 msgr2=0x55dbe99f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.892+0000 7f630a23f740 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe99f8ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e6d0 tx=0x7f62f800eb90 comp rx=0 tx=0).stop 2024-05-09T09:15:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.892+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 shutdown_connections 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.892+0000 7f630a23f740 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbe998dac0 0x55dbe99f8ad0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.892+0000 7f630a23f740 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55dbe99faa90 0x55dbe99fced0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f62e8ff9640 0 client.26893 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f630a23f740 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dbe997e930 0x55dbe99a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f630a23f740 1 --2- 172.21.15.64:0/206312776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dbe997df40 0x55dbe99a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 >> 172.21.15.64:0/206312776 conn(0x55dbe99795b0 msgr2=0x55dbe9988550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 shutdown_connections 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f630a23f740 1 -- 172.21.15.64:0/206312776 wait complete. 2024-05-09T09:15:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.893+0000 7f630a23f740 20 client.26893 trim_cache size 0 max 16384 2024-05-09T09:15:50.894 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:15:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f62ec0a2bf0 msgr2=0x7f62ec0a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f62ec0a2bf0 0x7f62ec0a50b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f62f0035040 tx=0x7f62f0003f90 comp rx=0 tx=0).stop 2024-05-09T09:15:50.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 msgr2=0x7f6300195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300195ad0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e9a0 tx=0x7f62fc00ee60 comp rx=0 tx=0).stop 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 shutdown_connections 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f62ec0a2bf0 0x7f62ec0a50b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f630019b070 0x7f630019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6300105640 0x7f6300196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 --2- 172.21.15.64:0/3956751860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6300103a90 0x7f6300195ad0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 >> 172.21.15.64:0/3956751860 conn(0x7f63000ff020 msgr2=0x7f6300077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 shutdown_connections 2024-05-09T09:15:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:50.896+0000 7f6307530640 1 -- 172.21.15.64:0/3956751860 wait complete. 2024-05-09T09:15:50.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:15:51.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 -- 172.21.15.64:0/3518730179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10104c80 msgr2=0x7ffa10105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 --2- 172.21.15.64:0/3518730179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10104c80 0x7ffa10105100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009900 tx=0x7ffa0c02a420 comp rx=0 tx=0).stop 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 -- 172.21.15.64:0/3518730179 shutdown_connections 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 --2- 172.21.15.64:0/3518730179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10105640 0x7ffa1010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 --2- 172.21.15.64:0/3518730179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10104c80 0x7ffa10105100 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 --2- 172.21.15.64:0/3518730179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa10103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 -- 172.21.15.64:0/3518730179 >> 172.21.15.64:0/3518730179 conn(0x7ffa100ff020 msgr2=0x7ffa10101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.126+0000 7ffa17864640 1 -- 172.21.15.64:0/3518730179 shutdown_connections 2024-05-09T09:15:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.127+0000 7ffa17864640 1 -- 172.21.15.64:0/3518730179 wait complete. 2024-05-09T09:15:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 Processor -- start 2024-05-09T09:15:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 -- start start 2024-05-09T09:15:51.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa101959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 0x7ffa10195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10105640 0x7ffa1019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010f5f0 con 0x7ffa10103a90 2024-05-09T09:15:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010f470 con 0x7ffa10104c80 2024-05-09T09:15:51.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.128+0000 7ffa17864640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010f770 con 0x7ffa10105640 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 0x7ffa10195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa101959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa101959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50328/0 (socket says 172.21.15.64:50328) 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 -- 172.21.15.64:0/2762956993 learned_addr learned my addr 172.21.15.64:0/2762956993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 0x7ffa10195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50082/0 (socket says 172.21.15.64:50082) 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 -- 172.21.15.64:0/2762956993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10105640 msgr2=0x7ffa1019ceb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10105640 0x7ffa1019ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 -- 172.21.15.64:0/2762956993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 msgr2=0x7ffa10195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 0x7ffa10195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa15600640 1 -- 172.21.15.64:0/2762956993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa101ae540 con 0x7ffa10103a90 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.129+0000 7ffa14dff640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 0x7ffa10195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:15:51.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.130+0000 7ffa15600640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa101959a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000d7a0 tx=0x7ffa0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.130+0000 7ffa067fc640 1 -- 172.21.15.64:0/2762956993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00017070 con 0x7ffa10103a90 2024-05-09T09:15:51.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.130+0000 7ffa067fc640 1 -- 172.21.15.64:0/2762956993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa00009c30 con 0x7ffa10103a90 2024-05-09T09:15:51.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.130+0000 7ffa067fc640 1 -- 172.21.15.64:0/2762956993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00003d30 con 0x7ffa10103a90 2024-05-09T09:15:51.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.131+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa101ae830 con 0x7ffa10103a90 2024-05-09T09:15:51.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.131+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101aed50 con 0x7ffa10103a90 2024-05-09T09:15:51.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.135+0000 7ffa1a572740 1 Processor -- start 2024-05-09T09:15:51.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.135+0000 7ffa1a572740 1 -- start start 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.135+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff14a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff14f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff151f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa1a572740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563dff0aec40 con 0x563dff14a930 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa1a572740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563dff056580 con 0x563dff149f40 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa1a572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563dff0aac00 con 0x563dff14faa0 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa15e01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff14a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa15e01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff14a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50098/0 (socket says 172.21.15.64:50098) 2024-05-09T09:15:51.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa15e01640 1 -- 172.21.15.64:0/1279128749 learned_addr learned my addr 172.21.15.64:0/1279128749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa14dff640 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.136+0000 7ffa067fc640 1 -- 172.21.15.64:0/2762956993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa00003ed0 con 0x7ffa10103a90 2024-05-09T09:15:51.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.137+0000 7ffa067fc640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9fc0a2d10 0x7ff9fc0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.137+0000 7ffa15600640 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.138+0000 7ffa067fc640 1 -- 172.21.15.64:0/2762956993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa000cecf0 con 0x7ffa10103a90 2024-05-09T09:15:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 -- 172.21.15.64:0/1279128749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 msgr2=0x563dff151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 -- 172.21.15.64:0/1279128749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 msgr2=0x563dff14f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff14f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 -- 172.21.15.64:0/1279128749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dfedc23f0 con 0x563dff149f40 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff14a360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e30 tx=0x7ffa0800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=372299fd24cdf7a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.140+0000 7ffa15e01640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9fc0a2d10 0x7ff9fc0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.141+0000 7ff9fb7fe640 1 -- 172.21.15.64:0/1279128749 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0801a5d0 con 0x563dff149f40 2024-05-09T09:15:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.141+0000 7ff9fb7fe640 1 -- 172.21.15.64:0/1279128749 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa0801aca0 con 0x563dff149f40 2024-05-09T09:15:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.142+0000 7ffa1a572740 1 -- 172.21.15.64:0/1279128749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 msgr2=0x563dff14a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.142+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff14a360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e30 tx=0x7ffa0800e720 comp rx=0 tx=0).stop 2024-05-09T09:15:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.142+0000 7ffa15e01640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9fc0a2d10 0x7ff9fc0a51d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800f010 tx=0x7ffa08019a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.142+0000 7ffa1a572740 1 -- 172.21.15.64:0/1279128749 shutdown_connections 2024-05-09T09:15:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.143+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.143+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.143+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1279128749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff14a360 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.143+0000 7ffa1a572740 1 -- 172.21.15.64:0/1279128749 >> 172.21.15.64:0/1279128749 conn(0x563dff1455b0 msgr2=0x563dff1459b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.144+0000 7ffa1a572740 1 -- 172.21.15.64:0/1279128749 shutdown_connections 2024-05-09T09:15:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.144+0000 7ffa1a572740 1 -- 172.21.15.64:0/1279128749 wait complete. 2024-05-09T09:15:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.144+0000 7ffa1a572740 1 -- start start 2024-05-09T09:15:51.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 Processor -- start 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff149f40 0x563dff1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 0x563dff1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 0x563dff1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563dff152aa0 con 0x563dff14faa0 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563dff069c40 con 0x563dff14a930 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa1a572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563dff152c20 con 0x563dff149f40 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 0x563dff1549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50112/0 (socket says 172.21.15.64:50112) 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.145+0000 7ffa14dff640 1 -- 172.21.15.64:0/2111927458 learned_addr learned my addr 172.21.15.64:0/2111927458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa15600640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff149f40 0x563dff1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa15e01640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa1a572740 10 client.0 _ll_register_callbacks cb 0x563dfedc1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa14dff640 1 -- 172.21.15.64:0/2111927458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff149f40 msgr2=0x563dff1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa14dff640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff149f40 0x563dff1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa14dff640 1 -- 172.21.15.64:0/2111927458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 msgr2=0x563dff15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa14dff640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa14dff640 1 -- 172.21.15.64:0/2111927458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dfee8c7d0 con 0x563dff14a930 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa15600640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff149f40 0x563dff1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.146+0000 7ffa14dff640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 0x563dff1549e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009900 tx=0x7ffa0c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295812bf850e1790 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.147+0000 7ff9f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.147+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/2111927458 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c035070 con 0x563dff14a930 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.147+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563dff0bc370 con 0x563dff14a930 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.147+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/2111927458 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa0c003da0 con 0x563dff14a930 2024-05-09T09:15:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.148+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/2111927458 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c04e050 con 0x563dff14a930 2024-05-09T09:15:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.148+0000 7ffa1a572740 10 client.26729 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.149+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563dff15eec0 con 0x563dff14a930 2024-05-09T09:15:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.149+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/2111927458 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa0c031a30 con 0x563dff14a930 2024-05-09T09:15:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.149+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/2111927458 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7ffa0c03c7c0 con 0x563dff14a930 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa1a572740 10 client.26729 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa1a572740 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa1a572740 20 client.26729 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa1a572740 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1605a0 0x563dff162a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa1a572740 4 client.26729 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dff0bc370 con 0x563dff1605a0 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.150+0000 7ffa15600640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1605a0 0x563dff162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.151+0000 7ffa15600640 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1605a0 0x563dff162a00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8bd35ecf94d4c51 server_cookie=ba19f2b512b59367 in_seq=0 out_seq=0 2024-05-09T09:15:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.151+0000 7ff9f8ff9640 10 client.26729 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.152+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/2111927458 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563dff0bc370 con 0x563dff1605a0 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.152+0000 7ff9f8ff9640 10 client.26729 handle_command_reply: tid=0 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.152+0000 7ffa1a572740 1 client.26729 shutdown 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 msgr2=0x563dff1549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 0x563dff1549e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009900 tx=0x7ffa0c003820 comp rx=0 tx=0).stop 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 shutdown_connections 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1605a0 0x563dff162a00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff14a930 0x563dff1549e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 --2- 172.21.15.64:0/2111927458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff149f40 0x563dff1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 >> 172.21.15.64:0/2111927458 conn(0x563dff1455b0 msgr2=0x563dff1447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ff9f8ff9640 0 client.26729 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 shutdown_connections 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.153+0000 7ffa1a572740 1 -- 172.21.15.64:0/2111927458 wait complete. 2024-05-09T09:15:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.154+0000 7ffa1a572740 20 client.26729 trim_cache size 0 max 16384 2024-05-09T09:15:51.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.185+0000 7ffa1a572740 1 Processor -- start 2024-05-09T09:15:51.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.186+0000 7ffa1a572740 1 -- start start 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.186+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff148330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.186+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14a930 0x563dff15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff148870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa15600640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa15600640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff148330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50120/0 (socket says 172.21.15.64:50120) 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14a930 0x563dff15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14a930 0x563dff15b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36890/0 (socket says 172.21.15.64:36890) 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa15600640 1 -- 172.21.15.64:0/3814259518 learned_addr learned my addr 172.21.15.64:0/3814259518 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101bbd30 con 0x563dff14faa0 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101b95a0 con 0x563dff149f40 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa15e01640 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.187+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101b73f0 con 0x563dff14a930 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.188+0000 7ffa15600640 1 -- 172.21.15.64:0/3814259518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14a930 msgr2=0x563dff15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.188+0000 7ffa15600640 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14a930 0x563dff15b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.188+0000 7ffa15600640 1 -- 172.21.15.64:0/3814259518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 msgr2=0x563dff148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.188+0000 7ffa15600640 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.188+0000 7ffa15600640 1 -- 172.21.15.64:0/3814259518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dff136d20 con 0x563dff149f40 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.189+0000 7ffa15600640 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff148330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa0009b0f0 tx=0x7ffa000d1140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533dfbc6c7c2f882 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.189+0000 7ff9faffd640 1 -- 172.21.15.64:0/3814259518 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000b3020 con 0x563dff149f40 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.189+0000 7ff9faffd640 1 -- 172.21.15.64:0/3814259518 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa0009b160 con 0x563dff149f40 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.190+0000 7ff9faffd640 1 -- 172.21.15.64:0/3814259518 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00020040 con 0x563dff149f40 2024-05-09T09:15:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 msgr2=0x563dff148330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff148330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa0009b0f0 tx=0x7ffa000d1140 comp rx=0 tx=0).stop 2024-05-09T09:15:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 shutdown_connections 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14faa0 0x563dff148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14a930 0x563dff15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 --2- 172.21.15.64:0/3814259518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff148330 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 >> 172.21.15.64:0/3814259518 conn(0x563dff1455b0 msgr2=0x563dff14bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 shutdown_connections 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 -- 172.21.15.64:0/3814259518 wait complete. 2024-05-09T09:15:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.191+0000 7ffa1a572740 1 -- start start 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.192+0000 7ffa1a572740 1 Processor -- start 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.192+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff159270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa1a572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff15a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa14dff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50374/0 (socket says 172.21.15.64:50374) 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa14dff640 1 -- 172.21.15.64:0/1550920419 learned_addr learned my addr 172.21.15.64:0/1550920419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101b7a60 con 0x563dff14a930 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101b8ed0 con 0x563dff149f40 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.193+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101b8130 con 0x563dff14faa0 2024-05-09T09:15:51.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa1a572740 10 client.0 _ll_register_callbacks cb 0x7ffa0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:51.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa14dff640 1 -- 172.21.15.64:0/1550920419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 msgr2=0x563dff15a230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:15:51.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa14dff640 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff15a230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa14dff640 1 -- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 msgr2=0x563dff158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa15600640 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa14dff640 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa14dff640 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dff15a820 con 0x563dff14a930 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.194+0000 7ffa14dff640 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff159270 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c004650 tx=0x7ffa0c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14570e48ed79d29e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.195+0000 7ff9f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.195+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/1550920419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c035040 con 0x563dff14a930 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.195+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/1550920419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa0c03c440 con 0x563dff14a930 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.195+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa0009b160 con 0x563dff14a930 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.195+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/1550920419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c009520 con 0x563dff14a930 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.195+0000 7ffa1a572740 10 client.17253 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:51.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.196+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563dff15bba0 con 0x563dff14a930 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.196+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/1550920419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa0c04b070 con 0x563dff14a930 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/1550920419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7ffa0c03bc80 con 0x563dff14a930 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ffa1a572740 10 client.17253 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ffa1a572740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ffa1a572740 20 client.17253 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1c7160 0x563dff1c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ffa1a572740 4 client.17253 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.197+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ffa0009b160 con 0x563dff1c7160 2024-05-09T09:15:51.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.198+0000 7ffa15600640 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1c7160 0x563dff1c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.198+0000 7ffa15600640 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1c7160 0x563dff1c9580 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1648cde5b2688eff server_cookie=9df08d3f3fd2c068 in_seq=0 out_seq=0 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.198+0000 7ff9f8ff9640 10 client.17253 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.199+0000 7ff9f8ff9640 1 -- 172.21.15.64:0/1550920419 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffa0009b160 con 0x563dff1c7160 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.199+0000 7ff9f8ff9640 10 client.17253 handle_command_reply: tid=0 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.199+0000 7ffa1a572740 1 client.17253 shutdown 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.200+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 msgr2=0x563dff159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.200+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff159270 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c004650 tx=0x7ffa0c005b50 comp rx=0 tx=0).stop 2024-05-09T09:15:51.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.201+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 shutdown_connections 2024-05-09T09:15:51.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.201+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563dff1c7160 0x563dff1c9580 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.201+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563dff14faa0 0x563dff15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.201+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563dff14a930 0x563dff159270 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.201+0000 7ffa1a572740 1 --2- 172.21.15.64:0/1550920419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dff149f40 0x563dff158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.201+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 >> 172.21.15.64:0/1550920419 conn(0x563dff1455b0 msgr2=0x563dff143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.203+0000 7ff9f8ff9640 0 client.17253 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.203+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 shutdown_connections 2024-05-09T09:15:51.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.203+0000 7ffa1a572740 1 -- 172.21.15.64:0/1550920419 wait complete. 2024-05-09T09:15:51.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.203+0000 7ffa1a572740 20 client.17253 trim_cache size 0 max 16384 2024-05-09T09:15:51.202 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:15:51.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9fc0a2d10 msgr2=0x7ff9fc0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9fc0a2d10 0x7ff9fc0a51d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800f010 tx=0x7ffa08019a80 comp rx=0 tx=0).stop 2024-05-09T09:15:51.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 msgr2=0x7ffa101959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa101959a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000d7a0 tx=0x7ffa0000dc60 comp rx=0 tx=0).stop 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 shutdown_connections 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9fc0a2d10 0x7ff9fc0a51d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa10105640 0x7ffa1019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa10104c80 0x7ffa10195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 --2- 172.21.15.64:0/2762956993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa10103a90 0x7ffa101959a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.205+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 >> 172.21.15.64:0/2762956993 conn(0x7ffa100ff020 msgr2=0x7ffa10077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.206+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 shutdown_connections 2024-05-09T09:15:51.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:51.206+0000 7ffa17864640 1 -- 172.21.15.64:0/2762956993 wait complete. 2024-05-09T09:15:51.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:51 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1276407794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:15:51.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:51 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1276407794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:15:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 330.0 s, 867.3 tps, lat 115.198 ms stddev 130.741 2024-05-09T09:15:51.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:51 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1276407794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:15:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.781+0000 7f1bacf8c640 1 -- 172.21.15.64:0/1840960354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba810d910 msgr2=0x7f1ba810ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.781+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/1840960354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba810d910 0x7f1ba810ddc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009860 tx=0x7f1b9c02a360 comp rx=0 tx=0).stop 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 -- 172.21.15.64:0/1840960354 shutdown_connections 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/1840960354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba810d910 0x7f1ba810ddc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/1840960354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ba810ae10 0x7f1ba810d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/1840960354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba806aa80 0x7f1ba810a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 -- 172.21.15.64:0/1840960354 >> 172.21.15.64:0/1840960354 conn(0x7f1ba806f550 msgr2=0x7f1ba8071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 -- 172.21.15.64:0/1840960354 shutdown_connections 2024-05-09T09:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.782+0000 7f1bacf8c640 1 -- 172.21.15.64:0/1840960354 wait complete. 2024-05-09T09:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.783+0000 7f1bacf8c640 1 Processor -- start 2024-05-09T09:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.783+0000 7f1bacf8c640 1 -- start start 2024-05-09T09:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1bacf8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba806aa80 0x7f1ba81a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1bacf8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 0x7f1ba81a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1bacf8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ba810d910 0x7f1ba81adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1bacf8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba811bfc0 con 0x7f1ba810d910 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1bacf8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba811be40 con 0x7f1ba810ae10 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1bacf8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba811c140 con 0x7f1ba806aa80 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1ba6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 0x7f1ba81a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1ba6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 0x7f1ba81a6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50160/0 (socket says 172.21.15.64:50160) 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/2455035893 learned_addr learned my addr 172.21.15.64:0/2455035893 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.784+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba806aa80 0x7f1ba81a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.785+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ba810d910 0x7f1ba81adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.785+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/2455035893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba806aa80 msgr2=0x7f1ba81a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.785+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba806aa80 0x7f1ba81a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.785+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ba810d910 msgr2=0x7f1ba81adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.785+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ba810d910 0x7f1ba81adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.785+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/2455035893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba81ae2c0 con 0x7f1ba810ae10 2024-05-09T09:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.786+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 0x7f1ba81a6be0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000ebc0 tx=0x7f1ba000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.786+0000 7f1ba4ff9640 1 -- 172.21.15.64:0/2455035893 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003970 con 0x7f1ba810ae10 2024-05-09T09:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.786+0000 7f1ba4ff9640 1 -- 172.21.15.64:0/2455035893 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba0004030 con 0x7f1ba810ae10 2024-05-09T09:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.786+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba81ae5b0 con 0x7f1ba810ae10 2024-05-09T09:15:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.786+0000 7f1ba4ff9640 1 -- 172.21.15.64:0/2455035893 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0019770 con 0x7f1ba810ae10 2024-05-09T09:15:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.786+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba81aeb50 con 0x7f1ba810ae10 2024-05-09T09:15:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.791+0000 7f1bb0fd9740 1 Processor -- start 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.791+0000 7f1bb0fd9740 1 -- start start 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.791+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b6413d0 0x56462b6417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.791+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 0x56462b6468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 0x56462b6468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 0x56462b6468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50170/0 (socket says 172.21.15.64:50170) 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b6493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56462b5a5340 con 0x56462b646f30 2024-05-09T09:15:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56462b54d290 con 0x56462b641dc0 2024-05-09T09:15:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56462b5a1300 con 0x56462b6413d0 2024-05-09T09:15:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba77fe640 1 -- 172.21.15.64:0/1243289900 learned_addr learned my addr 172.21.15.64:0/1243289900 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b6413d0 0x56462b6417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba77fe640 1 -- 172.21.15.64:0/1243289900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b6413d0 msgr2=0x56462b6417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b6413d0 0x56462b6417f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.792+0000 7f1ba77fe640 1 -- 172.21.15.64:0/1243289900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 msgr2=0x56462b6493c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.793+0000 7f1ba4ff9640 1 -- 172.21.15.64:0/2455035893 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ba0003b10 con 0x7f1ba810ae10 2024-05-09T09:15:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.793+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b6493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.793+0000 7f1ba77fe640 1 -- 172.21.15.64:0/1243289900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462b5718e0 con 0x56462b641dc0 2024-05-09T09:15:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.793+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 0x56462b6468c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009cd0 tx=0x7f1b98011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd15802548d0706 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.794+0000 7f1ba4ff9640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b900a2cc0 0x7f1b900a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.795+0000 7f1ba4ff9640 1 -- 172.21.15.64:0/2455035893 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ba00cfc40 con 0x7f1ba810ae10 2024-05-09T09:15:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.795+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b900a2cc0 0x7f1b900a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.795+0000 7f1b8dffb640 1 -- 172.21.15.64:0/1243289900 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9801e070 con 0x56462b641dc0 2024-05-09T09:15:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.796+0000 7f1b8dffb640 1 -- 172.21.15.64:0/1243289900 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b98019ae0 con 0x56462b641dc0 2024-05-09T09:15:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.796+0000 7f1b8dffb640 1 -- 172.21.15.64:0/1243289900 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9800ea40 con 0x56462b641dc0 2024-05-09T09:15:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.797+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1243289900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 msgr2=0x56462b6468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.797+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 0x56462b6468c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009cd0 tx=0x7f1b98011810 comp rx=0 tx=0).stop 2024-05-09T09:15:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.798+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b900a2cc0 0x7f1b900a5180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56462b642620 tx=0x7f1b9c030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.800+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1243289900 shutdown_connections 2024-05-09T09:15:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.800+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b6493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.800+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b641dc0 0x56462b6468c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.800+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1243289900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b6413d0 0x56462b6417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.800+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1243289900 >> 172.21.15.64:0/1243289900 conn(0x56462b63ca40 msgr2=0x56462b63ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.800+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1243289900 shutdown_connections 2024-05-09T09:15:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.801+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1243289900 wait complete. 2024-05-09T09:15:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.801+0000 7f1bb0fd9740 1 -- start start 2024-05-09T09:15:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.801+0000 7f1bb0fd9740 1 Processor -- start 2024-05-09T09:15:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b6413d0 0x56462b64b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b64bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b646f30 0x56462b652cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56462b649ee0 con 0x56462b6413d0 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56462b560790 con 0x56462b646f30 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56462b64a060 con 0x56462b641dc0 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.802+0000 7f1bb0fd9740 10 client.0 _ll_register_callbacks cb 0x56462b59ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.803+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b646f30 0x56462b652cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.803+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b646f30 0x56462b652cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50178/0 (socket says 172.21.15.64:50178) 2024-05-09T09:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.803+0000 7f1ba7fff640 1 -- 172.21.15.64:0/3146090440 learned_addr learned my addr 172.21.15.64:0/3146090440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.803+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b64bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.803+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b6413d0 0x56462b64b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.804+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b646f30 msgr2=0x56462b652cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.804+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b646f30 0x56462b652cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.804+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/3146090440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b6413d0 msgr2=0x56462b64b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.804+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b6413d0 0x56462b64b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.804+0000 7f1ba6ffd640 1 -- 172.21.15.64:0/3146090440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462b2b9210 con 0x56462b641dc0 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.805+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b64bcb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0005c20 tx=0x7f1ba0003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0a17b5470cd4a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.805+0000 7f1b86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.805+0000 7f1b86ffd640 1 -- 172.21.15.64:0/3146090440 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00a0460 con 0x56462b641dc0 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.805+0000 7f1b86ffd640 1 -- 172.21.15.64:0/3146090440 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ba00a00d0 con 0x56462b641dc0 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.805+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b98019ae0 con 0x56462b641dc0 2024-05-09T09:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.805+0000 7f1b86ffd640 1 -- 172.21.15.64:0/3146090440 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1ba00a4d20 con 0x56462b641dc0 2024-05-09T09:15:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.807+0000 7f1bb0fd9740 10 client.26917 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.807+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56462b385620 con 0x56462b641dc0 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1b86ffd640 1 -- 172.21.15.64:0/3146090440 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b98019ae0 con 0x56462b641dc0 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1b86ffd640 1 -- 172.21.15.64:0/3146090440 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f1ba0097e00 con 0x56462b641dc0 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1bb0fd9740 10 client.26917 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1bb0fd9740 10 client.26917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1bb0fd9740 20 client.26917 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6571f0 0x56462b659640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1bb0fd9740 4 client.26917 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56462b5b2a70 con 0x56462b6571f0 2024-05-09T09:15:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.816+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6571f0 0x56462b659640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.818+0000 7f1b86ffd640 10 client.26917 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.819+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6571f0 0x56462b659640 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5fcb4e36bb6908 server_cookie=120cceee4598309c in_seq=0 out_seq=0 2024-05-09T09:15:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.822+0000 7f1b86ffd640 1 -- 172.21.15.64:0/3146090440 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56462b5b2a70 con 0x56462b6571f0 2024-05-09T09:15:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.822+0000 7f1b86ffd640 10 client.26917 handle_command_reply: tid=0 2024-05-09T09:15:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.822+0000 7f1bb0fd9740 1 client.26917 shutdown 2024-05-09T09:15:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.822+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 msgr2=0x56462b64bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.822+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b64bcb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0005c20 tx=0x7f1ba0003e00 comp rx=0 tx=0).stop 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.823+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 shutdown_connections 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.823+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6571f0 0x56462b659640 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.823+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b646f30 0x56462b652cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.823+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b64bcb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.823+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/3146090440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b6413d0 0x56462b64b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.823+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 >> 172.21.15.64:0/3146090440 conn(0x56462b63ca40 msgr2=0x56462b63bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.824+0000 7f1b86ffd640 0 client.26917 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.824+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 shutdown_connections 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.824+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/3146090440 wait complete. 2024-05-09T09:15:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.825+0000 7f1bb0fd9740 20 client.26917 trim_cache size 0 max 16384 2024-05-09T09:15:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.864+0000 7f1bb0fd9740 1 Processor -- start 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 -- start start 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b6430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b652cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b643600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94012570 con 0x56462b646f30 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9400fde0 con 0x56462b6413d0 2024-05-09T09:15:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9400dc30 con 0x56462b641dc0 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b643600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b643600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50406/0 (socket says 172.21.15.64:50406) 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.865+0000 7f1ba7fff640 1 -- 172.21.15.64:0/1811102085 learned_addr learned my addr 172.21.15.64:0/1811102085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba6ffd640 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b652cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba7fff640 1 -- 172.21.15.64:0/1811102085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 msgr2=0x56462b652cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b652cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba7fff640 1 -- 172.21.15.64:0/1811102085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 msgr2=0x56462b6430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b6430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba7fff640 1 -- 172.21.15.64:0/1811102085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462b62e330 con 0x56462b646f30 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.866+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b643600 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c035040 tx=0x7f1b9c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9760271b75d0c744 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.867+0000 7f1b8d7fa640 1 -- 172.21.15.64:0/1811102085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c037040 con 0x56462b646f30 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.867+0000 7f1b8d7fa640 1 -- 172.21.15.64:0/1811102085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b9c04e070 con 0x56462b646f30 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.867+0000 7f1b8d7fa640 1 -- 172.21.15.64:0/1811102085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c004d40 con 0x56462b646f30 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1811102085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 msgr2=0x56462b643600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b643600 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c035040 tx=0x7f1b9c03d000 comp rx=0 tx=0).stop 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1811102085 shutdown_connections 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b646f30 0x56462b643600 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b641dc0 0x56462b652cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/1811102085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b6430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.868+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1811102085 >> 172.21.15.64:0/1811102085 conn(0x56462b63ca40 msgr2=0x56462b64b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.869+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1811102085 shutdown_connections 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.869+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/1811102085 wait complete. 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.869+0000 7f1bb0fd9740 1 -- start start 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 Processor -- start 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b650740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b641dc0 0x56462b650c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 0x56462b651c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9400e2a0 con 0x56462b641dc0 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9400f710 con 0x56462b6413d0 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9400e970 con 0x56462b646f30 2024-05-09T09:15:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.870+0000 7f1bb0fd9740 10 client.0 _ll_register_callbacks cb 0x7f1ba009bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 0x56462b651c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 0x56462b651c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36938/0 (socket says 172.21.15.64:36938) 2024-05-09T09:15:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 -- 172.21.15.64:0/2792300523 learned_addr learned my addr 172.21.15.64:0/2792300523 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b650740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 -- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 msgr2=0x56462b650740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b650740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 -- 172.21.15.64:0/2792300523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b641dc0 msgr2=0x56462b650c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b641dc0 0x56462b650c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 -- 172.21.15.64:0/2792300523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462b652210 con 0x56462b646f30 2024-05-09T09:15:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.871+0000 7f1ba7fff640 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 0x56462b651c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009250 tx=0x7f1b9c004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944e1a0055e42950 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.872+0000 7f1b86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.872+0000 7f1b86ffd640 1 -- 172.21.15.64:0/2792300523 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c056020 con 0x56462b646f30 2024-05-09T09:15:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.872+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9c04e070 con 0x56462b646f30 2024-05-09T09:15:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.872+0000 7f1b86ffd640 1 -- 172.21.15.64:0/2792300523 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1b9c003a00 con 0x56462b646f30 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.872+0000 7f1b86ffd640 1 -- 172.21.15.64:0/2792300523 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c061050 con 0x56462b646f30 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.872+0000 7f1bb0fd9740 10 client.26920 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56462b6ceba0 con 0x56462b646f30 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1b86ffd640 1 -- 172.21.15.64:0/2792300523 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b9c04e070 con 0x56462b646f30 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1b86ffd640 1 -- 172.21.15.64:0/2792300523 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f1b9c032c90 con 0x56462b646f30 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 10 client.26920 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 20 client.26920 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6cf5b0 0x56462b6d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 4 client.26920 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:15:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.873+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56462b642bd0 con 0x56462b6cf5b0 2024-05-09T09:15:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.874+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6cf5b0 0x56462b6d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.874+0000 7f1b86ffd640 10 client.26920 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.874+0000 7f1ba77fe640 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6cf5b0 0x56462b6d19d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403f4f3677d6d01b server_cookie=42607e77909cffba in_seq=0 out_seq=0 2024-05-09T09:15:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.905+0000 7f1b86ffd640 1 -- 172.21.15.64:0/2792300523 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56462b642bd0 con 0x56462b6cf5b0 2024-05-09T09:15:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.905+0000 7f1b86ffd640 10 client.26920 handle_command_reply: tid=0 2024-05-09T09:15:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.905+0000 7f1bb0fd9740 1 client.26920 shutdown 2024-05-09T09:15:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 msgr2=0x56462b651c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 0x56462b651c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009250 tx=0x7f1b9c004820 comp rx=0 tx=0).stop 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 shutdown_connections 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56462b6cf5b0 0x56462b6d19d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56462b646f30 0x56462b651c20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56462b641dc0 0x56462b650c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 --2- 172.21.15.64:0/2792300523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56462b6413d0 0x56462b650740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 >> 172.21.15.64:0/2792300523 conn(0x56462b63ca40 msgr2=0x56462b64b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1b86ffd640 0 client.26920 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 shutdown_connections 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 1 -- 172.21.15.64:0/2792300523 wait complete. 2024-05-09T09:15:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.908+0000 7f1bb0fd9740 20 client.26920 trim_cache size 0 max 16384 2024-05-09T09:15:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:15:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:15:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:15:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:15:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.909+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b900a2cc0 msgr2=0x7f1b900a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.909+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b900a2cc0 0x7f1b900a5180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56462b642620 tx=0x7f1b9c030450 comp rx=0 tx=0).stop 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.909+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 msgr2=0x7f1ba81a6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.909+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 0x7f1ba81a6be0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000ebc0 tx=0x7f1ba000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 shutdown_connections 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b900a2cc0 0x7f1b900a5180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ba810d910 0x7f1ba81adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ba810ae10 0x7f1ba81a6be0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 --2- 172.21.15.64:0/2455035893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba806aa80 0x7f1ba81a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 >> 172.21.15.64:0/2455035893 conn(0x7f1ba806f550 msgr2=0x7f1ba8108cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 shutdown_connections 2024-05-09T09:15:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:15:51.910+0000 7f1bacf8c640 1 -- 172.21.15.64:0/2455035893 wait complete. 2024-05-09T09:15:52.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:15:52.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:15:52.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:15:52.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:52 smithi139 ceph-mon[33599]: pgmap v572: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 851 op/s 2024-05-09T09:15:52.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:52 smithi064 ceph-mon[30149]: pgmap v572: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 851 op/s 2024-05-09T09:15:52.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 -- 172.21.15.64:0/2061682586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf810e370 msgr2=0x7f6bf8110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 --2- 172.21.15.64:0/2061682586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf810e370 0x7f6bf8110830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c860 tx=0x7f6be802b600 comp rx=0 tx=0).stop 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 -- 172.21.15.64:0/2061682586 shutdown_connections 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 --2- 172.21.15.64:0/2061682586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf810e370 0x7f6bf8110830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 --2- 172.21.15.64:0/2061682586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf810b9f0 0x7f6bf810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 --2- 172.21.15.64:0/2061682586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bf81090c0 0x7f6bf810b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 -- 172.21.15.64:0/2061682586 >> 172.21.15.64:0/2061682586 conn(0x7f6bf806ea60 msgr2=0x7f6bf8070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 -- 172.21.15.64:0/2061682586 shutdown_connections 2024-05-09T09:15:52.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.439+0000 7f6c005f4640 1 -- 172.21.15.64:0/2061682586 wait complete. 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.440+0000 7f6c005f4640 1 Processor -- start 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.440+0000 7f6c005f4640 1 -- start start 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6c005f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 0x7f6bf819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6c005f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bf810b9f0 0x7f6bf819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6c005f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf810e370 0x7f6bf81a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6c005f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8113a50 con 0x7f6bf810e370 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6c005f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81138d0 con 0x7f6bf81090c0 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6c005f4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8113bd0 con 0x7f6bf810b9f0 2024-05-09T09:15:52.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6bfe390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 0x7f6bf819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6bfdb8f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bf810b9f0 0x7f6bf819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6bfe390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 0x7f6bf819e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50208/0 (socket says 172.21.15.64:50208) 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.441+0000 7f6bfe390640 1 -- 172.21.15.64:0/744663068 learned_addr learned my addr 172.21.15.64:0/744663068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.442+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf810e370 0x7f6bf81a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.442+0000 7f6bfe390640 1 -- 172.21.15.64:0/744663068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bf810b9f0 msgr2=0x7f6bf819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.442+0000 7f6bfe390640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bf810b9f0 0x7f6bf819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.442+0000 7f6bfe390640 1 -- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf810e370 msgr2=0x7f6bf81a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.442+0000 7f6bfe390640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf810e370 0x7f6bf81a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.442+0000 7f6bfe390640 1 -- 172.21.15.64:0/744663068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf81b6db0 con 0x7f6bf81090c0 2024-05-09T09:15:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.443+0000 7f6bfe390640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 0x7f6bf819e170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000b480 tx=0x7f6bf000b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.443+0000 7f6bef7fe640 1 -- 172.21.15.64:0/744663068 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0015020 con 0x7f6bf81090c0 2024-05-09T09:15:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.443+0000 7f6bef7fe640 1 -- 172.21.15.64:0/744663068 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bf0011ce0 con 0x7f6bf81090c0 2024-05-09T09:15:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.443+0000 7f6bef7fe640 1 -- 172.21.15.64:0/744663068 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0010db0 con 0x7f6bf81090c0 2024-05-09T09:15:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.443+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81b70a0 con 0x7f6bf81090c0 2024-05-09T09:15:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.444+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81b7620 con 0x7f6bf81090c0 2024-05-09T09:15:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.447+0000 7f6c033d7740 1 Processor -- start 2024-05-09T09:15:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.448+0000 7f6c033d7740 1 -- start start 2024-05-09T09:15:52.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.448+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6bfc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6bfceb0 0x55faa6c019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6bfc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6bfc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36956/0 (socket says 172.21.15.64:36956) 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 -- 172.21.15.64:0/3348126357 learned_addr learned my addr 172.21.15.64:0/3348126357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6c033d7740 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c02020 0x55faa6c044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faa6b60990 con 0x55faa6bfceb0 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55faa6b085b0 con 0x55faa6c02020 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55faa6b5c950 con 0x55faa6bfc4c0 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 -- 172.21.15.64:0/3348126357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c02020 msgr2=0x55faa6c044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c02020 0x55faa6c044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 -- 172.21.15.64:0/3348126357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6bfceb0 msgr2=0x55faa6c019b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6bfceb0 0x55faa6c019b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.449+0000 7f6bfeb91640 1 -- 172.21.15.64:0/3348126357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa693c7e0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.450+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6bfc8e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6be8035000 tx=0x7f6be8031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eaf8dd95456008f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.450+0000 7f6becbfe640 1 -- 172.21.15.64:0/3348126357 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8031a70 con 0x55faa6bfc4c0 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.450+0000 7f6becbfe640 1 -- 172.21.15.64:0/3348126357 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6be80054f0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.450+0000 7f6becbfe640 1 -- 172.21.15.64:0/3348126357 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8004bf0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6bef7fe640 1 -- 172.21.15.64:0/744663068 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6bf0018020 con 0x7f6bf81090c0 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 msgr2=0x55faa6bfc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6bfc8e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6be8035000 tx=0x7f6be8031700 comp rx=0 tx=0).stop 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 shutdown_connections 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c02020 0x55faa6c044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6bfceb0 0x55faa6c019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 --2- 172.21.15.64:0/3348126357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6bfc8e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.451+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 >> 172.21.15.64:0/3348126357 conn(0x55faa6bc40a0 msgr2=0x55faa6bf7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.452+0000 7f6bef7fe640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6be40a2d10 0x7f6be40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.452+0000 7f6bef7fe640 1 -- 172.21.15.64:0/744663068 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6bf005ed00 con 0x7f6bf81090c0 2024-05-09T09:15:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.453+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 shutdown_connections 2024-05-09T09:15:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.454+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6be40a2d10 0x7f6be40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.454+0000 7f6c033d7740 1 -- 172.21.15.64:0/3348126357 wait complete. 2024-05-09T09:15:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.455+0000 7f6c033d7740 1 -- start start 2024-05-09T09:15:52.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.455+0000 7f6c033d7740 1 Processor -- start 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c06930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c0bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 0x55faa6c0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faa6c04f70 con 0x55faa6c06e70 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55faa6b1bb40 con 0x55faa6c072f0 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55faa6c050f0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6c033d7740 10 client.0 _ll_register_callbacks cb 0x55faa6b100f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6bfeb91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 0x55faa6c0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6bfeb91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 0x55faa6c0e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50228/0 (socket says 172.21.15.64:50228) 2024-05-09T09:15:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.456+0000 7f6bfeb91640 1 -- 172.21.15.64:0/2580168205 learned_addr learned my addr 172.21.15.64:0/2580168205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfe390640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c06930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfeb91640 1 -- 172.21.15.64:0/2580168205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 msgr2=0x55faa6c06930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c06930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfeb91640 1 -- 172.21.15.64:0/2580168205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 msgr2=0x55faa6c0bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c0bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfeb91640 1 -- 172.21.15.64:0/2580168205 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa699cf70 con 0x55faa6c072f0 2024-05-09T09:15:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfe390640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c06930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.457+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 0x55faa6c0e430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6be802bbf0 tx=0x7f6be8004f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9516a4b9af11dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.458+0000 7f6be1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.458+0000 7f6be1ffb640 1 -- 172.21.15.64:0/2580168205 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8053020 con 0x55faa6c072f0 2024-05-09T09:15:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.458+0000 7f6be1ffb640 1 -- 172.21.15.64:0/2580168205 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6be8045440 con 0x55faa6c072f0 2024-05-09T09:15:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.459+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be80054f0 con 0x55faa6c072f0 2024-05-09T09:15:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.459+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6be40a2d10 0x7f6be40a51d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55faa6bf8df0 tx=0x7f6bf4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.459+0000 7f6be1ffb640 1 -- 172.21.15.64:0/2580168205 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6be804a040 con 0x55faa6c072f0 2024-05-09T09:15:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.460+0000 7f6c033d7740 10 client.26756 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:52.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.460+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55faa6834170 con 0x55faa6c072f0 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6be1ffb640 1 -- 172.21.15.64:0/2580168205 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6be80054f0 con 0x55faa6c072f0 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6be1ffb640 1 -- 172.21.15.64:0/2580168205 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f6be8060020 con 0x55faa6c072f0 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6c033d7740 10 client.26756 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6c033d7740 10 client.26756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6c033d7740 20 client.26756 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6c033d7740 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c127d0 0x55faa6c14bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6c033d7740 4 client.26756 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:52.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa6b6e0c0 con 0x55faa6c127d0 2024-05-09T09:15:52.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.464+0000 7f6bfe390640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c127d0 0x55faa6c14bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.467+0000 7f6be1ffb640 10 client.26756 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.467+0000 7f6bfe390640 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c127d0 0x55faa6c14bc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c7c2de3078b3c5 server_cookie=5c110346d199ac8f in_seq=0 out_seq=0 2024-05-09T09:15:52.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6be1ffb640 1 -- 172.21.15.64:0/2580168205 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55faa6b6e0c0 con 0x55faa6c127d0 2024-05-09T09:15:52.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6be1ffb640 10 client.26756 handle_command_reply: tid=0 2024-05-09T09:15:52.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 client.26756 shutdown 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 msgr2=0x55faa6c0e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 0x55faa6c0e430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6be802bbf0 tx=0x7f6be8004f30 comp rx=0 tx=0).stop 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 shutdown_connections 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c127d0 0x55faa6c14bc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c072f0 0x55faa6c0e430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c0bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 --2- 172.21.15.64:0/2580168205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c06930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.471+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 >> 172.21.15.64:0/2580168205 conn(0x55faa6bc40a0 msgr2=0x55faa6bf7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.472+0000 7f6be1ffb640 0 client.26756 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.472+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 shutdown_connections 2024-05-09T09:15:52.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.472+0000 7f6c033d7740 1 -- 172.21.15.64:0/2580168205 wait complete. 2024-05-09T09:15:52.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.472+0000 7f6c033d7740 20 client.26756 trim_cache size 0 max 16384 2024-05-09T09:15:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:52 smithi179 ceph-mon[34086]: pgmap v572: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 851 op/s 2024-05-09T09:15:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.509+0000 7f6c033d7740 1 Processor -- start 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 -- start start 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6bfc4c0 0x55faa6c1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c1a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6c072f0 0x55faa6c0de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81c4430 con 0x55faa6c06e70 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81c1ca0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6c033d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81bfaf0 con 0x55faa6c072f0 2024-05-09T09:15:52.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6bfdb8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c1a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6bfdb8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c1a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50450/0 (socket says 172.21.15.64:50450) 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.510+0000 7f6bfdb8f640 1 -- 172.21.15.64:0/1121685933 learned_addr learned my addr 172.21.15.64:0/1121685933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfeb91640 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6c072f0 0x55faa6c0de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfdb8f640 1 -- 172.21.15.64:0/1121685933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6c072f0 msgr2=0x55faa6c0de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6c072f0 0x55faa6c0de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfdb8f640 1 -- 172.21.15.64:0/1121685933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6bfc4c0 msgr2=0x55faa6c1a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6bfc4c0 0x55faa6c1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfdb8f640 1 -- 172.21.15.64:0/1121685933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa6bf8bf0 con 0x55faa6c06e70 2024-05-09T09:15:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.511+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c1a920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8073a30 tx=0x7f6bf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285a8b0e7f554a62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.512+0000 7f6be3fff640 1 -- 172.21.15.64:0/1121685933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4004b20 con 0x55faa6c06e70 2024-05-09T09:15:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.512+0000 7f6be3fff640 1 -- 172.21.15.64:0/1121685933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bf4004cc0 con 0x55faa6c06e70 2024-05-09T09:15:52.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.512+0000 7f6be3fff640 1 -- 172.21.15.64:0/1121685933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4005450 con 0x55faa6c06e70 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 -- 172.21.15.64:0/1121685933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 msgr2=0x55faa6c1a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c1a920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8073a30 tx=0x7f6bf4004ab0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 -- 172.21.15.64:0/1121685933 shutdown_connections 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6c072f0 0x55faa6c0de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c06e70 0x55faa6c1a920 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1121685933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6bfc4c0 0x55faa6c1a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.513+0000 7f6c033d7740 1 -- 172.21.15.64:0/1121685933 >> 172.21.15.64:0/1121685933 conn(0x55faa6bc40a0 msgr2=0x55faa6bf2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.514+0000 7f6c033d7740 1 -- 172.21.15.64:0/1121685933 shutdown_connections 2024-05-09T09:15:52.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.514+0000 7f6c033d7740 1 -- 172.21.15.64:0/1121685933 wait complete. 2024-05-09T09:15:52.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.514+0000 7f6c033d7740 1 -- start start 2024-05-09T09:15:52.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.514+0000 7f6c033d7740 1 Processor -- start 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c0c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c06e70 0x55faa6c0c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c072f0 0x55faa6c0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81c0160 con 0x55faa6c072f0 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81c15d0 con 0x55faa6c06e70 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81c0830 con 0x55faa6bfc4c0 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6c033d7740 10 client.0 _ll_register_callbacks cb 0x7f6be8031650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c0c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c0c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36994/0 (socket says 172.21.15.64:36994) 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 -- 172.21.15.64:0/1013658726 learned_addr learned my addr 172.21.15.64:0/1013658726 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c06e70 0x55faa6c0c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 -- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c06e70 msgr2=0x55faa6c0c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c06e70 0x55faa6c0c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 -- 172.21.15.64:0/1013658726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c072f0 msgr2=0x55faa6c0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c072f0 0x55faa6c0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.515+0000 7f6bfe390640 1 -- 172.21.15.64:0/1013658726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa6c0df10 con 0x55faa6bfc4c0 2024-05-09T09:15:52.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.516+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c06e70 0x55faa6c0c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:15:52.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.516+0000 7f6bfe390640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c0c3d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00979d0 tx=0x7f6bf009aa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae07d7c051cd24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.516+0000 7f6be1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.516+0000 7f6be1ffb640 1 -- 172.21.15.64:0/1013658726 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00bb0b0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.516+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf4004cc0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.516+0000 7f6be1ffb640 1 -- 172.21.15.64:0/1013658726 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6bf00cc3b0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.517+0000 7f6be1ffb640 1 -- 172.21.15.64:0/1013658726 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6bf00104a0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.517+0000 7f6c033d7740 10 client.26932 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:52.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55faa6c79600 con 0x55faa6bfc4c0 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6be1ffb640 1 -- 172.21.15.64:0/1013658726 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bf4004cc0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6be1ffb640 1 -- 172.21.15.64:0/1013658726 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f6bf00a38f0 con 0x55faa6bfc4c0 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 10 client.26932 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 10 client.26932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 20 client.26932 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c79f90 0x55faa6c7c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 4 client.26932 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:15:52.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.518+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55faa6b68410 con 0x55faa6c79f90 2024-05-09T09:15:52.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.522+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c79f90 0x55faa6c7c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.523+0000 7f6bfdb8f640 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c79f90 0x55faa6c7c3b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9f8c2994dba74 server_cookie=b760b3f87e0ef89f in_seq=0 out_seq=0 2024-05-09T09:15:52.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.523+0000 7f6be1ffb640 10 client.26932 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.526+0000 7f6be1ffb640 1 -- 172.21.15.64:0/1013658726 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55faa6b68410 con 0x55faa6c79f90 2024-05-09T09:15:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.526+0000 7f6be1ffb640 10 client.26932 handle_command_reply: tid=0 2024-05-09T09:15:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.526+0000 7f6c033d7740 1 client.26932 shutdown 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.528+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 msgr2=0x55faa6c0c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.528+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c0c3d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00979d0 tx=0x7f6bf009aa90 comp rx=0 tx=0).stop 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.528+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 shutdown_connections 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55faa6c79f90 0x55faa6c7c3b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faa6c072f0 0x55faa6c0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faa6c06e70 0x55faa6c0c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 --2- 172.21.15.64:0/1013658726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55faa6bfc4c0 0x55faa6c0c3d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 >> 172.21.15.64:0/1013658726 conn(0x55faa6bc40a0 msgr2=0x55faa6bfae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6be1ffb640 0 client.26932 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 shutdown_connections 2024-05-09T09:15:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 1 -- 172.21.15.64:0/1013658726 wait complete. 2024-05-09T09:15:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.529+0000 7f6c033d7740 20 client.26932 trim_cache size 0 max 16384 2024-05-09T09:15:52.530 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:15:52.530 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:15:52.530 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "c6fb6324-3da7-4a4f-b3a3-0acf219e3551", 2024-05-09T09:15:52.530 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:15:52.530 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:15:52.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.531+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6be40a2d10 msgr2=0x7f6be40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.531+0000 7f6c005f4640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6be40a2d10 0x7f6be40a51d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55faa6bf8df0 tx=0x7f6bf4010040 comp rx=0 tx=0).stop 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.531+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 msgr2=0x7f6bf819e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.531+0000 7f6c005f4640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 0x7f6bf819e170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000b480 tx=0x7f6bf000b940 comp rx=0 tx=0).stop 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.532+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 shutdown_connections 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.532+0000 7f6c005f4640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6be40a2d10 0x7f6be40a51d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.532+0000 7f6c005f4640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf810e370 0x7f6bf81a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.532+0000 7f6c005f4640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bf810b9f0 0x7f6bf819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.532+0000 7f6c005f4640 1 --2- 172.21.15.64:0/744663068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bf81090c0 0x7f6bf819e170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.532+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 >> 172.21.15.64:0/744663068 conn(0x7f6bf806ea60 msgr2=0x7f6bf8109980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.534+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 shutdown_connections 2024-05-09T09:15:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.535+0000 7f6c005f4640 1 -- 172.21.15.64:0/744663068 wait complete. 2024-05-09T09:15:52.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:15:52.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:15:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.770+0000 7f77d194c640 1 -- 172.21.15.64:0/4046062375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc108830 msgr2=0x7f77cc10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.770+0000 7f77d194c640 1 --2- 172.21.15.64:0/4046062375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc108830 0x7f77cc10ac60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f77bc01a910 tx=0x7f77bc03b470 comp rx=0 tx=0).stop 2024-05-09T09:15:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.771+0000 7f77d194c640 1 -- 172.21.15.64:0/4046062375 shutdown_connections 2024-05-09T09:15:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.771+0000 7f77d194c640 1 --2- 172.21.15.64:0/4046062375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77cc10db70 0x7f77cc10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.771+0000 7f77d194c640 1 --2- 172.21.15.64:0/4046062375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10b1a0 0x7f77cc10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.771+0000 7f77d194c640 1 --2- 172.21.15.64:0/4046062375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc108830 0x7f77cc10ac60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.771+0000 7f77d194c640 1 -- 172.21.15.64:0/4046062375 >> 172.21.15.64:0/4046062375 conn(0x7f77cc06e6c0 msgr2=0x7f77cc070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.771+0000 7f77d194c640 1 -- 172.21.15.64:0/4046062375 shutdown_connections 2024-05-09T09:15:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.773+0000 7f77d194c640 1 -- 172.21.15.64:0/4046062375 wait complete. 2024-05-09T09:15:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 Processor -- start 2024-05-09T09:15:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 -- start start 2024-05-09T09:15:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77cc108830 0x7f77cc19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc10b1a0 0x7f77cc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10db70 0x7f77cc1c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc111af0 con 0x7f77cc10b1a0 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc111970 con 0x7f77cc10db70 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.774+0000 7f77d194c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc111c70 con 0x7f77cc108830 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.776+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10db70 0x7f77cc1c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.776+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10db70 0x7f77cc1c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50270/0 (socket says 172.21.15.64:50270) 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.776+0000 7f77cb7fe640 1 -- 172.21.15.64:0/368116946 learned_addr learned my addr 172.21.15.64:0/368116946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.776+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc10b1a0 0x7f77cc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77ca7fc640 1 -- 172.21.15.64:0/368116946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77cc108830 msgr2=0x7f77cc19e260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:15:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77cc108830 0x7f77cc19e260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77ca7fc640 1 -- 172.21.15.64:0/368116946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10db70 msgr2=0x7f77cc1c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10db70 0x7f77cc1c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77ca7fc640 1 -- 172.21.15.64:0/368116946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc1c7f00 con 0x7f77cc10b1a0 2024-05-09T09:15:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc10b1a0 0x7f77cc19e7a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009fd0 tx=0x7f77c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77d094a640 1 -- 172.21.15.64:0/368116946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0015020 con 0x7f77cc10b1a0 2024-05-09T09:15:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.777+0000 7f77d094a640 1 -- 172.21.15.64:0/368116946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77c0011ce0 con 0x7f77cc10b1a0 2024-05-09T09:15:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.778+0000 7f77d094a640 1 -- 172.21.15.64:0/368116946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0010e00 con 0x7f77cc10b1a0 2024-05-09T09:15:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.778+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77cc1c81f0 con 0x7f77cc10b1a0 2024-05-09T09:15:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.778+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77cc1c8810 con 0x7f77cc10b1a0 2024-05-09T09:15:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.783+0000 7f77d4658740 1 Processor -- start 2024-05-09T09:15:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 -- start start 2024-05-09T09:15:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a23420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a25f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a1e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50272/0 (socket says 172.21.15.64:50272) 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77cb7fe640 1 -- 172.21.15.64:0/373728419 learned_addr learned my addr 172.21.15.64:0/373728419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e65982d00 con 0x558e65a23a90 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e6592a4c0 con 0x558e65a1df30 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e6597ecc0 con 0x558e65a1e920 2024-05-09T09:15:52.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.784+0000 7f77caffd640 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a23420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77d094a640 1 -- 172.21.15.64:0/368116946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77c0018020 con 0x7f77cc10b1a0 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77caffd640 1 -- 172.21.15.64:0/373728419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 msgr2=0x558e65a1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77caffd640 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77caffd640 1 -- 172.21.15.64:0/373728419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 msgr2=0x558e65a25f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77caffd640 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a25f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77caffd640 1 -- 172.21.15.64:0/373728419 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e657607d0 con 0x558e65a1e920 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77caffd640 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a23420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f77bc006010 tx=0x7f77bc004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49eb3bf73db00cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.785+0000 7f77b1ffb640 1 -- 172.21.15.64:0/373728419 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc012020 con 0x558e65a1e920 2024-05-09T09:15:52.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.786+0000 7f77b1ffb640 1 -- 172.21.15.64:0/373728419 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77bc004960 con 0x558e65a1e920 2024-05-09T09:15:52.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.786+0000 7f77b1ffb640 1 -- 172.21.15.64:0/373728419 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc046030 con 0x558e65a1e920 2024-05-09T09:15:52.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.786+0000 7f77d094a640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f77b40a2ce0 0x7f77b40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d094a640 1 -- 172.21.15.64:0/368116946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f77c00cebe0 con 0x7f77cc10b1a0 2024-05-09T09:15:52.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 msgr2=0x558e65a23420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d4658740 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a23420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f77bc006010 tx=0x7f77bc004870 comp rx=0 tx=0).stop 2024-05-09T09:15:52.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f77b40a2ce0 0x7f77b40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 shutdown_connections 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d4658740 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a25f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d4658740 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a23420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.787+0000 7f77d4658740 1 --2- 172.21.15.64:0/373728419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.788+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 >> 172.21.15.64:0/373728419 conn(0x558e65a195a0 msgr2=0x558e65a199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.788+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f77b40a2ce0 0x7f77b40a51a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x558e65a1f180 tx=0x7f77c400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.789+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 shutdown_connections 2024-05-09T09:15:52.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.789+0000 7f77d4658740 1 -- 172.21.15.64:0/373728419 wait complete. 2024-05-09T09:15:52.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.789+0000 7f77d4658740 1 -- start start 2024-05-09T09:15:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.790+0000 7f77d4658740 1 Processor -- start 2024-05-09T09:15:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.790+0000 7f77d4658740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77d4658740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77d4658740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a2f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77caffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37018/0 (socket says 172.21.15.64:37018) 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 -- 172.21.15.64:0/2400576504 learned_addr learned my addr 172.21.15.64:0/2400576504 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e65a269e0 con 0x558e65a23a90 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e6593db90 con 0x558e65a1df30 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e65a26b60 con 0x558e65a1e920 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a2f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 -- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 msgr2=0x558e65a28260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a28260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 -- 172.21.15.64:0/2400576504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 msgr2=0x558e65a2f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a2f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77ca7fc640 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e656963f0 con 0x558e65a1e920 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.791+0000 7f77d4658740 10 client.0 _ll_register_callbacks cb 0x558e65979da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.792+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a287a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f77c00a3790 tx=0x7f77c009aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7dd0e588458b65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.792+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77bc004960 con 0x558e65a1e920 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.792+0000 7f77aaffd640 10 client.26938 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.792+0000 7f77aaffd640 1 -- 172.21.15.64:0/2400576504 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c009b050 con 0x558e65a1e920 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.792+0000 7f77aaffd640 1 -- 172.21.15.64:0/2400576504 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f77c009f080 con 0x558e65a1e920 2024-05-09T09:15:52.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.792+0000 7f77aaffd640 1 -- 172.21.15.64:0/2400576504 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f77c009f400 con 0x558e65a1e920 2024-05-09T09:15:52.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.793+0000 7f77d4658740 10 client.26938 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:52.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.793+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=42}) -- 0x558e656560f0 con 0x558e65a1e920 2024-05-09T09:15:52.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.793+0000 7f77aaffd640 1 -- 172.21.15.64:0/2400576504 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f77c00976c0 con 0x558e65a1e920 2024-05-09T09:15:52.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77d4658740 10 client.26938 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77d4658740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77d4658740 20 client.26938 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77d4658740 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a33e70 0x558e65a36310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77d4658740 4 client.26938 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e65990430 con 0x558e65a33e70 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.794+0000 7f77caffd640 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a33e70 0x558e65a36310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.795+0000 7f77caffd640 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a33e70 0x558e65a36310 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6c92f332977ed5 server_cookie=8df561e49a0896b5 in_seq=0 out_seq=0 2024-05-09T09:15:52.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.795+0000 7f77aaffd640 10 client.26938 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.796+0000 7f77aaffd640 1 -- 172.21.15.64:0/2400576504 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e65990430 con 0x558e65a33e70 2024-05-09T09:15:52.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.796+0000 7f77aaffd640 10 client.26938 handle_command_reply: tid=0 2024-05-09T09:15:52.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.796+0000 7f77d4658740 1 client.26938 shutdown 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 msgr2=0x558e65a287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a287a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f77c00a3790 tx=0x7f77c009aa70 comp rx=0 tx=0).stop 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 shutdown_connections 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a33e70 0x558e65a36310 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a2f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77aaffd640 0 client.26938 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a287a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 --2- 172.21.15.64:0/2400576504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a28260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.797+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 >> 172.21.15.64:0/2400576504 conn(0x558e65a195a0 msgr2=0x558e65a187e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.799+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 shutdown_connections 2024-05-09T09:15:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.799+0000 7f77d4658740 1 -- 172.21.15.64:0/2400576504 wait complete. 2024-05-09T09:15:52.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.799+0000 7f77d4658740 20 client.26938 trim_cache size 0 max 16384 2024-05-09T09:15:52.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 Processor -- start 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 -- start start 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a18660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8012480 con 0x558e65a23a90 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.840+0000 7f77d4658740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800fcf0 con 0x558e65a1df30 2024-05-09T09:15:52.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.841+0000 7f77d4658740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800db40 con 0x558e65a1e920 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.841+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.841+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a274d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50510/0 (socket says 172.21.15.64:50510) 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.841+0000 7f77cb7fe640 1 -- 172.21.15.64:0/2679062789 learned_addr learned my addr 172.21.15.64:0/2679062789 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.841+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.841+0000 7f77caffd640 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a18660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.842+0000 7f77cb7fe640 1 -- 172.21.15.64:0/2679062789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 msgr2=0x558e65a26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.842+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.842+0000 7f77cb7fe640 1 -- 172.21.15.64:0/2679062789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 msgr2=0x558e65a18660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.842+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a18660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.842+0000 7f77cb7fe640 1 -- 172.21.15.64:0/2679062789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e65a1d4c0 con 0x558e65a23a90 2024-05-09T09:15:52.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.842+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a274d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f77c4016500 tx=0x7f77c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235365f1e346b3d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.843+0000 7f77b17fa640 1 -- 172.21.15.64:0/2679062789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40145e0 con 0x558e65a23a90 2024-05-09T09:15:52.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.843+0000 7f77b17fa640 1 -- 172.21.15.64:0/2679062789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77c4027070 con 0x558e65a23a90 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77b17fa640 1 -- 172.21.15.64:0/2679062789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4022bd0 con 0x558e65a23a90 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 -- 172.21.15.64:0/2679062789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 msgr2=0x558e65a274d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a274d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f77c4016500 tx=0x7f77c4015000 comp rx=0 tx=0).stop 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 -- 172.21.15.64:0/2679062789 shutdown_connections 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a23a90 0x558e65a274d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1e920 0x558e65a26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 --2- 172.21.15.64:0/2679062789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a1df30 0x558e65a18660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.844+0000 7f77d4658740 1 -- 172.21.15.64:0/2679062789 >> 172.21.15.64:0/2679062789 conn(0x558e65a195a0 msgr2=0x558e65a17f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.845+0000 7f77d4658740 1 -- 172.21.15.64:0/2679062789 shutdown_connections 2024-05-09T09:15:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.845+0000 7f77d4658740 1 -- 172.21.15.64:0/2679062789 wait complete. 2024-05-09T09:15:52.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.845+0000 7f77d4658740 1 -- start start 2024-05-09T09:15:52.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.846+0000 7f77d4658740 1 Processor -- start 2024-05-09T09:15:52.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.846+0000 7f77d4658740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1df30 0x558e65a2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77d4658740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a1e920 0x558e65a2d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77d4658740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a23a90 0x558e65a2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77d4658740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800e1b0 con 0x558e65a1e920 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77d4658740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800f620 con 0x558e65a23a90 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77d4658740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800e880 con 0x558e65a1df30 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77d4658740 10 client.0 _ll_register_callbacks cb 0x7f77c009ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:15:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77caffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1df30 0x558e65a2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77caffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1df30 0x558e65a2cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37044/0 (socket says 172.21.15.64:37044) 2024-05-09T09:15:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a23a90 0x558e65a2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77caffd640 1 -- 172.21.15.64:0/753701273 learned_addr learned my addr 172.21.15.64:0/753701273 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:15:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77ca7fc640 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a1e920 0x558e65a2d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77cb7fe640 1 -- 172.21.15.64:0/753701273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1df30 msgr2=0x558e65a2cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1df30 0x558e65a2cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77cb7fe640 1 -- 172.21.15.64:0/753701273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a1e920 msgr2=0x558e65a2d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a1e920 0x558e65a2d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.847+0000 7f77cb7fe640 1 -- 172.21.15.64:0/753701273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e65a2e620 con 0x558e65a23a90 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.848+0000 7f77cb7fe640 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a23a90 0x558e65a2e030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77c4018010 tx=0x7f77c4022eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f634908c113cdfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.848+0000 7f77aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.848+0000 7f77aaffd640 1 -- 172.21.15.64:0/753701273 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4032020 con 0x558e65a23a90 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.848+0000 7f77aaffd640 1 -- 172.21.15.64:0/753701273 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f77c40147f0 con 0x558e65a23a90 2024-05-09T09:15:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.848+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c4027070 con 0x558e65a23a90 2024-05-09T09:15:52.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.849+0000 7f77aaffd640 1 -- 172.21.15.64:0/753701273 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f77c4014a10 con 0x558e65a23a90 2024-05-09T09:15:52.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.849+0000 7f77d4658740 10 client.26771 fetch_fsmap learned FSMap version 42 2024-05-09T09:15:52.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.849+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558e65a9ab40 con 0x558e65a23a90 2024-05-09T09:15:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.850+0000 7f77aaffd640 1 -- 172.21.15.64:0/753701273 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77c4027070 con 0x558e65a23a90 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.850+0000 7f77aaffd640 1 -- 172.21.15.64:0/753701273 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f77c402bc00 con 0x558e65a23a90 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77d4658740 10 client.26771 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77d4658740 10 client.26771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77d4658740 20 client.26771 populate_metadata read hostname 'smithi064' 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77d4658740 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a9b580 0x558e65a9d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77d4658740 4 client.26771 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e6598a780 con 0x558e65a9b580 2024-05-09T09:15:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.851+0000 7f77caffd640 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a9b580 0x558e65a9d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:15:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.852+0000 7f77caffd640 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a9b580 0x558e65a9d9a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71932c242422eb26 server_cookie=dae39b943e6b6771 in_seq=0 out_seq=0 2024-05-09T09:15:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.852+0000 7f77aaffd640 10 client.26771 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.852+0000 7f77aaffd640 1 -- 172.21.15.64:0/753701273 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e6598a780 con 0x558e65a9b580 2024-05-09T09:15:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.852+0000 7f77aaffd640 10 client.26771 handle_command_reply: tid=0 2024-05-09T09:15:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.852+0000 7f77d4658740 1 client.26771 shutdown 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a23a90 msgr2=0x558e65a2e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a23a90 0x558e65a2e030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77c4018010 tx=0x7f77c4022eb0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 shutdown_connections 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e65a9b580 0x558e65a9d9a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e65a23a90 0x558e65a2e030 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e65a1e920 0x558e65a2d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 --2- 172.21.15.64:0/753701273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e65a1df30 0x558e65a2cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 >> 172.21.15.64:0/753701273 conn(0x558e65a195a0 msgr2=0x558e65a1cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77aaffd640 0 client.26771 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 shutdown_connections 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 1 -- 172.21.15.64:0/753701273 wait complete. 2024-05-09T09:15:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.853+0000 7f77d4658740 20 client.26771 trim_cache size 0 max 16384 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: "c6fb6324-3da7-4a4f-b3a3-0acf219e3551": { 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "c6fb6324-3da7-4a4f-b3a3-0acf219e3551", 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:15:52.854 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:15:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f77b40a2ce0 msgr2=0x7f77b40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f77b40a2ce0 0x7f77b40a51a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x558e65a1f180 tx=0x7f77c400b040 comp rx=0 tx=0).stop 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc10b1a0 msgr2=0x7f77cc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc10b1a0 0x7f77cc19e7a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009fd0 tx=0x7f77c000b870 comp rx=0 tx=0).stop 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 shutdown_connections 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f77b40a2ce0 0x7f77b40a51a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77cc10db70 0x7f77cc1c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77cc10b1a0 0x7f77cc19e7a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 --2- 172.21.15.64:0/368116946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77cc108830 0x7f77cc19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.854+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 >> 172.21.15.64:0/368116946 conn(0x7f77cc06e6c0 msgr2=0x7f77cc109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.855+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 shutdown_connections 2024-05-09T09:15:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:15:52.855+0000 7f77d194c640 1 -- 172.21.15.64:0/368116946 wait complete. 2024-05-09T09:15:52.873 INFO:tasks.cephfs.filesystem:scrub status for tag:c6fb6324-3da7-4a4f-b3a3-0acf219e3551 - {'path': '/', 'tag': 'c6fb6324-3da7-4a4f-b3a3-0acf219e3551', 'options': 'recursive,force'} 2024-05-09T09:15:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:53 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:15:53.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:53 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:15:53.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:53 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:15:54.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:54 smithi139 ceph-mon[33599]: pgmap v573: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 13 MiB/s wr, 983 op/s 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:54 smithi064 ceph-mon[30149]: pgmap v573: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 13 MiB/s wr, 983 op/s 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:15:54.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:54 smithi179 ceph-mon[34086]: pgmap v573: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 13 MiB/s wr, 983 op/s 2024-05-09T09:15:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 335.0 s, 845.2 tps, lat 118.585 ms stddev 53.685 2024-05-09T09:15:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:56 smithi064 ceph-mon[30149]: pgmap v574: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 12 MiB/s wr, 1.00k op/s 2024-05-09T09:15:56.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:56 smithi179 ceph-mon[34086]: pgmap v574: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 12 MiB/s wr, 1.00k op/s 2024-05-09T09:15:56.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:56 smithi139 ceph-mon[33599]: pgmap v574: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 12 MiB/s wr, 1.00k op/s 2024-05-09T09:15:58.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:15:58 smithi064 ceph-mon[30149]: pgmap v575: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 9.6 MiB/s wr, 881 op/s 2024-05-09T09:15:58.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:15:58 smithi179 ceph-mon[34086]: pgmap v575: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 9.6 MiB/s wr, 881 op/s 2024-05-09T09:15:58.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:15:58 smithi139 ceph-mon[33599]: pgmap v575: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 9.6 MiB/s wr, 881 op/s 2024-05-09T09:16:00.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:00 smithi064 ceph-mon[30149]: pgmap v576: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-09T09:16:00.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:00 smithi179 ceph-mon[34086]: pgmap v576: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-09T09:16:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:00 smithi139 ceph-mon[33599]: pgmap v576: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-09T09:16:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 340.0 s, 924.8 tps, lat 108.550 ms stddev 35.517 2024-05-09T09:16:02.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:02 smithi064 ceph-mon[30149]: pgmap v577: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.3 MiB/s wr, 864 op/s 2024-05-09T09:16:02.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:02 smithi179 ceph-mon[34086]: pgmap v577: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.3 MiB/s wr, 864 op/s 2024-05-09T09:16:02.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:02 smithi139 ceph-mon[33599]: pgmap v577: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.3 MiB/s wr, 864 op/s 2024-05-09T09:16:04.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:04 smithi064 ceph-mon[30149]: pgmap v578: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 8.8 MiB/s wr, 957 op/s 2024-05-09T09:16:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:04 smithi179 ceph-mon[34086]: pgmap v578: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 8.8 MiB/s wr, 957 op/s 2024-05-09T09:16:04.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:04 smithi139 ceph-mon[33599]: pgmap v578: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 8.8 MiB/s wr, 957 op/s 2024-05-09T09:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.068+0000 7feaad542640 1 -- 172.21.15.64:0/3533200793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 msgr2=0x7feaa00aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.068+0000 7feaad542640 1 --2- 172.21.15.64:0/3533200793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 0x7feaa00aafc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fea98009860 tx=0x7fea9802a360 comp rx=0 tx=0).stop 2024-05-09T09:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 -- 172.21.15.64:0/3533200793 shutdown_connections 2024-05-09T09:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 --2- 172.21.15.64:0/3533200793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 0x7feaa00aafc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 --2- 172.21.15.64:0/3533200793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaa00a5d60 0x7feaa00a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 --2- 172.21.15.64:0/3533200793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a4c30 0x7feaa00a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 -- 172.21.15.64:0/3533200793 >> 172.21.15.64:0/3533200793 conn(0x7feaa00a00c0 msgr2=0x7feaa00a2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 -- 172.21.15.64:0/3533200793 shutdown_connections 2024-05-09T09:16:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.069+0000 7feaad542640 1 -- 172.21.15.64:0/3533200793 wait complete. 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 Processor -- start 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 -- start start 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaa00a4c30 0x7feaa01434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 0x7feaa0143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 0x7feaa014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feaa00b8ec0 con 0x7feaa00a4c30 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feaa00b8d40 con 0x7feaa00a6700 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaad542640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feaa00b9040 con 0x7feaa00a5d60 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaa67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 0x7feaa0143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaa67fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 0x7feaa0143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44406/0 (socket says 172.21.15.64:44406) 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.070+0000 7feaa67fc640 1 -- 172.21.15.64:0/87789071 learned_addr learned my addr 172.21.15.64:0/87789071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa77fe640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 0x7feaa014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa67fc640 1 -- 172.21.15.64:0/87789071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 msgr2=0x7feaa014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa67fc640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 0x7feaa014aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa67fc640 1 -- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaa00a4c30 msgr2=0x7feaa01434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa67fc640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaa00a4c30 0x7feaa01434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa67fc640 1 -- 172.21.15.64:0/87789071 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa014b0c0 con 0x7feaa00a5d60 2024-05-09T09:16:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.071+0000 7feaa67fc640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 0x7feaa0143a30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9c0 tx=0x7fea9c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.072+0000 7fea8ffff640 1 -- 172.21.15.64:0/87789071 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00cde0 con 0x7feaa00a5d60 2024-05-09T09:16:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.072+0000 7fea8ffff640 1 -- 172.21.15.64:0/87789071 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea9c0035f0 con 0x7feaa00a5d60 2024-05-09T09:16:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.072+0000 7fea8ffff640 1 -- 172.21.15.64:0/87789071 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c010690 con 0x7feaa00a5d60 2024-05-09T09:16:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.073+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa014b3b0 con 0x7feaa00a5d60 2024-05-09T09:16:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.073+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa014b880 con 0x7feaa00a5d60 2024-05-09T09:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 Processor -- start 2024-05-09T09:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7fea8ffff640 1 -- 172.21.15.64:0/87789071 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea9c0037a0 con 0x7feaa00a5d60 2024-05-09T09:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 -- start start 2024-05-09T09:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 0x559a141da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141daf60 0x559a141dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a14139340 con 0x559a141daf60 2024-05-09T09:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559a140e1290 con 0x559a141d5400 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.077+0000 7feab024e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559a14135300 con 0x559a141d5df0 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 0x559a141da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 0x559a141da8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44412/0 (socket says 172.21.15.64:44412) 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1495198161 learned_addr learned my addr 172.21.15.64:0/1495198161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1495198161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 msgr2=0x559a141d5820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141d5820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1495198161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141daf60 msgr2=0x559a141dd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141daf60 0x559a141dd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.078+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1495198161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a141058e0 con 0x559a141d5df0 2024-05-09T09:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.079+0000 7fea8ffff640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea900a2d60 0x7fea900a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.079+0000 7fea8ffff640 1 -- 172.21.15.64:0/87789071 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea9c0cf6a0 con 0x7feaa00a5d60 2024-05-09T09:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.079+0000 7feaa77fe640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea900a2d60 0x7fea900a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.079+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 0x559a141da8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaa804f8c0 tx=0x7feaa806c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b072f2cfd55435 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.080+0000 7fea8d7fa640 1 -- 172.21.15.64:0/1495198161 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa806a750 con 0x559a141d5df0 2024-05-09T09:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.080+0000 7fea8d7fa640 1 -- 172.21.15.64:0/1495198161 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaa806ada0 con 0x559a141d5df0 2024-05-09T09:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 -- 172.21.15.64:0/1495198161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 msgr2=0x559a141da8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 0x559a141da8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaa804f8c0 tx=0x7feaa806c7e0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feaa77fe640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea900a2d60 0x7fea900a5220 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fea9802a880 tx=0x7fea98009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 -- 172.21.15.64:0/1495198161 shutdown_connections 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141daf60 0x559a141dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5df0 0x559a141da8f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 --2- 172.21.15.64:0/1495198161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141d5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 -- 172.21.15.64:0/1495198161 >> 172.21.15.64:0/1495198161 conn(0x559a141d0a70 msgr2=0x559a141d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 -- 172.21.15.64:0/1495198161 shutdown_connections 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.081+0000 7feab024e740 1 -- 172.21.15.64:0/1495198161 wait complete. 2024-05-09T09:16:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.082+0000 7feab024e740 1 -- start start 2024-05-09T09:16:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.082+0000 7feab024e740 1 Processor -- start 2024-05-09T09:16:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.082+0000 7feab024e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 0x559a141f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feab024e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5df0 0x559a141f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feab024e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feab024e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a141eef20 con 0x559a141d5400 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feab024e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559a140f4790 con 0x559a141d5df0 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feab024e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559a141ef0a0 con 0x559a141daf60 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feab024e740 10 client.0 _ll_register_callbacks cb 0x559a1412fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 0x559a141f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 0x559a141f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52188/0 (socket says 172.21.15.64:52188) 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feaa6ffd640 1 -- 172.21.15.64:0/4125337993 learned_addr learned my addr 172.21.15.64:0/4125337993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.083+0000 7feaa67fc640 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5df0 0x559a141f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7feaa6ffd640 1 -- 172.21.15.64:0/4125337993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 msgr2=0x559a141f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7feaa6ffd640 1 -- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5df0 msgr2=0x559a141f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5df0 0x559a141f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7feaa6ffd640 1 -- 172.21.15.64:0/4125337993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a13e4d210 con 0x559a141d5400 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 0x559a141f0610 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7feaa8065080 tx=0x7feaa806a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9944c55afee10610 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.084+0000 7fea8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.085+0000 7fea8aff5640 1 -- 172.21.15.64:0/4125337993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa808a060 con 0x559a141d5400 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.085+0000 7fea8aff5640 1 -- 172.21.15.64:0/4125337993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7feaa806aad0 con 0x559a141d5400 2024-05-09T09:16:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.085+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa806ada0 con 0x559a141d5400 2024-05-09T09:16:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.085+0000 7fea8aff5640 1 -- 172.21.15.64:0/4125337993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feaa807e040 con 0x559a141d5400 2024-05-09T09:16:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.086+0000 7feab024e740 10 client.17295 fetch_fsmap learned FSMap version 42 2024-05-09T09:16:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.086+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559a13f19620 con 0x559a141d5400 2024-05-09T09:16:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.086+0000 7fea8aff5640 1 -- 172.21.15.64:0/4125337993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaa806ada0 con 0x559a141d5400 2024-05-09T09:16:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.086+0000 7fea8aff5640 1 -- 172.21.15.64:0/4125337993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7feaa8077400 con 0x559a141d5400 2024-05-09T09:16:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feab024e740 10 client.17295 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:16:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feab024e740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feab024e740 20 client.17295 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feab024e740 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a141fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feab024e740 4 client.17295 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a14146a70 con 0x559a141fbe60 2024-05-09T09:16:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.087+0000 7feaa67fc640 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a141fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.091+0000 7feaa67fc640 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a141fe250 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bbd2432ee344680 server_cookie=e79954c7e8b89983 in_seq=0 out_seq=0 2024-05-09T09:16:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.091+0000 7fea8aff5640 10 client.17295 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.092+0000 7fea8aff5640 1 -- 172.21.15.64:0/4125337993 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a14146a70 con 0x559a141fbe60 2024-05-09T09:16:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.092+0000 7fea8aff5640 10 client.17295 handle_command_reply: tid=0 2024-05-09T09:16:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.092+0000 7feab024e740 1 client.17295 shutdown 2024-05-09T09:16:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.092+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 msgr2=0x559a141f0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 0x559a141f0610 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7feaa8065080 tx=0x7feaa806a760 comp rx=0 tx=0).stop 2024-05-09T09:16:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 shutdown_connections 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a141fe250 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7fea8aff5640 0 client.17295 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5df0 0x559a141f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 --2- 172.21.15.64:0/4125337993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5400 0x559a141f0610 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 >> 172.21.15.64:0/4125337993 conn(0x559a141d0a70 msgr2=0x559a141cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 shutdown_connections 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 1 -- 172.21.15.64:0/4125337993 wait complete. 2024-05-09T09:16:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.093+0000 7feab024e740 20 client.17295 trim_cache size 0 max 16384 2024-05-09T09:16:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.123+0000 7feab024e740 1 Processor -- start 2024-05-09T09:16:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.124+0000 7feab024e740 1 -- start start 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.124+0000 7feab024e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 0x559a141cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.124+0000 7feab024e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.124+0000 7feab024e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141daf60 0x559a141cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.124+0000 7feab024e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0150050 con 0x559a141d5df0 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feab024e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feaa014d8c0 con 0x559a141daf60 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feab024e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feaa00b4bd0 con 0x559a141d5400 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 0x559a141cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 0x559a141cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44430/0 (socket says 172.21.15.64:44430) 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feaa6ffd640 1 -- 172.21.15.64:0/160931301 learned_addr learned my addr 172.21.15.64:0/160931301 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feaa67fc640 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.125+0000 7feaa77fe640 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141daf60 0x559a141cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.126+0000 7feaa6ffd640 1 -- 172.21.15.64:0/160931301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141daf60 msgr2=0x559a141cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.126+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141daf60 0x559a141cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.126+0000 7feaa6ffd640 1 -- 172.21.15.64:0/160931301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 msgr2=0x559a141cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.126+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.126+0000 7feaa6ffd640 1 -- 172.21.15.64:0/160931301 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a141fe5c0 con 0x559a141d5400 2024-05-09T09:16:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.126+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 0x559a141cf320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feaa80506b0 tx=0x7feaa806c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e592eb2b8ad2fab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.127+0000 7fea8cff9640 1 -- 172.21.15.64:0/160931301 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8093020 con 0x559a141d5400 2024-05-09T09:16:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.127+0000 7fea8cff9640 1 -- 172.21.15.64:0/160931301 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaa8077e60 con 0x559a141d5400 2024-05-09T09:16:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.127+0000 7fea8cff9640 1 -- 172.21.15.64:0/160931301 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa808a040 con 0x559a141d5400 2024-05-09T09:16:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 -- 172.21.15.64:0/160931301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 msgr2=0x559a141cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 0x559a141cf320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feaa80506b0 tx=0x7feaa806c7e0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 -- 172.21.15.64:0/160931301 shutdown_connections 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141daf60 0x559a141cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 --2- 172.21.15.64:0/160931301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141d5400 0x559a141cf320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 -- 172.21.15.64:0/160931301 >> 172.21.15.64:0/160931301 conn(0x559a141d0a70 msgr2=0x559a141d6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.128+0000 7feab024e740 1 -- 172.21.15.64:0/160931301 shutdown_connections 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.129+0000 7feab024e740 1 -- 172.21.15.64:0/160931301 wait complete. 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.129+0000 7feab024e740 1 -- start start 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.129+0000 7feab024e740 1 Processor -- start 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feaa014bd80 con 0x559a141d5df0 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feaa014d1f0 con 0x559a141d5400 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feaa014c450 con 0x559a141daf60 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feab024e740 10 client.0 _ll_register_callbacks cb 0x7feaa804fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57296/0 (socket says 172.21.15.64:57296) 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1595414938 learned_addr learned my addr 172.21.15.64:0/1595414938 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa77fe640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa67fc640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1595414938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 msgr2=0x559a141f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1595414938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 msgr2=0x559a141f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.130+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.131+0000 7feaa6ffd640 1 -- 172.21.15.64:0/1595414938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a141f5cf0 con 0x559a141d5400 2024-05-09T09:16:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.131+0000 7feaa6ffd640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141f41b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feaa8065080 tx=0x7feaa8077bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8879e6fed4735899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.131+0000 7fea8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.131+0000 7fea8aff5640 1 -- 172.21.15.64:0/1595414938 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8093070 con 0x559a141d5400 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.132+0000 7fea8aff5640 1 -- 172.21.15.64:0/1595414938 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7feaa806a760 con 0x559a141d5400 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.132+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a14140dc0 con 0x559a141d5400 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.132+0000 7fea8aff5640 1 -- 172.21.15.64:0/1595414938 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feaa808a070 con 0x559a141d5400 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.132+0000 7feab024e740 10 client.26783 fetch_fsmap learned FSMap version 42 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.132+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559a141f7000 con 0x559a141d5400 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7fea8aff5640 1 -- 172.21.15.64:0/1595414938 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559a14140dc0 con 0x559a141d5400 2024-05-09T09:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7fea8aff5640 1 -- 172.21.15.64:0/1595414938 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7feaa8084b20 con 0x559a141d5400 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7feab024e740 10 client.26783 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7feab024e740 10 client.26783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7feab024e740 20 client.26783 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7feab024e740 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a142645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7feab024e740 4 client.26783 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.133+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a140be8e0 con 0x559a141fbe60 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.134+0000 7feaa67fc640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a142645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.135+0000 7feaa67fc640 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a142645d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771fdbb7bca93fd server_cookie=b3d8c9bd6be90ef2 in_seq=0 out_seq=0 2024-05-09T09:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.135+0000 7fea8aff5640 10 client.26783 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7fea8aff5640 1 -- 172.21.15.64:0/1595414938 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a140be8e0 con 0x559a141fbe60 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7fea8aff5640 10 client.26783 handle_command_reply: tid=0 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 client.26783 shutdown 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 msgr2=0x559a141f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141f41b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feaa8065080 tx=0x7feaa8077bf0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 shutdown_connections 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559a141fbe60 0x559a142645d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a141daf60 0x559a141f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a141d5df0 0x559a141f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 --2- 172.21.15.64:0/1595414938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a141d5400 0x559a141f41b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 >> 172.21.15.64:0/1595414938 conn(0x559a141d0a70 msgr2=0x559a141d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.155+0000 7fea8aff5640 0 client.26783 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.156+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 shutdown_connections 2024-05-09T09:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.156+0000 7feab024e740 1 -- 172.21.15.64:0/1595414938 wait complete. 2024-05-09T09:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.156+0000 7feab024e740 20 client.26783 trim_cache size 0 max 16384 2024-05-09T09:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea900a2d60 msgr2=0x7fea900a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea900a2d60 0x7fea900a5220 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fea9802a880 tx=0x7fea98009430 comp rx=0 tx=0).stop 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 msgr2=0x7feaa0143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 0x7feaa0143a30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9c0 tx=0x7fea9c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 shutdown_connections 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fea900a2d60 0x7fea900a5220 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa00a6700 0x7feaa014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaa00a5d60 0x7feaa0143a30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 --2- 172.21.15.64:0/87789071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaa00a4c30 0x7feaa01434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 >> 172.21.15.64:0/87789071 conn(0x7feaa00a00c0 msgr2=0x7feaa00a6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.157+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 shutdown_connections 2024-05-09T09:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:05.158+0000 7feaad542640 1 -- 172.21.15.64:0/87789071 wait complete. 2024-05-09T09:16:05.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:16:05.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:16:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:16:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:16:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:16:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:16:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 345.0 s, 951.6 tps, lat 104.697 ms stddev 59.048 2024-05-09T09:16:06.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:06 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:16:06.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:06 smithi064 ceph-mon[30149]: pgmap v579: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.0 MiB/s wr, 935 op/s 2024-05-09T09:16:06.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:06 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:16:06.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:06 smithi179 ceph-mon[34086]: pgmap v579: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.0 MiB/s wr, 935 op/s 2024-05-09T09:16:06.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:06 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:16:06.869 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:06 smithi139 ceph-mon[33599]: pgmap v579: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.0 MiB/s wr, 935 op/s 2024-05-09T09:16:08.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:08 smithi064 ceph-mon[30149]: pgmap v580: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.8 MiB/s wr, 797 op/s 2024-05-09T09:16:08.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:08 smithi179 ceph-mon[34086]: pgmap v580: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.8 MiB/s wr, 797 op/s 2024-05-09T09:16:08.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:08 smithi139 ceph-mon[33599]: pgmap v580: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.8 MiB/s wr, 797 op/s 2024-05-09T09:16:10.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:10 smithi064 ceph-mon[30149]: pgmap v581: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:16:10.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:10 smithi179 ceph-mon[34086]: pgmap v581: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:16:10.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:10 smithi139 ceph-mon[33599]: pgmap v581: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:16:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 350.0 s, 972.1 tps, lat 102.198 ms stddev 54.164 2024-05-09T09:16:12.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:12 smithi064 ceph-mon[30149]: pgmap v582: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 831 op/s 2024-05-09T09:16:12.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:12 smithi179 ceph-mon[34086]: pgmap v582: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 831 op/s 2024-05-09T09:16:12.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:12 smithi139 ceph-mon[33599]: pgmap v582: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 831 op/s 2024-05-09T09:16:14.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:14 smithi064 ceph-mon[30149]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 959 op/s 2024-05-09T09:16:14.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:14 smithi179 ceph-mon[34086]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 959 op/s 2024-05-09T09:16:14.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:14 smithi139 ceph-mon[33599]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 959 op/s 2024-05-09T09:16:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 355.0 s, 1006.3 tps, lat 100.377 ms stddev 42.753 2024-05-09T09:16:16.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:16 smithi179 ceph-mon[34086]: pgmap v584: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 919 op/s 2024-05-09T09:16:16.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:16 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:16:16.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:16 smithi139 ceph-mon[33599]: pgmap v584: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 919 op/s 2024-05-09T09:16:16.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:16 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:16:16.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:16 smithi064 ceph-mon[30149]: pgmap v584: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 919 op/s 2024-05-09T09:16:16.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:16 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:16:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.313+0000 7fa60f39e640 1 -- 172.21.15.64:0/2964047142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810d990 msgr2=0x7fa60810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.313+0000 7fa60f39e640 1 --2- 172.21.15.64:0/2964047142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810d990 0x7fa60810ddb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.314+0000 7fa60f39e640 1 -- 172.21.15.64:0/2964047142 shutdown_connections 2024-05-09T09:16:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.314+0000 7fa60f39e640 1 --2- 172.21.15.64:0/2964047142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa6081146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.314+0000 7fa60f39e640 1 --2- 172.21.15.64:0/2964047142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa60810bf10 0x7fa60810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.314+0000 7fa60f39e640 1 --2- 172.21.15.64:0/2964047142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810d990 0x7fa60810ddb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.314+0000 7fa60f39e640 1 -- 172.21.15.64:0/2964047142 >> 172.21.15.64:0/2964047142 conn(0x7fa60806c470 msgr2=0x7fa60806c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.315+0000 7fa60f39e640 1 -- 172.21.15.64:0/2964047142 shutdown_connections 2024-05-09T09:16:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.316+0000 7fa60f39e640 1 -- 172.21.15.64:0/2964047142 wait complete. 2024-05-09T09:16:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.317+0000 7fa60f39e640 1 Processor -- start 2024-05-09T09:16:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.317+0000 7fa60f39e640 1 -- start start 2024-05-09T09:16:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.318+0000 7fa60f39e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 0x7fa60819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.318+0000 7fa60f39e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa60819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.318+0000 7fa60d13a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 0x7fa60819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.318+0000 7fa60d13a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 0x7fa60819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41596/0 (socket says 172.21.15.64:41596) 2024-05-09T09:16:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.318+0000 7fa60c939640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa60819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.318+0000 7fa60c939640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa60819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39698/0 (socket says 172.21.15.64:39698) 2024-05-09T09:16:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.319+0000 7fa60d13a640 1 -- 172.21.15.64:0/3098690382 learned_addr learned my addr 172.21.15.64:0/3098690382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.319+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa60810d990 0x7fa6081a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.319+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa608117f40 con 0x7fa60810d990 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.319+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa608117dc0 con 0x7fa60810bf10 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.319+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081180c0 con 0x7fa60810c8d0 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.319+0000 7fa60d93b640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa60810d990 0x7fa6081a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60d13a640 1 -- 172.21.15.64:0/3098690382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 msgr2=0x7fa60819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60d13a640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa60819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60d13a640 1 -- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa60810d990 msgr2=0x7fa6081a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60d13a640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa60810d990 0x7fa6081a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60d13a640 1 -- 172.21.15.64:0/3098690382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081b6e00 con 0x7fa60810bf10 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60c939640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa60819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60d13a640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 0x7fa60819e100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80314e0 tx=0x7fa5f8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa5fe7fc640 1 -- 172.21.15.64:0/3098690382 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f803f070 con 0x7fa60810bf10 2024-05-09T09:16:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa5fe7fc640 1 -- 172.21.15.64:0/3098690382 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5f8031c40 con 0x7fa60810bf10 2024-05-09T09:16:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa5fe7fc640 1 -- 172.21.15.64:0/3098690382 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8035030 con 0x7fa60810bf10 2024-05-09T09:16:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.320+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6081b6fd0 con 0x7fa60810bf10 2024-05-09T09:16:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.321+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081b7380 con 0x7fa60810bf10 2024-05-09T09:16:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa5fe7fc640 1 -- 172.21.15.64:0/3098690382 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5f8042070 con 0x7fa60810bf10 2024-05-09T09:16:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 Processor -- start 2024-05-09T09:16:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 -- start start 2024-05-09T09:16:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25493670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e25498740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e2549b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e253f7340 con 0x562e25493c40 2024-05-09T09:16:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562e2539f290 con 0x562e25493250 2024-05-09T09:16:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.325+0000 7fa612184740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562e253f3300 con 0x562e25498db0 2024-05-09T09:16:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.326+0000 7fa60d93b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25493670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.327+0000 7fa60d93b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25493670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41606/0 (socket says 172.21.15.64:41606) 2024-05-09T09:16:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.327+0000 7fa60d93b640 1 -- 172.21.15.64:0/1442899455 learned_addr learned my addr 172.21.15.64:0/1442899455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.328+0000 7fa60c939640 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e2549b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.328+0000 7fa60c939640 1 -- 172.21.15.64:0/1442899455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 msgr2=0x562e25493670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.329+0000 7fa60d13a640 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e25498740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.329+0000 7fa60c939640 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25493670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.329+0000 7fa60c939640 1 -- 172.21.15.64:0/1442899455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 msgr2=0x562e25498740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.329+0000 7fa60c939640 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e25498740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.329+0000 7fa60c939640 1 -- 172.21.15.64:0/1442899455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e253c38e0 con 0x562e25498db0 2024-05-09T09:16:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.330+0000 7fa60c939640 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e2549b240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa604009fa0 tx=0x7fa6040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2b87337ec6dd2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.330+0000 7fa5fe7fc640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5f40a2c30 0x7fa5f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.330+0000 7fa5fe7fc640 1 -- 172.21.15.64:0/3098690382 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5f80efb20 con 0x7fa60810bf10 2024-05-09T09:16:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.330+0000 7fa5f37fe640 1 -- 172.21.15.64:0/1442899455 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604004030 con 0x562e25498db0 2024-05-09T09:16:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.330+0000 7fa5f37fe640 1 -- 172.21.15.64:0/1442899455 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6040054d0 con 0x562e25498db0 2024-05-09T09:16:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.330+0000 7fa5f37fe640 1 -- 172.21.15.64:0/1442899455 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604004ce0 con 0x562e25498db0 2024-05-09T09:16:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.331+0000 7fa60d93b640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5f40a2c30 0x7fa5f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.331+0000 7fa612184740 1 -- 172.21.15.64:0/1442899455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 msgr2=0x562e2549b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.331+0000 7fa612184740 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e2549b240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa604009fa0 tx=0x7fa6040118d0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.332+0000 7fa60d93b640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5f40a2c30 0x7fa5f40a50f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x562e254944a0 tx=0x7fa600009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.334+0000 7fa612184740 1 -- 172.21.15.64:0/1442899455 shutdown_connections 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.334+0000 7fa612184740 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e2549b240 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.334+0000 7fa612184740 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e25498740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.334+0000 7fa612184740 1 --2- 172.21.15.64:0/1442899455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25493670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.334+0000 7fa612184740 1 -- 172.21.15.64:0/1442899455 >> 172.21.15.64:0/1442899455 conn(0x562e2548e8c0 msgr2=0x562e2548ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.334+0000 7fa612184740 1 -- 172.21.15.64:0/1442899455 shutdown_connections 2024-05-09T09:16:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.335+0000 7fa612184740 1 -- 172.21.15.64:0/1442899455 wait complete. 2024-05-09T09:16:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.335+0000 7fa612184740 1 -- start start 2024-05-09T09:16:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.335+0000 7fa612184740 1 Processor -- start 2024-05-09T09:16:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 0x562e2549d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493c40 0x562e2549da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25498db0 0x562e254a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e2549bd60 con 0x562e25498db0 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562e253b2790 con 0x562e25493c40 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562e2549bee0 con 0x562e25493250 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa612184740 10 client.0 _ll_register_callbacks cb 0x562e253eda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa60d13a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 0x562e2549d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa60d13a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 0x562e2549d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39714/0 (socket says 172.21.15.64:39714) 2024-05-09T09:16:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa60d13a640 1 -- 172.21.15.64:0/2986928460 learned_addr learned my addr 172.21.15.64:0/2986928460 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.336+0000 7fa60c939640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493c40 0x562e2549da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d93b640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25498db0 0x562e254a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d13a640 1 -- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493c40 msgr2=0x562e2549da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493c40 0x562e2549da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d13a640 1 -- 172.21.15.64:0/2986928460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25498db0 msgr2=0x562e254a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25498db0 0x562e254a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d13a640 1 -- 172.21.15.64:0/2986928460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2510b210 con 0x562e25493250 2024-05-09T09:16:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 0x562e2549d4d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000c00 tx=0x7fa5f80bbb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35957fc37d98b346 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.337+0000 7fa5f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.338+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2986928460 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8049050 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.338+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2986928460 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5f803b4f0 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.338+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6040054d0 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.338+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2986928460 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa5f80bc6f0 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.339+0000 7fa612184740 10 client.26968 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.339+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562e251d7620 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.339+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2986928460 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6040054d0 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2986928460 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f80bc250 con 0x562e25493250 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa612184740 10 client.26968 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa612184740 10 client.26968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa612184740 20 client.26968 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa612184740 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e254a8d30 0x562e254ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa612184740 4 client.26968 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.340+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e25404a70 con 0x562e254a8d30 2024-05-09T09:16:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.344+0000 7fa60c939640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e254a8d30 0x562e254ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.348+0000 7fa60c939640 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e254a8d30 0x562e254ab140 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351276ee4a33262e server_cookie=2b8831234683cd11 in_seq=0 out_seq=0 2024-05-09T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.348+0000 7fa5f0ff9640 10 client.26968 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.349+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2986928460 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e25404a70 con 0x562e254a8d30 2024-05-09T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.349+0000 7fa5f0ff9640 10 client.26968 handle_command_reply: tid=0 2024-05-09T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.349+0000 7fa612184740 1 client.26968 shutdown 2024-05-09T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.350+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 msgr2=0x562e2549d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.350+0000 7fa612184740 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 0x562e2549d4d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000c00 tx=0x7fa5f80bbb30 comp rx=0 tx=0).stop 2024-05-09T09:16:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.351+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 shutdown_connections 2024-05-09T09:16:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.351+0000 7fa612184740 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e254a8d30 0x562e254ab140 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.351+0000 7fa612184740 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25498db0 0x562e254a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.351+0000 7fa612184740 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493c40 0x562e2549da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.351+0000 7fa612184740 1 --2- 172.21.15.64:0/2986928460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493250 0x562e2549d4d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.351+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 >> 172.21.15.64:0/2986928460 conn(0x562e2548e8c0 msgr2=0x562e2548da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.352+0000 7fa5f0ff9640 0 client.26968 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.354+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 shutdown_connections 2024-05-09T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.354+0000 7fa612184740 1 -- 172.21.15.64:0/2986928460 wait complete. 2024-05-09T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.355+0000 7fa612184740 20 client.26968 trim_cache size 0 max 16384 2024-05-09T09:16:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.405+0000 7fa612184740 1 Processor -- start 2024-05-09T09:16:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.405+0000 7fa612184740 1 -- start start 2024-05-09T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.405+0000 7fa612184740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25489720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.405+0000 7fa612184740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e254a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.405+0000 7fa612184740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e254b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.405+0000 7fa612184740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081c4330 con 0x562e25493c40 2024-05-09T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60c939640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e254a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60c939640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e254a4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36026/0 (socket says 172.21.15.64:36026) 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60c939640 1 -- 172.21.15.64:0/2415915778 learned_addr learned my addr 172.21.15.64:0/2415915778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081c1ba0 con 0x562e25493250 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081bf9f0 con 0x562e25498db0 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25489720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d93b640 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e254b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 -- 172.21.15.64:0/2415915778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 msgr2=0x562e254b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e254b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 -- 172.21.15.64:0/2415915778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 msgr2=0x562e254a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e254a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 -- 172.21.15.64:0/2415915778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e2549c5b0 con 0x562e25493250 2024-05-09T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60c939640 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e254a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.406+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25489720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80ba2a0 tx=0x7fa5f80bbc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73efb90c2efe60a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.407+0000 7fa5f2ffd640 1 -- 172.21.15.64:0/2415915778 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8049070 con 0x562e25493250 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.407+0000 7fa5f2ffd640 1 -- 172.21.15.64:0/2415915778 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5f80dc0b0 con 0x562e25493250 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.407+0000 7fa5f2ffd640 1 -- 172.21.15.64:0/2415915778 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f809ae60 con 0x562e25493250 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.408+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 msgr2=0x562e25489720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.408+0000 7fa612184740 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25489720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80ba2a0 tx=0x7fa5f80bbc60 comp rx=0 tx=0).stop 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.408+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 shutdown_connections 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25498db0 0x562e254b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493c40 0x562e254a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 --2- 172.21.15.64:0/2415915778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25493250 0x562e25489720 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 >> 172.21.15.64:0/2415915778 conn(0x562e2548e8c0 msgr2=0x562e2548eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 shutdown_connections 2024-05-09T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 -- 172.21.15.64:0/2415915778 wait complete. 2024-05-09T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.409+0000 7fa612184740 1 -- start start 2024-05-09T09:16:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.410+0000 7fa612184740 1 Processor -- start 2024-05-09T09:16:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493250 0x562e254a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 0x562e254a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25498db0 0x562e2550d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081c0060 con 0x562e25493250 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081c14d0 con 0x562e25498db0 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081c0730 con 0x562e25493c40 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa612184740 10 client.0 _ll_register_callbacks cb 0x7fa5f8005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60d93b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25498db0 0x562e2550d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 0x562e254a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60d93b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25498db0 0x562e2550d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41628/0 (socket says 172.21.15.64:41628) 2024-05-09T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60d93b640 1 -- 172.21.15.64:0/2721355280 learned_addr learned my addr 172.21.15.64:0/2721355280 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 0x562e254a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39734/0 (socket says 172.21.15.64:39734) 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493250 0x562e254a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 -- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25498db0 msgr2=0x562e2550d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25498db0 0x562e2550d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 -- 172.21.15.64:0/2721355280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493250 msgr2=0x562e254a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493250 0x562e254a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.411+0000 7fa60c939640 1 -- 172.21.15.64:0/2721355280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2550d840 con 0x562e25493c40 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.412+0000 7fa60c939640 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 0x562e254a1ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa604009380 tx=0x7fa6040045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ed5bdd7aa0924 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.412+0000 7fa5f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.412+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2721355280 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60401d040 con 0x562e25493c40 2024-05-09T09:16:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.412+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2721355280 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa60402b070 con 0x562e25493c40 2024-05-09T09:16:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.412+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f80dc0b0 con 0x562e25493c40 2024-05-09T09:16:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.421+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2721355280 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa60403a050 con 0x562e25493c40 2024-05-09T09:16:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.422+0000 7fa612184740 10 client.26974 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.422+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562e2550ebc0 con 0x562e25493c40 2024-05-09T09:16:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.422+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2721355280 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa60402f020 con 0x562e25493c40 2024-05-09T09:16:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.422+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2721355280 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa60400f5a0 con 0x562e25493c40 2024-05-09T09:16:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa612184740 10 client.26974 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa612184740 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa612184740 20 client.26974 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa612184740 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e2550f380 0x562e255117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa612184740 4 client.26974 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:16:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa5f80dc0b0 con 0x562e2550f380 2024-05-09T09:16:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.423+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e2550f380 0x562e255117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.424+0000 7fa60d13a640 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e2550f380 0x562e255117a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c98923396eba07d server_cookie=e91673d6ddc84260 in_seq=0 out_seq=0 2024-05-09T09:16:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.424+0000 7fa5f0ff9640 10 client.26974 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.447+0000 7fa5f0ff9640 1 -- 172.21.15.64:0/2721355280 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa5f80dc0b0 con 0x562e2550f380 2024-05-09T09:16:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.447+0000 7fa5f0ff9640 10 client.26974 handle_command_reply: tid=0 2024-05-09T09:16:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.447+0000 7fa612184740 1 client.26974 shutdown 2024-05-09T09:16:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 msgr2=0x562e254a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 0x562e254a1ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa604009380 tx=0x7fa6040045b0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 shutdown_connections 2024-05-09T09:16:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562e2550f380 0x562e255117a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa5f0ff9640 0 client.26974 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e25498db0 0x562e2550d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e25493c40 0x562e254a1ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 --2- 172.21.15.64:0/2721355280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e25493250 0x562e254a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.448+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 >> 172.21.15.64:0/2721355280 conn(0x562e2548e8c0 msgr2=0x562e25491870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.449+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 shutdown_connections 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.449+0000 7fa612184740 1 -- 172.21.15.64:0/2721355280 wait complete. 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.449+0000 7fa612184740 20 client.26974 trim_cache size 0 max 16384 2024-05-09T09:16:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:16:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:16:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:16:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:16:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.451+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5f40a2c30 msgr2=0x7fa5f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.451+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5f40a2c30 0x7fa5f40a50f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x562e254944a0 tx=0x7fa600009500 comp rx=0 tx=0).stop 2024-05-09T09:16:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.451+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 msgr2=0x7fa60819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 0x7fa60819e100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80314e0 tx=0x7fa5f8003c10 comp rx=0 tx=0).stop 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 shutdown_connections 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5f40a2c30 0x7fa5f40a50f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa60810d990 0x7fa6081a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa60810c8d0 0x7fa60819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 --2- 172.21.15.64:0/3098690382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa60810bf10 0x7fa60819e100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 >> 172.21.15.64:0/3098690382 conn(0x7fa60806c470 msgr2=0x7fa60806f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 shutdown_connections 2024-05-09T09:16:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:18.452+0000 7fa60f39e640 1 -- 172.21.15.64:0/3098690382 wait complete. 2024-05-09T09:16:18.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:18 smithi179 ceph-mon[34086]: pgmap v585: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 795 op/s 2024-05-09T09:16:18.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:18 smithi139 ceph-mon[33599]: pgmap v585: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 795 op/s 2024-05-09T09:16:18.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:18 smithi064 ceph-mon[30149]: pgmap v585: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 795 op/s 2024-05-09T09:16:20.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:20 smithi064 ceph-mon[30149]: pgmap v586: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:16:20.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:20 smithi179 ceph-mon[34086]: pgmap v586: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:16:20.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:20 smithi139 ceph-mon[33599]: pgmap v586: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:16:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 360.0 s, 935.8 tps, lat 106.163 ms stddev 48.660 2024-05-09T09:16:22.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:22 smithi179 ceph-mon[34086]: pgmap v587: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.9 MiB/s wr, 885 op/s 2024-05-09T09:16:22.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:22 smithi139 ceph-mon[33599]: pgmap v587: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.9 MiB/s wr, 885 op/s 2024-05-09T09:16:22.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:16:22.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:16:22.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:22 smithi064 ceph-mon[30149]: pgmap v587: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.9 MiB/s wr, 885 op/s 2024-05-09T09:16:23.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 -- 172.21.15.64:0/3634922088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 msgr2=0x7f4858077890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 --2- 172.21.15.64:0/3634922088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f4858077890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4848009be0 tx=0x7f484802a6c0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 -- 172.21.15.64:0/3634922088 shutdown_connections 2024-05-09T09:16:23.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 --2- 172.21.15.64:0/3634922088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 0x7f4858078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 --2- 172.21.15.64:0/3634922088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f4858077890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 --2- 172.21.15.64:0/3634922088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f4858076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.062+0000 7f4860514640 1 -- 172.21.15.64:0/3634922088 >> 172.21.15.64:0/3634922088 conn(0x7f48580ff020 msgr2=0x7f4858101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.064+0000 7f4860514640 1 -- 172.21.15.64:0/3634922088 shutdown_connections 2024-05-09T09:16:23.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.064+0000 7f4860514640 1 -- 172.21.15.64:0/3634922088 wait complete. 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 Processor -- start 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 -- start start 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f485819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f485819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 0x7f48581a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4858113a10 con 0x7f48580761f0 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4858113890 con 0x7f4858077430 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.065+0000 7f4860514640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4858113b90 con 0x7f4858077dd0 2024-05-09T09:16:23.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f485819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f485819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36062/0 (socket says 172.21.15.64:36062) 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 -- 172.21.15.64:0/264147126 learned_addr learned my addr 172.21.15.64:0/264147126 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485daaf640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f485819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485eab1640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 0x7f48581a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 -- 172.21.15.64:0/264147126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 msgr2=0x7f48581a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 0x7f48581a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 -- 172.21.15.64:0/264147126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 msgr2=0x7f485819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f485819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 -- 172.21.15.64:0/264147126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485806b380 con 0x7f48580761f0 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485daaf640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f485819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485eab1640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 0x7f48581a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:23.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.066+0000 7f485e2b0640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f485819e100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f484c00e9c0 tx=0x7f484c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.067+0000 7f48577fe640 1 -- 172.21.15.64:0/264147126 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c00cde0 con 0x7f48580761f0 2024-05-09T09:16:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.067+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485806b670 con 0x7f48580761f0 2024-05-09T09:16:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.067+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485806bbe0 con 0x7f48580761f0 2024-05-09T09:16:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.067+0000 7f48577fe640 1 -- 172.21.15.64:0/264147126 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f484c0035f0 con 0x7f48580761f0 2024-05-09T09:16:23.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.068+0000 7f48577fe640 1 -- 172.21.15.64:0/264147126 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0049a0 con 0x7f48580761f0 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.071+0000 7f4863224740 1 Processor -- start 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 -- start start 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28438f30 0x55ea28439350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2843e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea2843ea90 0x55ea28440f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ea2839dd00 con 0x55ea28439920 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ea283454c0 con 0x55ea2843ea90 2024-05-09T09:16:23.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.072+0000 7f4863224740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ea28399cc0 con 0x55ea28438f30 2024-05-09T09:16:23.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.073+0000 7f48577fe640 1 -- 172.21.15.64:0/264147126 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f484c003af0 con 0x7f48580761f0 2024-05-09T09:16:23.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.073+0000 7f485eab1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28438f30 0x55ea28439350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.074+0000 7f485eab1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28438f30 0x55ea28439350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39766/0 (socket says 172.21.15.64:39766) 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.074+0000 7f485eab1640 1 -- 172.21.15.64:0/129868057 learned_addr learned my addr 172.21.15.64:0/129868057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.074+0000 7f485e2b0640 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2843e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.076+0000 7f485e2b0640 1 -- 172.21.15.64:0/129868057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28438f30 msgr2=0x55ea28439350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.076+0000 7f485e2b0640 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28438f30 0x55ea28439350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.076+0000 7f485e2b0640 1 -- 172.21.15.64:0/129868057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea2843ea90 msgr2=0x55ea28440f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.076+0000 7f485e2b0640 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea2843ea90 0x55ea28440f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.076+0000 7f485e2b0640 1 -- 172.21.15.64:0/129868057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2817b7d0 con 0x55ea28439920 2024-05-09T09:16:23.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.076+0000 7f485e2b0640 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2843e420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f484c0cce80 tx=0x7f484c0d5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b304928f1a0278 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.077+0000 7f48577fe640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48400a2be0 0x7f48400a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.077+0000 7f485eab1640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48400a2be0 0x7f48400a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.077+0000 7f48577fe640 1 -- 172.21.15.64:0/264147126 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f484c0cf740 con 0x7f48580761f0 2024-05-09T09:16:23.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.077+0000 7f4854bfe640 1 -- 172.21.15.64:0/129868057 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0de070 con 0x55ea28439920 2024-05-09T09:16:23.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.078+0000 7f4854bfe640 1 -- 172.21.15.64:0/129868057 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f484c09c310 con 0x55ea28439920 2024-05-09T09:16:23.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.078+0000 7f4854bfe640 1 -- 172.21.15.64:0/129868057 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c09b750 con 0x55ea28439920 2024-05-09T09:16:23.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.079+0000 7f4863224740 1 -- 172.21.15.64:0/129868057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 msgr2=0x55ea2843e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.079+0000 7f4863224740 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2843e420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f484c0cce80 tx=0x7f484c0d5f10 comp rx=0 tx=0).stop 2024-05-09T09:16:23.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.079+0000 7f485eab1640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48400a2be0 0x7f48400a50a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4850005fd0 tx=0x7f485000b5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 -- 172.21.15.64:0/129868057 shutdown_connections 2024-05-09T09:16:23.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea2843ea90 0x55ea28440f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2843e420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 --2- 172.21.15.64:0/129868057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28438f30 0x55ea28439350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 -- 172.21.15.64:0/129868057 >> 172.21.15.64:0/129868057 conn(0x55ea284345a0 msgr2=0x55ea284349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 -- 172.21.15.64:0/129868057 shutdown_connections 2024-05-09T09:16:23.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.080+0000 7f4863224740 1 -- 172.21.15.64:0/129868057 wait complete. 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.081+0000 7f4863224740 1 -- start start 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.081+0000 7f4863224740 1 Processor -- start 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.081+0000 7f4863224740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28438f30 0x55ea28454390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f4863224740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 0x55ea284548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f4863224740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea2845b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f4863224740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ea28452a50 con 0x55ea28438f30 2024-05-09T09:16:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f4863224740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ea28358b90 con 0x55ea28439920 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f4863224740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ea28452bd0 con 0x55ea2843ea90 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 0x55ea284548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 0x55ea284548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41674/0 (socket says 172.21.15.64:41674) 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 -- 172.21.15.64:0/492758931 learned_addr learned my addr 172.21.15.64:0/492758931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485eab1640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea2845b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f4863224740 10 client.0 _ll_register_callbacks cb 0x55ea28394da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485e2b0640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28438f30 0x55ea28454390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 -- 172.21.15.64:0/492758931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 msgr2=0x55ea2845b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea2845b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 -- 172.21.15.64:0/492758931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28438f30 msgr2=0x55ea28454390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28438f30 0x55ea28454390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485daaf640 1 -- 172.21.15.64:0/492758931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea280b13f0 con 0x55ea28439920 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485eab1640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea2845b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:23.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.082+0000 7f485e2b0640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28438f30 0x55ea28454390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:23.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f485daaf640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 0x55ea284548d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4848030850 tx=0x7f4848003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b8b193ac428676 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f483dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:23.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f483dffb640 1 -- 172.21.15.64:0/492758931 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848041030 con 0x55ea28439920 2024-05-09T09:16:23.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484c09c310 con 0x55ea28439920 2024-05-09T09:16:23.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f483dffb640 1 -- 172.21.15.64:0/492758931 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4848004040 con 0x55ea28439920 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f483dffb640 1 -- 172.21.15.64:0/492758931 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f484804e050 con 0x55ea28439920 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f4863224740 10 client.26810 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.083+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea280710f0 con 0x55ea28439920 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f483dffb640 1 -- 172.21.15.64:0/492758931 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f484c09c310 con 0x55ea28439920 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f483dffb640 1 -- 172.21.15.64:0/492758931 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f484803c040 con 0x55ea28439920 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f4863224740 10 client.26810 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f4863224740 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f4863224740 20 client.26810 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f4863224740 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284601b0 0x55ea28462650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f4863224740 4 client.26810 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.084+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea283ab430 con 0x55ea284601b0 2024-05-09T09:16:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.085+0000 7f485e2b0640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284601b0 0x55ea28462650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.086+0000 7f485e2b0640 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284601b0 0x55ea28462650 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3145b7a7eb22c467 server_cookie=1c9f2ba9395aba82 in_seq=0 out_seq=0 2024-05-09T09:16:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.086+0000 7f483dffb640 10 client.26810 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:23.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.088+0000 7f483dffb640 1 -- 172.21.15.64:0/492758931 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea283ab430 con 0x55ea284601b0 2024-05-09T09:16:23.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.088+0000 7f483dffb640 10 client.26810 handle_command_reply: tid=0 2024-05-09T09:16:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.090+0000 7f4863224740 1 client.26810 shutdown 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 msgr2=0x55ea284548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 0x55ea284548d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4848030850 tx=0x7f4848003ba0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 shutdown_connections 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284601b0 0x55ea28462650 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea2845b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f483dffb640 0 client.26810 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28439920 0x55ea284548d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 --2- 172.21.15.64:0/492758931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28438f30 0x55ea28454390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 >> 172.21.15.64:0/492758931 conn(0x55ea284345a0 msgr2=0x55ea284337e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 shutdown_connections 2024-05-09T09:16:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 1 -- 172.21.15.64:0/492758931 wait complete. 2024-05-09T09:16:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.091+0000 7f4863224740 20 client.26810 trim_cache size 0 max 16384 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.146+0000 7f4863224740 1 Processor -- start 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 -- start start 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea2843a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2845b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea28454390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48440124e0 con 0x55ea28439920 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f484400fd50 con 0x55ea28438f30 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.147+0000 7f4863224740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f484400dba0 con 0x55ea2843ea90 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485eab1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea28454390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485eab1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea28454390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39778/0 (socket says 172.21.15.64:39778) 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485eab1640 1 -- 172.21.15.64:0/524545722 learned_addr learned my addr 172.21.15.64:0/524545722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485e2b0640 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea2843a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485daaf640 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2845b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485e2b0640 1 -- 172.21.15.64:0/524545722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 msgr2=0x55ea28454390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485e2b0640 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea28454390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485e2b0640 1 -- 172.21.15.64:0/524545722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 msgr2=0x55ea2845b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485e2b0640 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2845b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.148+0000 7f485e2b0640 1 -- 172.21.15.64:0/524545722 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea28455230 con 0x55ea28438f30 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.149+0000 7f485e2b0640 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea2843a1b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f484c09bba0 tx=0x7f484c097eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319d49afd42d1311 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.149+0000 7f483ffff640 1 -- 172.21.15.64:0/524545722 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0de040 con 0x55ea28438f30 2024-05-09T09:16:23.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.149+0000 7f483ffff640 1 -- 172.21.15.64:0/524545722 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f484c0b2070 con 0x55ea28438f30 2024-05-09T09:16:23.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.149+0000 7f483ffff640 1 -- 172.21.15.64:0/524545722 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c09b570 con 0x55ea28438f30 2024-05-09T09:16:23.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.150+0000 7f4863224740 1 -- 172.21.15.64:0/524545722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 msgr2=0x55ea2843a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.150+0000 7f4863224740 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea2843a1b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f484c09bba0 tx=0x7f484c097eb0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.151+0000 7f4863224740 1 -- 172.21.15.64:0/524545722 shutdown_connections 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.151+0000 7f4863224740 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea2843ea90 0x55ea28454390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.151+0000 7f4863224740 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea28439920 0x55ea2845b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.151+0000 7f4863224740 1 --2- 172.21.15.64:0/524545722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea2843a1b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.151+0000 7f4863224740 1 -- 172.21.15.64:0/524545722 >> 172.21.15.64:0/524545722 conn(0x55ea284345a0 msgr2=0x55ea28434980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.152+0000 7f4863224740 1 -- 172.21.15.64:0/524545722 shutdown_connections 2024-05-09T09:16:23.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.152+0000 7f4863224740 1 -- 172.21.15.64:0/524545722 wait complete. 2024-05-09T09:16:23.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.152+0000 7f4863224740 1 -- start start 2024-05-09T09:16:23.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.153+0000 7f4863224740 1 Processor -- start 2024-05-09T09:16:23.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.153+0000 7f4863224740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea28458e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.153+0000 7f4863224740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 0x55ea28459380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f4863224740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea2843ea90 0x55ea2845a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f485daaf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 0x55ea28459380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f485daaf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 0x55ea28459380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39788/0 (socket says 172.21.15.64:39788) 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f485e2b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea28458e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f485e2b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea28458e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41694/0 (socket says 172.21.15.64:41694) 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f485daaf640 1 -- 172.21.15.64:0/3268966208 learned_addr learned my addr 172.21.15.64:0/3268966208 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f484400e210 con 0x55ea2843ea90 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f485eab1640 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea2843ea90 0x55ea2845a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f484400f680 con 0x55ea28438f30 2024-05-09T09:16:23.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.154+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f484400e8e0 con 0x55ea28439920 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f4863224740 10 client.0 _ll_register_callbacks cb 0x7f484c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f485daaf640 1 -- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 msgr2=0x55ea28458e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f485daaf640 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea28458e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f485daaf640 1 -- 172.21.15.64:0/3268966208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea2843ea90 msgr2=0x55ea2845a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f485daaf640 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea2843ea90 0x55ea2845a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f485daaf640 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea28438a90 con 0x55ea28439920 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484c0b2070 con 0x55ea28439920 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f485daaf640 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 0x55ea28459380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4848009560 tx=0x7f4848009af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a194f33fed9e40c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.155+0000 7f483dffb640 10 client.26992 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.156+0000 7f483dffb640 1 -- 172.21.15.64:0/3268966208 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848041070 con 0x55ea28439920 2024-05-09T09:16:23.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.156+0000 7f483dffb640 1 -- 172.21.15.64:0/3268966208 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4848004330 con 0x55ea28439920 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.156+0000 7f483dffb640 1 -- 172.21.15.64:0/3268966208 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4848036040 con 0x55ea28439920 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.156+0000 7f4863224740 10 client.26992 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.156+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55ea2845ba20 con 0x55ea28439920 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.156+0000 7f483dffb640 1 -- 172.21.15.64:0/3268966208 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f484803c040 con 0x55ea28439920 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.157+0000 7f4863224740 10 client.26992 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.157+0000 7f4863224740 10 client.26992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.157+0000 7f4863224740 20 client.26992 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.157+0000 7f4863224740 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284c6c50 0x55ea284c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.157+0000 7f4863224740 4 client.26992 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:16:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.157+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea283a5780 con 0x55ea284c6c50 2024-05-09T09:16:23.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.158+0000 7f485e2b0640 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284c6c50 0x55ea284c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.159+0000 7f485e2b0640 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284c6c50 0x55ea284c9070 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a55c8f162c525a server_cookie=8068ae0c2f54c2ba in_seq=0 out_seq=0 2024-05-09T09:16:23.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.159+0000 7f483dffb640 10 client.26992 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:23.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.160+0000 7f483dffb640 1 -- 172.21.15.64:0/3268966208 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea283a5780 con 0x55ea284c6c50 2024-05-09T09:16:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.160+0000 7f483dffb640 10 client.26992 handle_command_reply: tid=0 2024-05-09T09:16:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 client.26992 shutdown 2024-05-09T09:16:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 msgr2=0x55ea28459380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 0x55ea28459380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4848009560 tx=0x7f4848009af0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 shutdown_connections 2024-05-09T09:16:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ea284c6c50 0x55ea284c9070 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea2843ea90 0x55ea2845a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea28439920 0x55ea28459380 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 --2- 172.21.15.64:0/3268966208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ea28438f30 0x55ea28458e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.161+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 >> 172.21.15.64:0/3268966208 conn(0x55ea284345a0 msgr2=0x55ea2843a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.162+0000 7f483dffb640 0 client.26992 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.162+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 shutdown_connections 2024-05-09T09:16:23.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.162+0000 7f4863224740 1 -- 172.21.15.64:0/3268966208 wait complete. 2024-05-09T09:16:23.165 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:16:23.165 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:16:23.165 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:16:23.165 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.163+0000 7f4863224740 20 client.26992 trim_cache size 0 max 16384 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.164+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48400a2be0 msgr2=0x7f48400a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.165+0000 7f4860514640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48400a2be0 0x7f48400a50a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4850005fd0 tx=0x7f485000b5e0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.165+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 msgr2=0x7f485819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.165+0000 7f4860514640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f485819e100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f484c00e9c0 tx=0x7f484c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.166+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 shutdown_connections 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.166+0000 7f4860514640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f48400a2be0 0x7f48400a50a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.166+0000 7f4860514640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4858077dd0 0x7f48581a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.166+0000 7f4860514640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4858077430 0x7f485819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.166+0000 7f4860514640 1 --2- 172.21.15.64:0/264147126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48580761f0 0x7f485819e100 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.166+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 >> 172.21.15.64:0/264147126 conn(0x7f48580ff020 msgr2=0x7f4858102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.167+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 shutdown_connections 2024-05-09T09:16:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.167+0000 7f4860514640 1 -- 172.21.15.64:0/264147126 wait complete. 2024-05-09T09:16:23.187 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:16:23.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:16:23.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 -- 172.21.15.64:0/1154986483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 msgr2=0x7fec90105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 --2- 172.21.15.64:0/1154986483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 0x7fec90105ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fec88009be0 tx=0x7fec8802a6c0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 -- 172.21.15.64:0/1154986483 shutdown_connections 2024-05-09T09:16:23.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 --2- 172.21.15.64:0/1154986483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 0x7fec90105ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 --2- 172.21.15.64:0/1154986483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec90100850 0x7fec90100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 --2- 172.21.15.64:0/1154986483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec900ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.413+0000 7fec96f42640 1 -- 172.21.15.64:0/1154986483 >> 172.21.15.64:0/1154986483 conn(0x7fec900fb020 msgr2=0x7fec900fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.414+0000 7fec96f42640 1 -- 172.21.15.64:0/1154986483 shutdown_connections 2024-05-09T09:16:23.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.414+0000 7fec96f42640 1 -- 172.21.15.64:0/1154986483 wait complete. 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.414+0000 7fec96f42640 1 Processor -- start 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.414+0000 7fec96f42640 1 -- start start 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec96f42640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec9019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec96f42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec90100850 0x7fec9019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec96f42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 0x7fec901a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec96f42640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec901139b0 con 0x7fec90100850 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec96f42640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec90113830 con 0x7fec901011f0 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec96f42640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec90113b30 con 0x7fec900ff720 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec94cde640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec9019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec954df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 0x7fec901a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec94cde640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec9019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39810/0 (socket says 172.21.15.64:39810) 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec94cde640 1 -- 172.21.15.64:0/2280778061 learned_addr learned my addr 172.21.15.64:0/2280778061 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.415+0000 7fec8ffff640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec90100850 0x7fec9019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.416+0000 7fec94cde640 1 -- 172.21.15.64:0/2280778061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 msgr2=0x7fec901a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.416+0000 7fec94cde640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 0x7fec901a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.416+0000 7fec94cde640 1 -- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec90100850 msgr2=0x7fec9019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.416+0000 7fec94cde640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec90100850 0x7fec9019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.416+0000 7fec94cde640 1 -- 172.21.15.64:0/2280778061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec901a5be0 con 0x7fec900ff720 2024-05-09T09:16:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.416+0000 7fec94cde640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec9019e010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec8000ea60 tx=0x7fec8000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.417+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8000cde0 con 0x7fec900ff720 2024-05-09T09:16:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.417+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec800035f0 con 0x7fec900ff720 2024-05-09T09:16:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.417+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80010690 con 0x7fec900ff720 2024-05-09T09:16:23.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.417+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec901a5ed0 con 0x7fec900ff720 2024-05-09T09:16:23.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.418+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec901a6330 con 0x7fec900ff720 2024-05-09T09:16:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.419+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c0052c0 con 0x7fec900ff720 2024-05-09T09:16:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.421+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec80004000 con 0x7fec900ff720 2024-05-09T09:16:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.423+0000 7fec8dffb640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec780a2ce0 0x7fec780a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.423+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fec800cf500 con 0x7fec900ff720 2024-05-09T09:16:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.423+0000 7fec8ffff640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec780a2ce0 0x7fec780a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:23.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.424+0000 7fec8ffff640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec780a2ce0 0x7fec780a51a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fec9019f4d0 tx=0x7fec84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.428+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec800042f0 con 0x7fec900ff720 2024-05-09T09:16:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.867+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec7c0055b0 con 0x7fec900ff720 2024-05-09T09:16:23.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:16:23.868 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:16:15.088273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:16:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 43 2024-05-09T09:16:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.868+0000 7fec8dffb640 1 -- 172.21.15.64:0/2280778061 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11882 (secure 0 0 0) 0x7fec8009c070 con 0x7fec900ff720 2024-05-09T09:16:23.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.872+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec780a2ce0 msgr2=0x7fec780a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.872+0000 7fec96f42640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec780a2ce0 0x7fec780a51a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fec9019f4d0 tx=0x7fec84009260 comp rx=0 tx=0).stop 2024-05-09T09:16:23.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.872+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 msgr2=0x7fec9019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:23.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.872+0000 7fec96f42640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec9019e010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec8000ea60 tx=0x7fec8000ef20 comp rx=0 tx=0).stop 2024-05-09T09:16:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 shutdown_connections 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fec780a2ce0 0x7fec780a51a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec901011f0 0x7fec901a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec90100850 0x7fec9019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 --2- 172.21.15.64:0/2280778061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec900ff720 0x7fec9019e010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 >> 172.21.15.64:0/2280778061 conn(0x7fec900fb020 msgr2=0x7fec90101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 shutdown_connections 2024-05-09T09:16:23.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:23.875+0000 7fec96f42640 1 -- 172.21.15.64:0/2280778061 wait complete. 2024-05-09T09:16:23.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 -- 172.21.15.64:0/2872892284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14078fa0 msgr2=0x7faa1407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 --2- 172.21.15.64:0/2872892284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14078fa0 0x7faa1407b430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009ca0 tx=0x7faa0c02a8d0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 -- 172.21.15.64:0/2872892284 shutdown_connections 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 --2- 172.21.15.64:0/2872892284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14078fa0 0x7faa1407b430 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 --2- 172.21.15.64:0/2872892284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa140765f0 0x7faa14078a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 --2- 172.21.15.64:0/2872892284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa14073710 0x7faa14073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.091+0000 7faa1ad70640 1 -- 172.21.15.64:0/2872892284 >> 172.21.15.64:0/2872892284 conn(0x7faa1406e6c0 msgr2=0x7faa14070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.092+0000 7faa1ad70640 1 -- 172.21.15.64:0/2872892284 shutdown_connections 2024-05-09T09:16:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.092+0000 7faa1ad70640 1 -- 172.21.15.64:0/2872892284 wait complete. 2024-05-09T09:16:24.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.093+0000 7faa1ad70640 1 Processor -- start 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.093+0000 7faa1ad70640 1 -- start start 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.093+0000 7faa1ad70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 0x7faa14082c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.093+0000 7faa1ad70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa140765f0 0x7faa14083190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa1ad70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14078fa0 0x7faa140836d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa1ad70640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa1412b650 con 0x7faa140765f0 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa1ad70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa1412b4d0 con 0x7faa14073710 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa1ad70640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa1412b7d0 con 0x7faa14078fa0 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 0x7faa14082c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 0x7faa14082c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41750/0 (socket says 172.21.15.64:41750) 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 -- 172.21.15.64:0/1227901861 learned_addr learned my addr 172.21.15.64:0/1227901861 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa1930d640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14078fa0 0x7faa140836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa13fff640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa140765f0 0x7faa14083190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 -- 172.21.15.64:0/1227901861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14078fa0 msgr2=0x7faa140836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14078fa0 0x7faa140836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 -- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa140765f0 msgr2=0x7faa14083190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa140765f0 0x7faa14083190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa18b0c640 1 -- 172.21.15.64:0/1227901861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa14132050 con 0x7faa14073710 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.094+0000 7faa1930d640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14078fa0 0x7faa140836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.095+0000 7faa18b0c640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 0x7faa14082c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faa0800d950 tx=0x7faa0800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.095+0000 7faa11ffb640 1 -- 172.21.15.64:0/1227901861 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08017070 con 0x7faa14073710 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.095+0000 7faa11ffb640 1 -- 172.21.15.64:0/1227901861 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa08009c30 con 0x7faa14073710 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.095+0000 7faa11ffb640 1 -- 172.21.15.64:0/1227901861 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800f620 con 0x7faa14073710 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.096+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa141322e0 con 0x7faa14073710 2024-05-09T09:16:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.097+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141327a0 con 0x7faa14073710 2024-05-09T09:16:24.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.102+0000 7faa11ffb640 1 -- 172.21.15.64:0/1227901861 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa080109a0 con 0x7faa14073710 2024-05-09T09:16:24.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.102+0000 7faa1da82740 1 Processor -- start 2024-05-09T09:16:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 -- start start 2024-05-09T09:16:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 0x562e55032360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55037430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55039f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e54f96c40 con 0x562e55037aa0 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562e54f3e580 con 0x562e55031f40 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1da82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562e54f92c00 con 0x562e55032930 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1930d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 0x562e55032360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1930d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 0x562e55032360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41766/0 (socket says 172.21.15.64:41766) 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa1930d640 1 -- 172.21.15.64:0/1890779787 learned_addr learned my addr 172.21.15.64:0/1890779787 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.103+0000 7faa18b0c640 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55037430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa11ffb640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faa040a2ce0 0x7faa040a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa11ffb640 1 -- 172.21.15.64:0/1227901861 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa080ce9f0 con 0x7faa14073710 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa1930d640 1 -- 172.21.15.64:0/1890779787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 msgr2=0x562e55037430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa1930d640 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55037430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa1930d640 1 -- 172.21.15.64:0/1890779787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 msgr2=0x562e55039f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa1930d640 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55039f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.104+0000 7faa1930d640 1 -- 172.21.15.64:0/1890779787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e54caa3f0 con 0x562e55031f40 2024-05-09T09:16:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.105+0000 7faa1930d640 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 0x562e55032360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa0c02f3b0 tx=0x7faa0c009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776d393269d922d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.105+0000 7faa1930d640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faa040a2ce0 0x7faa040a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.106+0000 7faa1930d640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faa040a2ce0 0x7faa040a51a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0049d0 tx=0x7faa0c004800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.107+0000 7fa9feffd640 1 -- 172.21.15.64:0/1890779787 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c04d070 con 0x562e55031f40 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.108+0000 7fa9feffd640 1 -- 172.21.15.64:0/1890779787 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0c036040 con 0x562e55031f40 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.108+0000 7fa9feffd640 1 -- 172.21.15.64:0/1890779787 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0045a0 con 0x562e55031f40 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 -- 172.21.15.64:0/1890779787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 msgr2=0x562e55032360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 0x562e55032360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa0c02f3b0 tx=0x7faa0c009900 comp rx=0 tx=0).stop 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 -- 172.21.15.64:0/1890779787 shutdown_connections 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55039f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55037430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 --2- 172.21.15.64:0/1890779787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55031f40 0x562e55032360 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.109+0000 7faa1da82740 1 -- 172.21.15.64:0/1890779787 >> 172.21.15.64:0/1890779787 conn(0x562e5502d5b0 msgr2=0x562e5502d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.110+0000 7faa1da82740 1 -- 172.21.15.64:0/1890779787 shutdown_connections 2024-05-09T09:16:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.110+0000 7faa1da82740 1 -- 172.21.15.64:0/1890779787 wait complete. 2024-05-09T09:16:24.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.111+0000 7faa1da82740 1 -- start start 2024-05-09T09:16:24.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.111+0000 7faa1da82740 1 Processor -- start 2024-05-09T09:16:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55043900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e5503aa50 con 0x562e55037aa0 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562e54f51c40 con 0x562e55032930 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562e5503abd0 con 0x562e55031f40 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.112+0000 7faa1da82740 10 client.0 _ll_register_callbacks cb 0x562e54ca9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa18b0c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55043900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa18b0c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39832/0 (socket says 172.21.15.64:39832) 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55043900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36144/0 (socket says 172.21.15.64:36144) 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa18b0c640 1 -- 172.21.15.64:0/1240207858 learned_addr learned my addr 172.21.15.64:0/1240207858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa13fff640 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 -- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55031f40 msgr2=0x562e5503c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 -- 172.21.15.64:0/1240207858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55032930 msgr2=0x562e5503c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 -- 172.21.15.64:0/1240207858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e54d747d0 con 0x562e55037aa0 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.113+0000 7faa1930d640 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55043900 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faa0c031980 tx=0x7faa0c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d701d2ac1557d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.114+0000 7fa9f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.114+0000 7fa9f3fff640 1 -- 172.21.15.64:0/1240207858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c05b060 con 0x562e55037aa0 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.114+0000 7fa9f3fff640 1 -- 172.21.15.64:0/1240207858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa0c031460 con 0x562e55037aa0 2024-05-09T09:16:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.114+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0c036040 con 0x562e55037aa0 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.115+0000 7fa9f3fff640 1 -- 172.21.15.64:0/1240207858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faa0c04d070 con 0x562e55037aa0 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.115+0000 7faa1da82740 10 client.17349 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.115+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562e55046c70 con 0x562e55037aa0 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7fa9f3fff640 1 -- 172.21.15.64:0/1240207858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0c036040 con 0x562e55037aa0 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7fa9f3fff640 1 -- 172.21.15.64:0/1240207858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7faa0c043050 con 0x562e55037aa0 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7faa1da82740 10 client.17349 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7faa1da82740 10 client.17349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7faa1da82740 20 client.17349 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7faa1da82740 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e55047fc0 0x562e5504a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7faa1da82740 4 client.17349 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.116+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e54fa4370 con 0x562e55047fc0 2024-05-09T09:16:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.117+0000 7faa18b0c640 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e55047fc0 0x562e5504a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.118+0000 7faa18b0c640 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e55047fc0 0x562e5504a3b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d7ba4ab6b03d9a server_cookie=96102d0fa7be5a73 in_seq=0 out_seq=0 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.118+0000 7fa9f3fff640 10 client.17349 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7fa9f3fff640 1 -- 172.21.15.64:0/1240207858 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e54fa4370 con 0x562e55047fc0 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7fa9f3fff640 10 client.17349 handle_command_reply: tid=0 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 client.17349 shutdown 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 msgr2=0x562e55043900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55043900 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faa0c031980 tx=0x7faa0c04e000 comp rx=0 tx=0).stop 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 shutdown_connections 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e55047fc0 0x562e5504a3b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55037aa0 0x562e55043900 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 --2- 172.21.15.64:0/1240207858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 >> 172.21.15.64:0/1240207858 conn(0x562e5502d5b0 msgr2=0x562e5502c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.119+0000 7fa9f3fff640 0 client.17349 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.120+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 shutdown_connections 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.121+0000 7faa1da82740 1 -- 172.21.15.64:0/1240207858 wait complete. 2024-05-09T09:16:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.121+0000 7faa1da82740 20 client.17349 trim_cache size 0 max 16384 2024-05-09T09:16:24.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.146+0000 7faa1da82740 1 Processor -- start 2024-05-09T09:16:24.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.146+0000 7faa1da82740 1 -- start start 2024-05-09T09:16:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.147+0000 7faa1da82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.147+0000 7faa1da82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.147+0000 7faa1da82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.147+0000 7faa1da82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa141d5400 con 0x562e55031f40 2024-05-09T09:16:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.147+0000 7faa1da82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa141d2c70 con 0x562e55037aa0 2024-05-09T09:16:24.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.147+0000 7faa1da82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa141d0ac0 con 0x562e55032930 2024-05-09T09:16:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41790/0 (socket says 172.21.15.64:41790) 2024-05-09T09:16:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 -- 172.21.15.64:0/3482555714 learned_addr learned my addr 172.21.15.64:0/3482555714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa13fff640 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 -- 172.21.15.64:0/3482555714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 msgr2=0x562e5503c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 -- 172.21.15.64:0/3482555714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 msgr2=0x562e5503c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.148+0000 7faa1930d640 1 -- 172.21.15.64:0/3482555714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e55032d10 con 0x562e55037aa0 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.149+0000 7faa1930d640 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550438c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faa0c006010 tx=0x7faa0c02f340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e60a631bcfc0144 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.149+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3482555714 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c05b040 con 0x562e55037aa0 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.149+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3482555714 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0c036070 con 0x562e55037aa0 2024-05-09T09:16:24.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.149+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3482555714 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c065080 con 0x562e55037aa0 2024-05-09T09:16:24.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 -- 172.21.15.64:0/3482555714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 msgr2=0x562e550438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550438c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faa0c006010 tx=0x7faa0c02f340 comp rx=0 tx=0).stop 2024-05-09T09:16:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 -- 172.21.15.64:0/3482555714 shutdown_connections 2024-05-09T09:16:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550438c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e5503c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 --2- 172.21.15.64:0/3482555714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e5503c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.151+0000 7faa1da82740 1 -- 172.21.15.64:0/3482555714 >> 172.21.15.64:0/3482555714 conn(0x562e5502d5b0 msgr2=0x562e55033df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.153+0000 7faa1da82740 1 -- 172.21.15.64:0/3482555714 shutdown_connections 2024-05-09T09:16:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.153+0000 7faa1da82740 1 -- 172.21.15.64:0/3482555714 wait complete. 2024-05-09T09:16:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.153+0000 7faa1da82740 1 -- start start 2024-05-09T09:16:24.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.154+0000 7faa1da82740 1 Processor -- start 2024-05-09T09:16:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e55040ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55041220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa141d1130 con 0x562e55031f40 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa141d25a0 con 0x562e55037aa0 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa141d1800 con 0x562e55032930 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa1da82740 10 client.0 _ll_register_callbacks cb 0x7faa0c04b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa18b0c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e55040ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa18b0c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e55040ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36150/0 (socket says 172.21.15.64:36150) 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa18b0c640 1 -- 172.21.15.64:0/45751888 learned_addr learned my addr 172.21.15.64:0/45751888 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.155+0000 7faa13fff640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55041220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.156+0000 7faa1930d640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.156+0000 7faa13fff640 1 -- 172.21.15.64:0/45751888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 msgr2=0x562e550421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.156+0000 7faa13fff640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.156+0000 7faa13fff640 1 -- 172.21.15.64:0/45751888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 msgr2=0x562e55040ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.156+0000 7faa13fff640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e55040ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.156+0000 7faa13fff640 1 -- 172.21.15.64:0/45751888 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e550427d0 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.157+0000 7faa13fff640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55041220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x562e55037700 tx=0x7faa00011630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ab1237e1c74cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.157+0000 7fa9f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.157+0000 7fa9f3fff640 1 -- 172.21.15.64:0/45751888 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00019070 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.157+0000 7fa9f3fff640 1 -- 172.21.15.64:0/45751888 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa00015440 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.157+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0c036070 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.157+0000 7fa9f3fff640 1 -- 172.21.15.64:0/45751888 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faa00023050 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7faa1da82740 10 client.27010 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562e55043b90 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7fa9f3fff640 1 -- 172.21.15.64:0/45751888 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa00015af0 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7fa9f3fff640 1 -- 172.21.15.64:0/45751888 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7faa0001e8d0 con 0x562e55032930 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7faa1da82740 10 client.27010 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7faa1da82740 10 client.27010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:16:24.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.158+0000 7faa1da82740 20 client.27010 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.159+0000 7faa1da82740 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e550aeca0 0x562e550b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.159+0000 7faa1da82740 4 client.27010 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:16:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.159+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7faa0c036070 con 0x562e550aeca0 2024-05-09T09:16:24.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.161+0000 7faa18b0c640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e550aeca0 0x562e550b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa18b0c640 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e550aeca0 0x562e550b10c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b180683a8476d3ac server_cookie=9bf770a9b42ffaab in_seq=0 out_seq=0 2024-05-09T09:16:24.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7fa9f3fff640 10 client.27010 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:24.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7fa9f3fff640 1 -- 172.21.15.64:0/45751888 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7faa0c036070 con 0x562e550aeca0 2024-05-09T09:16:24.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7fa9f3fff640 10 client.27010 handle_command_reply: tid=0 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 client.27010 shutdown 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 msgr2=0x562e55041220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55041220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x562e55037700 tx=0x7faa00011630 comp rx=0 tx=0).stop 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 shutdown_connections 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562e550aeca0 0x562e550b10c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e55037aa0 0x562e550421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7fa9f3fff640 0 client.27010 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e55032930 0x562e55041220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 --2- 172.21.15.64:0/45751888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e55031f40 0x562e55040ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.162+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 >> 172.21.15.64:0/45751888 conn(0x562e5502d5b0 msgr2=0x562e55033df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.163+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 shutdown_connections 2024-05-09T09:16:24.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.163+0000 7faa1da82740 1 -- 172.21.15.64:0/45751888 wait complete. 2024-05-09T09:16:24.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.163+0000 7faa1da82740 20 client.27010 trim_cache size 0 max 16384 2024-05-09T09:16:24.165 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:16:24.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.165+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faa040a2ce0 msgr2=0x7faa040a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.165+0000 7faa1ad70640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faa040a2ce0 0x7faa040a51a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0049d0 tx=0x7faa0c004800 comp rx=0 tx=0).stop 2024-05-09T09:16:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.165+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 msgr2=0x7faa14082c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.165+0000 7faa1ad70640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 0x7faa14082c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faa0800d950 tx=0x7faa0800de10 comp rx=0 tx=0).stop 2024-05-09T09:16:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 shutdown_connections 2024-05-09T09:16:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faa040a2ce0 0x7faa040a51a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14078fa0 0x7faa140836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa140765f0 0x7faa14083190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 --2- 172.21.15.64:0/1227901861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14073710 0x7faa14082c50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 >> 172.21.15.64:0/1227901861 conn(0x7faa1406e6c0 msgr2=0x7faa14076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.166+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 shutdown_connections 2024-05-09T09:16:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.167+0000 7faa1ad70640 1 -- 172.21.15.64:0/1227901861 wait complete. 2024-05-09T09:16:24.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:16:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 -- 172.21.15.64:0/3704372505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a6700 msgr2=0x7f7aac0aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/3704372505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a6700 0x7f7aac0aafc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009900 tx=0x7f7aa402a410 comp rx=0 tx=0).stop 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 -- 172.21.15.64:0/3704372505 shutdown_connections 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/3704372505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a6700 0x7f7aac0aafc0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/3704372505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a5d60 0x7f7aac0a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/3704372505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a4c30 0x7f7aac0a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 -- 172.21.15.64:0/3704372505 >> 172.21.15.64:0/3704372505 conn(0x7f7aac0a00c0 msgr2=0x7f7aac0a2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.397+0000 7f7ab3fff640 1 -- 172.21.15.64:0/3704372505 shutdown_connections 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.398+0000 7f7ab3fff640 1 -- 172.21.15.64:0/3704372505 wait complete. 2024-05-09T09:16:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.398+0000 7f7ab3fff640 1 Processor -- start 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 -- start start 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a4c30 0x7f7aac143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a5d60 0x7f7aac143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 0x7f7aac14ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac0b8f60 con 0x7f7aac0a5d60 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac0b8de0 con 0x7f7aac0a6700 2024-05-09T09:16:24.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab3fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac0b90e0 con 0x7f7aac0a4c30 2024-05-09T09:16:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 0x7f7aac14ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a4c30 0x7f7aac143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 0x7f7aac14ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:41822/0 (socket says 172.21.15.64:41822) 2024-05-09T09:16:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a4c30 0x7f7aac143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39872/0 (socket says 172.21.15.64:39872) 2024-05-09T09:16:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab37fe640 1 -- 172.21.15.64:0/1893457987 learned_addr learned my addr 172.21.15.64:0/1893457987 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.399+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a5d60 0x7f7aac143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.400+0000 7f7ab37fe640 1 -- 172.21.15.64:0/1893457987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a4c30 msgr2=0x7f7aac143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.400+0000 7f7ab37fe640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a4c30 0x7f7aac143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.400+0000 7f7ab37fe640 1 -- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a5d60 msgr2=0x7f7aac143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.400+0000 7f7ab37fe640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a5d60 0x7f7aac143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.400+0000 7f7ab37fe640 1 -- 172.21.15.64:0/1893457987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0ac980 con 0x7f7aac0a6700 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.400+0000 7f7ab37fe640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 0x7f7aac14ab20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4003820 tx=0x7f7aa4003850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.401+0000 7f7a9bfff640 1 -- 172.21.15.64:0/1893457987 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa403f030 con 0x7f7aac0a6700 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.401+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac0acc10 con 0x7f7aac0a6700 2024-05-09T09:16:24.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.401+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac0ad100 con 0x7f7aac0a6700 2024-05-09T09:16:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.401+0000 7f7a9bfff640 1 -- 172.21.15.64:0/1893457987 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa4003c00 con 0x7f7aac0a6700 2024-05-09T09:16:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.401+0000 7f7a9bfff640 1 -- 172.21.15.64:0/1893457987 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4035070 con 0x7f7aac0a6700 2024-05-09T09:16:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.405+0000 7f7a9bfff640 1 -- 172.21.15.64:0/1893457987 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7aa4042030 con 0x7f7aac0a6700 2024-05-09T09:16:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.405+0000 7f7abc5ee740 1 Processor -- start 2024-05-09T09:16:24.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 -- start start 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 0x55598e1cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1ccc60 0x55598e1cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55598e12bc40 con 0x55598e1ccc60 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55598e0d3580 con 0x55598e1c7100 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55598e127c00 con 0x55598e1c7af0 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7ab2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 0x55598e1cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7ab2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 0x55598e1cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39878/0 (socket says 172.21.15.64:39878) 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7ab2ffd640 1 -- 172.21.15.64:0/72846722 learned_addr learned my addr 172.21.15.64:0/72846722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.406+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1ccc60 0x55598e1cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab2ffd640 1 -- 172.21.15.64:0/72846722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 msgr2=0x55598e1c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab2ffd640 1 -- 172.21.15.64:0/72846722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1ccc60 msgr2=0x55598e1cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1ccc60 0x55598e1cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab2ffd640 1 -- 172.21.15.64:0/72846722 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598de3f3f0 con 0x55598e1c7af0 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7a9bfff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a9c0a2da0 0x7f7a9c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1ccc60 0x55598e1cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7a9bfff640 1 -- 172.21.15.64:0/1893457987 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7aa40ef8c0 con 0x7f7aac0a6700 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.407+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a9c0a2da0 0x7f7a9c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.408+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 0x55598e1cc5f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4050ef0 tx=0x7f7ab4072ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d84d372a0cd35a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.408+0000 7f7a997fa640 1 -- 172.21.15.64:0/72846722 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4075bf0 con 0x55598e1c7af0 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.408+0000 7f7a997fa640 1 -- 172.21.15.64:0/72846722 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ab4075d90 con 0x55598e1c7af0 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.409+0000 7f7a997fa640 1 -- 172.21.15.64:0/72846722 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40736f0 con 0x55598e1c7af0 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.409+0000 7f7abc5ee740 1 -- 172.21.15.64:0/72846722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 msgr2=0x55598e1cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.409+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 0x55598e1cc5f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4050ef0 tx=0x7f7ab4072ce0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a9c0a2da0 0x7f7a9c0a5260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55598e1c8350 tx=0x7f7aa8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 -- 172.21.15.64:0/72846722 shutdown_connections 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1ccc60 0x55598e1cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1c7af0 0x55598e1cc5f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/72846722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 -- 172.21.15.64:0/72846722 >> 172.21.15.64:0/72846722 conn(0x55598e1c2770 msgr2=0x55598e1c2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 -- 172.21.15.64:0/72846722 shutdown_connections 2024-05-09T09:16:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 -- 172.21.15.64:0/72846722 wait complete. 2024-05-09T09:16:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 -- start start 2024-05-09T09:16:24.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.410+0000 7f7abc5ee740 1 Processor -- start 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55598e1cfc10 con 0x55598e1c7af0 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55598e0e6c40 con 0x55598e1c7100 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55598e1cfd90 con 0x55598e1ccc60 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36196/0 (socket says 172.21.15.64:36196) 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1950985052 learned_addr learned my addr 172.21.15.64:0/1950985052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7abc5ee740 10 client.0 _ll_register_callbacks cb 0x55598de3ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab37fe640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 msgr2=0x55598e1d8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1950985052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 msgr2=0x55598e1d14f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1d14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.411+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1950985052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598df097d0 con 0x55598e1c7af0 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.412+0000 7f7ab37fe640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.412+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d1a30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8015a00 tx=0x7f7aa80090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8f0dffc1862406 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.412+0000 7f7a96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.412+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1950985052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8026020 con 0x55598e1c7af0 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.413+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab4075d90 con 0x55598e1c7af0 2024-05-09T09:16:24.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.413+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1950985052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7aa8014930 con 0x55598e1c7af0 2024-05-09T09:16:24.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.413+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1950985052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7aa8031050 con 0x55598e1c7af0 2024-05-09T09:16:24.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.413+0000 7f7abc5ee740 10 client.17361 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:24.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.413+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55598e1dbdf0 con 0x55598e1c7af0 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.414+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1950985052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ab4075d90 con 0x55598e1c7af0 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1950985052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa8022ba0 con 0x55598e1c7af0 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7abc5ee740 10 client.17361 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7abc5ee740 10 client.17361 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7abc5ee740 20 client.17361 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e1dda00 0x55598e1dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7abc5ee740 4 client.17361 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.415+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55598e139370 con 0x55598e1dda00 2024-05-09T09:16:24.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.418+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e1dda00 0x55598e1dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.418+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e1dda00 0x55598e1dfec0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad615786edf0210f server_cookie=12ac2d0d00d30190 in_seq=0 out_seq=0 2024-05-09T09:16:24.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.418+0000 7f7a96ff5640 10 client.17361 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:24.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.419+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1950985052 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55598e139370 con 0x55598e1dda00 2024-05-09T09:16:24.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.419+0000 7f7a96ff5640 10 client.17361 handle_command_reply: tid=0 2024-05-09T09:16:24.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.419+0000 7f7abc5ee740 1 client.17361 shutdown 2024-05-09T09:16:24.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.421+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 msgr2=0x55598e1d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.421+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d1a30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8015a00 tx=0x7f7aa80090e0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 shutdown_connections 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e1dda00 0x55598e1dfec0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d1a30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1950985052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 >> 172.21.15.64:0/1950985052 conn(0x55598e1c2770 msgr2=0x55598e1c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.423+0000 7f7a96ff5640 0 client.17361 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:24.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.426+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 shutdown_connections 2024-05-09T09:16:24.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.426+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1950985052 wait complete. 2024-05-09T09:16:24.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.429+0000 7f7abc5ee740 20 client.17361 trim_cache size 0 max 16384 2024-05-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.458+0000 7f7abc5ee740 1 Processor -- start 2024-05-09T09:16:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 -- start start 2024-05-09T09:16:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0012600 con 0x55598e1c7af0 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000fe70 con 0x55598e1c7100 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000dcc0 con 0x55598e1ccc60 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7ab37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7ab27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7ab37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39890/0 (socket says 172.21.15.64:39890) 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.459+0000 7f7ab37fe640 1 -- 172.21.15.64:0/2995672595 learned_addr learned my addr 172.21.15.64:0/2995672595 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab27fc640 1 -- 172.21.15.64:0/2995672595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 msgr2=0x55598e1d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab27fc640 1 -- 172.21.15.64:0/2995672595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 msgr2=0x55598e1c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab27fc640 1 -- 172.21.15.64:0/2995672595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598e1c20c0 con 0x55598e1c7af0 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.460+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d8a30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8017040 tx=0x7f7aa8021890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981a36ece1b18f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.461+0000 7f7a98ff9640 1 -- 172.21.15.64:0/2995672595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa805c070 con 0x55598e1c7af0 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.461+0000 7f7a98ff9640 1 -- 172.21.15.64:0/2995672595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa801b030 con 0x55598e1c7af0 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.461+0000 7f7a98ff9640 1 -- 172.21.15.64:0/2995672595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8026040 con 0x55598e1c7af0 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 -- 172.21.15.64:0/2995672595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 msgr2=0x55598e1d8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d8a30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8017040 tx=0x7f7aa8021890 comp rx=0 tx=0).stop 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 -- 172.21.15.64:0/2995672595 shutdown_connections 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7af0 0x55598e1d8a30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/2995672595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7100 0x55598e1c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 -- 172.21.15.64:0/2995672595 >> 172.21.15.64:0/2995672595 conn(0x55598e1c2770 msgr2=0x55598e1c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 -- 172.21.15.64:0/2995672595 shutdown_connections 2024-05-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 -- 172.21.15.64:0/2995672595 wait complete. 2024-05-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.462+0000 7f7abc5ee740 1 -- start start 2024-05-09T09:16:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 Processor -- start 2024-05-09T09:16:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7100 0x55598e1d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7af0 0x55598e1d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000e330 con 0x55598e1c7100 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000f7a0 con 0x55598e1c7af0 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000ea00 con 0x55598e1ccc60 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7abc5ee740 10 client.0 _ll_register_callbacks cb 0x7f7aa80098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7ab37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7ab37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39906/0 (socket says 172.21.15.64:39906) 2024-05-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7ab37fe640 1 -- 172.21.15.64:0/1626357613 learned_addr learned my addr 172.21.15.64:0/1626357613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.463+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7af0 0x55598e1d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7100 0x55598e1d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 msgr2=0x55598e1d58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1626357613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7100 msgr2=0x55598e1d8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7100 0x55598e1d8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab27fc640 1 -- 172.21.15.64:0/1626357613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598e242030 con 0x55598e1c7af0 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7ab27fc640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7af0 0x55598e1d53b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8006fd0 tx=0x7f7aa8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0349a16ba11c61e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7a96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.464+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1626357613 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa805c070 con 0x55598e1c7af0 2024-05-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.465+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1626357613 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7aa80149c0 con 0x55598e1c7af0 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.465+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa801b030 con 0x55598e1c7af0 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.465+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1626357613 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7aa8069050 con 0x55598e1c7af0 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.465+0000 7f7abc5ee740 10 client.26852 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.465+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55598e243140 con 0x55598e1c7af0 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1626357613 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa801b030 con 0x55598e1c7af0 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1626357613 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa8060c80 con 0x55598e1c7af0 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7abc5ee740 10 client.26852 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7abc5ee740 10 client.26852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7abc5ee740 20 client.26852 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e243b20 0x55598e245f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7abc5ee740 4 client.26852 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.466+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55598e1336c0 con 0x55598e243b20 2024-05-09T09:16:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.467+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e243b20 0x55598e245f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.469+0000 7f7ab2ffd640 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e243b20 0x55598e245f40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0443bfdfe6f657b server_cookie=94f9af965509295c in_seq=0 out_seq=0 2024-05-09T09:16:24.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.469+0000 7f7a96ff5640 10 client.26852 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:24.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.469+0000 7f7a96ff5640 1 -- 172.21.15.64:0/1626357613 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55598e1336c0 con 0x55598e243b20 2024-05-09T09:16:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.469+0000 7f7a96ff5640 10 client.26852 handle_command_reply: tid=0 2024-05-09T09:16:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.469+0000 7f7abc5ee740 1 client.26852 shutdown 2024-05-09T09:16:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.470+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7af0 msgr2=0x55598e1d53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.470+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7af0 0x55598e1d53b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8006fd0 tx=0x7f7aa8004770 comp rx=0 tx=0).stop 2024-05-09T09:16:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 shutdown_connections 2024-05-09T09:16:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55598e1ccc60 0x55598e1d58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e1c7af0 0x55598e1d53b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55598e243b20 0x55598e245f40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 --2- 172.21.15.64:0/1626357613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55598e1c7100 0x55598e1d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 >> 172.21.15.64:0/1626357613 conn(0x55598e1c2770 msgr2=0x55598e1c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7a96ff5640 0 client.26852 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 shutdown_connections 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 1 -- 172.21.15.64:0/1626357613 wait complete. 2024-05-09T09:16:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.471+0000 7f7abc5ee740 20 client.26852 trim_cache size 0 max 16384 2024-05-09T09:16:24.474 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.473+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a9c0a2da0 msgr2=0x7f7a9c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.473+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a9c0a2da0 0x7f7a9c0a5260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55598e1c8350 tx=0x7f7aa8008040 comp rx=0 tx=0).stop 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.473+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 msgr2=0x7f7aac14ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.473+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 0x7f7aac14ab20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4003820 tx=0x7f7aa4003850 comp rx=0 tx=0).stop 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 shutdown_connections 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a9c0a2da0 0x7f7a9c0a5260 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0a6700 0x7f7aac14ab20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aac0a5d60 0x7f7aac143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 --2- 172.21.15.64:0/1893457987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac0a4c30 0x7f7aac143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 >> 172.21.15.64:0/1893457987 conn(0x7f7aac0a00c0 msgr2=0x7f7aac0a0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 shutdown_connections 2024-05-09T09:16:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.474+0000 7f7ab3fff640 1 -- 172.21.15.64:0/1893457987 wait complete. 2024-05-09T09:16:24.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:24 smithi064 ceph-mon[30149]: pgmap v588: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 979 op/s 2024-05-09T09:16:24.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:24 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2280778061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:16:24.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:16:24.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.693+0000 7fe259116640 1 -- 172.21.15.64:0/2041145076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254101650 msgr2=0x7fe254107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.693+0000 7fe259116640 1 --2- 172.21.15.64:0/2041145076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254101650 0x7fe254107ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009ca0 tx=0x7fe24c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.694+0000 7fe259116640 1 -- 172.21.15.64:0/2041145076 shutdown_connections 2024-05-09T09:16:24.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.694+0000 7fe259116640 1 --2- 172.21.15.64:0/2041145076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254101650 0x7fe254107ee0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.694+0000 7fe259116640 1 --2- 172.21.15.64:0/2041145076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe254100c90 0x7fe254101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.694+0000 7fe259116640 1 --2- 172.21.15.64:0/2041145076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe2540ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.694+0000 7fe259116640 1 -- 172.21.15.64:0/2041145076 >> 172.21.15.64:0/2041145076 conn(0x7fe2540fb030 msgr2=0x7fe2540fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.698+0000 7fe259116640 1 -- 172.21.15.64:0/2041145076 shutdown_connections 2024-05-09T09:16:24.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.700+0000 7fe259116640 1 -- 172.21.15.64:0/2041145076 wait complete. 2024-05-09T09:16:24.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.702+0000 7fe259116640 1 Processor -- start 2024-05-09T09:16:24.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.702+0000 7fe259116640 1 -- start start 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe259116640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe254103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe259116640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254100c90 0x7fe254102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe259116640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe254101650 0x7fe2541025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe254103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe254103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42728/0 (socket says 172.21.15.64:42728) 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe252d9d640 1 -- 172.21.15.64:0/470966943 learned_addr learned my addr 172.21.15.64:0/470966943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410b170 con 0x7fe254101650 2024-05-09T09:16:24.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410aff0 con 0x7fe2540ffaa0 2024-05-09T09:16:24.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410b2f0 con 0x7fe254100c90 2024-05-09T09:16:24.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe252d9d640 1 -- 172.21.15.64:0/470966943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254100c90 msgr2=0x7fe254102070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:16:24.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.703+0000 7fe252d9d640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254100c90 0x7fe254102070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.704+0000 7fe252d9d640 1 -- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe254101650 msgr2=0x7fe2541025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.704+0000 7fe252d9d640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe254101650 0x7fe2541025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.704+0000 7fe252d9d640 1 -- 172.21.15.64:0/470966943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe254102e40 con 0x7fe2540ffaa0 2024-05-09T09:16:24.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.704+0000 7fe252d9d640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe254103a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe2440058c0 tx=0x7fe24400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.705+0000 7fe23bfff640 1 -- 172.21.15.64:0/470966943 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24400cdb0 con 0x7fe2540ffaa0 2024-05-09T09:16:24.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.705+0000 7fe23bfff640 1 -- 172.21.15.64:0/470966943 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2440035f0 con 0x7fe2540ffaa0 2024-05-09T09:16:24.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.705+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2541a1bb0 con 0x7fe2540ffaa0 2024-05-09T09:16:24.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.705+0000 7fe23bfff640 1 -- 172.21.15.64:0/470966943 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244016630 con 0x7fe2540ffaa0 2024-05-09T09:16:24.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.705+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2541a2100 con 0x7fe2540ffaa0 2024-05-09T09:16:24.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 Processor -- start 2024-05-09T09:16:24.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 -- start start 2024-05-09T09:16:24.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3203360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c3208430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c320af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a1c3167c40 con 0x55a1c3208aa0 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a1c310f580 con 0x55a1c3202f40 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25be1c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a1c3163c00 con 0x55a1c3203930 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3203360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3203360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42732/0 (socket says 172.21.15.64:42732) 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25359e640 1 -- 172.21.15.64:0/41245320 learned_addr learned my addr 172.21.15.64:0/41245320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.709+0000 7fe25259c640 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c320af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.710+0000 7fe25359e640 1 -- 172.21.15.64:0/41245320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 msgr2=0x55a1c3208430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.710+0000 7fe25359e640 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c3208430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.710+0000 7fe25359e640 1 -- 172.21.15.64:0/41245320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 msgr2=0x55a1c320af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.710+0000 7fe25359e640 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c320af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.710+0000 7fe25359e640 1 -- 172.21.15.64:0/41245320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c2e7b3f0 con 0x55a1c3202f40 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.710+0000 7fe25359e640 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3203360 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009bf0 tx=0x7fe24c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf791c9f52aab633 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.711+0000 7fe2397fa640 1 -- 172.21.15.64:0/41245320 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c031650 con 0x55a1c3202f40 2024-05-09T09:16:24.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.711+0000 7fe2397fa640 1 -- 172.21.15.64:0/41245320 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe24c03f070 con 0x55a1c3202f40 2024-05-09T09:16:24.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.711+0000 7fe2397fa640 1 -- 172.21.15.64:0/41245320 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c004330 con 0x55a1c3202f40 2024-05-09T09:16:24.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 -- 172.21.15.64:0/41245320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 msgr2=0x55a1c3203360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3203360 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009bf0 tx=0x7fe24c009a20 comp rx=0 tx=0).stop 2024-05-09T09:16:24.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 -- 172.21.15.64:0/41245320 shutdown_connections 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c320af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c3208430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 --2- 172.21.15.64:0/41245320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3203360 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 -- 172.21.15.64:0/41245320 >> 172.21.15.64:0/41245320 conn(0x55a1c31fe5b0 msgr2=0x55a1c31fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 -- 172.21.15.64:0/41245320 shutdown_connections 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.712+0000 7fe25be1c740 1 -- 172.21.15.64:0/41245320 wait complete. 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.713+0000 7fe25be1c740 1 -- start start 2024-05-09T09:16:24.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.713+0000 7fe23bfff640 1 -- 172.21.15.64:0/470966943 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe2440037a0 con 0x7fe2540ffaa0 2024-05-09T09:16:24.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.713+0000 7fe25be1c740 1 Processor -- start 2024-05-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.713+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c320d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c320d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3214920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25be1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a1c320b9f0 con 0x55a1c3208aa0 2024-05-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25be1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a1c3122c40 con 0x55a1c3202f40 2024-05-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25be1c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a1c320bb70 con 0x55a1c3203930 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25be1c740 10 client.0 _ll_register_callbacks cb 0x55a1c2e7abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c320d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c320d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42740/0 (socket says 172.21.15.64:42740) 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25359e640 1 -- 172.21.15.64:0/3332963183 learned_addr learned my addr 172.21.15.64:0/3332963183 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe252d9d640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c320d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe25259c640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3214920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.714+0000 7fe23bfff640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe23c0a2ca0 0x7fe23c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.715+0000 7fe25359e640 1 -- 172.21.15.64:0/3332963183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 msgr2=0x55a1c320d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.715+0000 7fe25359e640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c320d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.715+0000 7fe25359e640 1 -- 172.21.15.64:0/3332963183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 msgr2=0x55a1c3214920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.715+0000 7fe25359e640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3214920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.715+0000 7fe25359e640 1 -- 172.21.15.64:0/3332963183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c2f457d0 con 0x55a1c3202f40 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.715+0000 7fe25359e640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe23c0a2ca0 0x7fe23c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.716+0000 7fe23bfff640 1 -- 172.21.15.64:0/470966943 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe2440cf280 con 0x7fe2540ffaa0 2024-05-09T09:16:24.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.716+0000 7fe25359e640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c320d3a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009ca0 tx=0x7fe24c031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f931e9d6933e141 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.716+0000 7fe236ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.717+0000 7fe236ff5640 1 -- 172.21.15.64:0/3332963183 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c004750 con 0x55a1c3202f40 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.717+0000 7fe236ff5640 1 -- 172.21.15.64:0/3332963183 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe24c030a10 con 0x55a1c3202f40 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.717+0000 7fe25359e640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe23c0a2ca0 0x7fe23c0a5160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe24c057000 tx=0x7fe24c058000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.717+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe24c03f070 con 0x55a1c3202f40 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.718+0000 7fe236ff5640 1 -- 172.21.15.64:0/3332963183 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe24c030c10 con 0x55a1c3202f40 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.718+0000 7fe25be1c740 10 client.26864 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.718+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a1c320b9f0 con 0x55a1c3202f40 2024-05-09T09:16:24.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe236ff5640 1 -- 172.21.15.64:0/3332963183 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe24c03f070 con 0x55a1c3202f40 2024-05-09T09:16:24.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe236ff5640 1 -- 172.21.15.64:0/3332963183 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe24c03b400 con 0x55a1c3202f40 2024-05-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe25be1c740 10 client.26864 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe25be1c740 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe25be1c740 20 client.26864 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3218e60 0x55a1c321b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe25be1c740 4 client.26864 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1c3175370 con 0x55a1c3218e60 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.719+0000 7fe252d9d640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3218e60 0x55a1c321b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.721+0000 7fe252d9d640 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3218e60 0x55a1c321b250 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87774488308dbdcf server_cookie=8bc93aba613ac457 in_seq=0 out_seq=0 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.721+0000 7fe236ff5640 10 client.26864 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe236ff5640 1 -- 172.21.15.64:0/3332963183 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1c3175370 con 0x55a1c3218e60 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe236ff5640 10 client.26864 handle_command_reply: tid=0 2024-05-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 client.26864 shutdown 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 msgr2=0x55a1c320d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c320d3a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009ca0 tx=0x7fe24c031680 comp rx=0 tx=0).stop 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 shutdown_connections 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3218e60 0x55a1c321b250 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3214920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3203930 0x55a1c320d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3332963183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c320d3a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 >> 172.21.15.64:0/3332963183 conn(0x55a1c31fe5b0 msgr2=0x55a1c31fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.722+0000 7fe236ff5640 0 client.26864 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.723+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 shutdown_connections 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.723+0000 7fe25be1c740 1 -- 172.21.15.64:0/3332963183 wait complete. 2024-05-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.723+0000 7fe25be1c740 20 client.26864 trim_cache size 0 max 16384 2024-05-09T09:16:24.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:24 smithi179 ceph-mon[34086]: pgmap v588: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 979 op/s 2024-05-09T09:16:24.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:24 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2280778061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 Processor -- start 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 -- start start 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3201c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3203930 0x55a1c32021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 0x55a1c32148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541aef70 con 0x55a1c3203930 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541ac7e0 con 0x55a1c3202f40 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.754+0000 7fe25be1c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541aa630 con 0x55a1c3208aa0 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 0x55a1c32148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 0x55a1c32148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37208/0 (socket says 172.21.15.64:37208) 2024-05-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 -- 172.21.15.64:0/3195632899 learned_addr learned my addr 172.21.15.64:0/3195632899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe252d9d640 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3201c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25259c640 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3203930 0x55a1c32021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 -- 172.21.15.64:0/3195632899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 msgr2=0x55a1c3201c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3201c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 -- 172.21.15.64:0/3195632899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3203930 msgr2=0x55a1c32021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3203930 0x55a1c32021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.755+0000 7fe25359e640 1 -- 172.21.15.64:0/3195632899 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c321b5c0 con 0x55a1c3208aa0 2024-05-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.758+0000 7fe25359e640 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 0x55a1c32148e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0318d0 tx=0x7fe24c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9bbc219a49ce9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.759+0000 7fe238ff9640 1 -- 172.21.15.64:0/3195632899 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c004750 con 0x55a1c3208aa0 2024-05-09T09:16:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.759+0000 7fe238ff9640 1 -- 172.21.15.64:0/3195632899 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe24c03f040 con 0x55a1c3208aa0 2024-05-09T09:16:24.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.759+0000 7fe238ff9640 1 -- 172.21.15.64:0/3195632899 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c030ed0 con 0x55a1c3208aa0 2024-05-09T09:16:24.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.760+0000 7fe25be1c740 1 -- 172.21.15.64:0/3195632899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 msgr2=0x55a1c32148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.760+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 0x55a1c32148e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0318d0 tx=0x7fe24c009a20 comp rx=0 tx=0).stop 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 -- 172.21.15.64:0/3195632899 shutdown_connections 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3208aa0 0x55a1c32148e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3203930 0x55a1c32021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 --2- 172.21.15.64:0/3195632899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3202f40 0x55a1c3201c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 -- 172.21.15.64:0/3195632899 >> 172.21.15.64:0/3195632899 conn(0x55a1c31fe5b0 msgr2=0x55a1c3201560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 -- 172.21.15.64:0/3195632899 shutdown_connections 2024-05-09T09:16:24.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 -- 172.21.15.64:0/3195632899 wait complete. 2024-05-09T09:16:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.764+0000 7fe25be1c740 1 -- start start 2024-05-09T09:16:24.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.765+0000 7fe25be1c740 1 Processor -- start 2024-05-09T09:16:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.765+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 0x55a1c32125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.765+0000 7fe25be1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3203930 0x55a1c3212b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 0x55a1c32125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 0x55a1c32125c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37220/0 (socket says 172.21.15.64:37220) 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe252d9d640 1 -- 172.21.15.64:0/4196794084 learned_addr learned my addr 172.21.15.64:0/4196794084 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3213ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541aaca0 con 0x55a1c3208aa0 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541ac110 con 0x55a1c3203930 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe25359e640 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3213ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541ab370 con 0x55a1c3202f40 2024-05-09T09:16:24.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.766+0000 7fe25be1c740 10 client.0 _ll_register_callbacks cb 0x7fe244005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.767+0000 7fe252d9d640 1 -- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3203930 msgr2=0x55a1c3212b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.767+0000 7fe252d9d640 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3203930 0x55a1c3212b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.767+0000 7fe252d9d640 1 -- 172.21.15.64:0/4196794084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 msgr2=0x55a1c3213ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.767+0000 7fe252d9d640 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3213ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.767+0000 7fe252d9d640 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c32140b0 con 0x55a1c3202f40 2024-05-09T09:16:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.768+0000 7fe252d9d640 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 0x55a1c32125c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe24409b3f0 tx=0x7fe2440cc860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33c8d7cabea501e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.768+0000 7fe236ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe236ff5640 1 -- 172.21.15.64:0/4196794084 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24409b6f0 con 0x55a1c3202f40 2024-05-09T09:16:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe236ff5640 1 -- 172.21.15.64:0/4196794084 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe244097d60 con 0x55a1c3202f40 2024-05-09T09:16:24.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe24c03f040 con 0x55a1c3202f40 2024-05-09T09:16:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe236ff5640 1 -- 172.21.15.64:0/4196794084 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe24409faa0 con 0x55a1c3202f40 2024-05-09T09:16:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe25be1c740 10 client.27034 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a1c32881a0 con 0x55a1c3202f40 2024-05-09T09:16:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.769+0000 7fe236ff5640 1 -- 172.21.15.64:0/4196794084 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe24c03f040 con 0x55a1c3202f40 2024-05-09T09:16:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe236ff5640 1 -- 172.21.15.64:0/4196794084 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe244015030 con 0x55a1c3202f40 2024-05-09T09:16:24.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25be1c740 10 client.27034 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25be1c740 10 client.27034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25be1c740 20 client.27034 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3288b50 0x55a1c328af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25be1c740 4 client.27034 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:16:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1c316f6c0 con 0x55a1c3288b50 2024-05-09T09:16:24.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.770+0000 7fe25259c640 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3288b50 0x55a1c328af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.771+0000 7fe236ff5640 10 client.27034 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.771+0000 7fe25259c640 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3288b50 0x55a1c328af70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37844de6401c3460 server_cookie=d604d91ad882fed3 in_seq=0 out_seq=0 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.771+0000 7fe236ff5640 1 -- 172.21.15.64:0/4196794084 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1c316f6c0 con 0x55a1c3288b50 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.771+0000 7fe236ff5640 10 client.27034 handle_command_reply: tid=0 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.771+0000 7fe25be1c740 1 client.27034 shutdown 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.771+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 msgr2=0x55a1c32125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 0x55a1c32125c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe24409b3f0 tx=0x7fe2440cc860 comp rx=0 tx=0).stop 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 shutdown_connections 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a1c3288b50 0x55a1c328af70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a1c3208aa0 0x55a1c3213ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe236ff5640 0 client.27034 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a1c3203930 0x55a1c3212b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 --2- 172.21.15.64:0/4196794084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1c3202f40 0x55a1c32125c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 >> 172.21.15.64:0/4196794084 conn(0x55a1c31fe5b0 msgr2=0x55a1c3201df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.772+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 shutdown_connections 2024-05-09T09:16:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.773+0000 7fe25be1c740 1 -- 172.21.15.64:0/4196794084 wait complete. 2024-05-09T09:16:24.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.773+0000 7fe25be1c740 20 client.27034 trim_cache size 0 max 16384 2024-05-09T09:16:24.774 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe23c0a2ca0 msgr2=0x7fe23c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe23c0a2ca0 0x7fe23c0a5160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe24c057000 tx=0x7fe24c058000 comp rx=0 tx=0).stop 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 msgr2=0x7fe254103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe254103a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe2440058c0 tx=0x7fe24400eea0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 shutdown_connections 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe23c0a2ca0 0x7fe23c0a5160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe254101650 0x7fe2541025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe254100c90 0x7fe254102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 --2- 172.21.15.64:0/470966943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2540ffaa0 0x7fe254103a30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:24.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.774+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 >> 172.21.15.64:0/470966943 conn(0x7fe2540fb030 msgr2=0x7fe2541061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:24.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.776+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 shutdown_connections 2024-05-09T09:16:24.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:24.776+0000 7fe259116640 1 -- 172.21.15.64:0/470966943 wait complete. 2024-05-09T09:16:24.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:24 smithi139 ceph-mon[33599]: pgmap v588: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 979 op/s 2024-05-09T09:16:24.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:24 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2280778061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:16:25.802 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:16:25.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:16:25.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:16:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 -- 172.21.15.64:0/3493189409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31041099a0 msgr2=0x7f31041102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 --2- 172.21.15.64:0/3493189409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31041099a0 0x7f31041102c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009be0 tx=0x7f30fc02a6c0 comp rx=0 tx=0).stop 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 -- 172.21.15.64:0/3493189409 shutdown_connections 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 --2- 172.21.15.64:0/3493189409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31041099a0 0x7f31041102c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 --2- 172.21.15.64:0/3493189409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104108fe0 0x7f3104109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 --2- 172.21.15.64:0/3493189409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104107e90 0x7f31041082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.988+0000 7f310914f640 1 -- 172.21.15.64:0/3493189409 >> 172.21.15.64:0/3493189409 conn(0x7f3104076e30 msgr2=0x7f3104079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.989+0000 7f310914f640 1 -- 172.21.15.64:0/3493189409 shutdown_connections 2024-05-09T09:16:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.989+0000 7f310914f640 1 -- 172.21.15.64:0/3493189409 wait complete. 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 Processor -- start 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 -- start start 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104107e90 0x7f310419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3104108fe0 0x7f310419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041099a0 0x7f31041a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31041139f0 con 0x7f3104107e90 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3104113870 con 0x7f3104108fe0 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310914f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3104113b70 con 0x7f31041099a0 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104107e90 0x7f310419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104107e90 0x7f310419e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41216/0 (socket says 172.21.15.64:41216) 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f3102d9d640 1 -- 172.21.15.64:0/1736152134 learned_addr learned my addr 172.21.15.64:0/1736152134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.990+0000 7f310359e640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041099a0 0x7f31041a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f30fbfff640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3104108fe0 0x7f310419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f310359e640 1 -- 172.21.15.64:0/1736152134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3104108fe0 msgr2=0x7f310419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f310359e640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3104108fe0 0x7f310419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f310359e640 1 -- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104107e90 msgr2=0x7f310419e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f310359e640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104107e90 0x7f310419e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f310359e640 1 -- 172.21.15.64:0/1736152134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041a5c90 con 0x7f31041099a0 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f30fbfff640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3104108fe0 0x7f310419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.991+0000 7f310359e640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041099a0 0x7f31041a5620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30fc034040 tx=0x7f30fc030f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.992+0000 7f3100d99640 1 -- 172.21.15.64:0/1736152134 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc041020 con 0x7f31041099a0 2024-05-09T09:16:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.992+0000 7f3100d99640 1 -- 172.21.15.64:0/1736152134 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30fc037040 con 0x7f31041099a0 2024-05-09T09:16:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.992+0000 7f3100d99640 1 -- 172.21.15.64:0/1736152134 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc036070 con 0x7f31041099a0 2024-05-09T09:16:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.992+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041a5f20 con 0x7f31041099a0 2024-05-09T09:16:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.992+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a62d0 con 0x7f31041099a0 2024-05-09T09:16:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.996+0000 7f310be5c740 1 Processor -- start 2024-05-09T09:16:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.997+0000 7f3100d99640 1 -- 172.21.15.64:0/1736152134 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f30fc044020 con 0x7f31041099a0 2024-05-09T09:16:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.998+0000 7f310be5c740 1 -- start start 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f310be5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f310be5c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f310be5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f310be5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d80441990 con 0x561d804dd4c0 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f310be5c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d803e95b0 con 0x561d804ddeb0 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f310be5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561d8043d950 con 0x561d804e3020 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:25.999+0000 7f30fbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.000+0000 7f30fbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804e54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37262/0 (socket says 172.21.15.64:37262) 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.000+0000 7f30fbfff640 1 -- 172.21.15.64:0/735497182 learned_addr learned my addr 172.21.15.64:0/735497182 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.000+0000 7f30fbfff640 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.000+0000 7f3102d9d640 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f3100d99640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f30f00a2c90 0x7f30f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f30fbfff640 1 -- 172.21.15.64:0/735497182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 msgr2=0x561d804e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f30fbfff640 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f30fbfff640 1 -- 172.21.15.64:0/735497182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 msgr2=0x561d804dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f30fbfff640 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f30fbfff640 1 -- 172.21.15.64:0/735497182 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8021d7e0 con 0x561d804e3020 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f3100d99640 1 -- 172.21.15.64:0/1736152134 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30fc0efdd0 con 0x7f31041099a0 2024-05-09T09:16:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.001+0000 7f30fbfff640 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804e54b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f30f40094d0 tx=0x7f30f40097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1571395525856f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.002+0000 7f310359e640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f30f00a2c90 0x7f30f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.002+0000 7f30f97fa640 1 -- 172.21.15.64:0/735497182 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4004030 con 0x561d804e3020 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.002+0000 7f30f97fa640 1 -- 172.21.15.64:0/735497182 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f40107d0 con 0x561d804e3020 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f30f97fa640 1 -- 172.21.15.64:0/735497182 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4003850 con 0x561d804e3020 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 -- 172.21.15.64:0/735497182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 msgr2=0x561d804e54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804e54b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f30f40094d0 tx=0x7f30f40097c0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310359e640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f30f00a2c90 0x7f30f00a5150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0bbf40 tx=0x7f30fc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 -- 172.21.15.64:0/735497182 shutdown_connections 2024-05-09T09:16:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804e54b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 --2- 172.21.15.64:0/735497182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 -- 172.21.15.64:0/735497182 >> 172.21.15.64:0/735497182 conn(0x561d804a50a0 msgr2=0x561d804d8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 -- 172.21.15.64:0/735497182 shutdown_connections 2024-05-09T09:16:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.003+0000 7f310be5c740 1 -- 172.21.15.64:0/735497182 wait complete. 2024-05-09T09:16:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 -- start start 2024-05-09T09:16:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 Processor -- start 2024-05-09T09:16:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804dd4c0 0x561d804e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 0x561d804e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804e3020 0x561d804eef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d804e6020 con 0x561d804ddeb0 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d803fcb40 con 0x561d804e3020 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561d804e61a0 con 0x561d804dd4c0 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.004+0000 7f310be5c740 10 client.0 _ll_register_callbacks cb 0x561d803f10f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 0x561d804e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 0x561d804e7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41246/0 (socket says 172.21.15.64:41246) 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 -- 172.21.15.64:0/764031315 learned_addr learned my addr 172.21.15.64:0/764031315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f30fbfff640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804dd4c0 0x561d804e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f30fbfff640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804e3020 0x561d804eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 -- 172.21.15.64:0/764031315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804dd4c0 msgr2=0x561d804e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804dd4c0 0x561d804e79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 -- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804e3020 msgr2=0x561d804eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804e3020 0x561d804eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.005+0000 7f3102d9d640 1 -- 172.21.15.64:0/764031315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8027df70 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.006+0000 7f3102d9d640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 0x561d804e7ef0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0127d0 tx=0x7f30ec012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0cb26f78fb043b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.006+0000 7f30ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.006+0000 7f30ea7fc640 1 -- 172.21.15.64:0/764031315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00f8c0 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.006+0000 7f30ea7fc640 1 -- 172.21.15.64:0/764031315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30ec00fa60 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.006+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f40107d0 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.006+0000 7f30ea7fc640 1 -- 172.21.15.64:0/764031315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f30ec00fc60 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 10 client.17394 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561d80115170 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f30ea7fc640 1 -- 172.21.15.64:0/764031315 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f40107d0 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f30ea7fc640 1 -- 172.21.15.64:0/764031315 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f30ec010430 con 0x561d804ddeb0 2024-05-09T09:16:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 10 client.17394 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 20 client.17394 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d804f3530 0x561d804f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 4 client.17394 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.007+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d8044f0c0 con 0x561d804f3530 2024-05-09T09:16:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.008+0000 7f30fbfff640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d804f3530 0x561d804f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.008+0000 7f30fbfff640 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d804f3530 0x561d804f5990 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8015909c123f83 server_cookie=c3944ae2eb32e613 in_seq=0 out_seq=0 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.009+0000 7f30ea7fc640 10 client.17394 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.009+0000 7f30ea7fc640 1 -- 172.21.15.64:0/764031315 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d8044f0c0 con 0x561d804f3530 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.009+0000 7f30ea7fc640 10 client.17394 handle_command_reply: tid=0 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.009+0000 7f310be5c740 1 client.17394 shutdown 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 msgr2=0x561d804e7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 0x561d804e7ef0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0127d0 tx=0x7f30ec012c90 comp rx=0 tx=0).stop 2024-05-09T09:16:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 shutdown_connections 2024-05-09T09:16:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d804f3530 0x561d804f5990 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804e3020 0x561d804eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804ddeb0 0x561d804e7ef0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 --2- 172.21.15.64:0/764031315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804dd4c0 0x561d804e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 >> 172.21.15.64:0/764031315 conn(0x561d804a50a0 msgr2=0x561d804d8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f30ea7fc640 0 client.17394 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 shutdown_connections 2024-05-09T09:16:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.010+0000 7f310be5c740 1 -- 172.21.15.64:0/764031315 wait complete. 2024-05-09T09:16:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.011+0000 7f310be5c740 20 client.17394 trim_cache size 0 max 16384 2024-05-09T09:16:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.049+0000 7f310be5c740 1 Processor -- start 2024-05-09T09:16:26.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.049+0000 7f310be5c740 1 -- start start 2024-05-09T09:16:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f310be5c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804dd4c0 0x561d804df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f310be5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 0x561d804df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f310be5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804e3020 0x561d804eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f310be5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31041b32f0 con 0x561d804e3020 2024-05-09T09:16:26.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f310be5c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f31041b0b60 con 0x561d804dd4c0 2024-05-09T09:16:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f310be5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f31041ae9b0 con 0x561d804ddeb0 2024-05-09T09:16:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 0x561d804df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 0x561d804df7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37278/0 (socket says 172.21.15.64:37278) 2024-05-09T09:16:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f3102d9d640 1 -- 172.21.15.64:0/3588131555 learned_addr learned my addr 172.21.15.64:0/3588131555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.050+0000 7f30fbfff640 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804dd4c0 0x561d804df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.051+0000 7f3102d9d640 1 -- 172.21.15.64:0/3588131555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804dd4c0 msgr2=0x561d804df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.051+0000 7f3102d9d640 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804dd4c0 0x561d804df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.051+0000 7f3102d9d640 1 -- 172.21.15.64:0/3588131555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804e3020 msgr2=0x561d804eeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.051+0000 7f3102d9d640 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804e3020 0x561d804eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.051+0000 7f3102d9d640 1 -- 172.21.15.64:0/3588131555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d804de290 con 0x561d804ddeb0 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.051+0000 7f3102d9d640 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 0x561d804df7f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009640 tx=0x7f30ec0059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474b383099805f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.052+0000 7f30f8ff9640 1 -- 172.21.15.64:0/3588131555 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec0122e0 con 0x561d804ddeb0 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.052+0000 7f30f8ff9640 1 -- 172.21.15.64:0/3588131555 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ec02a070 con 0x561d804ddeb0 2024-05-09T09:16:26.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.052+0000 7f30f8ff9640 1 -- 172.21.15.64:0/3588131555 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec01c760 con 0x561d804ddeb0 2024-05-09T09:16:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.053+0000 7f310be5c740 1 -- 172.21.15.64:0/3588131555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 msgr2=0x561d804df7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.053+0000 7f310be5c740 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 0x561d804df7f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009640 tx=0x7f30ec0059b0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 -- 172.21.15.64:0/3588131555 shutdown_connections 2024-05-09T09:16:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804e3020 0x561d804eeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804ddeb0 0x561d804df7f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 --2- 172.21.15.64:0/3588131555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804dd4c0 0x561d804df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 -- 172.21.15.64:0/3588131555 >> 172.21.15.64:0/3588131555 conn(0x561d804a50a0 msgr2=0x561d804e79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 -- 172.21.15.64:0/3588131555 shutdown_connections 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.054+0000 7f310be5c740 1 -- 172.21.15.64:0/3588131555 wait complete. 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.055+0000 7f310be5c740 1 -- start start 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.055+0000 7f310be5c740 1 Processor -- start 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.055+0000 7f310be5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.055+0000 7f310be5c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.055+0000 7f310be5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804ed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.055+0000 7f310be5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31041af020 con 0x561d804dd4c0 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f30fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f30fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804ec3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41252/0 (socket says 172.21.15.64:41252) 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f30fbfff640 1 -- 172.21.15.64:0/3016577436 learned_addr learned my addr 172.21.15.64:0/3016577436 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f31041b0490 con 0x561d804ddeb0 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f31041af6f0 con 0x561d804e3020 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f310be5c740 10 client.0 _ll_register_callbacks cb 0x7f30ec0098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.056+0000 7f30fbfff640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f3102d9d640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30fbfff640 1 -- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 msgr2=0x561d804ec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30fbfff640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30fbfff640 1 -- 172.21.15.64:0/3016577436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 msgr2=0x561d804ec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30fbfff640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804ec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30fbfff640 1 -- 172.21.15.64:0/3016577436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d804ede30 con 0x561d804e3020 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30fbfff640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804ed840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f30f40092a0 tx=0x7f30ec009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46b5077793279e2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30ea7fc640 1 -- 172.21.15.64:0/3016577436 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4027040 con 0x561d804e3020 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.057+0000 7f30ea7fc640 1 -- 172.21.15.64:0/3016577436 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f30f4010c40 con 0x561d804e3020 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.058+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ec02a070 con 0x561d804e3020 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.058+0000 7f30ea7fc640 1 -- 172.21.15.64:0/3016577436 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f30f40107a0 con 0x561d804e3020 2024-05-09T09:16:26.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.058+0000 7f310be5c740 10 client.27052 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:26.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.058+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561d804ef170 con 0x561d804e3020 2024-05-09T09:16:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.061+0000 7f30ea7fc640 1 -- 172.21.15.64:0/3016577436 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ec02a070 con 0x561d804e3020 2024-05-09T09:16:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.061+0000 7f30ea7fc640 1 -- 172.21.15.64:0/3016577436 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f30f4003ae0 con 0x561d804e3020 2024-05-09T09:16:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.062+0000 7f310be5c740 10 client.27052 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:26.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.062+0000 7f310be5c740 10 client.27052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.062+0000 7f310be5c740 20 client.27052 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.062+0000 7f310be5c740 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d8056b090 0x561d8056d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.062+0000 7f310be5c740 4 client.27052 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:16:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.062+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d80449410 con 0x561d8056b090 2024-05-09T09:16:26.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.063+0000 7f3102d9d640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d8056b090 0x561d8056d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.065+0000 7f3102d9d640 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d8056b090 0x561d8056d4b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7cc8c262cee2ba server_cookie=da4b66812f435657 in_seq=0 out_seq=0 2024-05-09T09:16:26.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.065+0000 7f30ea7fc640 10 client.27052 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.066+0000 7f30ea7fc640 1 -- 172.21.15.64:0/3016577436 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d80449410 con 0x561d8056b090 2024-05-09T09:16:26.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.066+0000 7f30ea7fc640 10 client.27052 handle_command_reply: tid=0 2024-05-09T09:16:26.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.066+0000 7f310be5c740 1 client.27052 shutdown 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.074+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 msgr2=0x561d804ed840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.074+0000 7f310be5c740 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804ed840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f30f40092a0 tx=0x7f30ec009f00 comp rx=0 tx=0).stop 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.074+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 shutdown_connections 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.074+0000 7f310be5c740 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561d8056b090 0x561d8056d4b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.074+0000 7f310be5c740 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d804e3020 0x561d804ed840 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f30f40092a0 tx=0x7f30ec009f00 comp rx=0 tx=0).stop 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.074+0000 7f30ea7fc640 0 client.27052 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.075+0000 7f310be5c740 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d804ddeb0 0x561d804ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.075+0000 7f310be5c740 1 --2- 172.21.15.64:0/3016577436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d804dd4c0 0x561d804ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.075+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 >> 172.21.15.64:0/3016577436 conn(0x561d804a50a0 msgr2=0x561d804e78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.077+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 shutdown_connections 2024-05-09T09:16:26.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.077+0000 7f310be5c740 1 -- 172.21.15.64:0/3016577436 wait complete. 2024-05-09T09:16:26.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.077+0000 7f310be5c740 20 client.27052 trim_cache size 0 max 16384 2024-05-09T09:16:26.078 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:16:26.078 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:16:26.078 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "b0ffde3c-bc8c-4c8b-831a-61cb4c59705a", 2024-05-09T09:16:26.078 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:16:26.078 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:16:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f30f00a2c90 msgr2=0x7f30f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f30f00a2c90 0x7f30f00a5150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0bbf40 tx=0x7f30fc009500 comp rx=0 tx=0).stop 2024-05-09T09:16:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041099a0 msgr2=0x7f31041a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041099a0 0x7f31041a5620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30fc034040 tx=0x7f30fc030f10 comp rx=0 tx=0).stop 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 shutdown_connections 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f30f00a2c90 0x7f30f00a5150 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041099a0 0x7f31041a5620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3104108fe0 0x7f310419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 --2- 172.21.15.64:0/1736152134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3104107e90 0x7f310419e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 >> 172.21.15.64:0/1736152134 conn(0x7f3104076e30 msgr2=0x7f310410c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 shutdown_connections 2024-05-09T09:16:26.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.079+0000 7f310914f640 1 -- 172.21.15.64:0/1736152134 wait complete. 2024-05-09T09:16:26.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:16:26.101 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 -- 172.21.15.64:0/3234646429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200a62b0 msgr2=0x7fd1200a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 --2- 172.21.15.64:0/3234646429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200a62b0 0x7fd1200a6730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd1240074e0 tx=0x7fd12402b060 comp rx=0 tx=0).stop 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 -- 172.21.15.64:0/3234646429 shutdown_connections 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 --2- 172.21.15.64:0/3234646429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1200aafb0 0x7fd1200ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 --2- 172.21.15.64:0/3234646429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200a62b0 0x7fd1200a6730 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 --2- 172.21.15.64:0/3234646429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1200a4fd0 0x7fd1200a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.295+0000 7fd12e55f640 1 -- 172.21.15.64:0/3234646429 >> 172.21.15.64:0/3234646429 conn(0x7fd1200a00d0 msgr2=0x7fd1200a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.296+0000 7fd12e55f640 1 -- 172.21.15.64:0/3234646429 shutdown_connections 2024-05-09T09:16:26.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.296+0000 7fd12e55f640 1 -- 172.21.15.64:0/3234646429 wait complete. 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.297+0000 7fd12e55f640 1 Processor -- start 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.297+0000 7fd12e55f640 1 -- start start 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.297+0000 7fd12e55f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1200a4fd0 0x7fd12013f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.297+0000 7fd12e55f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1200a62b0 0x7fd12013f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.297+0000 7fd12e55f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 0x7fd1201466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.297+0000 7fd12e55f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200b0730 con 0x7fd1200a62b0 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12e55f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200b05b0 con 0x7fd1200a4fd0 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12e55f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200b08b0 con 0x7fd1200aafb0 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 0x7fd1201466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 0x7fd1201466d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37290/0 (socket says 172.21.15.64:37290) 2024-05-09T09:16:26.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 -- 172.21.15.64:0/1894770812 learned_addr learned my addr 172.21.15.64:0/1894770812 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12d55d640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1200a4fd0 0x7fd12013f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 -- 172.21.15.64:0/1894770812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1200a4fd0 msgr2=0x7fd12013f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1200a4fd0 0x7fd12013f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 -- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1200a62b0 msgr2=0x7fd12013f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1200a62b0 0x7fd12013f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.298+0000 7fd12dd5e640 1 -- 172.21.15.64:0/1894770812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd120146d40 con 0x7fd1200aafb0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.299+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 0x7fd1201466d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd1280673a0 tx=0x7fd128070990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.299+0000 7fd11e7fc640 1 -- 172.21.15.64:0/1894770812 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128078070 con 0x7fd1200aafb0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.299+0000 7fd11e7fc640 1 -- 172.21.15.64:0/1894770812 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd128074cf0 con 0x7fd1200aafb0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.299+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd120147030 con 0x7fd1200aafb0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.299+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd120147500 con 0x7fd1200aafb0 2024-05-09T09:16:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.299+0000 7fd11e7fc640 1 -- 172.21.15.64:0/1894770812 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128069a80 con 0x7fd1200aafb0 2024-05-09T09:16:26.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.303+0000 7fd11e7fc640 1 -- 172.21.15.64:0/1894770812 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd12807b070 con 0x7fd1200aafb0 2024-05-09T09:16:26.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.304+0000 7fd11e7fc640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1140a2d10 0x7fd1140a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.305+0000 7fd11e7fc640 1 -- 172.21.15.64:0/1894770812 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd128132e70 con 0x7fd1200aafb0 2024-05-09T09:16:26.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.305+0000 7fd12d55d640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1140a2d10 0x7fd1140a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 Processor -- start 2024-05-09T09:16:26.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 -- start start 2024-05-09T09:16:26.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd12d55d640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1140a2d10 0x7fd1140a51d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd1180058c0 tx=0x7fd118005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586dde540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586de3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586de6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d586d42d00 con 0x55d586dde120 2024-05-09T09:16:26.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d586cea4c0 con 0x55d586de3c80 2024-05-09T09:16:26.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd1325ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d586d3ecc0 con 0x55d586ddeb10 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd12d55d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586dde540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd12d55d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586dde540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41298/0 (socket says 172.21.15.64:41298) 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd12d55d640 1 -- 172.21.15.64:0/1092019943 learned_addr learned my addr 172.21.15.64:0/1092019943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.306+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586de6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.307+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586de3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.307+0000 7fd12cd5c640 1 -- 172.21.15.64:0/1092019943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 msgr2=0x55d586de6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.307+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586de6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.307+0000 7fd12cd5c640 1 -- 172.21.15.64:0/1092019943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 msgr2=0x55d586dde540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.307+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586dde540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.307+0000 7fd12cd5c640 1 -- 172.21.15.64:0/1092019943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d586b207d0 con 0x55d586ddeb10 2024-05-09T09:16:26.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.308+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586de3610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd120140630 tx=0x7fd124000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba7493178d1d79a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.308+0000 7fd1137fe640 1 -- 172.21.15.64:0/1092019943 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124007c50 con 0x55d586ddeb10 2024-05-09T09:16:26.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.309+0000 7fd1137fe640 1 -- 172.21.15.64:0/1092019943 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd124043070 con 0x55d586ddeb10 2024-05-09T09:16:26.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.310+0000 7fd1325ab740 1 -- 172.21.15.64:0/1092019943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 msgr2=0x55d586de3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.310+0000 7fd1325ab740 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586de3610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd120140630 tx=0x7fd124000bc0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.311+0000 7fd1325ab740 1 -- 172.21.15.64:0/1092019943 shutdown_connections 2024-05-09T09:16:26.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.311+0000 7fd1325ab740 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586de6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.311+0000 7fd1325ab740 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586de3610 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.311+0000 7fd1325ab740 1 --2- 172.21.15.64:0/1092019943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586dde540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.311+0000 7fd1325ab740 1 -- 172.21.15.64:0/1092019943 >> 172.21.15.64:0/1092019943 conn(0x55d586dd9790 msgr2=0x55d586dd9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.314+0000 7fd1325ab740 1 -- 172.21.15.64:0/1092019943 shutdown_connections 2024-05-09T09:16:26.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.314+0000 7fd1325ab740 1 -- 172.21.15.64:0/1092019943 wait complete. 2024-05-09T09:16:26.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.315+0000 7fd1325ab740 1 -- start start 2024-05-09T09:16:26.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.315+0000 7fd1325ab740 1 Processor -- start 2024-05-09T09:16:26.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.315+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e0a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.315+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586e118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd1325ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d586e08c30 con 0x55d586de3c80 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd12cd5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd12cd5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e0a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42856/0 (socket says 172.21.15.64:42856) 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd12cd5c640 1 -- 172.21.15.64:0/205922662 learned_addr learned my addr 172.21.15.64:0/205922662 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d586cfdb90 con 0x55d586ddeb10 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d586e08db0 con 0x55d586dde120 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd1325ab740 10 client.0 _ll_register_callbacks cb 0x55d586d39da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586e118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.316+0000 7fd12d55d640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e0a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12cd5c640 1 -- 172.21.15.64:0/205922662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 msgr2=0x55d586e0a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e0a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12cd5c640 1 -- 172.21.15.64:0/205922662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 msgr2=0x55d586e118e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586e118e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12cd5c640 1 -- 172.21.15.64:0/205922662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d586a563f0 con 0x55d586ddeb10 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12d55d640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e0a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e0a8a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd124046000 tx=0x7fd12400a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac4c8f36fb81497 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.317+0000 7fd110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.318+0000 7fd110ff9640 1 -- 172.21.15.64:0/205922662 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124038040 con 0x55d586ddeb10 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.318+0000 7fd110ff9640 1 -- 172.21.15.64:0/205922662 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd12400a450 con 0x55d586ddeb10 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.318+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd124043070 con 0x55d586ddeb10 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.318+0000 7fd110ff9640 1 -- 172.21.15.64:0/205922662 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd12404d050 con 0x55d586ddeb10 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.318+0000 7fd1325ab740 10 client.26891 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:26.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.318+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d586a160f0 con 0x55d586ddeb10 2024-05-09T09:16:26.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.319+0000 7fd110ff9640 1 -- 172.21.15.64:0/205922662 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd124008040 con 0x55d586ddeb10 2024-05-09T09:16:26.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.319+0000 7fd110ff9640 1 -- 172.21.15.64:0/205922662 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd12403b070 con 0x55d586ddeb10 2024-05-09T09:16:26.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.320+0000 7fd1325ab740 10 client.26891 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:26.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.320+0000 7fd1325ab740 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:26.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.320+0000 7fd1325ab740 20 client.26891 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:26.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.320+0000 7fd1325ab740 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.320+0000 7fd1325ab740 4 client.26891 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:26.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.320+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd124043070 con 0x55d586e15dd0 2024-05-09T09:16:26.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.322+0000 7fd12d55d640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.325+0000 7fd110ff9640 10 client.26891 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.325+0000 7fd12d55d640 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e181c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3021e7674707d3 server_cookie=51e3917a1f3ad5dc in_seq=0 out_seq=0 2024-05-09T09:16:26.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd110ff9640 1 -- 172.21.15.64:0/205922662 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fd124043070 con 0x55d586e15dd0 2024-05-09T09:16:26.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd110ff9640 10 client.26891 handle_command_reply: tid=0 2024-05-09T09:16:26.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 client.26891 shutdown 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 msgr2=0x55d586e0a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e0a8a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd124046000 tx=0x7fd12400a2e0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 shutdown_connections 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e181c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586e118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd110ff9640 0 client.26891 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e0a8a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.326+0000 7fd1325ab740 1 --2- 172.21.15.64:0/205922662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e0a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.327+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 >> 172.21.15.64:0/205922662 conn(0x55d586dd9790 msgr2=0x55d586dd8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.327+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 shutdown_connections 2024-05-09T09:16:26.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.330+0000 7fd1325ab740 1 -- 172.21.15.64:0/205922662 wait complete. 2024-05-09T09:16:26.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.330+0000 7fd1325ab740 20 client.26891 trim_cache size 0 max 16384 2024-05-09T09:16:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 365.0 s, 837.6 tps, lat 118.725 ms stddev 56.009 2024-05-09T09:16:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.376+0000 7fd1325ab740 1 Processor -- start 2024-05-09T09:16:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 -- start start 2024-05-09T09:16:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586dd81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c023630 con 0x55d586de3c80 2024-05-09T09:16:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c020ea0 con 0x55d586ddeb10 2024-05-09T09:16:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd1325ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c01ecf0 con 0x55d586dde120 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd12dd5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586dd81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd12dd5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586dd81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41306/0 (socket says 172.21.15.64:41306) 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.377+0000 7fd12dd5e640 1 -- 172.21.15.64:0/168645490 learned_addr learned my addr 172.21.15.64:0/168645490 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.378+0000 7fd12dd5e640 1 -- 172.21.15.64:0/168645490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 msgr2=0x55d586e09d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.378+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e09d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.378+0000 7fd12dd5e640 1 -- 172.21.15.64:0/168645490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 msgr2=0x55d586e118a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.378+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e118a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.378+0000 7fd12dd5e640 1 -- 172.21.15.64:0/168645490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d586e08b20 con 0x55d586de3c80 2024-05-09T09:16:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.378+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586dd81a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd1280503a0 tx=0x7fd128067600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff8193e72a30c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.379+0000 7fd112ffd640 1 -- 172.21.15.64:0/168645490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128082040 con 0x55d586de3c80 2024-05-09T09:16:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.379+0000 7fd112ffd640 1 -- 172.21.15.64:0/168645490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd128107fb0 con 0x55d586de3c80 2024-05-09T09:16:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.379+0000 7fd112ffd640 1 -- 172.21.15.64:0/168645490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280fea80 con 0x55d586de3c80 2024-05-09T09:16:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.380+0000 7fd1325ab740 1 -- 172.21.15.64:0/168645490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 msgr2=0x55d586dd81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.380+0000 7fd1325ab740 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586dd81a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd1280503a0 tx=0x7fd128067600 comp rx=0 tx=0).stop 2024-05-09T09:16:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 -- 172.21.15.64:0/168645490 shutdown_connections 2024-05-09T09:16:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586de3c80 0x55d586dd81a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586ddeb10 0x55d586e118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 --2- 172.21.15.64:0/168645490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586dde120 0x55d586e09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 -- 172.21.15.64:0/168645490 >> 172.21.15.64:0/168645490 conn(0x55d586dd9790 msgr2=0x55d586e09650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 -- 172.21.15.64:0/168645490 shutdown_connections 2024-05-09T09:16:26.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.381+0000 7fd1325ab740 1 -- 172.21.15.64:0/168645490 wait complete. 2024-05-09T09:16:26.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.382+0000 7fd1325ab740 1 -- start start 2024-05-09T09:16:26.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.382+0000 7fd1325ab740 1 Processor -- start 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586e0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586e0fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586e10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c01f360 con 0x55d586dde120 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c0207d0 con 0x55d586de3c80 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c01fa30 con 0x55d586ddeb10 2024-05-09T09:16:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd1325ab740 10 client.0 _ll_register_callbacks cb 0x7fd118014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:26.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd12d55d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586e0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd12d55d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586e0f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41318/0 (socket says 172.21.15.64:41318) 2024-05-09T09:16:26.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd12d55d640 1 -- 172.21.15.64:0/4263442390 learned_addr learned my addr 172.21.15.64:0/4263442390 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:26.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.383+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586e10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12cd5c640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586e0fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12dd5e640 1 -- 172.21.15.64:0/4263442390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 msgr2=0x55d586e0fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586e0fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12dd5e640 1 -- 172.21.15.64:0/4263442390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 msgr2=0x55d586e0f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586e0f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12dd5e640 1 -- 172.21.15.64:0/4263442390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d586e11060 con 0x55d586de3c80 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12dd5e640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586e10a70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd128130950 tx=0x7fd128103870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d27f12cfe2e156 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.384+0000 7fd12d55d640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586e0f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.385+0000 7fd110ff9640 1 -- 172.21.15.64:0/4263442390 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128082040 con 0x55d586de3c80 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.385+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd128107fb0 con 0x55d586de3c80 2024-05-09T09:16:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.385+0000 7fd110ff9640 1 -- 172.21.15.64:0/4263442390 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd128074840 con 0x55d586de3c80 2024-05-09T09:16:26.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.385+0000 7fd110ff9640 1 -- 172.21.15.64:0/4263442390 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd128074a80 con 0x55d586de3c80 2024-05-09T09:16:26.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.385+0000 7fd1325ab740 10 client.26894 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:26.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.385+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d586e7c720 con 0x55d586de3c80 2024-05-09T09:16:26.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.386+0000 7fd110ff9640 1 -- 172.21.15.64:0/4263442390 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd128107fb0 con 0x55d586de3c80 2024-05-09T09:16:26.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.386+0000 7fd110ff9640 1 -- 172.21.15.64:0/4263442390 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd128067ce0 con 0x55d586de3c80 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.387+0000 7fd1325ab740 10 client.26894 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.387+0000 7fd1325ab740 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.387+0000 7fd1325ab740 20 client.26894 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.387+0000 7fd1325ab740 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.387+0000 7fd1325ab740 4 client.26894 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.387+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d586d50430 con 0x55d586e15dd0 2024-05-09T09:16:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.388+0000 7fd12d55d640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.388+0000 7fd12d55d640 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e7efd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc38087c5492fe8 server_cookie=4d29d125b41dfb16 in_seq=0 out_seq=0 2024-05-09T09:16:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.388+0000 7fd110ff9640 10 client.26894 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.389+0000 7fd110ff9640 1 -- 172.21.15.64:0/4263442390 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d586d50430 con 0x55d586e15dd0 2024-05-09T09:16:26.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.389+0000 7fd110ff9640 10 client.26894 handle_command_reply: tid=0 2024-05-09T09:16:26.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 client.26894 shutdown 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 msgr2=0x55d586e10a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586e10a70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd128130950 tx=0x7fd128103870 comp rx=0 tx=0).stop 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 shutdown_connections 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d586e15dd0 0x55d586e7efd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d586de3c80 0x55d586e10a70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d586ddeb10 0x55d586e0fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 --2- 172.21.15.64:0/4263442390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d586dde120 0x55d586e0f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 >> 172.21.15.64:0/4263442390 conn(0x55d586dd9790 msgr2=0x55d586dd81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd110ff9640 0 client.26894 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.390+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 shutdown_connections 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.391+0000 7fd1325ab740 1 -- 172.21.15.64:0/4263442390 wait complete. 2024-05-09T09:16:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.391+0000 7fd1325ab740 20 client.26894 trim_cache size 0 max 16384 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "b0ffde3c-bc8c-4c8b-831a-61cb4c59705a": { 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "b0ffde3c-bc8c-4c8b-831a-61cb4c59705a", 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:16:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1140a2d10 msgr2=0x7fd1140a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1140a2d10 0x7fd1140a51d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd1180058c0 tx=0x7fd118005850 comp rx=0 tx=0).stop 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 msgr2=0x7fd1201466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 0x7fd1201466d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd1280673a0 tx=0x7fd128070990 comp rx=0 tx=0).stop 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 shutdown_connections 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1140a2d10 0x7fd1140a51d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200aafb0 0x7fd1201466d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1200a62b0 0x7fd12013f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 --2- 172.21.15.64:0/1894770812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1200a4fd0 0x7fd12013f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.393+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 >> 172.21.15.64:0/1894770812 conn(0x7fd1200a00d0 msgr2=0x7fd1200a6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.394+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 shutdown_connections 2024-05-09T09:16:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:26.394+0000 7fd12e55f640 1 -- 172.21.15.64:0/1894770812 wait complete. 2024-05-09T09:16:26.417 INFO:tasks.cephfs.filesystem:scrub status for tag:b0ffde3c-bc8c-4c8b-831a-61cb4c59705a - {'path': '/', 'tag': 'b0ffde3c-bc8c-4c8b-831a-61cb4c59705a', 'options': 'recursive,force'} 2024-05-09T09:16:26.434 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:16:26.086+0000 7f3d1eb03700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans/ auth{2=1} v2916 ap=2 f(v8 m2024-05-09T09:15:52.500587+0000 1=1+0) n(v9 rc2024-05-09T09:15:52.500587+0000 b8192 2=1+1) (inest sync->mix g=2) caps={15456=pAsLsXs/-@40} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55f5a7ce6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v501>,<0x10000000005/data v111>,<0x10000000003/postgres v28>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v39>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v45>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2916>,<0x10000000005/data v1325>,<0x10000000003/postgres v560>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v566>,<0x10000000001/sv_0 v577>,<0x10000000000/qa v562>,<0x1/volumes v533>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:15:52.484381+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:15:52.500587+0000 b8192 2=1+1)","memory_value.dirstat":"f(v8 m2024-05-09T09:15:52.500587+0000 1=1+0)","memory_value.rstat":"n(v9 rc2024-05-09T09:15:52.500587+0000 b8192 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: pgmap v589: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 918 op/s 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:26 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: pgmap v589: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 918 op/s 2024-05-09T09:16:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:16:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:16:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:16:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:16:26.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:26 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: pgmap v589: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 918 op/s 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:26 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:27.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:27 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:16:27.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:27 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:27.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:27 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:16:27.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:27 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:27.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:27 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:16:27.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:27 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:16:28.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:28 smithi179 ceph-mon[34086]: pgmap v590: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 824 op/s 2024-05-09T09:16:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:28 smithi139 ceph-mon[33599]: pgmap v590: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 824 op/s 2024-05-09T09:16:28.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:28 smithi064 ceph-mon[30149]: pgmap v590: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 824 op/s 2024-05-09T09:16:30.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:30 smithi179 ceph-mon[34086]: pgmap v591: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:16:30.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:30 smithi139 ceph-mon[33599]: pgmap v591: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:16:30.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:30 smithi064 ceph-mon[30149]: pgmap v591: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-09T09:16:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 370.0 s, 829.4 tps, lat 117.802 ms stddev 61.850 2024-05-09T09:16:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.625+0000 7f0507443640 1 -- 172.21.15.64:0/763927416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 msgr2=0x7f04f80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.625+0000 7f0507443640 1 --2- 172.21.15.64:0/763927416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f80a4790 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f050004f8c0 tx=0x7f050008d720 comp rx=0 tx=0).stop 2024-05-09T09:16:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.628+0000 7f0507443640 1 -- 172.21.15.64:0/763927416 shutdown_connections 2024-05-09T09:16:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.628+0000 7f0507443640 1 --2- 172.21.15.64:0/763927416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f80a5210 0x7f04f8005c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.628+0000 7f0507443640 1 --2- 172.21.15.64:0/763927416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 0x7f04f80a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.628+0000 7f0507443640 1 --2- 172.21.15.64:0/763927416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f80a4790 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.628+0000 7f0507443640 1 -- 172.21.15.64:0/763927416 >> 172.21.15.64:0/763927416 conn(0x7f04f80a00d0 msgr2=0x7f04f80a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.631+0000 7f0507443640 1 -- 172.21.15.64:0/763927416 shutdown_connections 2024-05-09T09:16:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.632+0000 7f0507443640 1 -- 172.21.15.64:0/763927416 wait complete. 2024-05-09T09:16:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.633+0000 7f0507443640 1 Processor -- start 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.633+0000 7f0507443640 1 -- start start 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.633+0000 7f0507443640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f80a5210 0x7f04f8012930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.633+0000 7f0507443640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f8012e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0507443640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 0x7f04f800f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0507443640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8013760 con 0x7f04f80a6520 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0507443640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80135e0 con 0x7f04f80a6940 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0507443640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80138e0 con 0x7f04f80a5210 2024-05-09T09:16:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0505c40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f8012e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0505c40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f8012e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41328/0 (socket says 172.21.15.64:41328) 2024-05-09T09:16:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0505c40640 1 -- 172.21.15.64:0/3269055461 learned_addr learned my addr 172.21.15.64:0/3269055461 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.634+0000 7f0506c42640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 0x7f04f800f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.635+0000 7f0506c42640 1 -- 172.21.15.64:0/3269055461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f80a5210 msgr2=0x7f04f8012930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:16:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.635+0000 7f0506c42640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f80a5210 0x7f04f8012930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.635+0000 7f0506c42640 1 -- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 msgr2=0x7f04f8012e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.635+0000 7f0506c42640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f8012e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.635+0000 7f0506c42640 1 -- 172.21.15.64:0/3269055461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f800fd60 con 0x7f04f80a6940 2024-05-09T09:16:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.635+0000 7f0506c42640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 0x7f04f800f4a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009fd0 tx=0x7f04f000eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.637+0000 7f04f77fe640 1 -- 172.21.15.64:0/3269055461 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0004230 con 0x7f04f80a6940 2024-05-09T09:16:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.637+0000 7f04f77fe640 1 -- 172.21.15.64:0/3269055461 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f00043d0 con 0x7f04f80a6940 2024-05-09T09:16:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.637+0000 7f04f77fe640 1 -- 172.21.15.64:0/3269055461 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00047c0 con 0x7f04f80a6940 2024-05-09T09:16:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.637+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f814fdc0 con 0x7f04f80a6940 2024-05-09T09:16:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.637+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81502c0 con 0x7f04f80a6940 2024-05-09T09:16:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.641+0000 7f050b3b8740 1 Processor -- start 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.642+0000 7f04f77fe640 1 -- 172.21.15.64:0/3269055461 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04f00049e0 con 0x7f04f80a6940 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.643+0000 7f04f77fe640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04ec0a2d90 0x7f04ec0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 -- start start 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f31820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 0x561680f393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f04f77fe640 1 -- 172.21.15.64:0/3269055461 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04f00cf5c0 con 0x7f04f80a6940 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561680e95340 con 0x561680f36f60 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561680e3d290 con 0x561680f31400 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f050b3b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561680e91300 con 0x561680f31df0 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f0506c42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 0x561680f393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f0506c42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 0x561680f393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41338/0 (socket says 172.21.15.64:41338) 2024-05-09T09:16:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.644+0000 7f0506c42640 1 -- 172.21.15.64:0/429989497 learned_addr learned my addr 172.21.15.64:0/429989497 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.645+0000 7f0506c42640 1 -- 172.21.15.64:0/429989497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 msgr2=0x561680f368f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.645+0000 7f0506c42640 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f368f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.645+0000 7f0506c42640 1 -- 172.21.15.64:0/429989497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 msgr2=0x561680f31820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.645+0000 7f0506c42640 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f31820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.645+0000 7f0506c42640 1 -- 172.21.15.64:0/429989497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561680e618e0 con 0x561680f36f60 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.645+0000 7f0505c40640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04ec0a2d90 0x7f04ec0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.646+0000 7f0506c42640 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 0x561680f393f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f04f00991d0 tx=0x7f04f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3ad46b41b6e8dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.648+0000 7f0505c40640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04ec0a2d90 0x7f04ec0a5250 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x561680f321f0 tx=0x7f04fc00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.650+0000 7f04f4ff9640 1 -- 172.21.15.64:0/429989497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f009a8a0 con 0x561680f36f60 2024-05-09T09:16:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.650+0000 7f04f4ff9640 1 -- 172.21.15.64:0/429989497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f009ab00 con 0x561680f36f60 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.651+0000 7f050b3b8740 1 -- 172.21.15.64:0/429989497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 msgr2=0x561680f393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.651+0000 7f050b3b8740 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 0x561680f393f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f04f00991d0 tx=0x7f04f0013000 comp rx=0 tx=0).stop 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.652+0000 7f050b3b8740 1 -- 172.21.15.64:0/429989497 shutdown_connections 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.652+0000 7f050b3b8740 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f36f60 0x561680f393f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.652+0000 7f050b3b8740 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.652+0000 7f050b3b8740 1 --2- 172.21.15.64:0/429989497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f31820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.652+0000 7f050b3b8740 1 -- 172.21.15.64:0/429989497 >> 172.21.15.64:0/429989497 conn(0x561680f2ca70 msgr2=0x561680f2ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.654+0000 7f050b3b8740 1 -- 172.21.15.64:0/429989497 shutdown_connections 2024-05-09T09:16:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.654+0000 7f050b3b8740 1 -- 172.21.15.64:0/429989497 wait complete. 2024-05-09T09:16:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.654+0000 7f050b3b8740 1 -- start start 2024-05-09T09:16:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 Processor -- start 2024-05-09T09:16:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f5dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561680f5bf70 con 0x561680f62cd0 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561680e50790 con 0x561680f31400 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561680f5c0f0 con 0x561680f31df0 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f050b3b8740 10 client.0 _ll_register_callbacks cb 0x561680e8ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f0505c40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f5dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f0505c40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f5dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37352/0 (socket says 172.21.15.64:37352) 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.655+0000 7f0505c40640 1 -- 172.21.15.64:0/3576762761 learned_addr learned my addr 172.21.15.64:0/3576762761 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0506c42640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0506441640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0505c40640 1 -- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 msgr2=0x561680f5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0505c40640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f5d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0505c40640 1 -- 172.21.15.64:0/3576762761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 msgr2=0x561680f650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0505c40640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0505c40640 1 -- 172.21.15.64:0/3576762761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561680ba9210 con 0x561680f31df0 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f0505c40640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f5dca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0116c0 tx=0x7f04fc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74837ae975e8c3c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.656+0000 7f04e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f04e9ffb640 1 -- 172.21.15.64:0/3576762761 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc005220 con 0x561680f31df0 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f04e9ffb640 1 -- 172.21.15.64:0/3576762761 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f04fc0053c0 con 0x561680f31df0 2024-05-09T09:16:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561680ea2a70 con 0x561680f31df0 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f04e9ffb640 1 -- 172.21.15.64:0/3576762761 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f04fc0055c0 con 0x561680f31df0 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f050b3b8740 10 client.27073 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561680c75620 con 0x561680f31df0 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.657+0000 7f04e9ffb640 1 -- 172.21.15.64:0/3576762761 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561680ea2a70 con 0x561680f31df0 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f04e9ffb640 1 -- 172.21.15.64:0/3576762761 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f04fc01ec30 con 0x561680f31df0 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f050b3b8740 10 client.27073 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f050b3b8740 10 client.27073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f050b3b8740 20 client.27073 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f050b3b8740 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680f69b70 0x561680f6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f050b3b8740 4 client.27073 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.659+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561680e9cdc0 con 0x561680f69b70 2024-05-09T09:16:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.660+0000 7f0506441640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680f69b70 0x561680f6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.660+0000 7f0506441640 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680f69b70 0x561680f6bfd0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb8da9d1b4f2eb6 server_cookie=436b224c4f8619b8 in_seq=0 out_seq=0 2024-05-09T09:16:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.661+0000 7f04e9ffb640 10 client.27073 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.661+0000 7f04e9ffb640 1 -- 172.21.15.64:0/3576762761 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561680e9cdc0 con 0x561680f69b70 2024-05-09T09:16:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.661+0000 7f04e9ffb640 10 client.27073 handle_command_reply: tid=0 2024-05-09T09:16:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.661+0000 7f050b3b8740 1 client.27073 shutdown 2024-05-09T09:16:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.664+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 msgr2=0x561680f5dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.664+0000 7f050b3b8740 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f5dca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0116c0 tx=0x7f04fc004730 comp rx=0 tx=0).stop 2024-05-09T09:16:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 shutdown_connections 2024-05-09T09:16:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f050b3b8740 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680f69b70 0x561680f6bfd0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f050b3b8740 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f050b3b8740 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f5dca0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f050b3b8740 1 --2- 172.21.15.64:0/3576762761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 >> 172.21.15.64:0/3576762761 conn(0x561680f2ca70 msgr2=0x561680f2bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.666+0000 7f04e9ffb640 0 client.27073 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.667+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 shutdown_connections 2024-05-09T09:16:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.667+0000 7f050b3b8740 1 -- 172.21.15.64:0/3576762761 wait complete. 2024-05-09T09:16:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.667+0000 7f050b3b8740 20 client.27073 trim_cache size 0 max 16384 2024-05-09T09:16:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.697+0000 7f050b3b8740 1 Processor -- start 2024-05-09T09:16:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.698+0000 7f050b3b8740 1 -- start start 2024-05-09T09:16:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.699+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.699+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.699+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f33570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.699+0000 7f050b3b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4012470 con 0x561680f62cd0 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.699+0000 7f050b3b8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04e400fce0 con 0x561680f31400 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.699+0000 7f050b3b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04e400dbe0 con 0x561680f31df0 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.700+0000 7f0506441640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.700+0000 7f0506441640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f2ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42922/0 (socket says 172.21.15.64:42922) 2024-05-09T09:16:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.700+0000 7f0506441640 1 -- 172.21.15.64:0/373201872 learned_addr learned my addr 172.21.15.64:0/373201872 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.700+0000 7f0505c40640 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.701+0000 7f0506441640 1 -- 172.21.15.64:0/373201872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 msgr2=0x561680f33030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.701+0000 7f0506441640 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f33030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.701+0000 7f0506441640 1 -- 172.21.15.64:0/373201872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 msgr2=0x561680f33570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.701+0000 7f0506441640 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.701+0000 7f0506441640 1 -- 172.21.15.64:0/373201872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561680f30a00 con 0x561680f31400 2024-05-09T09:16:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.701+0000 7f0506441640 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f2ce50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0500050dd0 tx=0x7f05000677a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26da3274e2e626a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.702+0000 7f04ebfff640 1 -- 172.21.15.64:0/373201872 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000a6020 con 0x561680f31400 2024-05-09T09:16:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.702+0000 7f04ebfff640 1 -- 172.21.15.64:0/373201872 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05000a2640 con 0x561680f31400 2024-05-09T09:16:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.702+0000 7f04ebfff640 1 -- 172.21.15.64:0/373201872 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500093ed0 con 0x561680f31400 2024-05-09T09:16:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.704+0000 7f050b3b8740 1 -- 172.21.15.64:0/373201872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 msgr2=0x561680f2ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.704+0000 7f050b3b8740 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f2ce50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0500050dd0 tx=0x7f05000677a0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.706+0000 7f050b3b8740 1 -- 172.21.15.64:0/373201872 shutdown_connections 2024-05-09T09:16:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.706+0000 7f050b3b8740 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680f33570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.706+0000 7f050b3b8740 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31df0 0x561680f33030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.706+0000 7f050b3b8740 1 --2- 172.21.15.64:0/373201872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31400 0x561680f2ce50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.706+0000 7f050b3b8740 1 -- 172.21.15.64:0/373201872 >> 172.21.15.64:0/373201872 conn(0x561680f2ca70 msgr2=0x561680f36680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.707+0000 7f050b3b8740 1 -- 172.21.15.64:0/373201872 shutdown_connections 2024-05-09T09:16:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.707+0000 7f050b3b8740 1 -- 172.21.15.64:0/373201872 wait complete. 2024-05-09T09:16:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.708+0000 7f050b3b8740 1 -- start start 2024-05-09T09:16:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.708+0000 7f050b3b8740 1 Processor -- start 2024-05-09T09:16:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 0x561680f7a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31df0 0x561680f7a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680fcdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f0506441640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 0x561680f7a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e400e1a0 con 0x561680f62cd0 2024-05-09T09:16:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04e400f610 con 0x561680f31df0 2024-05-09T09:16:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04e400e870 con 0x561680f31400 2024-05-09T09:16:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f0506441640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 0x561680f7a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:37376/0 (socket says 172.21.15.64:37376) 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f0506441640 1 -- 172.21.15.64:0/387183429 learned_addr learned my addr 172.21.15.64:0/387183429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f050b3b8740 10 client.0 _ll_register_callbacks cb 0x7f04fc00b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.709+0000 7f0505c40640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31df0 0x561680f7a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506c42640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680fcdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506441640 1 -- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31df0 msgr2=0x561680f7a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506441640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31df0 0x561680f7a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506441640 1 -- 172.21.15.64:0/387183429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 msgr2=0x561680fcdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506441640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680fcdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506441640 1 -- 172.21.15.64:0/387183429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561680fce190 con 0x561680f31400 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f0506441640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 0x561680f7a3b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f05000649a0 tx=0x7f05000a2a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d599bb17693086e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.710+0000 7f04e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.711+0000 7f04e9ffb640 1 -- 172.21.15.64:0/387183429 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000a6030 con 0x561680f31400 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.711+0000 7f04e9ffb640 1 -- 172.21.15.64:0/387183429 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0500093800 con 0x561680f31400 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.711+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05000a2640 con 0x561680f31400 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.711+0000 7f04e9ffb640 1 -- 172.21.15.64:0/387183429 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f05000be050 con 0x561680f31400 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.711+0000 7f050b3b8740 10 client.27079 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.711+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561680fcf4a0 con 0x561680f31400 2024-05-09T09:16:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f04e9ffb640 1 -- 172.21.15.64:0/387183429 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0500093a00 con 0x561680f31400 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f04e9ffb640 1 -- 172.21.15.64:0/387183429 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f05000ac430 con 0x561680f31400 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f050b3b8740 10 client.27079 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f050b3b8740 10 client.27079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f050b3b8740 20 client.27079 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f050b3b8740 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680fcfda0 0x561680fd21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f050b3b8740 4 client.27079 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f05000a2640 con 0x561680fcfda0 2024-05-09T09:16:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.712+0000 7f0505c40640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680fcfda0 0x561680fd21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.713+0000 7f0505c40640 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680fcfda0 0x561680fd21c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8870008f40abadd4 server_cookie=598f940b5c44104a in_seq=0 out_seq=0 2024-05-09T09:16:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.713+0000 7f04e9ffb640 10 client.27079 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.754+0000 7f04e9ffb640 1 -- 172.21.15.64:0/387183429 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f05000a2640 con 0x561680fcfda0 2024-05-09T09:16:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.754+0000 7f04e9ffb640 10 client.27079 handle_command_reply: tid=0 2024-05-09T09:16:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.754+0000 7f050b3b8740 1 client.27079 shutdown 2024-05-09T09:16:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.755+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 msgr2=0x561680f7a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.755+0000 7f050b3b8740 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 0x561680f7a3b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f05000649a0 tx=0x7f05000a2a20 comp rx=0 tx=0).stop 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.755+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 shutdown_connections 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.755+0000 7f050b3b8740 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561680fcfda0 0x561680fd21c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f050b3b8740 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561680f62cd0 0x561680fcdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f04e9ffb640 0 client.27079 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f050b3b8740 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561680f31df0 0x561680f7a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f050b3b8740 1 --2- 172.21.15.64:0/387183429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561680f31400 0x561680f7a3b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 >> 172.21.15.64:0/387183429 conn(0x561680f2ca70 msgr2=0x561680f36250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 shutdown_connections 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.756+0000 7f050b3b8740 1 -- 172.21.15.64:0/387183429 wait complete. 2024-05-09T09:16:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.757+0000 7f050b3b8740 20 client.27079 trim_cache size 0 max 16384 2024-05-09T09:16:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:16:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:16:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:16:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:16:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.760+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04ec0a2d90 msgr2=0x7f04ec0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.760+0000 7f0507443640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04ec0a2d90 0x7f04ec0a5250 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x561680f321f0 tx=0x7f04fc00b480 comp rx=0 tx=0).stop 2024-05-09T09:16:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.760+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 msgr2=0x7f04f800f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.760+0000 7f0507443640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 0x7f04f800f4a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009fd0 tx=0x7f04f000eba0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.761+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 shutdown_connections 2024-05-09T09:16:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.762+0000 7f0507443640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f04ec0a2d90 0x7f04ec0a5250 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.762+0000 7f0507443640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04f80a6940 0x7f04f800f4a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.762+0000 7f0507443640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f80a6520 0x7f04f8012e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.762+0000 7f0507443640 1 --2- 172.21.15.64:0/3269055461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f80a5210 0x7f04f8012930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.762+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 >> 172.21.15.64:0/3269055461 conn(0x7f04f80a00d0 msgr2=0x7f04f80140d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.765+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 shutdown_connections 2024-05-09T09:16:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:31.765+0000 7f0507443640 1 -- 172.21.15.64:0/3269055461 wait complete. 2024-05-09T09:16:32.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:32 smithi179 ceph-mon[34086]: pgmap v592: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.2 MiB/s wr, 830 op/s 2024-05-09T09:16:32.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:32 smithi139 ceph-mon[33599]: pgmap v592: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.2 MiB/s wr, 830 op/s 2024-05-09T09:16:32.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:32 smithi064 ceph-mon[30149]: pgmap v592: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.2 MiB/s wr, 830 op/s 2024-05-09T09:16:34.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:34 smithi179 ceph-mon[34086]: pgmap v593: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.0 MiB/s wr, 917 op/s 2024-05-09T09:16:34.822 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:34 smithi139 ceph-mon[33599]: pgmap v593: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.0 MiB/s wr, 917 op/s 2024-05-09T09:16:34.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:34 smithi064 ceph-mon[30149]: pgmap v593: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.0 MiB/s wr, 917 op/s 2024-05-09T09:16:35.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:35 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:35.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:35 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:35.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:35 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:16:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 375.0 s, 897.0 tps, lat 114.820 ms stddev 132.781 2024-05-09T09:16:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:36 smithi179 ceph-mon[34086]: pgmap v594: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.8 MiB/s wr, 950 op/s 2024-05-09T09:16:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:36 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:36 smithi139 ceph-mon[33599]: pgmap v594: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.8 MiB/s wr, 950 op/s 2024-05-09T09:16:36.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:36 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:36 smithi064 ceph-mon[30149]: pgmap v594: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.8 MiB/s wr, 950 op/s 2024-05-09T09:16:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:36 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:16:37.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:37 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi179 to 176.1M 2024-05-09T09:16:37.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:37 smithi179 ceph-mon[34086]: Unable to set osd_memory_target on smithi179 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T09:16:37.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:37.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:37.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:16:37.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:37 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi179 to 176.1M 2024-05-09T09:16:37.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:37 smithi139 ceph-mon[33599]: Unable to set osd_memory_target on smithi179 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T09:16:37.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:37.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:37.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:16:37.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:37 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi179 to 176.1M 2024-05-09T09:16:37.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:37 smithi064 ceph-mon[30149]: Unable to set osd_memory_target on smithi179 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T09:16:37.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:16:37.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:16:37.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:16:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:38 smithi179 ceph-mon[34086]: pgmap v595: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.9 MiB/s wr, 853 op/s 2024-05-09T09:16:39.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:38 smithi139 ceph-mon[33599]: pgmap v595: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.9 MiB/s wr, 853 op/s 2024-05-09T09:16:39.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:38 smithi064 ceph-mon[30149]: pgmap v595: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.9 MiB/s wr, 853 op/s 2024-05-09T09:16:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:40 smithi179 ceph-mon[34086]: pgmap v596: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.6 MiB/s wr, 1.09k op/s 2024-05-09T09:16:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:16:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:16:40.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:16:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:40 smithi139 ceph-mon[33599]: pgmap v596: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.6 MiB/s wr, 1.09k op/s 2024-05-09T09:16:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:16:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:16:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:16:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:40 smithi064 ceph-mon[30149]: pgmap v596: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.6 MiB/s wr, 1.09k op/s 2024-05-09T09:16:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:16:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:16:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:16:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 380.0 s, 877.0 tps, lat 113.984 ms stddev 57.177 2024-05-09T09:16:42.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:42 smithi179 ceph-mon[34086]: pgmap v597: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 817 op/s 2024-05-09T09:16:43.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:42 smithi139 ceph-mon[33599]: pgmap v597: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 817 op/s 2024-05-09T09:16:43.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:42 smithi064 ceph-mon[30149]: pgmap v597: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 817 op/s 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.922+0000 7f2b49d9e640 1 -- 172.21.15.64:0/4005199424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 msgr2=0x7f2b44101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.922+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/4005199424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44101510 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009900 tx=0x7f2b3802a410 comp rx=0 tx=0).stop 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.923+0000 7f2b49d9e640 1 -- 172.21.15.64:0/4005199424 shutdown_connections 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.923+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/4005199424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b44101a50 0x7f2b44108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.923+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/4005199424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44101510 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.923+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/4005199424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b440fff40 0x7f2b44100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.923+0000 7f2b49d9e640 1 -- 172.21.15.64:0/4005199424 >> 172.21.15.64:0/4005199424 conn(0x7f2b440fb4d0 msgr2=0x7f2b440fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.924+0000 7f2b49d9e640 1 -- 172.21.15.64:0/4005199424 shutdown_connections 2024-05-09T09:16:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.924+0000 7f2b49d9e640 1 -- 172.21.15.64:0/4005199424 wait complete. 2024-05-09T09:16:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.925+0000 7f2b49d9e640 1 Processor -- start 2024-05-09T09:16:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.925+0000 7f2b49d9e640 1 -- start start 2024-05-09T09:16:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b49d9e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b440fff40 0x7f2b44110b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b49d9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44111090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b49d9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b44101a50 0x7f2b441ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b49d9e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410acb0 con 0x7f2b44101090 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b49d9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410ab30 con 0x7f2b44101a50 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b49d9e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410ae30 con 0x7f2b440fff40 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44111090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36484/0 (socket says 172.21.15.64:36484) 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.926+0000 7f2b42ffd640 1 -- 172.21.15.64:0/439074965 learned_addr learned my addr 172.21.15.64:0/439074965 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.927+0000 7f2b42ffd640 1 -- 172.21.15.64:0/439074965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b440fff40 msgr2=0x7f2b44110b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.927+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b440fff40 0x7f2b44110b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.927+0000 7f2b42ffd640 1 -- 172.21.15.64:0/439074965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b44101a50 msgr2=0x7f2b441ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.927+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b44101a50 0x7f2b441ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.927+0000 7f2b42ffd640 1 -- 172.21.15.64:0/439074965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b441ae4a0 con 0x7f2b44101090 2024-05-09T09:16:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.928+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44111090 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009230 tx=0x7f2b38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.928+0000 7f2b40ff9640 1 -- 172.21.15.64:0/439074965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3803f030 con 0x7f2b44101090 2024-05-09T09:16:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.928+0000 7f2b40ff9640 1 -- 172.21.15.64:0/439074965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b38003aa0 con 0x7f2b44101090 2024-05-09T09:16:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.928+0000 7f2b40ff9640 1 -- 172.21.15.64:0/439074965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38035070 con 0x7f2b44101090 2024-05-09T09:16:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.928+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b441ae730 con 0x7f2b44101090 2024-05-09T09:16:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.929+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b441aebd0 con 0x7f2b44101090 2024-05-09T09:16:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.932+0000 7f2b4cb84740 1 Processor -- start 2024-05-09T09:16:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 -- start start 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aacf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aad7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55980aa33340 con 0x55980aacfc40 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55980a9db290 con 0x55980aacf250 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55980aa2f300 con 0x55980aad4db0 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aacf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aacf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:40178/0 (socket says 172.21.15.64:40178) 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b43fff640 1 -- 172.21.15.64:0/3652076932 learned_addr learned my addr 172.21.15.64:0/3652076932 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.933+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b437fe640 1 -- 172.21.15.64:0/3652076932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 msgr2=0x55980aad7240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aad7240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b437fe640 1 -- 172.21.15.64:0/3652076932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 msgr2=0x55980aacf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aacf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b437fe640 1 -- 172.21.15.64:0/3652076932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980a9ff8e0 con 0x55980aacfc40 2024-05-09T09:16:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad4740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009ad0 tx=0x7f2b3c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf589ff9a97f046 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b29ffb640 1 -- 172.21.15.64:0/3652076932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c00ed10 con 0x55980aacfc40 2024-05-09T09:16:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b29ffb640 1 -- 172.21.15.64:0/3652076932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b3c005020 con 0x55980aacfc40 2024-05-09T09:16:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.934+0000 7f2b29ffb640 1 -- 172.21.15.64:0/3652076932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0049e0 con 0x55980aacfc40 2024-05-09T09:16:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3652076932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 msgr2=0x55980aad4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad4740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009ad0 tx=0x7f2b3c009f90 comp rx=0 tx=0).stop 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3652076932 shutdown_connections 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aad7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad4740 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3652076932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aacf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3652076932 >> 172.21.15.64:0/3652076932 conn(0x55980aaca8c0 msgr2=0x55980aacacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b40ff9640 1 -- 172.21.15.64:0/439074965 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b38042030 con 0x7f2b44101090 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3652076932 shutdown_connections 2024-05-09T09:16:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.936+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3652076932 wait complete. 2024-05-09T09:16:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.937+0000 7f2b4cb84740 1 -- start start 2024-05-09T09:16:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.937+0000 7f2b40ff9640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2b2c0a2c40 0x7f2b2c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.938+0000 7f2b40ff9640 1 -- 172.21.15.64:0/439074965 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b380ef9d0 con 0x7f2b44101090 2024-05-09T09:16:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.938+0000 7f2b43fff640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2b2c0a2c40 0x7f2b2c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.938+0000 7f2b4cb84740 1 Processor -- start 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aad9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aae0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55980aad7d60 con 0x55980aacfc40 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55980a9ee790 con 0x55980aacf250 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55980aad7ee0 con 0x55980aad4db0 2024-05-09T09:16:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b4cb84740 10 client.0 _ll_register_callbacks cb 0x55980aa29a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b437fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aad9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b42ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36498/0 (socket says 172.21.15.64:36498) 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b42ffd640 1 -- 172.21.15.64:0/706211242 learned_addr learned my addr 172.21.15.64:0/706211242 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b43fff640 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aae0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.939+0000 7f2b43fff640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2b2c0a2c40 0x7f2b2c0a5100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b340049a0 tx=0x7f2b34010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.940+0000 7f2b42ffd640 1 -- 172.21.15.64:0/706211242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 msgr2=0x55980aae0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.940+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aae0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.940+0000 7f2b42ffd640 1 -- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 msgr2=0x55980aad9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.940+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aad9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.940+0000 7f2b42ffd640 1 -- 172.21.15.64:0/706211242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980a747210 con 0x55980aacfc40 2024-05-09T09:16:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.941+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad9a40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2b380bbdd0 tx=0x7f2b380bb980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37378dc9639dd4af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.942+0000 7f2b22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.942+0000 7f2b22ffd640 1 -- 172.21.15.64:0/706211242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380be030 con 0x55980aacfc40 2024-05-09T09:16:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.942+0000 7f2b22ffd640 1 -- 172.21.15.64:0/706211242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b380b84b0 con 0x55980aacfc40 2024-05-09T09:16:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.942+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b3c005020 con 0x55980aacfc40 2024-05-09T09:16:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.943+0000 7f2b22ffd640 1 -- 172.21.15.64:0/706211242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2b380da050 con 0x55980aacfc40 2024-05-09T09:16:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.943+0000 7f2b4cb84740 10 client.17439 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.943+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55980a813620 con 0x55980aacfc40 2024-05-09T09:16:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b22ffd640 1 -- 172.21.15.64:0/706211242 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b3c005020 con 0x55980aacfc40 2024-05-09T09:16:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b22ffd640 1 -- 172.21.15.64:0/706211242 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2b380bc0a0 con 0x55980aacfc40 2024-05-09T09:16:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b4cb84740 10 client.17439 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b4cb84740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b4cb84740 20 client.17439 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980aae5010 0x55980aae7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b4cb84740 4 client.17439 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980aa40a70 con 0x55980aae5010 2024-05-09T09:16:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.944+0000 7f2b437fe640 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980aae5010 0x55980aae7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.945+0000 7f2b22ffd640 10 client.17439 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.945+0000 7f2b437fe640 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980aae5010 0x55980aae7430 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae34ab0f4ec76fc server_cookie=7a0b655b9c2b2607 in_seq=0 out_seq=0 2024-05-09T09:16:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.946+0000 7f2b22ffd640 1 -- 172.21.15.64:0/706211242 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55980aa40a70 con 0x55980aae5010 2024-05-09T09:16:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.946+0000 7f2b22ffd640 10 client.17439 handle_command_reply: tid=0 2024-05-09T09:16:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.946+0000 7f2b4cb84740 1 client.17439 shutdown 2024-05-09T09:16:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.947+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 msgr2=0x55980aad9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.947+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad9a40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2b380bbdd0 tx=0x7f2b380bb980 comp rx=0 tx=0).stop 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 shutdown_connections 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980aae5010 0x55980aae7430 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aad4db0 0x55980aae0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacfc40 0x55980aad9a40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/706211242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aacf250 0x55980aad9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 >> 172.21.15.64:0/706211242 conn(0x55980aaca8c0 msgr2=0x55980aac9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b22ffd640 0 client.17439 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 shutdown_connections 2024-05-09T09:16:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 1 -- 172.21.15.64:0/706211242 wait complete. 2024-05-09T09:16:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.949+0000 7f2b4cb84740 20 client.17439 trim_cache size 0 max 16384 2024-05-09T09:16:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.978+0000 7f2b4cb84740 1 Processor -- start 2024-05-09T09:16:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.979+0000 7f2b4cb84740 1 -- start start 2024-05-09T09:16:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.979+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aaca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.979+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aae0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.979+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aacdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.979+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441bbbc0 con 0x55980aacf250 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441b9430 con 0x55980aad4db0 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441b7280 con 0x55980aacfc40 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aae0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aae0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35498/0 (socket says 172.21.15.64:35498) 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 -- 172.21.15.64:0/1363333771 learned_addr learned my addr 172.21.15.64:0/1363333771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b43fff640 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aacdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b437fe640 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aaca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 -- 172.21.15.64:0/1363333771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 msgr2=0x55980aacdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aacdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 -- 172.21.15.64:0/1363333771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 msgr2=0x55980aaca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aaca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b42ffd640 1 -- 172.21.15.64:0/1363333771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980aac8f00 con 0x55980aacfc40 2024-05-09T09:16:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.980+0000 7f2b43fff640 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aacdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:44.981 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:44 smithi179 ceph-mon[34086]: pgmap v598: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 938 op/s 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.981+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aae0a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b380ba0f0 tx=0x7f2b38009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36fb533280399fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.981+0000 7f2b297fa640 1 -- 172.21.15.64:0/1363333771 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3809e070 con 0x55980aacfc40 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.981+0000 7f2b297fa640 1 -- 172.21.15.64:0/1363333771 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b380c01f0 con 0x55980aacfc40 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.981+0000 7f2b297fa640 1 -- 172.21.15.64:0/1363333771 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380be070 con 0x55980aacfc40 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 -- 172.21.15.64:0/1363333771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 msgr2=0x55980aae0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aae0a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b380ba0f0 tx=0x7f2b38009260 comp rx=0 tx=0).stop 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 -- 172.21.15.64:0/1363333771 shutdown_connections 2024-05-09T09:16:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aacdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aae0a40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/1363333771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aaca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 -- 172.21.15.64:0/1363333771 >> 172.21.15.64:0/1363333771 conn(0x55980aaca8c0 msgr2=0x55980aacd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.982+0000 7f2b4cb84740 1 -- 172.21.15.64:0/1363333771 shutdown_connections 2024-05-09T09:16:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.983+0000 7f2b4cb84740 1 -- 172.21.15.64:0/1363333771 wait complete. 2024-05-09T09:16:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.983+0000 7f2b4cb84740 1 -- start start 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.983+0000 7f2b4cb84740 1 Processor -- start 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aade2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aade810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aadf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441b78f0 con 0x55980aacf250 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441b8d60 con 0x55980aad4db0 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441b7fc0 con 0x55980aacfc40 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b4cb84740 10 client.0 _ll_register_callbacks cb 0x7f2b38005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aadf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aadf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:40210/0 (socket says 172.21.15.64:40210) 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b43fff640 1 -- 172.21.15.64:0/3736130692 learned_addr learned my addr 172.21.15.64:0/3736130692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aade810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.984+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aade2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b43fff640 1 -- 172.21.15.64:0/3736130692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 msgr2=0x55980aade810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b43fff640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aade810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b43fff640 1 -- 172.21.15.64:0/3736130692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 msgr2=0x55980aade2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b43fff640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aade2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b43fff640 1 -- 172.21.15.64:0/3736130692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980aadfda0 con 0x55980aad4db0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b42ffd640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aade810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aade2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b43fff640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aadf7b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400dce0 tx=0x7f2b3400a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee479045ff874cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b22ffd640 1 -- 172.21.15.64:0/3736130692 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34004b20 con 0x55980aad4db0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b22ffd640 1 -- 172.21.15.64:0/3736130692 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b34009050 con 0x55980aad4db0 2024-05-09T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.985+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b380c01f0 con 0x55980aad4db0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.986+0000 7f2b22ffd640 1 -- 172.21.15.64:0/3736130692 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2b34004cc0 con 0x55980aad4db0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.986+0000 7f2b4cb84740 10 client.26918 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.986+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55980ab5c750 con 0x55980aad4db0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.986+0000 7f2b22ffd640 1 -- 172.21.15.64:0/3736130692 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b380c01f0 con 0x55980aad4db0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b22ffd640 1 -- 172.21.15.64:0/3736130692 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2b34018070 con 0x55980aad4db0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b4cb84740 10 client.26918 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b4cb84740 10 client.26918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b4cb84740 20 client.26918 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980ab5d1a0 0x55980ab5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b4cb84740 4 client.26918 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55980aad92f0 con 0x55980ab5d1a0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.987+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980ab5d1a0 0x55980ab5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.988+0000 7f2b437fe640 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980ab5d1a0 0x55980ab5f5c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df62a2240b84645 server_cookie=de8998343cac516f in_seq=0 out_seq=0 2024-05-09T09:16:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:44.988+0000 7f2b22ffd640 10 client.26918 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b22ffd640 1 -- 172.21.15.64:0/3736130692 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55980aad92f0 con 0x55980ab5d1a0 2024-05-09T09:16:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b22ffd640 10 client.26918 handle_command_reply: tid=0 2024-05-09T09:16:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 client.26918 shutdown 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 msgr2=0x55980aadf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aadf7b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400dce0 tx=0x7f2b3400a610 comp rx=0 tx=0).stop 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 shutdown_connections 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55980ab5d1a0 0x55980ab5f5c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55980aad4db0 0x55980aadf7b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55980aacfc40 0x55980aade810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 --2- 172.21.15.64:0/3736130692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55980aacf250 0x55980aade2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 >> 172.21.15.64:0/3736130692 conn(0x55980aaca8c0 msgr2=0x55980aac9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.004+0000 7f2b22ffd640 0 client.26918 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.005+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 shutdown_connections 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.005+0000 7f2b4cb84740 1 -- 172.21.15.64:0/3736130692 wait complete. 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.005+0000 7f2b4cb84740 20 client.26918 trim_cache size 0 max 16384 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:16:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:16:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.006+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2b2c0a2c40 msgr2=0x7f2b2c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.006+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2b2c0a2c40 0x7f2b2c0a5100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b340049a0 tx=0x7f2b34010040 comp rx=0 tx=0).stop 2024-05-09T09:16:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.006+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 msgr2=0x7f2b44111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44111090 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009230 tx=0x7f2b38004270 comp rx=0 tx=0).stop 2024-05-09T09:16:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 shutdown_connections 2024-05-09T09:16:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2b2c0a2c40 0x7f2b2c0a5100 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b44101a50 0x7f2b441ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b44101090 0x7f2b44111090 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 --2- 172.21.15.64:0/439074965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b440fff40 0x7f2b44110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.007+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 >> 172.21.15.64:0/439074965 conn(0x7f2b440fb4d0 msgr2=0x7f2b440fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.009+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 shutdown_connections 2024-05-09T09:16:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:45.009+0000 7f2b49d9e640 1 -- 172.21.15.64:0/439074965 wait complete. 2024-05-09T09:16:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:44 smithi139 ceph-mon[33599]: pgmap v598: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 938 op/s 2024-05-09T09:16:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:44 smithi064 ceph-mon[30149]: pgmap v598: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 938 op/s 2024-05-09T09:16:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 385.0 s, 806.8 tps, lat 123.894 ms stddev 94.813 2024-05-09T09:16:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:46 smithi179 ceph-mon[34086]: pgmap v599: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 932 op/s 2024-05-09T09:16:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:46 smithi139 ceph-mon[33599]: pgmap v599: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 932 op/s 2024-05-09T09:16:47.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:46 smithi064 ceph-mon[30149]: pgmap v599: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 932 op/s 2024-05-09T09:16:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:48 smithi179 ceph-mon[34086]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 764 op/s 2024-05-09T09:16:49.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:48 smithi139 ceph-mon[33599]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 764 op/s 2024-05-09T09:16:49.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:48 smithi064 ceph-mon[30149]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 764 op/s 2024-05-09T09:16:50.954 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:50 smithi064 ceph-mon[30149]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-05-09T09:16:50.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:50 smithi179 ceph-mon[34086]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-05-09T09:16:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:50 smithi139 ceph-mon[33599]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-05-09T09:16:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 390.0 s, 893.0 tps, lat 112.379 ms stddev 43.388 2024-05-09T09:16:52.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:52 smithi179 ceph-mon[34086]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 802 op/s 2024-05-09T09:16:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:52 smithi139 ceph-mon[33599]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 802 op/s 2024-05-09T09:16:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:52 smithi064 ceph-mon[30149]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 802 op/s 2024-05-09T09:16:54.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:54 smithi179 ceph-mon[34086]: pgmap v603: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 893 op/s 2024-05-09T09:16:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:54 smithi139 ceph-mon[33599]: pgmap v603: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 893 op/s 2024-05-09T09:16:55.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:54 smithi064 ceph-mon[30149]: pgmap v603: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 893 op/s 2024-05-09T09:16:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 395.0 s, 887.8 tps, lat 111.906 ms stddev 48.598 2024-05-09T09:16:56.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:16:56.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:16:56.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.612+0000 7f76127fc640 1 --2- 172.21.15.64:0/2801218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c0a5d60 0x7f760c0a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 -- 172.21.15.64:0/2801218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a6700 msgr2=0x7f760c0aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 --2- 172.21.15.64:0/2801218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a6700 0x7f760c0aafc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f760401a910 tx=0x7f760403b450 comp rx=0 tx=0).stop 2024-05-09T09:16:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 -- 172.21.15.64:0/2801218 shutdown_connections 2024-05-09T09:16:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 --2- 172.21.15.64:0/2801218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a6700 0x7f760c0aafc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 --2- 172.21.15.64:0/2801218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c0a5d60 0x7f760c0a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 --2- 172.21.15.64:0/2801218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a4c30 0x7f760c0a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.613+0000 7f7613fff640 1 -- 172.21.15.64:0/2801218 >> 172.21.15.64:0/2801218 conn(0x7f760c0a00c0 msgr2=0x7f760c0a2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.617+0000 7f7613fff640 1 -- 172.21.15.64:0/2801218 shutdown_connections 2024-05-09T09:16:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.619+0000 7f7613fff640 1 -- 172.21.15.64:0/2801218 wait complete. 2024-05-09T09:16:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.621+0000 7f7613fff640 1 Processor -- start 2024-05-09T09:16:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.621+0000 7f7613fff640 1 -- start start 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.622+0000 7f7613fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a4c30 0x7f760c13ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.622+0000 7f7613fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c0a5d60 0x7f760c13b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.622+0000 7f7613fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a6700 0x7f760c142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.622+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a4c30 0x7f760c13ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.622+0000 7f7613fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f760c0ae8c0 con 0x7f760c0a4c30 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a4c30 0x7f760c13ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34458/0 (socket says 172.21.15.64:34458) 2024-05-09T09:16:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f7612ffd640 1 -- 172.21.15.64:0/3545396759 learned_addr learned my addr 172.21.15.64:0/3545396759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f760c0ae740 con 0x7f760c0a5d60 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f760c0aea40 con 0x7f760c0a6700 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f76137fe640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a6700 0x7f760c142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f76137fe640 1 -- 172.21.15.64:0/3545396759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c0a5d60 msgr2=0x7f760c13b340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f76137fe640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c0a5d60 0x7f760c13b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f76137fe640 1 -- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a4c30 msgr2=0x7f760c13ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f76137fe640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a4c30 0x7f760c13ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.623+0000 7f76137fe640 1 -- 172.21.15.64:0/3545396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1429d0 con 0x7f760c0a6700 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.624+0000 7f76137fe640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a6700 0x7f760c142360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f760403b970 tx=0x7f7604005380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.624+0000 7f75fbfff640 1 -- 172.21.15.64:0/3545396759 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604046030 con 0x7f760c0a6700 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.624+0000 7f75fbfff640 1 -- 172.21.15.64:0/3545396759 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76040049d0 con 0x7f760c0a6700 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.624+0000 7f75fbfff640 1 -- 172.21.15.64:0/3545396759 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604003850 con 0x7f760c0a6700 2024-05-09T09:16:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.624+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c142c60 con 0x7f760c0a6700 2024-05-09T09:16:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.624+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c143100 con 0x7f760c0a6700 2024-05-09T09:16:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.628+0000 7f761c56b740 1 Processor -- start 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.629+0000 7f75fbfff640 1 -- 172.21.15.64:0/3545396759 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76040409a0 con 0x7f760c0a6700 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.629+0000 7f75fbfff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75fc0a2d60 0x7f75fc0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f75fbfff640 1 -- 172.21.15.64:0/3545396759 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7604100970 con 0x7f760c0a6700 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 -- start start 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x555564042500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x5555640475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555563fa6d00 con 0x555564047c40 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555563f4e4c0 con 0x555564042ad0 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.630+0000 7f761c56b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555563fa2cc0 con 0x5555640420e0 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.631+0000 7f76127fc640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75fc0a2d60 0x7f75fc0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.631+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x555564042500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.631+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x555564042500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39864/0 (socket says 172.21.15.64:39864) 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.631+0000 7f7612ffd640 1 -- 172.21.15.64:0/270828935 learned_addr learned my addr 172.21.15.64:0/270828935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.631+0000 7f76127fc640 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x5555640475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.632+0000 7f76137fe640 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.632+0000 7f76127fc640 1 -- 172.21.15.64:0/270828935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 msgr2=0x555564042500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.632+0000 7f76127fc640 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x555564042500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.632+0000 7f76127fc640 1 -- 172.21.15.64:0/270828935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 msgr2=0x55556404a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.632+0000 7f76127fc640 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.632+0000 7f76127fc640 1 -- 172.21.15.64:0/270828935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555563d847d0 con 0x555564042ad0 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.633+0000 7f76127fc640 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x5555640475d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7608005d00 tx=0x7f7608014d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfbfb52c872dfd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.633+0000 7f76127fc640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75fc0a2d60 0x7f75fc0a5220 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f760c13c2c0 tx=0x7f760800d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.633+0000 7f75f97fa640 1 -- 172.21.15.64:0/270828935 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608021020 con 0x555564042ad0 2024-05-09T09:16:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.633+0000 7f75f97fa640 1 -- 172.21.15.64:0/270828935 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f760800b8e0 con 0x555564042ad0 2024-05-09T09:16:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.633+0000 7f75f97fa640 1 -- 172.21.15.64:0/270828935 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608005390 con 0x555564042ad0 2024-05-09T09:16:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 -- 172.21.15.64:0/270828935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 msgr2=0x5555640475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x5555640475d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7608005d00 tx=0x7f7608014d50 comp rx=0 tx=0).stop 2024-05-09T09:16:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 -- 172.21.15.64:0/270828935 shutdown_connections 2024-05-09T09:16:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x5555640475d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 --2- 172.21.15.64:0/270828935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x555564042500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 -- 172.21.15.64:0/270828935 >> 172.21.15.64:0/270828935 conn(0x55556403d750 msgr2=0x55556403db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 -- 172.21.15.64:0/270828935 shutdown_connections 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.634+0000 7f761c56b740 1 -- 172.21.15.64:0/270828935 wait complete. 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 -- start start 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 Processor -- start 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556404c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x55556404cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564047c40 0x555564053b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55556404ab90 con 0x555564042ad0 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555563f61b90 con 0x555564047c40 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55556404ad10 con 0x5555640420e0 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.635+0000 7f761c56b740 10 client.0 _ll_register_callbacks cb 0x555563f9dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556404c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556404c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39876/0 (socket says 172.21.15.64:39876) 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 -- 172.21.15.64:0/394186890 learned_addr learned my addr 172.21.15.64:0/394186890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f76127fc640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x55556404cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f76137fe640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564047c40 0x555564053b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 -- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564047c40 msgr2=0x555564053b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564047c40 0x555564053b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 -- 172.21.15.64:0/394186890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 msgr2=0x55556404cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x55556404cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f7612ffd640 1 -- 172.21.15.64:0/394186890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555563cba3f0 con 0x5555640420e0 2024-05-09T09:16:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.636+0000 7f76127fc640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x55556404cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.637+0000 7f7612ffd640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556404c5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f761402d750 tx=0x7f7614075690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c543aea927f452af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.637+0000 7f75f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.637+0000 7f75f6ff5640 1 -- 172.21.15.64:0/394186890 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614078440 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.637+0000 7f75f6ff5640 1 -- 172.21.15.64:0/394186890 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f76140785e0 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.637+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760800b8e0 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f75f6ff5640 1 -- 172.21.15.64:0/394186890 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7614087050 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 10 client.27097 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555563c7a0f0 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f75f6ff5640 1 -- 172.21.15.64:0/394186890 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f760800b8e0 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f75f6ff5640 1 -- 172.21.15.64:0/394186890 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7614082880 con 0x5555640420e0 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 10 client.27097 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 10 client.27097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 20 client.27097 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555564058530 0x55556405a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 4 client.27097 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.638+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555563fb4430 con 0x555564058530 2024-05-09T09:16:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.639+0000 7f76127fc640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555564058530 0x55556405a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.639+0000 7f76127fc640 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555564058530 0x55556405a9a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd389ed8abe3b99 server_cookie=9e29b21e8e35c7be in_seq=0 out_seq=0 2024-05-09T09:16:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.639+0000 7f75f6ff5640 10 client.27097 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.640+0000 7f75f6ff5640 1 -- 172.21.15.64:0/394186890 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555563fb4430 con 0x555564058530 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.640+0000 7f75f6ff5640 10 client.27097 handle_command_reply: tid=0 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.640+0000 7f761c56b740 1 client.27097 shutdown 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.640+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 msgr2=0x55556404c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.640+0000 7f761c56b740 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556404c5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f761402d750 tx=0x7f7614075690 comp rx=0 tx=0).stop 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 shutdown_connections 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555564058530 0x55556405a9a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564047c40 0x555564053b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x55556404cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 --2- 172.21.15.64:0/394186890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556404c5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 >> 172.21.15.64:0/394186890 conn(0x55556403d750 msgr2=0x55556403ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f75f6ff5640 0 client.27097 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 shutdown_connections 2024-05-09T09:16:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.641+0000 7f761c56b740 1 -- 172.21.15.64:0/394186890 wait complete. 2024-05-09T09:16:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.642+0000 7f761c56b740 20 client.27097 trim_cache size 0 max 16384 2024-05-09T09:16:56.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.676+0000 7f761c56b740 1 Processor -- start 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.676+0000 7f761c56b740 1 -- start start 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.676+0000 7f761c56b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556403ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.676+0000 7f761c56b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x555564053b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.676+0000 7f761c56b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f761c56b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f760c14fee0 con 0x555564047c40 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f761c56b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f760c14d750 con 0x555564042ad0 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f761c56b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f760c14b5a0 con 0x5555640420e0 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f7612ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556403ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34490/0 (socket says 172.21.15.64:34490) 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 -- 172.21.15.64:0/3199484757 learned_addr learned my addr 172.21.15.64:0/3199484757 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:56.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 -- 172.21.15.64:0/3199484757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 msgr2=0x55556403ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556403ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 -- 172.21.15.64:0/3199484757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 msgr2=0x555564053b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x555564053b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.677+0000 7f76137fe640 1 -- 172.21.15.64:0/3199484757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556403d580 con 0x555564047c40 2024-05-09T09:16:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.678+0000 7f76137fe640 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404c8d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f76040ccae0 tx=0x7f760401a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d827e406e5538b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.678+0000 7f75f8ff9640 1 -- 172.21.15.64:0/3199484757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604058040 con 0x555564047c40 2024-05-09T09:16:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.678+0000 7f75f8ff9640 1 -- 172.21.15.64:0/3199484757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76040ccde0 con 0x555564047c40 2024-05-09T09:16:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.678+0000 7f75f8ff9640 1 -- 172.21.15.64:0/3199484757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76040d55c0 con 0x555564047c40 2024-05-09T09:16:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.680+0000 7f761c56b740 1 -- 172.21.15.64:0/3199484757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 msgr2=0x55556404c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.680+0000 7f761c56b740 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404c8d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f76040ccae0 tx=0x7f760401a270 comp rx=0 tx=0).stop 2024-05-09T09:16:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.680+0000 7f761c56b740 1 -- 172.21.15.64:0/3199484757 shutdown_connections 2024-05-09T09:16:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.681+0000 7f761c56b740 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564047c40 0x55556404c8d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.681+0000 7f761c56b740 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555564042ad0 0x555564053b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.681+0000 7f761c56b740 1 --2- 172.21.15.64:0/3199484757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555640420e0 0x55556403ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.681+0000 7f761c56b740 1 -- 172.21.15.64:0/3199484757 >> 172.21.15.64:0/3199484757 conn(0x55556403d750 msgr2=0x555564040430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.681+0000 7f761c56b740 1 -- 172.21.15.64:0/3199484757 shutdown_connections 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.681+0000 7f761c56b740 1 -- 172.21.15.64:0/3199484757 wait complete. 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.682+0000 7f761c56b740 1 -- start start 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.682+0000 7f761c56b740 1 Processor -- start 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.683+0000 7f761c56b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555640420e0 0x555564051970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.683+0000 7f761c56b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x555564051eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.683+0000 7f761c56b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 0x555564052ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.683+0000 7f76127fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x555564051eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.683+0000 7f76127fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x555564051eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34506/0 (socket says 172.21.15.64:34506) 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f76137fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 0x555564052ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f76137fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 0x555564052ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39894/0 (socket says 172.21.15.64:39894) 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f76127fc640 1 -- 172.21.15.64:0/2967490269 learned_addr learned my addr 172.21.15.64:0/2967490269 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f760c14bc10 con 0x555564042ad0 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f760c14d080 con 0x5555640420e0 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f760c14c2e0 con 0x555564047c40 2024-05-09T09:16:56.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.684+0000 7f761c56b740 10 client.0 _ll_register_callbacks cb 0x7f7614073240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f76137fe640 1 -- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555640420e0 msgr2=0x555564051970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f76137fe640 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555640420e0 0x555564051970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f76137fe640 1 -- 172.21.15.64:0/2967490269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 msgr2=0x555564051eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f76137fe640 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x555564051eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f76137fe640 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555564038f30 con 0x555564047c40 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f76137fe640 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 0x555564052ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f76040c97b0 tx=0x7f76040d57f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f041b3f7122b681 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f75f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:56.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.685+0000 7f75f6ff5640 1 -- 172.21.15.64:0/2967490269 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76040d9070 con 0x555564047c40 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f75f6ff5640 1 -- 172.21.15.64:0/2967490269 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f76040cc440 con 0x555564047c40 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76040ccde0 con 0x555564047c40 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f75f6ff5640 1 -- 172.21.15.64:0/2967490269 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7604058070 con 0x555564047c40 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 10 client.27103 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5555640bf060 con 0x555564047c40 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f75f6ff5640 1 -- 172.21.15.64:0/2967490269 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76040ccde0 con 0x555564047c40 2024-05-09T09:16:56.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f75f6ff5640 1 -- 172.21.15.64:0/2967490269 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f76040dc030 con 0x555564047c40 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 10 client.27103 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 10 client.27103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 20 client.27103 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5555640bf960 0x5555640c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 4 client.27103 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.686+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555563fae780 con 0x5555640bf960 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.687+0000 7f7612ffd640 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5555640bf960 0x5555640c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.688+0000 7f7612ffd640 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5555640bf960 0x5555640c1d80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae6027959c33da4 server_cookie=f6613456f5e57502 in_seq=0 out_seq=0 2024-05-09T09:16:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.688+0000 7f75f6ff5640 10 client.27103 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.688+0000 7f75f6ff5640 1 -- 172.21.15.64:0/2967490269 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555563fae780 con 0x5555640bf960 2024-05-09T09:16:56.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.688+0000 7f75f6ff5640 10 client.27103 handle_command_reply: tid=0 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.688+0000 7f761c56b740 1 client.27103 shutdown 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 msgr2=0x555564052ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 0x555564052ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f76040c97b0 tx=0x7f76040d57f0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 shutdown_connections 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5555640bf960 0x5555640c1d80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555564047c40 0x555564052ec0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555564042ad0 0x555564051eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 --2- 172.21.15.64:0/2967490269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555640420e0 0x555564051970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 >> 172.21.15.64:0/2967490269 conn(0x55556403d750 msgr2=0x555564040cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f75f6ff5640 0 client.27103 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 shutdown_connections 2024-05-09T09:16:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 1 -- 172.21.15.64:0/2967490269 wait complete. 2024-05-09T09:16:56.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.689+0000 7f761c56b740 20 client.27103 trim_cache size 0 max 16384 2024-05-09T09:16:56.697 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:16:56.698 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:16:56.698 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:16:56.698 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75fc0a2d60 msgr2=0x7f75fc0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75fc0a2d60 0x7f75fc0a5220 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f760c13c2c0 tx=0x7f760800d000 comp rx=0 tx=0).stop 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a6700 msgr2=0x7f760c142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a6700 0x7f760c142360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f760403b970 tx=0x7f7604005380 comp rx=0 tx=0).stop 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 shutdown_connections 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f75fc0a2d60 0x7f75fc0a5220 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f760c0a6700 0x7f760c142360 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c0a5d60 0x7f760c13b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 --2- 172.21.15.64:0/3545396759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f760c0a4c30 0x7f760c13ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.692+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 >> 172.21.15.64:0/3545396759 conn(0x7f760c0a00c0 msgr2=0x7f760c0a8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.694+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 shutdown_connections 2024-05-09T09:16:56.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.694+0000 7f7613fff640 1 -- 172.21.15.64:0/3545396759 wait complete. 2024-05-09T09:16:56.712 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:16:56.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:16:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:56 smithi064 ceph-mon[30149]: pgmap v604: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 883 op/s 2024-05-09T09:16:56.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.911+0000 7f16d7fe7640 1 -- 172.21.15.64:0/1991891648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 msgr2=0x7f16d0108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.911+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/1991891648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d0108c30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009ca0 tx=0x7f16c802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 -- 172.21.15.64:0/1991891648 shutdown_connections 2024-05-09T09:16:56.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/1991891648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d0108c30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/1991891648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 0x7f16d0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/1991891648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d01007f0 0x7f16d0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 -- 172.21.15.64:0/1991891648 >> 172.21.15.64:0/1991891648 conn(0x7f16d00fbd80 msgr2=0x7f16d00fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:56.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 -- 172.21.15.64:0/1991891648 shutdown_connections 2024-05-09T09:16:56.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.912+0000 7f16d7fe7640 1 -- 172.21.15.64:0/1991891648 wait complete. 2024-05-09T09:16:56.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.914+0000 7f16d7fe7640 1 Processor -- start 2024-05-09T09:16:56.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.914+0000 7f16d7fe7640 1 -- start start 2024-05-09T09:16:56.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.914+0000 7f16d7fe7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d01007f0 0x7f16d0197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.914+0000 7f16d7fe7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 0x7f16d0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d7fe7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d7fe7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d390 con 0x7f16d01023a0 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d6584640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d6584640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34516/0 (socket says 172.21.15.64:34516) 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d6584640 1 -- 172.21.15.64:0/2427850277 learned_addr learned my addr 172.21.15.64:0/2427850277 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d5d83640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d01007f0 0x7f16d0197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d5582640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 0x7f16d0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d210 con 0x7f16d01019e0 2024-05-09T09:16:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.915+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d510 con 0x7f16d01007f0 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d6584640 1 -- 172.21.15.64:0/2427850277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d01007f0 msgr2=0x7f16d0197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d6584640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d01007f0 0x7f16d0197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d6584640 1 -- 172.21.15.64:0/2427850277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 msgr2=0x7f16d0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d6584640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 0x7f16d0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d6584640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019f5a0 con 0x7f16d01023a0 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d5582640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 0x7f16d0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.916+0000 7f16d6584640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d019ef30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f16c8031040 tx=0x7f16c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.917+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004040 con 0x7f16d01023a0 2024-05-09T09:16:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.917+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8004690 con 0x7f16d01023a0 2024-05-09T09:16:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.917+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8030c70 con 0x7f16d01023a0 2024-05-09T09:16:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.919+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019f830 con 0x7f16d01023a0 2024-05-09T09:16:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.919+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d019fc40 con 0x7f16d01023a0 2024-05-09T09:16:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.921+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d0107b10 con 0x7f16d01023a0 2024-05-09T09:16:56.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.923+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f16c80041e0 con 0x7f16d01023a0 2024-05-09T09:16:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.924+0000 7f16c6ffd640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16bc0a2d10 0x7f16bc0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.924+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f16c80f08b0 con 0x7f16d01023a0 2024-05-09T09:16:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.924+0000 7f16d5d83640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16bc0a2d10 0x7f16bc0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.925+0000 7f16d5d83640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16bc0a2d10 0x7f16bc0a51d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f16d0101840 tx=0x7f16c000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:56.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:56.932+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16c80bd480 con 0x7f16d01023a0 2024-05-09T09:16:56.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:56 smithi179 ceph-mon[34086]: pgmap v604: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 883 op/s 2024-05-09T09:16:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:56 smithi139 ceph-mon[33599]: pgmap v604: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 883 op/s 2024-05-09T09:16:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.393+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d0062430 con 0x7f16d01023a0 2024-05-09T09:16:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.395+0000 7f16c6ffd640 1 -- 172.21.15.64:0/2427850277 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11882 (secure 0 0 0) 0x7f16c80bcd80 con 0x7f16d01023a0 2024-05-09T09:16:57.395 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:16:57.396 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:16:15.088273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:16:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 43 2024-05-09T09:16:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16bc0a2d10 msgr2=0x7f16bc0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16bc0a2d10 0x7f16bc0a51d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f16d0101840 tx=0x7f16c000b390 comp rx=0 tx=0).stop 2024-05-09T09:16:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 msgr2=0x7f16d019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d019ef30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f16c8031040 tx=0x7f16c8003ba0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 shutdown_connections 2024-05-09T09:16:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16bc0a2d10 0x7f16bc0a51d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d01023a0 0x7f16d019ef30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16d01019e0 0x7f16d0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 --2- 172.21.15.64:0/2427850277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16d01007f0 0x7f16d0197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.400+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 >> 172.21.15.64:0/2427850277 conn(0x7f16d00fbd80 msgr2=0x7f16d0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.401+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 shutdown_connections 2024-05-09T09:16:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.401+0000 7f16d7fe7640 1 -- 172.21.15.64:0/2427850277 wait complete. 2024-05-09T09:16:57.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:16:57.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 -- 172.21.15.64:0/1824677659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410db70 msgr2=0x7f8e8410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 --2- 172.21.15.64:0/1824677659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410db70 0x7f8e8410e020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009860 tx=0x7f8e7c02a360 comp rx=0 tx=0).stop 2024-05-09T09:16:57.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 -- 172.21.15.64:0/1824677659 shutdown_connections 2024-05-09T09:16:57.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 --2- 172.21.15.64:0/1824677659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410db70 0x7f8e8410e020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 --2- 172.21.15.64:0/1824677659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410b1a0 0x7f8e8410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 --2- 172.21.15.64:0/1824677659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 0x7f8e8410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.623+0000 7f8e8b530640 1 -- 172.21.15.64:0/1824677659 >> 172.21.15.64:0/1824677659 conn(0x7f8e8406e6c0 msgr2=0x7f8e84070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.624+0000 7f8e8b530640 1 -- 172.21.15.64:0/1824677659 shutdown_connections 2024-05-09T09:16:57.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.624+0000 7f8e8b530640 1 -- 172.21.15.64:0/1824677659 wait complete. 2024-05-09T09:16:57.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.624+0000 7f8e8b530640 1 Processor -- start 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.624+0000 7f8e8b530640 1 -- start start 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e8b530640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 0x7f8e8419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e8b530640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 0x7f8e8419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e8b530640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410db70 0x7f8e841a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e8b530640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841119a0 con 0x7f8e8410db70 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e8b530640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84111820 con 0x7f8e8410b1a0 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e8b530640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84111b20 con 0x7f8e84108830 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e88acb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 0x7f8e8419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 0x7f8e8419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e88acb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 0x7f8e8419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49196/0 (socket says 172.21.15.64:49196) 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e88acb640 1 -- 172.21.15.64:0/2403282079 learned_addr learned my addr 172.21.15.64:0/2403282079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.625+0000 7f8e89acd640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410db70 0x7f8e841a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e892cc640 1 -- 172.21.15.64:0/2403282079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 msgr2=0x7f8e8419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e892cc640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 0x7f8e8419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e892cc640 1 -- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410db70 msgr2=0x7f8e841a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e892cc640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410db70 0x7f8e841a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e892cc640 1 -- 172.21.15.64:0/2403282079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e841a5cd0 con 0x7f8e84108830 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e88acb640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 0x7f8e8419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e89acd640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410db70 0x7f8e841a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e892cc640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 0x7f8e8419e0b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e9f0 tx=0x7f8e7400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e827fc640 1 -- 172.21.15.64:0/2403282079 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400cd80 con 0x7f8e84108830 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e827fc640 1 -- 172.21.15.64:0/2403282079 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e74011920 con 0x7f8e84108830 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e827fc640 1 -- 172.21.15.64:0/2403282079 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74010980 con 0x7f8e84108830 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e841a5fc0 con 0x7f8e84108830 2024-05-09T09:16:57.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.626+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e841a63d0 con 0x7f8e84108830 2024-05-09T09:16:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e827fc640 1 -- 172.21.15.64:0/2403282079 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e74011ad0 con 0x7f8e84108830 2024-05-09T09:16:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 Processor -- start 2024-05-09T09:16:57.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 -- start start 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 0x563a36fc1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fc6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a36f25c40 con 0x563a36fc6aa0 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563a36ecd580 con 0x563a36fc1930 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a36f21c00 con 0x563a36fc0f40 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.631+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fc6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e88acb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fc6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49210/0 (socket says 172.21.15.64:49210) 2024-05-09T09:16:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 -- 172.21.15.64:0/3886420137 learned_addr learned my addr 172.21.15.64:0/3886420137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 -- 172.21.15.64:0/3886420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 msgr2=0x563a36fc1360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 0x563a36fc1360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 -- 172.21.15.64:0/3886420137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 msgr2=0x563a36fc8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fc8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.632+0000 7f8e892cc640 1 -- 172.21.15.64:0/3886420137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a36c393f0 con 0x563a36fc1930 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.633+0000 7f8e892cc640 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fc6430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e740d5c10 tx=0x7f8e740d5d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed62536bef3b13f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.633+0000 7f8e827fc640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e6c0a2c60 0x7f8e6c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.634+0000 7f8e827fc640 1 -- 172.21.15.64:0/2403282079 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e740cf970 con 0x7f8e84108830 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.634+0000 7f8e6b7fe640 1 -- 172.21.15.64:0/3886420137 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e740de070 con 0x563a36fc1930 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.634+0000 7f8e89acd640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e6c0a2c60 0x7f8e6c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.634+0000 7f8e6b7fe640 1 -- 172.21.15.64:0/3886420137 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e74011e20 con 0x563a36fc1930 2024-05-09T09:16:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.634+0000 7f8e6b7fe640 1 -- 172.21.15.64:0/3886420137 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7409b980 con 0x563a36fc1930 2024-05-09T09:16:57.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 -- 172.21.15.64:0/3886420137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 msgr2=0x563a36fc6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fc6430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e740d5c10 tx=0x7f8e740d5d00 comp rx=0 tx=0).stop 2024-05-09T09:16:57.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e89acd640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e6c0a2c60 0x7f8e6c0a5120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009830 tx=0x7f8e7c030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 -- 172.21.15.64:0/3886420137 shutdown_connections 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fc8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fc6430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/3886420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 0x563a36fc1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 -- 172.21.15.64:0/3886420137 >> 172.21.15.64:0/3886420137 conn(0x563a36fbc5b0 msgr2=0x563a36fbc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.635+0000 7f8e8e23e740 1 -- 172.21.15.64:0/3886420137 shutdown_connections 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.636+0000 7f8e8e23e740 1 -- 172.21.15.64:0/3886420137 wait complete. 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.636+0000 7f8e8e23e740 1 -- start start 2024-05-09T09:16:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.637+0000 7f8e8e23e740 1 Processor -- start 2024-05-09T09:16:57.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 0x563a36fcb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc1930 0x563a36fcb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fd2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a36fc9aa0 con 0x563a36fc6aa0 2024-05-09T09:16:57.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563a36ee0c40 con 0x563a36fc0f40 2024-05-09T09:16:57.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a36fc9c20 con 0x563a36fc1930 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e8e23e740 10 client.0 _ll_register_callbacks cb 0x563a36c38be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 0x563a36fcb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 0x563a36fcb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49220/0 (socket says 172.21.15.64:49220) 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.638+0000 7f8e892cc640 1 -- 172.21.15.64:0/2326033105 learned_addr learned my addr 172.21.15.64:0/2326033105 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e89acd640 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fd2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e892cc640 1 -- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc1930 msgr2=0x563a36fcb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e892cc640 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc1930 0x563a36fcb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e892cc640 1 -- 172.21.15.64:0/2326033105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 msgr2=0x563a36fd2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e892cc640 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fd2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e892cc640 1 -- 172.21.15.64:0/2326033105 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a36d037d0 con 0x563a36fc0f40 2024-05-09T09:16:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.639+0000 7f8e892cc640 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 0x563a36fcb410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e7401c040 tx=0x7f8e7401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f81eaeddec5df6d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e68ff9640 1 -- 172.21.15.64:0/2326033105 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7409e060 con 0x563a36fc0f40 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a36f33370 con 0x563a36fc0f40 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e68ff9640 1 -- 172.21.15.64:0/2326033105 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e7409a140 con 0x563a36fc0f40 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e68ff9640 1 -- 172.21.15.64:0/2326033105 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8e740de070 con 0x563a36fc0f40 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e8e23e740 10 client.26942 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.640+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563a36fd5d00 con 0x563a36fc0f40 2024-05-09T09:16:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.641+0000 7f8e68ff9640 1 -- 172.21.15.64:0/2326033105 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563a36f33370 con 0x563a36fc0f40 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e68ff9640 1 -- 172.21.15.64:0/2326033105 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8e7400f040 con 0x563a36fc0f40 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e8e23e740 10 client.26942 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e8e23e740 10 client.26942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e8e23e740 20 client.26942 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a36fd7050 0x563a36fd9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e8e23e740 4 client.26942 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.643+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a36f2d6c0 con 0x563a36fd7050 2024-05-09T09:16:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.645+0000 7f8e88acb640 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a36fd7050 0x563a36fd9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.648+0000 7f8e88acb640 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a36fd7050 0x563a36fd9440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e2af676b48c912f server_cookie=652f7f448ddd0ab in_seq=0 out_seq=0 2024-05-09T09:16:57.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.648+0000 7f8e68ff9640 10 client.26942 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:57.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.649+0000 7f8e68ff9640 1 -- 172.21.15.64:0/2326033105 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a36f2d6c0 con 0x563a36fd7050 2024-05-09T09:16:57.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.649+0000 7f8e68ff9640 10 client.26942 handle_command_reply: tid=0 2024-05-09T09:16:57.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.649+0000 7f8e8e23e740 1 client.26942 shutdown 2024-05-09T09:16:57.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.649+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 msgr2=0x563a36fcb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.649+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 0x563a36fcb410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e7401c040 tx=0x7f8e7401e000 comp rx=0 tx=0).stop 2024-05-09T09:16:57.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 shutdown_connections 2024-05-09T09:16:57.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a36fd7050 0x563a36fd9440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fd2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc1930 0x563a36fcb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/2326033105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc0f40 0x563a36fcb410 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 >> 172.21.15.64:0/2326033105 conn(0x563a36fbc5b0 msgr2=0x563a36fbb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.652+0000 7f8e68ff9640 0 client.26942 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:57.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.654+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 shutdown_connections 2024-05-09T09:16:57.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.655+0000 7f8e8e23e740 1 -- 172.21.15.64:0/2326033105 wait complete. 2024-05-09T09:16:57.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.655+0000 7f8e8e23e740 20 client.26942 trim_cache size 0 max 16384 2024-05-09T09:16:57.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 Processor -- start 2024-05-09T09:16:57.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 -- start start 2024-05-09T09:16:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 0x563a36fcb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fcb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841b32d0 con 0x563a36fc6aa0 2024-05-09T09:16:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841b0b40 con 0x563a36fc1930 2024-05-09T09:16:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.684+0000 7f8e8e23e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841aea40 con 0x563a36fc0f40 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fcb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e88acb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fcb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34566/0 (socket says 172.21.15.64:34566) 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 -- 172.21.15.64:0/327336141 learned_addr learned my addr 172.21.15.64:0/327336141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 -- 172.21.15.64:0/327336141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 msgr2=0x563a36fcb020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 0x563a36fcb020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 -- 172.21.15.64:0/327336141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 msgr2=0x563a36fd2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.685+0000 7f8e89acd640 1 -- 172.21.15.64:0/327336141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a36fadd20 con 0x563a36fc6aa0 2024-05-09T09:16:57.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.686+0000 7f8e89acd640 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fcb560 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c030e80 tx=0x7f8e7c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b99c78c0b99167 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.686+0000 7f8e6affd640 1 -- 172.21.15.64:0/327336141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c04d070 con 0x563a36fc6aa0 2024-05-09T09:16:57.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.686+0000 7f8e6affd640 1 -- 172.21.15.64:0/327336141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c0054f0 con 0x563a36fc6aa0 2024-05-09T09:16:57.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.686+0000 7f8e6affd640 1 -- 172.21.15.64:0/327336141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c037040 con 0x563a36fc6aa0 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 -- 172.21.15.64:0/327336141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 msgr2=0x563a36fcb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fcb560 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c030e80 tx=0x7f8e7c03b000 comp rx=0 tx=0).stop 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 -- 172.21.15.64:0/327336141 shutdown_connections 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc6aa0 0x563a36fcb560 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/327336141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc0f40 0x563a36fcb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.687+0000 7f8e8e23e740 1 -- 172.21.15.64:0/327336141 >> 172.21.15.64:0/327336141 conn(0x563a36fbc5b0 msgr2=0x563a36fc2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.688+0000 7f8e8e23e740 1 -- 172.21.15.64:0/327336141 shutdown_connections 2024-05-09T09:16:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.688+0000 7f8e8e23e740 1 -- 172.21.15.64:0/327336141 wait complete. 2024-05-09T09:16:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.688+0000 7f8e8e23e740 1 -- start start 2024-05-09T09:16:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.688+0000 7f8e8e23e740 1 Processor -- start 2024-05-09T09:16:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 0x563a36fcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc6aa0 0x563a36fd1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 0x563a36fcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e892cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 0x563a36fcfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34570/0 (socket says 172.21.15.64:34570) 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841af000 con 0x563a36fc0f40 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841b0470 con 0x563a36fc1930 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e841af6d0 con 0x563a36fc6aa0 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e8e23e740 10 client.0 _ll_register_callbacks cb 0x7f8e740d5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e88acb640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e89acd640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc6aa0 0x563a36fd1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.689+0000 7f8e892cc640 1 -- 172.21.15.64:0/995390113 learned_addr learned my addr 172.21.15.64:0/995390113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e89acd640 1 -- 172.21.15.64:0/995390113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 msgr2=0x563a36fd0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e89acd640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e89acd640 1 -- 172.21.15.64:0/995390113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 msgr2=0x563a36fcfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e89acd640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 0x563a36fcfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e89acd640 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a36fd1650 con 0x563a36fc6aa0 2024-05-09T09:16:57.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e892cc640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 0x563a36fcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:16:57.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e89acd640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc6aa0 0x563a36fd1060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02a950 tx=0x7f8e7c004d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de057e6efd4ccd50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.690+0000 7f8e68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e68ff9640 1 -- 172.21.15.64:0/995390113 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c059070 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e68ff9640 1 -- 172.21.15.64:0/995390113 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8e7c0035f0 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e7c0054f0 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e68ff9640 1 -- 172.21.15.64:0/995390113 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c065050 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e8e23e740 10 client.27118 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563a36fd2950 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e68ff9640 1 -- 172.21.15.64:0/995390113 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c0054f0 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e68ff9640 1 -- 172.21.15.64:0/995390113 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8e7c033450 con 0x563a36fc6aa0 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e8e23e740 10 client.27118 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e8e23e740 10 client.27118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.691+0000 7f8e8e23e740 20 client.27118 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.692+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a3703d7d0 0x563a3703fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.692+0000 7f8e8e23e740 4 client.27118 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:16:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.692+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a36eaa820 con 0x563a3703d7d0 2024-05-09T09:16:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.692+0000 7f8e892cc640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a3703d7d0 0x563a3703fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.695+0000 7f8e892cc640 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a3703d7d0 0x563a3703fbc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f9ea6eb9348cf5f server_cookie=4f7b415e0c15574c in_seq=0 out_seq=0 2024-05-09T09:16:57.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.695+0000 7f8e68ff9640 10 client.27118 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:57.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.696+0000 7f8e68ff9640 1 -- 172.21.15.64:0/995390113 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a36eaa820 con 0x563a3703d7d0 2024-05-09T09:16:57.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.696+0000 7f8e68ff9640 10 client.27118 handle_command_reply: tid=0 2024-05-09T09:16:57.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.696+0000 7f8e8e23e740 1 client.27118 shutdown 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.696+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc6aa0 msgr2=0x563a36fd1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.696+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc6aa0 0x563a36fd1060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02a950 tx=0x7f8e7c004d40 comp rx=0 tx=0).stop 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 shutdown_connections 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563a3703d7d0 0x563a3703fbc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e68ff9640 0 client.27118 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a36fc6aa0 0x563a36fd1060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a36fc1930 0x563a36fd0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 --2- 172.21.15.64:0/995390113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a36fc0f40 0x563a36fcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 >> 172.21.15.64:0/995390113 conn(0x563a36fbc5b0 msgr2=0x563a36fcb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 shutdown_connections 2024-05-09T09:16:57.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 1 -- 172.21.15.64:0/995390113 wait complete. 2024-05-09T09:16:57.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.697+0000 7f8e8e23e740 20 client.27118 trim_cache size 0 max 16384 2024-05-09T09:16:57.698 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:16:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.699+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e6c0a2c60 msgr2=0x7f8e6c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.699+0000 7f8e8b530640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e6c0a2c60 0x7f8e6c0a5120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009830 tx=0x7f8e7c030470 comp rx=0 tx=0).stop 2024-05-09T09:16:57.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.699+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 msgr2=0x7f8e8419e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.699+0000 7f8e8b530640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 0x7f8e8419e0b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e9f0 tx=0x7f8e7400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.701+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 shutdown_connections 2024-05-09T09:16:57.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.701+0000 7f8e8b530640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8e6c0a2c60 0x7f8e6c0a5120 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.701+0000 7f8e8b530640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8410db70 0x7f8e841a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.701+0000 7f8e8b530640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e8410b1a0 0x7f8e8419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.701+0000 7f8e8b530640 1 --2- 172.21.15.64:0/2403282079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e84108830 0x7f8e8419e0b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.701+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 >> 172.21.15.64:0/2403282079 conn(0x7f8e8406e6c0 msgr2=0x7f8e8410bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.704+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 shutdown_connections 2024-05-09T09:16:57.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.704+0000 7f8e8b530640 1 -- 172.21.15.64:0/2403282079 wait complete. 2024-05-09T09:16:57.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:16:57.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.925+0000 7f3be9e12640 1 -- 172.21.15.64:0/830096037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 msgr2=0x7f3be4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.925+0000 7f3be9e12640 1 --2- 172.21.15.64:0/830096037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 0x7f3be4108bb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009900 tx=0x7f3bd402a410 comp rx=0 tx=0).stop 2024-05-09T09:16:57.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 -- 172.21.15.64:0/830096037 shutdown_connections 2024-05-09T09:16:57.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 --2- 172.21.15.64:0/830096037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 0x7f3be4108bb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 --2- 172.21.15.64:0/830096037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4101960 0x7f3be4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 --2- 172.21.15.64:0/830096037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be4100770 0x7f3be4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 -- 172.21.15.64:0/830096037 >> 172.21.15.64:0/830096037 conn(0x7f3be40fbd20 msgr2=0x7f3be40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 -- 172.21.15.64:0/830096037 shutdown_connections 2024-05-09T09:16:57.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.927+0000 7f3be9e12640 1 -- 172.21.15.64:0/830096037 wait complete. 2024-05-09T09:16:57.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 Processor -- start 2024-05-09T09:16:57.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 -- start start 2024-05-09T09:16:57.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 0x7f3be4104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be4101960 0x7f3be4102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 0x7f3be4103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3be410c2f0 con 0x7f3be4100770 2024-05-09T09:16:57.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3be410c170 con 0x7f3be4101960 2024-05-09T09:16:57.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be9e12640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3be410c470 con 0x7f3be4102320 2024-05-09T09:16:57.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.928+0000 7f3be37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 0x7f3be4104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 0x7f3be4104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34586/0 (socket says 172.21.15.64:34586) 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 -- 172.21.15.64:0/1481242582 learned_addr learned my addr 172.21.15.64:0/1481242582 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be3fff640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 0x7f3be4103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 -- 172.21.15.64:0/1481242582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 msgr2=0x7f3be4103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 0x7f3be4103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 -- 172.21.15.64:0/1481242582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be4101960 msgr2=0x7f3be4102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be4101960 0x7f3be4102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 -- 172.21.15.64:0/1481242582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be4103b10 con 0x7f3be4100770 2024-05-09T09:16:57.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.929+0000 7f3be37fe640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 0x7f3be4104700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00eaa0 tx=0x7f3bdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.930+0000 7f3be0ff9640 1 -- 172.21.15.64:0/1481242582 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc003b70 con 0x7f3be4100770 2024-05-09T09:16:57.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.930+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be41976d0 con 0x7f3be4100770 2024-05-09T09:16:57.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.931+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be4197c20 con 0x7f3be4100770 2024-05-09T09:16:57.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.931+0000 7f3be0ff9640 1 -- 172.21.15.64:0/1481242582 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bdc00cd90 con 0x7f3be4100770 2024-05-09T09:16:57.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.931+0000 7f3be0ff9640 1 -- 172.21.15.64:0/1481242582 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0047a0 con 0x7f3be4100770 2024-05-09T09:16:57.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.936+0000 7f3becbfc740 1 Processor -- start 2024-05-09T09:16:57.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.936+0000 7f3becbfc740 1 -- start start 2024-05-09T09:16:57.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708f4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da708f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3becbfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da70858c40 con 0x55da708f9aa0 2024-05-09T09:16:57.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3becbfc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55da70800580 con 0x55da708f4930 2024-05-09T09:16:57.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3becbfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55da70854c00 con 0x55da708f3f40 2024-05-09T09:16:57.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34598/0 (socket says 172.21.15.64:34598) 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.937+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2639757879 learned_addr learned my addr 172.21.15.64:0/2639757879 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.938+0000 7f3be0ff9640 1 -- 172.21.15.64:0/1481242582 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3bdc01f3f0 con 0x7f3be4100770 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2639757879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 msgr2=0x55da708f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be37fe640 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da708f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2639757879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 msgr2=0x55da708f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da708f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2639757879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da7056c3f0 con 0x55da708f9aa0 2024-05-09T09:16:57.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.939+0000 7f3be37fe640 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da708f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:16:57.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.940+0000 7f3be0ff9640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bcc0a2c10 0x7f3bcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.940+0000 7f3be0ff9640 1 -- 172.21.15.64:0/1481242582 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3bdc0d7ac0 con 0x7f3be4100770 2024-05-09T09:16:57.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.940+0000 7f3be3fff640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bcc0a2c10 0x7f3bcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.941+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708fbf30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009b20 tx=0x7f3bd800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acc31845c5dcd00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.943+0000 7f3bc9ffb640 1 -- 172.21.15.64:0/2639757879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800ed80 con 0x55da708f9aa0 2024-05-09T09:16:57.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.944+0000 7f3bc9ffb640 1 -- 172.21.15.64:0/2639757879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bd80108e0 con 0x55da708f9aa0 2024-05-09T09:16:57.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.944+0000 7f3be3fff640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bcc0a2c10 0x7f3bcc0a50d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005c40 tx=0x7f3bd40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.945+0000 7f3becbfc740 1 -- 172.21.15.64:0/2639757879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 msgr2=0x55da708fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.945+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708fbf30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009b20 tx=0x7f3bd800e350 comp rx=0 tx=0).stop 2024-05-09T09:16:57.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.946+0000 7f3becbfc740 1 -- 172.21.15.64:0/2639757879 shutdown_connections 2024-05-09T09:16:57.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.946+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708fbf30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.946+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da708f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.946+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2639757879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.946+0000 7f3becbfc740 1 -- 172.21.15.64:0/2639757879 >> 172.21.15.64:0/2639757879 conn(0x55da708ef5b0 msgr2=0x55da708ef9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.947+0000 7f3becbfc740 1 -- 172.21.15.64:0/2639757879 shutdown_connections 2024-05-09T09:16:57.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.947+0000 7f3becbfc740 1 -- 172.21.15.64:0/2639757879 wait complete. 2024-05-09T09:16:57.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.947+0000 7f3becbfc740 1 -- start start 2024-05-09T09:16:57.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.947+0000 7f3becbfc740 1 Processor -- start 2024-05-09T09:16:57.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f4930 0x55da708fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 0x55da70905840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da708fcaa0 con 0x55da708f4930 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55da70813c40 con 0x55da708f9aa0 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55da708fcc20 con 0x55da708f3f40 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3becbfc740 10 client.0 _ll_register_callbacks cb 0x55da7056bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 0x55da70905840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 0x55da70905840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49276/0 (socket says 172.21.15.64:49276) 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3be3fff640 1 -- 172.21.15.64:0/2935640200 learned_addr learned my addr 172.21.15.64:0/2935640200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.948+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f4930 0x55da708fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3be3fff640 1 -- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 msgr2=0x55da708fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3be3fff640 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3be3fff640 1 -- 172.21.15.64:0/2935640200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f4930 msgr2=0x55da708fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3be3fff640 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f4930 0x55da708fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3be3fff640 1 -- 172.21.15.64:0/2935640200 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da706367d0 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3be3fff640 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 0x55da70905840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4033000 tx=0x7f3bd4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f84bfff4aba6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2935640200 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd404d070 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd80108e0 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2935640200 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3bd4004b90 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.949+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2935640200 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3bd4059050 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 10 client.26954 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55da70919c20 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2935640200 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bd80108e0 con 0x55da708f9aa0 2024-05-09T09:16:57.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2935640200 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3bd4032730 con 0x55da708f9aa0 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 10 client.26954 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 10 client.26954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 20 client.26954 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7091b320 0x55da7091d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.950+0000 7f3becbfc740 4 client.26954 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.951+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da70866370 con 0x55da7091b320 2024-05-09T09:16:57.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.951+0000 7f3be37fe640 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7091b320 0x55da7091d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.954+0000 7f3be37fe640 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7091b320 0x55da7091d7c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a49f54799e4b4dac server_cookie=41485b6165ae5699 in_seq=0 out_seq=0 2024-05-09T09:16:57.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.954+0000 7f3bc2ffd640 10 client.26954 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:57.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.954+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2935640200 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da70866370 con 0x55da7091b320 2024-05-09T09:16:57.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.955+0000 7f3bc2ffd640 10 client.26954 handle_command_reply: tid=0 2024-05-09T09:16:57.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.955+0000 7f3becbfc740 1 client.26954 shutdown 2024-05-09T09:16:57.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 msgr2=0x55da70905840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 0x55da70905840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4033000 tx=0x7f3bd4009260 comp rx=0 tx=0).stop 2024-05-09T09:16:57.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 shutdown_connections 2024-05-09T09:16:57.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f9aa0 0x55da70905840 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7091b320 0x55da7091d7c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f4930 0x55da708fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2935640200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f3f40 0x55da708fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 >> 172.21.15.64:0/2935640200 conn(0x55da708ef5b0 msgr2=0x55da708ee7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.956+0000 7f3bc2ffd640 0 client.26954 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:57.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.957+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 shutdown_connections 2024-05-09T09:16:57.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.957+0000 7f3becbfc740 1 -- 172.21.15.64:0/2935640200 wait complete. 2024-05-09T09:16:57.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.957+0000 7f3becbfc740 20 client.26954 trim_cache size 0 max 16384 2024-05-09T09:16:57.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2427850277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:16:57.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.984+0000 7f3becbfc740 1 Processor -- start 2024-05-09T09:16:57.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 -- start start 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f3f40 0x55da708fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 0x55da70905800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd0012580 con 0x55da708f9aa0 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd000fdf0 con 0x55da708f3f40 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3becbfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd000dc40 con 0x55da708f4930 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 0x55da70905800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.985+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 0x55da70905800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40018/0 (socket says 172.21.15.64:40018) 2024-05-09T09:16:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 -- 172.21.15.64:0/4038245349 learned_addr learned my addr 172.21.15.64:0/4038245349 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be3fff640 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 -- 172.21.15.64:0/4038245349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f3f40 msgr2=0x55da708fdcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f3f40 0x55da708fdcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 -- 172.21.15.64:0/4038245349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 msgr2=0x55da708f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 -- 172.21.15.64:0/4038245349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da708e0d20 con 0x55da708f4930 2024-05-09T09:16:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.986+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 0x55da70905800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009af0 tx=0x7f3bd8009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c037a83b0b3027b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.987+0000 7f3bc97fa640 1 -- 172.21.15.64:0/4038245349 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8027020 con 0x55da708f4930 2024-05-09T09:16:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.987+0000 7f3bc97fa640 1 -- 172.21.15.64:0/4038245349 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bd8018a60 con 0x55da708f4930 2024-05-09T09:16:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.987+0000 7f3bc97fa640 1 -- 172.21.15.64:0/4038245349 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8005040 con 0x55da708f4930 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.988+0000 7f3becbfc740 1 -- 172.21.15.64:0/4038245349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 msgr2=0x55da70905800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.988+0000 7f3becbfc740 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 0x55da70905800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009af0 tx=0x7f3bd8009850 comp rx=0 tx=0).stop 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 -- 172.21.15.64:0/4038245349 shutdown_connections 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f9aa0 0x55da708f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f4930 0x55da70905800 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 --2- 172.21.15.64:0/4038245349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f3f40 0x55da708fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 -- 172.21.15.64:0/4038245349 >> 172.21.15.64:0/4038245349 conn(0x55da708ef5b0 msgr2=0x55da708fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 -- 172.21.15.64:0/4038245349 shutdown_connections 2024-05-09T09:16:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.990+0000 7f3becbfc740 1 -- 172.21.15.64:0/4038245349 wait complete. 2024-05-09T09:16:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.991+0000 7f3becbfc740 1 -- start start 2024-05-09T09:16:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.991+0000 7f3becbfc740 1 Processor -- start 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.991+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f3f40 0x55da709029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.991+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da70902ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3becbfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f9aa0 0x55da70903ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da70902ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da70902ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49292/0 (socket says 172.21.15.64:49292) 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2171789109 learned_addr learned my addr 172.21.15.64:0/2171789109 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3becbfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd000e2b0 con 0x55da708f3f40 2024-05-09T09:16:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be3fff640 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f9aa0 0x55da70903ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd000f720 con 0x55da708f4930 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd000e980 con 0x55da708f9aa0 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3becbfc740 10 client.0 _ll_register_callbacks cb 0x7f3bd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f9aa0 msgr2=0x55da70903ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f9aa0 0x55da70903ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2171789109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f3f40 msgr2=0x55da709029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f3f40 0x55da709029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.992+0000 7f3be2ffd640 1 -- 172.21.15.64:0/2171789109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da709044c0 con 0x55da708f4930 2024-05-09T09:16:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.993+0000 7f3be2ffd640 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da70902ef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800e850 tx=0x7f3bd8004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5869f40fc72d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.993+0000 7f3bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.994+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2171789109 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8027030 con 0x55da708f4930 2024-05-09T09:16:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.994+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8018a60 con 0x55da708f4930 2024-05-09T09:16:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.994+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2171789109 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3bd80035f0 con 0x55da708f4930 2024-05-09T09:16:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.995+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2171789109 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3bd803d050 con 0x55da708f4930 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.995+0000 7f3becbfc740 10 client.26957 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.995+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55da70905780 con 0x55da708f4930 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2171789109 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bd8003ce0 con 0x55da708f4930 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2171789109 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3bd802c650 con 0x55da708f4930 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3becbfc740 10 client.26957 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3becbfc740 10 client.26957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3becbfc740 20 client.26957 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7097bb70 0x55da7097df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3becbfc740 4 client.26957 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:16:57.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.996+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3bd8018a60 con 0x55da7097bb70 2024-05-09T09:16:57.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.997+0000 7f3be37fe640 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7097bb70 0x55da7097df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.997+0000 7f3bc2ffd640 10 client.26957 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.997+0000 7f3be37fe640 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7097bb70 0x55da7097df90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89ed29d43af7ec4d server_cookie=cccf91d16ba282a in_seq=0 out_seq=0 2024-05-09T09:16:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3bc2ffd640 1 -- 172.21.15.64:0/2171789109 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3bd8018a60 con 0x55da7097bb70 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3bc2ffd640 10 client.26957 handle_command_reply: tid=0 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 client.26957 shutdown 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 msgr2=0x55da70902ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da70902ef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800e850 tx=0x7f3bd8004550 comp rx=0 tx=0).stop 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 shutdown_connections 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da708f9aa0 0x55da70903ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55da708f4930 0x55da70902ef0 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800e850 tx=0x7f3bd8004550 comp rx=0 tx=0).stop 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.998+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55da7097bb70 0x55da7097df90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.999+0000 7f3becbfc740 1 --2- 172.21.15.64:0/2171789109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da708f3f40 0x55da709029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.999+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 >> 172.21.15.64:0/2171789109 conn(0x55da708ef5b0 msgr2=0x55da708edf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.999+0000 7f3bc2ffd640 0 client.26957 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:16:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:57.999+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 shutdown_connections 2024-05-09T09:16:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.000+0000 7f3becbfc740 1 -- 172.21.15.64:0/2171789109 wait complete. 2024-05-09T09:16:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.000+0000 7f3becbfc740 20 client.26957 trim_cache size 0 max 16384 2024-05-09T09:16:58.002 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:16:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bcc0a2c10 msgr2=0x7f3bcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bcc0a2c10 0x7f3bcc0a50d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005c40 tx=0x7f3bd40314a0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 msgr2=0x7f3be4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 0x7f3be4104700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00eaa0 tx=0x7f3bdc00ef60 comp rx=0 tx=0).stop 2024-05-09T09:16:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 shutdown_connections 2024-05-09T09:16:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3bcc0a2c10 0x7f3bcc0a50d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be4102320 0x7f3be4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be4101960 0x7f3be4102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 --2- 172.21.15.64:0/1481242582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3be4100770 0x7f3be4104700 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.003+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 >> 172.21.15.64:0/1481242582 conn(0x7f3be40fbd20 msgr2=0x7f3be40fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.005+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 shutdown_connections 2024-05-09T09:16:58.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.006+0000 7f3be9e12640 1 -- 172.21.15.64:0/1481242582 wait complete. 2024-05-09T09:16:58.027 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2427850277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:16:58.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:16:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2427850277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:16:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.184+0000 7f0adf59e640 1 -- 172.21.15.64:0/281299963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006d470 msgr2=0x7f0ae006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.184+0000 7f0adf59e640 1 --2- 172.21.15.64:0/281299963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006d470 0x7f0ae006d890 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0ad801acb0 tx=0x7f0ad803b900 comp rx=0 tx=0).stop 2024-05-09T09:16:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.185+0000 7f0adf59e640 1 -- 172.21.15.64:0/281299963 shutdown_connections 2024-05-09T09:16:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.185+0000 7f0adf59e640 1 --2- 172.21.15.64:0/281299963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006f050 0x7f0ae006f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.185+0000 7f0adf59e640 1 --2- 172.21.15.64:0/281299963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae006e6b0 0x7f0ae006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.185+0000 7f0adf59e640 1 --2- 172.21.15.64:0/281299963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006d470 0x7f0ae006d890 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.185+0000 7f0adf59e640 1 -- 172.21.15.64:0/281299963 >> 172.21.15.64:0/281299963 conn(0x7f0ae006c120 msgr2=0x7f0ae006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.186+0000 7f0adf59e640 1 -- 172.21.15.64:0/281299963 shutdown_connections 2024-05-09T09:16:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.186+0000 7f0adf59e640 1 -- 172.21.15.64:0/281299963 wait complete. 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 Processor -- start 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 -- start start 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 0x7f0ae019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006e6b0 0x7f0ae019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae006f050 0x7f0ae01a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae010d890 con 0x7f0ae006e6b0 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae010d710 con 0x7f0ae006d470 2024-05-09T09:16:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0adf59e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae010da10 con 0x7f0ae006f050 2024-05-09T09:16:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 0x7f0ae019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 0x7f0ae019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49300/0 (socket says 172.21.15.64:49300) 2024-05-09T09:16:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0ade59c640 1 -- 172.21.15.64:0/4003784375 learned_addr learned my addr 172.21.15.64:0/4003784375 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.188+0000 7f0addd9b640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006e6b0 0x7f0ae019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0aded9d640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae006f050 0x7f0ae01a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0ade59c640 1 -- 172.21.15.64:0/4003784375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae006f050 msgr2=0x7f0ae01a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0ade59c640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae006f050 0x7f0ae01a5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0ade59c640 1 -- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006e6b0 msgr2=0x7f0ae019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0ade59c640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006e6b0 0x7f0ae019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0ade59c640 1 -- 172.21.15.64:0/4003784375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae01a59d0 con 0x7f0ae006d470 2024-05-09T09:16:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.189+0000 7f0ade59c640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 0x7f0ae019ddb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8003ba0 tx=0x7f0ad8003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.190+0000 7f0acf7fe640 1 -- 172.21.15.64:0/4003784375 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8045070 con 0x7f0ae006d470 2024-05-09T09:16:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.190+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae01a5c60 con 0x7f0ae006d470 2024-05-09T09:16:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.190+0000 7f0acf7fe640 1 -- 172.21.15.64:0/4003784375 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad8003df0 con 0x7f0ae006d470 2024-05-09T09:16:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.190+0000 7f0acf7fe640 1 -- 172.21.15.64:0/4003784375 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80046f0 con 0x7f0ae006d470 2024-05-09T09:16:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.190+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a61a0 con 0x7f0ae006d470 2024-05-09T09:16:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.193+0000 7f0acf7fe640 1 -- 172.21.15.64:0/4003784375 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ad8048020 con 0x7f0ae006d470 2024-05-09T09:16:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.194+0000 7f0ae7566740 1 Processor -- start 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 -- start start 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 0x555daee087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee0d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee0df30 0x555daee103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555daed6c340 con 0x555daee083d0 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555daed14290 con 0x555daee0df30 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ae7566740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555daed68300 con 0x555daee08dc0 2024-05-09T09:16:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0acf7fe640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ac80a2c40 0x7f0ac80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 0x555daee087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 0x555daee087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34652/0 (socket says 172.21.15.64:34652) 2024-05-09T09:16:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0aded9d640 1 -- 172.21.15.64:0/2728744623 learned_addr learned my addr 172.21.15.64:0/2728744623 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0acf7fe640 1 -- 172.21.15.64:0/4003784375 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ad81013a0 con 0x7f0ae006d470 2024-05-09T09:16:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.195+0000 7f0ade59c640 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee0d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.196+0000 7f0ade59c640 1 -- 172.21.15.64:0/2728744623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee0df30 msgr2=0x555daee103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.196+0000 7f0ade59c640 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee0df30 0x555daee103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.196+0000 7f0ade59c640 1 -- 172.21.15.64:0/2728744623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 msgr2=0x555daee087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.196+0000 7f0ade59c640 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 0x555daee087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.196+0000 7f0ade59c640 1 -- 172.21.15.64:0/2728744623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555daed388e0 con 0x555daee08dc0 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.197+0000 7f0aded9d640 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 0x555daee087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.197+0000 7f0aded9d640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ac80a2c40 0x7f0ac80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.197+0000 7f0ade59c640 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee0d8c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80feab0 tx=0x7f0ad80cd5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64af0acbad20f0d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.197+0000 7f0accff9640 1 -- 172.21.15.64:0/2728744623 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80554b0 con 0x555daee08dc0 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.197+0000 7f0accff9640 1 -- 172.21.15.64:0/2728744623 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad80cdf10 con 0x555daee08dc0 2024-05-09T09:16:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.197+0000 7f0accff9640 1 -- 172.21.15.64:0/2728744623 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8054430 con 0x555daee08dc0 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0aded9d640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ac80a2c40 0x7f0ac80a5100 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005d00 tx=0x7f0ad000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 -- 172.21.15.64:0/2728744623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 msgr2=0x555daee0d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee0d8c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80feab0 tx=0x7f0ad80cd5e0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 -- 172.21.15.64:0/2728744623 shutdown_connections 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee0df30 0x555daee103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee0d8c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 --2- 172.21.15.64:0/2728744623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee083d0 0x555daee087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.198+0000 7f0ae7566740 1 -- 172.21.15.64:0/2728744623 >> 172.21.15.64:0/2728744623 conn(0x555daee03a40 msgr2=0x555daee03e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.201+0000 7f0ae7566740 1 -- 172.21.15.64:0/2728744623 shutdown_connections 2024-05-09T09:16:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.201+0000 7f0ae7566740 1 -- 172.21.15.64:0/2728744623 wait complete. 2024-05-09T09:16:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.201+0000 7f0ae7566740 1 -- start start 2024-05-09T09:16:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 Processor -- start 2024-05-09T09:16:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee12360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555daee10ca0 con 0x555daee0df30 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555daed27790 con 0x555daee083d0 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555daee10e20 con 0x555daee08dc0 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.202+0000 7f0ae7566740 10 client.0 _ll_register_callbacks cb 0x555daed62a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee12360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee12360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49330/0 (socket says 172.21.15.64:49330) 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 -- 172.21.15.64:0/3829099743 learned_addr learned my addr 172.21.15.64:0/3829099743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 -- 172.21.15.64:0/3829099743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 msgr2=0x555daee128a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee128a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 -- 172.21.15.64:0/3829099743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 msgr2=0x555daee198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.203+0000 7f0ade59c640 1 -- 172.21.15.64:0/3829099743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555daea80210 con 0x555daee083d0 2024-05-09T09:16:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.204+0000 7f0ade59c640 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee12360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c9c70 tx=0x7f0ad81011f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16d38b985995bca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.204+0000 7f0ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.208+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3829099743 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80e6060 con 0x555daee083d0 2024-05-09T09:16:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.208+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad80cdf10 con 0x555daee083d0 2024-05-09T09:16:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.208+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3829099743 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ad80b80d0 con 0x555daee083d0 2024-05-09T09:16:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.211+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3829099743 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0ad80eb050 con 0x555daee083d0 2024-05-09T09:16:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.213+0000 7f0ae7566740 10 client.26966 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.213+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555daeb4c620 con 0x555daee083d0 2024-05-09T09:16:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3829099743 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad80cdf10 con 0x555daee083d0 2024-05-09T09:16:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3829099743 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0ad804c8e0 con 0x555daee083d0 2024-05-09T09:16:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ae7566740 10 client.26966 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ae7566740 10 client.26966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ae7566740 20 client.26966 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee1dcf0 0x555daee20150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ae7566740 4 client.26966 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.216+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555daed79a70 con 0x555daee1dcf0 2024-05-09T09:16:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.218+0000 7f0addd9b640 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee1dcf0 0x555daee20150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.223+0000 7f0addd9b640 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee1dcf0 0x555daee20150 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cef48be4f66ef16 server_cookie=bfdd7b56cc25fe25 in_seq=0 out_seq=0 2024-05-09T09:16:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.223+0000 7f0ac5ffb640 10 client.26966 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.227+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3829099743 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555daed79a70 con 0x555daee1dcf0 2024-05-09T09:16:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.227+0000 7f0ac5ffb640 10 client.26966 handle_command_reply: tid=0 2024-05-09T09:16:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.227+0000 7f0ae7566740 1 client.26966 shutdown 2024-05-09T09:16:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.229+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 msgr2=0x555daee12360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.229+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee12360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c9c70 tx=0x7f0ad81011f0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 shutdown_connections 2024-05-09T09:16:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee1dcf0 0x555daee20150 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee08dc0 0x555daee128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3829099743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee12360 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 >> 172.21.15.64:0/3829099743 conn(0x555daee03a40 msgr2=0x555daee02ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ac5ffb640 0 client.26966 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 shutdown_connections 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 1 -- 172.21.15.64:0/3829099743 wait complete. 2024-05-09T09:16:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.230+0000 7f0ae7566740 20 client.26966 trim_cache size 0 max 16384 2024-05-09T09:16:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.241+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2000308231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74102bc0 msgr2=0x7f1c74103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.241+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2000308231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74102bc0 0x7f1c74103040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009860 tx=0x7f1c6402a360 comp rx=0 tx=0).stop 2024-05-09T09:16:58.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.244+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2000308231 shutdown_connections 2024-05-09T09:16:58.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.244+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2000308231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74103580 0x7f1c74109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.244+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2000308231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74102bc0 0x7f1c74103040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.244+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2000308231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c740ffb50 0x7f1c740fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.244+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2000308231 >> 172.21.15.64:0/2000308231 conn(0x7f1c740fb460 msgr2=0x7f1c740fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.244+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2000308231 shutdown_connections 2024-05-09T09:16:58.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.245+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2000308231 wait complete. 2024-05-09T09:16:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.246+0000 7f1c7bad4640 1 Processor -- start 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.246+0000 7f1c7bad4640 1 -- start start 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.246+0000 7f1c7bad4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c740ffb50 0x7f1c7419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7bad4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 0x7f1c7419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7bad4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74103580 0x7f1c741a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7906f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 0x7f1c7419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7906f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 0x7f1c7419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49350/0 (socket says 172.21.15.64:49350) 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7a071640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74103580 0x7f1c741a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7a071640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74103580 0x7f1c741a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40092/0 (socket says 172.21.15.64:40092) 2024-05-09T09:16:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7906f640 1 -- 172.21.15.64:0/2281644282 learned_addr learned my addr 172.21.15.64:0/2281644282 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74102580 con 0x7f1c740ffb50 2024-05-09T09:16:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74102400 con 0x7f1c74102bc0 2024-05-09T09:16:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.247+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74102700 con 0x7f1c74103580 2024-05-09T09:16:58.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c79870640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c740ffb50 0x7f1c7419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c7906f640 1 -- 172.21.15.64:0/2281644282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74103580 msgr2=0x7f1c741a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c7906f640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74103580 0x7f1c741a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c7906f640 1 -- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c740ffb50 msgr2=0x7f1c7419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c7906f640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c740ffb50 0x7f1c7419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c7906f640 1 -- 172.21.15.64:0/2281644282 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c741a5df0 con 0x7f1c74102bc0 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.248+0000 7f1c7906f640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 0x7f1c7419e710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009780 tx=0x7f1c64032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.249+0000 7f1c6affd640 1 -- 172.21.15.64:0/2281644282 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64042070 con 0x7f1c74102bc0 2024-05-09T09:16:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.249+0000 7f1c6affd640 1 -- 172.21.15.64:0/2281644282 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c640307a0 con 0x7f1c74102bc0 2024-05-09T09:16:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.249+0000 7f1c6affd640 1 -- 172.21.15.64:0/2281644282 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64037040 con 0x7f1c74102bc0 2024-05-09T09:16:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.249+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741a6080 con 0x7f1c74102bc0 2024-05-09T09:16:58.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.249+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741a6460 con 0x7f1c74102bc0 2024-05-09T09:16:58.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.253+0000 7f1c6affd640 1 -- 172.21.15.64:0/2281644282 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c64045070 con 0x7f1c74102bc0 2024-05-09T09:16:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.254+0000 7f1c7e7e3740 1 Processor -- start 2024-05-09T09:16:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.254+0000 7f1c6affd640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c600a2d10 0x7f1c600a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.255+0000 7f1c6affd640 1 -- 172.21.15.64:0/2281644282 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c640eefb0 con 0x7f1c74102bc0 2024-05-09T09:16:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.255+0000 7f1c7a071640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c600a2d10 0x7f1c600a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.256+0000 7f1c7e7e3740 1 -- start start 2024-05-09T09:16:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.256+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560878732f40 0x560878733360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.256+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878733930 0x560878738430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.257+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 0x56087873af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.257+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560878697c40 con 0x560878738aa0 2024-05-09T09:16:58.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.257+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56087863f580 con 0x560878733930 2024-05-09T09:16:58.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.257+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560878693c00 con 0x560878732f40 2024-05-09T09:16:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.257+0000 7f1c7a071640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 0x56087873af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.257+0000 7f1c7a071640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 0x56087873af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34700/0 (socket says 172.21.15.64:34700) 2024-05-09T09:16:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 -- 172.21.15.64:0/3991064426 learned_addr learned my addr 172.21.15.64:0/3991064426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c600a2d10 0x7f1c600a51d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fa0 tx=0x7f1c6c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 -- 172.21.15.64:0/3991064426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560878732f40 msgr2=0x560878733360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560878732f40 0x560878733360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 -- 172.21.15.64:0/3991064426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878733930 msgr2=0x560878738430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878733930 0x560878738430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.258+0000 7f1c7a071640 1 -- 172.21.15.64:0/3991064426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608783ab3f0 con 0x560878738aa0 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.259+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 0x56087873af30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c016820 tx=0x7f1c6c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf24c337b0c2692 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.259+0000 7f1c5ffff640 1 -- 172.21.15.64:0/3991064426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c015a70 con 0x560878738aa0 2024-05-09T09:16:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.260+0000 7f1c5ffff640 1 -- 172.21.15.64:0/3991064426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c025070 con 0x560878738aa0 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3991064426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 msgr2=0x56087873af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 0x56087873af30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c016820 tx=0x7f1c6c011000 comp rx=0 tx=0).stop 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3991064426 shutdown_connections 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878738aa0 0x56087873af30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878733930 0x560878738430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3991064426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560878732f40 0x560878733360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.261+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3991064426 >> 172.21.15.64:0/3991064426 conn(0x56087872e5b0 msgr2=0x56087872e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.263+0000 7f0ae7566740 1 Processor -- start 2024-05-09T09:16:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.263+0000 7f0ae7566740 1 -- start start 2024-05-09T09:16:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.263+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee083d0 0x555daee12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.263+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3991064426 shutdown_connections 2024-05-09T09:16:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee08dc0 0x555daee19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee083d0 0x555daee12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee083d0 0x555daee12800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40108/0 (socket says 172.21.15.64:40108) 2024-05-09T09:16:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ade59c640 1 -- 172.21.15.64:0/1486819104 learned_addr learned my addr 172.21.15.64:0/1486819104 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.263+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3991064426 wait complete. 2024-05-09T09:16:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 -- start start 2024-05-09T09:16:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 Processor -- start 2024-05-09T09:16:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0addd9b640 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee08dc0 0x555daee19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ae7566740 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b3170 con 0x555daee0df30 2024-05-09T09:16:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b09e0 con 0x555daee08dc0 2024-05-09T09:16:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ae830 con 0x555daee083d0 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x560878742a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878744fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56087873baa0 con 0x560878732f40 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560878652c40 con 0x56087873def0 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.264+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56087873bc20 con 0x56087873da70 2024-05-09T09:16:58.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f1c7e7e3740 10 client.0 _ll_register_callbacks cb 0x5608783aabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f0aded9d640 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f1c79870640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f1c7906f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x560878742a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f1c79870640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34718/0 (socket says 172.21.15.64:34718) 2024-05-09T09:16:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f1c79870640 1 -- 172.21.15.64:0/3047666258 learned_addr learned my addr 172.21.15.64:0/3047666258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878744fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f0aded9d640 1 -- 172.21.15.64:0/1486819104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee083d0 msgr2=0x555daee12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f0aded9d640 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee083d0 0x555daee12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f0aded9d640 1 -- 172.21.15.64:0/1486819104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee08dc0 msgr2=0x555daee19870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f0aded9d640 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee08dc0 0x555daee19870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.265+0000 7f0aded9d640 1 -- 172.21.15.64:0/1486819104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555daedf5330 con 0x555daee0df30 2024-05-09T09:16:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.266+0000 7f0aded9d640 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee071c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x555daee09620 tx=0x7f0ad0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ec2a9a1e1474e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.266+0000 7f1c79870640 1 -- 172.21.15.64:0/3047666258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 msgr2=0x560878742a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.266+0000 7f1c79870640 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x560878742a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.266+0000 7f1c79870640 1 -- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 msgr2=0x560878744fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.266+0000 7f1c79870640 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878744fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.266+0000 7f1c79870640 1 -- 172.21.15.64:0/3047666258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608784757d0 con 0x560878732f40 2024-05-09T09:16:58.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.267+0000 7f1c79870640 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000dd90 tx=0x7f1c7000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d27d15a2dc2e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.267+0000 7f0ac7fff640 1 -- 172.21.15.64:0/1486819104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0026670 con 0x555daee0df30 2024-05-09T09:16:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.267+0000 7f0ac7fff640 1 -- 172.21.15.64:0/1486819104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad0021070 con 0x555daee0df30 2024-05-09T09:16:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ac7fff640 1 -- 172.21.15.64:0/1486819104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad001a650 con 0x555daee0df30 2024-05-09T09:16:58.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f1c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:58.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/3047666258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70005020 con 0x560878732f40 2024-05-09T09:16:58.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/3047666258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c700051c0 con 0x560878732f40 2024-05-09T09:16:58.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608786a5370 con 0x560878732f40 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 msgr2=0x555daee071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee071c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x555daee09620 tx=0x7f0ad0014070 comp rx=0 tx=0).stop 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 shutdown_connections 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee0df30 0x555daee071c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee08dc0 0x555daee19870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 --2- 172.21.15.64:0/1486819104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee083d0 0x555daee12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.268+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 >> 172.21.15.64:0/1486819104 conn(0x555daee03a40 msgr2=0x555daee06ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.269+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/3047666258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1c70020050 con 0x560878732f40 2024-05-09T09:16:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.269+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 shutdown_connections 2024-05-09T09:16:58.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.269+0000 7f1c7e7e3740 10 client.17520 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:58.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.269+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5608787596e0 con 0x560878732f40 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.269+0000 7f0ae7566740 1 -- 172.21.15.64:0/1486819104 wait complete. 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.269+0000 7f0ae7566740 1 -- start start 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 Processor -- start 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 0x555daee17620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee0df30 0x555daee18630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01aeea0 con 0x555daee08dc0 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01b0310 con 0x555daee083d0 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01af570 con 0x555daee0df30 2024-05-09T09:16:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f0ae7566740 10 client.0 _ll_register_callbacks cb 0x7f0ad000b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:58.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.270+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/3047666258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5608786a5370 con 0x560878732f40 2024-05-09T09:16:58.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/3047666258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1c70011860 con 0x560878732f40 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 0x555daee17620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 0x555daee17620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34724/0 (socket says 172.21.15.64:34724) 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 -- 172.21.15.64:0/3891223355 learned_addr learned my addr 172.21.15.64:0/3891223355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 -- 172.21.15.64:0/3891223355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee0df30 msgr2=0x555daee18630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee0df30 0x555daee18630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 -- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 msgr2=0x555daee170e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee170e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 -- 172.21.15.64:0/3891223355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555daee18c20 con 0x555daee08dc0 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.271+0000 7f0addd9b640 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 0x555daee17620 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x555daee20a00 tx=0x7f0ad400ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6021b13e8771e26b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f0ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:16:58.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7e7e3740 10 client.17520 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:58.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7e7e3740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:58.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7e7e3740 20 client.17520 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:58.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56087875b150 0x56087875d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7e7e3740 4 client.17520 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:58.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56087869f6c0 con 0x56087875b150 2024-05-09T09:16:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3891223355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4024060 con 0x555daee08dc0 2024-05-09T09:16:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad0021070 con 0x555daee08dc0 2024-05-09T09:16:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3891223355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ad401f430 con 0x555daee08dc0 2024-05-09T09:16:58.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f1c7906f640 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56087875b150 0x56087875d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.272+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3891223355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0ad402b050 con 0x555daee08dc0 2024-05-09T09:16:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.273+0000 7f0ae7566740 10 client.17523 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.273+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555daee84550 con 0x555daee08dc0 2024-05-09T09:16:58.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.273+0000 7f1c7906f640 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56087875b150 0x56087875d5c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc1e9946a96de5a server_cookie=3049e0984349aa6a in_seq=0 out_seq=0 2024-05-09T09:16:58.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.273+0000 7f1c5d7fa640 10 client.17520 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.273+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3891223355 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad0021070 con 0x555daee08dc0 2024-05-09T09:16:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.273+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3891223355 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0ad4013390 con 0x555daee08dc0 2024-05-09T09:16:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f0ae7566740 10 client.17523 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f0ae7566740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f0ae7566740 20 client.17523 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee84f10 0x555daee87330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f0ae7566740 4 client.17523 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:16:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555daed73dc0 con 0x555daee84f10 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/3047666258 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56087869f6c0 con 0x56087875b150 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f1c5d7fa640 10 client.17520 handle_command_reply: tid=0 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.274+0000 7f1c7e7e3740 1 client.17520 shutdown 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 msgr2=0x56087873d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000dd90 tx=0x7f1c7000f730 comp rx=0 tx=0).stop 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 shutdown_connections 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56087875b150 0x56087875d5c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878744fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x560878742a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3047666258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d530 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 >> 172.21.15.64:0/3047666258 conn(0x56087872e5b0 msgr2=0x56087872d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c5d7fa640 0 client.17520 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.275+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 shutdown_connections 2024-05-09T09:16:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.276+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3047666258 wait complete. 2024-05-09T09:16:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.276+0000 7f0ade59c640 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee84f10 0x555daee87330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.276+0000 7f1c7e7e3740 20 client.17520 trim_cache size 0 max 16384 2024-05-09T09:16:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.276+0000 7f0ade59c640 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee84f10 0x555daee87330 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb28dcd162d2cc69 server_cookie=4ed28c34869cbc20 in_seq=0 out_seq=0 2024-05-09T09:16:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.276+0000 7f0ac5ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ac5ffb640 1 -- 172.21.15.64:0/3891223355 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555daed73dc0 con 0x555daee84f10 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ac5ffb640 10 client.17523 handle_command_reply: tid=0 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ae7566740 1 client.17523 shutdown 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 msgr2=0x555daee17620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 0x555daee17620 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x555daee20a00 tx=0x7f0ad400ac90 comp rx=0 tx=0).stop 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 shutdown_connections 2024-05-09T09:16:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555daee84f10 0x555daee87330 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.299+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555daee0df30 0x555daee18630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.300+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555daee08dc0 0x555daee17620 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.300+0000 7f0ae7566740 1 --2- 172.21.15.64:0/3891223355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555daee083d0 0x555daee170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.300+0000 7f0ac5ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.300+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 >> 172.21.15.64:0/3891223355 conn(0x555daee03a40 msgr2=0x555daee12660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.300+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 shutdown_connections 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.300+0000 7f0ae7566740 1 -- 172.21.15.64:0/3891223355 wait complete. 2024-05-09T09:16:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.301+0000 7f0ae7566740 20 client.17523 trim_cache size 0 max 16384 2024-05-09T09:16:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:16:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:16:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:16:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:16:58.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.303+0000 7f1c7e7e3740 1 Processor -- start 2024-05-09T09:16:58.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.304+0000 7f1c7e7e3740 1 -- start start 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.304+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.304+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x56087872e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.304+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878734dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.304+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741aacb0 con 0x560878732f40 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741a8520 con 0x56087873def0 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74118110 con 0x56087873da70 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c79870640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c79870640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34732/0 (socket says 172.21.15.64:34732) 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c79870640 1 -- 172.21.15.64:0/3858169073 learned_addr learned my addr 172.21.15.64:0/3858169073 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878734dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7a071640 1 -- 172.21.15.64:0/3858169073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 msgr2=0x56087872e990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x56087872e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7a071640 1 -- 172.21.15.64:0/3858169073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 msgr2=0x56087873d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c7a071640 1 -- 172.21.15.64:0/3858169073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560878738cb0 con 0x56087873def0 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.305+0000 7f1c79870640 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.306+0000 7f1c7a071640 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878734dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c015740 tx=0x7f1c6c009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607860997c3be26c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.306+0000 7f1c5f7fe640 1 -- 172.21.15.64:0/3858169073 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c02e070 con 0x56087873def0 2024-05-09T09:16:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.306+0000 7f1c5f7fe640 1 -- 172.21.15.64:0/3858169073 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c025040 con 0x56087873def0 2024-05-09T09:16:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c5f7fe640 1 -- 172.21.15.64:0/3858169073 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0206f0 con 0x56087873def0 2024-05-09T09:16:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3858169073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 msgr2=0x560878734dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878734dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c015740 tx=0x7f1c6c009d80 comp rx=0 tx=0).stop 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3858169073 shutdown_connections 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56087873def0 0x560878734dd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873da70 0x56087872e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/3858169073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560878732f40 0x56087873d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3858169073 >> 172.21.15.64:0/3858169073 conn(0x56087872e5b0 msgr2=0x56087873cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3858169073 shutdown_connections 2024-05-09T09:16:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/3858169073 wait complete. 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ac80a2c40 msgr2=0x7f0ac80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ac80a2c40 0x7f0ac80a5100 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005d00 tx=0x7f0ad000b540 comp rx=0 tx=0).stop 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 msgr2=0x7f0ae019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 0x7f0ae019ddb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8003ba0 tx=0x7f0ad8003bd0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 shutdown_connections 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ac80a2c40 0x7f0ac80a5100 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae006f050 0x7f0ae01a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae006e6b0 0x7f0ae019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 --2- 172.21.15.64:0/4003784375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae006d470 0x7f0ae019ddb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 >> 172.21.15.64:0/4003784375 conn(0x7f0ae006c120 msgr2=0x7f0ae010a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 shutdown_connections 2024-05-09T09:16:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:16:58.307+0000 7f0adf59e640 1 -- 172.21.15.64:0/4003784375 wait complete. 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 -- start start 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 Processor -- start 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 0x56087876b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56087873da70 0x56087876be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873def0 0x5608787bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741a69e0 con 0x56087873da70 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741a7e50 con 0x560878732f40 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741a70b0 con 0x56087873def0 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.308+0000 7f1c7e7e3740 10 client.0 _ll_register_callbacks cb 0x7f1c7000d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 0x56087876b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 0x56087876b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49384/0 (socket says 172.21.15.64:49384) 2024-05-09T09:16:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 -- 172.21.15.64:0/2524193365 learned_addr learned my addr 172.21.15.64:0/2524193365 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c7906f640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56087873da70 0x56087876be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c7a071640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873def0 0x5608787bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 -- 172.21.15.64:0/2524193365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873def0 msgr2=0x5608787bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873def0 0x5608787bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 -- 172.21.15.64:0/2524193365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56087873da70 msgr2=0x56087876be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56087873da70 0x56087876be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 -- 172.21.15.64:0/2524193365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560878732550 con 0x560878732f40 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c79870640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 0x56087876b8c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c70001eb0 tx=0x7f1c70011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5631baa3dce33e6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.309+0000 7f1c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.310+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/2524193365 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70034060 con 0x560878732f40 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.310+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/2524193365 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c7001b060 con 0x560878732f40 2024-05-09T09:16:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.310+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6c025040 con 0x560878732f40 2024-05-09T09:16:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.315+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/2524193365 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1c7001b260 con 0x560878732f40 2024-05-09T09:16:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.315+0000 7f1c7e7e3740 10 client.26981 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.315+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5608787c0790 con 0x560878732f40 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.316+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/2524193365 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c025040 con 0x560878732f40 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.316+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/2524193365 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1c70004680 con 0x560878732f40 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.317+0000 7f1c7e7e3740 10 client.26981 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.317+0000 7f1c7e7e3740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.317+0000 7f1c7e7e3740 20 client.26981 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.317+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5608787c1170 0x5608787c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.317+0000 7f1c7e7e3740 4 client.26981 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:16:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.317+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56087861c820 con 0x5608787c1170 2024-05-09T09:16:58.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.319+0000 7f1c7906f640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5608787c1170 0x5608787c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:58.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.319+0000 7f1c7906f640 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5608787c1170 0x5608787c3590 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f548c5812429f58 server_cookie=53bdb76249afc678 in_seq=0 out_seq=0 2024-05-09T09:16:58.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.319+0000 7f1c5d7fa640 10 client.26981 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.319+0000 7f1c5d7fa640 1 -- 172.21.15.64:0/2524193365 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56087861c820 con 0x5608787c1170 2024-05-09T09:16:58.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.319+0000 7f1c5d7fa640 10 client.26981 handle_command_reply: tid=0 2024-05-09T09:16:58.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.319+0000 7f1c7e7e3740 1 client.26981 shutdown 2024-05-09T09:16:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 msgr2=0x56087876b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 0x56087876b8c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c70001eb0 tx=0x7f1c70011880 comp rx=0 tx=0).stop 2024-05-09T09:16:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 shutdown_connections 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5608787c1170 0x5608787c3590 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56087873def0 0x5608787bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56087873da70 0x56087876be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 --2- 172.21.15.64:0/2524193365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560878732f40 0x56087876b8c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 >> 172.21.15.64:0/2524193365 conn(0x56087872e5b0 msgr2=0x56087873ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.321+0000 7f1c5d7fa640 0 client.26981 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.322+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 shutdown_connections 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.322+0000 7f1c7e7e3740 1 -- 172.21.15.64:0/2524193365 wait complete. 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.322+0000 7f1c7e7e3740 20 client.26981 trim_cache size 0 max 16384 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.323+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c600a2d10 msgr2=0x7f1c600a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.323+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c600a2d10 0x7f1c600a51d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fa0 tx=0x7f1c6c00d040 comp rx=0 tx=0).stop 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.323+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 msgr2=0x7f1c7419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.323+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 0x7f1c7419e710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009780 tx=0x7f1c64032cd0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.325+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 shutdown_connections 2024-05-09T09:16:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.325+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c600a2d10 0x7f1c600a51d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.325+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c74103580 0x7f1c741a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.325+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c74102bc0 0x7f1c7419e710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.325+0000 7f1c7bad4640 1 --2- 172.21.15.64:0/2281644282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c740ffb50 0x7f1c7419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.325+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 >> 172.21.15.64:0/2281644282 conn(0x7f1c740fb460 msgr2=0x7f1c7410ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:58.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.327+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 shutdown_connections 2024-05-09T09:16:58.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:58.328+0000 7f1c7bad4640 1 -- 172.21.15.64:0/2281644282 wait complete. 2024-05-09T09:16:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:16:58 smithi179 ceph-mon[34086]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 773 op/s 2024-05-09T09:16:59.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:16:58 smithi139 ceph-mon[33599]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 773 op/s 2024-05-09T09:16:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:16:58 smithi064 ceph-mon[30149]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 773 op/s 2024-05-09T09:16:59.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:16:59.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:16:59.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:16:59.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.574+0000 7f3337210640 1 -- 172.21.15.64:0/317982124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330102a50 msgr2=0x7f3330104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.574+0000 7f3337210640 1 --2- 172.21.15.64:0/317982124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330102a50 0x7f3330104ec0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f332002a440 comp rx=0 tx=0).stop 2024-05-09T09:16:59.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.575+0000 7f3337210640 1 -- 172.21.15.64:0/317982124 shutdown_connections 2024-05-09T09:16:59.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.575+0000 7f3337210640 1 --2- 172.21.15.64:0/317982124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3330105400 0x7f3330105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.575+0000 7f3337210640 1 --2- 172.21.15.64:0/317982124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330102a50 0x7f3330104ec0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.575+0000 7f3337210640 1 --2- 172.21.15.64:0/317982124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33301000e0 0x7f3330102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.575+0000 7f3337210640 1 -- 172.21.15.64:0/317982124 >> 172.21.15.64:0/317982124 conn(0x7f33300fbce0 msgr2=0x7f33300fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.576+0000 7f3337210640 1 -- 172.21.15.64:0/317982124 shutdown_connections 2024-05-09T09:16:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.576+0000 7f3337210640 1 -- 172.21.15.64:0/317982124 wait complete. 2024-05-09T09:16:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.577+0000 7f3337210640 1 Processor -- start 2024-05-09T09:16:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 -- start start 2024-05-09T09:16:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 0x7f33301a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3330102a50 0x7f33301a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330105400 0x7f33301ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f333006dd00 con 0x7f3330105400 2024-05-09T09:16:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f333006db80 con 0x7f33301000e0 2024-05-09T09:16:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3337210640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f333006de80 con 0x7f3330102a50 2024-05-09T09:16:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3334fac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 0x7f33301a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3334fac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 0x7f33301a6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49400/0 (socket says 172.21.15.64:49400) 2024-05-09T09:16:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f3334fac640 1 -- 172.21.15.64:0/1602173919 learned_addr learned my addr 172.21.15.64:0/1602173919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.578+0000 7f33357ad640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330105400 0x7f33301ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f3334fac640 1 -- 172.21.15.64:0/1602173919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3330102a50 msgr2=0x7f33301a74c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f3334fac640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3330102a50 0x7f33301a74c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f3334fac640 1 -- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330105400 msgr2=0x7f33301ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f3334fac640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330105400 0x7f33301ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f3334fac640 1 -- 172.21.15.64:0/1602173919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33301ae700 con 0x7f33301000e0 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f33357ad640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330105400 0x7f33301ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.579+0000 7f3334fac640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 0x7f33301a6f80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f332800ba50 tx=0x7f332800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.580+0000 7f332dffb640 1 -- 172.21.15.64:0/1602173919 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328019070 con 0x7f33301000e0 2024-05-09T09:16:59.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.580+0000 7f332dffb640 1 -- 172.21.15.64:0/1602173919 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f332800cbe0 con 0x7f33301000e0 2024-05-09T09:16:59.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.580+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301ae9f0 con 0x7f33301000e0 2024-05-09T09:16:59.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.581+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301aefc0 con 0x7f33301000e0 2024-05-09T09:16:59.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.582+0000 7f332dffb640 1 -- 172.21.15.64:0/1602173919 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280076c0 con 0x7f33301000e0 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 Processor -- start 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 -- start start 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf408808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf408859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40886020 0x55bf408884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf407e4990 con 0x55bf408804c0 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf4078c5b0 con 0x55bf40880eb0 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.585+0000 7f3339ff4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf407e0950 con 0x55bf40886020 2024-05-09T09:16:59.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.588+0000 7f332dffb640 1 -- 172.21.15.64:0/1602173919 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f332801c070 con 0x7f33301000e0 2024-05-09T09:16:59.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.588+0000 7f3334fac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf408859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.588+0000 7f3334fac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf408859b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49410/0 (socket says 172.21.15.64:49410) 2024-05-09T09:16:59.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.588+0000 7f3334fac640 1 -- 172.21.15.64:0/3453065301 learned_addr learned my addr 172.21.15.64:0/3453065301 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.588+0000 7f33357ad640 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf408808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.588+0000 7f332ffff640 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40886020 0x55bf408884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.589+0000 7f3334fac640 1 -- 172.21.15.64:0/3453065301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40886020 msgr2=0x55bf408884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.589+0000 7f3334fac640 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40886020 0x55bf408884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.589+0000 7f3334fac640 1 -- 172.21.15.64:0/3453065301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 msgr2=0x55bf408808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.589+0000 7f3334fac640 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf408808e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.590+0000 7f3334fac640 1 -- 172.21.15.64:0/3453065301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf405c07e0 con 0x55bf40880eb0 2024-05-09T09:16:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.590+0000 7f3334fac640 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf408859b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33280d2910 tx=0x7f33280d2a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a88f4d860e1ab4d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.590+0000 7f3326ffd640 1 -- 172.21.15.64:0/3453065301 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280d9040 con 0x55bf40880eb0 2024-05-09T09:16:59.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.591+0000 7f3326ffd640 1 -- 172.21.15.64:0/3453065301 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33280dd880 con 0x55bf40880eb0 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 -- 172.21.15.64:0/3453065301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 msgr2=0x55bf408859b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf408859b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33280d2910 tx=0x7f33280d2a00 comp rx=0 tx=0).stop 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f332dffb640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f331c0a2d90 0x7f331c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 -- 172.21.15.64:0/3453065301 shutdown_connections 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40886020 0x55bf408884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf408859b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3453065301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf408808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f3339ff4740 1 -- 172.21.15.64:0/3453065301 >> 172.21.15.64:0/3453065301 conn(0x55bf408480a0 msgr2=0x55bf4087bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.592+0000 7f332dffb640 1 -- 172.21.15.64:0/1602173919 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33280ceaa0 con 0x7f33301000e0 2024-05-09T09:16:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.593+0000 7f33357ad640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f331c0a2d90 0x7f331c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.593+0000 7f3339ff4740 1 -- 172.21.15.64:0/3453065301 shutdown_connections 2024-05-09T09:16:59.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.595+0000 7f3339ff4740 1 -- 172.21.15.64:0/3453065301 wait complete. 2024-05-09T09:16:59.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.595+0000 7f3339ff4740 1 -- start start 2024-05-09T09:16:59.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.595+0000 7f3339ff4740 1 Processor -- start 2024-05-09T09:16:59.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf408804c0 0x55bf4088ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf4088b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf40886020 0x55bf40892110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf40889070 con 0x55bf40886020 2024-05-09T09:16:59.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf4079fb40 con 0x55bf40880eb0 2024-05-09T09:16:59.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf408891f0 con 0x55bf408804c0 2024-05-09T09:16:59.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.597+0000 7f3339ff4740 10 client.0 _ll_register_callbacks cb 0x55bf407940f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.600+0000 7f3334fac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf408804c0 0x55bf4088ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.600+0000 7f332ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf4088b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf4088b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49420/0 (socket says 172.21.15.64:49420) 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 -- 172.21.15.64:0/23484555 learned_addr learned my addr 172.21.15.64:0/23484555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 -- 172.21.15.64:0/23484555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf408804c0 msgr2=0x55bf4088ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf408804c0 0x55bf4088ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 -- 172.21.15.64:0/23484555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf40886020 msgr2=0x55bf40892110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf40886020 0x55bf40892110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.601+0000 7f332ffff640 1 -- 172.21.15.64:0/23484555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf40620f70 con 0x55bf40880eb0 2024-05-09T09:16:59.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.602+0000 7f332ffff640 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf4088b0d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f33200314e0 tx=0x7f3320005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1fd39bbe7dc8b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.603+0000 7f330ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.604+0000 7f330ffff640 1 -- 172.21.15.64:0/23484555 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320040070 con 0x55bf40880eb0 2024-05-09T09:16:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.604+0000 7f330ffff640 1 -- 172.21.15.64:0/23484555 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33200048c0 con 0x55bf40880eb0 2024-05-09T09:16:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.606+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf407f20c0 con 0x55bf40880eb0 2024-05-09T09:16:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.606+0000 7f33357ad640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f331c0a2d90 0x7f331c0a5250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55bf40881710 tx=0x7f3318008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.607+0000 7f330ffff640 1 -- 172.21.15.64:0/23484555 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f332004a050 con 0x55bf40880eb0 2024-05-09T09:16:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.608+0000 7f3339ff4740 10 client.26990 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.608+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bf404b8170 con 0x55bf40880eb0 2024-05-09T09:16:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.611+0000 7f330ffff640 1 -- 172.21.15.64:0/23484555 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bf407f20c0 con 0x55bf40880eb0 2024-05-09T09:16:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.611+0000 7f330ffff640 1 -- 172.21.15.64:0/23484555 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f332003a040 con 0x55bf40880eb0 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.613+0000 7f3339ff4740 10 client.26990 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.613+0000 7f3339ff4740 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.613+0000 7f3339ff4740 20 client.26990 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.613+0000 7f3339ff4740 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf40896760 0x55bf40898be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.613+0000 7f3339ff4740 4 client.26990 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.613+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf407ec410 con 0x55bf40896760 2024-05-09T09:16:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.614+0000 7f3334fac640 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf40896760 0x55bf40898be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.614+0000 7f3334fac640 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf40896760 0x55bf40898be0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d694cc9aabb9e267 server_cookie=fb25ee0601c7b37c in_seq=0 out_seq=0 2024-05-09T09:16:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.616+0000 7f330ffff640 10 client.26990 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.616+0000 7f330ffff640 1 -- 172.21.15.64:0/23484555 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf407ec410 con 0x55bf40896760 2024-05-09T09:16:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.616+0000 7f330ffff640 10 client.26990 handle_command_reply: tid=0 2024-05-09T09:16:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.617+0000 7f3339ff4740 1 client.26990 shutdown 2024-05-09T09:16:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.623+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 msgr2=0x55bf4088b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.623+0000 7f3339ff4740 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf4088b0d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f33200314e0 tx=0x7f3320005270 comp rx=0 tx=0).stop 2024-05-09T09:16:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.626+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 shutdown_connections 2024-05-09T09:16:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.626+0000 7f3339ff4740 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf40896760 0x55bf40898be0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.626+0000 7f3339ff4740 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf40886020 0x55bf40892110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.626+0000 7f3339ff4740 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40880eb0 0x55bf4088b0d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.626+0000 7f3339ff4740 1 --2- 172.21.15.64:0/23484555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf408804c0 0x55bf4088ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.626+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 >> 172.21.15.64:0/23484555 conn(0x55bf408480a0 msgr2=0x55bf4087b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.627+0000 7f330ffff640 0 client.26990 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.627+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 shutdown_connections 2024-05-09T09:16:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.628+0000 7f3339ff4740 1 -- 172.21.15.64:0/23484555 wait complete. 2024-05-09T09:16:59.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.630+0000 7f3339ff4740 20 client.26990 trim_cache size 0 max 16384 2024-05-09T09:16:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.676+0000 7f3339ff4740 1 Processor -- start 2024-05-09T09:16:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.679+0000 7f3339ff4740 1 -- start start 2024-05-09T09:16:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.680+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf40889670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.680+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf408920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.680+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf40889bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.680+0000 7f3339ff4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3310012620 con 0x55bf408804c0 2024-05-09T09:16:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.680+0000 7f3339ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f331000fe90 con 0x55bf40886020 2024-05-09T09:16:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.680+0000 7f3339ff4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f331000dce0 con 0x55bf40880eb0 2024-05-09T09:16:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf40889bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf40889bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49436/0 (socket says 172.21.15.64:49436) 2024-05-09T09:16:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 -- 172.21.15.64:0/3057307340 learned_addr learned my addr 172.21.15.64:0/3057307340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f332ffff640 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf408920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 -- 172.21.15.64:0/3057307340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 msgr2=0x55bf408920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf408920d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 -- 172.21.15.64:0/3057307340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 msgr2=0x55bf40889670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf40889670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.682+0000 7f33357ad640 1 -- 172.21.15.64:0/3057307340 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf40881740 con 0x55bf40886020 2024-05-09T09:16:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.689+0000 7f33357ad640 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf40889bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f331800eca0 tx=0x7f3318004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77e6610f8c7a172 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.690+0000 7f33267fc640 1 -- 172.21.15.64:0/3057307340 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331801a050 con 0x55bf40886020 2024-05-09T09:16:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.690+0000 7f33267fc640 1 -- 172.21.15.64:0/3057307340 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3318024070 con 0x55bf40886020 2024-05-09T09:16:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.691+0000 7f33267fc640 1 -- 172.21.15.64:0/3057307340 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180146f0 con 0x55bf40886020 2024-05-09T09:16:59.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.695+0000 7f3339ff4740 1 -- 172.21.15.64:0/3057307340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 msgr2=0x55bf40889bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.695+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf40889bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f331800eca0 tx=0x7f3318004770 comp rx=0 tx=0).stop 2024-05-09T09:16:59.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.697+0000 7f3339ff4740 1 -- 172.21.15.64:0/3057307340 shutdown_connections 2024-05-09T09:16:59.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.697+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf40889bb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.697+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf408920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.697+0000 7f3339ff4740 1 --2- 172.21.15.64:0/3057307340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf40889670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.697+0000 7f3339ff4740 1 -- 172.21.15.64:0/3057307340 >> 172.21.15.64:0/3057307340 conn(0x55bf408480a0 msgr2=0x55bf4087b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.699+0000 7f3339ff4740 1 -- 172.21.15.64:0/3057307340 shutdown_connections 2024-05-09T09:16:59.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.699+0000 7f3339ff4740 1 -- 172.21.15.64:0/3057307340 wait complete. 2024-05-09T09:16:59.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.699+0000 7f3339ff4740 1 -- start start 2024-05-09T09:16:59.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.700+0000 7f3339ff4740 1 Processor -- start 2024-05-09T09:16:59.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.700+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf4088f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf4088fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f3339ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf408909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f332ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf4088fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f332ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf4088fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40180/0 (socket says 172.21.15.64:40180) 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f332ffff640 1 -- 172.21.15.64:0/1832761019 learned_addr learned my addr 172.21.15.64:0/1832761019 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f331000e350 con 0x55bf408804c0 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.701+0000 7f33357ad640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf408909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f331000f7c0 con 0x55bf40886020 2024-05-09T09:16:59.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f331000ea20 con 0x55bf40880eb0 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f3339ff4740 10 client.0 _ll_register_callbacks cb 0x7f3320009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f332ffff640 1 -- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 msgr2=0x55bf408909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f332ffff640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf408909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f332ffff640 1 -- 172.21.15.64:0/1832761019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 msgr2=0x55bf4088f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f332ffff640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf4088f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f332ffff640 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf40890fd0 con 0x55bf40880eb0 2024-05-09T09:16:59.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.702+0000 7f33357ad640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf408909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:16:59.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.705+0000 7f332ffff640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf4088fa90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f33200317e0 tx=0x7f3320031810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca670429b8ebd099 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.705+0000 7f330ffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:16:59.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.705+0000 7f330ffff640 1 -- 172.21.15.64:0/1832761019 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320040040 con 0x55bf40880eb0 2024-05-09T09:16:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.706+0000 7f330ffff640 1 -- 172.21.15.64:0/1832761019 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3320031970 con 0x55bf40880eb0 2024-05-09T09:16:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.706+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3318024070 con 0x55bf40880eb0 2024-05-09T09:16:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.706+0000 7f330ffff640 1 -- 172.21.15.64:0/1832761019 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3320035040 con 0x55bf40880eb0 2024-05-09T09:16:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.706+0000 7f3339ff4740 10 client.27157 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:59.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.706+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bf40892300 con 0x55bf40880eb0 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f330ffff640 1 -- 172.21.15.64:0/1832761019 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3318024070 con 0x55bf40880eb0 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f330ffff640 1 -- 172.21.15.64:0/1832761019 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f33200046c0 con 0x55bf40880eb0 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f3339ff4740 10 client.27157 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f3339ff4740 10 client.27157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f3339ff4740 20 client.27157 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f3339ff4740 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf408fd450 0x55bf408ff870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f3339ff4740 4 client.27157 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.707+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bf40769da0 con 0x55bf408fd450 2024-05-09T09:16:59.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.708+0000 7f3334fac640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf408fd450 0x55bf408ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.709+0000 7f3334fac640 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf408fd450 0x55bf408ff870 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8620779562962531 server_cookie=9713d7a3d317a7eb in_seq=0 out_seq=0 2024-05-09T09:16:59.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.709+0000 7f330ffff640 10 client.27157 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.714+0000 7f330ffff640 1 -- 172.21.15.64:0/1832761019 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bf40769da0 con 0x55bf408fd450 2024-05-09T09:16:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.714+0000 7f330ffff640 10 client.27157 handle_command_reply: tid=0 2024-05-09T09:16:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.714+0000 7f3339ff4740 1 client.27157 shutdown 2024-05-09T09:16:59.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.716+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 msgr2=0x55bf4088fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.716+0000 7f3339ff4740 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf4088fa90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f33200317e0 tx=0x7f3320031810 comp rx=0 tx=0).stop 2024-05-09T09:16:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.717+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 shutdown_connections 2024-05-09T09:16:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.717+0000 7f3339ff4740 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bf408fd450 0x55bf408ff870 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.717+0000 7f3339ff4740 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf40886020 0x55bf408909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.717+0000 7f3339ff4740 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf40880eb0 0x55bf4088fa90 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f33200317e0 tx=0x7f3320031810 comp rx=0 tx=0).stop 2024-05-09T09:16:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.717+0000 7f3339ff4740 1 --2- 172.21.15.64:0/1832761019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf408804c0 0x55bf4088f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.717+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 >> 172.21.15.64:0/1832761019 conn(0x55bf408480a0 msgr2=0x55bf4087ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.718+0000 7f330ffff640 0 client.27157 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.718+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 shutdown_connections 2024-05-09T09:16:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.718+0000 7f3339ff4740 1 -- 172.21.15.64:0/1832761019 wait complete. 2024-05-09T09:16:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.719+0000 7f3339ff4740 20 client.27157 trim_cache size 0 max 16384 2024-05-09T09:16:59.719 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:16:59.719 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:16:59.719 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "b38b5c9f-d20a-4359-be53-810a01b19ba7", 2024-05-09T09:16:59.719 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:16:59.719 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.720+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f331c0a2d90 msgr2=0x7f331c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.720+0000 7f3337210640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f331c0a2d90 0x7f331c0a5250 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55bf40881710 tx=0x7f3318008040 comp rx=0 tx=0).stop 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 msgr2=0x7f33301a6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 0x7f33301a6f80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f332800ba50 tx=0x7f332800bf10 comp rx=0 tx=0).stop 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 shutdown_connections 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f331c0a2d90 0x7f331c0a5250 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3330105400 0x7f33301ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3330102a50 0x7f33301a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 --2- 172.21.15.64:0/1602173919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33301000e0 0x7f33301a6f80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 >> 172.21.15.64:0/1602173919 conn(0x7f33300fbce0 msgr2=0x7f3330101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 shutdown_connections 2024-05-09T09:16:59.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.721+0000 7f3337210640 1 -- 172.21.15.64:0/1602173919 wait complete. 2024-05-09T09:16:59.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:16:59.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:16:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 -- 172.21.15.64:0/2251293632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 msgr2=0x7ffb6010e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/2251293632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb6010e660 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009900 tx=0x7ffb5002a400 comp rx=0 tx=0).stop 2024-05-09T09:16:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 -- 172.21.15.64:0/2251293632 shutdown_connections 2024-05-09T09:16:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/2251293632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6010ec80 0x7ffb60111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/2251293632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb6010e660 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/2251293632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb60109940 0x7ffb6010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.976+0000 7ffb64ab3640 1 -- 172.21.15.64:0/2251293632 >> 172.21.15.64:0/2251293632 conn(0x7ffb6006ea60 msgr2=0x7ffb60070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.977+0000 7ffb64ab3640 1 -- 172.21.15.64:0/2251293632 shutdown_connections 2024-05-09T09:16:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.977+0000 7ffb64ab3640 1 -- 172.21.15.64:0/2251293632 wait complete. 2024-05-09T09:16:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 Processor -- start 2024-05-09T09:16:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 -- start start 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb60109940 0x7ffb60109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb601a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6010ec80 0x7ffb601a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60114b00 con 0x7ffb60109940 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60114980 con 0x7ffb6010c270 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb64ab3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60114c80 con 0x7ffb6010ec80 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb5dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb601a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb5dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb601a4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49470/0 (socket says 172.21.15.64:49470) 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb5dd9b640 1 -- 172.21.15.64:0/3927547304 learned_addr learned my addr 172.21.15.64:0/3927547304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb60109940 0x7ffb60109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.978+0000 7ffb5ed9d640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6010ec80 0x7ffb601a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.979+0000 7ffb5dd9b640 1 -- 172.21.15.64:0/3927547304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6010ec80 msgr2=0x7ffb601a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.979+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6010ec80 0x7ffb601a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.979+0000 7ffb5dd9b640 1 -- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb60109940 msgr2=0x7ffb60109400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.979+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb60109940 0x7ffb60109400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.979+0000 7ffb5dd9b640 1 -- 172.21.15.64:0/3927547304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb601a7f20 con 0x7ffb6010c270 2024-05-09T09:16:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.979+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb601a4ff0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb500314e0 tx=0x7ffb50003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.980+0000 7ffb4f7fe640 1 -- 172.21.15.64:0/3927547304 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50035030 con 0x7ffb6010c270 2024-05-09T09:16:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.980+0000 7ffb4f7fe640 1 -- 172.21.15.64:0/3927547304 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb50041070 con 0x7ffb6010c270 2024-05-09T09:16:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.980+0000 7ffb4f7fe640 1 -- 172.21.15.64:0/3927547304 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50046470 con 0x7ffb6010c270 2024-05-09T09:16:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.980+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6019edc0 con 0x7ffb6010c270 2024-05-09T09:16:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.980+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6019f260 con 0x7ffb6010c270 2024-05-09T09:16:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.984+0000 7ffb4f7fe640 1 -- 172.21.15.64:0/3927547304 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb50031580 con 0x7ffb6010c270 2024-05-09T09:16:59.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.985+0000 7ffb4f7fe640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb480a2c90 0x7ffb480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.985+0000 7ffb4f7fe640 1 -- 172.21.15.64:0/3927547304 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffb500f3eb0 con 0x7ffb6010c270 2024-05-09T09:16:59.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.985+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb480a2c90 0x7ffb480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 Processor -- start 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 -- start start 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d4130 0x55e76f0d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 0x55e76f0db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e76f02fd00 con 0x55e76f0d4130 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e76efd74c0 con 0x55e76f0d92a0 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.986+0000 7ffb677be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e76f02bcc0 con 0x55e76f0d3740 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 0x55e76f0db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 0x55e76f0db730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49482/0 (socket says 172.21.15.64:49482) 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 -- 172.21.15.64:0/937035220 learned_addr learned my addr 172.21.15.64:0/937035220 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb480a2c90 0x7ffb480a5150 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009b60 tx=0x7ffb54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 -- 172.21.15.64:0/937035220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 msgr2=0x55e76f0d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 -- 172.21.15.64:0/937035220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d4130 msgr2=0x55e76f0d8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d4130 0x55e76f0d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.987+0000 7ffb5ed9d640 1 -- 172.21.15.64:0/937035220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76ee0d7d0 con 0x55e76f0d92a0 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.988+0000 7ffb5ed9d640 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 0x55e76f0db730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb58010370 tx=0x7ffb58010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701a671fea68ea7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.988+0000 7ffb4cff9640 1 -- 172.21.15.64:0/937035220 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58016070 con 0x55e76f0d92a0 2024-05-09T09:16:59.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.988+0000 7ffb4cff9640 1 -- 172.21.15.64:0/937035220 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb58004d20 con 0x55e76f0d92a0 2024-05-09T09:16:59.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.988+0000 7ffb4cff9640 1 -- 172.21.15.64:0/937035220 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58005400 con 0x55e76f0d92a0 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 -- 172.21.15.64:0/937035220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 msgr2=0x55e76f0db730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 0x55e76f0db730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb58010370 tx=0x7ffb58010830 comp rx=0 tx=0).stop 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 -- 172.21.15.64:0/937035220 shutdown_connections 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d92a0 0x55e76f0db730 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d4130 0x55e76f0d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 --2- 172.21.15.64:0/937035220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.989+0000 7ffb677be740 1 -- 172.21.15.64:0/937035220 >> 172.21.15.64:0/937035220 conn(0x55e76f0cedb0 msgr2=0x55e76f0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.990+0000 7ffb677be740 1 -- 172.21.15.64:0/937035220 shutdown_connections 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.990+0000 7ffb677be740 1 -- 172.21.15.64:0/937035220 wait complete. 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.990+0000 7ffb677be740 1 -- start start 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb677be740 1 Processor -- start 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb677be740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 0x55e76f0ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb677be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d4130 0x55e76f0de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb677be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 0x55e76f0ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 0x55e76f0ddc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:49488/0 (socket says 172.21.15.64:49488) 2024-05-09T09:16:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb5e59c640 1 -- 172.21.15.64:0/1680538907 learned_addr learned my addr 172.21.15.64:0/1680538907 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:16:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d4130 0x55e76f0de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.991+0000 7ffb677be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e76f0dc250 con 0x55e76f0d92a0 2024-05-09T09:16:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e76efeab90 con 0x55e76f0d3740 2024-05-09T09:16:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e76f0dc3d0 con 0x55e76f0d4130 2024-05-09T09:16:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb677be740 10 client.0 _ll_register_callbacks cb 0x55e76f026da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:16:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb5e59c640 1 -- 172.21.15.64:0/1680538907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d4130 msgr2=0x55e76f0de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d4130 0x55e76f0de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb5e59c640 1 -- 172.21.15.64:0/1680538907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 msgr2=0x55e76f0e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb5e59c640 1 -- 172.21.15.64:0/1680538907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76ed433f0 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb58004d20 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.992+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 0x55e76f0ddc20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb54012c10 tx=0x7ffb54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4399130c5efff167 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.993+0000 7ffb45ffb640 10 client.27008 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.993+0000 7ffb45ffb640 1 -- 172.21.15.64:0/1680538907 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54012c40 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.993+0000 7ffb45ffb640 1 -- 172.21.15.64:0/1680538907 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffb54012de0 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.993+0000 7ffb45ffb640 1 -- 172.21.15.64:0/1680538907 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffb54029430 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.993+0000 7ffb677be740 10 client.27008 fetch_fsmap learned FSMap version 43 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.993+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55e76ed030f0 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb45ffb640 1 -- 172.21.15.64:0/1680538907 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ffb540295d0 con 0x55e76f0d3740 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb677be740 10 client.27008 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb677be740 10 client.27008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb677be740 20 client.27008 populate_metadata read hostname 'smithi064' 2024-05-09T09:16:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb677be740 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f0e98c0 0x55e76f0ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:16:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb677be740 4 client.27008 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:16:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e76f03d430 con 0x55e76f0e98c0 2024-05-09T09:16:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.994+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f0e98c0 0x55e76f0ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.995+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f0e98c0 0x55e76f0ebd50 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9eace5365772a88 server_cookie=f9a831a93c0c6d1 in_seq=0 out_seq=0 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.995+0000 7ffb45ffb640 10 client.27008 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb45ffb640 1 -- 172.21.15.64:0/1680538907 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e76f03d430 con 0x55e76f0e98c0 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb45ffb640 10 client.27008 handle_command_reply: tid=0 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 client.27008 shutdown 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 msgr2=0x55e76f0ddc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 0x55e76f0ddc20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb54012c10 tx=0x7ffb54004730 comp rx=0 tx=0).stop 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 shutdown_connections 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f0e98c0 0x55e76f0ebd50 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d4130 0x55e76f0de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 --2- 172.21.15.64:0/1680538907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d3740 0x55e76f0ddc20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:16:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb45ffb640 0 client.27008 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:16:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 >> 172.21.15.64:0/1680538907 conn(0x55e76f0cedb0 msgr2=0x55e76f0c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:16:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 shutdown_connections 2024-05-09T09:16:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 1 -- 172.21.15.64:0/1680538907 wait complete. 2024-05-09T09:16:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:16:59.996+0000 7ffb677be740 20 client.27008 trim_cache size 0 max 16384 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.033+0000 7ffb677be740 1 Processor -- start 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.033+0000 7ffb677be740 1 -- start start 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.033+0000 7ffb677be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb677be740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb677be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb677be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601b3420 con 0x55e76f0d92a0 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb677be740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601b0c90 con 0x55e76f0d4130 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb677be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601aeae0 con 0x55e76f0d3740 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34844/0 (socket says 172.21.15.64:34844) 2024-05-09T09:17:00.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5ed9d640 1 -- 172.21.15.64:0/2186585061 learned_addr learned my addr 172.21.15.64:0/2186585061 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:40218/0 (socket says 172.21.15.64:40218) 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 -- 172.21.15.64:0/2186585061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 msgr2=0x55e76f0ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 -- 172.21.15.64:0/2186585061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 msgr2=0x55e76f0c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.034+0000 7ffb5e59c640 1 -- 172.21.15.64:0/2186585061 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76f0d2cd0 con 0x55e76f0d3740 2024-05-09T09:17:00.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.035+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0dd830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb5401fc20 tx=0x7ffb54022070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f1e9d1049440f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:00.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.035+0000 7ffb47fff640 1 -- 172.21.15.64:0/2186585061 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb540295d0 con 0x55e76f0d3740 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.035+0000 7ffb47fff640 1 -- 172.21.15.64:0/2186585061 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb54039070 con 0x55e76f0d3740 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.035+0000 7ffb47fff640 1 -- 172.21.15.64:0/2186585061 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54004990 con 0x55e76f0d3740 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 -- 172.21.15.64:0/2186585061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 msgr2=0x55e76f0dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0dd830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb5401fc20 tx=0x7ffb54022070 comp rx=0 tx=0).stop 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 -- 172.21.15.64:0/2186585061 shutdown_connections 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d92a0 0x55e76f0c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 --2- 172.21.15.64:0/2186585061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d3740 0x55e76f0dd830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 -- 172.21.15.64:0/2186585061 >> 172.21.15.64:0/2186585061 conn(0x55e76f0cedb0 msgr2=0x55e76f0d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 -- 172.21.15.64:0/2186585061 shutdown_connections 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.036+0000 7ffb677be740 1 -- 172.21.15.64:0/2186585061 wait complete. 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.037+0000 7ffb677be740 1 -- start start 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.037+0000 7ffb677be740 1 Processor -- start 2024-05-09T09:17:00.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.037+0000 7ffb677be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 0x55e76f0e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 0x55e76f0e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb677be740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 0x55e76f0e2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34856/0 (socket says 172.21.15.64:34856) 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb677be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d92a0 0x55e76f0e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb677be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601af150 con 0x55e76f0d3740 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601b05c0 con 0x55e76f0d4130 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb5e59c640 1 -- 172.21.15.64:0/26931045 learned_addr learned my addr 172.21.15.64:0/26931045 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601af820 con 0x55e76f0d92a0 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.038+0000 7ffb677be740 10 client.0 _ll_register_callbacks cb 0x7ffb54011c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5ed9d640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d92a0 0x55e76f0e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5e59c640 1 -- 172.21.15.64:0/26931045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d92a0 msgr2=0x55e76f0e3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d92a0 0x55e76f0e3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5e59c640 1 -- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 msgr2=0x55e76f0e26b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0e26b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5e59c640 1 -- 172.21.15.64:0/26931045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76f0e3c60 con 0x55e76f0d3740 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5ed9d640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d92a0 0x55e76f0e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb5e59c640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 0x55e76f0e2170 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffb54029a50 tx=0x7ffb54009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2f259a25e20e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.039+0000 7ffb45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.040+0000 7ffb45ffb640 1 -- 172.21.15.64:0/26931045 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54004220 con 0x55e76f0d3740 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.040+0000 7ffb45ffb640 1 -- 172.21.15.64:0/26931045 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffb540043c0 con 0x55e76f0d3740 2024-05-09T09:17:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.040+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb54039070 con 0x55e76f0d3740 2024-05-09T09:17:00.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.040+0000 7ffb45ffb640 1 -- 172.21.15.64:0/26931045 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffb54029d10 con 0x55e76f0d3740 2024-05-09T09:17:00.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.040+0000 7ffb677be740 10 client.17547 fetch_fsmap learned FSMap version 43 2024-05-09T09:17:00.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.040+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e76f1501c0 con 0x55e76f0d3740 2024-05-09T09:17:00.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb45ffb640 1 -- 172.21.15.64:0/26931045 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb58004d20 con 0x55e76f0d3740 2024-05-09T09:17:00.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb45ffb640 1 -- 172.21.15.64:0/26931045 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ffb54033500 con 0x55e76f0d3740 2024-05-09T09:17:00.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb677be740 10 client.17547 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:17:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb677be740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb677be740 20 client.17547 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb677be740 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f150a50 0x55e76f152e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb677be740 4 client.17547 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:17:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.041+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ffb54039070 con 0x55e76f150a50 2024-05-09T09:17:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.042+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f150a50 0x55e76f152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.042+0000 7ffb5dd9b640 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f150a50 0x55e76f152e40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ea094364664938 server_cookie=acaa05dbea2803e5 in_seq=0 out_seq=0 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.043+0000 7ffb45ffb640 10 client.17547 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.043+0000 7ffb45ffb640 1 -- 172.21.15.64:0/26931045 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7ffb54039070 con 0x55e76f150a50 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.043+0000 7ffb45ffb640 10 client.17547 handle_command_reply: tid=0 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.043+0000 7ffb677be740 1 client.17547 shutdown 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.043+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 msgr2=0x55e76f0e2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.043+0000 7ffb677be740 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 0x55e76f0e2170 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffb54029a50 tx=0x7ffb54009400 comp rx=0 tx=0).stop 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 shutdown_connections 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e76f150a50 0x55e76f152e40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e76f0d92a0 0x55e76f0e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e76f0d4130 0x55e76f0e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb45ffb640 0 client.17547 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 --2- 172.21.15.64:0/26931045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e76f0d3740 0x55e76f0e2170 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 >> 172.21.15.64:0/26931045 conn(0x55e76f0cedb0 msgr2=0x55e76f0dd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 shutdown_connections 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 1 -- 172.21.15.64:0/26931045 wait complete. 2024-05-09T09:17:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.044+0000 7ffb677be740 20 client.17547 trim_cache size 0 max 16384 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "b38b5c9f-d20a-4359-be53-810a01b19ba7": { 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "b38b5c9f-d20a-4359-be53-810a01b19ba7", 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:17:00.045 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:17:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.045+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb480a2c90 msgr2=0x7ffb480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.045+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb480a2c90 0x7ffb480a5150 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009b60 tx=0x7ffb54005f90 comp rx=0 tx=0).stop 2024-05-09T09:17:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.045+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 msgr2=0x7ffb601a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.045+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb601a4ff0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb500314e0 tx=0x7ffb50003c10 comp rx=0 tx=0).stop 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 shutdown_connections 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffb480a2c90 0x7ffb480a5150 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6010ec80 0x7ffb601a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6010c270 0x7ffb601a4ff0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 --2- 172.21.15.64:0/3927547304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb60109940 0x7ffb60109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 >> 172.21.15.64:0/3927547304 conn(0x7ffb6006ea60 msgr2=0x7ffb6010a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 shutdown_connections 2024-05-09T09:17:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:00.046+0000 7ffb64ab3640 1 -- 172.21.15.64:0/3927547304 wait complete. 2024-05-09T09:17:00.062 INFO:tasks.cephfs.filesystem:scrub status for tag:b38b5c9f-d20a-4359-be53-810a01b19ba7 - {'path': '/', 'tag': 'b38b5c9f-d20a-4359-be53-810a01b19ba7', 'options': 'recursive,force'} 2024-05-09T09:17:00.073 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:16:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:16:59.744+0000 7f3d1eb03700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans/ auth{2=1} v2916 ap=2 f(v8 m2024-05-09T09:15:52.500587+0000 1=1+0) n(v9 rc2024-05-09T09:15:52.500587+0000 b8192 2=1+1) (inest sync->mix g=2) caps={15456=pAsLsXs/-@40} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55f5a7ce6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v501>,<0x10000000005/data v111>,<0x10000000003/postgres v28>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v39>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v45>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2916>,<0x10000000005/data v1350>,<0x10000000003/postgres v584>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v590>,<0x10000000001/sv_0 v604>,<0x10000000000/qa v589>,<0x1/volumes v552>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:15:52.484381+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:15:52.500587+0000 b8192 2=1+1)","memory_value.dirstat":"f(v8 m2024-05-09T09:15:52.500587+0000 1=1+0)","memory_value.rstat":"n(v9 rc2024-05-09T09:15:52.500587+0000 b8192 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:17:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-05-09T09:17:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:17:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:17:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:17:00.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:17:00.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:17:00.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:00.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:00.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:00 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:00 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:17:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:00 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 400.0 s, 672.4 tps, lat 148.152 ms stddev 174.193 2024-05-09T09:17:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:02 smithi179 ceph-mon[34086]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.1 MiB/s wr, 844 op/s 2024-05-09T09:17:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:02 smithi139 ceph-mon[33599]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.1 MiB/s wr, 844 op/s 2024-05-09T09:17:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:02 smithi064 ceph-mon[30149]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.1 MiB/s wr, 844 op/s 2024-05-09T09:17:04.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:04 smithi179 ceph-mon[34086]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.9 MiB/s wr, 962 op/s 2024-05-09T09:17:05.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:04 smithi139 ceph-mon[33599]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.9 MiB/s wr, 962 op/s 2024-05-09T09:17:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:04 smithi064 ceph-mon[30149]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.9 MiB/s wr, 962 op/s 2024-05-09T09:17:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 405.0 s, 908.0 tps, lat 110.707 ms stddev 58.779 2024-05-09T09:17:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:06 smithi139 ceph-mon[33599]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 997 op/s 2024-05-09T09:17:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:06 smithi064 ceph-mon[30149]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 997 op/s 2024-05-09T09:17:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:06 smithi179 ceph-mon[34086]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 997 op/s 2024-05-09T09:17:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:08 smithi139 ceph-mon[33599]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.2 MiB/s wr, 858 op/s 2024-05-09T09:17:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:08 smithi064 ceph-mon[30149]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.2 MiB/s wr, 858 op/s 2024-05-09T09:17:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:08 smithi179 ceph-mon[34086]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.2 MiB/s wr, 858 op/s 2024-05-09T09:17:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:10 smithi139 ceph-mon[33599]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-09T09:17:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:17:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:17:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:17:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:10 smithi064 ceph-mon[30149]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-09T09:17:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:17:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:17:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:17:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:10 smithi179 ceph-mon[34086]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-09T09:17:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:17:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:17:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:17:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 410.0 s, 754.6 tps, lat 97.681 ms stddev 49.390 2024-05-09T09:17:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.472+0000 7f2818f9a640 1 -- 172.21.15.64:0/1760651043 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810030850 con 0x7f281410b1a0 2024-05-09T09:17:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.473+0000 7f281c200640 1 -- 172.21.15.64:0/1760651043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 msgr2=0x7f281410b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.473+0000 7f281c200640 1 --2- 172.21.15.64:0/1760651043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410b600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2810009860 tx=0x7f281002a340 comp rx=0 tx=0).stop 2024-05-09T09:17:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.475+0000 7f281c200640 1 -- 172.21.15.64:0/1760651043 shutdown_connections 2024-05-09T09:17:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.475+0000 7f281c200640 1 --2- 172.21.15.64:0/1760651043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814111f70 0x7f28141143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.475+0000 7f281c200640 1 --2- 172.21.15.64:0/1760651043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410b600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.475+0000 7f281c200640 1 --2- 172.21.15.64:0/1760651043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814108830 0x7f281410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.475+0000 7f281c200640 1 -- 172.21.15.64:0/1760651043 >> 172.21.15.64:0/1760651043 conn(0x7f281406e8a0 msgr2=0x7f2814070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.479+0000 7f281c200640 1 -- 172.21.15.64:0/1760651043 shutdown_connections 2024-05-09T09:17:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.479+0000 7f281c200640 1 -- 172.21.15.64:0/1760651043 wait complete. 2024-05-09T09:17:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 Processor -- start 2024-05-09T09:17:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 -- start start 2024-05-09T09:17:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814108830 0x7f281410f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814111f70 0x7f281410c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2814117e20 con 0x7f2814108830 2024-05-09T09:17:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2814117ca0 con 0x7f281410b1a0 2024-05-09T09:17:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.480+0000 7f281c200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2814117fa0 con 0x7f2814111f70 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43216/0 (socket says 172.21.15.64:43216) 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f2819f9c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814108830 0x7f281410f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 -- 172.21.15.64:0/231553836 learned_addr learned my addr 172.21.15.64:0/231553836 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281a79d640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814111f70 0x7f281410c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 -- 172.21.15.64:0/231553836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814111f70 msgr2=0x7f281410c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814111f70 0x7f281410c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 -- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814108830 msgr2=0x7f281410f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814108830 0x7f281410f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 -- 172.21.15.64:0/231553836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281410c8f0 con 0x7f281410b1a0 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.481+0000 7f281979b640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410fa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2810035040 tx=0x7f2810030d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.482+0000 7f280affd640 1 -- 172.21.15.64:0/231553836 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810040020 con 0x7f281410b1a0 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.482+0000 7f280affd640 1 -- 172.21.15.64:0/231553836 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2810038040 con 0x7f281410b1a0 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.482+0000 7f280affd640 1 -- 172.21.15.64:0/231553836 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810037070 con 0x7f281410b1a0 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.482+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141a20c0 con 0x7f281410b1a0 2024-05-09T09:17:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.482+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a2560 con 0x7f281410b1a0 2024-05-09T09:17:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.486+0000 7f280affd640 1 -- 172.21.15.64:0/231553836 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2810043020 con 0x7f281410b1a0 2024-05-09T09:17:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.487+0000 7f280affd640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2df0 0x7f28000a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.487+0000 7f280affd640 1 -- 172.21.15.64:0/231553836 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28100efdd0 con 0x7f281410b1a0 2024-05-09T09:17:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 Processor -- start 2024-05-09T09:17:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 -- start start 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f2819f9c640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2df0 0x7f28000a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861c250 0x563b6861c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 0x563b68621740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b68621db0 0x563b68624240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563b68580340 con 0x563b68621db0 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563b68528290 con 0x563b6861c250 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.488+0000 7f281efed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563b6857c300 con 0x563b6861cc40 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.489+0000 7f281979b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 0x563b68621740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.489+0000 7f281979b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 0x563b68621740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34790/0 (socket says 172.21.15.64:34790) 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.489+0000 7f281979b640 1 -- 172.21.15.64:0/4034744041 learned_addr learned my addr 172.21.15.64:0/4034744041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.489+0000 7f281a79d640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b68621db0 0x563b68624240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.489+0000 7f281979b640 1 -- 172.21.15.64:0/4034744041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861c250 msgr2=0x563b6861c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.489+0000 7f2819f9c640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861c250 0x563b6861c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f281979b640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861c250 0x563b6861c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f281979b640 1 -- 172.21.15.64:0/4034744041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b68621db0 msgr2=0x563b68624240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f281979b640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b68621db0 0x563b68624240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f281979b640 1 -- 172.21.15.64:0/4034744041 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6854c8e0 con 0x563b6861cc40 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f281a79d640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b68621db0 0x563b68624240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:17:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f2819f9c640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861c250 0x563b6861c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:17:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.490+0000 7f281979b640 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 0x563b68621740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28100c49e0 tx=0x7f281002a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42ff9ef42c6e5d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.491+0000 7f2819f9c640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2df0 0x7f28000a52b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2804005fd0 tx=0x7f2804009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.491+0000 7f27fffff640 1 -- 172.21.15.64:0/4034744041 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100bc2a0 con 0x563b6861cc40 2024-05-09T09:17:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.491+0000 7f27fffff640 1 -- 172.21.15.64:0/4034744041 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28100c0650 con 0x563b6861cc40 2024-05-09T09:17:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.491+0000 7f27fffff640 1 -- 172.21.15.64:0/4034744041 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100c4a80 con 0x563b6861cc40 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.492+0000 7f281efed740 1 -- 172.21.15.64:0/4034744041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 msgr2=0x563b68621740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.492+0000 7f281efed740 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 0x563b68621740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28100c49e0 tx=0x7f281002a840 comp rx=0 tx=0).stop 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.495+0000 7f281efed740 1 -- 172.21.15.64:0/4034744041 shutdown_connections 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.495+0000 7f281efed740 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b68621db0 0x563b68624240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.495+0000 7f281efed740 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b6861cc40 0x563b68621740 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.495+0000 7f281efed740 1 --2- 172.21.15.64:0/4034744041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861c250 0x563b6861c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.495+0000 7f281efed740 1 -- 172.21.15.64:0/4034744041 >> 172.21.15.64:0/4034744041 conn(0x563b686178c0 msgr2=0x563b68617cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.496+0000 7f281efed740 1 -- 172.21.15.64:0/4034744041 shutdown_connections 2024-05-09T09:17:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.496+0000 7f281efed740 1 -- 172.21.15.64:0/4034744041 wait complete. 2024-05-09T09:17:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.497+0000 7f281efed740 1 -- start start 2024-05-09T09:17:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.497+0000 7f281efed740 1 Processor -- start 2024-05-09T09:17:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.498+0000 7f281efed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b686261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281efed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b68626720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281efed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b6862d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281979b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b68626720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281979b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b68626720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43230/0 (socket says 172.21.15.64:43230) 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281979b640 1 -- 172.21.15.64:0/4285465057 learned_addr learned my addr 172.21.15.64:0/4285465057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563b68624b20 con 0x563b6861c250 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563b6853b790 con 0x563b6861cc40 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.499+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563b68624ca0 con 0x563b68621db0 2024-05-09T09:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.500+0000 7f281efed740 10 client.0 _ll_register_callbacks cb 0x563b68576a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.501+0000 7f281979b640 1 -- 172.21.15.64:0/4285465057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 msgr2=0x563b6862d760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.501+0000 7f281979b640 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b6862d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.501+0000 7f281979b640 1 -- 172.21.15.64:0/4285465057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 msgr2=0x563b686261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.501+0000 7f281979b640 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b686261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.501+0000 7f281979b640 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b68294210 con 0x563b6861cc40 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.502+0000 7f281979b640 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b68626720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2810009830 tx=0x7f28100b8a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd6dd2d9f4ae7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.502+0000 7f27fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.502+0000 7f27fd7fa640 1 -- 172.21.15.64:0/4285465057 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100d9030 con 0x563b6861cc40 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.502+0000 7f27fd7fa640 1 -- 172.21.15.64:0/4285465057 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28100ba5a0 con 0x563b6861cc40 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.503+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28100c0650 con 0x563b6861cc40 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.503+0000 7f27fd7fa640 1 -- 172.21.15.64:0/4285465057 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f28100d0050 con 0x563b6861cc40 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.503+0000 7f281efed740 10 client.27020 fetch_fsmap learned FSMap version 43 2024-05-09T09:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.503+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563b68360620 con 0x563b6861cc40 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.503+0000 7f27fd7fa640 1 -- 172.21.15.64:0/4285465057 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28100c4f10 con 0x563b6861cc40 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.503+0000 7f27fd7fa640 1 -- 172.21.15.64:0/4285465057 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f28100dc0b0 con 0x563b6861cc40 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f281efed740 10 client.27020 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f281efed740 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f281efed740 20 client.27020 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f281efed740 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b68633ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f281efed740 4 client.27020 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f28100c0650 con 0x563b68631a90 2024-05-09T09:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f2819f9c640 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b68633ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f2819f9c640 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b68633ec0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2209258384d7c8b9 server_cookie=4da8c383ced6666a in_seq=0 out_seq=0 2024-05-09T09:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.504+0000 7f27fd7fa640 10 client.27020 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.505+0000 7f27fd7fa640 1 -- 172.21.15.64:0/4285465057 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f28100c0650 con 0x563b68631a90 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.505+0000 7f27fd7fa640 10 client.27020 handle_command_reply: tid=0 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.505+0000 7f281efed740 1 client.27020 shutdown 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 msgr2=0x563b68626720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b68626720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2810009830 tx=0x7f28100b8a00 comp rx=0 tx=0).stop 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 shutdown_connections 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b68633ec0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f27fd7fa640 0 client.27020 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b6862d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b68626720 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 --2- 172.21.15.64:0/4285465057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b686261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 >> 172.21.15.64:0/4285465057 conn(0x563b686178c0 msgr2=0x563b68616880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 shutdown_connections 2024-05-09T09:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.506+0000 7f281efed740 1 -- 172.21.15.64:0/4285465057 wait complete. 2024-05-09T09:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.507+0000 7f281efed740 20 client.27020 trim_cache size 0 max 16384 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.537+0000 7f281efed740 1 Processor -- start 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.538+0000 7f281efed740 1 -- start start 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.538+0000 7f281efed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b68615f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.538+0000 7f2819f9c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b68615f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.538+0000 7f2819f9c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b68615f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51328/0 (socket says 172.21.15.64:51328) 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f2819f9c640 1 -- 172.21.15.64:0/4044195906 learned_addr learned my addr 172.21.15.64:0/4044195906 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281efed740 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281efed740 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68616440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28141c42d0 con 0x563b6861c250 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28141c1b40 con 0x563b6861cc40 2024-05-09T09:17:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28141bf9c0 con 0x563b68621db0 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281979b640 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.539+0000 7f281a79d640 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68616440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f281979b640 1 -- 172.21.15.64:0/4044195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 msgr2=0x563b68616440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f281979b640 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68616440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f281979b640 1 -- 172.21.15.64:0/4044195906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 msgr2=0x563b68615f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f281979b640 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b68615f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f281979b640 1 -- 172.21.15.64:0/4044195906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b68611ba0 con 0x563b6861cc40 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f281979b640 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862d720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28100b8700 tx=0x7f28100035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e6beb2924acd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f27ff7fe640 1 -- 172.21.15.64:0/4044195906 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100d9070 con 0x563b6861cc40 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f27ff7fe640 1 -- 172.21.15.64:0/4044195906 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28100b8a80 con 0x563b6861cc40 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.540+0000 7f27ff7fe640 1 -- 172.21.15.64:0/4044195906 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100bc2a0 con 0x563b6861cc40 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.542+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 msgr2=0x563b6862d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.542+0000 7f281efed740 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862d720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28100b8700 tx=0x7f28100035f0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.544+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 shutdown_connections 2024-05-09T09:17:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.544+0000 7f281efed740 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68616440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.544+0000 7f281efed740 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862d720 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.544+0000 7f281efed740 1 --2- 172.21.15.64:0/4044195906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b68615f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.544+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 >> 172.21.15.64:0/4044195906 conn(0x563b686178c0 msgr2=0x563b6861dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.546+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 shutdown_connections 2024-05-09T09:17:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.546+0000 7f281efed740 1 -- 172.21.15.64:0/4044195906 wait complete. 2024-05-09T09:17:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.546+0000 7f281efed740 1 -- start start 2024-05-09T09:17:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.547+0000 7f281efed740 1 Processor -- start 2024-05-09T09:17:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.547+0000 7f281efed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b6862a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.547+0000 7f281efed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281efed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68696030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281efed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28141c0000 con 0x563b6861c250 2024-05-09T09:17:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281efed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28141c1470 con 0x563b6861cc40 2024-05-09T09:17:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281a79d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68696030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281a79d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68696030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34798/0 (socket says 172.21.15.64:34798) 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281a79d640 1 -- 172.21.15.64:0/561149864 learned_addr learned my addr 172.21.15.64:0/561149864 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281979b640 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28141c06d0 con 0x563b68621db0 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281efed740 10 client.0 _ll_register_callbacks cb 0x7f28100bc070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281a79d640 1 -- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 msgr2=0x563b6862ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.548+0000 7f281a79d640 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.549+0000 7f281a79d640 1 -- 172.21.15.64:0/561149864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 msgr2=0x563b6862a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.549+0000 7f281a79d640 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b6862a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.549+0000 7f281a79d640 1 -- 172.21.15.64:0/561149864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b686965f0 con 0x563b68621db0 2024-05-09T09:17:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.549+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28100b8a80 con 0x563b68621db0 2024-05-09T09:17:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.550+0000 7f281a79d640 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68696030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f280c012d20 tx=0x7f280c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6e80c099d8002 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.551+0000 7f27fd7fa640 10 client.27187 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.551+0000 7f27fd7fa640 1 -- 172.21.15.64:0/561149864 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c004b20 con 0x563b68621db0 2024-05-09T09:17:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.551+0000 7f27fd7fa640 1 -- 172.21.15.64:0/561149864 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f280c004cc0 con 0x563b68621db0 2024-05-09T09:17:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.551+0000 7f27fd7fa640 1 -- 172.21.15.64:0/561149864 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f280c003970 con 0x563b68621db0 2024-05-09T09:17:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.552+0000 7f281efed740 10 client.27187 fetch_fsmap learned FSMap version 43 2024-05-09T09:17:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.552+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=43}) -- 0x563b68697870 con 0x563b68621db0 2024-05-09T09:17:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f27fd7fa640 1 -- 172.21.15.64:0/561149864 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f280c01a070 con 0x563b68621db0 2024-05-09T09:17:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f281efed740 10 client.27187 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T09:17:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f281efed740 10 client.27187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f281efed740 20 client.27187 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f281efed740 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b6869a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f281efed740 4 client.27187 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:17:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.556+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b6858da70 con 0x563b68631a90 2024-05-09T09:17:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.558+0000 7f2819f9c640 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b6869a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.559+0000 7f2819f9c640 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b6869a1a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6f75326d3407e server_cookie=817de5d85b0448ce in_seq=0 out_seq=0 2024-05-09T09:17:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.560+0000 7f27fd7fa640 10 client.27187 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.586+0000 7f27fd7fa640 1 -- 172.21.15.64:0/561149864 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b6858da70 con 0x563b68631a90 2024-05-09T09:17:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.586+0000 7f27fd7fa640 10 client.27187 handle_command_reply: tid=0 2024-05-09T09:17:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.587+0000 7f281efed740 1 client.27187 shutdown 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 msgr2=0x563b68696030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68696030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f280c012d20 tx=0x7f280c0107a0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 shutdown_connections 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563b68631a90 0x563b6869a1a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b68621db0 0x563b68696030 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563b6861cc40 0x563b6862ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 --2- 172.21.15.64:0/561149864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b6861c250 0x563b6862a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 >> 172.21.15.64:0/561149864 conn(0x563b686178c0 msgr2=0x563b6861a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.588+0000 7f27fd7fa640 0 client.27187 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.590+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 shutdown_connections 2024-05-09T09:17:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.590+0000 7f281efed740 1 -- 172.21.15.64:0/561149864 wait complete. 2024-05-09T09:17:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.591+0000 7f281efed740 20 client.27187 trim_cache size 0 max 16384 2024-05-09T09:17:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:17:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:17:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:17:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:17:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.594+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2df0 msgr2=0x7f28000a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.594+0000 7f281c200640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2df0 0x7f28000a52b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2804005fd0 tx=0x7f2804009780 comp rx=0 tx=0).stop 2024-05-09T09:17:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.594+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 msgr2=0x7f281410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.594+0000 7f281c200640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410fa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2810035040 tx=0x7f2810030d80 comp rx=0 tx=0).stop 2024-05-09T09:17:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.596+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 shutdown_connections 2024-05-09T09:17:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.596+0000 7f281c200640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f28000a2df0 0x7f28000a52b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.596+0000 7f281c200640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2814111f70 0x7f281410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.596+0000 7f281c200640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281410b1a0 0x7f281410fa00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.596+0000 7f281c200640 1 --2- 172.21.15.64:0/231553836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2814108830 0x7f281410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.596+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 >> 172.21.15.64:0/231553836 conn(0x7f281406e8a0 msgr2=0x7f2814109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.599+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 shutdown_connections 2024-05-09T09:17:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:11.599+0000 7f281c200640 1 -- 172.21.15.64:0/231553836 wait complete. 2024-05-09T09:17:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:12 smithi139 ceph-mon[33599]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.0 MiB/s wr, 918 op/s 2024-05-09T09:17:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:12 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:17:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:12 smithi064 ceph-mon[30149]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.0 MiB/s wr, 918 op/s 2024-05-09T09:17:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:12 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:17:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:12 smithi179 ceph-mon[34086]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.0 MiB/s wr, 918 op/s 2024-05-09T09:17:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:12 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:17:15.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:14 smithi139 ceph-mon[33599]: pgmap v613: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-05-09T09:17:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:14 smithi064 ceph-mon[30149]: pgmap v613: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-05-09T09:17:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:14 smithi179 ceph-mon[34086]: pgmap v613: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-05-09T09:17:16.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 415.0 s, 938.0 tps, lat 135.046 ms stddev 590.345 2024-05-09T09:17:17.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:16 smithi139 ceph-mon[33599]: pgmap v614: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.7 MiB/s wr, 969 op/s 2024-05-09T09:17:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:16 smithi064 ceph-mon[30149]: pgmap v614: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.7 MiB/s wr, 969 op/s 2024-05-09T09:17:17.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:16 smithi179 ceph-mon[34086]: pgmap v614: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.7 MiB/s wr, 969 op/s 2024-05-09T09:17:18.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:17 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:17:18.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:17 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:17:18.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:17 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:17:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:18 smithi139 ceph-mon[33599]: pgmap v615: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 817 op/s 2024-05-09T09:17:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:18 smithi064 ceph-mon[30149]: pgmap v615: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 817 op/s 2024-05-09T09:17:19.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:18 smithi179 ceph-mon[34086]: pgmap v615: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 817 op/s 2024-05-09T09:17:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:20 smithi139 ceph-mon[33599]: pgmap v616: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-05-09T09:17:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:20 smithi064 ceph-mon[30149]: pgmap v616: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-05-09T09:17:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:20 smithi179 ceph-mon[34086]: pgmap v616: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-05-09T09:17:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 420.0 s, 826.4 tps, lat 120.603 ms stddev 80.200 2024-05-09T09:17:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:22 smithi139 ceph-mon[33599]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 896 op/s 2024-05-09T09:17:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:22 smithi064 ceph-mon[30149]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 896 op/s 2024-05-09T09:17:23.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:22 smithi179 ceph-mon[34086]: pgmap v617: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 896 op/s 2024-05-09T09:17:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.764+0000 7ffa81563640 1 -- 172.21.15.64:0/44015695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c100160 msgr2=0x7ffa7c100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.764+0000 7ffa81563640 1 --2- 172.21.15.64:0/44015695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c100160 0x7ffa7c100580 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009860 tx=0x7ffa6c02a340 comp rx=0 tx=0).stop 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.765+0000 7ffa81563640 1 -- 172.21.15.64:0/44015695 shutdown_connections 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.765+0000 7ffa81563640 1 --2- 172.21.15.64:0/44015695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa7c101460 0x7ffa7c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.765+0000 7ffa81563640 1 --2- 172.21.15.64:0/44015695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100ac0 0x7ffa7c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.765+0000 7ffa81563640 1 --2- 172.21.15.64:0/44015695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c100160 0x7ffa7c100580 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.765+0000 7ffa81563640 1 -- 172.21.15.64:0/44015695 >> 172.21.15.64:0/44015695 conn(0x7ffa7c0fbd00 msgr2=0x7ffa7c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.765+0000 7ffa81563640 1 -- 172.21.15.64:0/44015695 shutdown_connections 2024-05-09T09:17:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.766+0000 7ffa81563640 1 -- 172.21.15.64:0/44015695 wait complete. 2024-05-09T09:17:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.767+0000 7ffa81563640 1 Processor -- start 2024-05-09T09:17:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.767+0000 7ffa81563640 1 -- start start 2024-05-09T09:17:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.767+0000 7ffa81563640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 0x7ffa7c105ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.767+0000 7ffa81563640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa7c100ac0 0x7ffa7c106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.767+0000 7ffa81563640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101460 0x7ffa7c102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.767+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 0x7ffa7c105ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa81563640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10f5f0 con 0x7ffa7c100160 2024-05-09T09:17:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 0x7ffa7c105ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53830/0 (socket says 172.21.15.64:53830) 2024-05-09T09:17:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 -- 172.21.15.64:0/3964383860 learned_addr learned my addr 172.21.15.64:0/3964383860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10f470 con 0x7ffa7c100ac0 2024-05-09T09:17:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10f770 con 0x7ffa7c101460 2024-05-09T09:17:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa7c100ac0 0x7ffa7c106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101460 0x7ffa7c102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 -- 172.21.15.64:0/3964383860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101460 msgr2=0x7ffa7c102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101460 0x7ffa7c102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 -- 172.21.15.64:0/3964383860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa7c100ac0 msgr2=0x7ffa7c106220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa7c100ac0 0x7ffa7c106220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.768+0000 7ffa7affd640 1 -- 172.21.15.64:0/3964383860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c103110 con 0x7ffa7c100160 2024-05-09T09:17:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.769+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 0x7ffa7c105ce0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c035040 tx=0x7ffa6c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.770+0000 7ffa63fff640 1 -- 172.21.15.64:0/3964383860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c040020 con 0x7ffa7c100160 2024-05-09T09:17:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.770+0000 7ffa63fff640 1 -- 172.21.15.64:0/3964383860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6c038040 con 0x7ffa7c100160 2024-05-09T09:17:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.770+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c1a60b0 con 0x7ffa7c100160 2024-05-09T09:17:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.770+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1a6460 con 0x7ffa7c100160 2024-05-09T09:17:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.770+0000 7ffa63fff640 1 -- 172.21.15.64:0/3964383860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c037070 con 0x7ffa7c100160 2024-05-09T09:17:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.774+0000 7ffa8426a740 1 Processor -- start 2024-05-09T09:17:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.774+0000 7ffa63fff640 1 -- 172.21.15.64:0/3964383860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa6c043020 con 0x7ffa7c100160 2024-05-09T09:17:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.774+0000 7ffa8426a740 1 -- start start 2024-05-09T09:17:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aa3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aa8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa8426a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559d50a07340 con 0x559d50aa3c40 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa8426a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559d509af290 con 0x559d50aa8db0 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa8426a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559d50a03300 con 0x559d50aa3250 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa7b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aa3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa7b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aa3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38764/0 (socket says 172.21.15.64:38764) 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa7b7fe640 1 -- 172.21.15.64:0/1466642744 learned_addr learned my addr 172.21.15.64:0/1466642744 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.775+0000 7ffa7affd640 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aa8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.776+0000 7ffa7a7fc640 1 -- 172.21.15.64:0/1466642744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 msgr2=0x559d50aa3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.776+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aa3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.776+0000 7ffa7a7fc640 1 -- 172.21.15.64:0/1466642744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 msgr2=0x559d50aa8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.776+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aa8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.776+0000 7ffa7a7fc640 1 -- 172.21.15.64:0/1466642744 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d509d38e0 con 0x559d50aa8db0 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.776+0000 7ffa63fff640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa640a2e70 0x7ffa640a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.777+0000 7ffa63fff640 1 -- 172.21.15.64:0/3964383860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa6c0efba0 con 0x7ffa7c100160 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.777+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aab240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005fd0 tx=0x7ffa70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11aaafb2f33fb684 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.777+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa640a2e70 0x7ffa640a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.777+0000 7ffa617fa640 1 -- 172.21.15.64:0/1466642744 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7001e070 con 0x559d50aa8db0 2024-05-09T09:17:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.777+0000 7ffa617fa640 1 -- 172.21.15.64:0/1466642744 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa700035f0 con 0x559d50aa8db0 2024-05-09T09:17:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.777+0000 7ffa617fa640 1 -- 172.21.15.64:0/1466642744 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70019690 con 0x559d50aa8db0 2024-05-09T09:17:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 -- 172.21.15.64:0/1466642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 msgr2=0x559d50aab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aab240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005fd0 tx=0x7ffa70011810 comp rx=0 tx=0).stop 2024-05-09T09:17:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa640a2e70 0x7ffa640a5330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x559d50aa44a0 tx=0x7ffa74009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 -- 172.21.15.64:0/1466642744 shutdown_connections 2024-05-09T09:17:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aab240 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aa8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 --2- 172.21.15.64:0/1466642744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aa3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 -- 172.21.15.64:0/1466642744 >> 172.21.15.64:0/1466642744 conn(0x559d50a9e8c0 msgr2=0x559d50a9ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 -- 172.21.15.64:0/1466642744 shutdown_connections 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 -- 172.21.15.64:0/1466642744 wait complete. 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.779+0000 7ffa8426a740 1 -- start start 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 Processor -- start 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa3250 0x559d50aad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 0x559d50ab4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559d50aabd60 con 0x559d50aa3c40 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559d509c2790 con 0x559d50aa3250 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559d50aabee0 con 0x559d50aa8db0 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa8426a740 10 client.0 _ll_register_callbacks cb 0x559d509fda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa7b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 0x559d50ab4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa7b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 0x559d50ab4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38772/0 (socket says 172.21.15.64:38772) 2024-05-09T09:17:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa3250 0x559d50aad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.780+0000 7ffa7b7fe640 1 -- 172.21.15.64:0/3321412304 learned_addr learned my addr 172.21.15.64:0/3321412304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7b7fe640 1 -- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa3250 msgr2=0x559d50aad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa3250 0x559d50aad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7b7fe640 1 -- 172.21.15.64:0/3321412304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 msgr2=0x559d50aada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7b7fe640 1 -- 172.21.15.64:0/3321412304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5071b210 con 0x559d50aa8db0 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa3250 0x559d50aad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:17:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 0x559d50ab4a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400f700 tx=0x7ffa74004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5e7f68628724c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.781+0000 7ffa5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.782+0000 7ffa5eff5640 1 -- 172.21.15.64:0/3321412304 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74003970 con 0x559d50aa8db0 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.782+0000 7ffa5eff5640 1 -- 172.21.15.64:0/3321412304 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa74017030 con 0x559d50aa8db0 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.782+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa700035f0 con 0x559d50aa8db0 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.782+0000 7ffa5eff5640 1 -- 172.21.15.64:0/3321412304 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffa74032050 con 0x559d50aa8db0 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.782+0000 7ffa8426a740 10 client.27199 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.782+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559d507e7620 con 0x559d50aa8db0 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa5eff5640 1 -- 172.21.15.64:0/3321412304 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa700035f0 con 0x559d50aa8db0 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa5eff5640 1 -- 172.21.15.64:0/3321412304 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ffa74004240 con 0x559d50aa8db0 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa8426a740 10 client.27199 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa8426a740 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa8426a740 20 client.27199 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50aca560 0x559d50acc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa8426a740 4 client.27199 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.783+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d50a14a70 con 0x559d50aca560 2024-05-09T09:17:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.786+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50aca560 0x559d50acc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.789+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50aca560 0x559d50acc9e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bb246f69bdb0197 server_cookie=9be80e4b50672fca in_seq=0 out_seq=0 2024-05-09T09:17:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.789+0000 7ffa5eff5640 10 client.27199 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.789+0000 7ffa5eff5640 1 -- 172.21.15.64:0/3321412304 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d50a14a70 con 0x559d50aca560 2024-05-09T09:17:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.789+0000 7ffa5eff5640 10 client.27199 handle_command_reply: tid=0 2024-05-09T09:17:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.790+0000 7ffa8426a740 1 client.27199 shutdown 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 msgr2=0x559d50ab4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 0x559d50ab4a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400f700 tx=0x7ffa74004ab0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 shutdown_connections 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50aca560 0x559d50acc9e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa8db0 0x559d50ab4a40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50aada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3321412304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa3250 0x559d50aad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.793+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 >> 172.21.15.64:0/3321412304 conn(0x559d50a9e8c0 msgr2=0x559d50a9da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.794+0000 7ffa5eff5640 0 client.27199 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.794+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 shutdown_connections 2024-05-09T09:17:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.794+0000 7ffa8426a740 1 -- 172.21.15.64:0/3321412304 wait complete. 2024-05-09T09:17:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.795+0000 7ffa8426a740 20 client.27199 trim_cache size 0 max 16384 2024-05-09T09:17:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.826+0000 7ffa8426a740 1 Processor -- start 2024-05-09T09:17:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.826+0000 7ffa8426a740 1 -- start start 2024-05-09T09:17:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aaccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa8426a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa680124d0 con 0x559d50aa3c40 2024-05-09T09:17:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa8426a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6800fda0 con 0x559d50aa8db0 2024-05-09T09:17:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa8426a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6800dbf0 con 0x559d50aa3250 2024-05-09T09:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aac790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38774/0 (socket says 172.21.15.64:38774) 2024-05-09T09:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa7affd640 1 -- 172.21.15.64:0/3854323204 learned_addr learned my addr 172.21.15.64:0/3854323204 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.827+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aaccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7affd640 1 -- 172.21.15.64:0/3854323204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 msgr2=0x559d50aaccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aaccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7affd640 1 -- 172.21.15.64:0/3854323204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 msgr2=0x559d50ab4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7affd640 1 -- 172.21.15.64:0/3854323204 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d50aac0a0 con 0x559d50aa3250 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.828+0000 7ffa7affd640 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aac790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000c00 tx=0x7ffa6c0ed180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d42ad1f13dd2ae6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.829+0000 7ffa60ff9640 1 -- 172.21.15.64:0/3854323204 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0bbde0 con 0x559d50aa3250 2024-05-09T09:17:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.829+0000 7ffa60ff9640 1 -- 172.21.15.64:0/3854323204 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6c0bc710 con 0x559d50aa3250 2024-05-09T09:17:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.830+0000 7ffa8426a740 1 -- 172.21.15.64:0/3854323204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 msgr2=0x559d50aac790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.830+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aac790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000c00 tx=0x7ffa6c0ed180 comp rx=0 tx=0).stop 2024-05-09T09:17:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.834+0000 7ffa8426a740 1 -- 172.21.15.64:0/3854323204 shutdown_connections 2024-05-09T09:17:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.834+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50aaccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.834+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.834+0000 7ffa8426a740 1 --2- 172.21.15.64:0/3854323204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50aac790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.834+0000 7ffa8426a740 1 -- 172.21.15.64:0/3854323204 >> 172.21.15.64:0/3854323204 conn(0x559d50a9e8c0 msgr2=0x559d50aad4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.834+0000 7ffa8426a740 1 -- 172.21.15.64:0/3854323204 shutdown_connections 2024-05-09T09:17:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.838+0000 7ffa8426a740 1 -- 172.21.15.64:0/3854323204 wait complete. 2024-05-09T09:17:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.838+0000 7ffa8426a740 1 -- start start 2024-05-09T09:17:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.838+0000 7ffa8426a740 1 Processor -- start 2024-05-09T09:17:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.838+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50ab4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.838+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa8426a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50ab1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa8426a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6800e260 con 0x559d50aa3c40 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa8426a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6800f6d0 con 0x559d50aa8db0 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa8426a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6800e930 con 0x559d50aa3250 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa8426a740 10 client.0 _ll_register_callbacks cb 0x7ffa74009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50ab4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa7affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50ab4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38788/0 (socket says 172.21.15.64:38788) 2024-05-09T09:17:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.839+0000 7ffa7affd640 1 -- 172.21.15.64:0/2438246095 learned_addr learned my addr 172.21.15.64:0/2438246095 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7b7fe640 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50ab1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7affd640 1 -- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 msgr2=0x559d50ab1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7affd640 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50ab1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7affd640 1 -- 172.21.15.64:0/2438246095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 msgr2=0x559d50ab1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7affd640 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7affd640 1 -- 172.21.15.64:0/2438246095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d50b2ebd0 con 0x559d50aa3250 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa7affd640 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50ab4a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009980 tx=0x7ffa6c0095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e2ae547abd04b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.840+0000 7ffa5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.841+0000 7ffa5eff5640 1 -- 172.21.15.64:0/2438246095 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0bbb50 con 0x559d50aa3250 2024-05-09T09:17:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.841+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c0bc710 con 0x559d50aa3250 2024-05-09T09:17:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.841+0000 7ffa5eff5640 1 -- 172.21.15.64:0/2438246095 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffa6c003d50 con 0x559d50aa3250 2024-05-09T09:17:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.842+0000 7ffa5eff5640 1 -- 172.21.15.64:0/2438246095 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c0bc010 con 0x559d50aa3250 2024-05-09T09:17:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.842+0000 7ffa8426a740 10 client.27205 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.842+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559d50b2ff50 con 0x559d50aa3250 2024-05-09T09:17:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa5eff5640 1 -- 172.21.15.64:0/2438246095 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6c0bc710 con 0x559d50aa3250 2024-05-09T09:17:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa5eff5640 1 -- 172.21.15.64:0/2438246095 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ffa6c0c48e0 con 0x559d50aa3250 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa8426a740 10 client.27205 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa8426a740 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa8426a740 20 client.27205 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa8426a740 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50b30940 0x559d50b32d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa8426a740 4 client.27205 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.843+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d50a0edc0 con 0x559d50b30940 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.844+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50b30940 0x559d50b32d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.844+0000 7ffa7a7fc640 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50b30940 0x559d50b32d60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0b67dac7728ab4 server_cookie=6197f7fc79d590d7 in_seq=0 out_seq=0 2024-05-09T09:17:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.844+0000 7ffa5eff5640 10 client.27205 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.874+0000 7ffa5eff5640 1 -- 172.21.15.64:0/2438246095 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d50a0edc0 con 0x559d50b30940 2024-05-09T09:17:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.874+0000 7ffa5eff5640 10 client.27205 handle_command_reply: tid=0 2024-05-09T09:17:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.874+0000 7ffa8426a740 1 client.27205 shutdown 2024-05-09T09:17:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.879+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 msgr2=0x559d50ab4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.879+0000 7ffa8426a740 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50ab4a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009980 tx=0x7ffa6c0095b0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 shutdown_connections 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559d50b30940 0x559d50b32d60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559d50aa8db0 0x559d50ab1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d50aa3c40 0x559d50ab1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 --2- 172.21.15.64:0/2438246095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d50aa3250 0x559d50ab4a20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 >> 172.21.15.64:0/2438246095 conn(0x559d50a9e8c0 msgr2=0x559d50aac790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa5eff5640 0 client.27205 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 shutdown_connections 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.881+0000 7ffa8426a740 1 -- 172.21.15.64:0/2438246095 wait complete. 2024-05-09T09:17:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.882+0000 7ffa8426a740 20 client.27205 trim_cache size 0 max 16384 2024-05-09T09:17:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:17:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:17:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:17:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:17:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa640a2e70 msgr2=0x7ffa640a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa640a2e70 0x7ffa640a5330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x559d50aa44a0 tx=0x7ffa74009500 comp rx=0 tx=0).stop 2024-05-09T09:17:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 msgr2=0x7ffa7c105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 0x7ffa7c105ce0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c035040 tx=0x7ffa6c030d80 comp rx=0 tx=0).stop 2024-05-09T09:17:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 shutdown_connections 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffa640a2e70 0x7ffa640a5330 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101460 0x7ffa7c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa7c100ac0 0x7ffa7c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.883+0000 7ffa81563640 1 --2- 172.21.15.64:0/3964383860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7c100160 0x7ffa7c105ce0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.884+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 >> 172.21.15.64:0/3964383860 conn(0x7ffa7c0fbd00 msgr2=0x7ffa7c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.884+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 shutdown_connections 2024-05-09T09:17:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:24.884+0000 7ffa81563640 1 -- 172.21.15.64:0/3964383860 wait complete. 2024-05-09T09:17:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:24 smithi139 ceph-mon[33599]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-05-09T09:17:25.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:24 smithi064 ceph-mon[30149]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-05-09T09:17:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:24 smithi179 ceph-mon[34086]: pgmap v618: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-05-09T09:17:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 425.0 s, 779.0 tps, lat 128.261 ms stddev 53.782 2024-05-09T09:17:27.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:26 smithi139 ceph-mon[33599]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.6 MiB/s wr, 978 op/s 2024-05-09T09:17:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:26 smithi064 ceph-mon[30149]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.6 MiB/s wr, 978 op/s 2024-05-09T09:17:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:26 smithi179 ceph-mon[34086]: pgmap v619: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.6 MiB/s wr, 978 op/s 2024-05-09T09:17:29.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:28 smithi139 ceph-mon[33599]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.0 MiB/s wr, 873 op/s 2024-05-09T09:17:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:28 smithi064 ceph-mon[30149]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.0 MiB/s wr, 873 op/s 2024-05-09T09:17:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:28 smithi179 ceph-mon[34086]: pgmap v620: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.0 MiB/s wr, 873 op/s 2024-05-09T09:17:30.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:17:30.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:17:30.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 -- 172.21.15.64:0/2895135311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 msgr2=0x7f47a41144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 --2- 172.21.15.64:0/2895135311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 0x7f47a41144c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f479c009cd0 tx=0x7f479c02a8f0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 -- 172.21.15.64:0/2895135311 shutdown_connections 2024-05-09T09:17:30.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 --2- 172.21.15.64:0/2895135311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 0x7f47a41144c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 --2- 172.21.15.64:0/2895135311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47a410b1a0 0x7f47a4111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 --2- 172.21.15.64:0/2895135311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a4108830 0x7f47a410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.251+0000 7f47aacae640 1 -- 172.21.15.64:0/2895135311 >> 172.21.15.64:0/2895135311 conn(0x7f47a406e6c0 msgr2=0x7f47a4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.254+0000 7f47aacae640 1 -- 172.21.15.64:0/2895135311 shutdown_connections 2024-05-09T09:17:30.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.255+0000 7f47aacae640 1 -- 172.21.15.64:0/2895135311 wait complete. 2024-05-09T09:17:30.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47aacae640 1 Processor -- start 2024-05-09T09:17:30.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47aacae640 1 -- start start 2024-05-09T09:17:30.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47aacae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47a4108830 0x7f47a41a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47aacae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 0x7f47a41a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47aacae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 0x7f47a41a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47aacae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4117e80 con 0x7f47a4108830 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 0x7f47a41a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 0x7f47a41a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38810/0 (socket says 172.21.15.64:38810) 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.256+0000 7f47a3fff640 1 -- 172.21.15.64:0/953378462 learned_addr learned my addr 172.21.15.64:0/953378462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4117d00 con 0x7f47a410b620 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4118000 con 0x7f47a410b1a0 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47a924b640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 0x7f47a41a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47a3fff640 1 -- 172.21.15.64:0/953378462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 msgr2=0x7f47a41a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47a3fff640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 0x7f47a41a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47a3fff640 1 -- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47a4108830 msgr2=0x7f47a41a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47a3fff640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47a4108830 0x7f47a41a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.257+0000 7f47a3fff640 1 -- 172.21.15.64:0/953378462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a41aa190 con 0x7f47a410b1a0 2024-05-09T09:17:30.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.258+0000 7f47a3fff640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 0x7f47a41a2ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f479800b450 tx=0x7f479800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.258+0000 7f47a1ffb640 1 -- 172.21.15.64:0/953378462 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798015020 con 0x7f47a410b1a0 2024-05-09T09:17:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.258+0000 7f47a1ffb640 1 -- 172.21.15.64:0/953378462 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4798011d40 con 0x7f47a410b1a0 2024-05-09T09:17:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.258+0000 7f47a1ffb640 1 -- 172.21.15.64:0/953378462 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479800f340 con 0x7f47a410b1a0 2024-05-09T09:17:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.258+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41aa480 con 0x7f47a410b1a0 2024-05-09T09:17:30.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.259+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a410bdb0 con 0x7f47a410b1a0 2024-05-09T09:17:30.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.264+0000 7f47a1ffb640 1 -- 172.21.15.64:0/953378462 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4798018020 con 0x7f47a410b1a0 2024-05-09T09:17:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.265+0000 7f47a1ffb640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f478c0a2c10 0x7f478c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.266+0000 7f47a1ffb640 1 -- 172.21.15.64:0/953378462 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f47980ceef0 con 0x7f47a410b1a0 2024-05-09T09:17:30.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.268+0000 7f47ad9b7740 1 Processor -- start 2024-05-09T09:17:30.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.268+0000 7f47ad9b7740 1 -- start start 2024-05-09T09:17:30.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae291f30 0x55acae292350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae297420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 0x55acae299f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55acae1f6d00 con 0x55acae291f30 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55acae19e4c0 con 0x55acae292920 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55acae1f2cc0 con 0x55acae297a90 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f478c0a2c10 0x7f478c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47a924b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 0x55acae299f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47a924b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 0x55acae299f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38814/0 (socket says 172.21.15.64:38814) 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47a924b640 1 -- 172.21.15.64:0/3607392078 learned_addr learned my addr 172.21.15.64:0/3607392078 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47a3fff640 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae297420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.269+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae291f30 0x55acae292350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a924b640 1 -- 172.21.15.64:0/3607392078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 msgr2=0x55acae297420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a924b640 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae297420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a924b640 1 -- 172.21.15.64:0/3607392078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae291f30 msgr2=0x55acae292350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a924b640 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae291f30 0x55acae292350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a924b640 1 -- 172.21.15.64:0/3607392078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acadfd47d0 con 0x55acae297a90 2024-05-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae291f30 0x55acae292350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a924b640 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 0x55acae299f20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f479c009cd0 tx=0x7f479c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5560d453baa5ba75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.270+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f478c0a2c10 0x7f478c0a50d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55acae293180 tx=0x7f4794005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.271+0000 7f478affd640 1 -- 172.21.15.64:0/3607392078 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c030960 con 0x55acae297a90 2024-05-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.271+0000 7f478affd640 1 -- 172.21.15.64:0/3607392078 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c036030 con 0x55acae297a90 2024-05-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.271+0000 7f478affd640 1 -- 172.21.15.64:0/3607392078 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c004510 con 0x55acae297a90 2024-05-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 -- 172.21.15.64:0/3607392078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 msgr2=0x55acae299f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 0x55acae299f20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f479c009cd0 tx=0x7f479c009a20 comp rx=0 tx=0).stop 2024-05-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 -- 172.21.15.64:0/3607392078 shutdown_connections 2024-05-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae297a90 0x55acae299f20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae297420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/3607392078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae291f30 0x55acae292350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.272+0000 7f47ad9b7740 1 -- 172.21.15.64:0/3607392078 >> 172.21.15.64:0/3607392078 conn(0x55acae28d5a0 msgr2=0x55acae28d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.273+0000 7f47ad9b7740 1 -- 172.21.15.64:0/3607392078 shutdown_connections 2024-05-09T09:17:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.273+0000 7f47ad9b7740 1 -- 172.21.15.64:0/3607392078 wait complete. 2024-05-09T09:17:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.273+0000 7f47ad9b7740 1 -- start start 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.273+0000 7f47ad9b7740 1 Processor -- start 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.274+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae29c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.274+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.274+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae2a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55acae29a9e0 con 0x55acae297a90 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55acae1b1b90 con 0x55acae292920 2024-05-09T09:17:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55acae29ab60 con 0x55acae291f30 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47ad9b7740 10 client.0 _ll_register_callbacks cb 0x55acae1edda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47a924b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae2a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47a924b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae2a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53910/0 (socket says 172.21.15.64:53910) 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.275+0000 7f47a924b640 1 -- 172.21.15.64:0/1995514411 learned_addr learned my addr 172.21.15.64:0/1995514411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a3fff640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a924b640 1 -- 172.21.15.64:0/1995514411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 msgr2=0x55acae29c360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a924b640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae29c360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a924b640 1 -- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 msgr2=0x55acae29c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a924b640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a924b640 1 -- 172.21.15.64:0/1995514411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acadf0a3f0 con 0x55acae297a90 2024-05-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a3fff640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.276+0000 7f47a924b640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae2a38e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ca0 tx=0x7f479c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6d87dce4332189 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.277+0000 7f4783fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.277+0000 7f4783fff640 1 -- 172.21.15.64:0/1995514411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c052020 con 0x55acae297a90 2024-05-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.277+0000 7f4783fff640 1 -- 172.21.15.64:0/1995514411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f479c0042d0 con 0x55acae297a90 2024-05-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.277+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479c036030 con 0x55acae297a90 2024-05-09T09:17:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.279+0000 7f4783fff640 1 -- 172.21.15.64:0/1995514411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f479c004510 con 0x55acae297a90 2024-05-09T09:17:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.279+0000 7f47ad9b7740 10 client.17574 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.279+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55acadeca0f0 con 0x55acae297a90 2024-05-09T09:17:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.280+0000 7f4783fff640 1 -- 172.21.15.64:0/1995514411 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c036030 con 0x55acae297a90 2024-05-09T09:17:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f4783fff640 1 -- 172.21.15.64:0/1995514411 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f479c043070 con 0x55acae297a90 2024-05-09T09:17:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f47ad9b7740 10 client.17574 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f47ad9b7740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f47ad9b7740 20 client.17574 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae2aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f47ad9b7740 4 client.17574 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.281+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acae204430 con 0x55acae2a7dd0 2024-05-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.282+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae2aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.283+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae2aa1c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756501f21f33269d server_cookie=85779030c35ad3f7 in_seq=0 out_seq=0 2024-05-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.283+0000 7f4783fff640 10 client.17574 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.284+0000 7f4783fff640 1 -- 172.21.15.64:0/1995514411 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acae204430 con 0x55acae2a7dd0 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.284+0000 7f4783fff640 10 client.17574 handle_command_reply: tid=0 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.284+0000 7f47ad9b7740 1 client.17574 shutdown 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.284+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 msgr2=0x55acae2a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.284+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae2a38e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ca0 tx=0x7f479c009800 comp rx=0 tx=0).stop 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 shutdown_connections 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae2aa1c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae2a38e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f4783fff640 0 client.17574 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1995514411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae29c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 >> 172.21.15.64:0/1995514411 conn(0x55acae28d5a0 msgr2=0x55acae28c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 shutdown_connections 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1995514411 wait complete. 2024-05-09T09:17:30.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.285+0000 7f47ad9b7740 20 client.17574 trim_cache size 0 max 16384 2024-05-09T09:17:30.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.320+0000 7f47ad9b7740 1 Processor -- start 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 -- start start 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae290340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae2a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae290880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41bbb00 con 0x55acae297a90 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b93d0 con 0x55acae292920 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b7240 con 0x55acae291f30 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47a924b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae290880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47a924b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae290880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53914/0 (socket says 172.21.15.64:53914) 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.321+0000 7f47a924b640 1 -- 172.21.15.64:0/1959436539 learned_addr learned my addr 172.21.15.64:0/1959436539 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.323+0000 7f47a3fff640 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae2a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.323+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae290340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.324+0000 7f47a3fff640 1 -- 172.21.15.64:0/1959436539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 msgr2=0x55acae290340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.324+0000 7f47a3fff640 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae290340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.324+0000 7f47a3fff640 1 -- 172.21.15.64:0/1959436539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 msgr2=0x55acae290880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.324+0000 7f47a3fff640 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae290880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.324+0000 7f47a3fff640 1 -- 172.21.15.64:0/1959436539 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acae2aa7a0 con 0x55acae292920 2024-05-09T09:17:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.325+0000 7f47a3fff640 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae2a38a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f479800be10 tx=0x7f47980ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e873717e46d798 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.325+0000 7f478a7fc640 1 -- 172.21.15.64:0/1959436539 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980cc710 con 0x55acae292920 2024-05-09T09:17:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.325+0000 7f478a7fc640 1 -- 172.21.15.64:0/1959436539 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47980b4240 con 0x55acae292920 2024-05-09T09:17:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.325+0000 7f478a7fc640 1 -- 172.21.15.64:0/1959436539 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479809aae0 con 0x55acae292920 2024-05-09T09:17:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1959436539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 msgr2=0x55acae2a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae2a38a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f479800be10 tx=0x7f47980ce040 comp rx=0 tx=0).stop 2024-05-09T09:17:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1959436539 shutdown_connections 2024-05-09T09:17:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae290880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae2a38a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1959436539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae290340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.326+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1959436539 >> 172.21.15.64:0/1959436539 conn(0x55acae28d5a0 msgr2=0x55acae29c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.327+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1959436539 shutdown_connections 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.327+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1959436539 wait complete. 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.327+0000 7f47ad9b7740 1 -- start start 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.327+0000 7f47ad9b7740 1 Processor -- start 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae29e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae31d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b7890 con 0x55acae297a90 2024-05-09T09:17:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b8d00 con 0x55acae292920 2024-05-09T09:17:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b7f60 con 0x55acae291f30 2024-05-09T09:17:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47ad9b7740 10 client.0 _ll_register_callbacks cb 0x7f4798097600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47a924b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae31d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47a924b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae31d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53926/0 (socket says 172.21.15.64:53926) 2024-05-09T09:17:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47a924b640 1 -- 172.21.15.64:0/1178853854 learned_addr learned my addr 172.21.15.64:0/1178853854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47a924b640 1 -- 172.21.15.64:0/1178853854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 msgr2=0x55acae29e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.328+0000 7f47a924b640 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae29e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.329+0000 7f47a924b640 1 -- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 msgr2=0x55acae29ed30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.329+0000 7f47a924b640 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29ed30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.329+0000 7f47a924b640 1 -- 172.21.15.64:0/1178853854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acae31d7f0 con 0x55acae297a90 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.329+0000 7f47a924b640 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae31d2b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f479c006010 tx=0x7f479c031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59d52f0a0efae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.329+0000 7f4783fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:17:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.330+0000 7f4783fff640 1 -- 172.21.15.64:0/1178853854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c052070 con 0x55acae297a90 2024-05-09T09:17:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.330+0000 7f4783fff640 1 -- 172.21.15.64:0/1178853854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f479c004d50 con 0x55acae297a90 2024-05-09T09:17:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.330+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47980b4240 con 0x55acae297a90 2024-05-09T09:17:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.330+0000 7f4783fff640 1 -- 172.21.15.64:0/1178853854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f479c003970 con 0x55acae297a90 2024-05-09T09:17:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.330+0000 7f47ad9b7740 10 client.17580 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.330+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55acae31ea80 con 0x55acae297a90 2024-05-09T09:17:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.331+0000 7f4783fff640 1 -- 172.21.15.64:0/1178853854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47980b4240 con 0x55acae297a90 2024-05-09T09:17:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.331+0000 7f4783fff640 1 -- 172.21.15.64:0/1178853854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f479c043070 con 0x55acae297a90 2024-05-09T09:17:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47ad9b7740 10 client.17580 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47ad9b7740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47ad9b7740 20 client.17580 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae3213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47ad9b7740 4 client.17580 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:17:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acae29c150 con 0x55acae2a7dd0 2024-05-09T09:17:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.332+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae3213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.335+0000 7f47a8a4a640 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae3213a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e031dd0452abd65 server_cookie=71415d63afb0a5ad in_seq=0 out_seq=0 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.335+0000 7f4783fff640 10 client.17580 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.336+0000 7f4783fff640 1 -- 172.21.15.64:0/1178853854 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55acae29c150 con 0x55acae2a7dd0 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f4783fff640 10 client.17580 handle_command_reply: tid=0 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 client.17580 shutdown 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 msgr2=0x55acae31d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae31d2b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f479c006010 tx=0x7f479c031cb0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 shutdown_connections 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55acae2a7dd0 0x55acae3213a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55acae297a90 0x55acae31d2b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55acae292920 0x55acae29ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 --2- 172.21.15.64:0/1178853854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55acae291f30 0x55acae29e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 >> 172.21.15.64:0/1178853854 conn(0x55acae28d5a0 msgr2=0x55acae29c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.337+0000 7f4783fff640 0 client.17580 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.338+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 shutdown_connections 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.338+0000 7f47ad9b7740 1 -- 172.21.15.64:0/1178853854 wait complete. 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.338+0000 7f47ad9b7740 20 client.17580 trim_cache size 0 max 16384 2024-05-09T09:17:30.340 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:17:30.341 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:17:30.341 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:17:30.341 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:17:30.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.339+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f478c0a2c10 msgr2=0x7f478c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.339+0000 7f47aacae640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f478c0a2c10 0x7f478c0a50d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55acae293180 tx=0x7f4794005850 comp rx=0 tx=0).stop 2024-05-09T09:17:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.339+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 msgr2=0x7f47a41a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.339+0000 7f47aacae640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 0x7f47a41a2ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f479800b450 tx=0x7f479800b910 comp rx=0 tx=0).stop 2024-05-09T09:17:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 shutdown_connections 2024-05-09T09:17:30.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f478c0a2c10 0x7f478c0a50d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47a410b620 0x7f47a41a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47a410b1a0 0x7f47a41a2ab0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 --2- 172.21.15.64:0/953378462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47a4108830 0x7f47a41a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 >> 172.21.15.64:0/953378462 conn(0x7f47a406e6c0 msgr2=0x7f47a4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.340+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 shutdown_connections 2024-05-09T09:17:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.341+0000 7f47aacae640 1 -- 172.21.15.64:0/953378462 wait complete. 2024-05-09T09:17:30.361 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:17:30.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:17:30.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.569+0000 7fed8ca10640 1 -- 172.21.15.64:0/4280258487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed881099a0 msgr2=0x7fed881102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.569+0000 7fed8ca10640 1 --2- 172.21.15.64:0/4280258487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed881099a0 0x7fed881102c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fed8001acb0 tx=0x7fed8003b920 comp rx=0 tx=0).stop 2024-05-09T09:17:30.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.570+0000 7fed8ca10640 1 -- 172.21.15.64:0/4280258487 shutdown_connections 2024-05-09T09:17:30.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.570+0000 7fed8ca10640 1 --2- 172.21.15.64:0/4280258487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed881099a0 0x7fed881102c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.570+0000 7fed8ca10640 1 --2- 172.21.15.64:0/4280258487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88108fe0 0x7fed88109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.570+0000 7fed8ca10640 1 --2- 172.21.15.64:0/4280258487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88107e90 0x7fed881082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.570+0000 7fed8ca10640 1 -- 172.21.15.64:0/4280258487 >> 172.21.15.64:0/4280258487 conn(0x7fed88076e40 msgr2=0x7fed88079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:30.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.571+0000 7fed8ca10640 1 -- 172.21.15.64:0/4280258487 shutdown_connections 2024-05-09T09:17:30.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.573+0000 7fed8ca10640 1 -- 172.21.15.64:0/4280258487 wait complete. 2024-05-09T09:17:30.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.574+0000 7fed8ca10640 1 Processor -- start 2024-05-09T09:17:30.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.575+0000 7fed8ca10640 1 -- start start 2024-05-09T09:17:30.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.575+0000 7fed8ca10640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 0x7fed88195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8ca10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88108fe0 0x7fed881963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 0x7fed88195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8ca10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed881099a0 0x7fed8819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 0x7fed88195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38854/0 (socket says 172.21.15.64:38854) 2024-05-09T09:17:30.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8ca10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed8806de10 con 0x7fed88108fe0 2024-05-09T09:17:30.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8ca10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed8806dc90 con 0x7fed881099a0 2024-05-09T09:17:30.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8ca10640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed8806df90 con 0x7fed88107e90 2024-05-09T09:17:30.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed8659c640 1 -- 172.21.15.64:0/268949031 learned_addr learned my addr 172.21.15.64:0/268949031 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.576+0000 7fed86d9d640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed881099a0 0x7fed8819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.577+0000 7fed8659c640 1 -- 172.21.15.64:0/268949031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed881099a0 msgr2=0x7fed8819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.577+0000 7fed8659c640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed881099a0 0x7fed8819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.577+0000 7fed8659c640 1 -- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88108fe0 msgr2=0x7fed881963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.577+0000 7fed8659c640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88108fe0 0x7fed881963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.577+0000 7fed8659c640 1 -- 172.21.15.64:0/268949031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8819d5c0 con 0x7fed88107e90 2024-05-09T09:17:30.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.577+0000 7fed8659c640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 0x7fed88195e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b450 tx=0x7fed7800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:30.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.578+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78015020 con 0x7fed88107e90 2024-05-09T09:17:30.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.578+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed78011d40 con 0x7fed88107e90 2024-05-09T09:17:30.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.579+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7800f340 con 0x7fed88107e90 2024-05-09T09:17:30.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.579+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8819d8b0 con 0x7fed88107e90 2024-05-09T09:17:30.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.579+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8819de00 con 0x7fed88107e90 2024-05-09T09:17:30.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.582+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed88067dc0 con 0x7fed88107e90 2024-05-09T09:17:30.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.583+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed78018020 con 0x7fed88107e90 2024-05-09T09:17:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.584+0000 7fed777fe640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed700a2c80 0x7fed700a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.585+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fed780ced10 con 0x7fed88107e90 2024-05-09T09:17:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.585+0000 7fed85d9b640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed700a2c80 0x7fed700a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:30.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.589+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed7800d040 con 0x7fed88107e90 2024-05-09T09:17:30.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:30.590+0000 7fed85d9b640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed700a2c80 0x7fed700a5140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fed88196eb0 tx=0x7fed7c006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.040+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed88062430 con 0x7fed88107e90 2024-05-09T09:17:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.041+0000 7fed777fe640 1 -- 172.21.15.64:0/268949031 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11879 (secure 0 0 0) 0x7fed7809b880 con 0x7fed88107e90 2024-05-09T09:17:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:17:31.041 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:17:16.247105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:17:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 45 2024-05-09T09:17:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.045+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed700a2c80 msgr2=0x7fed700a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.046+0000 7fed8ca10640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed700a2c80 0x7fed700a5140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fed88196eb0 tx=0x7fed7c006c40 comp rx=0 tx=0).stop 2024-05-09T09:17:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.046+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 msgr2=0x7fed88195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.046+0000 7fed8ca10640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 0x7fed88195e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b450 tx=0x7fed7800b910 comp rx=0 tx=0).stop 2024-05-09T09:17:31.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.047+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 shutdown_connections 2024-05-09T09:17:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.047+0000 7fed8ca10640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fed700a2c80 0x7fed700a5140 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.047+0000 7fed8ca10640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed881099a0 0x7fed8819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.047+0000 7fed8ca10640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed88108fe0 0x7fed881963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.047+0000 7fed8ca10640 1 --2- 172.21.15.64:0/268949031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed88107e90 0x7fed88195e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.047+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 >> 172.21.15.64:0/268949031 conn(0x7fed88076e40 msgr2=0x7fed880708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.050+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 shutdown_connections 2024-05-09T09:17:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.050+0000 7fed8ca10640 1 -- 172.21.15.64:0/268949031 wait complete. 2024-05-09T09:17:31.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:30 smithi064 ceph-mon[30149]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:17:31.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:17:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:30 smithi139 ceph-mon[33599]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:17:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:30 smithi179 ceph-mon[34086]: pgmap v621: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-05-09T09:17:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.272+0000 7f6ca0cff640 1 -- 172.21.15.64:0/1307355376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10b340 msgr2=0x7f6c9c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.272+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/1307355376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10b340 0x7f6c9c10d750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bd70 tx=0x7f6c8c02e680 comp rx=0 tx=0).stop 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.273+0000 7f6ca0cff640 1 -- 172.21.15.64:0/1307355376 shutdown_connections 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.273+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/1307355376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.273+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/1307355376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10b340 0x7f6c9c10d750 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bd70 tx=0x7f6c8c02e680 comp rx=0 tx=0).stop 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.273+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/1307355376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c9c108a10 0x7f6c9c10ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.273+0000 7f6ca0cff640 1 -- 172.21.15.64:0/1307355376 >> 172.21.15.64:0/1307355376 conn(0x7f6c9c06e740 msgr2=0x7f6c9c070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.275+0000 7f6ca0cff640 1 -- 172.21.15.64:0/1307355376 shutdown_connections 2024-05-09T09:17:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.276+0000 7f6ca0cff640 1 -- 172.21.15.64:0/1307355376 wait complete. 2024-05-09T09:17:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.277+0000 7f6ca0cff640 1 Processor -- start 2024-05-09T09:17:31.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.277+0000 7f6ca0cff640 1 -- start start 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.277+0000 7f6ca0cff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c108a10 0x7f6c9c1a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.277+0000 7f6ca0cff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6ca0cff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c9c1abf50 0x7f6c9c1ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6ca0cff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c11c0d0 con 0x7f6c9c108a10 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6ca0cff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c11bf50 con 0x7f6c9c10dde0 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6ca0cff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c11c250 con 0x7f6c9c1abf50 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6c9affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6c9affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1a6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42564/0 (socket says 172.21.15.64:42564) 2024-05-09T09:17:31.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6c9affd640 1 -- 172.21.15.64:0/2899711712 learned_addr learned my addr 172.21.15.64:0/2899711712 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6c9affd640 1 -- 172.21.15.64:0/2899711712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c9c1abf50 msgr2=0x7f6c9c1ae340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:17:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.278+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c9c1abf50 0x7f6c9c1ae340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c9affd640 1 -- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c108a10 msgr2=0x7f6c9c1a69b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:17:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c108a10 0x7f6c9c1a69b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c9affd640 1 -- 172.21.15.64:0/2899711712 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c1ae9b0 con 0x7f6c9c10dde0 2024-05-09T09:17:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1a6ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c02ebf0 tx=0x7f6c8c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c98ff9640 1 -- 172.21.15.64:0/2899711712 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c03a030 con 0x7f6c9c10dde0 2024-05-09T09:17:31.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c98ff9640 1 -- 172.21.15.64:0/2899711712 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c8c03c070 con 0x7f6c9c10dde0 2024-05-09T09:17:31.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.279+0000 7f6c98ff9640 1 -- 172.21.15.64:0/2899711712 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c009840 con 0x7f6c9c10dde0 2024-05-09T09:17:31.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.280+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9c1aec40 con 0x7f6c9c10dde0 2024-05-09T09:17:31.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.280+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9c1af050 con 0x7f6c9c10dde0 2024-05-09T09:17:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.283+0000 7f6ca4c6b740 1 Processor -- start 2024-05-09T09:17:31.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 -- start start 2024-05-09T09:17:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b9c90 0x558b576bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558b57618c40 con 0x558b576b9c90 2024-05-09T09:17:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558b575c0580 con 0x558b576b4130 2024-05-09T09:17:31.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b57614c00 con 0x558b576b4b20 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42572/0 (socket says 172.21.15.64:42572) 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6c9bfff640 1 -- 172.21.15.64:0/846876889 learned_addr learned my addr 172.21.15.64:0/846876889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.284+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c98ff9640 1 -- 172.21.15.64:0/2899711712 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c8c047060 con 0x7f6c9c10dde0 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/846876889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 msgr2=0x558b576b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/846876889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b9c90 msgr2=0x558b576bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b9c90 0x558b576bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/846876889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b5732c3f0 con 0x558b576b4b20 2024-05-09T09:17:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.285+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576b9620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000a5f0 tx=0x7f6c9000ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e13f84309d0beee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.286+0000 7f6c81ffb640 1 -- 172.21.15.64:0/846876889 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90017030 con 0x558b576b4b20 2024-05-09T09:17:31.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.286+0000 7f6c81ffb640 1 -- 172.21.15.64:0/846876889 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c900054d0 con 0x558b576b4b20 2024-05-09T09:17:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.286+0000 7f6c98ff9640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c840a2d90 0x7f6c840a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.286+0000 7f6c98ff9640 1 -- 172.21.15.64:0/2899711712 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c8c0ee640 con 0x7f6c9c10dde0 2024-05-09T09:17:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.287+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c840a2d90 0x7f6c840a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.287+0000 7f6c81ffb640 1 -- 172.21.15.64:0/846876889 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90004cf0 con 0x558b576b4b20 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/846876889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 msgr2=0x558b576b9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576b9620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000a5f0 tx=0x7f6c9000ac00 comp rx=0 tx=0).stop 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c840a2d90 0x7f6c840a5250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x558b576b5380 tx=0x7f6c9400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/846876889 shutdown_connections 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b9c90 0x558b576bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576b9620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/846876889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/846876889 >> 172.21.15.64:0/846876889 conn(0x558b576af7a0 msgr2=0x558b576afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/846876889 shutdown_connections 2024-05-09T09:17:31.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.288+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/846876889 wait complete. 2024-05-09T09:17:31.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 -- start start 2024-05-09T09:17:31.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 Processor -- start 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576be560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4b20 0x558b576beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558b576bcc90 con 0x558b576b4b20 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558b575d3c40 con 0x558b576b4130 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b576bce10 con 0x558b576b9c90 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.289+0000 7f6ca4c6b740 10 client.0 _ll_register_callbacks cb 0x558b5732bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576be560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42588/0 (socket says 172.21.15.64:42588) 2024-05-09T09:17:31.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/2690947147 learned_addr learned my addr 172.21.15.64:0/2690947147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4b20 0x558b576beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 msgr2=0x558b576c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/2690947147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4b20 msgr2=0x558b576beaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4b20 0x558b576beaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/2690947147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b573f67d0 con 0x558b576b4130 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4b20 0x558b576beaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576be560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006010 tx=0x7f6c90005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a6959154b480b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c7affd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.290+0000 7f6c7affd640 1 -- 172.21.15.64:0/2690947147 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90005d10 con 0x558b576b4130 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.291+0000 7f6c7affd640 1 -- 172.21.15.64:0/2690947147 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c90004020 con 0x558b576b4130 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.291+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c900054d0 con 0x558b576b4130 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.291+0000 7f6c7affd640 1 -- 172.21.15.64:0/2690947147 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6c9002e050 con 0x558b576b4130 2024-05-09T09:17:31.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.291+0000 7f6ca4c6b740 10 client.27071 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:31.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.291+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558b576c8e50 con 0x558b576b4130 2024-05-09T09:17:31.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6c7affd640 1 -- 172.21.15.64:0/2690947147 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c900054d0 con 0x558b576b4130 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6c7affd640 1 -- 172.21.15.64:0/2690947147 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6c9001c450 con 0x558b576b4130 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6ca4c6b740 10 client.27071 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6ca4c6b740 10 client.27071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6ca4c6b740 20 client.27071 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b576ca1f0 0x558b576cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6ca4c6b740 4 client.27071 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b57626370 con 0x558b576ca1f0 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.292+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b576ca1f0 0x558b576cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.293+0000 7f6c7affd640 10 client.27071 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.293+0000 7f6c9affd640 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b576ca1f0 0x558b576cc5e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6eb04748b3706e3d server_cookie=e71568ed40e55aad in_seq=0 out_seq=0 2024-05-09T09:17:31.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.294+0000 7f6c7affd640 1 -- 172.21.15.64:0/2690947147 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b57626370 con 0x558b576ca1f0 2024-05-09T09:17:31.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.294+0000 7f6c7affd640 10 client.27071 handle_command_reply: tid=0 2024-05-09T09:17:31.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.294+0000 7f6ca4c6b740 1 client.27071 shutdown 2024-05-09T09:17:31.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 msgr2=0x558b576be560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576be560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006010 tx=0x7f6c90005250 comp rx=0 tx=0).stop 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 shutdown_connections 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b576ca1f0 0x558b576cc5e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4b20 0x558b576beaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/2690947147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4130 0x558b576be560 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 >> 172.21.15.64:0/2690947147 conn(0x558b576af7a0 msgr2=0x558b576ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6c7affd640 0 client.27071 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 shutdown_connections 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.295+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/2690947147 wait complete. 2024-05-09T09:17:31.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.296+0000 7f6ca4c6b740 20 client.27071 trim_cache size 0 max 16384 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 Processor -- start 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 -- start start 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4b20 0x558b576c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1bbe70 con 0x558b576b4130 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b9740 con 0x558b576b4b20 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.326+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b7590 con 0x558b576b9c90 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53970/0 (socket says 172.21.15.64:53970) 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/1155002331 learned_addr learned my addr 172.21.15.64:0/1155002331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9affd640 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4b20 0x558b576c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 -- 172.21.15.64:0/1155002331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4b20 msgr2=0x558b576c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4b20 0x558b576c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 -- 172.21.15.64:0/1155002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 msgr2=0x558b576b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 -- 172.21.15.64:0/1155002331 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b576af2e0 con 0x558b576b9c90 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9affd640 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4b20 0x558b576c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:31.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.327+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576bdb80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400f790 tx=0x7f6c94012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fdbc4eb0b4c8571 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.328+0000 7f6c817fa640 1 -- 172.21.15.64:0/1155002331 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94016070 con 0x558b576b9c90 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.328+0000 7f6c817fa640 1 -- 172.21.15.64:0/1155002331 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c9401ee50 con 0x558b576b9c90 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.328+0000 7f6c817fa640 1 -- 172.21.15.64:0/1155002331 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94026bd0 con 0x558b576b9c90 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/1155002331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 msgr2=0x558b576bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576bdb80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400f790 tx=0x7f6c94012000 comp rx=0 tx=0).stop 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/1155002331 shutdown_connections 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b9c90 0x558b576bdb80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b4b20 0x558b576c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/1155002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/1155002331 >> 172.21.15.64:0/1155002331 conn(0x558b576af7a0 msgr2=0x558b576b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/1155002331 shutdown_connections 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/1155002331 wait complete. 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.329+0000 7f6ca4c6b740 1 -- start start 2024-05-09T09:17:31.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 Processor -- start 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b9c90 0x558b576c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b7c00 con 0x558b576b4130 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b9070 con 0x558b576b9c90 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b82d0 con 0x558b576b4b20 2024-05-09T09:17:31.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.330+0000 7f6ca4c6b740 10 client.0 _ll_register_callbacks cb 0x7f6c900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:31.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53978/0 (socket says 172.21.15.64:53978) 2024-05-09T09:17:31.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9b7fe640 1 -- 172.21.15.64:0/4021796432 learned_addr learned my addr 172.21.15.64:0/4021796432 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9affd640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9bfff640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b9c90 0x558b576c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9affd640 1 -- 172.21.15.64:0/4021796432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b9c90 msgr2=0x558b576c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9affd640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b9c90 0x558b576c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9affd640 1 -- 172.21.15.64:0/4021796432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 msgr2=0x558b576c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9affd640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.331+0000 7f6c9affd640 1 -- 172.21.15.64:0/4021796432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b576c4ab0 con 0x558b576b4b20 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.332+0000 7f6c9affd640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576c34b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bd40 tx=0x7f6c8c0121d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a3df525aa6d4cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.332+0000 7f6c7affd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.332+0000 7f6c7affd640 1 -- 172.21.15.64:0/4021796432 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0d3060 con 0x558b576b4b20 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.333+0000 7f6c7affd640 1 -- 172.21.15.64:0/4021796432 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c8c0ebe60 con 0x558b576b4b20 2024-05-09T09:17:31.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.333+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9401ee50 con 0x558b576b4b20 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.333+0000 7f6c7affd640 1 -- 172.21.15.64:0/4021796432 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6c8c070050 con 0x558b576b4b20 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 10 client.27235 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558b57741670 con 0x558b576b4b20 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6c7affd640 1 -- 172.21.15.64:0/4021796432 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c9401ee50 con 0x558b576b4b20 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6c7affd640 1 -- 172.21.15.64:0/4021796432 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6c8c0b7120 con 0x558b576b4b20 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 10 client.27235 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 10 client.27235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 20 client.27235 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b57741eb0 0x558b577442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 4 client.27235 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:17:31.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.334+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b576bd9a0 con 0x558b57741eb0 2024-05-09T09:17:31.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.335+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b57741eb0 0x558b577442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.337+0000 7f6c9b7fe640 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b57741eb0 0x558b577442d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14adf6c97f92c99c server_cookie=a79e0ae953471b79 in_seq=0 out_seq=0 2024-05-09T09:17:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.337+0000 7f6c7affd640 10 client.27235 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:17:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.337+0000 7f6c7affd640 1 -- 172.21.15.64:0/4021796432 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b576bd9a0 con 0x558b57741eb0 2024-05-09T09:17:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.337+0000 7f6c7affd640 10 client.27235 handle_command_reply: tid=0 2024-05-09T09:17:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.338+0000 7f6ca4c6b740 1 client.27235 shutdown 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 msgr2=0x558b576c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576c34b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bd40 tx=0x7f6c8c0121d0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 shutdown_connections 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x558b57741eb0 0x558b577442d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558b576b9c90 0x558b576c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b576b4b20 0x558b576c34b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 --2- 172.21.15.64:0/4021796432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b576b4130 0x558b576c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 >> 172.21.15.64:0/4021796432 conn(0x558b576af7a0 msgr2=0x558b576b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.339+0000 7f6c7affd640 0 client.27235 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.340+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 shutdown_connections 2024-05-09T09:17:31.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.340+0000 7f6ca4c6b740 1 -- 172.21.15.64:0/4021796432 wait complete. 2024-05-09T09:17:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.340+0000 7f6ca4c6b740 20 client.27235 trim_cache size 0 max 16384 2024-05-09T09:17:31.340 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:17:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.341+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c840a2d90 msgr2=0x7f6c840a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.341+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c840a2d90 0x7f6c840a5250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x558b576b5380 tx=0x7f6c9400a2d0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.341+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 msgr2=0x7f6c9c1a6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.342+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1a6ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c02ebf0 tx=0x7f6c8c003ba0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.343+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 shutdown_connections 2024-05-09T09:17:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.343+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c840a2d90 0x7f6c840a5250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.343+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c9c1abf50 0x7f6c9c1ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.343+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c9c10dde0 0x7f6c9c1a6ef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.343+0000 7f6ca0cff640 1 --2- 172.21.15.64:0/2899711712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c9c108a10 0x7f6c9c1a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.343+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 >> 172.21.15.64:0/2899711712 conn(0x7f6c9c06e740 msgr2=0x7f6c9c10f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.344+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 shutdown_connections 2024-05-09T09:17:31.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.344+0000 7f6ca0cff640 1 -- 172.21.15.64:0/2899711712 wait complete. 2024-05-09T09:17:31.355 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 430.0 s, 818.8 tps, lat 122.180 ms stddev 56.785 2024-05-09T09:17:31.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:17:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.573+0000 7fa55acf8640 1 -- 172.21.15.64:0/2105339519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 msgr2=0x7fa554110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.573+0000 7fa55acf8640 1 --2- 172.21.15.64:0/2105339519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 0x7fa554110190 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa550009900 tx=0x7fa55002a400 comp rx=0 tx=0).stop 2024-05-09T09:17:31.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.574+0000 7fa55acf8640 1 -- 172.21.15.64:0/2105339519 shutdown_connections 2024-05-09T09:17:31.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.574+0000 7fa55acf8640 1 --2- 172.21.15.64:0/2105339519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 0x7fa554110190 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.574+0000 7fa55acf8640 1 --2- 172.21.15.64:0/2105339519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554108f40 0x7fa5541093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.574+0000 7fa55acf8640 1 --2- 172.21.15.64:0/2105339519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa554107d50 0x7fa554108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.574+0000 7fa55acf8640 1 -- 172.21.15.64:0/2105339519 >> 172.21.15.64:0/2105339519 conn(0x7fa554076cf0 msgr2=0x7fa554079130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.577+0000 7fa55acf8640 1 -- 172.21.15.64:0/2105339519 shutdown_connections 2024-05-09T09:17:31.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.577+0000 7fa55acf8640 1 -- 172.21.15.64:0/2105339519 wait complete. 2024-05-09T09:17:31.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.578+0000 7fa55acf8640 1 Processor -- start 2024-05-09T09:17:31.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.578+0000 7fa55acf8640 1 -- start start 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa55acf8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 0x7fa55419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa55acf8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa554108f40 0x7fa55419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa55acf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 0x7fa5541a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa55acf8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541138b0 con 0x7fa554109900 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa55acf8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa554113730 con 0x7fa554107d50 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa55acf8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa554113a30 con 0x7fa554108f40 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa559cf6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 0x7fa55419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa559cf6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 0x7fa55419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42612/0 (socket says 172.21.15.64:42612) 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa559cf6640 1 -- 172.21.15.64:0/108017028 learned_addr learned my addr 172.21.15.64:0/108017028 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.579+0000 7fa5594f5640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa554108f40 0x7fa55419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.580+0000 7fa559cf6640 1 -- 172.21.15.64:0/108017028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa554108f40 msgr2=0x7fa55419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.580+0000 7fa559cf6640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa554108f40 0x7fa55419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.580+0000 7fa559cf6640 1 -- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 msgr2=0x7fa5541a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.580+0000 7fa559cf6640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 0x7fa5541a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.580+0000 7fa559cf6640 1 -- 172.21.15.64:0/108017028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5541a5b40 con 0x7fa554107d50 2024-05-09T09:17:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.581+0000 7fa559cf6640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 0x7fa55419df20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00ebc0 tx=0x7fa54c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.584+0000 7fa54affd640 1 -- 172.21.15.64:0/108017028 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c003b70 con 0x7fa554107d50 2024-05-09T09:17:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.584+0000 7fa54affd640 1 -- 172.21.15.64:0/108017028 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa54c003d10 con 0x7fa554107d50 2024-05-09T09:17:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.584+0000 7fa54affd640 1 -- 172.21.15.64:0/108017028 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c019800 con 0x7fa554107d50 2024-05-09T09:17:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.586+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55406b2b0 con 0x7fa554107d50 2024-05-09T09:17:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.586+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55406b780 con 0x7fa554107d50 2024-05-09T09:17:31.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.591+0000 7fa55ec69740 1 Processor -- start 2024-05-09T09:17:31.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 -- start start 2024-05-09T09:17:31.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40130 0x55f717a40550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a45620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717a48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f7179a4c40 con 0x55f717a40130 2024-05-09T09:17:31.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f71794c580 con 0x55f717a40b20 2024-05-09T09:17:31.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.592+0000 7fa55ec69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f7179a0c00 con 0x55f717a45c90 2024-05-09T09:17:31.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.594+0000 7fa5594f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717a48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.594+0000 7fa5594f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717a48120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38904/0 (socket says 172.21.15.64:38904) 2024-05-09T09:17:31.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.594+0000 7fa5594f5640 1 -- 172.21.15.64:0/3794950186 learned_addr learned my addr 172.21.15.64:0/3794950186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.594+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40130 0x55f717a40550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.594+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a45620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa5594f5640 1 -- 172.21.15.64:0/3794950186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 msgr2=0x55f717a45620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a45620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa5594f5640 1 -- 172.21.15.64:0/3794950186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40130 msgr2=0x55f717a40550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40130 0x55f717a40550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa54affd640 1 -- 172.21.15.64:0/108017028 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa54c0199a0 con 0x7fa554107d50 2024-05-09T09:17:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa5594f5640 1 -- 172.21.15.64:0/3794950186 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7176b83f0 con 0x55f717a45c90 2024-05-09T09:17:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.595+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40130 0x55f717a40550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:31.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.596+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717a48120 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa544009c20 tx=0x7fa5440227c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6d2729de992315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.597+0000 7fa53ffff640 1 -- 172.21.15.64:0/3794950186 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54402f070 con 0x55f717a45c90 2024-05-09T09:17:31.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.597+0000 7fa53ffff640 1 -- 172.21.15.64:0/3794950186 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa544009d10 con 0x55f717a45c90 2024-05-09T09:17:31.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.597+0000 7fa53ffff640 1 -- 172.21.15.64:0/3794950186 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544020810 con 0x55f717a45c90 2024-05-09T09:17:31.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.597+0000 7fa54affd640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5400a2d60 0x7fa5400a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.597+0000 7fa54affd640 1 -- 172.21.15.64:0/108017028 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa54c0cfe50 con 0x7fa554107d50 2024-05-09T09:17:31.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.598+0000 7fa55ec69740 1 -- 172.21.15.64:0/3794950186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 msgr2=0x55f717a48120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.598+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717a48120 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa544009c20 tx=0x7fa5440227c0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.598+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5400a2d60 0x7fa5400a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.599+0000 7fa55ec69740 1 -- 172.21.15.64:0/3794950186 shutdown_connections 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.599+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717a48120 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.599+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a45620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.599+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3794950186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40130 0x55f717a40550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.599+0000 7fa55ec69740 1 -- 172.21.15.64:0/3794950186 >> 172.21.15.64:0/3794950186 conn(0x55f717a3b7a0 msgr2=0x55f717a3bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.599+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5400a2d60 0x7fa5400a5220 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa550005730 tx=0x7fa5500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.600+0000 7fa55ec69740 1 -- 172.21.15.64:0/3794950186 shutdown_connections 2024-05-09T09:17:31.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.602+0000 7fa55ec69740 1 -- 172.21.15.64:0/3794950186 wait complete. 2024-05-09T09:17:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.602+0000 7fa55ec69740 1 -- start start 2024-05-09T09:17:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.602+0000 7fa55ec69740 1 Processor -- start 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f717a48c40 con 0x55f717a45c90 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f71795fc40 con 0x55f717a40b20 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f717a48dc0 con 0x55f717a40130 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55ec69740 10 client.0 _ll_register_callbacks cb 0x55f7176b7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55a4f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55a4f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54020/0 (socket says 172.21.15.64:54020) 2024-05-09T09:17:31.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.603+0000 7fa55a4f7640 1 -- 172.21.15.64:0/3591776872 learned_addr learned my addr 172.21.15.64:0/3591776872 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa55a4f7640 1 -- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 msgr2=0x55f717a4a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa55a4f7640 1 -- 172.21.15.64:0/3591776872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 msgr2=0x55f717a4ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.604+0000 7fa55a4f7640 1 -- 172.21.15.64:0/3591776872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7177827d0 con 0x55f717a45c90 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.605+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa550033000 tx=0x7fa550005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716a1fdf16fe100a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.605+0000 7fa53d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:17:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.606+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3591776872 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550035040 con 0x55f717a45c90 2024-05-09T09:17:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.606+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa544009d10 con 0x55f717a45c90 2024-05-09T09:17:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.606+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3591776872 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa550031990 con 0x55f717a45c90 2024-05-09T09:17:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.606+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3591776872 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa550056050 con 0x55f717a45c90 2024-05-09T09:17:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.606+0000 7fa55ec69740 10 client.17601 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.606+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f717a54f90 con 0x55f717a45c90 2024-05-09T09:17:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.607+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3591776872 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa544009d10 con 0x55f717a45c90 2024-05-09T09:17:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.608+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3591776872 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa55003a040 con 0x55f717a45c90 2024-05-09T09:17:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.609+0000 7fa55ec69740 10 client.17601 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.609+0000 7fa55ec69740 10 client.17601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:17:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.609+0000 7fa55ec69740 20 client.17601 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.609+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717a566d0 0x55f717a58b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.609+0000 7fa55ec69740 4 client.17601 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.609+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7179b2370 con 0x55f717a566d0 2024-05-09T09:17:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.611+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717a566d0 0x55f717a58b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.612+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717a566d0 0x55f717a58b50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddf892435cb64630 server_cookie=7ee195291c8b24e4 in_seq=0 out_seq=0 2024-05-09T09:17:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.612+0000 7fa53d7fa640 10 client.17601 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:17:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.613+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3591776872 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa54c0abae0 con 0x55f717a566d0 2024-05-09T09:17:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.613+0000 7fa53d7fa640 10 client.17601 handle_command_reply: tid=0 2024-05-09T09:17:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.613+0000 7fa55ec69740 1 client.17601 shutdown 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 msgr2=0x55f717a51b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa550033000 tx=0x7fa550005fd0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 shutdown_connections 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717a566d0 0x55f717a58b50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3591776872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.615+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 >> 172.21.15.64:0/3591776872 conn(0x55f717a3b7a0 msgr2=0x55f717a3a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.616+0000 7fa53d7fa640 0 client.17601 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:17:31.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.619+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 shutdown_connections 2024-05-09T09:17:31.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.619+0000 7fa55ec69740 1 -- 172.21.15.64:0/3591776872 wait complete. 2024-05-09T09:17:31.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.620+0000 7fa55ec69740 20 client.17601 trim_cache size 0 max 16384 2024-05-09T09:17:31.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.649+0000 7fa55ec69740 1 Processor -- start 2024-05-09T09:17:31.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.650+0000 7fa55ec69740 1 -- start start 2024-05-09T09:17:31.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.650+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.650+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa55ec69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541aa940 con 0x55f717a45c90 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa55ec69740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541a8210 con 0x55f717a40b20 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa55ec69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541a6060 con 0x55f717a40130 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa5594f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa5594f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42630/0 (socket says 172.21.15.64:42630) 2024-05-09T09:17:31.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.651+0000 7fa5594f5640 1 -- 172.21.15.64:0/1283781479 learned_addr learned my addr 172.21.15.64:0/1283781479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa5594f5640 1 -- 172.21.15.64:0/1283781479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 msgr2=0x55f717a4a5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa5594f5640 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa5594f5640 1 -- 172.21.15.64:0/1283781479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 msgr2=0x55f717a51b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa5594f5640 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa5594f5640 1 -- 172.21.15.64:0/1283781479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f717a3c510 con 0x55f717a40b20 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa5594f5640 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55f717a3c710 tx=0x7fa544005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f5ef3ebe87df88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.652+0000 7fa53f7fe640 1 -- 172.21.15.64:0/1283781479 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544037020 con 0x55f717a40b20 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.653+0000 7fa53f7fe640 1 -- 172.21.15.64:0/1283781479 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa544009e10 con 0x55f717a40b20 2024-05-09T09:17:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.653+0000 7fa53f7fe640 1 -- 172.21.15.64:0/1283781479 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54402f040 con 0x55f717a40b20 2024-05-09T09:17:31.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.654+0000 7fa55ec69740 1 -- 172.21.15.64:0/1283781479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 msgr2=0x55f717a4ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.654+0000 7fa55ec69740 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55f717a3c710 tx=0x7fa544005d10 comp rx=0 tx=0).stop 2024-05-09T09:17:31.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.656+0000 7fa55ec69740 1 -- 172.21.15.64:0/1283781479 shutdown_connections 2024-05-09T09:17:31.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.656+0000 7fa55ec69740 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a45c90 0x55f717a51b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.656+0000 7fa55ec69740 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40b20 0x55f717a4ab20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.656+0000 7fa55ec69740 1 --2- 172.21.15.64:0/1283781479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a40130 0x55f717a4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.656+0000 7fa55ec69740 1 -- 172.21.15.64:0/1283781479 >> 172.21.15.64:0/1283781479 conn(0x55f717a3b7a0 msgr2=0x55f717a3bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.657+0000 7fa55ec69740 1 -- 172.21.15.64:0/1283781479 shutdown_connections 2024-05-09T09:17:31.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.657+0000 7fa55ec69740 1 -- 172.21.15.64:0/1283781479 wait complete. 2024-05-09T09:17:31.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.659+0000 7fa55ec69740 1 -- start start 2024-05-09T09:17:31.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.660+0000 7fa55ec69740 1 Processor -- start 2024-05-09T09:17:31.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.665+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40130 0x55f717a4eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.665+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 0x55f717a4f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.665+0000 7fa55ec69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717acbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.665+0000 7fa55ec69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541a66d0 con 0x55f717a40b20 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.665+0000 7fa55ec69740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541a7b40 con 0x55f717a40130 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.665+0000 7fa55ec69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541a6da0 con 0x55f717a45c90 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 0x55f717a4f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa55ec69740 10 client.0 _ll_register_callbacks cb 0x7fa544005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 0x55f717a4f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54026/0 (socket says 172.21.15.64:54026) 2024-05-09T09:17:31.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 -- 172.21.15.64:0/3198326369 learned_addr learned my addr 172.21.15.64:0/3198326369 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40130 0x55f717a4eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa55a4f7640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717acbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 -- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 msgr2=0x55f717acbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717acbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 -- 172.21.15.64:0/3198326369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40130 msgr2=0x55f717a4eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40130 0x55f717a4eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.666+0000 7fa5594f5640 1 -- 172.21.15.64:0/3198326369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f717acc250 con 0x55f717a40b20 2024-05-09T09:17:31.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.667+0000 7fa5594f5640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 0x55f717a4f070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa544004ed0 tx=0x7fa544004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e01ee02095b2931 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.668+0000 7fa53d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:17:31.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.668+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3198326369 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544040020 con 0x55f717a40b20 2024-05-09T09:17:31.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.668+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3198326369 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa544005250 con 0x55f717a40b20 2024-05-09T09:17:31.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.668+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa544009e10 con 0x55f717a40b20 2024-05-09T09:17:31.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.668+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3198326369 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa54402f040 con 0x55f717a40b20 2024-05-09T09:17:31.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.668+0000 7fa55ec69740 10 client.17604 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:31.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.669+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f717acd550 con 0x55f717a40b20 2024-05-09T09:17:31.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.669+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3198326369 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa544009e10 con 0x55f717a40b20 2024-05-09T09:17:31.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.669+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3198326369 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa544004040 con 0x55f717a40b20 2024-05-09T09:17:31.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.669+0000 7fa55ec69740 10 client.17604 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:31.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.670+0000 7fa55ec69740 10 client.17604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:17:31.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.670+0000 7fa55ec69740 20 client.17604 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:31.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.670+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717acdc60 0x55f717ad0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.670+0000 7fa55ec69740 4 client.17604 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:17:31.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.670+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7179ac6c0 con 0x55f717acdc60 2024-05-09T09:17:31.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.671+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717acdc60 0x55f717ad0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.673+0000 7fa53d7fa640 10 client.17604 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:17:31.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.673+0000 7fa559cf6640 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717acdc60 0x55f717ad0080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65062c1f5c61a35 server_cookie=5fc444041d8b5fa2 in_seq=0 out_seq=0 2024-05-09T09:17:31.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.676+0000 7fa53d7fa640 1 -- 172.21.15.64:0/3198326369 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7179ac6c0 con 0x55f717acdc60 2024-05-09T09:17:31.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.676+0000 7fa53d7fa640 10 client.17604 handle_command_reply: tid=0 2024-05-09T09:17:31.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.676+0000 7fa55ec69740 1 client.17604 shutdown 2024-05-09T09:17:31.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.676+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 msgr2=0x55f717a4f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.676+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 0x55f717a4f070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa544004ed0 tx=0x7fa544004f00 comp rx=0 tx=0).stop 2024-05-09T09:17:31.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.676+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 shutdown_connections 2024-05-09T09:17:31.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f717a45c90 0x55f717acbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f717acdc60 0x55f717ad0080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f717a40b20 0x55f717a4f070 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 --2- 172.21.15.64:0/3198326369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f717a40130 0x55f717a4eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 >> 172.21.15.64:0/3198326369 conn(0x55f717a3b7a0 msgr2=0x55f717a4a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa53d7fa640 0 client.17604 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:17:31.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 shutdown_connections 2024-05-09T09:17:31.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.677+0000 7fa55ec69740 1 -- 172.21.15.64:0/3198326369 wait complete. 2024-05-09T09:17:31.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.678+0000 7fa55ec69740 20 client.17604 trim_cache size 0 max 16384 2024-05-09T09:17:31.678 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:17:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.679+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5400a2d60 msgr2=0x7fa5400a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.679+0000 7fa55acf8640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5400a2d60 0x7fa5400a5220 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa550005730 tx=0x7fa5500314a0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.679+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 msgr2=0x7fa55419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.679+0000 7fa55acf8640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 0x7fa55419df20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00ebc0 tx=0x7fa54c00c6a0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 shutdown_connections 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5400a2d60 0x7fa5400a5220 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa554109900 0x7fa5541a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa554108f40 0x7fa55419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 --2- 172.21.15.64:0/108017028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa554107d50 0x7fa55419df20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 >> 172.21.15.64:0/108017028 conn(0x7fa554076cf0 msgr2=0x7fa55410c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.680+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 shutdown_connections 2024-05-09T09:17:31.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.681+0000 7fa55acf8640 1 -- 172.21.15.64:0/108017028 wait complete. 2024-05-09T09:17:31.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:17:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 -- 172.21.15.64:0/3801210469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 msgr2=0x7f030c0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 --2- 172.21.15.64:0/3801210469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 0x7f030c0a6200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f030002a420 comp rx=0 tx=0).stop 2024-05-09T09:17:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 -- 172.21.15.64:0/3801210469 shutdown_connections 2024-05-09T09:17:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 --2- 172.21.15.64:0/3801210469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a6740 0x7f030c0ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 --2- 172.21.15.64:0/3801210469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 0x7f030c0a6200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 --2- 172.21.15.64:0/3801210469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f030c0a4c70 0x7f030c0a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.916+0000 7f031b1dc640 1 -- 172.21.15.64:0/3801210469 >> 172.21.15.64:0/3801210469 conn(0x7f030c0a0100 msgr2=0x7f030c0a2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.919+0000 7f031b1dc640 1 -- 172.21.15.64:0/3801210469 shutdown_connections 2024-05-09T09:17:31.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.919+0000 7f031b1dc640 1 -- 172.21.15.64:0/3801210469 wait complete. 2024-05-09T09:17:31.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.920+0000 7f031b1dc640 1 Processor -- start 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.920+0000 7f031b1dc640 1 -- start start 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.920+0000 7f031b1dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 0x7f030c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.920+0000 7f031b1dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 0x7f030c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f031b1dc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f030c0a6740 0x7f030c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f031b1dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0b8fa0 con 0x7f030c0a4c70 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f031b1dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0b8e20 con 0x7f030c0a5da0 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f031b1dc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0b9120 con 0x7f030c0a6740 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f0318f78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 0x7f030c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f0318f78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 0x7f030c0b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54048/0 (socket says 172.21.15.64:54048) 2024-05-09T09:17:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f0318f78640 1 -- 172.21.15.64:0/2442670472 learned_addr learned my addr 172.21.15.64:0/2442670472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f0313fff640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 0x7f030c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.921+0000 7f0319779640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f030c0a6740 0x7f030c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.922+0000 7f0318f78640 1 -- 172.21.15.64:0/2442670472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f030c0a6740 msgr2=0x7f030c1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.922+0000 7f0318f78640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f030c0a6740 0x7f030c1426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.922+0000 7f0318f78640 1 -- 172.21.15.64:0/2442670472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 msgr2=0x7f030c0b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.922+0000 7f0318f78640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 0x7f030c0b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.922+0000 7f0318f78640 1 -- 172.21.15.64:0/2442670472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c0b4d30 con 0x7f030c0a4c70 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.922+0000 7f0318f78640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 0x7f030c0b1670 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f030800e9f0 tx=0x7f030800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.923+0000 7f0311ffb640 1 -- 172.21.15.64:0/2442670472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030800cd80 con 0x7f030c0a4c70 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.923+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c142be0 con 0x7f030c0a4c70 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.923+0000 7f0311ffb640 1 -- 172.21.15.64:0/2442670472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03080108e0 con 0x7f030c0a4c70 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.923+0000 7f0311ffb640 1 -- 172.21.15.64:0/2442670472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308018920 con 0x7f030c0a4c70 2024-05-09T09:17:31.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.924+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c143130 con 0x7f030c0a4c70 2024-05-09T09:17:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.929+0000 7f031dfc9740 1 Processor -- start 2024-05-09T09:17:31.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 -- start start 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087130 0x561288087550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x56128808c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x56128808f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561287febc40 con 0x561288087130 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561287f93580 con 0x561288087b20 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f031dfc9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561287fe7c00 con 0x56128808cc90 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.932+0000 7f0311ffb640 1 -- 172.21.15.64:0/2442670472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0308010a90 con 0x7f030c0a4c70 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.933+0000 7f0311ffb640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03040a2d60 0x7f03040a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.933+0000 7f0311ffb640 1 -- 172.21.15.64:0/2442670472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03080cf630 con 0x7f030c0a4c70 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.934+0000 7f0313fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x56128808f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.934+0000 7f0313fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x56128808f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38960/0 (socket says 172.21.15.64:38960) 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.934+0000 7f0313fff640 1 -- 172.21.15.64:0/1420459911 learned_addr learned my addr 172.21.15.64:0/1420459911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.934+0000 7f0319779640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03040a2d60 0x7f03040a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.934+0000 7f0318f78640 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x56128808c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0319779640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03040a2d60 0x7f03040a5220 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0314064d50 tx=0x7f0314067a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0313fff640 1 -- 172.21.15.64:0/1420459911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 msgr2=0x56128808c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0313fff640 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x56128808c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0313fff640 1 -- 172.21.15.64:0/1420459911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087130 msgr2=0x561288087550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0313fff640 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087130 0x561288087550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0313fff640 1 -- 172.21.15.64:0/1420459911 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561287cff3f0 con 0x56128808cc90 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f0313fff640 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x56128808f120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f0300004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f6db9177b73247 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.935+0000 7f02feffd640 1 -- 172.21.15.64:0/1420459911 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035040 con 0x56128808cc90 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.936+0000 7f02feffd640 1 -- 172.21.15.64:0/1420459911 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03000044b0 con 0x56128808cc90 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.936+0000 7f02feffd640 1 -- 172.21.15.64:0/1420459911 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300003a10 con 0x56128808cc90 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.936+0000 7f031dfc9740 1 -- 172.21.15.64:0/1420459911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 msgr2=0x56128808f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.936+0000 7f031dfc9740 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x56128808f120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f0300004200 comp rx=0 tx=0).stop 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 -- 172.21.15.64:0/1420459911 shutdown_connections 2024-05-09T09:17:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x56128808f120 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x56128808c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 --2- 172.21.15.64:0/1420459911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087130 0x561288087550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 -- 172.21.15.64:0/1420459911 >> 172.21.15.64:0/1420459911 conn(0x5612880827a0 msgr2=0x561288082ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 -- 172.21.15.64:0/1420459911 shutdown_connections 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.937+0000 7f031dfc9740 1 -- 172.21.15.64:0/1420459911 wait complete. 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.938+0000 7f031dfc9740 1 -- start start 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.938+0000 7f031dfc9740 1 Processor -- start 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 0x561288091500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x561288091a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56128808cc90 0x561288098a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56128808fbe0 con 0x56128808cc90 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561287fa6c40 con 0x561288087b20 2024-05-09T09:17:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56128808fd60 con 0x561288087130 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f031dfc9740 10 client.0 _ll_register_callbacks cb 0x561287cfebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 0x561288091500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 0x561288091500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38976/0 (socket says 172.21.15.64:38976) 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 -- 172.21.15.64:0/2539724307 learned_addr learned my addr 172.21.15.64:0/2539724307 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0319779640 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56128808cc90 0x561288098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 -- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 msgr2=0x561288091a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x561288091a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 -- 172.21.15.64:0/2539724307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56128808cc90 msgr2=0x561288098a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56128808cc90 0x561288098a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 -- 172.21.15.64:0/2539724307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561287dc97d0 con 0x561288087130 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.939+0000 7f0318f78640 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 0x561288091500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0308098520 tx=0x7f030809bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fc80b310d924f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.940+0000 7f02f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.940+0000 7f02f3fff640 1 -- 172.21.15.64:0/2539724307 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03080b4060 con 0x561288087130 2024-05-09T09:17:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.942+0000 7f02f3fff640 1 -- 172.21.15.64:0/2539724307 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0308010430 con 0x561288087130 2024-05-09T09:17:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.947+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03000044b0 con 0x561288087130 2024-05-09T09:17:31.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.948+0000 7f02f3fff640 1 -- 172.21.15.64:0/2539724307 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f03080b9050 con 0x561288087130 2024-05-09T09:17:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 10 client.27262 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56128809bf10 con 0x561288087130 2024-05-09T09:17:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f02f3fff640 1 -- 172.21.15.64:0/2539724307 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03000044b0 con 0x561288087130 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f02f3fff640 1 -- 172.21.15.64:0/2539724307 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f03080978a0 con 0x561288087130 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 10 client.27262 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 20 client.27262 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56128809d610 0x56128809fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 4 client.27262 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.949+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561287ff9370 con 0x56128809d610 2024-05-09T09:17:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.950+0000 7f0313fff640 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56128809d610 0x56128809fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.950+0000 7f02f3fff640 10 client.27262 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.951+0000 7f0313fff640 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56128809d610 0x56128809fab0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70230bb459207af2 server_cookie=c4a63de069badcf3 in_seq=0 out_seq=0 2024-05-09T09:17:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.952+0000 7f02f3fff640 1 -- 172.21.15.64:0/2539724307 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561287ff9370 con 0x56128809d610 2024-05-09T09:17:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.952+0000 7f02f3fff640 10 client.27262 handle_command_reply: tid=0 2024-05-09T09:17:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.952+0000 7f031dfc9740 1 client.27262 shutdown 2024-05-09T09:17:31.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 msgr2=0x561288091500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 0x561288091500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0308098520 tx=0x7f030809bb50 comp rx=0 tx=0).stop 2024-05-09T09:17:31.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 shutdown_connections 2024-05-09T09:17:31.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56128809d610 0x56128809fab0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56128808cc90 0x561288098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288087b20 0x561288091a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 --2- 172.21.15.64:0/2539724307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087130 0x561288091500 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0308098520 tx=0x7f030809bb50 comp rx=0 tx=0).stop 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 >> 172.21.15.64:0/2539724307 conn(0x5612880827a0 msgr2=0x5612880819c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.955+0000 7f02f3fff640 0 client.27262 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.956+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 shutdown_connections 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.956+0000 7f031dfc9740 1 -- 172.21.15.64:0/2539724307 wait complete. 2024-05-09T09:17:31.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.956+0000 7f031dfc9740 20 client.27262 trim_cache size 0 max 16384 2024-05-09T09:17:31.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.982+0000 7f031dfc9740 1 Processor -- start 2024-05-09T09:17:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.982+0000 7f031dfc9740 1 -- start start 2024-05-09T09:17:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.982+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 0x561288091110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56128808cc90 0x561288098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f0318f78640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 0x561288091110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f0318f78640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 0x561288091110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:38980/0 (socket says 172.21.15.64:38980) 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288091650 0x561288091a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f0318f78640 1 -- 172.21.15.64:0/495261787 learned_addr learned my addr 172.21.15.64:0/495261787 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f030c171e10 con 0x561288091650 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f030c14d790 con 0x56128808cc90 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.983+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f030c14b600 con 0x561288087b20 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.984+0000 7f0318f78640 1 -- 172.21.15.64:0/495261787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56128808cc90 msgr2=0x561288098a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:17:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.984+0000 7f0318f78640 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56128808cc90 0x561288098a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.984+0000 7f0318f78640 1 -- 172.21.15.64:0/495261787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288091650 msgr2=0x561288091a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.984+0000 7f0318f78640 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288091650 0x561288091a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.984+0000 7f0318f78640 1 -- 172.21.15.64:0/495261787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128809bd30 con 0x561288087b20 2024-05-09T09:17:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.984+0000 7f0318f78640 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 0x561288091110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0308097d90 tx=0x7f03080cf960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8c7f840fdf20c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.985+0000 7f02fe7fc640 1 -- 172.21.15.64:0/495261787 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030809b920 con 0x561288087b20 2024-05-09T09:17:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.985+0000 7f02fe7fc640 1 -- 172.21.15.64:0/495261787 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03080978a0 con 0x561288087b20 2024-05-09T09:17:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.985+0000 7f02fe7fc640 1 -- 172.21.15.64:0/495261787 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030809afa0 con 0x561288087b20 2024-05-09T09:17:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.986+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 msgr2=0x561288091110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.986+0000 7f031dfc9740 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 0x561288091110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0308097d90 tx=0x7f03080cf960 comp rx=0 tx=0).stop 2024-05-09T09:17:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.986+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 shutdown_connections 2024-05-09T09:17:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.986+0000 7f031dfc9740 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288091650 0x561288091a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.986+0000 7f031dfc9740 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56128808cc90 0x561288098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.987+0000 7f031dfc9740 1 --2- 172.21.15.64:0/495261787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561288087b20 0x561288091110 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.987+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 >> 172.21.15.64:0/495261787 conn(0x5612880827a0 msgr2=0x561288082b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.987+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 shutdown_connections 2024-05-09T09:17:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.988+0000 7f031dfc9740 1 -- 172.21.15.64:0/495261787 wait complete. 2024-05-09T09:17:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.988+0000 7f031dfc9740 1 -- start start 2024-05-09T09:17:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.988+0000 7f031dfc9740 1 Processor -- start 2024-05-09T09:17:31.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.989+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087b20 0x5612880961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.989+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x561288096720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.989+0000 7f031dfc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 0x561288097730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:31.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f031dfc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f030c14bc50 con 0x561288087b20 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f031dfc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f030c14d0c0 con 0x561288091650 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f031dfc9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f030c14c320 con 0x56128808cc90 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 0x561288097730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0313fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x561288096720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 0x561288097730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42688/0 (socket says 172.21.15.64:42688) 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 -- 172.21.15.64:0/3346957222 learned_addr learned my addr 172.21.15.64:0/3346957222 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f031dfc9740 10 client.0 _ll_register_callbacks cb 0x7f0308097df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0318f78640 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087b20 0x5612880961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 -- 172.21.15.64:0/3346957222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 msgr2=0x561288096720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x561288096720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 -- 172.21.15.64:0/3346957222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087b20 msgr2=0x5612880961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:31.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087b20 0x5612880961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:31.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.990+0000 7f0319779640 1 -- 172.21.15.64:0/3346957222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561288097d20 con 0x561288091650 2024-05-09T09:17:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f0319779640 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 0x561288097730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0314050860 tx=0x7f0314067700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b8c16b6a61f9a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f02f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:17:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f02f3fff640 1 -- 172.21.15.64:0/3346957222 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314088020 con 0x561288091650 2024-05-09T09:17:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f02f3fff640 1 -- 172.21.15.64:0/3346957222 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0314077dd0 con 0x561288091650 2024-05-09T09:17:31.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03080978a0 con 0x561288091650 2024-05-09T09:17:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f02f3fff640 1 -- 172.21.15.64:0/3346957222 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0314093050 con 0x561288091650 2024-05-09T09:17:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f031dfc9740 10 client.27104 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.991+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561288103cc0 con 0x561288091650 2024-05-09T09:17:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.992+0000 7f02f3fff640 1 -- 172.21.15.64:0/3346957222 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031407f040 con 0x561288091650 2024-05-09T09:17:31.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.993+0000 7f02f3fff640 1 -- 172.21.15.64:0/3346957222 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f031408c5b0 con 0x561288091650 2024-05-09T09:17:31.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f031dfc9740 10 client.27104 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:31.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f031dfc9740 10 client.27104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:31.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f031dfc9740 20 client.27104 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f031dfc9740 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561288104740 0x561288106b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f031dfc9740 4 client.27104 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f03080978a0 con 0x561288104740 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.995+0000 7f0318f78640 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561288104740 0x561288106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f02f3fff640 10 client.27104 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f0318f78640 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561288104740 0x561288106b60 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db2897391f2880f server_cookie=d5ca30bc4365f7c6 in_seq=0 out_seq=0 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f02f3fff640 1 -- 172.21.15.64:0/3346957222 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f03080978a0 con 0x561288104740 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f02f3fff640 10 client.27104 handle_command_reply: tid=0 2024-05-09T09:17:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f031dfc9740 1 client.27104 shutdown 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 msgr2=0x561288097730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.996+0000 7f031dfc9740 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 0x561288097730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0314050860 tx=0x7f0314067700 comp rx=0 tx=0).stop 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 shutdown_connections 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x561288104740 0x561288106b60 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561288091650 0x561288097730 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56128808cc90 0x561288096720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 --2- 172.21.15.64:0/3346957222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561288087b20 0x5612880961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 >> 172.21.15.64:0/3346957222 conn(0x5612880827a0 msgr2=0x561288081130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:32.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f02f3fff640 0 client.27104 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:32.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.997+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 shutdown_connections 2024-05-09T09:17:32.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.998+0000 7f031dfc9740 1 -- 172.21.15.64:0/3346957222 wait complete. 2024-05-09T09:17:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.998+0000 7f031dfc9740 20 client.27104 trim_cache size 0 max 16384 2024-05-09T09:17:32.003 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03040a2d60 msgr2=0x7f03040a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03040a2d60 0x7f03040a5220 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0314064d50 tx=0x7f0314067a30 comp rx=0 tx=0).stop 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 msgr2=0x7f030c0b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 0x7f030c0b1670 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f030800e9f0 tx=0x7f030800eeb0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 shutdown_connections 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f03040a2d60 0x7f03040a5220 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f030c0a6740 0x7f030c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f030c0a5da0 0x7f030c0b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 --2- 172.21.15.64:0/2442670472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f030c0a4c70 0x7f030c0b1670 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:31.999+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 >> 172.21.15.64:0/2442670472 conn(0x7f030c0a0100 msgr2=0x7f030c0a6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:32.000+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 shutdown_connections 2024-05-09T09:17:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:32.000+0000 7f031b1dc640 1 -- 172.21.15.64:0/2442670472 wait complete. 2024-05-09T09:17:32.017 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:31 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/268949031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:17:32.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:31 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/268949031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:17:32.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:31 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/268949031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:17:33.020 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:17:33.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:17:33.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:17:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:32 smithi139 ceph-mon[33599]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 817 op/s 2024-05-09T09:17:33.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:32 smithi064 ceph-mon[30149]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 817 op/s 2024-05-09T09:17:33.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 -- 172.21.15.64:0/2127304263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d24108830 msgr2=0x7f4d2410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/2127304263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d24108830 0x7f4d2410ac60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4d1401a870 tx=0x7f4d1403b360 comp rx=0 tx=0).stop 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 -- 172.21.15.64:0/2127304263 shutdown_connections 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/2127304263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2410db70 0x7f4d2410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/2127304263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410b1a0 0x7f4d2410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/2127304263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d24108830 0x7f4d2410ac60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.220+0000 7f4d2bac1640 1 -- 172.21.15.64:0/2127304263 >> 172.21.15.64:0/2127304263 conn(0x7f4d2406e6c0 msgr2=0x7f4d24070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.221+0000 7f4d2bac1640 1 -- 172.21.15.64:0/2127304263 shutdown_connections 2024-05-09T09:17:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.222+0000 7f4d2bac1640 1 -- 172.21.15.64:0/2127304263 wait complete. 2024-05-09T09:17:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 Processor -- start 2024-05-09T09:17:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 -- start start 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2410b1a0 0x7f4d2419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410db70 0x7f4d2419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241a3890 0x7f4d241a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241119a0 con 0x7f4d2410b1a0 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24111820 con 0x7f4d241a3890 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.223+0000 7f4d2bac1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24111b20 con 0x7f4d2410db70 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2985d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2410b1a0 0x7f4d2419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2a05e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241a3890 0x7f4d241a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2985d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2410b1a0 0x7f4d2419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54092/0 (socket says 172.21.15.64:54092) 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2985d640 1 -- 172.21.15.64:0/4026315692 learned_addr learned my addr 172.21.15.64:0/4026315692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2905c640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410db70 0x7f4d2419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2905c640 1 -- 172.21.15.64:0/4026315692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241a3890 msgr2=0x7f4d241a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2905c640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241a3890 0x7f4d241a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2905c640 1 -- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2410b1a0 msgr2=0x7f4d2419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2905c640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2410b1a0 0x7f4d2419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2905c640 1 -- 172.21.15.64:0/4026315692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d241a62f0 con 0x7f4d2410db70 2024-05-09T09:17:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.224+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241a3890 0x7f4d241a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:17:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.225+0000 7f4d2905c640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410db70 0x7f4d2419e830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d20005fd0 tx=0x7f4d2000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.225+0000 7f4d1affd640 1 -- 172.21.15.64:0/4026315692 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000ccc0 con 0x7f4d2410db70 2024-05-09T09:17:33.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.225+0000 7f4d1affd640 1 -- 172.21.15.64:0/4026315692 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2000ee70 con 0x7f4d2410db70 2024-05-09T09:17:33.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.225+0000 7f4d1affd640 1 -- 172.21.15.64:0/4026315692 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20003e10 con 0x7f4d2410db70 2024-05-09T09:17:33.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.226+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d241a65e0 con 0x7f4d2410db70 2024-05-09T09:17:33.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.226+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d241a6ae0 con 0x7f4d2410db70 2024-05-09T09:17:33.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:32 smithi179 ceph-mon[34086]: pgmap v622: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 817 op/s 2024-05-09T09:17:33.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.230+0000 7f4d1affd640 1 -- 172.21.15.64:0/4026315692 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d20003fb0 con 0x7f4d2410db70 2024-05-09T09:17:33.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.232+0000 7f4d2e7cd740 1 Processor -- start 2024-05-09T09:17:33.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.232+0000 7f4d2e7cd740 1 -- start start 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x5627832f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562783255990 con 0x5627832f1eb0 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5627831fd5b0 con 0x5627832f7020 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562783251950 con 0x5627832f14c0 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2905c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x5627832f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2905c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x5627832f94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42708/0 (socket says 172.21.15.64:42708) 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2905c640 1 -- 172.21.15.64:0/622197191 learned_addr learned my addr 172.21.15.64:0/622197191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39022/0 (socket says 172.21.15.64:39022) 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 -- 172.21.15.64:0/622197191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 msgr2=0x5627832f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x5627832f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 -- 172.21.15.64:0/622197191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 msgr2=0x5627832f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d2a05e640 1 -- 172.21.15.64:0/622197191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627830317e0 con 0x5627832f14c0 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.233+0000 7f4d1affd640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d100a2d60 0x7f4d100a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.234+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f18e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005fd0 tx=0x7f4d1c010b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8ad2e26a0b8df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.234+0000 7f4d1affd640 1 -- 172.21.15.64:0/4026315692 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4d200cf520 con 0x7f4d2410db70 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.234+0000 7f4d2985d640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d100a2d60 0x7f4d100a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.234+0000 7f4d0ffff640 1 -- 172.21.15.64:0/622197191 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c004b20 con 0x5627832f14c0 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.234+0000 7f4d0ffff640 1 -- 172.21.15.64:0/622197191 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d1c00ed80 con 0x5627832f14c0 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.234+0000 7f4d0ffff640 1 -- 172.21.15.64:0/622197191 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c005750 con 0x5627832f14c0 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/622197191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 msgr2=0x5627832f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2985d640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d100a2d60 0x7f4d100a5220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d1403b950 tx=0x7f4d14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f18e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005fd0 tx=0x7f4d1c010b20 comp rx=0 tx=0).stop 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/622197191 shutdown_connections 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x5627832f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/622197191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f18e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.235+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/622197191 >> 172.21.15.64:0/622197191 conn(0x5627832b90a0 msgr2=0x5627832ecf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.236+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/622197191 shutdown_connections 2024-05-09T09:17:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.236+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/622197191 wait complete. 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.236+0000 7f4d2e7cd740 1 -- start start 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.236+0000 7f4d2e7cd740 1 Processor -- start 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x562783302ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2985d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2985d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39024/0 (socket says 172.21.15.64:39024) 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2985d640 1 -- 172.21.15.64:0/1375076304 learned_addr learned my addr 172.21.15.64:0/1375076304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5627832f9fd0 con 0x5627832f1eb0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562783210b40 con 0x5627832f7020 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5627832fa150 con 0x5627832f14c0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2905c640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x562783302ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.237+0000 7f4d2e7cd740 10 client.0 _ll_register_callbacks cb 0x5627832050f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d2985d640 1 -- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 msgr2=0x562783302ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d2985d640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x562783302ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d2985d640 1 -- 172.21.15.64:0/1375076304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 msgr2=0x5627832fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d2985d640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d2985d640 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562783091f70 con 0x5627832f14c0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d2985d640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832fb920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d14046040 tx=0x7f4d1403b360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270209d951a18f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.238+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/1375076304 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1405c020 con 0x5627832f14c0 2024-05-09T09:17:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.239+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/1375076304 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4d14003cc0 con 0x5627832f14c0 2024-05-09T09:17:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.239+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1c00ed80 con 0x5627832f14c0 2024-05-09T09:17:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.242+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/1375076304 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4d14054040 con 0x5627832f14c0 2024-05-09T09:17:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.242+0000 7f4d2e7cd740 10 client.27280 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.242+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562782f29170 con 0x5627832f14c0 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.243+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/1375076304 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d1c00ed80 con 0x5627832f14c0 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/1375076304 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4d14057030 con 0x5627832f14c0 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d2e7cd740 10 client.27280 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d2e7cd740 10 client.27280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d2e7cd740 20 client.27280 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562783307330 0x5627833097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d2e7cd740 4 client.27280 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.244+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627832630c0 con 0x562783307330 2024-05-09T09:17:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.245+0000 7f4d2905c640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562783307330 0x5627833097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.246+0000 7f4d2905c640 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562783307330 0x5627833097c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accffab3616b10d7 server_cookie=1d02425b0a9c1eb6 in_seq=0 out_seq=0 2024-05-09T09:17:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.246+0000 7f4d0d7fa640 10 client.27280 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.246+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/1375076304 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627832630c0 con 0x562783307330 2024-05-09T09:17:33.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.246+0000 7f4d0d7fa640 10 client.27280 handle_command_reply: tid=0 2024-05-09T09:17:33.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.249+0000 7f4d2e7cd740 1 client.27280 shutdown 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.250+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 msgr2=0x5627832fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.250+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832fb920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d14046040 tx=0x7f4d1403b360 comp rx=0 tx=0).stop 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 shutdown_connections 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562783307330 0x5627833097c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f7020 0x562783302ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x5627832fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/1375076304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832fb920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 >> 172.21.15.64:0/1375076304 conn(0x5627832b90a0 msgr2=0x5627832ec080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d0d7fa640 0 client.27280 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 shutdown_connections 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/1375076304 wait complete. 2024-05-09T09:17:33.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.251+0000 7f4d2e7cd740 20 client.27280 trim_cache size 0 max 16384 2024-05-09T09:17:33.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.304+0000 7f4d2e7cd740 1 Processor -- start 2024-05-09T09:17:33.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.304+0000 7f4d2e7cd740 1 -- start start 2024-05-09T09:17:33.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f1eb0 0x562783302e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 0x5627832ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241b3800 con 0x5627832f7020 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241b10d0 con 0x5627832f1eb0 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241aefd0 con 0x5627832f14c0 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2a05e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 0x5627832ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2a05e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 0x5627832ebf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54118/0 (socket says 172.21.15.64:54118) 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.305+0000 7f4d2a05e640 1 -- 172.21.15.64:0/2990576334 learned_addr learned my addr 172.21.15.64:0/2990576334 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2905c640 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f1eb0 0x562783302e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2a05e640 1 -- 172.21.15.64:0/2990576334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 msgr2=0x5627832f07a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f07a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2a05e640 1 -- 172.21.15.64:0/2990576334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f1eb0 msgr2=0x562783302e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f1eb0 0x562783302e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2a05e640 1 -- 172.21.15.64:0/2990576334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627832eb470 con 0x5627832f7020 2024-05-09T09:17:33.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.306+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 0x5627832ebf00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x5627832f2710 tx=0x7f4d1c010410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc64cc2809b1c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.308+0000 7f4d0f7fe640 1 -- 172.21.15.64:0/2990576334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c04e070 con 0x5627832f7020 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.308+0000 7f4d0f7fe640 1 -- 172.21.15.64:0/2990576334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d1c00ed80 con 0x5627832f7020 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.308+0000 7f4d0f7fe640 1 -- 172.21.15.64:0/2990576334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0057d0 con 0x5627832f7020 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/2990576334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 msgr2=0x5627832ebf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 0x5627832ebf00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x5627832f2710 tx=0x7f4d1c010410 comp rx=0 tx=0).stop 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/2990576334 shutdown_connections 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f7020 0x5627832ebf00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f1eb0 0x562783302e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/2990576334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f14c0 0x5627832f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/2990576334 >> 172.21.15.64:0/2990576334 conn(0x5627832b90a0 msgr2=0x5627832eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/2990576334 shutdown_connections 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.309+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/2990576334 wait complete. 2024-05-09T09:17:33.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 -- start start 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 Processor -- start 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f14c0 0x5627833026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x562783302bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f7020 0x562783300cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241af590 con 0x5627832f1eb0 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241b0a00 con 0x5627832f14c0 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241afc60 con 0x5627832f7020 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.310+0000 7f4d2e7cd740 10 client.0 _ll_register_callbacks cb 0x7f4d20005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2905c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x562783302bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2905c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x562783302bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54132/0 (socket says 172.21.15.64:54132) 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2905c640 1 -- 172.21.15.64:0/812801123 learned_addr learned my addr 172.21.15.64:0/812801123 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f7020 0x562783300cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2985d640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f14c0 0x5627833026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2a05e640 1 -- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f14c0 msgr2=0x5627833026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f14c0 0x5627833026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2a05e640 1 -- 172.21.15.64:0/812801123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 msgr2=0x562783302bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x562783302bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.311+0000 7f4d2a05e640 1 -- 172.21.15.64:0/812801123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627833013b0 con 0x5627832f7020 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.312+0000 7f4d2a05e640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f7020 0x562783300cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c010660 tx=0x7f4d1c004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eefe99705e20b34e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.312+0000 7f4d0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.312+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/812801123 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c057020 con 0x5627832f7020 2024-05-09T09:17:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.312+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/812801123 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4d1c00ead0 con 0x5627832f7020 2024-05-09T09:17:33.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.312+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1c00ed80 con 0x5627832f7020 2024-05-09T09:17:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.317+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/812801123 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4d1c04e040 con 0x5627832f7020 2024-05-09T09:17:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.318+0000 7f4d2e7cd740 10 client.27283 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:33.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.318+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56278336e2e0 con 0x5627832f7020 2024-05-09T09:17:33.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/812801123 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d1c00ed80 con 0x5627832f7020 2024-05-09T09:17:33.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/812801123 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4d1c00f040 con 0x5627832f7020 2024-05-09T09:17:33.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d2e7cd740 10 client.27283 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:33.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d2e7cd740 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:33.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d2e7cd740 20 client.27283 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:33.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56278336ea20 0x562783370e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d2e7cd740 4 client.27283 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:17:33.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.321+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56278325d410 con 0x56278336ea20 2024-05-09T09:17:33.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.323+0000 7f4d2985d640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56278336ea20 0x562783370e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.323+0000 7f4d2985d640 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56278336ea20 0x562783370e40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a852c1f9029297 server_cookie=30d38bb9dda74726 in_seq=0 out_seq=0 2024-05-09T09:17:33.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.323+0000 7f4d0d7fa640 10 client.27283 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.325+0000 7f4d0d7fa640 1 -- 172.21.15.64:0/812801123 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56278325d410 con 0x56278336ea20 2024-05-09T09:17:33.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.325+0000 7f4d0d7fa640 10 client.27283 handle_command_reply: tid=0 2024-05-09T09:17:33.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.325+0000 7f4d2e7cd740 1 client.27283 shutdown 2024-05-09T09:17:33.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f7020 msgr2=0x562783300cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f7020 0x562783300cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c010660 tx=0x7f4d1c004db0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 shutdown_connections 2024-05-09T09:17:33.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56278336ea20 0x562783370e40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627832f7020 0x562783300cc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627832f1eb0 0x562783302bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 --2- 172.21.15.64:0/812801123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5627832f14c0 0x5627833026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 >> 172.21.15.64:0/812801123 conn(0x5627832b90a0 msgr2=0x5627832ec080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.328+0000 7f4d0d7fa640 0 client.27283 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.331+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 shutdown_connections 2024-05-09T09:17:33.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.332+0000 7f4d2e7cd740 1 -- 172.21.15.64:0/812801123 wait complete. 2024-05-09T09:17:33.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.333+0000 7f4d2e7cd740 20 client.27283 trim_cache size 0 max 16384 2024-05-09T09:17:33.332 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:17:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:17:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "469a85a1-f626-48bc-9591-1f8bad8f6528", 2024-05-09T09:17:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:17:33.333 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:17:33.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.337+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d100a2d60 msgr2=0x7f4d100a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.337+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d100a2d60 0x7f4d100a5220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d1403b950 tx=0x7f4d14000bc0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.338+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410db70 msgr2=0x7f4d2419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.338+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410db70 0x7f4d2419e830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d20005fd0 tx=0x7f4d2000ec20 comp rx=0 tx=0).stop 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.339+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 shutdown_connections 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.339+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d100a2d60 0x7f4d100a5220 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.339+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d241a3890 0x7f4d241a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.339+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d2410db70 0x7f4d2419e830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.339+0000 7f4d2bac1640 1 --2- 172.21.15.64:0/4026315692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2410b1a0 0x7f4d2419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.339+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 >> 172.21.15.64:0/4026315692 conn(0x7f4d2406e6c0 msgr2=0x7f4d24109260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.342+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 shutdown_connections 2024-05-09T09:17:33.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.343+0000 7f4d2bac1640 1 -- 172.21.15.64:0/4026315692 wait complete. 2024-05-09T09:17:33.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:17:33.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:17:33.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 -- 172.21.15.64:0/736639027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02810b1a0 msgr2=0x7fa02810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 --2- 172.21.15.64:0/736639027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02810b1a0 0x7fa02810d630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009860 tx=0x7fa01c02a360 comp rx=0 tx=0).stop 2024-05-09T09:17:33.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 -- 172.21.15.64:0/736639027 shutdown_connections 2024-05-09T09:17:33.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 --2- 172.21.15.64:0/736639027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa02810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 --2- 172.21.15.64:0/736639027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02810b1a0 0x7fa02810d630 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 --2- 172.21.15.64:0/736639027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa028108830 0x7fa02810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.613+0000 7fa02e721640 1 -- 172.21.15.64:0/736639027 >> 172.21.15.64:0/736639027 conn(0x7fa02806e6c0 msgr2=0x7fa028070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.614+0000 7fa02e721640 1 -- 172.21.15.64:0/736639027 shutdown_connections 2024-05-09T09:17:33.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.614+0000 7fa02e721640 1 -- 172.21.15.64:0/736639027 wait complete. 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.615+0000 7fa02e721640 1 Processor -- start 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.615+0000 7fa02e721640 1 -- start start 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.615+0000 7fa02e721640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa028108830 0x7fa028115030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02e721640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02810b1a0 0x7fa028115570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02e721640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa0281b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02e721640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa02811c1e0 con 0x7fa02810b1a0 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02e721640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa02811c060 con 0x7fa028108830 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02e721640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa02811c360 con 0x7fa02810db70 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02ccbe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa0281b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02ccbe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa0281b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39054/0 (socket says 172.21.15.64:39054) 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa02ccbe640 1 -- 172.21.15.64:0/7926256 learned_addr learned my addr 172.21.15.64:0/7926256 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa027fff640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa028108830 0x7fa028115030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.616+0000 7fa0277fe640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02810b1a0 0x7fa028115570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.617+0000 7fa02ccbe640 1 -- 172.21.15.64:0/7926256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa028108830 msgr2=0x7fa028115030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.617+0000 7fa02ccbe640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa028108830 0x7fa028115030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.617+0000 7fa02ccbe640 1 -- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02810b1a0 msgr2=0x7fa028115570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.617+0000 7fa02ccbe640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02810b1a0 0x7fa028115570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.617+0000 7fa02ccbe640 1 -- 172.21.15.64:0/7926256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0281b6dd0 con 0x7fa02810db70 2024-05-09T09:17:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.618+0000 7fa02ccbe640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa0281b6890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa02000e9d0 tx=0x7fa02000edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.618+0000 7fa0257fa640 1 -- 172.21.15.64:0/7926256 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020004d20 con 0x7fa02810db70 2024-05-09T09:17:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.618+0000 7fa0257fa640 1 -- 172.21.15.64:0/7926256 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa020005020 con 0x7fa02810db70 2024-05-09T09:17:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.618+0000 7fa0257fa640 1 -- 172.21.15.64:0/7926256 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200116d0 con 0x7fa02810db70 2024-05-09T09:17:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.619+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0281b7060 con 0x7fa02810db70 2024-05-09T09:17:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.619+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0281b75b0 con 0x7fa02810db70 2024-05-09T09:17:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.622+0000 7fa03142a740 1 Processor -- start 2024-05-09T09:17:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.623+0000 7fa0257fa640 1 -- 172.21.15.64:0/7926256 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa020011870 con 0x7fa02810db70 2024-05-09T09:17:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.623+0000 7fa03142a740 1 -- start start 2024-05-09T09:17:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.624+0000 7fa03142a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d63350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.624+0000 7fa03142a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d68420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.624+0000 7fa03142a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.624+0000 7fa03142a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a3f0cc7d00 con 0x55a3f0d63920 2024-05-09T09:17:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.624+0000 7fa03142a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a3f0c6f4c0 con 0x55a3f0d68a90 2024-05-09T09:17:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.624+0000 7fa03142a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a3f0cc3cc0 con 0x55a3f0d62f30 2024-05-09T09:17:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.625+0000 7fa0257fa640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0100a2ce0 0x7fa0100a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.625+0000 7fa0257fa640 1 -- 172.21.15.64:0/7926256 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0200cf3c0 con 0x7fa02810db70 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.626+0000 7fa0277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.626+0000 7fa0277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:42768/0 (socket says 172.21.15.64:42768) 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.626+0000 7fa0277fe640 1 -- 172.21.15.64:0/3297231932 learned_addr learned my addr 172.21.15.64:0/3297231932 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.626+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d63350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.626+0000 7fa027fff640 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d68420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.627+0000 7fa0277fe640 1 -- 172.21.15.64:0/3297231932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 msgr2=0x55a3f0d63350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.627+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d63350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.627+0000 7fa0277fe640 1 -- 172.21.15.64:0/3297231932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 msgr2=0x55a3f0d68420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.627+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d68420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.627+0000 7fa0277fe640 1 -- 172.21.15.64:0/3297231932 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f0aa57d0 con 0x55a3f0d68a90 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.627+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6af20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa01c000c00 tx=0x7fa01c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a492aceea0f9fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.628+0000 7fa02ccbe640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0100a2ce0 0x7fa0100a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.628+0000 7fa00e7fc640 1 -- 172.21.15.64:0/3297231932 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c048070 con 0x55a3f0d68a90 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.628+0000 7fa00e7fc640 1 -- 172.21.15.64:0/3297231932 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa01c038040 con 0x55a3f0d68a90 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.628+0000 7fa00e7fc640 1 -- 172.21.15.64:0/3297231932 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c037070 con 0x55a3f0d68a90 2024-05-09T09:17:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.628+0000 7fa02ccbe640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0100a2ce0 0x7fa0100a51a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa02009c790 tx=0x7fa020004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 -- 172.21.15.64:0/3297231932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 msgr2=0x55a3f0d6af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6af20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa01c000c00 tx=0x7fa01c005f90 comp rx=0 tx=0).stop 2024-05-09T09:17:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 -- 172.21.15.64:0/3297231932 shutdown_connections 2024-05-09T09:17:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6af20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d68420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 --2- 172.21.15.64:0/3297231932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d63350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 -- 172.21.15.64:0/3297231932 >> 172.21.15.64:0/3297231932 conn(0x55a3f0d5e5a0 msgr2=0x55a3f0d5e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 -- 172.21.15.64:0/3297231932 shutdown_connections 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.630+0000 7fa03142a740 1 -- 172.21.15.64:0/3297231932 wait complete. 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.631+0000 7fa03142a740 1 -- start start 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.631+0000 7fa03142a740 1 Processor -- start 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.631+0000 7fa03142a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d6d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.631+0000 7fa03142a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d6d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.631+0000 7fa03142a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d74940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.631+0000 7fa03142a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a3f0d6b9e0 con 0x55a3f0d63920 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa03142a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a3f0c82b90 con 0x55a3f0d68a90 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa03142a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a3f0d6bb60 con 0x55a3f0d62f30 2024-05-09T09:17:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa03142a740 10 client.0 _ll_register_callbacks cb 0x55a3f0cbeda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa0277fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d6d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa0277fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d6d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39074/0 (socket says 172.21.15.64:39074) 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa0277fe640 1 -- 172.21.15.64:0/1711501426 learned_addr learned my addr 172.21.15.64:0/1711501426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d74940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.632+0000 7fa027fff640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d6d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.633+0000 7fa0277fe640 1 -- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 msgr2=0x55a3f0d74940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.633+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d74940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.633+0000 7fa0277fe640 1 -- 172.21.15.64:0/1711501426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 msgr2=0x55a3f0d6d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.633+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d6d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.633+0000 7fa0277fe640 1 -- 172.21.15.64:0/1711501426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f09db3f0 con 0x55a3f0d62f30 2024-05-09T09:17:33.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.635+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d6d3c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0304b0 tx=0x7fa01c030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5850a39bcb098048 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.635+0000 7fa0077fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.636+0000 7fa0077fe640 1 -- 172.21.15.64:0/1711501426 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c048070 con 0x55a3f0d62f30 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.636+0000 7fa0077fe640 1 -- 172.21.15.64:0/1711501426 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa01c032ce0 con 0x55a3f0d62f30 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.636+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa01c038040 con 0x55a3f0d62f30 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.636+0000 7fa0077fe640 1 -- 172.21.15.64:0/1711501426 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa01c037040 con 0x55a3f0d62f30 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.636+0000 7fa03142a740 10 client.27295 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.636+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a3f099b0f0 con 0x55a3f0d62f30 2024-05-09T09:17:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa0077fe640 1 -- 172.21.15.64:0/1711501426 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa01c038040 con 0x55a3f0d62f30 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa0077fe640 1 -- 172.21.15.64:0/1711501426 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa01c03d030 con 0x55a3f0d62f30 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa03142a740 10 client.27295 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa03142a740 10 client.27295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa03142a740 20 client.27295 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa03142a740 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0d790d0 0x55a3f0d7b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa03142a740 4 client.27295 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.637+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f0cd5430 con 0x55a3f0d790d0 2024-05-09T09:17:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.638+0000 7fa027fff640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0d790d0 0x55a3f0d7b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.639+0000 7fa027fff640 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0d790d0 0x55a3f0d7b510 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61cb69e1dbfcac3d server_cookie=ecc4cea926a7fcda in_seq=0 out_seq=0 2024-05-09T09:17:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.639+0000 7fa0077fe640 10 client.27295 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.641+0000 7fa0077fe640 1 -- 172.21.15.64:0/1711501426 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3f0cd5430 con 0x55a3f0d790d0 2024-05-09T09:17:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.641+0000 7fa0077fe640 10 client.27295 handle_command_reply: tid=0 2024-05-09T09:17:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 client.27295 shutdown 2024-05-09T09:17:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 msgr2=0x55a3f0d6d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d6d3c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0304b0 tx=0x7fa01c030bd0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 shutdown_connections 2024-05-09T09:17:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0d790d0 0x55a3f0d7b510 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d68a90 0x55a3f0d74940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 --2- 172.21.15.64:0/1711501426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d62f30 0x55a3f0d6d3c0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0304b0 tx=0x7fa01c030bd0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.642+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 >> 172.21.15.64:0/1711501426 conn(0x55a3f0d5e5a0 msgr2=0x55a3f0d5d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.643+0000 7fa0077fe640 0 client.27295 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.643+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 shutdown_connections 2024-05-09T09:17:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.643+0000 7fa03142a740 1 -- 172.21.15.64:0/1711501426 wait complete. 2024-05-09T09:17:33.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.646+0000 7fa03142a740 20 client.27295 trim_cache size 0 max 16384 2024-05-09T09:17:33.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.688+0000 7fa03142a740 1 Processor -- start 2024-05-09T09:17:33.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.688+0000 7fa03142a740 1 -- start start 2024-05-09T09:17:33.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.688+0000 7fa03142a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 0x55a3f0d7b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa03142a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa03142a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa03142a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa014012480 con 0x55a3f0d68a90 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa03142a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa01400fd50 con 0x55a3f0d5df60 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa03142a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa01400dba0 con 0x55a3f0d63920 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 0x55a3f0d7b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 0x55a3f0d7b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:39090/0 (socket says 172.21.15.64:39090) 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 -- 172.21.15.64:0/1918072240 learned_addr learned my addr 172.21.15.64:0/1918072240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa027fff640 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 -- 172.21.15.64:0/1918072240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 msgr2=0x55a3f0d748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 -- 172.21.15.64:0/1918072240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d68a90 msgr2=0x55a3f0d6d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.689+0000 7fa0277fe640 1 -- 172.21.15.64:0/1918072240 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f0d5e3d0 con 0x55a3f0d63920 2024-05-09T09:17:33.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.690+0000 7fa0277fe640 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 0x55a3f0d7b4b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa01c005820 tx=0x7fa01c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0796b914527a0d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.690+0000 7fa00dffb640 1 -- 172.21.15.64:0/1918072240 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c061030 con 0x55a3f0d63920 2024-05-09T09:17:33.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.690+0000 7fa00dffb640 1 -- 172.21.15.64:0/1918072240 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa01c049040 con 0x55a3f0d63920 2024-05-09T09:17:33.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.690+0000 7fa00dffb640 1 -- 172.21.15.64:0/1918072240 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c048040 con 0x55a3f0d63920 2024-05-09T09:17:33.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.692+0000 7fa03142a740 1 -- 172.21.15.64:0/1918072240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 msgr2=0x55a3f0d7b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.692+0000 7fa03142a740 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 0x55a3f0d7b4b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa01c005820 tx=0x7fa01c005850 comp rx=0 tx=0).stop 2024-05-09T09:17:33.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.693+0000 7fa03142a740 1 -- 172.21.15.64:0/1918072240 shutdown_connections 2024-05-09T09:17:33.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.693+0000 7fa03142a740 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.693+0000 7fa03142a740 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d68a90 0x55a3f0d6d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.693+0000 7fa03142a740 1 --2- 172.21.15.64:0/1918072240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d63920 0x55a3f0d7b4b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.693+0000 7fa03142a740 1 -- 172.21.15.64:0/1918072240 >> 172.21.15.64:0/1918072240 conn(0x55a3f0d5e5a0 msgr2=0x55a3f0d614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.694+0000 7fa03142a740 1 -- 172.21.15.64:0/1918072240 shutdown_connections 2024-05-09T09:17:33.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.694+0000 7fa03142a740 1 -- 172.21.15.64:0/1918072240 wait complete. 2024-05-09T09:17:33.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.694+0000 7fa03142a740 1 -- start start 2024-05-09T09:17:33.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 Processor -- start 2024-05-09T09:17:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d71830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d71d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d68a90 0x55a3f0ddd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa01400e210 con 0x55a3f0d63920 2024-05-09T09:17:33.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa01400f680 con 0x55a3f0d5df60 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa01400e8e0 con 0x55a3f0d68a90 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.695+0000 7fa03142a740 10 client.0 _ll_register_callbacks cb 0x7fa01c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d71d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d71d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54174/0 (socket says 172.21.15.64:54174) 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 -- 172.21.15.64:0/3407821532 learned_addr learned my addr 172.21.15.64:0/3407821532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d68a90 0x55a3f0ddd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d71830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 -- 172.21.15.64:0/3407821532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d68a90 msgr2=0x55a3f0ddd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d68a90 0x55a3f0ddd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 -- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 msgr2=0x55a3f0d71830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d71830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.696+0000 7fa027fff640 1 -- 172.21.15.64:0/3407821532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f0dddc10 con 0x55a3f0d63920 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.697+0000 7fa027fff640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d71d70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa018004700 tx=0x7fa018020410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abf6173ea314717 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:33.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.697+0000 7fa0077fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:17:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.698+0000 7fa0077fe640 1 -- 172.21.15.64:0/3407821532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018005220 con 0x55a3f0d63920 2024-05-09T09:17:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.698+0000 7fa0077fe640 1 -- 172.21.15.64:0/3407821532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0180053c0 con 0x55a3f0d63920 2024-05-09T09:17:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.698+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa01c049040 con 0x55a3f0d63920 2024-05-09T09:17:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.699+0000 7fa0077fe640 1 -- 172.21.15.64:0/3407821532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa0180055e0 con 0x55a3f0d63920 2024-05-09T09:17:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.699+0000 7fa03142a740 10 client.17643 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:33.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.699+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a3f0ddef40 con 0x55a3f0d63920 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.699+0000 7fa0077fe640 1 -- 172.21.15.64:0/3407821532 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa01c049040 con 0x55a3f0d63920 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa0077fe640 1 -- 172.21.15.64:0/3407821532 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa018004bd0 con 0x55a3f0d63920 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa03142a740 10 client.17643 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa03142a740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa03142a740 20 client.17643 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa03142a740 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0ddf880 0x55a3f0de1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa03142a740 4 client.17643 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:17:33.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.700+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3f0ccf780 con 0x55a3f0ddf880 2024-05-09T09:17:33.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.701+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0ddf880 0x55a3f0de1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:33.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.702+0000 7fa0277fe640 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0ddf880 0x55a3f0de1ca0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c22a760ddad70bd server_cookie=f554c079490220c in_seq=0 out_seq=0 2024-05-09T09:17:33.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.702+0000 7fa0077fe640 10 client.17643 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa0077fe640 1 -- 172.21.15.64:0/3407821532 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a3f0ccf780 con 0x55a3f0ddf880 2024-05-09T09:17:33.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa0077fe640 10 client.17643 handle_command_reply: tid=0 2024-05-09T09:17:33.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa03142a740 1 client.17643 shutdown 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 msgr2=0x55a3f0d71d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa03142a740 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d71d70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa018004700 tx=0x7fa018020410 comp rx=0 tx=0).stop 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 shutdown_connections 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa03142a740 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a3f0ddf880 0x55a3f0de1ca0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.703+0000 7fa03142a740 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a3f0d68a90 0x55a3f0ddd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa03142a740 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3f0d63920 0x55a3f0d71d70 secure :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa018004700 tx=0x7fa018020410 comp rx=0 tx=0).stop 2024-05-09T09:17:33.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa03142a740 1 --2- 172.21.15.64:0/3407821532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a3f0d5df60 0x55a3f0d71830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 >> 172.21.15.64:0/3407821532 conn(0x55a3f0d5e5a0 msgr2=0x55a3f0d6d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa0077fe640 0 client.17643 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:33.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 shutdown_connections 2024-05-09T09:17:33.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa03142a740 1 -- 172.21.15.64:0/3407821532 wait complete. 2024-05-09T09:17:33.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.704+0000 7fa03142a740 20 client.17643 trim_cache size 0 max 16384 2024-05-09T09:17:33.704 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "469a85a1-f626-48bc-9591-1f8bad8f6528": { 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "469a85a1-f626-48bc-9591-1f8bad8f6528", 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:17:33.705 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:17:33.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.706+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0100a2ce0 msgr2=0x7fa0100a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.706+0000 7fa02e721640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0100a2ce0 0x7fa0100a51a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa02009c790 tx=0x7fa020004a40 comp rx=0 tx=0).stop 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.706+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 msgr2=0x7fa0281b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.706+0000 7fa02e721640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa0281b6890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa02000e9d0 tx=0x7fa02000edd0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 shutdown_connections 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0100a2ce0 0x7fa0100a51a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa02810db70 0x7fa0281b6890 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02810b1a0 0x7fa028115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 --2- 172.21.15.64:0/7926256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa028108830 0x7fa028115030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 >> 172.21.15.64:0/7926256 conn(0x7fa02806e6c0 msgr2=0x7fa028108c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 shutdown_connections 2024-05-09T09:17:33.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:17:33.707+0000 7fa02e721640 1 -- 172.21.15.64:0/7926256 wait complete. 2024-05-09T09:17:33.728 INFO:tasks.cephfs.filesystem:scrub status for tag:469a85a1-f626-48bc-9591-1f8bad8f6528 - {'path': '/', 'tag': '469a85a1-f626-48bc-9591-1f8bad8f6528', 'options': 'recursive,force'} 2024-05-09T09:17:34.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:33 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:17:34.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:33 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:33 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:17:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:33 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:33 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:17:34.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:33 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 925 op/s 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:34 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:17:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:17:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 925 op/s 2024-05-09T09:17:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:17:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:34 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: pgmap v623: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 925 op/s 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:34 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:17:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 435.0 s, 764.1 tps, lat 131.102 ms stddev 42.695 2024-05-09T09:17:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:36 smithi064 ceph-mon[30149]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.2 MiB/s wr, 962 op/s 2024-05-09T09:17:37.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:36 smithi179 ceph-mon[34086]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.2 MiB/s wr, 962 op/s 2024-05-09T09:17:37.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:36 smithi139 ceph-mon[33599]: pgmap v624: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.2 MiB/s wr, 962 op/s 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.056+0000 7fa6ac693640 1 -- 172.21.15.64:0/3345004437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4109050 msgr2=0x7fa6a41094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.056+0000 7fa6ac693640 1 --2- 172.21.15.64:0/3345004437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4109050 0x7fa6a41094d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009be0 tx=0x7fa69c02a6d0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.057+0000 7fa6ac693640 1 -- 172.21.15.64:0/3345004437 shutdown_connections 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.057+0000 7fa6ac693640 1 --2- 172.21.15.64:0/3345004437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a4109a10 0x7fa6a4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.057+0000 7fa6ac693640 1 --2- 172.21.15.64:0/3345004437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4109050 0x7fa6a41094d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.057+0000 7fa6ac693640 1 --2- 172.21.15.64:0/3345004437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a4107e70 0x7fa6a4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.057+0000 7fa6ac693640 1 -- 172.21.15.64:0/3345004437 >> 172.21.15.64:0/3345004437 conn(0x7fa6a4076e80 msgr2=0x7fa6a40792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.058+0000 7fa6ac693640 1 -- 172.21.15.64:0/3345004437 shutdown_connections 2024-05-09T09:17:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.060+0000 7fa6ac693640 1 -- 172.21.15.64:0/3345004437 wait complete. 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.061+0000 7fa6ac693640 1 Processor -- start 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.061+0000 7fa6ac693640 1 -- start start 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.061+0000 7fa6ac693640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 0x7fa6a406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6ac693640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a4109050 0x7fa6a40704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 0x7fa6a406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6ac693640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a4109a10 0x7fa6a41ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6ac693640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4113a00 con 0x7fa6a4109050 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6ac693640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4113880 con 0x7fa6a4107e70 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6ac693640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4113b80 con 0x7fa6a4109a10 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 0x7fa6a406ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51484/0 (socket says 172.21.15.64:51484) 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 -- 172.21.15.64:0/206984706 learned_addr learned my addr 172.21.15.64:0/206984706 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 -- 172.21.15.64:0/206984706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a4109a10 msgr2=0x7fa6a41ae110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a4109a10 0x7fa6a41ae110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 -- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a4109050 msgr2=0x7fa6a40704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a4109050 0x7fa6a40704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.062+0000 7fa6aa42f640 1 -- 172.21.15.64:0/206984706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a4073660 con 0x7fa6a4107e70 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.063+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 0x7fa6a406ffa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d770 tx=0x7fa69400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.063+0000 7fa69b7fe640 1 -- 172.21.15.64:0/206984706 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694017070 con 0x7fa6a4107e70 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.063+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a407a080 con 0x7fa6a4107e70 2024-05-09T09:17:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.064+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a407a5d0 con 0x7fa6a4107e70 2024-05-09T09:17:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.064+0000 7fa69b7fe640 1 -- 172.21.15.64:0/206984706 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa694009c30 con 0x7fa6a4107e70 2024-05-09T09:17:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.067+0000 7fa6af39e740 1 Processor -- start 2024-05-09T09:17:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.068+0000 7fa69b7fe640 1 -- 172.21.15.64:0/206984706 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694003d00 con 0x7fa6a4107e70 2024-05-09T09:17:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.069+0000 7fa6af39e740 1 -- start start 2024-05-09T09:17:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214d250 0x556eb214d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb2152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb2152db0 0x556eb2155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6af39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556eb20b1340 con 0x556eb214d250 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6af39e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556eb2059290 con 0x556eb214dc40 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6af39e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556eb20ad300 con 0x556eb2152db0 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa69b7fe640 1 -- 172.21.15.64:0/206984706 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa694010460 con 0x7fa6a4107e70 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6aa42f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb2152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6aa42f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb2152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51498/0 (socket says 172.21.15.64:51498) 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6aa42f640 1 -- 172.21.15.64:0/1027760172 learned_addr learned my addr 172.21.15.64:0/1027760172 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.070+0000 7fa6a9c2e640 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb2152db0 0x556eb2155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.071+0000 7fa6aa42f640 1 -- 172.21.15.64:0/1027760172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb2152db0 msgr2=0x556eb2155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.071+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb2152db0 0x556eb2155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.071+0000 7fa6aa42f640 1 -- 172.21.15.64:0/1027760172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214d250 msgr2=0x556eb214d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.071+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214d250 0x556eb214d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.071+0000 7fa6aa42f640 1 -- 172.21.15.64:0/1027760172 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb207d8e0 con 0x556eb214dc40 2024-05-09T09:17:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.071+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb2152740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6940d2ec0 tx=0x7fa6940d7700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5405a76d2e7d282 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.072+0000 7fa698ff9640 1 -- 172.21.15.64:0/1027760172 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940d9820 con 0x556eb214dc40 2024-05-09T09:17:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.072+0000 7fa698ff9640 1 -- 172.21.15.64:0/1027760172 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6940e1400 con 0x556eb214dc40 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.073+0000 7fa69b7fe640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6900a2cf0 0x7fa6900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.073+0000 7fa698ff9640 1 -- 172.21.15.64:0/1027760172 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940d55f0 con 0x556eb214dc40 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.073+0000 7fa69b7fe640 1 -- 172.21.15.64:0/206984706 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa6940ced20 con 0x7fa6a4107e70 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.073+0000 7fa6aac30640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6900a2cf0 0x7fa6900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 -- 172.21.15.64:0/1027760172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 msgr2=0x556eb2152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb2152740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6940d2ec0 tx=0x7fa6940d7700 comp rx=0 tx=0).stop 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 -- 172.21.15.64:0/1027760172 shutdown_connections 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb2152db0 0x556eb2155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb2152740 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 --2- 172.21.15.64:0/1027760172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214d250 0x556eb214d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6af39e740 1 -- 172.21.15.64:0/1027760172 >> 172.21.15.64:0/1027760172 conn(0x556eb21488c0 msgr2=0x556eb2148cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.074+0000 7fa6aac30640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6900a2cf0 0x7fa6900a51b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00058c0 tx=0x7fa6a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.075+0000 7fa6af39e740 1 -- 172.21.15.64:0/1027760172 shutdown_connections 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.075+0000 7fa6af39e740 1 -- 172.21.15.64:0/1027760172 wait complete. 2024-05-09T09:17:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.075+0000 7fa6af39e740 1 -- start start 2024-05-09T09:17:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.075+0000 7fa6af39e740 1 Processor -- start 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb21575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb2157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb215eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556eb2155db0 con 0x556eb214dc40 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556eb206c790 con 0x556eb2152db0 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556eb2155f30 con 0x556eb214d250 2024-05-09T09:17:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6a9c2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb2157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6a9c2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb2157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44420/0 (socket says 172.21.15.64:44420) 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6a9c2e640 1 -- 172.21.15.64:0/793819135 learned_addr learned my addr 172.21.15.64:0/793819135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6af39e740 10 client.0 _ll_register_callbacks cb 0x556eb20a7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.076+0000 7fa6aac30640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb215eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb21575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 -- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 msgr2=0x556eb215eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb215eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 -- 172.21.15.64:0/793819135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 msgr2=0x556eb2157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb2157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 -- 172.21.15.64:0/793819135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb1dc5210 con 0x556eb214d250 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb21575a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d740 tx=0x7fa6940d7cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e536ad17400edb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa68dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa68dffb640 1 -- 172.21.15.64:0/793819135 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694020050 con 0x556eb214d250 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.077+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6940e1400 con 0x556eb214d250 2024-05-09T09:17:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.078+0000 7fa68dffb640 1 -- 172.21.15.64:0/793819135 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa6940d29d0 con 0x556eb214d250 2024-05-09T09:17:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.078+0000 7fa68dffb640 1 -- 172.21.15.64:0/793819135 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa694097490 con 0x556eb214d250 2024-05-09T09:17:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.080+0000 7fa6af39e740 10 client.27307 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.080+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556eb1e91620 con 0x556eb214d250 2024-05-09T09:17:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.082+0000 7fa68dffb640 1 -- 172.21.15.64:0/793819135 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6940e1400 con 0x556eb214d250 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.082+0000 7fa68dffb640 1 -- 172.21.15.64:0/793819135 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa694097860 con 0x556eb214d250 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.083+0000 7fa6af39e740 10 client.27307 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.083+0000 7fa6af39e740 10 client.27307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.083+0000 7fa6af39e740 20 client.27307 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.083+0000 7fa6af39e740 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb2162c60 0x556eb2165050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.083+0000 7fa6af39e740 4 client.27307 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.083+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb20bea70 con 0x556eb2162c60 2024-05-09T09:17:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.085+0000 7fa6a9c2e640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb2162c60 0x556eb2165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.093+0000 7fa68dffb640 10 client.27307 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.093+0000 7fa6a9c2e640 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb2162c60 0x556eb2165050 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20867e7439c433a server_cookie=73b0cc71e9bb2dde in_seq=0 out_seq=0 2024-05-09T09:17:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.094+0000 7fa68dffb640 1 -- 172.21.15.64:0/793819135 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556eb20bea70 con 0x556eb2162c60 2024-05-09T09:17:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.094+0000 7fa68dffb640 10 client.27307 handle_command_reply: tid=0 2024-05-09T09:17:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.094+0000 7fa6af39e740 1 client.27307 shutdown 2024-05-09T09:17:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 msgr2=0x556eb21575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb21575a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d740 tx=0x7fa6940d7cb0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 shutdown_connections 2024-05-09T09:17:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb2162c60 0x556eb2165050 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb215eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb2157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 --2- 172.21.15.64:0/793819135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb21575a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.099+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 >> 172.21.15.64:0/793819135 conn(0x556eb21488c0 msgr2=0x556eb2147a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.102+0000 7fa68dffb640 0 client.27307 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.104+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 shutdown_connections 2024-05-09T09:17:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.104+0000 7fa6af39e740 1 -- 172.21.15.64:0/793819135 wait complete. 2024-05-09T09:17:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.104+0000 7fa6af39e740 20 client.27307 trim_cache size 0 max 16384 2024-05-09T09:17:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.136+0000 7fa6af39e740 1 Processor -- start 2024-05-09T09:17:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 -- start start 2024-05-09T09:17:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb2157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb215eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb214bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa688012480 con 0x556eb214dc40 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa68800fd50 con 0x556eb2152db0 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6af39e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa68800dba0 con 0x556eb214d250 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6aac30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb214bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6aac30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb214bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51516/0 (socket says 172.21.15.64:51516) 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.137+0000 7fa6aac30640 1 -- 172.21.15.64:0/3624486544 learned_addr learned my addr 172.21.15.64:0/3624486544 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa6aac30640 1 -- 172.21.15.64:0/3624486544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 msgr2=0x556eb2157a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:17:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa6aac30640 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb2157a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa6aac30640 1 -- 172.21.15.64:0/3624486544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 msgr2=0x556eb215eae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:17:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa6aac30640 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb215eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa6aac30640 1 -- 172.21.15.64:0/3624486544 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb2146f00 con 0x556eb2152db0 2024-05-09T09:17:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa6aac30640 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb214bd50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000ee70 tx=0x7fa6a0011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bddec3f7e0af5da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.138+0000 7fa68ffff640 1 -- 172.21.15.64:0/3624486544 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004730 con 0x556eb2152db0 2024-05-09T09:17:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.139+0000 7fa68ffff640 1 -- 172.21.15.64:0/3624486544 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6a0022070 con 0x556eb2152db0 2024-05-09T09:17:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa68ffff640 1 -- 172.21.15.64:0/3624486544 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0010d80 con 0x556eb2152db0 2024-05-09T09:17:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 -- 172.21.15.64:0/3624486544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 msgr2=0x556eb214bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb214bd50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000ee70 tx=0x7fa6a0011000 comp rx=0 tx=0).stop 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 -- 172.21.15.64:0/3624486544 shutdown_connections 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb2152db0 0x556eb214bd50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb214dc40 0x556eb215eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3624486544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb2157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.140+0000 7fa6af39e740 1 -- 172.21.15.64:0/3624486544 >> 172.21.15.64:0/3624486544 conn(0x556eb21488c0 msgr2=0x556eb214b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.141+0000 7fa6af39e740 1 -- 172.21.15.64:0/3624486544 shutdown_connections 2024-05-09T09:17:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.141+0000 7fa6af39e740 1 -- 172.21.15.64:0/3624486544 wait complete. 2024-05-09T09:17:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.141+0000 7fa6af39e740 1 -- start start 2024-05-09T09:17:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.141+0000 7fa6af39e740 1 Processor -- start 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb215ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb215cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb2152db0 0x556eb215dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa68800e210 con 0x556eb2152db0 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa68800f680 con 0x556eb214dc40 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa68800e8e0 con 0x556eb214d250 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6af39e740 10 client.0 _ll_register_callbacks cb 0x7fa6940d2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6aa42f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb215ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6aa42f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb215ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:41348/0 (socket says 172.21.15.64:41348) 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.142+0000 7fa6aa42f640 1 -- 172.21.15.64:0/3920961335 learned_addr learned my addr 172.21.15.64:0/3920961335 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6a9c2e640 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb215cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6aa42f640 1 -- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 msgr2=0x556eb215cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb215cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6aa42f640 1 -- 172.21.15.64:0/3920961335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb2152db0 msgr2=0x556eb215dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb2152db0 0x556eb215dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6aa42f640 1 -- 172.21.15.64:0/3920961335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb215e5a0 con 0x556eb214d250 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa6aa42f640 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb215ca60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa69400dfd0 tx=0x7fa69400df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4bf93de484896a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa68dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.143+0000 7fa68dffb640 1 -- 172.21.15.64:0/3920961335 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694020050 con 0x556eb214d250 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.144+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a0022070 con 0x556eb214d250 2024-05-09T09:17:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.144+0000 7fa68dffb640 1 -- 172.21.15.64:0/3920961335 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa694097860 con 0x556eb214d250 2024-05-09T09:17:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.144+0000 7fa68dffb640 1 -- 172.21.15.64:0/3920961335 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa69409ab50 con 0x556eb214d250 2024-05-09T09:17:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 10 client.27310 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556eb21c9720 con 0x556eb214d250 2024-05-09T09:17:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa68dffb640 1 -- 172.21.15.64:0/3920961335 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6a0022070 con 0x556eb214d250 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa68dffb640 1 -- 172.21.15.64:0/3920961335 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa6940bb0b0 con 0x556eb214d250 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 10 client.27310 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 10 client.27310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 20 client.27310 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb21c9ea0 0x556eb21cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 4 client.27310 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.145+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556eb20b8dc0 con 0x556eb21c9ea0 2024-05-09T09:17:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.146+0000 7fa6a9c2e640 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb21c9ea0 0x556eb21cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.147+0000 7fa68dffb640 10 client.27310 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.148+0000 7fa6a9c2e640 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb21c9ea0 0x556eb21cc2c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed096f745901125 server_cookie=f3de76ca551bc252 in_seq=0 out_seq=0 2024-05-09T09:17:38.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:37 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.181+0000 7fa68dffb640 1 -- 172.21.15.64:0/3920961335 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556eb20b8dc0 con 0x556eb21c9ea0 2024-05-09T09:17:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.181+0000 7fa68dffb640 10 client.27310 handle_command_reply: tid=0 2024-05-09T09:17:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.182+0000 7fa6af39e740 1 client.27310 shutdown 2024-05-09T09:17:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.182+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 msgr2=0x556eb215ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.182+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb215ca60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa69400dfd0 tx=0x7fa69400df00 comp rx=0 tx=0).stop 2024-05-09T09:17:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 shutdown_connections 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556eb21c9ea0 0x556eb21cc2c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556eb2152db0 0x556eb215dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556eb214dc40 0x556eb215cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 --2- 172.21.15.64:0/3920961335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556eb214d250 0x556eb215ca60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 >> 172.21.15.64:0/3920961335 conn(0x556eb21488c0 msgr2=0x556eb214bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa68dffb640 0 client.27310 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 shutdown_connections 2024-05-09T09:17:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.183+0000 7fa6af39e740 1 -- 172.21.15.64:0/3920961335 wait complete. 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.184+0000 7fa6af39e740 20 client.27310 trim_cache size 0 max 16384 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.185+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6900a2cf0 msgr2=0x7fa6900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.185+0000 7fa6ac693640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6900a2cf0 0x7fa6900a51b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00058c0 tx=0x7fa6a0005850 comp rx=0 tx=0).stop 2024-05-09T09:17:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.185+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 msgr2=0x7fa6a406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.185+0000 7fa6ac693640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 0x7fa6a406ffa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d770 tx=0x7fa69400dc30 comp rx=0 tx=0).stop 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.186+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 shutdown_connections 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.186+0000 7fa6ac693640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa6900a2cf0 0x7fa6900a51b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.186+0000 7fa6ac693640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a4109a10 0x7fa6a41ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.186+0000 7fa6ac693640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6a4109050 0x7fa6a40704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.186+0000 7fa6ac693640 1 --2- 172.21.15.64:0/206984706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6a4107e70 0x7fa6a406ffa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.186+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 >> 172.21.15.64:0/206984706 conn(0x7fa6a4076e80 msgr2=0x7fa6a410c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.187+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 shutdown_connections 2024-05-09T09:17:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:38.187+0000 7fa6ac693640 1 -- 172.21.15.64:0/206984706 wait complete. 2024-05-09T09:17:38.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:37 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:38.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:37 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:17:39.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:38 smithi064 ceph-mon[30149]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-05-09T09:17:39.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:39.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:39.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:17:39.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:38 smithi179 ceph-mon[34086]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-05-09T09:17:39.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:39.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:39.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:17:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:38 smithi139 ceph-mon[33599]: pgmap v625: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-05-09T09:17:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:17:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:17:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:17:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:40 smithi139 ceph-mon[33599]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.4 MiB/s wr, 1.11k op/s 2024-05-09T09:17:40.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:40 smithi064 ceph-mon[30149]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.4 MiB/s wr, 1.11k op/s 2024-05-09T09:17:40.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:40 smithi179 ceph-mon[34086]: pgmap v626: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.4 MiB/s wr, 1.11k op/s 2024-05-09T09:17:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 440.0 s, 611.5 tps, lat 162.475 ms stddev 66.681 2024-05-09T09:17:42.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:42 smithi064 ceph-mon[30149]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.0 MiB/s wr, 913 op/s 2024-05-09T09:17:42.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:42 smithi179 ceph-mon[34086]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.0 MiB/s wr, 913 op/s 2024-05-09T09:17:42.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:42 smithi139 ceph-mon[33599]: pgmap v627: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.0 MiB/s wr, 913 op/s 2024-05-09T09:17:44.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:44 smithi064 ceph-mon[30149]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.4 MiB/s wr, 981 op/s 2024-05-09T09:17:44.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:44 smithi179 ceph-mon[34086]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.4 MiB/s wr, 981 op/s 2024-05-09T09:17:44.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:44 smithi139 ceph-mon[33599]: pgmap v628: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.4 MiB/s wr, 981 op/s 2024-05-09T09:17:45.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:45 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:17:45.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:45 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:17:45.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:45 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:17:45.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:45 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:17:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:45 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:17:45.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:45 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:17:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 445.0 s, 732.0 tps, lat 137.123 ms stddev 78.793 2024-05-09T09:17:46.731 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:46 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:17:46.731 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:46 smithi179 ceph-mon[34086]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.2 MiB/s wr, 941 op/s 2024-05-09T09:17:46.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:46 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:17:46.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:46 smithi139 ceph-mon[33599]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.2 MiB/s wr, 941 op/s 2024-05-09T09:17:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:46 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:17:46.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:46 smithi064 ceph-mon[30149]: pgmap v629: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.2 MiB/s wr, 941 op/s 2024-05-09T09:17:48.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:48 smithi179 ceph-mon[34086]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.0 MiB/s wr, 779 op/s 2024-05-09T09:17:48.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:48 smithi139 ceph-mon[33599]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.0 MiB/s wr, 779 op/s 2024-05-09T09:17:48.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:48 smithi064 ceph-mon[30149]: pgmap v630: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.0 MiB/s wr, 779 op/s 2024-05-09T09:17:50.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:50 smithi064 ceph-mon[30149]: pgmap v631: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2024-05-09T09:17:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:50 smithi179 ceph-mon[34086]: pgmap v631: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2024-05-09T09:17:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:50 smithi139 ceph-mon[33599]: pgmap v631: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2024-05-09T09:17:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.349+0000 7f5c9d7db640 1 -- 172.21.15.64:0/1856066048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 msgr2=0x7f5c9009da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.349+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/1856066048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 0x7f5c9009da10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009900 tx=0x7f5c8c02a410 comp rx=0 tx=0).stop 2024-05-09T09:17:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 -- 172.21.15.64:0/1856066048 shutdown_connections 2024-05-09T09:17:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/1856066048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009df50 0x7f5c900a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/1856066048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 0x7f5c9009da10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/1856066048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009c3f0 0x7f5c9009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 -- 172.21.15.64:0/1856066048 >> 172.21.15.64:0/1856066048 conn(0x7f5c9000b340 msgr2=0x7f5c9000b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 -- 172.21.15.64:0/1856066048 shutdown_connections 2024-05-09T09:17:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.351+0000 7f5c9d7db640 1 -- 172.21.15.64:0/1856066048 wait complete. 2024-05-09T09:17:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.352+0000 7f5c9d7db640 1 Processor -- start 2024-05-09T09:17:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.353+0000 7f5c9d7db640 1 -- start start 2024-05-09T09:17:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 450.0 s, 714.8 tps, lat 139.780 ms stddev 74.420 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.353+0000 7f5c9d7db640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009c3f0 0x7f5c9012a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.353+0000 7f5c9d7db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 0x7f5c9012a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.353+0000 7f5c9d7db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 0x7f5c90131380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.353+0000 7f5c9d7db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000fe30 con 0x7f5c9009d590 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.353+0000 7f5c9d7db640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000fcb0 con 0x7f5c9009df50 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c9d7db640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000ffb0 con 0x7f5c9009c3f0 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 0x7f5c90131380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 0x7f5c90131380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:45582/0 (socket says 172.21.15.64:45582) 2024-05-09T09:17:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 -- 172.21.15.64:0/2659171799 learned_addr learned my addr 172.21.15.64:0/2659171799 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009c3f0 0x7f5c9012a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 -- 172.21.15.64:0/2659171799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009c3f0 msgr2=0x7f5c9012a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009c3f0 0x7f5c9012a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 -- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 msgr2=0x7f5c9012a8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 0x7f5c9012a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.354+0000 7f5c977fe640 1 -- 172.21.15.64:0/2659171799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c901319f0 con 0x7f5c9009df50 2024-05-09T09:17:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.355+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009c3f0 0x7f5c9012a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:17:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.361+0000 7f5c977fe640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 0x7f5c90131380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800e9b0 tx=0x7f5c8800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.362+0000 7f5c7ffff640 1 -- 172.21.15.64:0/2659171799 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800cd80 con 0x7f5c9009df50 2024-05-09T09:17:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.362+0000 7f5c7ffff640 1 -- 172.21.15.64:0/2659171799 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c880035f0 con 0x7f5c9009df50 2024-05-09T09:17:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.362+0000 7f5c7ffff640 1 -- 172.21.15.64:0/2659171799 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88010690 con 0x7f5c9009df50 2024-05-09T09:17:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.362+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c90131ce0 con 0x7f5c9009df50 2024-05-09T09:17:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.362+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c90132180 con 0x7f5c9009df50 2024-05-09T09:17:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.366+0000 7f5c7ffff640 1 -- 172.21.15.64:0/2659171799 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c88010830 con 0x7f5c9009df50 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.367+0000 7f5ca05bd740 1 Processor -- start 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.369+0000 7f5c7ffff640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5c800a2d90 0x7f5c800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.369+0000 7f5c7ffff640 1 -- 172.21.15.64:0/2659171799 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c880cf790 con 0x7f5c9009df50 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 -- start start 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e35820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e3a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e3d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5585b4d99340 con 0x5585b4e35df0 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5585b4d41290 con 0x5585b4e3af60 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5ca05bd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5585b4d95300 con 0x5585b4e35400 2024-05-09T09:17:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5c967fc640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5c800a2d90 0x7f5c800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e35820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e35820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42132/0 (socket says 172.21.15.64:42132) 2024-05-09T09:17:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.370+0000 7f5c96ffd640 1 -- 172.21.15.64:0/1375538347 learned_addr learned my addr 172.21.15.64:0/1375538347 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c977fe640 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e3d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c967fc640 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e3a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c96ffd640 1 -- 172.21.15.64:0/1375538347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 msgr2=0x5585b4e3d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e3d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c96ffd640 1 -- 172.21.15.64:0/1375538347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 msgr2=0x5585b4e3a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e3a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c96ffd640 1 -- 172.21.15.64:0/1375538347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b4d658e0 con 0x5585b4e35400 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.371+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e35820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c98050ef0 tx=0x7f5c9808f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cdb115971363ccf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.372+0000 7f5c967fc640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5c800a2d90 0x7f5c800a5250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009230 tx=0x7f5c8c004040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.372+0000 7f5c7d7fa640 1 -- 172.21.15.64:0/1375538347 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98073e60 con 0x5585b4e35400 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.372+0000 7f5c7d7fa640 1 -- 172.21.15.64:0/1375538347 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c98094030 con 0x5585b4e35400 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.373+0000 7f5c7d7fa640 1 -- 172.21.15.64:0/1375538347 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98098790 con 0x5585b4e35400 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.373+0000 7f5ca05bd740 1 -- 172.21.15.64:0/1375538347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 msgr2=0x5585b4e35820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.373+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e35820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c98050ef0 tx=0x7f5c9808f7b0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.374+0000 7f5ca05bd740 1 -- 172.21.15.64:0/1375538347 shutdown_connections 2024-05-09T09:17:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.374+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e3d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.374+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e3a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.374+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/1375538347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e35820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.374+0000 7f5ca05bd740 1 -- 172.21.15.64:0/1375538347 >> 172.21.15.64:0/1375538347 conn(0x5585b4e30a70 msgr2=0x5585b4e30e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.377+0000 7f5ca05bd740 1 -- 172.21.15.64:0/1375538347 shutdown_connections 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.377+0000 7f5ca05bd740 1 -- 172.21.15.64:0/1375538347 wait complete. 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.377+0000 7f5ca05bd740 1 -- start start 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.377+0000 7f5ca05bd740 1 Processor -- start 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35df0 0x5585b4e3fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 0x5585b4e46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5585b4e3df10 con 0x5585b4e35400 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5585b4d54790 con 0x5585b4e35df0 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5585b4e3e090 con 0x5585b4e3af60 2024-05-09T09:17:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5ca05bd740 10 client.0 _ll_register_callbacks cb 0x5585b4d8fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5c967fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35df0 0x5585b4e3fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 0x5585b4e46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5c967fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35df0 0x5585b4e3fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:45600/0 (socket says 172.21.15.64:45600) 2024-05-09T09:17:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 0x5585b4e46ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42136/0 (socket says 172.21.15.64:42136) 2024-05-09T09:17:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.378+0000 7f5c967fc640 1 -- 172.21.15.64:0/43287556 learned_addr learned my addr 172.21.15.64:0/43287556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c977fe640 1 -- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35df0 msgr2=0x5585b4e3fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c977fe640 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35df0 0x5585b4e3fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c977fe640 1 -- 172.21.15.64:0/43287556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35400 msgr2=0x5585b4e3f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c977fe640 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c977fe640 1 -- 172.21.15.64:0/43287556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b4aad210 con 0x5585b4e3af60 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c977fe640 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 0x5585b4e46ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5c880ccea0 tx=0x7f5c880095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5624ee524ca27a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.379+0000 7f5c7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.380+0000 7f5c7aff5640 1 -- 172.21.15.64:0/43287556 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8809ffb0 con 0x5585b4e3af60 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.380+0000 7f5c7aff5640 1 -- 172.21.15.64:0/43287556 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c880ba580 con 0x5585b4e3af60 2024-05-09T09:17:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.380+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c98094030 con 0x5585b4e3af60 2024-05-09T09:17:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.381+0000 7f5c7aff5640 1 -- 172.21.15.64:0/43287556 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5c880a4870 con 0x5585b4e3af60 2024-05-09T09:17:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.381+0000 7f5ca05bd740 10 client.27322 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.381+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5585b4b79620 con 0x5585b4e3af60 2024-05-09T09:17:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.381+0000 7f5c7aff5640 1 -- 172.21.15.64:0/43287556 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c98094030 con 0x5585b4e3af60 2024-05-09T09:17:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5c7aff5640 1 -- 172.21.15.64:0/43287556 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f5c880cf3e0 con 0x5585b4e3af60 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5ca05bd740 10 client.27322 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5ca05bd740 10 client.27322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5ca05bd740 20 client.27322 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4e5c1c0 0x5585b4e5e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5ca05bd740 4 client.27322 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585b4da6a70 con 0x5585b4e5c1c0 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.382+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4e5c1c0 0x5585b4e5e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.384+0000 7f5c7aff5640 10 client.27322 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.384+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4e5c1c0 0x5585b4e5e640 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b069a2c30cfc0ed server_cookie=d65b40c4a888f450 in_seq=0 out_seq=0 2024-05-09T09:17:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.386+0000 7f5c7aff5640 1 -- 172.21.15.64:0/43287556 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585b4da6a70 con 0x5585b4e5c1c0 2024-05-09T09:17:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.386+0000 7f5c7aff5640 10 client.27322 handle_command_reply: tid=0 2024-05-09T09:17:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.386+0000 7f5ca05bd740 1 client.27322 shutdown 2024-05-09T09:17:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 msgr2=0x5585b4e46ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 0x5585b4e46ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5c880ccea0 tx=0x7f5c880095b0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 shutdown_connections 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4e5c1c0 0x5585b4e5e640 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e3af60 0x5585b4e46ba0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35df0 0x5585b4e3fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/43287556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.388+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 >> 172.21.15.64:0/43287556 conn(0x5585b4e30a70 msgr2=0x5585b4e2fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.389+0000 7f5c7aff5640 0 client.27322 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.389+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 shutdown_connections 2024-05-09T09:17:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.390+0000 7f5ca05bd740 1 -- 172.21.15.64:0/43287556 wait complete. 2024-05-09T09:17:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.390+0000 7f5ca05bd740 20 client.27322 trim_cache size 0 max 16384 2024-05-09T09:17:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.419+0000 7f5ca05bd740 1 Processor -- start 2024-05-09T09:17:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.419+0000 7f5ca05bd740 1 -- start start 2024-05-09T09:17:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.420+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.420+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35df0 0x5585b4e46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.420+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e3af60 0x5585b4e3fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.420+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.420+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:45614/0 (socket says 172.21.15.64:45614) 2024-05-09T09:17:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.420+0000 7f5c96ffd640 1 -- 172.21.15.64:0/257571417 learned_addr learned my addr 172.21.15.64:0/257571417 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.422+0000 7f5c967fc640 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35df0 0x5585b4e46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.425+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9013f070 con 0x5585b4e3af60 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.425+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9013c940 con 0x5585b4e35400 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.425+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9013a790 con 0x5585b4e35df0 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.425+0000 7f5c977fe640 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e3af60 0x5585b4e3fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.426+0000 7f5c96ffd640 1 -- 172.21.15.64:0/257571417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35df0 msgr2=0x5585b4e46b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.426+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35df0 0x5585b4e46b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.426+0000 7f5c96ffd640 1 -- 172.21.15.64:0/257571417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e3af60 msgr2=0x5585b4e3fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.426+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e3af60 0x5585b4e3fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.426+0000 7f5c96ffd640 1 -- 172.21.15.64:0/257571417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b4e36680 con 0x5585b4e35400 2024-05-09T09:17:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.426+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000b60 tx=0x7f5c98062db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b871481d48b33f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.427+0000 7f5c7cff9640 1 -- 172.21.15.64:0/257571417 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98073620 con 0x5585b4e35400 2024-05-09T09:17:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.427+0000 7f5c7cff9640 1 -- 172.21.15.64:0/257571417 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c9809d070 con 0x5585b4e35400 2024-05-09T09:17:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.427+0000 7f5c7cff9640 1 -- 172.21.15.64:0/257571417 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98090c80 con 0x5585b4e35400 2024-05-09T09:17:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.428+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 msgr2=0x5585b4e3f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.428+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000b60 tx=0x7f5c98062db0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 shutdown_connections 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e3af60 0x5585b4e3fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35df0 0x5585b4e46b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/257571417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e35400 0x5585b4e3f690 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 >> 172.21.15.64:0/257571417 conn(0x5585b4e30a70 msgr2=0x5585b4e30e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 shutdown_connections 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.429+0000 7f5ca05bd740 1 -- 172.21.15.64:0/257571417 wait complete. 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.430+0000 7f5ca05bd740 1 -- start start 2024-05-09T09:17:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.430+0000 7f5ca05bd740 1 Processor -- start 2024-05-09T09:17:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.431+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e43f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.431+0000 7f5ca05bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.431+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e43f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.431+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e43f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42152/0 (socket says 172.21.15.64:42152) 2024-05-09T09:17:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.431+0000 7f5c96ffd640 1 -- 172.21.15.64:0/45911510 learned_addr learned my addr 172.21.15.64:0/45911510 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:17:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.431+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e45470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9013ae00 con 0x5585b4e35df0 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c967fc640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c977fe640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e45470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9013c270 con 0x5585b4e3af60 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9013b4d0 con 0x5585b4e35400 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5ca05bd740 10 client.0 _ll_register_callbacks cb 0x7f5c88097f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c96ffd640 1 -- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 msgr2=0x5585b4e45470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e45470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c96ffd640 1 -- 172.21.15.64:0/45911510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 msgr2=0x5585b4e44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:17:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e44460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.432+0000 7f5c96ffd640 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b4e45a60 con 0x5585b4e35400 2024-05-09T09:17:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.433+0000 7f5c96ffd640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e43f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5c98050b40 tx=0x7f5c98064aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51a50e3b10282ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:17:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.434+0000 7f5c7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:17:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.434+0000 7f5c7aff5640 1 -- 172.21.15.64:0/45911510 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98090a00 con 0x5585b4e35400 2024-05-09T09:17:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.434+0000 7f5c7aff5640 1 -- 172.21.15.64:0/45911510 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c98090ba0 con 0x5585b4e35400 2024-05-09T09:17:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.434+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9809d070 con 0x5585b4e35400 2024-05-09T09:17:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.434+0000 7f5c7aff5640 1 -- 172.21.15.64:0/45911510 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5c98090dc0 con 0x5585b4e35400 2024-05-09T09:17:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.435+0000 7f5ca05bd740 10 client.27328 fetch_fsmap learned FSMap version 45 2024-05-09T09:17:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.435+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5585b4e46d50 con 0x5585b4e35400 2024-05-09T09:17:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.435+0000 7f5c7aff5640 1 -- 172.21.15.64:0/45911510 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c9809d070 con 0x5585b4e35400 2024-05-09T09:17:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.435+0000 7f5c7aff5640 1 -- 172.21.15.64:0/45911510 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f5c980998b0 con 0x5585b4e35400 2024-05-09T09:17:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.435+0000 7f5ca05bd740 10 client.27328 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.436+0000 7f5ca05bd740 10 client.27328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.436+0000 7f5ca05bd740 20 client.27328 populate_metadata read hostname 'smithi064' 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.436+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4ec2ba0 0x5585b4ec4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.436+0000 7f5ca05bd740 4 client.27328 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.436+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585b4da0dc0 con 0x5585b4ec2ba0 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.436+0000 7f5c967fc640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4ec2ba0 0x5585b4ec4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.437+0000 7f5c967fc640 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4ec2ba0 0x5585b4ec4fc0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a19bf28b87f596 server_cookie=b53fabb9f94a98df in_seq=0 out_seq=0 2024-05-09T09:17:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.437+0000 7f5c7aff5640 10 client.27328 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5c7aff5640 1 -- 172.21.15.64:0/45911510 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585b4da0dc0 con 0x5585b4ec2ba0 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5c7aff5640 10 client.27328 handle_command_reply: tid=0 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 client.27328 shutdown 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 msgr2=0x5585b4e43f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e43f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5c98050b40 tx=0x7f5c98064aa0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 shutdown_connections 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5585b4ec2ba0 0x5585b4ec4fc0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5585b4e3af60 0x5585b4e45470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585b4e35df0 0x5585b4e44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 --2- 172.21.15.64:0/45911510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585b4e35400 0x5585b4e43f20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 >> 172.21.15.64:0/45911510 conn(0x5585b4e30a70 msgr2=0x5585b4e3f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.468+0000 7f5c7aff5640 0 client.27328 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:17:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.469+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 shutdown_connections 2024-05-09T09:17:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.470+0000 7f5ca05bd740 1 -- 172.21.15.64:0/45911510 wait complete. 2024-05-09T09:17:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.470+0000 7f5ca05bd740 20 client.27328 trim_cache size 0 max 16384 2024-05-09T09:17:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:17:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:17:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:17:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5c800a2d90 msgr2=0x7f5c800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5c800a2d90 0x7f5c800a5250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009230 tx=0x7f5c8c004040 comp rx=0 tx=0).stop 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 msgr2=0x7f5c90131380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 0x7f5c90131380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800e9b0 tx=0x7f5c8800ee70 comp rx=0 tx=0).stop 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 shutdown_connections 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5c800a2d90 0x7f5c800a5250 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.473+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9009df50 0x7f5c90131380 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.474+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c9009d590 0x7f5c9012a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.474+0000 7f5c9d7db640 1 --2- 172.21.15.64:0/2659171799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c9009c3f0 0x7f5c9012a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:17:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.474+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 >> 172.21.15.64:0/2659171799 conn(0x7f5c9000b340 msgr2=0x7f5c90012970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:17:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.475+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 shutdown_connections 2024-05-09T09:17:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:17:51.475+0000 7f5c9d7db640 1 -- 172.21.15.64:0/2659171799 wait complete. 2024-05-09T09:17:52.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:52 smithi179 ceph-mon[34086]: pgmap v632: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 791 op/s 2024-05-09T09:17:52.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:52 smithi139 ceph-mon[33599]: pgmap v632: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 791 op/s 2024-05-09T09:17:52.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:52 smithi064 ceph-mon[30149]: pgmap v632: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 791 op/s 2024-05-09T09:17:54.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:54 smithi179 ceph-mon[34086]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 847 op/s 2024-05-09T09:17:54.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:54 smithi139 ceph-mon[33599]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 847 op/s 2024-05-09T09:17:54.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:54 smithi064 ceph-mon[30149]: pgmap v633: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 847 op/s 2024-05-09T09:17:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 455.0 s, 702.6 tps, lat 81.786 ms stddev 54.988 2024-05-09T09:17:56.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:56 smithi179 ceph-mon[34086]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 857 op/s 2024-05-09T09:17:56.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:56 smithi139 ceph-mon[33599]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 857 op/s 2024-05-09T09:17:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:56 smithi064 ceph-mon[30149]: pgmap v634: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 857 op/s 2024-05-09T09:17:58.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:17:58 smithi179 ceph-mon[34086]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 787 op/s 2024-05-09T09:17:58.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:17:58 smithi139 ceph-mon[33599]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 787 op/s 2024-05-09T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:17:58 smithi064 ceph-mon[30149]: pgmap v635: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 787 op/s 2024-05-09T09:18:00.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:00 smithi179 ceph-mon[34086]: pgmap v636: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-05-09T09:18:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:00 smithi139 ceph-mon[33599]: pgmap v636: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-05-09T09:18:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:00 smithi064 ceph-mon[30149]: pgmap v636: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-05-09T09:18:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 460.0 s, 740.2 tps, lat 192.458 ms stddev 568.049 2024-05-09T09:18:02.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:02 smithi179 ceph-mon[34086]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 840 op/s 2024-05-09T09:18:02.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:02 smithi139 ceph-mon[33599]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 840 op/s 2024-05-09T09:18:02.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:02 smithi064 ceph-mon[30149]: pgmap v637: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 840 op/s 2024-05-09T09:18:03.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:18:03.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:18:03.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.933+0000 7f5190e5d640 1 -- 172.21.15.64:0/3843830599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c10db70 msgr2=0x7f518c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:03.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.933+0000 7f5190e5d640 1 --2- 172.21.15.64:0/3843830599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c10db70 0x7f518c10e020 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f517c009900 tx=0x7f517c02a440 comp rx=0 tx=0).stop 2024-05-09T09:18:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 -- 172.21.15.64:0/3843830599 shutdown_connections 2024-05-09T09:18:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 --2- 172.21.15.64:0/3843830599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c10db70 0x7f518c10e020 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 --2- 172.21.15.64:0/3843830599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f518c10b1a0 0x7f518c10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 --2- 172.21.15.64:0/3843830599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c108830 0x7f518c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 -- 172.21.15.64:0/3843830599 >> 172.21.15.64:0/3843830599 conn(0x7f518c06e6c0 msgr2=0x7f518c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 -- 172.21.15.64:0/3843830599 shutdown_connections 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.935+0000 7f5190e5d640 1 -- 172.21.15.64:0/3843830599 wait complete. 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 Processor -- start 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 -- start start 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108830 0x7f518c114f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f518c10b1a0 0x7f518c1154d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 0x7f518c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f518c11c220 con 0x7f518c108830 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f518c11c0a0 con 0x7f518c10b1a0 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.936+0000 7f5190e5d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f518c11c3a0 con 0x7f518c10db70 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 0x7f518c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 0x7f518c1a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45120/0 (socket says 172.21.15.64:45120) 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 -- 172.21.15.64:0/1903621281 learned_addr learned my addr 172.21.15.64:0/1903621281 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 -- 172.21.15.64:0/1903621281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f518c10b1a0 msgr2=0x7f518c1154d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f518c10b1a0 0x7f518c1154d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 -- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108830 msgr2=0x7f518c114f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.937+0000 7f518ad9d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108830 0x7f518c114f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f518ad9d640 1 -- 172.21.15.64:0/1903621281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a5e60 con 0x7f518c10db70 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f518ad9d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 0x7f518c1a5920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f517c009850 tx=0x7f517c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:03.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f517b7fe640 1 -- 172.21.15.64:0/1903621281 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c03f070 con 0x7f518c10db70 2024-05-09T09:18:03.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f517b7fe640 1 -- 172.21.15.64:0/1903621281 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f517c031bc0 con 0x7f518c10db70 2024-05-09T09:18:03.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a60f0 con 0x7f518c10db70 2024-05-09T09:18:03.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f517b7fe640 1 -- 172.21.15.64:0/1903621281 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c035030 con 0x7f518c10db70 2024-05-09T09:18:03.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.938+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a65e0 con 0x7f518c10db70 2024-05-09T09:18:03.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.945+0000 7f5193b6c740 1 Processor -- start 2024-05-09T09:18:03.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.946+0000 7f517b7fe640 1 -- 172.21.15.64:0/1903621281 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f517c042070 con 0x7f518c10db70 2024-05-09T09:18:03.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.947+0000 7f5193b6c740 1 -- start start 2024-05-09T09:18:03.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.947+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd17420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f517b7fe640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f51740a2d10 0x7f51740a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f518a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd17420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f518a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd17420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:46382/0 (socket says 172.21.15.64:46382) 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f518a59c640 1 -- 172.21.15.64:0/3943631962 learned_addr learned my addr 172.21.15.64:0/3943631962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5189d9b640 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f518ad9d640 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a5bc76d00 con 0x560a5bd11f30 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a5bc1e4c0 con 0x560a5bd12920 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a5bc72cc0 con 0x560a5bd17a90 2024-05-09T09:18:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f5189d9b640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f51740a2d10 0x7f51740a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.948+0000 7f517b7fe640 1 -- 172.21.15.64:0/1903621281 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f517c0f03b0 con 0x7f518c10db70 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f518ad9d640 1 -- 172.21.15.64:0/3943631962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 msgr2=0x560a5bd17420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f518ad9d640 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd17420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f518ad9d640 1 -- 172.21.15.64:0/3943631962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 msgr2=0x560a5bd12350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f518ad9d640 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd12350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f518ad9d640 1 -- 172.21.15.64:0/3943631962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5ba547d0 con 0x560a5bd17a90 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f518ad9d640 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd19f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f517c031470 tx=0x7f517c02a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e92d7aac2cef3b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.949+0000 7f5178ff9640 1 -- 172.21.15.64:0/3943631962 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0bcf20 con 0x560a5bd17a90 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.950+0000 7f5189d9b640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f51740a2d10 0x7f51740a51d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f518c1164a0 tx=0x7f5180009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.950+0000 7f5178ff9640 1 -- 172.21.15.64:0/3943631962 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f517c0bc820 con 0x560a5bd17a90 2024-05-09T09:18:03.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.950+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 msgr2=0x560a5bd19f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.950+0000 7f5193b6c740 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd19f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f517c031470 tx=0x7f517c02a440 comp rx=0 tx=0).stop 2024-05-09T09:18:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.951+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 shutdown_connections 2024-05-09T09:18:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.951+0000 7f5193b6c740 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd19f20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.951+0000 7f5193b6c740 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd17420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.951+0000 7f5193b6c740 1 --2- 172.21.15.64:0/3943631962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.951+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 >> 172.21.15.64:0/3943631962 conn(0x560a5bd0d5a0 msgr2=0x560a5bd0d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:03.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.952+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 shutdown_connections 2024-05-09T09:18:03.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.954+0000 7f5193b6c740 1 -- 172.21.15.64:0/3943631962 wait complete. 2024-05-09T09:18:03.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.954+0000 7f5193b6c740 1 -- start start 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.954+0000 7f5193b6c740 1 Processor -- start 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd1bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd12920 0x560a5bd1c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd17a90 0x560a5bd23360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a5bc72cc0 con 0x560a5bd12920 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560a5bc31b90 con 0x560a5bd17a90 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a5bd1a800 con 0x560a5bd11f30 2024-05-09T09:18:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f518a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd1bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f518a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd1bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45140/0 (socket says 172.21.15.64:45140) 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f518a59c640 1 -- 172.21.15.64:0/4292444150 learned_addr learned my addr 172.21.15.64:0/4292444150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f518ad9d640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd17a90 0x560a5bd23360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.955+0000 7f5193b6c740 10 client.0 _ll_register_callbacks cb 0x560a5bc6dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f518a59c640 1 -- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd17a90 msgr2=0x560a5bd23360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f518a59c640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd17a90 0x560a5bd23360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd12920 0x560a5bd1c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f518a59c640 1 -- 172.21.15.64:0/4292444150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd12920 msgr2=0x560a5bd1c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f518a59c640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd12920 0x560a5bd1c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f518a59c640 1 -- 172.21.15.64:0/4292444150 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5b98a3f0 con 0x560a5bd11f30 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd12920 0x560a5bd1c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f518a59c640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd1bde0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f518400e6f0 tx=0x7f518400ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc698ef79aaf5c35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f5171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f5171ffb640 1 -- 172.21.15.64:0/4292444150 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184018020 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517c0bc820 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.956+0000 7f5171ffb640 1 -- 172.21.15.64:0/4292444150 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f518400b7b0 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5171ffb640 1 -- 172.21.15.64:0/4292444150 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5184023050 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 10 client.27337 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560a5b94a0f0 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5171ffb640 1 -- 172.21.15.64:0/4292444150 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f517c0bc820 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5171ffb640 1 -- 172.21.15.64:0/4292444150 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f5184014c10 con 0x560a5bd11f30 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 10 client.27337 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 10 client.27337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 20 client.27337 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:03.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd27cc0 0x560a5bd2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:03.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 4 client.27337 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:03.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.957+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a5bc84430 con 0x560a5bd27cc0 2024-05-09T09:18:03.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.958+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd27cc0 0x560a5bd2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.959+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd27cc0 0x560a5bd2a180 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a0170d001413b5 server_cookie=782212a229644df2 in_seq=0 out_seq=0 2024-05-09T09:18:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.959+0000 7f5171ffb640 10 client.27337 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.960+0000 7f5171ffb640 1 -- 172.21.15.64:0/4292444150 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a5bc84430 con 0x560a5bd27cc0 2024-05-09T09:18:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.960+0000 7f5171ffb640 10 client.27337 handle_command_reply: tid=0 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.960+0000 7f5193b6c740 1 client.27337 shutdown 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.962+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 msgr2=0x560a5bd1bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.962+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd1bde0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f518400e6f0 tx=0x7f518400ebb0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 shutdown_connections 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd27cc0 0x560a5bd2a180 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd17a90 0x560a5bd23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd12920 0x560a5bd1c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4292444150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd1bde0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 >> 172.21.15.64:0/4292444150 conn(0x560a5bd0d5a0 msgr2=0x560a5bd0c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5171ffb640 0 client.27337 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 shutdown_connections 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 1 -- 172.21.15.64:0/4292444150 wait complete. 2024-05-09T09:18:03.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.963+0000 7f5193b6c740 20 client.27337 trim_cache size 0 max 16384 2024-05-09T09:18:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:03.999+0000 7f5193b6c740 1 Processor -- start 2024-05-09T09:18:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 -- start start 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd1c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1d5320 con 0x560a5bd11f30 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b0be0 con 0x560a5bd12920 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5193b6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1aea30 con 0x560a5bd17a90 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f518a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f518a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd2a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39188/0 (socket says 172.21.15.64:39188) 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f518a59c640 1 -- 172.21.15.64:0/4123368238 learned_addr learned my addr 172.21.15.64:0/4123368238 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.000+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd1c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5189d9b640 1 -- 172.21.15.64:0/4123368238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 msgr2=0x560a5bd232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5189d9b640 1 -- 172.21.15.64:0/4123368238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 msgr2=0x560a5bd2a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd2a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5189d9b640 1 -- 172.21.15.64:0/4123368238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5bd08830 con 0x560a5bd12920 2024-05-09T09:18:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5189d9b640 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd1c280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x560a5bd2a990 tx=0x7f518001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b98a1c575045d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5173fff640 1 -- 172.21.15.64:0/4123368238 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180010d20 con 0x560a5bd12920 2024-05-09T09:18:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5173fff640 1 -- 172.21.15.64:0/4123368238 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5180029850 con 0x560a5bd12920 2024-05-09T09:18:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.001+0000 7f5173fff640 1 -- 172.21.15.64:0/4123368238 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800047a0 con 0x560a5bd12920 2024-05-09T09:18:04.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 -- 172.21.15.64:0/4123368238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 msgr2=0x560a5bd1c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd1c280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x560a5bd2a990 tx=0x7f518001d000 comp rx=0 tx=0).stop 2024-05-09T09:18:04.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 -- 172.21.15.64:0/4123368238 shutdown_connections 2024-05-09T09:18:04.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd17a90 0x560a5bd232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd1c280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 --2- 172.21.15.64:0/4123368238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd11f30 0x560a5bd2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 -- 172.21.15.64:0/4123368238 >> 172.21.15.64:0/4123368238 conn(0x560a5bd0d5a0 msgr2=0x560a5bd171d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 -- 172.21.15.64:0/4123368238 shutdown_connections 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 -- 172.21.15.64:0/4123368238 wait complete. 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 -- start start 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.003+0000 7f5193b6c740 1 Processor -- start 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd20cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5193b6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 0x560a5bd21d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5193b6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1af0a0 con 0x560a5bd17a90 2024-05-09T09:18:04.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5193b6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b0510 con 0x560a5bd12920 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f518ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 0x560a5bd21d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f518a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f518ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 0x560a5bd21d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39192/0 (socket says 172.21.15.64:39192) 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f518ad9d640 1 -- 172.21.15.64:0/1484121462 learned_addr learned my addr 172.21.15.64:0/1484121462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f518a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd207b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45154/0 (socket says 172.21.15.64:45154) 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5189d9b640 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd20cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.004+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1af770 con 0x560a5bd11f30 2024-05-09T09:18:04.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.005+0000 7f5193b6c740 10 client.0 _ll_register_callbacks cb 0x7f51840099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:04.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f518ad9d640 1 -- 172.21.15.64:0/1484121462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 msgr2=0x560a5bd207b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f518ad9d640 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd207b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f518ad9d640 1 -- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 msgr2=0x560a5bd20cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f518ad9d640 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd20cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f518ad9d640 1 -- 172.21.15.64:0/1484121462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5bd222f0 con 0x560a5bd17a90 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f518ad9d640 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 0x560a5bd21d00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b8c80 tx=0x7f517c03b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5207d02635c6f7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.006+0000 7f5171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.007+0000 7f5171ffb640 1 -- 172.21.15.64:0/1484121462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0bc820 con 0x560a5bd17a90 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.007+0000 7f5171ffb640 1 -- 172.21.15.64:0/1484121462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f517c0edb40 con 0x560a5bd17a90 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.007+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5180029850 con 0x560a5bd17a90 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.007+0000 7f5171ffb640 1 -- 172.21.15.64:0/1484121462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f517c0bcf20 con 0x560a5bd17a90 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.007+0000 7f5193b6c740 10 client.17673 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:04.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.007+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560a5bd23610 con 0x560a5bd17a90 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.008+0000 7f5171ffb640 1 -- 172.21.15.64:0/1484121462 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5180029850 con 0x560a5bd17a90 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.008+0000 7f5171ffb640 1 -- 172.21.15.64:0/1484121462 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f517c04c020 con 0x560a5bd17a90 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.008+0000 7f5193b6c740 10 client.17673 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.008+0000 7f5193b6c740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.008+0000 7f5193b6c740 20 client.17673 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.008+0000 7f5193b6c740 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd8e630 0x560a5bd90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.009+0000 7f5193b6c740 4 client.17673 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.009+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a5bc7e780 con 0x560a5bd8e630 2024-05-09T09:18:04.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.009+0000 7f518a59c640 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd8e630 0x560a5bd90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.011+0000 7f5171ffb640 10 client.17673 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:04.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.011+0000 7f518a59c640 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd8e630 0x560a5bd90a20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38698f85d2f5cbe6 server_cookie=8219ac90bebe318 in_seq=0 out_seq=0 2024-05-09T09:18:04.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.011+0000 7f5171ffb640 1 -- 172.21.15.64:0/1484121462 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a5bc7e780 con 0x560a5bd8e630 2024-05-09T09:18:04.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.011+0000 7f5171ffb640 10 client.17673 handle_command_reply: tid=0 2024-05-09T09:18:04.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.012+0000 7f5193b6c740 1 client.17673 shutdown 2024-05-09T09:18:04.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.012+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 msgr2=0x560a5bd21d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.012+0000 7f5193b6c740 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 0x560a5bd21d00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b8c80 tx=0x7f517c03b670 comp rx=0 tx=0).stop 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.012+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 shutdown_connections 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.012+0000 7f5193b6c740 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560a5bd8e630 0x560a5bd90a20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5171ffb640 0 client.17673 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5193b6c740 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a5bd17a90 0x560a5bd21d00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5193b6c740 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560a5bd12920 0x560a5bd20cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5193b6c740 1 --2- 172.21.15.64:0/1484121462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a5bd11f30 0x560a5bd207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 >> 172.21.15.64:0/1484121462 conn(0x560a5bd0d5a0 msgr2=0x560a5bd1b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 shutdown_connections 2024-05-09T09:18:04.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.013+0000 7f5193b6c740 1 -- 172.21.15.64:0/1484121462 wait complete. 2024-05-09T09:18:04.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.015+0000 7f5193b6c740 20 client.17673 trim_cache size 0 max 16384 2024-05-09T09:18:04.016 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:18:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:18:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:18:04.016 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f51740a2d10 msgr2=0x7f51740a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f51740a2d10 0x7f51740a51d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f518c1164a0 tx=0x7f5180009780 comp rx=0 tx=0).stop 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 msgr2=0x7f518c1a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 0x7f518c1a5920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f517c009850 tx=0x7f517c003b90 comp rx=0 tx=0).stop 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 shutdown_connections 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f51740a2d10 0x7f51740a51d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c10db70 0x7f518c1a5920 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f518c10b1a0 0x7f518c1154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 --2- 172.21.15.64:0/1903621281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108830 0x7f518c114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 >> 172.21.15.64:0/1903621281 conn(0x7f518c06e6c0 msgr2=0x7f518c109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.019+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 shutdown_connections 2024-05-09T09:18:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.020+0000 7f5190e5d640 1 -- 172.21.15.64:0/1903621281 wait complete. 2024-05-09T09:18:04.041 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:18:04.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:18:04.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.223+0000 7facae18c640 1 -- 172.21.15.64:0/2717818111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0096f70 msgr2=0x7faca00993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.223+0000 7facae18c640 1 --2- 172.21.15.64:0/2717818111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0096f70 0x7faca00993e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fac98009900 tx=0x7fac9802a420 comp rx=0 tx=0).stop 2024-05-09T09:18:04.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 -- 172.21.15.64:0/2717818111 shutdown_connections 2024-05-09T09:18:04.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 --2- 172.21.15.64:0/2717818111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0099920 0x7faca0099da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 --2- 172.21.15.64:0/2717818111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0096f70 0x7faca00993e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 --2- 172.21.15.64:0/2717818111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faca0094600 0x7faca0096a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 -- 172.21.15.64:0/2717818111 >> 172.21.15.64:0/2717818111 conn(0x7faca00901e0 msgr2=0x7faca0092640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 -- 172.21.15.64:0/2717818111 shutdown_connections 2024-05-09T09:18:04.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.225+0000 7facae18c640 1 -- 172.21.15.64:0/2717818111 wait complete. 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.226+0000 7facae18c640 1 Processor -- start 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.226+0000 7facae18c640 1 -- start start 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facae18c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faca0094600 0x7faca0132520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facae18c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 0x7faca0132a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facae18c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0099920 0x7faca0139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facae18c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faca00a7ff0 con 0x7faca0099920 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facae18c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faca00a7e70 con 0x7faca0094600 2024-05-09T09:18:04.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facae18c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faca00a8170 con 0x7faca0096f70 2024-05-09T09:18:04.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facac989640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 0x7faca0132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facac989640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 0x7faca0132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45168/0 (socket says 172.21.15.64:45168) 2024-05-09T09:18:04.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facac989640 1 -- 172.21.15.64:0/4071671187 learned_addr learned my addr 172.21.15.64:0/4071671187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.227+0000 7facad98b640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0099920 0x7faca0139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facac989640 1 -- 172.21.15.64:0/4071671187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faca0094600 msgr2=0x7faca0132520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:18:04.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facac989640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faca0094600 0x7faca0132520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facac989640 1 -- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0099920 msgr2=0x7faca0139ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facac989640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0099920 0x7faca0139ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facac989640 1 -- 172.21.15.64:0/4071671187 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca013a140 con 0x7faca0096f70 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facad98b640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0099920 0x7faca0139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.228+0000 7facac989640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 0x7faca0132a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac98009850 tx=0x7fac98032520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.229+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9803f030 con 0x7faca0096f70 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.229+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac98032dd0 con 0x7faca0096f70 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.229+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035070 con 0x7faca0096f70 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.229+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca013a3d0 con 0x7faca0096f70 2024-05-09T09:18:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.229+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca013a8f0 con 0x7faca0096f70 2024-05-09T09:18:04.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.231+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca013ab50 con 0x7faca0096f70 2024-05-09T09:18:04.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.233+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac98042030 con 0x7faca0096f70 2024-05-09T09:18:04.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.235+0000 7fac9e7fc640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac940a2d60 0x7fac940a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.235+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fac9806de00 con 0x7faca0096f70 2024-05-09T09:18:04.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.236+0000 7facad18a640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac940a2d60 0x7fac940a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.236+0000 7facad18a640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac940a2d60 0x7fac940a5220 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faca40096b0 tx=0x7faca4006c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.239+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fac980bc820 con 0x7faca0096f70 2024-05-09T09:18:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.645+0000 7fbdd553b640 1 -- 172.21.15.64:0/4107679642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0108ed0 msgr2=0x7fbdd0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.645+0000 7fbdd553b640 1 --2- 172.21.15.64:0/4107679642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0108ed0 0x7fbdd0109330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009860 tx=0x7fbdc002a340 comp rx=0 tx=0).stop 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 -- 172.21.15.64:0/4107679642 shutdown_connections 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 --2- 172.21.15.64:0/4107679642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0109870 0x7fbdd010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 --2- 172.21.15.64:0/4107679642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0108ed0 0x7fbdd0109330 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 --2- 172.21.15.64:0/4107679642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd0072c20 0x7fbdd0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 -- 172.21.15.64:0/4107679642 >> 172.21.15.64:0/4107679642 conn(0x7fbdd006e6d0 msgr2=0x7fbdd0070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 -- 172.21.15.64:0/4107679642 shutdown_connections 2024-05-09T09:18:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.646+0000 7fbdd553b640 1 -- 172.21.15.64:0/4107679642 wait complete. 2024-05-09T09:18:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.647+0000 7fbdd553b640 1 Processor -- start 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.647+0000 7fbdd553b640 1 -- start start 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdd553b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 0x7fbdd0114f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdd553b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd0108ed0 0x7fbdd01154a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdd553b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0109870 0x7fbdd01b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdd553b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd011c080 con 0x7fbdd0108ed0 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdd553b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd011bf00 con 0x7fbdd0072c20 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdd553b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd011c200 con 0x7fbdd0109870 2024-05-09T09:18:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 0x7fbdd0114f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 0x7fbdd0114f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60618/0 (socket says 172.21.15.64:60618) 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 -- 172.21.15.64:0/596363914 learned_addr learned my addr 172.21.15.64:0/596363914 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdcf7fe640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0109870 0x7fbdd01b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 -- 172.21.15.64:0/596363914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0109870 msgr2=0x7fbdd01b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0109870 0x7fbdd01b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 -- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd0108ed0 msgr2=0x7fbdd01154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd0108ed0 0x7fbdd01154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.648+0000 7fbdceffd640 1 -- 172.21.15.64:0/596363914 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd01b6c90 con 0x7fbdd0072c20 2024-05-09T09:18:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.649+0000 7fbdceffd640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 0x7fbdd0114f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c420 tx=0x7fbdc8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.649+0000 7fbdb7fff640 1 -- 172.21.15.64:0/596363914 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8003970 con 0x7fbdd0072c20 2024-05-09T09:18:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.650+0000 7fbdb7fff640 1 -- 172.21.15.64:0/596363914 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc8004020 con 0x7fbdd0072c20 2024-05-09T09:18:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.651+0000 7fbdb7fff640 1 -- 172.21.15.64:0/596363914 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800b980 con 0x7fbdd0072c20 2024-05-09T09:18:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.651+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd01b6e60 con 0x7fbdd0072c20 2024-05-09T09:18:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.652+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01b7210 con 0x7fbdd0072c20 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 Processor -- start 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 -- start start 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965943820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 0x5579659488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x55796594b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5579658a7340 con 0x557965943df0 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55796584f290 con 0x557965943400 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.657+0000 7fbdd8246740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579658a3300 con 0x557965948f60 2024-05-09T09:18:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 0x5579659488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 0x5579659488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44690/0 (socket says 172.21.15.64:44690) 2024-05-09T09:18:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 -- 172.21.15.64:0/4138999590 learned_addr learned my addr 172.21.15.64:0/4138999590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 -- 172.21.15.64:0/4138999590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 msgr2=0x55796594b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x55796594b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 -- 172.21.15.64:0/4138999590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 msgr2=0x557965943820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965943820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.658+0000 7fbdceffd640 1 -- 172.21.15.64:0/4138999590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579658738e0 con 0x557965943df0 2024-05-09T09:18:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.660+0000 7fbdb7fff640 1 -- 172.21.15.64:0/596363914 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbdc801d020 con 0x7fbdd0072c20 2024-05-09T09:18:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.662+0000 7fbdb7fff640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbdb80a2c00 0x7fbdb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.662+0000 7fbdb7fff640 1 -- 172.21.15.64:0/596363914 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbdc8025040 con 0x7fbdd0072c20 2024-05-09T09:18:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.662+0000 7fbdceffd640 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 0x5579659488f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8018040 tx=0x7fbdc8004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f803e259b01943 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.663+0000 7fbdcf7fe640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbdb80a2c00 0x7fbdb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.663+0000 7fbdb57fa640 1 -- 172.21.15.64:0/4138999590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8099460 con 0x557965943df0 2024-05-09T09:18:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.663+0000 7fbdb57fa640 1 -- 172.21.15.64:0/4138999590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc809d8c0 con 0x557965943df0 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdb57fa640 1 -- 172.21.15.64:0/4138999590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8003cd0 con 0x557965943df0 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 -- 172.21.15.64:0/4138999590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 msgr2=0x5579659488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 0x5579659488f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8018040 tx=0x7fbdc8004ee0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 -- 172.21.15.64:0/4138999590 shutdown_connections 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x55796594b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943df0 0x5579659488f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 --2- 172.21.15.64:0/4138999590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965943820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 -- 172.21.15.64:0/4138999590 >> 172.21.15.64:0/4138999590 conn(0x55796593ea70 msgr2=0x55796593ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.664+0000 7fbdd8246740 1 -- 172.21.15.64:0/4138999590 shutdown_connections 2024-05-09T09:18:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 -- 172.21.15.64:0/4138999590 wait complete. 2024-05-09T09:18:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 -- start start 2024-05-09T09:18:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 Processor -- start 2024-05-09T09:18:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdcf7fe640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbdb80a2c00 0x7fbdb80a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x557965944650 tx=0x7fbdc4009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 0x55796594d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x55796594d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965954970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55796594bd90 con 0x557965948f60 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557965862790 con 0x557965943df0 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.665+0000 7fbdd8246740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55796594bf10 con 0x557965943400 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.666+0000 7fbdd8246740 10 client.0 _ll_register_callbacks cb 0x55796589da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.666+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 0x55796594d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.666+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 0x55796594d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49518/0 (socket says 172.21.15.64:49518) 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.666+0000 7fbdceffd640 1 -- 172.21.15.64:0/3221750461 learned_addr learned my addr 172.21.15.64:0/3221750461 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.666+0000 7fbdce7fc640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x55796594d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdcf7fe640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965954970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdceffd640 1 -- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 msgr2=0x55796594d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdceffd640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x55796594d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdceffd640 1 -- 172.21.15.64:0/3221750461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 msgr2=0x557965954970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdceffd640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965954970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdceffd640 1 -- 172.21.15.64:0/3221750461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579655bb210 con 0x557965943400 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdceffd640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 0x55796594d3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c3f0 tx=0x7fbdc8099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2392da0bee07919c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/3221750461 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80b6060 con 0x557965943400 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.667+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/3221750461 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbdc80a5b20 con 0x557965943400 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.668+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc809d8c0 con 0x557965943400 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.668+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/3221750461 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbdc809cb20 con 0x557965943400 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.668+0000 7fbdd8246740 10 client.27352 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.668+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557965687620 con 0x557965943400 2024-05-09T09:18:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/3221750461 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc809d8c0 con 0x557965943400 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/3221750461 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc800b040 con 0x557965943400 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdd8246740 10 client.27352 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdd8246740 10 client.27352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdd8246740 20 client.27352 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdd8246740 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557965958bc0 0x55796595afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdd8246740 4 client.27352 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.670+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579658b4a70 con 0x557965958bc0 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.672+0000 7fbdce7fc640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557965958bc0 0x55796595afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.672+0000 7fbdb2ff5640 10 client.27352 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.672+0000 7fbdce7fc640 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557965958bc0 0x55796595afb0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c657ccd243524a server_cookie=34aaf5e935424b52 in_seq=0 out_seq=0 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.673+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/3221750461 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579658b4a70 con 0x557965958bc0 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.673+0000 7fbdb2ff5640 10 client.27352 handle_command_reply: tid=0 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.673+0000 7fbdd8246740 1 client.27352 shutdown 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.673+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 msgr2=0x55796594d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.673+0000 7fbdd8246740 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 0x55796594d3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c3f0 tx=0x7fbdc8099a20 comp rx=0 tx=0).stop 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 shutdown_connections 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557965958bc0 0x55796595afb0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965954970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x55796594d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 --2- 172.21.15.64:0/3221750461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943400 0x55796594d3f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 >> 172.21.15.64:0/3221750461 conn(0x55796593ea70 msgr2=0x55796593dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdb2ff5640 0 client.27352 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 shutdown_connections 2024-05-09T09:18:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 1 -- 172.21.15.64:0/3221750461 wait complete. 2024-05-09T09:18:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.674+0000 7fbdd8246740 20 client.27352 trim_cache size 0 max 16384 2024-05-09T09:18:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.707+0000 7fbdd8246740 1 Processor -- start 2024-05-09T09:18:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 -- start start 2024-05-09T09:18:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965941a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943df0 0x557965954900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965941f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01c4010 con 0x557965948f60 2024-05-09T09:18:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01c18e0 con 0x557965943400 2024-05-09T09:18:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdd8246740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01bf730 con 0x557965943df0 2024-05-09T09:18:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.708+0000 7fbdcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965941f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965941a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965941a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60654/0 (socket says 172.21.15.64:60654) 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 -- 172.21.15.64:0/1678949960 learned_addr learned my addr 172.21.15.64:0/1678949960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965941f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44712/0 (socket says 172.21.15.64:44712) 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 -- 172.21.15.64:0/1678949960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943df0 msgr2=0x557965954900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943df0 0x557965954900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 -- 172.21.15.64:0/1678949960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 msgr2=0x557965941f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965941f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.709+0000 7fbdceffd640 1 -- 172.21.15.64:0/1678949960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557965948210 con 0x557965943400 2024-05-09T09:18:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.710+0000 7fbdceffd640 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965941a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800cdf0 tx=0x7fbdc800e2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7e061fa59395dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.710+0000 7fbdb4ff9640 1 -- 172.21.15.64:0/1678949960 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80b6040 con 0x557965943400 2024-05-09T09:18:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.710+0000 7fbdb4ff9640 1 -- 172.21.15.64:0/1678949960 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc809d8c0 con 0x557965943400 2024-05-09T09:18:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.710+0000 7fbdb4ff9640 1 -- 172.21.15.64:0/1678949960 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc809cd00 con 0x557965943400 2024-05-09T09:18:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 -- 172.21.15.64:0/1678949960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 msgr2=0x557965941a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965941a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800cdf0 tx=0x7fbdc800e2f0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 -- 172.21.15.64:0/1678949960 shutdown_connections 2024-05-09T09:18:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965948f60 0x557965941f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965943df0 0x557965954900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 --2- 172.21.15.64:0/1678949960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943400 0x557965941a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.712+0000 7fbdd8246740 1 -- 172.21.15.64:0/1678949960 >> 172.21.15.64:0/1678949960 conn(0x55796593ea70 msgr2=0x55796594d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.713+0000 7fbdd8246740 1 -- 172.21.15.64:0/1678949960 shutdown_connections 2024-05-09T09:18:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.713+0000 7fbdd8246740 1 -- 172.21.15.64:0/1678949960 wait complete. 2024-05-09T09:18:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.714+0000 7fbdd8246740 1 -- start start 2024-05-09T09:18:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.714+0000 7fbdd8246740 1 Processor -- start 2024-05-09T09:18:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.715+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943400 0x557965952320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.715+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x557965952860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.715+0000 7fbdd8246740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x557965953820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdd8246740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01bfda0 con 0x557965943400 2024-05-09T09:18:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdd8246740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01c1210 con 0x557965943df0 2024-05-09T09:18:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdce7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x557965952860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdce7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x557965952860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60666/0 (socket says 172.21.15.64:60666) 2024-05-09T09:18:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdce7fc640 1 -- 172.21.15.64:0/2594852134 learned_addr learned my addr 172.21.15.64:0/2594852134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01c0470 con 0x557965948f60 2024-05-09T09:18:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdd8246740 10 client.0 _ll_register_callbacks cb 0x7fbdc80a5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdcf7fe640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x557965953820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.716+0000 7fbdce7fc640 1 -- 172.21.15.64:0/2594852134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 msgr2=0x557965953820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdce7fc640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x557965953820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdceffd640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943400 0x557965952320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdce7fc640 1 -- 172.21.15.64:0/2594852134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943400 msgr2=0x557965952320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdce7fc640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943400 0x557965952320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdce7fc640 1 -- 172.21.15.64:0/2594852134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557965953e10 con 0x557965943df0 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdcf7fe640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x557965953820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdceffd640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943400 0x557965952320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdce7fc640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x557965952860 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009830 tx=0x7fbdc0005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab135d785cba05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.717+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/2594852134 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0037040 con 0x557965943df0 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.718+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc809d8c0 con 0x557965943df0 2024-05-09T09:18:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.718+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/2594852134 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbdc0004240 con 0x557965943df0 2024-05-09T09:18:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.718+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/2594852134 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbdc004e050 con 0x557965943df0 2024-05-09T09:18:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.718+0000 7fbdd8246740 10 client.27200 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.718+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579659bf3e0 con 0x557965943df0 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.718+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/2594852134 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc809d8c0 con 0x557965943df0 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/2594852134 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc0005240 con 0x557965943df0 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdd8246740 10 client.27200 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdd8246740 10 client.27200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdd8246740 20 client.27200 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdd8246740 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5579659bfd60 0x5579659c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdd8246740 4 client.27200 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:18:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.719+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579658aedc0 con 0x5579659bfd60 2024-05-09T09:18:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.720+0000 7fbdceffd640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5579659bfd60 0x5579659c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.720+0000 7fbdceffd640 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5579659bfd60 0x5579659c2180 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7f1d3014af52e3 server_cookie=64914dbe6cca6252 in_seq=0 out_seq=0 2024-05-09T09:18:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.720+0000 7fbdb2ff5640 10 client.27200 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:04.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:04 smithi179 ceph-mon[34086]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.8 MiB/s wr, 938 op/s 2024-05-09T09:18:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.740+0000 7fbdb2ff5640 1 -- 172.21.15.64:0/2594852134 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579658aedc0 con 0x5579659bfd60 2024-05-09T09:18:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.740+0000 7fbdb2ff5640 10 client.27200 handle_command_reply: tid=0 2024-05-09T09:18:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.741+0000 7fbdd8246740 1 client.27200 shutdown 2024-05-09T09:18:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.742+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 msgr2=0x557965952860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.742+0000 7fbdd8246740 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x557965952860 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009830 tx=0x7fbdc0005a90 comp rx=0 tx=0).stop 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 shutdown_connections 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5579659bfd60 0x5579659c2180 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdb2ff5640 0 client.27200 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557965948f60 0x557965953820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557965943df0 0x557965952860 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 --2- 172.21.15.64:0/2594852134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557965943400 0x557965952320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 >> 172.21.15.64:0/2594852134 conn(0x55796593ea70 msgr2=0x557965941a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 shutdown_connections 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 1 -- 172.21.15.64:0/2594852134 wait complete. 2024-05-09T09:18:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.743+0000 7fbdd8246740 20 client.27200 trim_cache size 0 max 16384 2024-05-09T09:18:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:18:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:18:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:18:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:18:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.745+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbdb80a2c00 msgr2=0x7fbdb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.745+0000 7fbdd553b640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbdb80a2c00 0x7fbdb80a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x557965944650 tx=0x7fbdc4009530 comp rx=0 tx=0).stop 2024-05-09T09:18:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.745+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 msgr2=0x7fbdd0114f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.745+0000 7fbdd553b640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 0x7fbdd0114f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c420 tx=0x7fbdc8009f90 comp rx=0 tx=0).stop 2024-05-09T09:18:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.747+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 shutdown_connections 2024-05-09T09:18:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.747+0000 7fbdd553b640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbdb80a2c00 0x7fbdb80a50c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.747+0000 7fbdd553b640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd0109870 0x7fbdd01b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.747+0000 7fbdd553b640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdd0108ed0 0x7fbdd01154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.747+0000 7fbdd553b640 1 --2- 172.21.15.64:0/596363914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdd0072c20 0x7fbdd0114f60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.747+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 >> 172.21.15.64:0/596363914 conn(0x7fbdd006e6d0 msgr2=0x7fbdd006f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.748+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 shutdown_connections 2024-05-09T09:18:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:04.748+0000 7fbdd553b640 1 -- 172.21.15.64:0/596363914 wait complete. 2024-05-09T09:18:04.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.753+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faca013aee0 con 0x7faca0096f70 2024-05-09T09:18:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.758+0000 7fac9e7fc640 1 -- 172.21.15.64:0/4071671187 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11879 (secure 0 0 0) 0x7fac980bc120 con 0x7faca0096f70 2024-05-09T09:18:04.758 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:18:04.758 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:17:16.247105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:18:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 45 2024-05-09T09:18:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac940a2d60 msgr2=0x7fac940a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac940a2d60 0x7fac940a5220 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faca40096b0 tx=0x7faca4006c20 comp rx=0 tx=0).stop 2024-05-09T09:18:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 msgr2=0x7faca0132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 0x7faca0132a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac98009850 tx=0x7fac98032520 comp rx=0 tx=0).stop 2024-05-09T09:18:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 shutdown_connections 2024-05-09T09:18:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac940a2d60 0x7fac940a5220 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca0099920 0x7faca0139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faca0096f70 0x7faca0132a60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 --2- 172.21.15.64:0/4071671187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faca0094600 0x7faca0132520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.762+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 >> 172.21.15.64:0/4071671187 conn(0x7faca00901e0 msgr2=0x7faca0094f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.765+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 shutdown_connections 2024-05-09T09:18:04.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.765+0000 7facae18c640 1 -- 172.21.15.64:0/4071671187 wait complete. 2024-05-09T09:18:04.783 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:04 smithi064 ceph-mon[30149]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.8 MiB/s wr, 938 op/s 2024-05-09T09:18:04.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:18:04.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:04 smithi139 ceph-mon[33599]: pgmap v638: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.8 MiB/s wr, 938 op/s 2024-05-09T09:18:04.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.987+0000 7fd1e262a640 1 -- 172.21.15.64:0/2274506885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc107dd0 msgr2=0x7fd1dc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.987+0000 7fd1e262a640 1 --2- 172.21.15.64:0/2274506885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc1081f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009900 tx=0x7fd1cc02a420 comp rx=0 tx=0).stop 2024-05-09T09:18:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.989+0000 7fd1e262a640 1 -- 172.21.15.64:0/2274506885 shutdown_connections 2024-05-09T09:18:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.989+0000 7fd1e262a640 1 --2- 172.21.15.64:0/2274506885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc109930 0x7fd1dc1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.989+0000 7fd1e262a640 1 --2- 172.21.15.64:0/2274506885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1dc108f70 0x7fd1dc1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.989+0000 7fd1e262a640 1 --2- 172.21.15.64:0/2274506885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc1081f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.989+0000 7fd1e262a640 1 -- 172.21.15.64:0/2274506885 >> 172.21.15.64:0/2274506885 conn(0x7fd1dc076d00 msgr2=0x7fd1dc079160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.991+0000 7fd1e262a640 1 -- 172.21.15.64:0/2274506885 shutdown_connections 2024-05-09T09:18:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.992+0000 7fd1e262a640 1 -- 172.21.15.64:0/2274506885 wait complete. 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.994+0000 7fd1e262a640 1 Processor -- start 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.994+0000 7fd1e262a640 1 -- start start 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1e262a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1e262a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1dc108f70 0x7fd1dc070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1e262a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc109930 0x7fd1dc1adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1e262a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1138d0 con 0x7fd1dc109930 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1e262a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc113750 con 0x7fd1dc108f70 2024-05-09T09:18:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1e262a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc113a50 con 0x7fd1dc107dd0 2024-05-09T09:18:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49528/0 (socket says 172.21.15.64:49528) 2024-05-09T09:18:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.995+0000 7fd1dbfff640 1 -- 172.21.15.64:0/828948313 learned_addr learned my addr 172.21.15.64:0/828948313 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.996+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc109930 0x7fd1dc1adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.996+0000 7fd1dbfff640 1 -- 172.21.15.64:0/828948313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1dc108f70 msgr2=0x7fd1dc070810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:18:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.996+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1dc108f70 0x7fd1dc070810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.996+0000 7fd1dbfff640 1 -- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc109930 msgr2=0x7fd1dc1adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.996+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc109930 0x7fd1dc1adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.996+0000 7fd1dbfff640 1 -- 172.21.15.64:0/828948313 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc07a080 con 0x7fd1dc107dd0 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.997+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc0702d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000c00 tx=0x7fd1cc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.998+0000 7fd1d97fa640 1 -- 172.21.15.64:0/828948313 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc03f070 con 0x7fd1dc107dd0 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.998+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc07a310 con 0x7fd1dc107dd0 2024-05-09T09:18:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:04.998+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc07a800 con 0x7fd1dc107dd0 2024-05-09T09:18:05.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.000+0000 7fd1d97fa640 1 -- 172.21.15.64:0/828948313 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1cc031c30 con 0x7fd1dc107dd0 2024-05-09T09:18:05.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.000+0000 7fd1d97fa640 1 -- 172.21.15.64:0/828948313 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc035030 con 0x7fd1dc107dd0 2024-05-09T09:18:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 Processor -- start 2024-05-09T09:18:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 -- start start 2024-05-09T09:18:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d856550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d85b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d85bc90 0x55f47d85e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f47d7bac40 con 0x55f47d85bc90 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f47d762580 con 0x55f47d856130 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.002+0000 7fd1e5413740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f47d7b6c00 con 0x55f47d856b20 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d856550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d856550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60678/0 (socket says 172.21.15.64:60678) 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/2751869889 learned_addr learned my addr 172.21.15.64:0/2751869889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d85b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/2751869889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 msgr2=0x55f47d85b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d85b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/2751869889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d85bc90 msgr2=0x55f47d85e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d85bc90 0x55f47d85e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.003+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/2751869889 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47d4ce3f0 con 0x55f47d856130 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.004+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d856550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009ee0 tx=0x7fd1d4010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63cf3f5167506f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.004+0000 7fd1c27fc640 1 -- 172.21.15.64:0/2751869889 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4016070 con 0x55f47d856130 2024-05-09T09:18:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.005+0000 7fd1c27fc640 1 -- 172.21.15.64:0/2751869889 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d4004d20 con 0x55f47d856130 2024-05-09T09:18:05.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.005+0000 7fd1c27fc640 1 -- 172.21.15.64:0/2751869889 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40127c0 con 0x55f47d856130 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.005+0000 7fd1d97fa640 1 -- 172.21.15.64:0/828948313 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1cc042070 con 0x7fd1dc107dd0 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 -- 172.21.15.64:0/2751869889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 msgr2=0x55f47d856550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d856550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009ee0 tx=0x7fd1d4010740 comp rx=0 tx=0).stop 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 -- 172.21.15.64:0/2751869889 shutdown_connections 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d85bc90 0x55f47d85e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d85b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2751869889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d856550 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 -- 172.21.15.64:0/2751869889 >> 172.21.15.64:0/2751869889 conn(0x55f47d8517a0 msgr2=0x55f47d851ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 -- 172.21.15.64:0/2751869889 shutdown_connections 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.006+0000 7fd1e5413740 1 -- 172.21.15.64:0/2751869889 wait complete. 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.007+0000 7fd1d97fa640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1c40a2ca0 0x7fd1c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.007+0000 7fd1e5413740 1 -- start start 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.007+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1c40a2ca0 0x7fd1c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.007+0000 7fd1e5413740 1 Processor -- start 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.007+0000 7fd1d97fa640 1 -- 172.21.15.64:0/828948313 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd1cc0f55a0 con 0x7fd1dc107dd0 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856130 0x55f47d860670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d860bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d867bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f47d85ec90 con 0x55f47d856b20 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f47d775c40 con 0x55f47d85bc90 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f47d85ee10 con 0x55f47d856130 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e5413740 10 client.0 _ll_register_callbacks cb 0x55f47d4cdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e0bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d867bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e0bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d867bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60690/0 (socket says 172.21.15.64:60690) 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/3972499779 learned_addr learned my addr 172.21.15.64:0/3972499779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856130 0x55f47d860670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.008+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d860bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856130 msgr2=0x55f47d860670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856130 0x55f47d860670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/3972499779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 msgr2=0x55f47d860bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d860bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 -- 172.21.15.64:0/3972499779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47d5987d0 con 0x55f47d85bc90 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1c40a2ca0 0x7fd1c40a5160 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55f47d857380 tx=0x7fd1d4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d4004d20 con 0x55f47d85bc90 2024-05-09T09:18:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1e0bc7640 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d867bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4019000 tx=0x7fd1d4026000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a642631e098e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.009+0000 7fd1bb7fe640 10 client.27209 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.010+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/3972499779 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4039050 con 0x55f47d85bc90 2024-05-09T09:18:05.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.010+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/3972499779 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd1d4013ad0 con 0x55f47d85bc90 2024-05-09T09:18:05.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.010+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/3972499779 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd1d4016050 con 0x55f47d85bc90 2024-05-09T09:18:05.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.011+0000 7fd1e5413740 10 client.27209 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.011+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55f47d86aeb0 con 0x55f47d85bc90 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.012+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/3972499779 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fd1d400f040 con 0x55f47d85bc90 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.012+0000 7fd1e5413740 10 client.27209 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.012+0000 7fd1e5413740 10 client.27209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.012+0000 7fd1e5413740 20 client.27209 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.012+0000 7fd1e5413740 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d86c200 0x55f47d86e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.012+0000 7fd1e5413740 4 client.27209 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.013+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d86c200 0x55f47d86e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.013+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f47d7c8370 con 0x55f47d86c200 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.013+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d86c200 0x55f47d86e5f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55c660cd4ee2827c server_cookie=b2608d5e7842fcfa in_seq=0 out_seq=0 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.013+0000 7fd1bb7fe640 10 client.27209 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.014+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/3972499779 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f47d7c8370 con 0x55f47d86c200 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.014+0000 7fd1bb7fe640 10 client.27209 handle_command_reply: tid=0 2024-05-09T09:18:05.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.014+0000 7fd1e5413740 1 client.27209 shutdown 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 msgr2=0x55f47d867bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d867bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4019000 tx=0x7fd1d4026000 comp rx=0 tx=0).stop 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 shutdown_connections 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d86c200 0x55f47d86e5f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d867bf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d860bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 --2- 172.21.15.64:0/3972499779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856130 0x55f47d860670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 >> 172.21.15.64:0/3972499779 conn(0x55f47d8517a0 msgr2=0x55f47d8509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1bb7fe640 0 client.27209 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 shutdown_connections 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 1 -- 172.21.15.64:0/3972499779 wait complete. 2024-05-09T09:18:05.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.015+0000 7fd1e5413740 20 client.27209 trim_cache size 0 max 16384 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.043+0000 7fd1e5413740 1 Processor -- start 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.043+0000 7fd1e5413740 1 -- start start 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.043+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856130 0x55f47d85f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.043+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d867bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.043+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d854750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1e5413740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8012550 con 0x55f47d856130 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1e5413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800fe20 con 0x55f47d85bc90 2024-05-09T09:18:05.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1e5413740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800dc70 con 0x55f47d856b20 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1e0bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d854750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856130 0x55f47d85f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856130 0x55f47d85f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44782/0 (socket says 172.21.15.64:44782) 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1dbfff640 1 -- 172.21.15.64:0/2403753073 learned_addr learned my addr 172.21.15.64:0/2403753073 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.044+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d867bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.045+0000 7fd1db7fe640 1 -- 172.21.15.64:0/2403753073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 msgr2=0x55f47d854750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.045+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d854750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.045+0000 7fd1db7fe640 1 -- 172.21.15.64:0/2403753073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856130 msgr2=0x55f47d85f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.045+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856130 0x55f47d85f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.045+0000 7fd1db7fe640 1 -- 172.21.15.64:0/2403753073 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47d852510 con 0x55f47d856b20 2024-05-09T09:18:05.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.045+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d867bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55f47d861b50 tx=0x7fd1d0009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dab3cd973a3a5ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.046+0000 7fd1c1ffb640 1 -- 172.21.15.64:0/2403753073 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0004230 con 0x55f47d856b20 2024-05-09T09:18:05.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.046+0000 7fd1c1ffb640 1 -- 172.21.15.64:0/2403753073 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d0011e80 con 0x55f47d856b20 2024-05-09T09:18:05.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.046+0000 7fd1c1ffb640 1 -- 172.21.15.64:0/2403753073 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d001b450 con 0x55f47d856b20 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 -- 172.21.15.64:0/2403753073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 msgr2=0x55f47d867bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d867bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55f47d861b50 tx=0x7fd1d0009e70 comp rx=0 tx=0).stop 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 -- 172.21.15.64:0/2403753073 shutdown_connections 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d85bc90 0x55f47d854750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d856b20 0x55f47d867bb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 --2- 172.21.15.64:0/2403753073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856130 0x55f47d85f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 -- 172.21.15.64:0/2403753073 >> 172.21.15.64:0/2403753073 conn(0x55f47d8517a0 msgr2=0x55f47d85f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 -- 172.21.15.64:0/2403753073 shutdown_connections 2024-05-09T09:18:05.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.047+0000 7fd1e5413740 1 -- 172.21.15.64:0/2403753073 wait complete. 2024-05-09T09:18:05.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.048+0000 7fd1e5413740 1 -- start start 2024-05-09T09:18:05.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.048+0000 7fd1e5413740 1 Processor -- start 2024-05-09T09:18:05.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d864e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d865360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d85bc90 0x55f47d866370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800e2e0 con 0x55f47d856b20 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800f750 con 0x55f47d856130 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800e9b0 con 0x55f47d85bc90 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e5413740 10 client.0 _ll_register_callbacks cb 0x7fd1d4009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1e0bc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d85bc90 0x55f47d866370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d864e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d864e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60700/0 (socket says 172.21.15.64:60700) 2024-05-09T09:18:05.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.049+0000 7fd1dbfff640 1 -- 172.21.15.64:0/4058331094 learned_addr learned my addr 172.21.15.64:0/4058331094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d865360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1dbfff640 1 -- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d85bc90 msgr2=0x55f47d866370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d85bc90 0x55f47d866370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1dbfff640 1 -- 172.21.15.64:0/4058331094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 msgr2=0x55f47d865360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d865360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1dbfff640 1 -- 172.21.15.64:0/4058331094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f47d866960 con 0x55f47d856130 2024-05-09T09:18:05.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1dbfff640 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d864e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0314e0 tx=0x7fd1cc04b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ec2e1c93e0ae75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.050+0000 7fd1bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:05.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.051+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/4058331094 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc0f4040 con 0x55f47d856130 2024-05-09T09:18:05.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.051+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f47d7c26c0 con 0x55f47d856130 2024-05-09T09:18:05.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.051+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/4058331094 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd1cc0f5260 con 0x55f47d856130 2024-05-09T09:18:05.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.051+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/4058331094 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd1cc049040 con 0x55f47d856130 2024-05-09T09:18:05.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.052+0000 7fd1e5413740 10 client.27215 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:05.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.052+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f47d867d20 con 0x55f47d856130 2024-05-09T09:18:05.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.052+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/4058331094 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f47d7c26c0 con 0x55f47d856130 2024-05-09T09:18:05.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/4058331094 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fd1cc0bdff0 con 0x55f47d856130 2024-05-09T09:18:05.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1e5413740 10 client.27215 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:05.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1e5413740 10 client.27215 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:18:05.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1e5413740 20 client.27215 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1e5413740 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d8d2c40 0x55f47d8d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1e5413740 4 client.27215 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:18:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.053+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f47d73f820 con 0x55f47d8d2c40 2024-05-09T09:18:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.056+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d8d2c40 0x55f47d8d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.057+0000 7fd1db7fe640 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d8d2c40 0x55f47d8d5060 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1d3ceeaca1fdb1f server_cookie=b2fc494dd672df55 in_seq=0 out_seq=0 2024-05-09T09:18:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.057+0000 7fd1bb7fe640 10 client.27215 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.057+0000 7fd1bb7fe640 1 -- 172.21.15.64:0/4058331094 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f47d73f820 con 0x55f47d8d2c40 2024-05-09T09:18:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.057+0000 7fd1bb7fe640 10 client.27215 handle_command_reply: tid=0 2024-05-09T09:18:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.057+0000 7fd1e5413740 1 client.27215 shutdown 2024-05-09T09:18:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 msgr2=0x55f47d864e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d864e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0314e0 tx=0x7fd1cc04b000 comp rx=0 tx=0).stop 2024-05-09T09:18:05.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 shutdown_connections 2024-05-09T09:18:05.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55f47d8d2c40 0x55f47d8d5060 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f47d85bc90 0x55f47d866370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f47d856b20 0x55f47d865360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 --2- 172.21.15.64:0/4058331094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f47d856130 0x55f47d864e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 >> 172.21.15.64:0/4058331094 conn(0x55f47d8517a0 msgr2=0x55f47d8509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1bb7fe640 0 client.27215 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.058+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 shutdown_connections 2024-05-09T09:18:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.059+0000 7fd1e5413740 1 -- 172.21.15.64:0/4058331094 wait complete. 2024-05-09T09:18:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.059+0000 7fd1e5413740 20 client.27215 trim_cache size 0 max 16384 2024-05-09T09:18:05.065 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:18:05.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.060+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1c40a2ca0 msgr2=0x7fd1c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.060+0000 7fd1e262a640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1c40a2ca0 0x7fd1c40a5160 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55f47d857380 tx=0x7fd1d4004a40 comp rx=0 tx=0).stop 2024-05-09T09:18:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.061+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 msgr2=0x7fd1dc0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.061+0000 7fd1e262a640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc0702d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000c00 tx=0x7fd1cc003b90 comp rx=0 tx=0).stop 2024-05-09T09:18:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.062+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 shutdown_connections 2024-05-09T09:18:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.062+0000 7fd1e262a640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1c40a2ca0 0x7fd1c40a5160 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.062+0000 7fd1e262a640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1dc109930 0x7fd1dc1adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.062+0000 7fd1e262a640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1dc108f70 0x7fd1dc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.062+0000 7fd1e262a640 1 --2- 172.21.15.64:0/828948313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1dc107dd0 0x7fd1dc0702d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.062+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 >> 172.21.15.64:0/828948313 conn(0x7fd1dc076d00 msgr2=0x7fd1dc10c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.063+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 shutdown_connections 2024-05-09T09:18:05.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.063+0000 7fd1e262a640 1 -- 172.21.15.64:0/828948313 wait complete. 2024-05-09T09:18:05.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:18:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.289+0000 7f7b7359e640 1 -- 172.21.15.64:0/3936848265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b74100770 msgr2=0x7f7b74100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.289+0000 7f7b7359e640 1 --2- 172.21.15.64:0/3936848265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b74100770 0x7f7b74100bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009ca0 tx=0x7f7b6c02a8f0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.292+0000 7f7b7359e640 1 -- 172.21.15.64:0/3936848265 shutdown_connections 2024-05-09T09:18:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.292+0000 7f7b7359e640 1 --2- 172.21.15.64:0/3936848265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b74101130 0x7f7b741059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.292+0000 7f7b7359e640 1 --2- 172.21.15.64:0/3936848265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b74100770 0x7f7b74100bf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.292+0000 7f7b7359e640 1 --2- 172.21.15.64:0/3936848265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b740ffa10 0x7f7b740ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.292+0000 7f7b7359e640 1 -- 172.21.15.64:0/3936848265 >> 172.21.15.64:0/3936848265 conn(0x7f7b740fb360 msgr2=0x7f7b740fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.293+0000 7f7b7359e640 1 -- 172.21.15.64:0/3936848265 shutdown_connections 2024-05-09T09:18:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.293+0000 7f7b7359e640 1 -- 172.21.15.64:0/3936848265 wait complete. 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 Processor -- start 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 -- start start 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b740ffa10 0x7f7b7419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b74100770 0x7f7b7419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 0x7f7b741a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113900 con 0x7f7b74101130 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113780 con 0x7f7b740ffa10 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.295+0000 7f7b7359e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113a80 con 0x7f7b74100770 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 0x7f7b741a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 0x7f7b741a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44818/0 (socket says 172.21.15.64:44818) 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 -- 172.21.15.64:0/2522543002 learned_addr learned my addr 172.21.15.64:0/2522543002 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b7259c640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b740ffa10 0x7f7b7419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 -- 172.21.15.64:0/2522543002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b74100770 msgr2=0x7f7b7419e470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b74100770 0x7f7b7419e470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 -- 172.21.15.64:0/2522543002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b740ffa10 msgr2=0x7f7b7419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b740ffa10 0x7f7b7419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.296+0000 7f7b72d9d640 1 -- 172.21.15.64:0/2522543002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7406b380 con 0x7f7b74101130 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.297+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 0x7f7b741a54e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e970 tx=0x7f7b6400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.297+0000 7f7b637fe640 1 -- 172.21.15.64:0/2522543002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400ccc0 con 0x7f7b74101130 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.297+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7406b670 con 0x7f7b74101130 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.297+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7406bbc0 con 0x7f7b74101130 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.298+0000 7f7b637fe640 1 -- 172.21.15.64:0/2522543002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6400ce60 con 0x7f7b74101130 2024-05-09T09:18:05.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.298+0000 7f7b637fe640 1 -- 172.21.15.64:0/2522543002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400ccc0 con 0x7f7b74101130 2024-05-09T09:18:05.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.302+0000 7f7b7bda5740 1 Processor -- start 2024-05-09T09:18:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.303+0000 7f7b637fe640 1 -- 172.21.15.64:0/2522543002 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b64021440 con 0x7f7b74101130 2024-05-09T09:18:05.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.304+0000 7f7b637fe640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b5c0a2e40 0x7f7b5c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.305+0000 7f7b637fe640 1 -- 172.21.15.64:0/2522543002 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b64099920 con 0x7f7b74101130 2024-05-09T09:18:05.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.305+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b5c0a2e40 0x7f7b5c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.305+0000 7f7b7bda5740 1 -- start start 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d2f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d025c93c40 con 0x55d025d2f130 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d025c3b580 con 0x55d025d34c90 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d025c8fc00 con 0x55d025d2fb20 2024-05-09T09:18:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d2f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d2f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44820/0 (socket says 172.21.15.64:44820) 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.306+0000 7f7b7259c640 1 -- 172.21.15.64:0/3824525075 learned_addr learned my addr 172.21.15.64:0/3824525075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b7259c640 1 -- 172.21.15.64:0/3824525075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 msgr2=0x55d025d34620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b7259c640 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b7259c640 1 -- 172.21.15.64:0/3824525075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 msgr2=0x55d025d37120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b7259c640 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d37120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b7259c640 1 -- 172.21.15.64:0/3824525075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0259a73f0 con 0x55d025d2f130 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.307+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b5c0a2e40 0x7f7b5c0a5300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c004af0 tx=0x7f7b6c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.308+0000 7f7b7259c640 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d2f550 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bd10 tx=0x7f7b6800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5be8ad24a3816d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.308+0000 7f7b60ff9640 1 -- 172.21.15.64:0/3824525075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68015070 con 0x55d025d2f130 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.308+0000 7f7b60ff9640 1 -- 172.21.15.64:0/3824525075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b680118f0 con 0x55d025d2f130 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.308+0000 7f7b60ff9640 1 -- 172.21.15.64:0/3824525075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68010890 con 0x55d025d2f130 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.309+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3824525075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 msgr2=0x55d025d2f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.309+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d2f550 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bd10 tx=0x7f7b6800f730 comp rx=0 tx=0).stop 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3824525075 shutdown_connections 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d37120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d34620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3824525075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d2f550 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3824525075 >> 172.21.15.64:0/3824525075 conn(0x55d025d2a7a0 msgr2=0x55d025d2aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3824525075 shutdown_connections 2024-05-09T09:18:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.310+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3824525075 wait complete. 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 -- start start 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 Processor -- start 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d40b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d025d37c40 con 0x55d025d2f130 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d025c4ec40 con 0x55d025d34c90 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d025d37dc0 con 0x55d025d2fb20 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.311+0000 7f7b7bda5740 10 client.0 _ll_register_callbacks cb 0x55d0259a6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44836/0 (socket says 172.21.15.64:44836) 2024-05-09T09:18:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 -- 172.21.15.64:0/2109484788 learned_addr learned my addr 172.21.15.64:0/2109484788 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d40b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 -- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 msgr2=0x55d025d39af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d39af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 -- 172.21.15.64:0/2109484788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 msgr2=0x55d025d40b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d40b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b7259c640 1 -- 172.21.15.64:0/2109484788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d025a717d0 con 0x55d025d2f130 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.312+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d40b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.313+0000 7f7b7259c640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d395b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800fc30 tx=0x7f7b6800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20232379457ae53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.313+0000 7f7b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.314+0000 7f7b59ffb640 1 -- 172.21.15.64:0/2109484788 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68028060 con 0x55d025d2f130 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.314+0000 7f7b59ffb640 1 -- 172.21.15.64:0/2109484788 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b68011ca0 con 0x55d025d2f130 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.314+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b680118f0 con 0x55d025d2f130 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.314+0000 7f7b59ffb640 1 -- 172.21.15.64:0/2109484788 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7b6802f050 con 0x55d025d2f130 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.314+0000 7f7b7bda5740 10 client.17718 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.314+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d025d43ed0 con 0x55d025d2f130 2024-05-09T09:18:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b59ffb640 1 -- 172.21.15.64:0/2109484788 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b680118f0 con 0x55d025d2f130 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b59ffb640 1 -- 172.21.15.64:0/2109484788 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7b68022030 con 0x55d025d2f130 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b7bda5740 10 client.17718 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b7bda5740 10 client.17718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b7bda5740 20 client.17718 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025d459d0 0x55d025d47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b7bda5740 4 client.17718 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.315+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d025ca1370 con 0x55d025d459d0 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.316+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025d459d0 0x55d025d47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.317+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025d459d0 0x55d025d47e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac6f4b42cbe9afd server_cookie=e414266c3d801b4e in_seq=0 out_seq=0 2024-05-09T09:18:05.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.317+0000 7f7b59ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.317+0000 7f7b59ffb640 1 -- 172.21.15.64:0/2109484788 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d025ca1370 con 0x55d025d459d0 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.317+0000 7f7b59ffb640 10 client.17718 handle_command_reply: tid=0 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.318+0000 7f7b7bda5740 1 client.17718 shutdown 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.318+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 msgr2=0x55d025d395b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.318+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d395b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800fc30 tx=0x7f7b6800ba50 comp rx=0 tx=0).stop 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 shutdown_connections 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d34c90 0x55d025d40b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d39af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025d459d0 0x55d025d47e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/2109484788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d2f130 0x55d025d395b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 >> 172.21.15.64:0/2109484788 conn(0x55d025d2a7a0 msgr2=0x55d025d299c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b59ffb640 0 client.17718 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.319+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 shutdown_connections 2024-05-09T09:18:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.320+0000 7f7b7bda5740 1 -- 172.21.15.64:0/2109484788 wait complete. 2024-05-09T09:18:05.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.320+0000 7f7b7bda5740 20 client.17718 trim_cache size 0 max 16384 2024-05-09T09:18:05.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 Processor -- start 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 -- start start 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d30f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d31490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540124d0 con 0x55d025d34c90 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d30f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b71d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d31490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d30f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60752/0 (socket says 172.21.15.64:60752) 2024-05-09T09:18:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7259c640 1 -- 172.21.15.64:0/695326978 learned_addr learned my addr 172.21.15.64:0/695326978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400fda0 con 0x55d025d2f130 2024-05-09T09:18:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.346+0000 7f7b7bda5740 1 -- 172.21.15.64:0/695326978 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400dbf0 con 0x55d025d2fb20 2024-05-09T09:18:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b71d9b640 1 -- 172.21.15.64:0/695326978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 msgr2=0x55d025d30f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d30f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b71d9b640 1 -- 172.21.15.64:0/695326978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 msgr2=0x55d025d40ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d40ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b71d9b640 1 -- 172.21.15.64:0/695326978 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d025d2b510 con 0x55d025d2fb20 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b72d9d640 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d40ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.347+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d31490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c031ab0 tx=0x7f7b6c03d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119640117837cd40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.349+0000 7f7b5bfff640 1 -- 172.21.15.64:0/695326978 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c003dc0 con 0x55d025d2fb20 2024-05-09T09:18:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.349+0000 7f7b5bfff640 1 -- 172.21.15.64:0/695326978 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6c036050 con 0x55d025d2fb20 2024-05-09T09:18:05.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.349+0000 7f7b5bfff640 1 -- 172.21.15.64:0/695326978 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c030850 con 0x55d025d2fb20 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.350+0000 7f7b7bda5740 1 -- 172.21.15.64:0/695326978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 msgr2=0x55d025d31490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.350+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d31490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c031ab0 tx=0x7f7b6c03d070 comp rx=0 tx=0).stop 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.352+0000 7f7b7bda5740 1 -- 172.21.15.64:0/695326978 shutdown_connections 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.352+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d40ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.352+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d31490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.352+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/695326978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d30f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.352+0000 7f7b7bda5740 1 -- 172.21.15.64:0/695326978 >> 172.21.15.64:0/695326978 conn(0x55d025d2a7a0 msgr2=0x55d025d479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.353+0000 7f7b7bda5740 1 -- 172.21.15.64:0/695326978 shutdown_connections 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.353+0000 7f7b7bda5740 1 -- 172.21.15.64:0/695326978 wait complete. 2024-05-09T09:18:05.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.353+0000 7f7b7bda5740 1 -- start start 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.353+0000 7f7b7bda5740 1 Processor -- start 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d3d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d3d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400e260 con 0x55d025d34c90 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400f6d0 con 0x55d025d2f130 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400e930 con 0x55d025d2fb20 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b7bda5740 10 client.0 _ll_register_callbacks cb 0x7f7b6800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b71d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d3d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b71d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d3d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49606/0 (socket says 172.21.15.64:49606) 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.354+0000 7f7b71d9b640 1 -- 172.21.15.64:0/3621807538 learned_addr learned my addr 172.21.15.64:0/3621807538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b71d9b640 1 -- 172.21.15.64:0/3621807538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 msgr2=0x55d025d405e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d405e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b71d9b640 1 -- 172.21.15.64:0/3621807538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 msgr2=0x55d025d3d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d3d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b71d9b640 1 -- 172.21.15.64:0/3621807538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d025d40c10 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b71d9b640 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d3d080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c02a930 tx=0x7f7b6c0048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec373127061eeeb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.355+0000 7f7b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.356+0000 7f7b59ffb640 1 -- 172.21.15.64:0/3621807538 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c071060 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.356+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c036050 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.356+0000 7f7b59ffb640 1 -- 172.21.15.64:0/3621807538 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b6c068040 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.356+0000 7f7b59ffb640 1 -- 172.21.15.64:0/3621807538 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c004000 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.356+0000 7f7b7bda5740 10 client.27376 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.356+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d025daaee0 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b59ffb640 1 -- 172.21.15.64:0/3621807538 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6c036050 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b59ffb640 1 -- 172.21.15.64:0/3621807538 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7b6c06c480 con 0x55d025d2fb20 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7bda5740 10 client.27376 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7bda5740 10 client.27376 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:18:05.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7bda5740 20 client.27376 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025dab820 0x55d025dadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7bda5740 4 client.27376 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d025c9b6c0 con 0x55d025dab820 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.357+0000 7f7b7259c640 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025dab820 0x55d025dadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.358+0000 7f7b59ffb640 10 client.27376 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.358+0000 7f7b7259c640 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025dab820 0x55d025dadc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dcecd5090dbd43 server_cookie=9661bdd3fc66c114 in_seq=0 out_seq=0 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.358+0000 7f7b59ffb640 1 -- 172.21.15.64:0/3621807538 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d025c9b6c0 con 0x55d025dab820 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.358+0000 7f7b59ffb640 10 client.27376 handle_command_reply: tid=0 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.358+0000 7f7b7bda5740 1 client.27376 shutdown 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 msgr2=0x55d025d3d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d3d080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c02a930 tx=0x7f7b6c0048e0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 shutdown_connections 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d025dab820 0x55d025dadc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d025d34c90 0x55d025d3d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d025d2fb20 0x55d025d3d080 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b59ffb640 0 client.27376 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 --2- 172.21.15.64:0/3621807538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d025d2f130 0x55d025d405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 >> 172.21.15.64:0/3621807538 conn(0x55d025d2a7a0 msgr2=0x55d025d30f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 shutdown_connections 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 1 -- 172.21.15.64:0/3621807538 wait complete. 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.359+0000 7f7b7bda5740 20 client.27376 trim_cache size 0 max 16384 2024-05-09T09:18:05.363 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:18:05.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.360+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b5c0a2e40 msgr2=0x7f7b5c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.360+0000 7f7b7359e640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b5c0a2e40 0x7f7b5c0a5300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c004af0 tx=0x7f7b6c000f40 comp rx=0 tx=0).stop 2024-05-09T09:18:05.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.360+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 msgr2=0x7f7b741a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.360+0000 7f7b7359e640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 0x7f7b741a54e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e970 tx=0x7f7b6400ed70 comp rx=0 tx=0).stop 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 shutdown_connections 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b5c0a2e40 0x7f7b5c0a5300 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74101130 0x7f7b741a54e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b74100770 0x7f7b7419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 --2- 172.21.15.64:0/2522543002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b740ffa10 0x7f7b7419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 >> 172.21.15.64:0/2522543002 conn(0x7f7b740fb360 msgr2=0x7f7b741019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 shutdown_connections 2024-05-09T09:18:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.361+0000 7f7b7359e640 1 -- 172.21.15.64:0/2522543002 wait complete. 2024-05-09T09:18:05.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/3923961627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a5da0 msgr2=0x7f7c8c0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/3923961627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c0a6200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009900 tx=0x7f7c9002a410 comp rx=0 tx=0).stop 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/3923961627 shutdown_connections 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/3923961627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a6740 0x7f7c8c0a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/3923961627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c0a6200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/3923961627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a4c70 0x7f7c8c0a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/3923961627 >> 172.21.15.64:0/3923961627 conn(0x7f7c8c0a0100 msgr2=0x7f7c8c0a2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/3923961627 shutdown_connections 2024-05-09T09:18:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.571+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/3923961627 wait complete. 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 Processor -- start 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 -- start start 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a4c70 0x7f7c8c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c13f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a6740 0x7f7c8c146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0b4b10 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0b4990 con 0x7f7c8c0a6740 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.572+0000 7f7c9b9d5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0b4c90 con 0x7f7c8c0a5da0 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c98f70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c99771640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a4c70 0x7f7c8c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c98f70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c13f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49618/0 (socket says 172.21.15.64:49618) 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c98f70640 1 -- 172.21.15.64:0/345840514 learned_addr learned my addr 172.21.15.64:0/345840514 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c99f72640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a6740 0x7f7c8c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c99771640 1 -- 172.21.15.64:0/345840514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 msgr2=0x7f7c8c13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c99771640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c99771640 1 -- 172.21.15.64:0/345840514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a6740 msgr2=0x7f7c8c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.573+0000 7f7c99771640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a6740 0x7f7c8c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c99771640 1 -- 172.21.15.64:0/345840514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c146de0 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c98f70640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:18:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c99771640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a4c70 0x7f7c8c13f1c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7c94067530 tx=0x7f7c94070800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c8a7fc640 1 -- 172.21.15.64:0/345840514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94078070 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c8a7fc640 1 -- 172.21.15.64:0/345840514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c94074db0 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c1470d0 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c8a7fc640 1 -- 172.21.15.64:0/345840514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940699d0 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.574+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c1475a0 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.579+0000 7f7c9e7bf740 1 Processor -- start 2024-05-09T09:18:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.579+0000 7f7c8a7fc640 1 -- 172.21.15.64:0/345840514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c9407b070 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.579+0000 7f7c9e7bf740 1 -- start start 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd553d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd558e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd55b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5557fd4afc40 con 0x5557fd554330 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5557fd457580 con 0x5557fd553940 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5557fd4abc00 con 0x5557fd5594a0 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c99f72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd553d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c99f72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd553d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60788/0 (socket says 172.21.15.64:60788) 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c99f72640 1 -- 172.21.15.64:0/2950076457 learned_addr learned my addr 172.21.15.64:0/2950076457 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.580+0000 7f7c99771640 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd558e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd55b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c8a7fc640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7c800a2c90 0x7f7c800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c98f70640 1 -- 172.21.15.64:0/2950076457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 msgr2=0x5557fd553d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd553d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c98f70640 1 -- 172.21.15.64:0/2950076457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 msgr2=0x5557fd558e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd558e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c98f70640 1 -- 172.21.15.64:0/2950076457 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fd1c33f0 con 0x5557fd5594a0 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.581+0000 7f7c8a7fc640 1 -- 172.21.15.64:0/345840514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7c94132e00 con 0x7f7c8c0a4c70 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.582+0000 7f7c99771640 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd558e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.582+0000 7f7c99f72640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7c800a2c90 0x7f7c800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.582+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd55b930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c90004770 tx=0x7f7c90032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc5b8f51cd0eff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.582+0000 7f7c99f72640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7c800a2c90 0x7f7c800a5150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5557fd554b90 tx=0x7f7c840046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.586+0000 7f7c7f7fe640 1 -- 172.21.15.64:0/2950076457 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90035070 con 0x5557fd5594a0 2024-05-09T09:18:05.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.586+0000 7f7c7f7fe640 1 -- 172.21.15.64:0/2950076457 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c900035f0 con 0x5557fd5594a0 2024-05-09T09:18:05.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.586+0000 7f7c7f7fe640 1 -- 172.21.15.64:0/2950076457 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90031900 con 0x5557fd5594a0 2024-05-09T09:18:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2950076457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 msgr2=0x5557fd55b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd55b930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c90004770 tx=0x7f7c90032f60 comp rx=0 tx=0).stop 2024-05-09T09:18:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2950076457 shutdown_connections 2024-05-09T09:18:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd55b930 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd558e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2950076457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd553d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2950076457 >> 172.21.15.64:0/2950076457 conn(0x5557fd54efb0 msgr2=0x5557fd54f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2950076457 shutdown_connections 2024-05-09T09:18:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.589+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2950076457 wait complete. 2024-05-09T09:18:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.590+0000 7f7c9e7bf740 1 -- start start 2024-05-09T09:18:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.590+0000 7f7c9e7bf740 1 Processor -- start 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.591+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.591+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd55e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.591+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd5594a0 0x5557fd5652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.591+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5557fd55c3f0 con 0x5557fd554330 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.591+0000 7f7c98f70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd55e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd55e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44878/0 (socket says 172.21.15.64:44878) 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 -- 172.21.15.64:0/2489065542 learned_addr learned my addr 172.21.15.64:0/2489065542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c99f72640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd5594a0 0x5557fd5652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c99771640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5557fd46ac40 con 0x5557fd5594a0 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5557fd55c570 con 0x5557fd553940 2024-05-09T09:18:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c9e7bf740 10 client.0 _ll_register_callbacks cb 0x5557fd1c2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 -- 172.21.15.64:0/2489065542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 msgr2=0x5557fd55dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 -- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd5594a0 msgr2=0x5557fd5652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd5594a0 0x5557fd5652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.592+0000 7f7c98f70640 1 -- 172.21.15.64:0/2489065542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fd28d7d0 con 0x5557fd554330 2024-05-09T09:18:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.593+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd55e290 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c90005940 tx=0x7f7c90031790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e134fcbd38873e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.593+0000 7f7c7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.593+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c900035f0 con 0x5557fd554330 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.593+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2489065542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90053060 con 0x5557fd554330 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.593+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2489065542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7c900037a0 con 0x5557fd554330 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.593+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2489065542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c9005a050 con 0x5557fd554330 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.594+0000 7f7c9e7bf740 10 client.17730 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.594+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5557fd5684c0 con 0x5557fd554330 2024-05-09T09:18:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.594+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2489065542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7c9004d030 con 0x5557fd554330 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.595+0000 7f7c9e7bf740 10 client.17730 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.595+0000 7f7c9e7bf740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.595+0000 7f7c9e7bf740 20 client.17730 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.595+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd569810 0x5557fd56bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.595+0000 7f7c9e7bf740 4 client.17730 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.595+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c900035f0 con 0x5557fd569810 2024-05-09T09:18:05.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.596+0000 7f7c99771640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd569810 0x5557fd56bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.597+0000 7f7c99771640 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd569810 0x5557fd56bc00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe7158418d185c7 server_cookie=f0010800e7689b9c in_seq=0 out_seq=0 2024-05-09T09:18:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.597+0000 7f7c7cff9640 10 client.17730 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.598+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2489065542 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7c900035f0 con 0x5557fd569810 2024-05-09T09:18:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.598+0000 7f7c7cff9640 10 client.17730 handle_command_reply: tid=0 2024-05-09T09:18:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.598+0000 7f7c9e7bf740 1 client.17730 shutdown 2024-05-09T09:18:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 msgr2=0x5557fd55e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd55e290 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c90005940 tx=0x7f7c90031790 comp rx=0 tx=0).stop 2024-05-09T09:18:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 shutdown_connections 2024-05-09T09:18:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd569810 0x5557fd56bc00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd5594a0 0x5557fd5652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd55e290 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2489065542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.600+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 >> 172.21.15.64:0/2489065542 conn(0x5557fd54efb0 msgr2=0x5557fd5459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.601+0000 7f7c7cff9640 0 client.17730 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.601+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 shutdown_connections 2024-05-09T09:18:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.602+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2489065542 wait complete. 2024-05-09T09:18:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.602+0000 7f7c9e7bf740 20 client.17730 trim_cache size 0 max 16384 2024-05-09T09:18:05.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.634+0000 7f7c9e7bf740 1 Processor -- start 2024-05-09T09:18:05.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.635+0000 7f7c9e7bf740 1 -- start start 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd554330 0x5557fd565290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd5594a0 0x5557fd56b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c14ff70 con 0x5557fd5594a0 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c14d840 con 0x5557fd554330 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c14b6d0 con 0x5557fd553940 2024-05-09T09:18:05.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c99771640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c99771640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49646/0 (socket says 172.21.15.64:49646) 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c99771640 1 -- 172.21.15.64:0/1760492060 learned_addr learned my addr 172.21.15.64:0/1760492060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c98f70640 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd554330 0x5557fd565290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.636+0000 7f7c99f72640 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd5594a0 0x5557fd56b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.637+0000 7f7c98f70640 1 -- 172.21.15.64:0/1760492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 msgr2=0x5557fd55e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.637+0000 7f7c98f70640 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.637+0000 7f7c98f70640 1 -- 172.21.15.64:0/1760492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd5594a0 msgr2=0x5557fd56b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.637+0000 7f7c98f70640 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd5594a0 0x5557fd56b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.637+0000 7f7c98f70640 1 -- 172.21.15.64:0/1760492060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fd54fd20 con 0x5557fd554330 2024-05-09T09:18:05.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.637+0000 7f7c98f70640 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd554330 0x5557fd565290 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009a20 tx=0x7f7c90035000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=586f72d097f94314 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.639+0000 7f7c7effd640 1 -- 172.21.15.64:0/1760492060 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90053020 con 0x5557fd554330 2024-05-09T09:18:05.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.639+0000 7f7c7effd640 1 -- 172.21.15.64:0/1760492060 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c90003e30 con 0x5557fd554330 2024-05-09T09:18:05.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.639+0000 7f7c7effd640 1 -- 172.21.15.64:0/1760492060 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90035070 con 0x5557fd554330 2024-05-09T09:18:05.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.640+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/1760492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd554330 msgr2=0x5557fd565290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.640+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd554330 0x5557fd565290 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009a20 tx=0x7f7c90035000 comp rx=0 tx=0).stop 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/1760492060 shutdown_connections 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd5594a0 0x5557fd56b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd554330 0x5557fd565290 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/1760492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd553940 0x5557fd55e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/1760492060 >> 172.21.15.64:0/1760492060 conn(0x5557fd54efb0 msgr2=0x5557fd545130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/1760492060 shutdown_connections 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.644+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/1760492060 wait complete. 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 -- start start 2024-05-09T09:18:05.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 Processor -- start 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd5630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd563600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd5645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c14bd20 con 0x5557fd554330 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c14d170 con 0x5557fd553940 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.645+0000 7f7c9e7bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c14c3d0 con 0x5557fd5594a0 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c9e7bf740 10 client.0 _ll_register_callbacks cb 0x7f7c84004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd5630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd5630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60806/0 (socket says 172.21.15.64:60806) 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 -- 172.21.15.64:0/2352902145 learned_addr learned my addr 172.21.15.64:0/2352902145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd563600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99f72640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd5645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 -- 172.21.15.64:0/2352902145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 msgr2=0x5557fd5645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd5645c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 -- 172.21.15.64:0/2352902145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 msgr2=0x5557fd563600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd563600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.646+0000 7f7c99771640 1 -- 172.21.15.64:0/2352902145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fd564bb0 con 0x5557fd553940 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c99771640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd5630c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7c940673a0 tx=0x7f7c9404ffe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e040eb3a0b37102 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:05.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2352902145 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940ff320 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2352902145 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7c940fb990 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c90003e30 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2352902145 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c94103620 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c9e7bf740 10 client.27245 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.647+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5557fd5d01c0 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2352902145 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c90003e30 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2352902145 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7c94137040 con 0x5557fd553940 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c9e7bf740 10 client.27245 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c9e7bf740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:05.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c9e7bf740 20 client.27245 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd5d0ba0 0x5557fd5d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c9e7bf740 4 client.27245 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557fd4bd370 con 0x5557fd5d0ba0 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.648+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd5d0ba0 0x5557fd5d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.649+0000 7f7c98f70640 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd5d0ba0 0x5557fd5d2f90 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92512ed1d6a1e52 server_cookie=c442d713ef64c196 in_seq=0 out_seq=0 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.649+0000 7f7c7cff9640 10 client.27245 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.649+0000 7f7c7cff9640 1 -- 172.21.15.64:0/2352902145 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557fd4bd370 con 0x5557fd5d0ba0 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.649+0000 7f7c7cff9640 10 client.27245 handle_command_reply: tid=0 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.649+0000 7f7c9e7bf740 1 client.27245 shutdown 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 msgr2=0x5557fd5630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd5630c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7c940673a0 tx=0x7f7c9404ffe0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 shutdown_connections 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5557fd5d0ba0 0x5557fd5d2f90 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557fd5594a0 0x5557fd5645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557fd554330 0x5557fd563600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 --2- 172.21.15.64:0/2352902145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fd553940 0x5557fd5630c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 >> 172.21.15.64:0/2352902145 conn(0x5557fd54efb0 msgr2=0x5557fd55e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c7cff9640 0 client.27245 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:05.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 shutdown_connections 2024-05-09T09:18:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 1 -- 172.21.15.64:0/2352902145 wait complete. 2024-05-09T09:18:05.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.650+0000 7f7c9e7bf740 20 client.27245 trim_cache size 0 max 16384 2024-05-09T09:18:05.652 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.651+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7c800a2c90 msgr2=0x7f7c800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7c800a2c90 0x7f7c800a5150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5557fd554b90 tx=0x7f7c840046c0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a4c70 msgr2=0x7f7c8c13f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a4c70 0x7f7c8c13f1c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7c94067530 tx=0x7f7c94070800 comp rx=0 tx=0).stop 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 shutdown_connections 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7c800a2c90 0x7f7c800a5150 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c8c0a6740 0x7f7c8c146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c8c0a5da0 0x7f7c8c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 --2- 172.21.15.64:0/345840514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c8c0a4c70 0x7f7c8c13f1c0 secure :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7c94067530 tx=0x7f7c94070800 comp rx=0 tx=0).stop 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.652+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 >> 172.21.15.64:0/345840514 conn(0x7f7c8c0a0100 msgr2=0x7f7c8c0a1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.653+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 shutdown_connections 2024-05-09T09:18:05.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:05.653+0000 7f7c9b9d5640 1 -- 172.21.15.64:0/345840514 wait complete. 2024-05-09T09:18:05.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:05 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4071671187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:18:05.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:05 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4071671187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:18:05.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:05 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4071671187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:18:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 465.0 s, 779.2 tps, lat 128.581 ms stddev 43.279 2024-05-09T09:18:06.672 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:18:06.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:18:06.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:18:06.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:06 smithi064 ceph-mon[30149]: pgmap v639: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 MiB/s wr, 905 op/s 2024-05-09T09:18:06.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:06 smithi179 ceph-mon[34086]: pgmap v639: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 MiB/s wr, 905 op/s 2024-05-09T09:18:06.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:06 smithi139 ceph-mon[33599]: pgmap v639: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 MiB/s wr, 905 op/s 2024-05-09T09:18:06.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.887+0000 7ffbbca01640 1 -- 172.21.15.64:0/1762404799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb81016a0 msgr2=0x7ffbb8101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.887+0000 7ffbbca01640 1 --2- 172.21.15.64:0/1762404799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb81016a0 0x7ffbb8101ac0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffba800fb60 tx=0x7ffba802e4f0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.888+0000 7ffbbca01640 1 -- 172.21.15.64:0/1762404799 shutdown_connections 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.888+0000 7ffbbca01640 1 --2- 172.21.15.64:0/1762404799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbb8103250 0x7ffbb8109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.888+0000 7ffbbca01640 1 --2- 172.21.15.64:0/1762404799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbb8102890 0x7ffbb8102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.888+0000 7ffbbca01640 1 --2- 172.21.15.64:0/1762404799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb81016a0 0x7ffbb8101ac0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.888+0000 7ffbbca01640 1 -- 172.21.15.64:0/1762404799 >> 172.21.15.64:0/1762404799 conn(0x7ffbb80fcc10 msgr2=0x7ffbb80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:06.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.888+0000 7ffbbca01640 1 -- 172.21.15.64:0/1762404799 shutdown_connections 2024-05-09T09:18:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.889+0000 7ffbbca01640 1 -- 172.21.15.64:0/1762404799 wait complete. 2024-05-09T09:18:06.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.889+0000 7ffbbca01640 1 Processor -- start 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 -- start start 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbb8102890 0x7ffbb8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbb8103250 0x7ffbb80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 0x7ffbb80728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb806b720 con 0x7ffbb8103250 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb806b5a0 con 0x7ffbb8072440 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbbca01640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb806b8a0 con 0x7ffbb8102890 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbb759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 0x7ffbb80728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbb759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 0x7ffbb80728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60826/0 (socket says 172.21.15.64:60826) 2024-05-09T09:18:06.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.890+0000 7ffbb759e640 1 -- 172.21.15.64:0/972457576 learned_addr learned my addr 172.21.15.64:0/972457576 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.891+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbb8102890 0x7ffbb8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.891+0000 7ffbb759e640 1 -- 172.21.15.64:0/972457576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbb8102890 msgr2=0x7ffbb8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.891+0000 7ffbb759e640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbb8102890 0x7ffbb8070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.891+0000 7ffbb759e640 1 -- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbb8103250 msgr2=0x7ffbb80713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.891+0000 7ffbb759e640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbb8103250 0x7ffbb80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.891+0000 7ffbb759e640 1 -- 172.21.15.64:0/972457576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb8072f30 con 0x7ffbb8072440 2024-05-09T09:18:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.892+0000 7ffbb759e640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 0x7ffbb80728f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b4a0 tx=0x7ffbac00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.892+0000 7ffb9ffff640 1 -- 172.21.15.64:0/972457576 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac021070 con 0x7ffbb8072440 2024-05-09T09:18:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.892+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb8073220 con 0x7ffbb8072440 2024-05-09T09:18:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.892+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb8073740 con 0x7ffbb8072440 2024-05-09T09:18:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.892+0000 7ffb9ffff640 1 -- 172.21.15.64:0/972457576 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbac01c8f0 con 0x7ffbb8072440 2024-05-09T09:18:06.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.892+0000 7ffb9ffff640 1 -- 172.21.15.64:0/972457576 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac01a960 con 0x7ffbb8072440 2024-05-09T09:18:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.896+0000 7ffb9ffff640 1 -- 172.21.15.64:0/972457576 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffbac024070 con 0x7ffbb8072440 2024-05-09T09:18:06.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.896+0000 7ffbbffea740 1 Processor -- start 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 -- start start 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24ee9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24eeebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24ef16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555a24e4d990 con 0x555a24eea0c0 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555a24df55b0 con 0x555a24eef230 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffbbffea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555a24e49950 con 0x555a24ee96d0 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.897+0000 7ffb9ffff640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffba00a2cd0 0x7ffba00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.898+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24ee9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.898+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24ef16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.898+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24ee9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49670/0 (socket says 172.21.15.64:49670) 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.898+0000 7ffbb659c640 1 -- 172.21.15.64:0/4112548103 learned_addr learned my addr 172.21.15.64:0/4112548103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.898+0000 7ffb9ffff640 1 -- 172.21.15.64:0/972457576 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffbac0d69c0 con 0x7ffbb8072440 2024-05-09T09:18:06.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.898+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24eeebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.899+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/4112548103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 msgr2=0x555a24ee9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.899+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24ee9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.899+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/4112548103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 msgr2=0x555a24ef16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.899+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24ef16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.899+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/4112548103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a24c297e0 con 0x555a24eea0c0 2024-05-09T09:18:06.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.900+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24eeebc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ffba802eae0 tx=0x7ffba80343a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62838995496eb15d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.901+0000 7ffb9d7fa640 1 -- 172.21.15.64:0/4112548103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8042020 con 0x555a24eea0c0 2024-05-09T09:18:06.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.901+0000 7ffb9d7fa640 1 -- 172.21.15.64:0/4112548103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba80349c0 con 0x555a24eea0c0 2024-05-09T09:18:06.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.901+0000 7ffb9d7fa640 1 -- 172.21.15.64:0/4112548103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba803ea30 con 0x555a24eea0c0 2024-05-09T09:18:06.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.902+0000 7ffbbffea740 1 -- 172.21.15.64:0/4112548103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 msgr2=0x555a24eeebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.902+0000 7ffbbffea740 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24eeebc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ffba802eae0 tx=0x7ffba80343a0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.903+0000 7ffbb759e640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffba00a2cd0 0x7ffba00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.903+0000 7ffbbffea740 1 -- 172.21.15.64:0/4112548103 shutdown_connections 2024-05-09T09:18:06.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.903+0000 7ffbbffea740 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24ef16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.903+0000 7ffbbffea740 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24eeebc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.903+0000 7ffbbffea740 1 --2- 172.21.15.64:0/4112548103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24ee9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.903+0000 7ffbbffea740 1 -- 172.21.15.64:0/4112548103 >> 172.21.15.64:0/4112548103 conn(0x555a24eb10a0 msgr2=0x555a24ee5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:06.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.906+0000 7ffbbffea740 1 -- 172.21.15.64:0/4112548103 shutdown_connections 2024-05-09T09:18:06.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.907+0000 7ffbb759e640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffba00a2cd0 0x7ffba00a5190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffbac014f80 tx=0x7ffbac014f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.907+0000 7ffbbffea740 1 -- 172.21.15.64:0/4112548103 wait complete. 2024-05-09T09:18:06.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.908+0000 7ffbbffea740 1 -- start start 2024-05-09T09:18:06.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.908+0000 7ffbbffea740 1 Processor -- start 2024-05-09T09:18:06.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24ee96d0 0x555a24f04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eef230 0x555a24f0c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555a24f031e0 con 0x555a24eea0c0 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555a24e08b40 con 0x555a24ee96d0 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555a24f03360 con 0x555a24eef230 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbbffea740 10 client.0 _ll_register_callbacks cb 0x555a24dfd0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbb6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbb6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f05080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44940/0 (socket says 172.21.15.64:44940) 2024-05-09T09:18:06.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.909+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/639583476 learned_addr learned my addr 172.21.15.64:0/639583476 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:06.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.910+0000 7ffbb659c640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24ee96d0 0x555a24f04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.910+0000 7ffbb659c640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eef230 0x555a24f0c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.912+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/639583476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eef230 msgr2=0x555a24f0c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.912+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eef230 0x555a24f0c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.912+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24ee96d0 msgr2=0x555a24f04b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.912+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24ee96d0 0x555a24f04b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.912+0000 7ffbb6d9d640 1 -- 172.21.15.64:0/639583476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a24c89f70 con 0x555a24eea0c0 2024-05-09T09:18:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.913+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f05080 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffba800fa80 tx=0x7ffba8034740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0d8dfec195ecb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.913+0000 7ffb9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.914+0000 7ffb9aff5640 1 -- 172.21.15.64:0/639583476 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8042070 con 0x555a24eea0c0 2024-05-09T09:18:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.914+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba80349c0 con 0x555a24eea0c0 2024-05-09T09:18:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.914+0000 7ffb9aff5640 1 -- 172.21.15.64:0/639583476 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffba800f5d0 con 0x555a24eea0c0 2024-05-09T09:18:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.914+0000 7ffb9aff5640 1 -- 172.21.15.64:0/639583476 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffba8057050 con 0x555a24eea0c0 2024-05-09T09:18:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.916+0000 7ffbbffea740 10 client.17745 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.916+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555a24b21170 con 0x555a24eea0c0 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.918+0000 7ffb9aff5640 1 -- 172.21.15.64:0/639583476 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba80349c0 con 0x555a24eea0c0 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.918+0000 7ffb9aff5640 1 -- 172.21.15.64:0/639583476 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ffba803eae0 con 0x555a24eea0c0 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.918+0000 7ffbbffea740 10 client.17745 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.918+0000 7ffbbffea740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.918+0000 7ffbbffea740 20 client.17745 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.918+0000 7ffbbffea740 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f10760 0x555a24f12be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.919+0000 7ffbbffea740 4 client.17745 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:06.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.919+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a24e5b0c0 con 0x555a24f10760 2024-05-09T09:18:06.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.919+0000 7ffbb659c640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f10760 0x555a24f12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.931+0000 7ffbb659c640 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f10760 0x555a24f12be0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba67049e99627a8 server_cookie=bc57e2156a1b2919 in_seq=0 out_seq=0 2024-05-09T09:18:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.933+0000 7ffb9aff5640 10 client.17745 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.933+0000 7ffb9aff5640 1 -- 172.21.15.64:0/639583476 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a24e5b0c0 con 0x555a24f10760 2024-05-09T09:18:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.933+0000 7ffb9aff5640 10 client.17745 handle_command_reply: tid=0 2024-05-09T09:18:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.933+0000 7ffbbffea740 1 client.17745 shutdown 2024-05-09T09:18:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.939+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 msgr2=0x555a24f05080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.939+0000 7ffbbffea740 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f05080 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffba800fa80 tx=0x7ffba8034740 comp rx=0 tx=0).stop 2024-05-09T09:18:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 shutdown_connections 2024-05-09T09:18:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffbbffea740 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f10760 0x555a24f12be0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffbbffea740 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eef230 0x555a24f0c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffbbffea740 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f05080 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffbbffea740 1 --2- 172.21.15.64:0/639583476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24ee96d0 0x555a24f04b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 >> 172.21.15.64:0/639583476 conn(0x555a24eb10a0 msgr2=0x555a24ee4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.941+0000 7ffb9aff5640 0 client.17745 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:06.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.949+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 shutdown_connections 2024-05-09T09:18:06.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.949+0000 7ffbbffea740 1 -- 172.21.15.64:0/639583476 wait complete. 2024-05-09T09:18:06.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.949+0000 7ffbbffea740 20 client.17745 trim_cache size 0 max 16384 2024-05-09T09:18:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.993+0000 7ffbbffea740 1 Processor -- start 2024-05-09T09:18:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.994+0000 7ffbbffea740 1 -- start start 2024-05-09T09:18:06.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.994+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 0x555a24ee4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.994+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eea0c0 0x555a24ee4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.994+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 0x555a24ee4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.994+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 0x555a24ee4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44946/0 (socket says 172.21.15.64:44946) 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.995+0000 7ffbb659c640 1 -- 172.21.15.64:0/2404149765 learned_addr learned my addr 172.21.15.64:0/2404149765 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.995+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.995+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eea0c0 0x555a24ee4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.995+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40124d0 con 0x555a24ee96d0 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.995+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffba400fda0 con 0x555a24eef230 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.995+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffba400dbf0 con 0x555a24eea0c0 2024-05-09T09:18:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.996+0000 7ffbb659c640 1 -- 172.21.15.64:0/2404149765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eea0c0 msgr2=0x555a24ee4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.996+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eea0c0 0x555a24ee4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.996+0000 7ffbb659c640 1 -- 172.21.15.64:0/2404149765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 msgr2=0x555a24f0c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.996+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.996+0000 7ffbb659c640 1 -- 172.21.15.64:0/2404149765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a24ee9020 con 0x555a24ee96d0 2024-05-09T09:18:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.996+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 0x555a24ee4110 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005fd0 tx=0x7ffbb00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a253947fb4defb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.997+0000 7ffb9cff9640 1 -- 172.21.15.64:0/2404149765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0009b40 con 0x555a24ee96d0 2024-05-09T09:18:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.997+0000 7ffb9cff9640 1 -- 172.21.15.64:0/2404149765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbb0003e20 con 0x555a24ee96d0 2024-05-09T09:18:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.997+0000 7ffb9cff9640 1 -- 172.21.15.64:0/2404149765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004020 con 0x555a24ee96d0 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 msgr2=0x555a24ee4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 0x555a24ee4110 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005fd0 tx=0x7ffbb00095c0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 shutdown_connections 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24eea0c0 0x555a24ee4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2404149765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24ee96d0 0x555a24ee4110 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.998+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 >> 172.21.15.64:0/2404149765 conn(0x555a24eb10a0 msgr2=0x555a24ee3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.999+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 shutdown_connections 2024-05-09T09:18:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.999+0000 7ffbbffea740 1 -- 172.21.15.64:0/2404149765 wait complete. 2024-05-09T09:18:06.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.999+0000 7ffbbffea740 1 -- start start 2024-05-09T09:18:06.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:06.999+0000 7ffbbffea740 1 Processor -- start 2024-05-09T09:18:06.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24f0be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:06.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f0a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffba400e260 con 0x555a24eea0c0 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffba400f6d0 con 0x555a24eef230 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffba400e930 con 0x555a24ee96d0 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbbffea740 10 client.0 _ll_register_callbacks cb 0x7ffbb0010470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24f0be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24f0be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49678/0 (socket says 172.21.15.64:49678) 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbb659c640 1 -- 172.21.15.64:0/2597303967 learned_addr learned my addr 172.21.15.64:0/2597303967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f0a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.000+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffbb659c640 1 -- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 msgr2=0x555a24f0a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffbb659c640 1 -- 172.21.15.64:0/2597303967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 msgr2=0x555a24f0a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f0a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffbb659c640 1 -- 172.21.15.64:0/2597303967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a24f0b070 con 0x555a24ee96d0 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffbb659c640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24f0be30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0024040 tx=0x7ffbb0021950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4166df210bebac39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.001+0000 7ffb9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.002+0000 7ffb9aff5640 1 -- 172.21.15.64:0/2597303967 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004000 con 0x555a24ee96d0 2024-05-09T09:18:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.002+0000 7ffb9aff5640 1 -- 172.21.15.64:0/2597303967 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ffbb00041a0 con 0x555a24ee96d0 2024-05-09T09:18:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.002+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb0003e20 con 0x555a24ee96d0 2024-05-09T09:18:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.003+0000 7ffb9aff5640 1 -- 172.21.15.64:0/2597303967 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffbb0034050 con 0x555a24ee96d0 2024-05-09T09:18:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.003+0000 7ffbbffea740 10 client.27403 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.003+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555a24f77c10 con 0x555a24ee96d0 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffb9aff5640 1 -- 172.21.15.64:0/2597303967 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbb0003e20 con 0x555a24ee96d0 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffb9aff5640 1 -- 172.21.15.64:0/2597303967 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ffbb000b040 con 0x555a24ee96d0 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffbbffea740 10 client.27403 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffbbffea740 10 client.27403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffbbffea740 20 client.27403 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f78470 0x555a24f7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffbbffea740 4 client.27403 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.006+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a24e55410 con 0x555a24f78470 2024-05-09T09:18:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.007+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f78470 0x555a24f7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.008+0000 7ffbb6d9d640 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f78470 0x555a24f7a890 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06b71fefa683975 server_cookie=f0f27725391228fb in_seq=0 out_seq=0 2024-05-09T09:18:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.008+0000 7ffb9aff5640 10 client.27403 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.012+0000 7ffb9aff5640 1 -- 172.21.15.64:0/2597303967 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a24e55410 con 0x555a24f78470 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.012+0000 7ffb9aff5640 10 client.27403 handle_command_reply: tid=0 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.012+0000 7ffbbffea740 1 client.27403 shutdown 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.012+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 msgr2=0x555a24f0be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.012+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24f0be30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0024040 tx=0x7ffbb0021950 comp rx=0 tx=0).stop 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 shutdown_connections 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555a24f78470 0x555a24f7a890 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a24eef230 0x555a24f0a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffb9aff5640 0 client.27403 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a24eea0c0 0x555a24f0a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 --2- 172.21.15.64:0/2597303967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a24ee96d0 0x555a24f0be30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 >> 172.21.15.64:0/2597303967 conn(0x555a24eb10a0 msgr2=0x555a24ee88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 shutdown_connections 2024-05-09T09:18:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.013+0000 7ffbbffea740 1 -- 172.21.15.64:0/2597303967 wait complete. 2024-05-09T09:18:07.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.014+0000 7ffbbffea740 20 client.27403 trim_cache size 0 max 16384 2024-05-09T09:18:07.014 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:18:07.014 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:18:07.014 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "ce4d9c10-c387-4689-8fa3-43df962b8f53", 2024-05-09T09:18:07.014 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:18:07.014 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:18:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.017+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffba00a2cd0 msgr2=0x7ffba00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffba00a2cd0 0x7ffba00a5190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffbac014f80 tx=0x7ffbac014f10 comp rx=0 tx=0).stop 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 msgr2=0x7ffbb80728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 0x7ffbb80728f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b4a0 tx=0x7ffbac00b960 comp rx=0 tx=0).stop 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 shutdown_connections 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ffba00a2cd0 0x7ffba00a5190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbb8072440 0x7ffbb80728f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbb8103250 0x7ffbb80713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 --2- 172.21.15.64:0/972457576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbb8102890 0x7ffbb8070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.018+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 >> 172.21.15.64:0/972457576 conn(0x7ffbb80fcc10 msgr2=0x7ffbb8105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.019+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 shutdown_connections 2024-05-09T09:18:07.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.019+0000 7ffbbca01640 1 -- 172.21.15.64:0/972457576 wait complete. 2024-05-09T09:18:07.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:18:07.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:18:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.232+0000 7fb245b43640 1 -- 172.21.15.64:0/1851006468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb240108f10 msgr2=0x7fb240109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.232+0000 7fb245b43640 1 --2- 172.21.15.64:0/1851006468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb240108f10 0x7fb240109370 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb23801acb0 tx=0x7fb23803b8e0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.236+0000 7fb245b43640 1 -- 172.21.15.64:0/1851006468 shutdown_connections 2024-05-09T09:18:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.236+0000 7fb245b43640 1 --2- 172.21.15.64:0/1851006468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2401098b0 0x7fb24010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.236+0000 7fb245b43640 1 --2- 172.21.15.64:0/1851006468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb240108f10 0x7fb240109370 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.236+0000 7fb245b43640 1 --2- 172.21.15.64:0/1851006468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 0x7fb240073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.236+0000 7fb245b43640 1 -- 172.21.15.64:0/1851006468 >> 172.21.15.64:0/1851006468 conn(0x7fb24006e740 msgr2=0x7fb240070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.237+0000 7fb245b43640 1 -- 172.21.15.64:0/1851006468 shutdown_connections 2024-05-09T09:18:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.237+0000 7fb245b43640 1 -- 172.21.15.64:0/1851006468 wait complete. 2024-05-09T09:18:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.238+0000 7fb245b43640 1 Processor -- start 2024-05-09T09:18:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.238+0000 7fb245b43640 1 -- start start 2024-05-09T09:18:07.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245b43640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 0x7fb24019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245b43640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240108f10 0x7fb24019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245b43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2401098b0 0x7fb2401a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245b43640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb240111a00 con 0x7fb2401098b0 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245b43640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb240111880 con 0x7fb240108f10 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245b43640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb240111b80 con 0x7fb240072c70 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb244b41640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 0x7fb24019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb244b41640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 0x7fb24019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49710/0 (socket says 172.21.15.64:49710) 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240108f10 0x7fb24019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb244b41640 1 -- 172.21.15.64:0/701307853 learned_addr learned my addr 172.21.15.64:0/701307853 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.239+0000 7fb245342640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2401098b0 0x7fb2401a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.240+0000 7fb23ffff640 1 -- 172.21.15.64:0/701307853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 msgr2=0x7fb24019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.240+0000 7fb23ffff640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 0x7fb24019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.240+0000 7fb23ffff640 1 -- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2401098b0 msgr2=0x7fb2401a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.240+0000 7fb23ffff640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2401098b0 0x7fb2401a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.240+0000 7fb23ffff640 1 -- 172.21.15.64:0/701307853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2401a5bc0 con 0x7fb240108f10 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.240+0000 7fb23ffff640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240108f10 0x7fb24019e4e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb23803bed0 tx=0x7fb238003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.241+0000 7fb23dffb640 1 -- 172.21.15.64:0/701307853 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238050070 con 0x7fb240108f10 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.241+0000 7fb23dffb640 1 -- 172.21.15.64:0/701307853 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2380044f0 con 0x7fb240108f10 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.241+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2401a5e50 con 0x7fb240108f10 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.241+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2401a63e0 con 0x7fb240108f10 2024-05-09T09:18:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.241+0000 7fb23dffb640 1 -- 172.21.15.64:0/701307853 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238041d00 con 0x7fb240108f10 2024-05-09T09:18:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.245+0000 7fb249b8d740 1 Processor -- start 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.246+0000 7fb249b8d740 1 -- start start 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.246+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b25120 0x5647a6b25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 0x5647a6b2a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b2ac80 0x5647a6b2d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb249b8d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647a6a89d00 con 0x5647a6b2ac80 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb249b8d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5647a6a314c0 con 0x5647a6b25b10 2024-05-09T09:18:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb249b8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5647a6a85cc0 con 0x5647a6b25120 2024-05-09T09:18:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb245342640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b25120 0x5647a6b25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.247+0000 7fb244b41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 0x5647a6b2a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.248+0000 7fb244b41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 0x5647a6b2a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60878/0 (socket says 172.21.15.64:60878) 2024-05-09T09:18:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.248+0000 7fb244b41640 1 -- 172.21.15.64:0/3844356935 learned_addr learned my addr 172.21.15.64:0/3844356935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.248+0000 7fb23dffb640 1 -- 172.21.15.64:0/701307853 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb238055460 con 0x7fb240108f10 2024-05-09T09:18:07.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.250+0000 7fb23ffff640 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b2ac80 0x5647a6b2d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.251+0000 7fb244b41640 1 -- 172.21.15.64:0/3844356935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b25120 msgr2=0x5647a6b25540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.251+0000 7fb244b41640 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b25120 0x5647a6b25540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.251+0000 7fb244b41640 1 -- 172.21.15.64:0/3844356935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b2ac80 msgr2=0x5647a6b2d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.251+0000 7fb244b41640 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b2ac80 0x5647a6b2d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.251+0000 7fb244b41640 1 -- 172.21.15.64:0/3844356935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a68677d0 con 0x5647a6b25b10 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.251+0000 7fb244b41640 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 0x5647a6b2a610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb234009b20 tx=0x7fb23400e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13f9b11a8c080be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.253+0000 7fb226ffd640 1 -- 172.21.15.64:0/3844356935 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234017070 con 0x5647a6b25b10 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.253+0000 7fb226ffd640 1 -- 172.21.15.64:0/3844356935 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb23400ec10 con 0x5647a6b25b10 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.253+0000 7fb226ffd640 1 -- 172.21.15.64:0/3844356935 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234012630 con 0x5647a6b25b10 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.253+0000 7fb23dffb640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2280a2ce0 0x7fb2280a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.253+0000 7fb23dffb640 1 -- 172.21.15.64:0/701307853 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2381011b0 con 0x7fb240108f10 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 -- 172.21.15.64:0/3844356935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 msgr2=0x5647a6b2a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 0x5647a6b2a610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb234009b20 tx=0x7fb23400e1d0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 -- 172.21.15.64:0/3844356935 shutdown_connections 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b2ac80 0x5647a6b2d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25b10 0x5647a6b2a610 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3844356935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b25120 0x5647a6b25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.254+0000 7fb249b8d740 1 -- 172.21.15.64:0/3844356935 >> 172.21.15.64:0/3844356935 conn(0x5647a6b20790 msgr2=0x5647a6b20b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.255+0000 7fb249b8d740 1 -- 172.21.15.64:0/3844356935 shutdown_connections 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.255+0000 7fb245342640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2280a2ce0 0x7fb2280a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.256+0000 7fb249b8d740 1 -- 172.21.15.64:0/3844356935 wait complete. 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.256+0000 7fb249b8d740 1 -- start start 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.256+0000 7fb249b8d740 1 Processor -- start 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b2f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b2fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647a6b2dc30 con 0x5647a6b25b10 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5647a6a44b90 con 0x5647a6b25120 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5647a6b2ddb0 con 0x5647a6b2ac80 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.257+0000 7fb249b8d740 10 client.0 _ll_register_callbacks cb 0x5647a6a80da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.259+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b2fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.259+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b2fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44972/0 (socket says 172.21.15.64:44972) 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.259+0000 7fb23ffff640 1 -- 172.21.15.64:0/3132196892 learned_addr learned my addr 172.21.15.64:0/3132196892 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.260+0000 7fb244b41640 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb244b41640 1 -- 172.21.15.64:0/3132196892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 msgr2=0x5647a6b36b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb244b41640 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b36b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb244b41640 1 -- 172.21.15.64:0/3132196892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 msgr2=0x5647a6b2fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb244b41640 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b2fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb244b41640 1 -- 172.21.15.64:0/3132196892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a679d3f0 con 0x5647a6b25120 2024-05-09T09:18:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb244b41640 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b2f600 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5647a6b21660 tx=0x7fb2340136a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9eb9db74ffd997 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb21ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.261+0000 7fb21ffff640 1 -- 172.21.15.64:0/3132196892 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234029060 con 0x5647a6b25120 2024-05-09T09:18:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.262+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23400ec10 con 0x5647a6b25120 2024-05-09T09:18:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.262+0000 7fb21ffff640 1 -- 172.21.15.64:0/3132196892 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb23401c450 con 0x5647a6b25120 2024-05-09T09:18:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.262+0000 7fb21ffff640 1 -- 172.21.15.64:0/3132196892 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb234030050 con 0x5647a6b25120 2024-05-09T09:18:07.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.263+0000 7fb249b8d740 10 client.27272 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:07.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.263+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5647a675d0f0 con 0x5647a6b25120 2024-05-09T09:18:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb21ffff640 1 -- 172.21.15.64:0/3132196892 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb23400ec10 con 0x5647a6b25120 2024-05-09T09:18:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb21ffff640 1 -- 172.21.15.64:0/3132196892 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fb234012600 con 0x5647a6b25120 2024-05-09T09:18:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb249b8d740 10 client.27272 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb249b8d740 10 client.27272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb249b8d740 20 client.27272 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6b5d580 0x5647a6b5fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb249b8d740 4 client.27272 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647a6a97430 con 0x5647a6b5d580 2024-05-09T09:18:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.264+0000 7fb245342640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2280a2ce0 0x7fb2280a51a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5647a6b26370 tx=0x7fb230005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.268+0000 7fb23ffff640 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6b5d580 0x5647a6b5fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.271+0000 7fb23ffff640 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6b5d580 0x5647a6b5fa00 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c20a4a2b5e31b4 server_cookie=fb08aa64ffb6c3d4 in_seq=0 out_seq=0 2024-05-09T09:18:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.272+0000 7fb21ffff640 10 client.27272 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.273+0000 7fb21ffff640 1 -- 172.21.15.64:0/3132196892 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647a6a97430 con 0x5647a6b5d580 2024-05-09T09:18:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.273+0000 7fb21ffff640 10 client.27272 handle_command_reply: tid=0 2024-05-09T09:18:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.273+0000 7fb249b8d740 1 client.27272 shutdown 2024-05-09T09:18:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.273+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 msgr2=0x5647a6b2f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.273+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b2f600 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5647a6b21660 tx=0x7fb2340136a0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 shutdown_connections 2024-05-09T09:18:07.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6b5d580 0x5647a6b5fa00 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b36b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b2fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 --2- 172.21.15.64:0/3132196892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b2f600 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 >> 172.21.15.64:0/3132196892 conn(0x5647a6b20790 msgr2=0x5647a6b1fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb21ffff640 0 client.27272 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:07.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 shutdown_connections 2024-05-09T09:18:07.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.274+0000 7fb249b8d740 1 -- 172.21.15.64:0/3132196892 wait complete. 2024-05-09T09:18:07.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.275+0000 7fb249b8d740 20 client.27272 trim_cache size 0 max 16384 2024-05-09T09:18:07.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.311+0000 7fb249b8d740 1 Processor -- start 2024-05-09T09:18:07.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.311+0000 7fb249b8d740 1 -- start start 2024-05-09T09:18:07.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.311+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b1b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb244b41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b1b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb244b41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b1b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60900/0 (socket says 172.21.15.64:60900) 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb244b41640 1 -- 172.21.15.64:0/1483219339 learned_addr learned my addr 172.21.15.64:0/1483219339 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb249b8d740 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c012480 con 0x5647a6b25b10 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c00fd50 con 0x5647a6b25120 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.312+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c00dba0 con 0x5647a6b2ac80 2024-05-09T09:18:07.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.313+0000 7fb245342640 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.314+0000 7fb245342640 1 -- 172.21.15.64:0/1483219339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 msgr2=0x5647a6b1b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.314+0000 7fb245342640 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b1b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.314+0000 7fb245342640 1 -- 172.21.15.64:0/1483219339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 msgr2=0x5647a6b36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.314+0000 7fb245342640 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.314+0000 7fb245342640 1 -- 172.21.15.64:0/1483219339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a6b205c0 con 0x5647a6b2ac80 2024-05-09T09:18:07.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.315+0000 7fb245342640 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b2e8d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb230012ee0 tx=0x7fb230017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b9171b545427c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.316+0000 7fb2267fc640 1 -- 172.21.15.64:0/1483219339 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23001eaa0 con 0x5647a6b2ac80 2024-05-09T09:18:07.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.317+0000 7fb2267fc640 1 -- 172.21.15.64:0/1483219339 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb23001ec40 con 0x5647a6b2ac80 2024-05-09T09:18:07.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.317+0000 7fb2267fc640 1 -- 172.21.15.64:0/1483219339 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230003b60 con 0x5647a6b2ac80 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 msgr2=0x5647a6b2e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b2e8d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb230012ee0 tx=0x7fb230017000 comp rx=0 tx=0).stop 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 shutdown_connections 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6b2e8d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 --2- 172.21.15.64:0/1483219339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b1b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.318+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 >> 172.21.15.64:0/1483219339 conn(0x5647a6b20790 msgr2=0x5647a6b1fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.319+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 shutdown_connections 2024-05-09T09:18:07.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.319+0000 7fb249b8d740 1 -- 172.21.15.64:0/1483219339 wait complete. 2024-05-09T09:18:07.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.319+0000 7fb249b8d740 1 -- start start 2024-05-09T09:18:07.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.319+0000 7fb249b8d740 1 Processor -- start 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b33ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b34020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6bc1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c00e210 con 0x5647a6b25b10 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c00f680 con 0x5647a6b25120 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c00e8e0 con 0x5647a6b2ac80 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb249b8d740 10 client.0 _ll_register_callbacks cb 0x7fb230009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb244b41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b33ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb244b41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b33ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:60910/0 (socket says 172.21.15.64:60910) 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb244b41640 1 -- 172.21.15.64:0/251324297 learned_addr learned my addr 172.21.15.64:0/251324297 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb245342640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6bc1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.320+0000 7fb23ffff640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b34020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb245342640 1 -- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 msgr2=0x5647a6b33ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb245342640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b33ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb245342640 1 -- 172.21.15.64:0/251324297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 msgr2=0x5647a6b34020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb245342640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b34020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb245342640 1 -- 172.21.15.64:0/251324297 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a6b11930 con 0x5647a6b2ac80 2024-05-09T09:18:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb23ffff640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b34020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:18:07.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb245342640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6bc1b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb23002b040 tx=0x7fb230005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0fa5a5a449d3063 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:07.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb21ffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:07.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.321+0000 7fb21ffff640 1 -- 172.21.15.64:0/251324297 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23001eaa0 con 0x5647a6b2ac80 2024-05-09T09:18:07.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.322+0000 7fb21ffff640 1 -- 172.21.15.64:0/251324297 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb23001edf0 con 0x5647a6b2ac80 2024-05-09T09:18:07.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.322+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23001ec40 con 0x5647a6b2ac80 2024-05-09T09:18:07.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.322+0000 7fb21ffff640 1 -- 172.21.15.64:0/251324297 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb2300046c0 con 0x5647a6b2ac80 2024-05-09T09:18:07.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.323+0000 7fb249b8d740 10 client.27415 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:07.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.323+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5647a6bc3290 con 0x5647a6b2ac80 2024-05-09T09:18:07.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.323+0000 7fb21ffff640 1 -- 172.21.15.64:0/251324297 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb23001ec40 con 0x5647a6b2ac80 2024-05-09T09:18:07.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.323+0000 7fb21ffff640 1 -- 172.21.15.64:0/251324297 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fb23003d820 con 0x5647a6b2ac80 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb249b8d740 10 client.27415 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb249b8d740 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb249b8d740 20 client.27415 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb249b8d740 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6bc3b90 0x5647a6bc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb249b8d740 4 client.27415 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647a6a91780 con 0x5647a6bc3b90 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.324+0000 7fb244b41640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6bc3b90 0x5647a6bc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.325+0000 7fb244b41640 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6bc3b90 0x5647a6bc5fb0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4244477a9851a37b server_cookie=1ca83b77df0ff7bc in_seq=0 out_seq=0 2024-05-09T09:18:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.328+0000 7fb21ffff640 10 client.27415 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.328+0000 7fb21ffff640 1 -- 172.21.15.64:0/251324297 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5647a6a91780 con 0x5647a6bc3b90 2024-05-09T09:18:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.328+0000 7fb21ffff640 10 client.27415 handle_command_reply: tid=0 2024-05-09T09:18:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 client.27415 shutdown 2024-05-09T09:18:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 msgr2=0x5647a6bc1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6bc1b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb23002b040 tx=0x7fb230005560 comp rx=0 tx=0).stop 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 shutdown_connections 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647a6bc3b90 0x5647a6bc5fb0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a6b2ac80 0x5647a6bc1b70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a6b25b10 0x5647a6b34020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 --2- 172.21.15.64:0/251324297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647a6b25120 0x5647a6b33ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 >> 172.21.15.64:0/251324297 conn(0x5647a6b20790 msgr2=0x5647a6b23760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.329+0000 7fb21ffff640 0 client.27415 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.330+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 shutdown_connections 2024-05-09T09:18:07.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.337+0000 7fb249b8d740 1 -- 172.21.15.64:0/251324297 wait complete. 2024-05-09T09:18:07.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.337+0000 7fb249b8d740 20 client.27415 trim_cache size 0 max 16384 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: "ce4d9c10-c387-4689-8fa3-43df962b8f53": { 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "ce4d9c10-c387-4689-8fa3-43df962b8f53", 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:18:07.337 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:18:07.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2280a2ce0 msgr2=0x7fb2280a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2280a2ce0 0x7fb2280a51a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5647a6b26370 tx=0x7fb230005f90 comp rx=0 tx=0).stop 2024-05-09T09:18:07.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240108f10 msgr2=0x7fb24019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:07.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240108f10 0x7fb24019e4e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb23803bed0 tx=0x7fb238003ba0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 shutdown_connections 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb2280a2ce0 0x7fb2280a51a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.338+0000 7fb245b43640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2401098b0 0x7fb2401a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.339+0000 7fb245b43640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240108f10 0x7fb24019e4e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.339+0000 7fb245b43640 1 --2- 172.21.15.64:0/701307853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb240072c70 0x7fb24019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.339+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 >> 172.21.15.64:0/701307853 conn(0x7fb24006e740 msgr2=0x7fb24010c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.339+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 shutdown_connections 2024-05-09T09:18:07.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:07.339+0000 7fb245b43640 1 -- 172.21.15.64:0/701307853 wait complete. 2024-05-09T09:18:07.363 INFO:tasks.cephfs.filesystem:scrub status for tag:ce4d9c10-c387-4689-8fa3-43df962b8f53 - {'path': '/', 'tag': 'ce4d9c10-c387-4689-8fa3-43df962b8f53', 'options': 'recursive,force'} 2024-05-09T09:18:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:07 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:18:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:07 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:07 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:18:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:07 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:18:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:07 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:07 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:07 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:18:07.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:07 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:07 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:18:07.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:07 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:18:07.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:07 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:07 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:07 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:18:07.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:07 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:07 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:18:07.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:07 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:18:07.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:07 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:07.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:07 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:08.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:08 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:18:08.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:08 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:08.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:08 smithi179 ceph-mon[34086]: pgmap v640: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 803 op/s 2024-05-09T09:18:08.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:08 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:18:08.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:08 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:08.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:08 smithi139 ceph-mon[33599]: pgmap v640: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 803 op/s 2024-05-09T09:18:08.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:08 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:18:08.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:08 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:08.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:08 smithi064 ceph-mon[30149]: pgmap v640: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 803 op/s 2024-05-09T09:18:10.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:10 smithi139 ceph-mon[33599]: pgmap v641: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:18:10.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:10 smithi064 ceph-mon[30149]: pgmap v641: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:18:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:10 smithi179 ceph-mon[34086]: pgmap v641: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-05-09T09:18:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 470.0 s, 663.7 tps, lat 151.154 ms stddev 153.671 2024-05-09T09:18:12.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:12 smithi139 ceph-mon[33599]: pgmap v642: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.1 MiB/s wr, 867 op/s 2024-05-09T09:18:12.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:12 smithi064 ceph-mon[30149]: pgmap v642: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.1 MiB/s wr, 867 op/s 2024-05-09T09:18:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:12 smithi179 ceph-mon[34086]: pgmap v642: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.1 MiB/s wr, 867 op/s 2024-05-09T09:18:14.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:14 smithi139 ceph-mon[33599]: pgmap v643: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:18:14.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:14 smithi064 ceph-mon[30149]: pgmap v643: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:18:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:14 smithi179 ceph-mon[34086]: pgmap v643: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-05-09T09:18:16.354 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 475.0 s, 730.1 tps, lat 137.394 ms stddev 52.321 2024-05-09T09:18:16.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:16 smithi139 ceph-mon[33599]: pgmap v644: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 997 op/s 2024-05-09T09:18:16.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:16 smithi064 ceph-mon[30149]: pgmap v644: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 997 op/s 2024-05-09T09:18:16.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:16 smithi179 ceph-mon[34086]: pgmap v644: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 997 op/s 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 -- 172.21.15.64:0/726413810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec41019c0 msgr2=0x7f6ec4101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/726413810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec41019c0 0x7f6ec4101e40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009900 tx=0x7f6eb402a410 comp rx=0 tx=0).stop 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 -- 172.21.15.64:0/726413810 shutdown_connections 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/726413810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec4102380 0x7f6ec4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/726413810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec41019c0 0x7f6ec4101e40 secure :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009900 tx=0x7f6eb402a410 comp rx=0 tx=0).stop 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/726413810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41007d0 0x7f6ec4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.908+0000 7f6eca6ba640 1 -- 172.21.15.64:0/726413810 >> 172.21.15.64:0/726413810 conn(0x7f6ec40fbd40 msgr2=0x7f6ec40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:17.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.911+0000 7f6eca6ba640 1 -- 172.21.15.64:0/726413810 shutdown_connections 2024-05-09T09:18:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.912+0000 7f6eca6ba640 1 -- 172.21.15.64:0/726413810 wait complete. 2024-05-09T09:18:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.913+0000 7f6eca6ba640 1 Processor -- start 2024-05-09T09:18:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.913+0000 7f6eca6ba640 1 -- start start 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.913+0000 7f6eca6ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec41007d0 0x7f6ec419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.913+0000 7f6eca6ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 0x7f6ec419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.913+0000 7f6eca6ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41a3880 0x7f6ec41a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6eca6ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec410c3b0 con 0x7f6ec4102380 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6eca6ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec410c230 con 0x7f6ec41007d0 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6eca6ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec410c530 con 0x7f6ec41a3880 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 0x7f6ec419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 0x7f6ec419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41196/0 (socket says 172.21.15.64:41196) 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6ec37fe640 1 -- 172.21.15.64:0/1516582832 learned_addr learned my addr 172.21.15.64:0/1516582832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41a3880 0x7f6ec41a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.914+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec41007d0 0x7f6ec419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec8c57640 1 -- 172.21.15.64:0/1516582832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec41007d0 msgr2=0x7f6ec419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec41007d0 0x7f6ec419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec8c57640 1 -- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 msgr2=0x7f6ec419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 0x7f6ec419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec8c57640 1 -- 172.21.15.64:0/1516582832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec41a6340 con 0x7f6ec41a3880 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 0x7f6ec419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.915+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec41007d0 0x7f6ec419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.916+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41a3880 0x7f6ec41a5c70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00be50 tx=0x7f6ebc00bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.917+0000 7f6ec17fa640 1 -- 172.21.15.64:0/1516582832 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc019070 con 0x7f6ec41a3880 2024-05-09T09:18:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.917+0000 7f6ec17fa640 1 -- 172.21.15.64:0/1516582832 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ebc0148f0 con 0x7f6ec41a3880 2024-05-09T09:18:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.917+0000 7f6ec17fa640 1 -- 172.21.15.64:0/1516582832 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc012910 con 0x7f6ec41a3880 2024-05-09T09:18:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.918+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec41a65d0 con 0x7f6ec41a3880 2024-05-09T09:18:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.918+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec41a6ac0 con 0x7f6ec41a3880 2024-05-09T09:18:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.921+0000 7f6ecd3c6740 1 Processor -- start 2024-05-09T09:18:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ec17fa640 1 -- 172.21.15.64:0/1516582832 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ebc014aa0 con 0x7f6ec41a3880 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 -- start start 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 0x55d92c707670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c707c40 0x55d92c70c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c70f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d92c66b340 con 0x55d92c70cdb0 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d92c613290 con 0x55d92c707c40 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d92c667300 con 0x55d92c707250 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.922+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 0x55d92c707670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 0x55d92c707670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34994/0 (socket says 172.21.15.64:34994) 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 -- 172.21.15.64:0/1216054017 learned_addr learned my addr 172.21.15.64:0/1216054017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c707c40 0x55d92c70c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c70f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 -- 172.21.15.64:0/1216054017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c707c40 msgr2=0x55d92c70c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c707c40 0x55d92c70c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 -- 172.21.15.64:0/1216054017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 msgr2=0x55d92c70f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c70f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.923+0000 7f6ec37fe640 1 -- 172.21.15.64:0/1216054017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92c6378e0 con 0x55d92c707250 2024-05-09T09:18:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.924+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 0x55d92c707670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009230 tx=0x7f6eb4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b2fcf0bf2d27f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.924+0000 7f6eaa7fc640 1 -- 172.21.15.64:0/1216054017 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4035040 con 0x55d92c707250 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.924+0000 7f6eaa7fc640 1 -- 172.21.15.64:0/1216054017 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb4003ba0 con 0x55d92c707250 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.924+0000 7f6eaa7fc640 1 -- 172.21.15.64:0/1216054017 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb403d6b0 con 0x55d92c707250 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ec17fa640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6eac0a2e20 0x7f6eac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1216054017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 msgr2=0x55d92c707670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 0x55d92c707670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009230 tx=0x7f6eb4009260 comp rx=0 tx=0).stop 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1216054017 shutdown_connections 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c70f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c707c40 0x55d92c70c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1216054017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707250 0x55d92c707670 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009230 tx=0x7f6eb4009260 comp rx=0 tx=0).stop 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.925+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1216054017 >> 172.21.15.64:0/1216054017 conn(0x55d92c7028c0 msgr2=0x55d92c702cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.926+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6eac0a2e20 0x7f6eac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.926+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1216054017 shutdown_connections 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.926+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1216054017 wait complete. 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.926+0000 7f6ecd3c6740 1 -- start start 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.926+0000 7f6ec17fa640 1 -- 172.21.15.64:0/1516582832 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ebc0cf010 con 0x7f6ec41a3880 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.926+0000 7f6ecd3c6740 1 Processor -- start 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c711660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c711ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c716bf0 0x55d92c718fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d92c70fd00 con 0x55d92c70cdb0 2024-05-09T09:18:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d92c626790 con 0x55d92c716bf0 2024-05-09T09:18:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d92c70fe80 con 0x55d92c707c40 2024-05-09T09:18:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ecd3c6740 10 client.0 _ll_register_callbacks cb 0x55d92c661a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c711660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c711660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35010/0 (socket says 172.21.15.64:35010) 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.927+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1644534808 learned_addr learned my addr 172.21.15.64:0/1644534808 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c716bf0 0x55d92c718fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c711ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c716bf0 msgr2=0x55d92c718fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c716bf0 0x55d92c718fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1644534808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 msgr2=0x55d92c711ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c711ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1644534808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92c37f210 con 0x55d92c707c40 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c711660 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8011be0 tx=0x7f6eb80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=490d829387ba8aab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6eac0a2e20 0x7f6eac0a52e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d92c703830 tx=0x7f6eb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ea37fe640 1 -- 172.21.15.64:0/1644534808 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8022030 con 0x55d92c707c40 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb4003ba0 con 0x55d92c707c40 2024-05-09T09:18:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.928+0000 7f6ea37fe640 1 -- 172.21.15.64:0/1644534808 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6eb8011cd0 con 0x55d92c707c40 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.929+0000 7f6ea37fe640 1 -- 172.21.15.64:0/1644534808 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6eb8031050 con 0x55d92c707c40 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.929+0000 7f6ecd3c6740 10 client.27427 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.929+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d92c44b620 con 0x55d92c707c40 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ea37fe640 1 -- 172.21.15.64:0/1644534808 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb4003ba0 con 0x55d92c707c40 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ea37fe640 1 -- 172.21.15.64:0/1644534808 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6eb801d040 con 0x55d92c707c40 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ecd3c6740 10 client.27427 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ecd3c6740 10 client.27427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ecd3c6740 20 client.27427 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c72ea60 0x55d92c730f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ecd3c6740 4 client.27427 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d92c678a70 con 0x55d92c72ea60 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.930+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c72ea60 0x55d92c730f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.931+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c72ea60 0x55d92c730f20 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f7ff2ea34753b5 server_cookie=9b5a2af20c0caad9 in_seq=0 out_seq=0 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.931+0000 7f6ea37fe640 10 client.27427 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.932+0000 7f6ea37fe640 1 -- 172.21.15.64:0/1644534808 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d92c678a70 con 0x55d92c72ea60 2024-05-09T09:18:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.932+0000 7f6ea37fe640 10 client.27427 handle_command_reply: tid=0 2024-05-09T09:18:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.932+0000 7f6ecd3c6740 1 client.27427 shutdown 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.932+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 msgr2=0x55d92c711660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.932+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c711660 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8011be0 tx=0x7f6eb80107a0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 shutdown_connections 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c72ea60 0x55d92c730f20 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c716bf0 0x55d92c718fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c70cdb0 0x55d92c711ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1644534808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c711660 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 >> 172.21.15.64:0/1644534808 conn(0x55d92c7028c0 msgr2=0x55d92c701a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ea37fe640 0 client.27427 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 shutdown_connections 2024-05-09T09:18:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1644534808 wait complete. 2024-05-09T09:18:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.933+0000 7f6ecd3c6740 20 client.27427 trim_cache size 0 max 16384 2024-05-09T09:18:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.964+0000 7f6ecd3c6740 1 Processor -- start 2024-05-09T09:18:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.964+0000 7f6ecd3c6740 1 -- start start 2024-05-09T09:18:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.964+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c707c40 0x55d92c7090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.964+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c7095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.964+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c716bf0 0x55d92c711290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41ab140 con 0x55d92c707c40 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41a8a10 con 0x55d92c70cdb0 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec40735c0 con 0x55d92c716bf0 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c7095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c707c40 0x55d92c7090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c7095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48604/0 (socket says 172.21.15.64:48604) 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec37fe640 1 -- 172.21.15.64:0/1940753741 learned_addr learned my addr 172.21.15.64:0/1940753741 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c716bf0 0x55d92c711290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1940753741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c716bf0 msgr2=0x55d92c711290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c716bf0 0x55d92c711290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1940753741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 msgr2=0x55d92c7095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c7095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.965+0000 7f6ec3fff640 1 -- 172.21.15.64:0/1940753741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92c719350 con 0x55d92c707c40 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.966+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c707c40 0x55d92c7090a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8026fd0 tx=0x7f6eb8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4131d7b15c0b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.966+0000 7f6ea9ffb640 1 -- 172.21.15.64:0/1940753741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8022070 con 0x55d92c707c40 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.967+0000 7f6ea9ffb640 1 -- 172.21.15.64:0/1940753741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb8011cd0 con 0x55d92c707c40 2024-05-09T09:18:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.967+0000 7f6ea9ffb640 1 -- 172.21.15.64:0/1940753741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8005620 con 0x55d92c707c40 2024-05-09T09:18:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1940753741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c707c40 msgr2=0x55d92c7090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c707c40 0x55d92c7090a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8026fd0 tx=0x7f6eb8000bc0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1940753741 shutdown_connections 2024-05-09T09:18:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c716bf0 0x55d92c711290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c7095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/1940753741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c707c40 0x55d92c7090a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1940753741 >> 172.21.15.64:0/1940753741 conn(0x55d92c7028c0 msgr2=0x55d92c702ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1940753741 shutdown_connections 2024-05-09T09:18:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.968+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/1940753741 wait complete. 2024-05-09T09:18:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.969+0000 7f6ecd3c6740 1 -- start start 2024-05-09T09:18:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.969+0000 7f6ecd3c6740 1 Processor -- start 2024-05-09T09:18:17.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c73f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c73fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c716bf0 0x55d92c740db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41a6ed0 con 0x55d92c716bf0 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41a8340 con 0x55d92c70cdb0 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41a75a0 con 0x55d92c707c40 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ecd3c6740 10 client.0 _ll_register_callbacks cb 0x7f6eb8009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c73fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c73fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48606/0 (socket says 172.21.15.64:48606) 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 -- 172.21.15.64:0/554717420 learned_addr learned my addr 172.21.15.64:0/554717420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c73f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec8c57640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c716bf0 0x55d92c740db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 -- 172.21.15.64:0/554717420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 msgr2=0x55d92c73f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c73f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 -- 172.21.15.64:0/554717420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c716bf0 msgr2=0x55d92c740db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c716bf0 0x55d92c740db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.970+0000 7f6ec37fe640 1 -- 172.21.15.64:0/554717420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92c7413a0 con 0x55d92c70cdb0 2024-05-09T09:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ec37fe640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c73fdf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009230 tx=0x7f6eb4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b357b88dbb0941 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ea37fe640 1 -- 172.21.15.64:0/554717420 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb405a060 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ea37fe640 1 -- 172.21.15.64:0/554717420 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6eb4003d30 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb8011cd0 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ea37fe640 1 -- 172.21.15.64:0/554717420 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6eb4061050 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ecd3c6740 10 client.27293 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.971+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d92c7426e0 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ea37fe640 1 -- 172.21.15.64:0/554717420 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb8011cd0 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ea37fe640 1 -- 172.21.15.64:0/554717420 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6eb4048070 con 0x55d92c70cdb0 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ecd3c6740 10 client.27293 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ecd3c6740 10 client.27293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ecd3c6740 20 client.27293 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c795020 0x55d92c797410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ecd3c6740 4 client.27293 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d92c672dc0 con 0x55d92c795020 2024-05-09T09:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.972+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c795020 0x55d92c797410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.973+0000 7f6ec3fff640 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c795020 0x55d92c797410 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d73447b7f5e64a7 server_cookie=3f2c6902fc93a7f7 in_seq=0 out_seq=0 2024-05-09T09:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.973+0000 7f6ea37fe640 10 client.27293 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ea37fe640 1 -- 172.21.15.64:0/554717420 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d92c672dc0 con 0x55d92c795020 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ea37fe640 10 client.27293 handle_command_reply: tid=0 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 client.27293 shutdown 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 msgr2=0x55d92c73fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c73fdf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009230 tx=0x7f6eb4005c90 comp rx=0 tx=0).stop 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 shutdown_connections 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d92c795020 0x55d92c797410 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d92c716bf0 0x55d92c740db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d92c70cdb0 0x55d92c73fdf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.995+0000 7f6ecd3c6740 1 --2- 172.21.15.64:0/554717420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d92c707c40 0x55d92c73f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.996+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 >> 172.21.15.64:0/554717420 conn(0x55d92c7028c0 msgr2=0x55d92c7090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.996+0000 7f6ea37fe640 0 client.27293 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.997+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 shutdown_connections 2024-05-09T09:18:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.998+0000 7f6ecd3c6740 1 -- 172.21.15.64:0/554717420 wait complete. 2024-05-09T09:18:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:17.998+0000 7f6ecd3c6740 20 client.27293 trim_cache size 0 max 16384 2024-05-09T09:18:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:18:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:18:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:18:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:18:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.000+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6eac0a2e20 msgr2=0x7f6eac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.000+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6eac0a2e20 0x7f6eac0a52e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d92c703830 tx=0x7f6eb8005f90 comp rx=0 tx=0).stop 2024-05-09T09:18:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.000+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41a3880 msgr2=0x7f6ec41a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.000+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41a3880 0x7f6ec41a5c70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00be50 tx=0x7f6ebc00bf40 comp rx=0 tx=0).stop 2024-05-09T09:18:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 shutdown_connections 2024-05-09T09:18:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6eac0a2e20 0x7f6eac0a52e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ec41a3880 0x7f6ec41a5c70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec4102380 0x7f6ec419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 --2- 172.21.15.64:0/1516582832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec41007d0 0x7f6ec419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 >> 172.21.15.64:0/1516582832 conn(0x7f6ec40fbd40 msgr2=0x7f6ec40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 shutdown_connections 2024-05-09T09:18:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:18.001+0000 7f6eca6ba640 1 -- 172.21.15.64:0/1516582832 wait complete. 2024-05-09T09:18:18.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:18 smithi139 ceph-mon[33599]: pgmap v645: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.0 MiB/s wr, 914 op/s 2024-05-09T09:18:18.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:18 smithi064 ceph-mon[30149]: pgmap v645: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.0 MiB/s wr, 914 op/s 2024-05-09T09:18:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:18 smithi179 ceph-mon[34086]: pgmap v645: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.0 MiB/s wr, 914 op/s 2024-05-09T09:18:20.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:20 smithi139 ceph-mon[33599]: pgmap v646: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-05-09T09:18:20.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:20 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:18:20.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:20 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:18:20.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:20 smithi064 ceph-mon[30149]: pgmap v646: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-05-09T09:18:20.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:20 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:18:20.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:20 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:18:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:20 smithi179 ceph-mon[34086]: pgmap v646: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-05-09T09:18:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:20 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:18:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:20 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:18:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 480.0 s, 805.6 tps, lat 123.095 ms stddev 42.694 2024-05-09T09:18:21.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:21 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:18:21.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:21 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:18:21.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:21 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:18:22.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:22 smithi139 ceph-mon[33599]: pgmap v647: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 856 op/s 2024-05-09T09:18:23.642 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:22 smithi064 ceph-mon[30149]: pgmap v647: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 856 op/s 2024-05-09T09:18:23.643 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:22 smithi179 ceph-mon[34086]: pgmap v647: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 856 op/s 2024-05-09T09:18:24.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:24 smithi139 ceph-mon[33599]: pgmap v648: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.9 MiB/s wr, 943 op/s 2024-05-09T09:18:24.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:24 smithi064 ceph-mon[30149]: pgmap v648: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.9 MiB/s wr, 943 op/s 2024-05-09T09:18:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:24 smithi179 ceph-mon[34086]: pgmap v648: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.9 MiB/s wr, 943 op/s 2024-05-09T09:18:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 485.0 s, 752.1 tps, lat 133.439 ms stddev 57.822 2024-05-09T09:18:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:26 smithi139 ceph-mon[33599]: pgmap v649: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 905 op/s 2024-05-09T09:18:26.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:26 smithi064 ceph-mon[30149]: pgmap v649: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 905 op/s 2024-05-09T09:18:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:26 smithi179 ceph-mon[34086]: pgmap v649: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 905 op/s 2024-05-09T09:18:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:28 smithi139 ceph-mon[33599]: pgmap v650: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 798 op/s 2024-05-09T09:18:28.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:28 smithi064 ceph-mon[30149]: pgmap v650: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 798 op/s 2024-05-09T09:18:28.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:28 smithi179 ceph-mon[34086]: pgmap v650: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 798 op/s 2024-05-09T09:18:30.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:30 smithi139 ceph-mon[33599]: pgmap v651: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2024-05-09T09:18:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:30 smithi064 ceph-mon[30149]: pgmap v651: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2024-05-09T09:18:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:30 smithi179 ceph-mon[34086]: pgmap v651: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2024-05-09T09:18:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/2185910105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 msgr2=0x7fc4c0110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/2185910105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 0x7fc4c0110ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009900 tx=0x7fc4b002a440 comp rx=0 tx=0).stop 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/2185910105 shutdown_connections 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/2185910105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 0x7fc4c0110ee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/2185910105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/2185910105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0073680 0x7fc4c010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.160+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/2185910105 >> 172.21.15.64:0/2185910105 conn(0x7fc4c006e890 msgr2=0x7fc4c0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.161+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/2185910105 shutdown_connections 2024-05-09T09:18:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.161+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/2185910105 wait complete. 2024-05-09T09:18:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 Processor -- start 2024-05-09T09:18:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 -- start start 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0073680 0x7fc4c01191f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c0119730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 0x7fc4c011e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0113840 con 0x7fc4c0073680 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01136c0 con 0x7fc4c010ea20 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.163+0000 7fc4c4d2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01139c0 con 0x7fc4c010bf20 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.164+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c0119730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.164+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c0119730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57252/0 (socket says 172.21.15.64:57252) 2024-05-09T09:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.164+0000 7fc4beffd640 1 -- 172.21.15.64:0/1908397120 learned_addr learned my addr 172.21.15.64:0/1908397120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.164+0000 7fc4bffff640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 0x7fc4c011e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.165+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0073680 0x7fc4c01191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.166+0000 7fc4beffd640 1 -- 172.21.15.64:0/1908397120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 msgr2=0x7fc4c011e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.166+0000 7fc4beffd640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 0x7fc4c011e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.166+0000 7fc4beffd640 1 -- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0073680 msgr2=0x7fc4c01191f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.166+0000 7fc4beffd640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0073680 0x7fc4c01191f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.166+0000 7fc4beffd640 1 -- 172.21.15.64:0/1908397120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c011ee00 con 0x7fc4c010bf20 2024-05-09T09:18:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.166+0000 7fc4beffd640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c0119730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.167+0000 7fc4bcff9640 1 -- 172.21.15.64:0/1908397120 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4c010bf20 2024-05-09T09:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.167+0000 7fc4bcff9640 1 -- 172.21.15.64:0/1908397120 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b4009c30 con 0x7fc4c010bf20 2024-05-09T09:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.167+0000 7fc4bcff9640 1 -- 172.21.15.64:0/1908397120 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400f620 con 0x7fc4c010bf20 2024-05-09T09:18:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.169+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c011f0f0 con 0x7fc4c010bf20 2024-05-09T09:18:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.169+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c011f5c0 con 0x7fc4c010bf20 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.173+0000 7fc4bcff9640 1 -- 172.21.15.64:0/1908397120 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4b401a070 con 0x7fc4c010bf20 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.173+0000 7fc4c8d7f740 1 Processor -- start 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.173+0000 7fc4c8d7f740 1 -- start start 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.173+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96ede820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.173+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96ee63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ae96e42340 con 0x55ae96ede400 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ae96dea290 con 0x55ae96ededf0 2024-05-09T09:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae96e3e300 con 0x55ae96ee3f60 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96ee63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96ee63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57264/0 (socket says 172.21.15.64:57264) 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4beffd640 1 -- 172.21.15.64:0/3806488269 learned_addr learned my addr 172.21.15.64:0/3806488269 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.174+0000 7fc4bffff640 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96ede820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/3806488269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 msgr2=0x55ae96ee63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96ee63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/3806488269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 msgr2=0x55ae96ede820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96ede820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/3806488269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae96e0e8e0 con 0x55ae96ededf0 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4beffd640 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96ee63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bcff9640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4a80a2d90 0x7fc4a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.175+0000 7fc4bcff9640 1 -- 172.21.15.64:0/1908397120 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4b40ce880 con 0x7fc4c010bf20 2024-05-09T09:18:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.176+0000 7fc4bffff640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4a80a2d90 0x7fc4a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.176+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee38f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8007d50 tx=0x7fc4b800c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c5afedbc2f2514 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.176+0000 7fc4bffff640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4a80a2d90 0x7fc4a80a5250 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0005ab0 tx=0x7fc4b0004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.176+0000 7fc4a5ffb640 1 -- 172.21.15.64:0/3806488269 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8015020 con 0x55ae96ededf0 2024-05-09T09:18:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.177+0000 7fc4a5ffb640 1 -- 172.21.15.64:0/3806488269 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b8011ae0 con 0x55ae96ededf0 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/3806488269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 msgr2=0x55ae96ee38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee38f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8007d50 tx=0x7fc4b800c560 comp rx=0 tx=0).stop 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/3806488269 shutdown_connections 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96ee63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee38f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/3806488269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96ede820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/3806488269 >> 172.21.15.64:0/3806488269 conn(0x55ae96ed9a70 msgr2=0x55ae96ed9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/3806488269 shutdown_connections 2024-05-09T09:18:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.178+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/3806488269 wait complete. 2024-05-09T09:18:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.179+0000 7fc4c8d7f740 1 -- start start 2024-05-09T09:18:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.179+0000 7fc4c8d7f740 1 Processor -- start 2024-05-09T09:18:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96ee8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ee3f60 0x55ae96eefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ae96ee6fb0 con 0x55ae96ee3f60 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ae96dfd790 con 0x55ae96ededf0 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae96ee7130 con 0x55ae96ede400 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4c8d7f740 10 client.0 _ll_register_callbacks cb 0x55ae96e38a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36814/0 (socket says 172.21.15.64:36814) 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4beffd640 1 -- 172.21.15.64:0/1422800995 learned_addr learned my addr 172.21.15.64:0/1422800995 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.180+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96ee8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 msgr2=0x55ae96ee8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1422800995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ee3f60 msgr2=0x55ae96eefbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ee3f60 0x55ae96eefbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1422800995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae96b56210 con 0x55ae96ede400 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96ee8670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55ae96eda9e0 tx=0x7fc4b8007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a663e3cd36845e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc49effd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc49effd640 1 -- 172.21.15.64:0/1422800995 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8028060 con 0x55ae96ede400 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.181+0000 7fc49effd640 1 -- 172.21.15.64:0/1422800995 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4b8011840 con 0x55ae96ede400 2024-05-09T09:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.182+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b8011ae0 con 0x55ae96ede400 2024-05-09T09:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.182+0000 7fc49effd640 1 -- 172.21.15.64:0/1422800995 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8015070 con 0x55ae96ede400 2024-05-09T09:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.183+0000 7fc4c8d7f740 10 client.27445 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.183+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ae96c22620 con 0x55ae96ede400 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.183+0000 7fc49effd640 1 -- 172.21.15.64:0/1422800995 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b8011ae0 con 0x55ae96ede400 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc49effd640 1 -- 172.21.15.64:0/1422800995 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fc4b8022030 con 0x55ae96ede400 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4c8d7f740 10 client.27445 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4c8d7f740 10 client.27445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4c8d7f740 20 client.27445 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96ef6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4c8d7f740 4 client.27445 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae96e4fa70 con 0x55ae96ef42b0 2024-05-09T09:18:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.184+0000 7fc4beffd640 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96ef6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.185+0000 7fc4beffd640 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96ef6740 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7560d1edf96b3971 server_cookie=a4cba9ef5ad72250 in_seq=0 out_seq=0 2024-05-09T09:18:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.185+0000 7fc49effd640 10 client.27445 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.186+0000 7fc49effd640 1 -- 172.21.15.64:0/1422800995 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae96e4fa70 con 0x55ae96ef42b0 2024-05-09T09:18:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.186+0000 7fc49effd640 10 client.27445 handle_command_reply: tid=0 2024-05-09T09:18:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.189+0000 7fc4c8d7f740 1 client.27445 shutdown 2024-05-09T09:18:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.195+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 msgr2=0x55ae96ee8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.195+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96ee8670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55ae96eda9e0 tx=0x7fc4b8007a90 comp rx=0 tx=0).stop 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.198+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 shutdown_connections 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.198+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96ef6740 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.198+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ee3f60 0x55ae96eefbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.198+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96ee8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.199+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1422800995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96ee8670 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.199+0000 7fc49effd640 0 client.27445 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.199+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 >> 172.21.15.64:0/1422800995 conn(0x55ae96ed9a70 msgr2=0x55ae96ed8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.203+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 shutdown_connections 2024-05-09T09:18:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.203+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1422800995 wait complete. 2024-05-09T09:18:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.204+0000 7fc4c8d7f740 20 client.27445 trim_cache size 0 max 16384 2024-05-09T09:18:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.233+0000 7fc4c8d7f740 1 Processor -- start 2024-05-09T09:18:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.233+0000 7fc4c8d7f740 1 -- start start 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96edd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ededf0 0x55ae96eefbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96edd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96edd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57282/0 (socket says 172.21.15.64:57282) 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ee3f60 0x55ae96ed8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1053949624 learned_addr learned my addr 172.21.15.64:0/1053949624 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01c4190 con 0x55ae96ededf0 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01c1a60 con 0x55ae96ee3f60 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4bffff640 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ee3f60 0x55ae96ed8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.234+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01bf8b0 con 0x55ae96ede400 2024-05-09T09:18:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.235+0000 7fc4beffd640 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ededf0 0x55ae96eefbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.235+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1053949624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ee3f60 msgr2=0x55ae96ed8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.235+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ee3f60 0x55ae96ed8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.235+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1053949624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ededf0 msgr2=0x55ae96eefbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.235+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ededf0 0x55ae96eefbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.235+0000 7fc4bf7fe640 1 -- 172.21.15.64:0/1053949624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae96ee72b0 con 0x55ae96ede400 2024-05-09T09:18:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.236+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96edd2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8002230 tx=0x7fc4b800c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca33366aa7a9dbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.236+0000 7fc4a57fa640 1 -- 172.21.15.64:0/1053949624 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8028040 con 0x55ae96ede400 2024-05-09T09:18:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.236+0000 7fc4a57fa640 1 -- 172.21.15.64:0/1053949624 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b80075c0 con 0x55ae96ede400 2024-05-09T09:18:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.236+0000 7fc4a57fa640 1 -- 172.21.15.64:0/1053949624 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8032080 con 0x55ae96ede400 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 msgr2=0x55ae96edd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96edd2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8002230 tx=0x7fc4b800c560 comp rx=0 tx=0).stop 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 shutdown_connections 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ee3f60 0x55ae96ed8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ededf0 0x55ae96eefbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/1053949624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ede400 0x55ae96edd2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.237+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 >> 172.21.15.64:0/1053949624 conn(0x55ae96ed9a70 msgr2=0x55ae96ed8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.238+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 shutdown_connections 2024-05-09T09:18:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.238+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/1053949624 wait complete. 2024-05-09T09:18:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.238+0000 7fc4c8d7f740 1 -- start start 2024-05-09T09:18:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 Processor -- start 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96eecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96eed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96eee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01bff20 con 0x55ae96ede400 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01c1390 con 0x55ae96ededf0 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01c05f0 con 0x55ae96ee3f60 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4c8d7f740 10 client.0 _ll_register_callbacks cb 0x7fc4b0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96eed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96eed250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36832/0 (socket says 172.21.15.64:36832) 2024-05-09T09:18:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.239+0000 7fc4beffd640 1 -- 172.21.15.64:0/2046633866 learned_addr learned my addr 172.21.15.64:0/2046633866 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96eecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4bffff640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96eee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4beffd640 1 -- 172.21.15.64:0/2046633866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 msgr2=0x55ae96eee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4beffd640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96eee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4beffd640 1 -- 172.21.15.64:0/2046633866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 msgr2=0x55ae96eecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4beffd640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96eecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4beffd640 1 -- 172.21.15.64:0/2046633866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae96eee850 con 0x55ae96ededf0 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc4beffd640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96eed250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4b409ad00 tx=0x7fc4b409ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c147f519b43932 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.240+0000 7fc49effd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.241+0000 7fc49effd640 1 -- 172.21.15.64:0/2046633866 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4009de0 con 0x55ae96ededf0 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.241+0000 7fc49effd640 1 -- 172.21.15.64:0/2046633866 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4b40973e0 con 0x55ae96ededf0 2024-05-09T09:18:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.241+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b80075c0 con 0x55ae96ededf0 2024-05-09T09:18:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.244+0000 7fc49effd640 1 -- 172.21.15.64:0/2046633866 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc4b409b470 con 0x55ae96ededf0 2024-05-09T09:18:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.244+0000 7fc4c8d7f740 10 client.27311 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.244+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ae96eefc00 con 0x55ae96ededf0 2024-05-09T09:18:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc49effd640 1 -- 172.21.15.64:0/2046633866 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b409ad70 con 0x55ae96ededf0 2024-05-09T09:18:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc49effd640 1 -- 172.21.15.64:0/2046633866 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fc4b4096ac0 con 0x55ae96ededf0 2024-05-09T09:18:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc4c8d7f740 10 client.27311 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc4c8d7f740 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc4c8d7f740 20 client.27311 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96f5cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc4c8d7f740 4 client.27311 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:18:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.245+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc4b80075c0 con 0x55ae96ef42b0 2024-05-09T09:18:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.246+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96f5cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.247+0000 7fc49effd640 10 client.27311 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.247+0000 7fc4bf7fe640 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96f5cd60 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d4f404566ef55 server_cookie=a00c8c232f591c7a in_seq=0 out_seq=0 2024-05-09T09:18:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.281+0000 7fc49effd640 1 -- 172.21.15.64:0/2046633866 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc4b80075c0 con 0x55ae96ef42b0 2024-05-09T09:18:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.281+0000 7fc49effd640 10 client.27311 handle_command_reply: tid=0 2024-05-09T09:18:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.281+0000 7fc4c8d7f740 1 client.27311 shutdown 2024-05-09T09:18:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.282+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 msgr2=0x55ae96eed250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.282+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96eed250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4b409ad00 tx=0x7fc4b409ac00 comp rx=0 tx=0).stop 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.282+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 shutdown_connections 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.282+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae96ef42b0 0x55ae96f5cd60 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.282+0000 7fc49effd640 0 client.27311 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.283+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae96ee3f60 0x55ae96eee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.283+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae96ededf0 0x55ae96eed250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.283+0000 7fc4c8d7f740 1 --2- 172.21.15.64:0/2046633866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae96ede400 0x55ae96eecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.283+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 >> 172.21.15.64:0/2046633866 conn(0x55ae96ed9a70 msgr2=0x55ae96edd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.283+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 shutdown_connections 2024-05-09T09:18:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.283+0000 7fc4c8d7f740 1 -- 172.21.15.64:0/2046633866 wait complete. 2024-05-09T09:18:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.284+0000 7fc4c8d7f740 20 client.27311 trim_cache size 0 max 16384 2024-05-09T09:18:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:18:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:18:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:18:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4a80a2d90 msgr2=0x7fc4a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4a80a2d90 0x7fc4a80a5250 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0005ab0 tx=0x7fc4b0004b40 comp rx=0 tx=0).stop 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 msgr2=0x7fc4c0119730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c0119730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).stop 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 shutdown_connections 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc4a80a2d90 0x7fc4a80a5250 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c010ea20 0x7fc4c011e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c010bf20 0x7fc4c0119730 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 --2- 172.21.15.64:0/1908397120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0073680 0x7fc4c01191f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.285+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 >> 172.21.15.64:0/1908397120 conn(0x7fc4c006e890 msgr2=0x7fc4c0109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.286+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 shutdown_connections 2024-05-09T09:18:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:31.286+0000 7fc4c4d2e640 1 -- 172.21.15.64:0/1908397120 wait complete. 2024-05-09T09:18:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 490.0 s, 723.1 tps, lat 137.030 ms stddev 49.505 2024-05-09T09:18:32.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:32 smithi139 ceph-mon[33599]: pgmap v652: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.5 MiB/s wr, 834 op/s 2024-05-09T09:18:32.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:32 smithi064 ceph-mon[30149]: pgmap v652: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.5 MiB/s wr, 834 op/s 2024-05-09T09:18:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:32 smithi179 ceph-mon[34086]: pgmap v652: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.5 MiB/s wr, 834 op/s 2024-05-09T09:18:34.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:34 smithi139 ceph-mon[33599]: pgmap v653: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.3 MiB/s wr, 931 op/s 2024-05-09T09:18:34.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:34 smithi064 ceph-mon[30149]: pgmap v653: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.3 MiB/s wr, 931 op/s 2024-05-09T09:18:34.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:34 smithi179 ceph-mon[34086]: pgmap v653: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.3 MiB/s wr, 931 op/s 2024-05-09T09:18:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 495.0 s, 492.0 tps, lat 164.602 ms stddev 221.086 2024-05-09T09:18:36.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:36 smithi139 ceph-mon[33599]: pgmap v654: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 MiB/s wr, 897 op/s 2024-05-09T09:18:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:36 smithi064 ceph-mon[30149]: pgmap v654: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 MiB/s wr, 897 op/s 2024-05-09T09:18:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:36 smithi179 ceph-mon[34086]: pgmap v654: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 MiB/s wr, 897 op/s 2024-05-09T09:18:37.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:18:37.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:18:37.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.554+0000 7f9d9a037640 1 -- 172.21.15.64:0/4064186579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d94109940 msgr2=0x7f9d9410bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.554+0000 7f9d9a037640 1 --2- 172.21.15.64:0/4064186579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d94109940 0x7f9d9410bd30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009ca0 tx=0x7f9d8402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.555+0000 7f9d9a037640 1 -- 172.21.15.64:0/4064186579 shutdown_connections 2024-05-09T09:18:37.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.555+0000 7f9d9a037640 1 --2- 172.21.15.64:0/4064186579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410ebf0 0x7f9d941110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.555+0000 7f9d9a037640 1 --2- 172.21.15.64:0/4064186579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9410c270 0x7f9d9410e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.555+0000 7f9d9a037640 1 --2- 172.21.15.64:0/4064186579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d94109940 0x7f9d9410bd30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.555+0000 7f9d9a037640 1 -- 172.21.15.64:0/4064186579 >> 172.21.15.64:0/4064186579 conn(0x7f9d9406e6c0 msgr2=0x7f9d94070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.556+0000 7f9d9a037640 1 -- 172.21.15.64:0/4064186579 shutdown_connections 2024-05-09T09:18:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.556+0000 7f9d9a037640 1 -- 172.21.15.64:0/4064186579 wait complete. 2024-05-09T09:18:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 Processor -- start 2024-05-09T09:18:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 -- start start 2024-05-09T09:18:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d94109940 0x7f9d94119a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410c270 0x7f9d94119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9410ebf0 0x7f9d94118120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941128a0 con 0x7f9d9410c270 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94112720 con 0x7f9d9410ebf0 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.557+0000 7f9d9a037640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94112a20 con 0x7f9d94109940 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.558+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9410ebf0 0x7f9d94118120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.558+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9410ebf0 0x7f9d94118120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54374/0 (socket says 172.21.15.64:54374) 2024-05-09T09:18:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.558+0000 7f9d93fff640 1 -- 172.21.15.64:0/627224530 learned_addr learned my addr 172.21.15.64:0/627224530 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:37.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.558+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410c270 0x7f9d94119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.559+0000 7f9d92ffd640 1 -- 172.21.15.64:0/627224530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d94109940 msgr2=0x7f9d94119a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:18:37.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.559+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d94109940 0x7f9d94119a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.559+0000 7f9d92ffd640 1 -- 172.21.15.64:0/627224530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9410ebf0 msgr2=0x7f9d94118120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.559+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9410ebf0 0x7f9d94118120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.559+0000 7f9d92ffd640 1 -- 172.21.15.64:0/627224530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d94118820 con 0x7f9d9410c270 2024-05-09T09:18:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.559+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410c270 0x7f9d94119fd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.560+0000 7f9d90ff9640 1 -- 172.21.15.64:0/627224530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00ccc0 con 0x7f9d9410c270 2024-05-09T09:18:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.560+0000 7f9d90ff9640 1 -- 172.21.15.64:0/627224530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c0108e0 con 0x7f9d9410c270 2024-05-09T09:18:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.561+0000 7f9d90ff9640 1 -- 172.21.15.64:0/627224530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c018840 con 0x7f9d9410c270 2024-05-09T09:18:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.561+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d94118b10 con 0x7f9d9410c270 2024-05-09T09:18:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.561+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941b7660 con 0x7f9d9410c270 2024-05-09T09:18:37.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.564+0000 7f9d9ce16740 1 Processor -- start 2024-05-09T09:18:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 -- start start 2024-05-09T09:18:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e735c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 0x55d5e7361420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7363f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d5e72c0d00 con 0x55d5e7361a90 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d5e72684c0 con 0x55d5e735bf30 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.566+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5e72bccc0 con 0x55d5e735c920 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.567+0000 7f9d90ff9640 1 -- 172.21.15.64:0/627224530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d8c010a90 con 0x7f9d9410c270 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.567+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e735c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.567+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7363f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.567+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e735c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54386/0 (socket says 172.21.15.64:54386) 2024-05-09T09:18:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.567+0000 7f9d93fff640 1 -- 172.21.15.64:0/1951555879 learned_addr learned my addr 172.21.15.64:0/1951555879 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.568+0000 7f9d93fff640 1 -- 172.21.15.64:0/1951555879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 msgr2=0x55d5e7361420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.568+0000 7f9d93fff640 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 0x55d5e7361420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.568+0000 7f9d93fff640 1 -- 172.21.15.64:0/1951555879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 msgr2=0x55d5e7363f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.568+0000 7f9d93fff640 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7363f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.568+0000 7f9d93fff640 1 -- 172.21.15.64:0/1951555879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e709e7d0 con 0x55d5e735bf30 2024-05-09T09:18:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.568+0000 7f9d93fff640 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e735c350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800bcd0 tx=0x7f9d8800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af71c8d3f0b06b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.569+0000 7f9d90ff9640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2d90 0x7f9d7c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.569+0000 7f9d90ff9640 1 -- 172.21.15.64:0/627224530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c0cf790 con 0x7f9d9410c270 2024-05-09T09:18:37.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.570+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1951555879 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88015070 con 0x55d5e735bf30 2024-05-09T09:18:37.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.570+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1951555879 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8801a910 con 0x55d5e735bf30 2024-05-09T09:18:37.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.570+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1951555879 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d880118d0 con 0x55d5e735bf30 2024-05-09T09:18:37.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.572+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1951555879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 msgr2=0x55d5e735c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.572+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e735c350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800bcd0 tx=0x7f9d8800f730 comp rx=0 tx=0).stop 2024-05-09T09:18:37.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.572+0000 7f9d937fe640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2d90 0x7f9d7c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.573+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1951555879 shutdown_connections 2024-05-09T09:18:37.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.573+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7363f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.573+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 0x55d5e7361420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.573+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1951555879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e735c350 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.573+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1951555879 >> 172.21.15.64:0/1951555879 conn(0x55d5e73575a0 msgr2=0x55d5e73579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.574+0000 7f9d937fe640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2d90 0x7f9d7c0a5250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9d84031970 tx=0x7f9d84000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.574+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1951555879 shutdown_connections 2024-05-09T09:18:37.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.574+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1951555879 wait complete. 2024-05-09T09:18:37.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.575+0000 7f9d9ce16740 1 -- start start 2024-05-09T09:18:37.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.575+0000 7f9d9ce16740 1 Processor -- start 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e73663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e7366930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7361a90 0x55d5e736d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d5e7364a40 con 0x55d5e735c920 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d5e727bb90 con 0x55d5e735bf30 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5e7364bc0 con 0x55d5e7361a90 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.579+0000 7f9d9ce16740 10 client.0 _ll_register_callbacks cb 0x55d5e72b7da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.580+0000 7f9d937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e73663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.580+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e7366930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.580+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e7366930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42488/0 (socket says 172.21.15.64:42488) 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.580+0000 7f9d92ffd640 1 -- 172.21.15.64:0/1247552559 learned_addr learned my addr 172.21.15.64:0/1247552559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:37.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d92ffd640 1 -- 172.21.15.64:0/1247552559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7361a90 msgr2=0x55d5e736d970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7361a90 0x55d5e736d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d92ffd640 1 -- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 msgr2=0x55d5e73663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e73663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d92ffd640 1 -- 172.21.15.64:0/1247552559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e6fd43f0 con 0x55d5e735c920 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d92ffd640 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e7366930 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0985d0 tx=0x7f9d8c09ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667ca7d6e9ed3a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.581+0000 7f9d72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.582+0000 7f9d72ffd640 1 -- 172.21.15.64:0/1247552559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0cfdc0 con 0x55d5e735c920 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.582+0000 7f9d72ffd640 1 -- 172.21.15.64:0/1247552559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c010430 con 0x55d5e735c920 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.582+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8801a910 con 0x55d5e735c920 2024-05-09T09:18:37.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.583+0000 7f9d72ffd640 1 -- 172.21.15.64:0/1247552559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c0b2050 con 0x55d5e735c920 2024-05-09T09:18:37.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.584+0000 7f9d9ce16740 10 client.17802 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:37.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.584+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d5e6f940f0 con 0x55d5e735c920 2024-05-09T09:18:37.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.585+0000 7f9d72ffd640 1 -- 172.21.15.64:0/1247552559 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8801a910 con 0x55d5e735c920 2024-05-09T09:18:37.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d72ffd640 1 -- 172.21.15.64:0/1247552559 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9d8c0d2ce0 con 0x55d5e735c920 2024-05-09T09:18:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d9ce16740 10 client.17802 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d9ce16740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d9ce16740 20 client.17802 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73721e0 0x55d5e7374680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d9ce16740 4 client.17802 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:37.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.586+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e72ce430 con 0x55d5e73721e0 2024-05-09T09:18:37.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.587+0000 7f9d937fe640 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73721e0 0x55d5e7374680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.588+0000 7f9d937fe640 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73721e0 0x55d5e7374680 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf81e736150b7f0 server_cookie=e19d143698c374d0 in_seq=0 out_seq=0 2024-05-09T09:18:37.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.589+0000 7f9d72ffd640 10 client.17802 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:37.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.590+0000 7f9d72ffd640 1 -- 172.21.15.64:0/1247552559 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5e72ce430 con 0x55d5e73721e0 2024-05-09T09:18:37.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.590+0000 7f9d72ffd640 10 client.17802 handle_command_reply: tid=0 2024-05-09T09:18:37.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.590+0000 7f9d9ce16740 1 client.17802 shutdown 2024-05-09T09:18:37.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.590+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 msgr2=0x55d5e7366930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.590+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e7366930 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0985d0 tx=0x7f9d8c09ba90 comp rx=0 tx=0).stop 2024-05-09T09:18:37.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 shutdown_connections 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73721e0 0x55d5e7374680 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7361a90 0x55d5e736d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e7366930 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1247552559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e73663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 >> 172.21.15.64:0/1247552559 conn(0x55d5e73575a0 msgr2=0x55d5e73567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d72ffd640 0 client.17802 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:37.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 shutdown_connections 2024-05-09T09:18:37.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1247552559 wait complete. 2024-05-09T09:18:37.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.591+0000 7f9d9ce16740 20 client.17802 trim_cache size 0 max 16384 2024-05-09T09:18:37.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.632+0000 7f9d9ce16740 1 Processor -- start 2024-05-09T09:18:37.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.632+0000 7f9d9ce16740 1 -- start start 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e7352670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 0x55d5e736d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7356660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94080600 con 0x55d5e7361a90 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9407ded0 con 0x55d5e735bf30 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9407bd20 con 0x55d5e735c920 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7356660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7356660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42496/0 (socket says 172.21.15.64:42496) 2024-05-09T09:18:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.633+0000 7f9d93fff640 1 -- 172.21.15.64:0/1069415302 learned_addr learned my addr 172.21.15.64:0/1069415302 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.634+0000 7f9d93fff640 1 -- 172.21.15.64:0/1069415302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 msgr2=0x55d5e736d930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.634+0000 7f9d93fff640 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 0x55d5e736d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.634+0000 7f9d93fff640 1 -- 172.21.15.64:0/1069415302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 msgr2=0x55d5e7352670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.634+0000 7f9d93fff640 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e7352670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.634+0000 7f9d93fff640 1 -- 172.21.15.64:0/1069415302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e73573d0 con 0x55d5e7361a90 2024-05-09T09:18:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.635+0000 7f9d93fff640 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7356660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005d00 tx=0x7f9d8800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789abad80a584e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.635+0000 7f9d797fa640 1 -- 172.21.15.64:0/1069415302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88015070 con 0x55d5e7361a90 2024-05-09T09:18:37.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.635+0000 7f9d797fa640 1 -- 172.21.15.64:0/1069415302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8800b7e0 con 0x55d5e7361a90 2024-05-09T09:18:37.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.635+0000 7f9d797fa640 1 -- 172.21.15.64:0/1069415302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88004d30 con 0x55d5e7361a90 2024-05-09T09:18:37.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1069415302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 msgr2=0x55d5e7356660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7356660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005d00 tx=0x7f9d8800bf50 comp rx=0 tx=0).stop 2024-05-09T09:18:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1069415302 shutdown_connections 2024-05-09T09:18:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e7361a90 0x55d5e7356660 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735c920 0x55d5e736d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/1069415302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e735bf30 0x55d5e7352670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.640+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1069415302 >> 172.21.15.64:0/1069415302 conn(0x55d5e73575a0 msgr2=0x55d5e7355f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.644+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1069415302 shutdown_connections 2024-05-09T09:18:37.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.647+0000 7f9d9ce16740 1 -- 172.21.15.64:0/1069415302 wait complete. 2024-05-09T09:18:37.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.647+0000 7f9d9ce16740 1 -- start start 2024-05-09T09:18:37.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.648+0000 7f9d9ce16740 1 Processor -- start 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.648+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735bf30 0x55d5e736b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.648+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e736ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.648+0000 7f9d9ce16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 0x55d5e736ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9407c390 con 0x55d5e735c920 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9407d800 con 0x55d5e7361a90 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d9ce16740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9407ca60 con 0x55d5e735bf30 2024-05-09T09:18:37.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d9ce16740 10 client.0 _ll_register_callbacks cb 0x7f9d8c097ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:37.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d937fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735bf30 0x55d5e736b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 0x55d5e736ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 0x55d5e736ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54402/0 (socket says 172.21.15.64:54402) 2024-05-09T09:18:37.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 -- 172.21.15.64:0/2683354373 learned_addr learned my addr 172.21.15.64:0/2683354373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 -- 172.21.15.64:0/2683354373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735bf30 msgr2=0x55d5e736b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735bf30 0x55d5e736b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 -- 172.21.15.64:0/2683354373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 msgr2=0x55d5e736ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e736ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.649+0000 7f9d93fff640 1 -- 172.21.15.64:0/2683354373 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e736d020 con 0x55d5e7361a90 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.650+0000 7f9d93fff640 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 0x55d5e736ca30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800b4e0 tx=0x7f9d8801aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8759f1cc2a49ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.650+0000 7f9d72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.650+0000 7f9d72ffd640 1 -- 172.21.15.64:0/2683354373 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88015070 con 0x55d5e7361a90 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.650+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8800b7e0 con 0x55d5e7361a90 2024-05-09T09:18:37.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.650+0000 7f9d72ffd640 1 -- 172.21.15.64:0/2683354373 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8802d470 con 0x55d5e7361a90 2024-05-09T09:18:37.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.652+0000 7f9d72ffd640 1 -- 172.21.15.64:0/2683354373 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9d88036050 con 0x55d5e7361a90 2024-05-09T09:18:37.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.652+0000 7f9d9ce16740 10 client.27326 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:37.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.652+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d5e73d8c30 con 0x55d5e7361a90 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d72ffd640 1 -- 172.21.15.64:0/2683354373 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d88028070 con 0x55d5e7361a90 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d72ffd640 1 -- 172.21.15.64:0/2683354373 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9d880114e0 con 0x55d5e7361a90 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d9ce16740 10 client.27326 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d9ce16740 10 client.27326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d9ce16740 20 client.27326 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73d9450 0x55d5e73db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d9ce16740 4 client.27326 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:18:37.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.653+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9d8800b7e0 con 0x55d5e73d9450 2024-05-09T09:18:37.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.654+0000 7f9d937fe640 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73d9450 0x55d5e73db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d937fe640 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73d9450 0x55d5e73db870 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662f0358184eb07f server_cookie=291c0545b8ccc40e in_seq=0 out_seq=0 2024-05-09T09:18:37.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d72ffd640 10 client.27326 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:37.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d72ffd640 1 -- 172.21.15.64:0/2683354373 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9d8800b7e0 con 0x55d5e73d9450 2024-05-09T09:18:37.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d72ffd640 10 client.27326 handle_command_reply: tid=0 2024-05-09T09:18:37.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d9ce16740 1 client.27326 shutdown 2024-05-09T09:18:37.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 msgr2=0x55d5e736ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.655+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 0x55d5e736ca30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800b4e0 tx=0x7f9d8801aa00 comp rx=0 tx=0).stop 2024-05-09T09:18:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.657+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 shutdown_connections 2024-05-09T09:18:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.657+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d5e73d9450 0x55d5e73db870 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.657+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d5e7361a90 0x55d5e736ca30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.657+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5e735c920 0x55d5e736ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.657+0000 7f9d9ce16740 1 --2- 172.21.15.64:0/2683354373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e735bf30 0x55d5e736b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.657+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 >> 172.21.15.64:0/2683354373 conn(0x55d5e73575a0 msgr2=0x55d5e73567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.659+0000 7f9d72ffd640 0 client.27326 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.659+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 shutdown_connections 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.660+0000 7f9d9ce16740 1 -- 172.21.15.64:0/2683354373 wait complete. 2024-05-09T09:18:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.660+0000 7f9d9ce16740 20 client.27326 trim_cache size 0 max 16384 2024-05-09T09:18:37.660 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:18:37.660 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:18:37.660 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:18:37.660 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:18:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.664+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2d90 msgr2=0x7f9d7c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.664+0000 7f9d9a037640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2d90 0x7f9d7c0a5250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9d84031970 tx=0x7f9d84000f40 comp rx=0 tx=0).stop 2024-05-09T09:18:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.664+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410c270 msgr2=0x7f9d94119fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.664+0000 7f9d9a037640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410c270 0x7f9d94119fd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00eeb0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.666+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 shutdown_connections 2024-05-09T09:18:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.666+0000 7f9d9a037640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2d90 0x7f9d7c0a5250 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.666+0000 7f9d9a037640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9410ebf0 0x7f9d94118120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.666+0000 7f9d9a037640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9410c270 0x7f9d94119fd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.667+0000 7f9d9a037640 1 --2- 172.21.15.64:0/627224530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d94109940 0x7f9d94119a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.667+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 >> 172.21.15.64:0/627224530 conn(0x7f9d9406e6c0 msgr2=0x7f9d94109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.668+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 shutdown_connections 2024-05-09T09:18:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.670+0000 7f9d9a037640 1 -- 172.21.15.64:0/627224530 wait complete. 2024-05-09T09:18:37.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:18:37.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:18:37.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.903+0000 7f94ce693640 1 -- 172.21.15.64:0/2254227774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 msgr2=0x7f94c810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:37.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.903+0000 7f94ce693640 1 --2- 172.21.15.64:0/2254227774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 0x7f94c810ac60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009900 tx=0x7f94b802a410 comp rx=0 tx=0).stop 2024-05-09T09:18:37.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.904+0000 7f94ce693640 1 -- 172.21.15.64:0/2254227774 shutdown_connections 2024-05-09T09:18:37.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.904+0000 7f94ce693640 1 --2- 172.21.15.64:0/2254227774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810db70 0x7f94c810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.904+0000 7f94ce693640 1 --2- 172.21.15.64:0/2254227774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94c810b1a0 0x7f94c810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.904+0000 7f94ce693640 1 --2- 172.21.15.64:0/2254227774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 0x7f94c810ac60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.904+0000 7f94ce693640 1 -- 172.21.15.64:0/2254227774 >> 172.21.15.64:0/2254227774 conn(0x7f94c806e6d0 msgr2=0x7f94c8070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:37.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.904+0000 7f94ce693640 1 -- 172.21.15.64:0/2254227774 shutdown_connections 2024-05-09T09:18:37.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.905+0000 7f94ce693640 1 -- 172.21.15.64:0/2254227774 wait complete. 2024-05-09T09:18:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.906+0000 7f94ce693640 1 Processor -- start 2024-05-09T09:18:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.906+0000 7f94ce693640 1 -- start start 2024-05-09T09:18:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94ce693640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 0x7f94c8114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94ce693640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 0x7f94c81154b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94ce693640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94c810db70 0x7f94c81b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94ce693640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94c811c220 con 0x7f94c810b1a0 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94ce693640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94c811c0a0 con 0x7f94c8108830 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94ce693640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94c811c3a0 con 0x7f94c810db70 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 0x7f94c81154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 0x7f94c81154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42514/0 (socket says 172.21.15.64:42514) 2024-05-09T09:18:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.907+0000 7f94c77fe640 1 -- 172.21.15.64:0/301040085 learned_addr learned my addr 172.21.15.64:0/301040085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.909+0000 7f94c77fe640 1 -- 172.21.15.64:0/301040085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94c810db70 msgr2=0x7f94c81b6910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:18:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.909+0000 7f94c77fe640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94c810db70 0x7f94c81b6910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.909+0000 7f94c77fe640 1 -- 172.21.15.64:0/301040085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 msgr2=0x7f94c8114f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:18:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.909+0000 7f94c77fe640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 0x7f94c8114f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.909+0000 7f94c77fe640 1 -- 172.21.15.64:0/301040085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c81b6e50 con 0x7f94c810b1a0 2024-05-09T09:18:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.909+0000 7f94c77fe640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 0x7f94c81154b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d950 tx=0x7f94bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.910+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc017070 con 0x7f94c810b1a0 2024-05-09T09:18:37.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.911+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94bc009c30 con 0x7f94c810b1a0 2024-05-09T09:18:37.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.912+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc017070 con 0x7f94c810b1a0 2024-05-09T09:18:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.913+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c81b7020 con 0x7f94c810b1a0 2024-05-09T09:18:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.913+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81b7520 con 0x7f94c810b1a0 2024-05-09T09:18:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.915+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c8068650 con 0x7f94c810b1a0 2024-05-09T09:18:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.924+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94bc01a070 con 0x7f94c810b1a0 2024-05-09T09:18:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.925+0000 7f94c57fa640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94b00a3050 0x7f94b00a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.925+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f94bc009a30 con 0x7f94c810b1a0 2024-05-09T09:18:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.926+0000 7f94c7fff640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94b00a3050 0x7f94b00a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.927+0000 7f94c7fff640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94b00a3050 0x7f94b00a5510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f94b80096c0 tx=0x7f94b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:37.927+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f94bc09b000 con 0x7f94c810b1a0 2024-05-09T09:18:38.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.415+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94c80711d0 con 0x7f94c810b1a0 2024-05-09T09:18:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.417+0000 7f94c57fa640 1 -- 172.21.15.64:0/301040085 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11879 (secure 0 0 0) 0x7f94bc09b000 con 0x7f94c810b1a0 2024-05-09T09:18:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:18:38.417 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:17:16.247105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:18:38.418 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 45 2024-05-09T09:18:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94b00a3050 msgr2=0x7f94b00a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94b00a3050 0x7f94b00a5510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f94b80096c0 tx=0x7f94b8005fd0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 msgr2=0x7f94c81154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 0x7f94c81154b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d950 tx=0x7f94bc00de10 comp rx=0 tx=0).stop 2024-05-09T09:18:38.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 shutdown_connections 2024-05-09T09:18:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f94b00a3050 0x7f94b00a5510 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94c810db70 0x7f94c81b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94c810b1a0 0x7f94c81154b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.421+0000 7f94ce693640 1 --2- 172.21.15.64:0/301040085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c8108830 0x7f94c8114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.422+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 >> 172.21.15.64:0/301040085 conn(0x7f94c806e6d0 msgr2=0x7f94c810c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.423+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 shutdown_connections 2024-05-09T09:18:38.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.423+0000 7f94ce693640 1 -- 172.21.15.64:0/301040085 wait complete. 2024-05-09T09:18:38.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:18:38.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 -- 172.21.15.64:0/417513179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c100280 msgr2=0x7f341c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 --2- 172.21.15.64:0/417513179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c100280 0x7f341c100fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f340c009860 tx=0x7f340c02a360 comp rx=0 tx=0).stop 2024-05-09T09:18:38.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 -- 172.21.15.64:0/417513179 shutdown_connections 2024-05-09T09:18:38.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 --2- 172.21.15.64:0/417513179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c101500 0x7f341c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 --2- 172.21.15.64:0/417513179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c100280 0x7f341c100fc0 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f340c009860 tx=0x7f340c02a360 comp rx=0 tx=0).stop 2024-05-09T09:18:38.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 --2- 172.21.15.64:0/417513179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f341c0681c0 0x7f341c0ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.684+0000 7f34223d6640 1 -- 172.21.15.64:0/417513179 >> 172.21.15.64:0/417513179 conn(0x7f341c0fb360 msgr2=0x7f341c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.686+0000 7f34223d6640 1 -- 172.21.15.64:0/417513179 shutdown_connections 2024-05-09T09:18:38.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.687+0000 7f34223d6640 1 -- 172.21.15.64:0/417513179 wait complete. 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.687+0000 7f34223d6640 1 Processor -- start 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.687+0000 7f34223d6640 1 -- start start 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f34223d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c0681c0 0x7f341c0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f34223d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f341c101500 0x7f341c070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f34223d6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 0x7f341c0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f34223d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1050d0 con 0x7f341c101500 2024-05-09T09:18:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f34223d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f341c104f50 con 0x7f341c0681c0 2024-05-09T09:18:38.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f34223d6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f341c105250 con 0x7f341c071f40 2024-05-09T09:18:38.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.688+0000 7f3421bd5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 0x7f341c0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 0x7f341c0723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35148/0 (socket says 172.21.15.64:35148) 2024-05-09T09:18:38.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 -- 172.21.15.64:0/1251799059 learned_addr learned my addr 172.21.15.64:0/1251799059 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:38.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f34213d4640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c0681c0 0x7f341c0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 -- 172.21.15.64:0/1251799059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c0681c0 msgr2=0x7f341c0709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c0681c0 0x7f341c0709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 -- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f341c101500 msgr2=0x7f341c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f341c101500 0x7f341c070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 -- 172.21.15.64:0/1251799059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c0729b0 con 0x7f341c071f40 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.689+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 0x7f341c0723f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f341400d680 tx=0x7f34140071c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:38.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.690+0000 7f34127fc640 1 -- 172.21.15.64:0/1251799059 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414019020 con 0x7f341c071f40 2024-05-09T09:18:38.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.690+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c072ca0 con 0x7f341c071f40 2024-05-09T09:18:38.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.690+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c0731f0 con 0x7f341c071f40 2024-05-09T09:18:38.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.690+0000 7f34127fc640 1 -- 172.21.15.64:0/1251799059 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3414007e00 con 0x7f341c071f40 2024-05-09T09:18:38.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.690+0000 7f34127fc640 1 -- 172.21.15.64:0/1251799059 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414015400 con 0x7f341c071f40 2024-05-09T09:18:38.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.693+0000 7f342641e740 1 Processor -- start 2024-05-09T09:18:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.697+0000 7f34127fc640 1 -- 172.21.15.64:0/1251799059 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f341400c030 con 0x7f341c071f40 2024-05-09T09:18:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.698+0000 7f34127fc640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34080a2c60 0x7f34080a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.698+0000 7f34127fc640 1 -- 172.21.15.64:0/1251799059 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f34140cf200 con 0x7f341c071f40 2024-05-09T09:18:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.698+0000 7f3420bd3640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34080a2c60 0x7f34080a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.699+0000 7f342641e740 1 -- start start 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.699+0000 7f342641e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c4e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.699+0000 7f342641e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c53620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.699+0000 7f342641e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566c56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.699+0000 7f342641e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562566bb2c40 con 0x562566c53c90 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.699+0000 7f342641e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562566b5a580 con 0x562566c4e130 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.700+0000 7f342641e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562566baec00 con 0x562566c4eb20 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566c56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566c56120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42538/0 (socket says 172.21.15.64:42538) 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 -- 172.21.15.64:0/1821324655 learned_addr learned my addr 172.21.15.64:0/1821324655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 -- 172.21.15.64:0/1821324655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 msgr2=0x562566c53620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c53620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 -- 172.21.15.64:0/1821324655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 msgr2=0x562566c4e550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c4e550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.701+0000 7f3421bd5640 1 -- 172.21.15.64:0/1821324655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625668c63f0 con 0x562566c53c90 2024-05-09T09:18:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.702+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566c56120 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f341409b690 tx=0x7f341409b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bfa80fa6692acd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.704+0000 7f3420bd3640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34080a2c60 0x7f34080a5120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f340c000c00 tx=0x7f340c0056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.704+0000 7f34077fe640 1 -- 172.21.15.64:0/1821324655 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414099a00 con 0x562566c53c90 2024-05-09T09:18:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.706+0000 7f34077fe640 1 -- 172.21.15.64:0/1821324655 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34140cca20 con 0x562566c53c90 2024-05-09T09:18:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.706+0000 7f34077fe640 1 -- 172.21.15.64:0/1821324655 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140a3e50 con 0x562566c53c90 2024-05-09T09:18:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.709+0000 7f342641e740 1 -- 172.21.15.64:0/1821324655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 msgr2=0x562566c56120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.709+0000 7f342641e740 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566c56120 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f341409b690 tx=0x7f341409b700 comp rx=0 tx=0).stop 2024-05-09T09:18:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.710+0000 7f342641e740 1 -- 172.21.15.64:0/1821324655 shutdown_connections 2024-05-09T09:18:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.710+0000 7f342641e740 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566c56120 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.710+0000 7f342641e740 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c53620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.710+0000 7f342641e740 1 --2- 172.21.15.64:0/1821324655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c4e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.710+0000 7f342641e740 1 -- 172.21.15.64:0/1821324655 >> 172.21.15.64:0/1821324655 conn(0x562566c497a0 msgr2=0x562566c49ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.712+0000 7f342641e740 1 -- 172.21.15.64:0/1821324655 shutdown_connections 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.712+0000 7f342641e740 1 -- 172.21.15.64:0/1821324655 wait complete. 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.713+0000 7f342641e740 1 -- start start 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.713+0000 7f342641e740 1 Processor -- start 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4e130 0x562566c58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4eb20 0x562566c58b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c53c90 0x562566c5fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562566c56c40 con 0x562566c4eb20 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562566b6dc40 con 0x562566c53c90 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562566c56dc0 con 0x562566c4e130 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f342641e740 10 client.0 _ll_register_callbacks cb 0x5625668c5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f34213d4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4e130 0x562566c58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f34213d4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4e130 0x562566c58610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35158/0 (socket says 172.21.15.64:35158) 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f34213d4640 1 -- 172.21.15.64:0/3213101058 learned_addr learned my addr 172.21.15.64:0/3213101058 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.716+0000 7f3421bd5640 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c53c90 0x562566c5fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3421bd5640 1 -- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4e130 msgr2=0x562566c58610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3421bd5640 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4e130 0x562566c58610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3421bd5640 1 -- 172.21.15.64:0/3213101058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4eb20 msgr2=0x562566c58b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3421bd5640 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4eb20 0x562566c58b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3421bd5640 1 -- 172.21.15.64:0/3213101058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625669907d0 con 0x562566c53c90 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3421bd5640 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c53c90 0x562566c5fb90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f341409b690 tx=0x7f34140ccbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd9766d5476a25f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.717+0000 7f3404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.718+0000 7f3404ff9640 1 -- 172.21.15.64:0/3213101058 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140afb90 con 0x562566c53c90 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.718+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34140cca20 con 0x562566c53c90 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.718+0000 7f3404ff9640 1 -- 172.21.15.64:0/3213101058 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f34140afd30 con 0x562566c53c90 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.718+0000 7f3404ff9640 1 -- 172.21.15.64:0/3213101058 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f341409ab50 con 0x562566c53c90 2024-05-09T09:18:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.718+0000 7f342641e740 10 client.27338 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.718+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562566c62f00 con 0x562566c53c90 2024-05-09T09:18:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.719+0000 7f3404ff9640 1 -- 172.21.15.64:0/3213101058 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34140cca20 con 0x562566c53c90 2024-05-09T09:18:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.719+0000 7f3404ff9640 1 -- 172.21.15.64:0/3213101058 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f3414005140 con 0x562566c53c90 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.719+0000 7f342641e740 10 client.27338 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.719+0000 7f342641e740 10 client.27338 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.719+0000 7f342641e740 20 client.27338 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.719+0000 7f342641e740 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566c646a0 0x562566c66ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.720+0000 7f342641e740 4 client.27338 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.720+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562566bc0370 con 0x562566c646a0 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.720+0000 7f34213d4640 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566c646a0 0x562566c66ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.720+0000 7f3404ff9640 10 client.27338 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.721+0000 7f34213d4640 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566c646a0 0x562566c66ae0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0d3de5508078cbc server_cookie=e9f94bdcb3e5855f in_seq=0 out_seq=0 2024-05-09T09:18:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.722+0000 7f3404ff9640 1 -- 172.21.15.64:0/3213101058 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562566bc0370 con 0x562566c646a0 2024-05-09T09:18:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.722+0000 7f3404ff9640 10 client.27338 handle_command_reply: tid=0 2024-05-09T09:18:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.723+0000 7f342641e740 1 client.27338 shutdown 2024-05-09T09:18:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c53c90 msgr2=0x562566c5fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c53c90 0x562566c5fb90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f341409b690 tx=0x7f34140ccbd0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 shutdown_connections 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566c646a0 0x562566c66ae0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c53c90 0x562566c5fb90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4eb20 0x562566c58b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 --2- 172.21.15.64:0/3213101058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4e130 0x562566c58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 >> 172.21.15.64:0/3213101058 conn(0x562566c497a0 msgr2=0x562566c489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f3404ff9640 0 client.27338 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 shutdown_connections 2024-05-09T09:18:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.724+0000 7f342641e740 1 -- 172.21.15.64:0/3213101058 wait complete. 2024-05-09T09:18:38.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.728+0000 7f342641e740 20 client.27338 trim_cache size 0 max 16384 2024-05-09T09:18:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.758+0000 7f342641e740 1 Processor -- start 2024-05-09T09:18:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 -- start start 2024-05-09T09:18:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 0x562566c666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4eb20 0x562566c5fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c53c90 0x562566c48840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34000125a0 con 0x562566c4e130 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f340000fe70 con 0x562566c4eb20 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.759+0000 7f342641e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f340000dcc0 con 0x562566c53c90 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 0x562566c666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 0x562566c666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42558/0 (socket says 172.21.15.64:42558) 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 -- 172.21.15.64:0/1806489693 learned_addr learned my addr 172.21.15.64:0/1806489693 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c53c90 0x562566c48840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 -- 172.21.15.64:0/1806489693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c53c90 msgr2=0x562566c48840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c53c90 0x562566c48840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 -- 172.21.15.64:0/1806489693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4eb20 msgr2=0x562566c5fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4eb20 0x562566c5fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f34213d4640 1 -- 172.21.15.64:0/1806489693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562566c49340 con 0x562566c4e130 2024-05-09T09:18:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.760+0000 7f3421bd5640 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c53c90 0x562566c48840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.761+0000 7f34213d4640 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 0x562566c666b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3418009b90 tx=0x7f3418004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b905e4ed814e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.761+0000 7f3406ffd640 1 -- 172.21.15.64:0/1806489693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418022070 con 0x562566c4e130 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.762+0000 7f3406ffd640 1 -- 172.21.15.64:0/1806489693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341801d8e0 con 0x562566c4e130 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.762+0000 7f3406ffd640 1 -- 172.21.15.64:0/1806489693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418010bf0 con 0x562566c4e130 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 -- 172.21.15.64:0/1806489693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 msgr2=0x562566c666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 0x562566c666b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3418009b90 tx=0x7f3418004410 comp rx=0 tx=0).stop 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 -- 172.21.15.64:0/1806489693 shutdown_connections 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c53c90 0x562566c48840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4eb20 0x562566c5fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 --2- 172.21.15.64:0/1806489693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c4e130 0x562566c666b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 -- 172.21.15.64:0/1806489693 >> 172.21.15.64:0/1806489693 conn(0x562566c497a0 msgr2=0x562566c48130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 -- 172.21.15.64:0/1806489693 shutdown_connections 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 -- 172.21.15.64:0/1806489693 wait complete. 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 -- start start 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.763+0000 7f342641e740 1 Processor -- start 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c5ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c5cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566cc8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f340000e330 con 0x562566c53c90 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f340000f7a0 con 0x562566c4e130 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f340000ea00 con 0x562566c4eb20 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f342641e740 10 client.0 _ll_register_callbacks cb 0x7f341409f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f34213d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c5ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f34213d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c5ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54474/0 (socket says 172.21.15.64:54474) 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f34213d4640 1 -- 172.21.15.64:0/4220274471 learned_addr learned my addr 172.21.15.64:0/4220274471 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:38.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.764+0000 7f3420bd3640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c5cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.765+0000 7f3421bd5640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566cc8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f34213d4640 1 -- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 msgr2=0x562566c5cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f34213d4640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c5cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f34213d4640 1 -- 172.21.15.64:0/4220274471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 msgr2=0x562566cc8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f34213d4640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566cc8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f34213d4640 1 -- 172.21.15.64:0/4220274471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562566cc9150 con 0x562566c4e130 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f34213d4640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c5ca40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3418009f20 tx=0x7f3418010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64630773b171f11a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.766+0000 7f3404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.767+0000 7f3404ff9640 1 -- 172.21.15.64:0/4220274471 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341802c030 con 0x562566c4e130 2024-05-09T09:18:38.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.767+0000 7f3404ff9640 1 -- 172.21.15.64:0/4220274471 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f341801d640 con 0x562566c4e130 2024-05-09T09:18:38.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.769+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341801d8e0 con 0x562566c4e130 2024-05-09T09:18:38.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.775+0000 7f3404ff9640 1 -- 172.21.15.64:0/4220274471 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3418022070 con 0x562566c4e130 2024-05-09T09:18:38.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.776+0000 7f342641e740 10 client.27341 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:38.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.776+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562566cca400 con 0x562566c4e130 2024-05-09T09:18:38.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f3404ff9640 1 -- 172.21.15.64:0/4220274471 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341801d8e0 con 0x562566c4e130 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f3404ff9640 1 -- 172.21.15.64:0/4220274471 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f3418028660 con 0x562566c4e130 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f342641e740 10 client.27341 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f342641e740 10 client.27341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f342641e740 20 client.27341 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f342641e740 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566ccad80 0x562566ccd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f342641e740 4 client.27341 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:18:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.777+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562566bba6c0 con 0x562566ccad80 2024-05-09T09:18:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.782+0000 7f3420bd3640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566ccad80 0x562566ccd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.783+0000 7f3420bd3640 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566ccad80 0x562566ccd1a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7c389e34e186203 server_cookie=c2a5e1d4a6865d13 in_seq=0 out_seq=0 2024-05-09T09:18:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.783+0000 7f3404ff9640 10 client.27341 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.783+0000 7f3404ff9640 1 -- 172.21.15.64:0/4220274471 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562566bba6c0 con 0x562566ccad80 2024-05-09T09:18:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.783+0000 7f3404ff9640 10 client.27341 handle_command_reply: tid=0 2024-05-09T09:18:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.783+0000 7f342641e740 1 client.27341 shutdown 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 msgr2=0x562566c5ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c5ca40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3418009f20 tx=0x7f3418010430 comp rx=0 tx=0).stop 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 shutdown_connections 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562566ccad80 0x562566ccd1a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562566c53c90 0x562566cc8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562566c4eb20 0x562566c5cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 --2- 172.21.15.64:0/4220274471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562566c4e130 0x562566c5ca40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.784+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 >> 172.21.15.64:0/4220274471 conn(0x562566c497a0 msgr2=0x562566c489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.785+0000 7f3404ff9640 0 client.27341 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:18:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.788+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 shutdown_connections 2024-05-09T09:18:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.789+0000 7f342641e740 1 -- 172.21.15.64:0/4220274471 wait complete. 2024-05-09T09:18:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.789+0000 7f342641e740 20 client.27341 trim_cache size 0 max 16384 2024-05-09T09:18:38.789 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:18:38.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34080a2c60 msgr2=0x7f34080a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34080a2c60 0x7f34080a5120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f340c000c00 tx=0x7f340c0056d0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 msgr2=0x7f341c0723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:38.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 0x7f341c0723f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f341400d680 tx=0x7f34140071c0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 shutdown_connections 2024-05-09T09:18:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34080a2c60 0x7f34080a5120 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c071f40 0x7f341c0723f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f341c101500 0x7f341c070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 --2- 172.21.15.64:0/1251799059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f341c0681c0 0x7f341c0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.792+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 >> 172.21.15.64:0/1251799059 conn(0x7f341c0fb360 msgr2=0x7f341c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:38.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.800+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 shutdown_connections 2024-05-09T09:18:38.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:38.801+0000 7f34223d6640 1 -- 172.21.15.64:0/1251799059 wait complete. 2024-05-09T09:18:38.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:38 smithi139 ceph-mon[33599]: pgmap v655: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 823 op/s 2024-05-09T09:18:38.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/301040085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:18:38.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:38 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:38.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:18:38.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:38 smithi064 ceph-mon[30149]: pgmap v655: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 823 op/s 2024-05-09T09:18:38.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/301040085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:18:38.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:38 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:38.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:38 smithi179 ceph-mon[34086]: pgmap v655: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 823 op/s 2024-05-09T09:18:38.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/301040085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:18:38.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:38 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:18:39.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.056+0000 7f9e60b8c640 1 -- 172.21.15.64:0/607403101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 msgr2=0x7f9e5c074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.056+0000 7f9e60b8c640 1 --2- 172.21.15.64:0/607403101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 0x7f9e5c074920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009900 tx=0x7f9e5002a410 comp rx=0 tx=0).stop 2024-05-09T09:18:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.058+0000 7f9e60b8c640 1 -- 172.21.15.64:0/607403101 shutdown_connections 2024-05-09T09:18:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.058+0000 7f9e60b8c640 1 --2- 172.21.15.64:0/607403101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c074e60 0x7f9e5c07d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.058+0000 7f9e60b8c640 1 --2- 172.21.15.64:0/607403101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 0x7f9e5c074920 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.058+0000 7f9e60b8c640 1 --2- 172.21.15.64:0/607403101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c073360 0x7f9e5c073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.058+0000 7f9e60b8c640 1 -- 172.21.15.64:0/607403101 >> 172.21.15.64:0/607403101 conn(0x7f9e5c06e740 msgr2=0x7f9e5c070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.060+0000 7f9e60b8c640 1 -- 172.21.15.64:0/607403101 shutdown_connections 2024-05-09T09:18:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.060+0000 7f9e60b8c640 1 -- 172.21.15.64:0/607403101 wait complete. 2024-05-09T09:18:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 Processor -- start 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 -- start start 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 0x7f9e5c1b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 0x7f9e5c1b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c074e60 0x7f9e5c1b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c07eb20 con 0x7f9e5c0744a0 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c07e9a0 con 0x7f9e5c073360 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e60b8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c07eca0 con 0x7f9e5c074e60 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.061+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c074e60 0x7f9e5c1b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 0x7f9e5c1b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 0x7f9e5c1b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54490/0 (socket says 172.21.15.64:54490) 2024-05-09T09:18:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/547378130 learned_addr learned my addr 172.21.15.64:0/547378130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5affd640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 0x7f9e5c1b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/547378130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c074e60 msgr2=0x7f9e5c1b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c074e60 0x7f9e5c1b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 msgr2=0x7f9e5c1b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 0x7f9e5c1b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 -- 172.21.15.64:0/547378130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c1b6bc0 con 0x7f9e5c073360 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.062+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 0x7f9e5c1b1740 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400ebc0 tx=0x7f9e5400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.063+0000 7f9e58ff9640 1 -- 172.21.15.64:0/547378130 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54003b70 con 0x7f9e5c073360 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.063+0000 7f9e58ff9640 1 -- 172.21.15.64:0/547378130 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e54003d10 con 0x7f9e5c073360 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.063+0000 7f9e58ff9640 1 -- 172.21.15.64:0/547378130 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540107f0 con 0x7f9e5c073360 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.063+0000 7f9e60b8c640 1 -- 172.21.15.64:0/547378130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c1b6eb0 con 0x7f9e5c073360 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.063+0000 7f9e60b8c640 1 -- 172.21.15.64:0/547378130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c1b72c0 con 0x7f9e5c073360 2024-05-09T09:18:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.067+0000 7f9e64bdb740 1 Processor -- start 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.067+0000 7f9e58ff9640 1 -- 172.21.15.64:0/547378130 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e54004030 con 0x7f9e5c073360 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 -- start start 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 0x563610fd3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd4320 0x563610fd9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fdb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563610f2fc40 con 0x563610fd3930 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563610ed7580 con 0x563610fd4320 2024-05-09T09:18:39.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563610f2bc00 con 0x563610fd9550 2024-05-09T09:18:39.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e58ff9640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2cc0 0x7f9e440a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.068+0000 7f9e58ff9640 1 -- 172.21.15.64:0/547378130 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e540cfab0 con 0x7f9e5c073360 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 0x563610fd3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 0x563610fd3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42594/0 (socket says 172.21.15.64:42594) 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 -- 172.21.15.64:0/3013861284 learned_addr learned my addr 172.21.15.64:0/3013861284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2cc0 0x7f9e440a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5affd640 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fdb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 -- 172.21.15.64:0/3013861284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 msgr2=0x563610fdb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fdb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 -- 172.21.15.64:0/3013861284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd4320 msgr2=0x563610fd9010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd4320 0x563610fd9010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.069+0000 7f9e5bfff640 1 -- 172.21.15.64:0/3013861284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563610c433f0 con 0x563610fd3930 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.070+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2cc0 0x7f9e440a5180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005fd0 tx=0x7f9e4c00e670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.070+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 0x563610fd3d50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00e8a0 tx=0x7f9e4c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f35112dcd223505 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.070+0000 7f9e417fa640 1 -- 172.21.15.64:0/3013861284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c013030 con 0x563610fd3930 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.071+0000 7f9e417fa640 1 -- 172.21.15.64:0/3013861284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c0131d0 con 0x563610fd3930 2024-05-09T09:18:39.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.071+0000 7f9e417fa640 1 -- 172.21.15.64:0/3013861284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0054c0 con 0x563610fd3930 2024-05-09T09:18:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 -- 172.21.15.64:0/3013861284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 msgr2=0x563610fd3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 0x563610fd3d50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00e8a0 tx=0x7f9e4c01a000 comp rx=0 tx=0).stop 2024-05-09T09:18:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 -- 172.21.15.64:0/3013861284 shutdown_connections 2024-05-09T09:18:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fdb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd4320 0x563610fd9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/3013861284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd3930 0x563610fd3d50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 -- 172.21.15.64:0/3013861284 >> 172.21.15.64:0/3013861284 conn(0x563610fcf290 msgr2=0x563610fcf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 -- 172.21.15.64:0/3013861284 shutdown_connections 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.072+0000 7f9e64bdb740 1 -- 172.21.15.64:0/3013861284 wait complete. 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 -- start start 2024-05-09T09:18:39.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 Processor -- start 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 0x563610fde320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563610fdc3e0 con 0x563610fd4320 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563610eeac40 con 0x563610fd3930 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563610fdc560 con 0x563610fd9550 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.073+0000 7f9e64bdb740 10 client.0 _ll_register_callbacks cb 0x563610c42be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 0x563610fde320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 0x563610fde320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42600/0 (socket says 172.21.15.64:42600) 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 -- 172.21.15.64:0/4064114503 learned_addr learned my addr 172.21.15.64:0/4064114503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 -- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 msgr2=0x563610fe5360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 -- 172.21.15.64:0/4064114503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 msgr2=0x563610fddde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:39.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.074+0000 7f9e5affd640 1 -- 172.21.15.64:0/4064114503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563610d0d7d0 con 0x563610fd4320 2024-05-09T09:18:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.075+0000 7f9e5affd640 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 0x563610fde320 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9e50000c00 tx=0x7f9e50005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8818f2b81b95f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.075+0000 7f9e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.075+0000 7f9e3affd640 1 -- 172.21.15.64:0/4064114503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50035030 con 0x563610fd4320 2024-05-09T09:18:39.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.075+0000 7f9e3affd640 1 -- 172.21.15.64:0/4064114503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e500048c0 con 0x563610fd4320 2024-05-09T09:18:39.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.075+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c0131d0 con 0x563610fd4320 2024-05-09T09:18:39.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.076+0000 7f9e3affd640 1 -- 172.21.15.64:0/4064114503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e50005570 con 0x563610fd4320 2024-05-09T09:18:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.078+0000 7f9e64bdb740 10 client.17835 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.078+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563610fc93e0 con 0x563610fd4320 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.082+0000 7f9e3affd640 1 -- 172.21.15.64:0/4064114503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c0131d0 con 0x563610fd4320 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e3affd640 1 -- 172.21.15.64:0/4064114503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9e50003ca0 con 0x563610fd4320 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e64bdb740 10 client.17835 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e64bdb740 10 client.17835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e64bdb740 20 client.17835 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563610fcaa00 0x563610fccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e64bdb740 4 client.17835 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:39.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563610f3d370 con 0x563610fcaa00 2024-05-09T09:18:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.083+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563610fcaa00 0x563610fccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.084+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563610fcaa00 0x563610fccea0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=492b2e00bf06df4b server_cookie=60b549ca392b5dd5 in_seq=0 out_seq=0 2024-05-09T09:18:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.084+0000 7f9e3affd640 10 client.17835 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e3affd640 1 -- 172.21.15.64:0/4064114503 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563610f3d370 con 0x563610fcaa00 2024-05-09T09:18:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e3affd640 10 client.17835 handle_command_reply: tid=0 2024-05-09T09:18:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 client.17835 shutdown 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 msgr2=0x563610fde320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 0x563610fde320 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9e50000c00 tx=0x7f9e50005270 comp rx=0 tx=0).stop 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 shutdown_connections 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563610fcaa00 0x563610fccea0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd4320 0x563610fde320 secure :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9e50000c00 tx=0x7f9e50005270 comp rx=0 tx=0).stop 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/4064114503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 >> 172.21.15.64:0/4064114503 conn(0x563610fcf290 msgr2=0x563610fc5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.085+0000 7f9e3affd640 0 client.17835 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.086+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 shutdown_connections 2024-05-09T09:18:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.086+0000 7f9e64bdb740 1 -- 172.21.15.64:0/4064114503 wait complete. 2024-05-09T09:18:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.087+0000 7f9e64bdb740 20 client.17835 trim_cache size 0 max 16384 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 Processor -- start 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 -- start start 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 0x563610fd2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1c41f0 con 0x563610fd2a30 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1c1ac0 con 0x563610fd3930 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.119+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1bf910 con 0x563610fd9550 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35212/0 (socket says 172.21.15.64:35212) 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 -- 172.21.15.64:0/2488873786 learned_addr learned my addr 172.21.15.64:0/2488873786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 -- 172.21.15.64:0/2488873786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 msgr2=0x563610fddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 -- 172.21.15.64:0/2488873786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 msgr2=0x563610fd2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 0x563610fd2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.120+0000 7f9e5affd640 1 -- 172.21.15.64:0/2488873786 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563610fd8c80 con 0x563610fd9550 2024-05-09T09:18:39.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.123+0000 7f9e5affd640 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009900 tx=0x7f9e5002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821c0785f3f5ce17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.125+0000 7f9e40ff9640 1 -- 172.21.15.64:0/2488873786 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50031a40 con 0x563610fd9550 2024-05-09T09:18:39.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.125+0000 7f9e40ff9640 1 -- 172.21.15.64:0/2488873786 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50009520 con 0x563610fd9550 2024-05-09T09:18:39.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.125+0000 7f9e40ff9640 1 -- 172.21.15.64:0/2488873786 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500039e0 con 0x563610fd9550 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.126+0000 7f9e64bdb740 1 -- 172.21.15.64:0/2488873786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 msgr2=0x563610fe5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.126+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009900 tx=0x7f9e5002a410 comp rx=0 tx=0).stop 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.127+0000 7f9e64bdb740 1 -- 172.21.15.64:0/2488873786 shutdown_connections 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.127+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 0x563610fd2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.127+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe5320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.127+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/2488873786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.127+0000 7f9e64bdb740 1 -- 172.21.15.64:0/2488873786 >> 172.21.15.64:0/2488873786 conn(0x563610fcf290 msgr2=0x563610fd2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.132+0000 7f9e64bdb740 1 -- 172.21.15.64:0/2488873786 shutdown_connections 2024-05-09T09:18:39.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.132+0000 7f9e64bdb740 1 -- 172.21.15.64:0/2488873786 wait complete. 2024-05-09T09:18:39.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 -- start start 2024-05-09T09:18:39.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 Processor -- start 2024-05-09T09:18:39.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 0x563610fe5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fe1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1bff80 con 0x563610fd2a30 2024-05-09T09:18:39.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1c13f0 con 0x563610fd3930 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.133+0000 7f9e64bdb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1c0650 con 0x563610fd9550 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e64bdb740 10 client.0 _ll_register_callbacks cb 0x7f9e540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35220/0 (socket says 172.21.15.64:35220) 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 -- 172.21.15.64:0/1469018247 learned_addr learned my addr 172.21.15.64:0/1469018247 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 -- 172.21.15.64:0/1469018247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 msgr2=0x563610fe1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fe1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 -- 172.21.15.64:0/1469018247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 msgr2=0x563610fe5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 0x563610fe5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.134+0000 7f9e5bfff640 1 -- 172.21.15.64:0/1469018247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636110581f0 con 0x563610fd9550 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.135+0000 7f9e5bfff640 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe2460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c013590 tx=0x7f9e4c0135c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278795a64ec5dfd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.135+0000 7f9e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.135+0000 7f9e3affd640 1 -- 172.21.15.64:0/1469018247 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c02e070 con 0x563610fd9550 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.135+0000 7f9e3affd640 1 -- 172.21.15.64:0/1469018247 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e4c009680 con 0x563610fd9550 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.135+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e50009520 con 0x563610fd9550 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.136+0000 7f9e3affd640 1 -- 172.21.15.64:0/1469018247 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c03a050 con 0x563610fd9550 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.136+0000 7f9e64bdb740 10 client.27484 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:39.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.136+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563611059520 con 0x563610fd9550 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e3affd640 1 -- 172.21.15.64:0/1469018247 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50009520 con 0x563610fd9550 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e3affd640 1 -- 172.21.15.64:0/1469018247 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9e4c00f040 con 0x563610fd9550 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e64bdb740 10 client.27484 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e64bdb740 10 client.27484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e64bdb740 20 client.27484 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563611059bf0 0x56361105bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e64bdb740 4 client.27484 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:18:39.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.138+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563610f376c0 con 0x563611059bf0 2024-05-09T09:18:39.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.139+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563611059bf0 0x56361105bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.148+0000 7f9e5b7fe640 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563611059bf0 0x56361105bfe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53968f27b4c835e7 server_cookie=81833175123d7b36 in_seq=0 out_seq=0 2024-05-09T09:18:39.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.148+0000 7f9e3affd640 10 client.27484 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:39.150 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.148+0000 7f9e3affd640 1 -- 172.21.15.64:0/1469018247 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563610f376c0 con 0x563611059bf0 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.148+0000 7f9e3affd640 10 client.27484 handle_command_reply: tid=0 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.148+0000 7f9e64bdb740 1 client.27484 shutdown 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 msgr2=0x563610fe2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe2460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c013590 tx=0x7f9e4c0135c0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 shutdown_connections 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563610fd9550 0x563610fe2460 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563610fd3930 0x563610fe1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563611059bf0 0x56361105bfe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 --2- 172.21.15.64:0/1469018247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563610fd2a30 0x563610fe5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 >> 172.21.15.64:0/1469018247 conn(0x563610fcf290 msgr2=0x563610fc5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e3affd640 0 client.27484 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 shutdown_connections 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.150+0000 7f9e64bdb740 1 -- 172.21.15.64:0/1469018247 wait complete. 2024-05-09T09:18:39.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.151+0000 7f9e64bdb740 20 client.27484 trim_cache size 0 max 16384 2024-05-09T09:18:39.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.160+0000 7f9e427fc640 1 -- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2cc0 msgr2=0x7f9e440a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.160+0000 7f9e427fc640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2cc0 0x7f9e440a5180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005fd0 tx=0x7f9e4c00e670 comp rx=0 tx=0).stop 2024-05-09T09:18:39.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.160+0000 7f9e427fc640 1 -- 172.21.15.64:0/547378130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 msgr2=0x7f9e5c1b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.160+0000 7f9e427fc640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 0x7f9e5c1b1740 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400ebc0 tx=0x7f9e5400c6a0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.164+0000 7f9e427fc640 1 -- 172.21.15.64:0/547378130 shutdown_connections 2024-05-09T09:18:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.164+0000 7f9e427fc640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9e440a2cc0 0x7f9e440a5180 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.164+0000 7f9e427fc640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e5c074e60 0x7f9e5c1b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.164+0000 7f9e427fc640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e5c0744a0 0x7f9e5c1b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.164+0000 7f9e427fc640 1 --2- 172.21.15.64:0/547378130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e5c073360 0x7f9e5c1b1740 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.164+0000 7f9e427fc640 1 -- 172.21.15.64:0/547378130 >> 172.21.15.64:0/547378130 conn(0x7f9e5c06e740 msgr2=0x7f9e5c070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.166+0000 7f9e427fc640 1 -- 172.21.15.64:0/547378130 shutdown_connections 2024-05-09T09:18:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.166+0000 7f9e427fc640 1 -- 172.21.15.64:0/547378130 wait complete. 2024-05-09T09:18:39.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:18:39.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.428+0000 7f73ce7fc640 1 -- 172.21.15.64:0/4094494368 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4030880 con 0x7f73d0108fe0 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.428+0000 7f73d5be7640 1 -- 172.21.15.64:0/4094494368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108fe0 msgr2=0x7f73d0109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.428+0000 7f73d5be7640 1 --2- 172.21.15.64:0/4094494368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108fe0 0x7f73d0109460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009860 tx=0x7f73c402a360 comp rx=0 tx=0).stop 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 -- 172.21.15.64:0/4094494368 shutdown_connections 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 --2- 172.21.15.64:0/4094494368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d01099a0 0x7f73d01102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 --2- 172.21.15.64:0/4094494368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108fe0 0x7f73d0109460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 --2- 172.21.15.64:0/4094494368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d0107e90 0x7f73d01082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 -- 172.21.15.64:0/4094494368 >> 172.21.15.64:0/4094494368 conn(0x7f73d0076e30 msgr2=0x7f73d0079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 -- 172.21.15.64:0/4094494368 shutdown_connections 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.429+0000 7f73d5be7640 1 -- 172.21.15.64:0/4094494368 wait complete. 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 Processor -- start 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 -- start start 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0107e90 0x7f73d007f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 0x7f73d007f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01099a0 0x7f73d019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0113ab0 con 0x7f73d0108fe0 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0113930 con 0x7f73d0107e90 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.430+0000 7f73d5be7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0113c30 con 0x7f73d01099a0 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 0x7f73d007f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73cffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01099a0 0x7f73d019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 0x7f73d007f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42626/0 (socket says 172.21.15.64:42626) 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73cffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01099a0 0x7f73d019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35228/0 (socket says 172.21.15.64:35228) 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73ceffd640 1 -- 172.21.15.64:0/147540270 learned_addr learned my addr 172.21.15.64:0/147540270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73ceffd640 1 -- 172.21.15.64:0/147540270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01099a0 msgr2=0x7f73d019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.431+0000 7f73ceffd640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01099a0 0x7f73d019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.432+0000 7f73ceffd640 1 -- 172.21.15.64:0/147540270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0107e90 msgr2=0x7f73d007f020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:18:39.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.432+0000 7f73ceffd640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0107e90 0x7f73d007f020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.432+0000 7f73ceffd640 1 -- 172.21.15.64:0/147540270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d019d6f0 con 0x7f73d0108fe0 2024-05-09T09:18:39.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.432+0000 7f73ceffd640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 0x7f73d007f560 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009780 tx=0x7f73c4032790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.435+0000 7f73ccff9640 1 -- 172.21.15.64:0/147540270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4042070 con 0x7f73d0108fe0 2024-05-09T09:18:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.435+0000 7f73ccff9640 1 -- 172.21.15.64:0/147540270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c4004050 con 0x7f73d0108fe0 2024-05-09T09:18:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.435+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d019d980 con 0x7f73d0108fe0 2024-05-09T09:18:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.435+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d019de70 con 0x7f73d0108fe0 2024-05-09T09:18:39.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.435+0000 7f73ccff9640 1 -- 172.21.15.64:0/147540270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4037040 con 0x7f73d0108fe0 2024-05-09T09:18:39.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 Processor -- start 2024-05-09T09:18:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 -- start start 2024-05-09T09:18:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56472f91ec40 con 0x56472f9bfaa0 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56472f8c6580 con 0x56472f9b9f40 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.440+0000 7f73d89d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56472f91ac00 con 0x56472f9ba930 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.441+0000 7f73cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.442+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.442+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9c1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42642/0 (socket says 172.21.15.64:42642) 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.442+0000 7f73ceffd640 1 -- 172.21.15.64:0/2642329894 learned_addr learned my addr 172.21.15.64:0/2642329894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.442+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.442+0000 7f73ccff9640 1 -- 172.21.15.64:0/147540270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73c4004200 con 0x7f73d0108fe0 2024-05-09T09:18:39.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.443+0000 7f73ceffd640 1 -- 172.21.15.64:0/2642329894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 msgr2=0x56472f9bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.443+0000 7f73ceffd640 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.443+0000 7f73ceffd640 1 -- 172.21.15.64:0/2642329894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 msgr2=0x56472f9ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.443+0000 7f73ceffd640 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.443+0000 7f73ceffd640 1 -- 172.21.15.64:0/2642329894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472f6323f0 con 0x56472f9bfaa0 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.443+0000 7f73cffff640 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.444+0000 7f73ceffd640 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9c1f30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f73c40f2c70 tx=0x7f73c40f2d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2e9f696b10d66c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.444+0000 7f73ccff9640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73b80a2d90 0x7f73b80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.445+0000 7f73cffff640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73b80a2d90 0x7f73b80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.445+0000 7f73ccff9640 1 -- 172.21.15.64:0/147540270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f73c40eec90 con 0x7f73d0108fe0 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.446+0000 7f73cffff640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73b80a2d90 0x7f73b80a5250 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009fd0 tx=0x7f73c80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.446+0000 7f73b5ffb640 1 -- 172.21.15.64:0/2642329894 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c40b7390 con 0x56472f9bfaa0 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.446+0000 7f73b5ffb640 1 -- 172.21.15.64:0/2642329894 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c40bb880 con 0x56472f9bfaa0 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.446+0000 7f73b5ffb640 1 -- 172.21.15.64:0/2642329894 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c40b92c0 con 0x56472f9bfaa0 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 -- 172.21.15.64:0/2642329894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 msgr2=0x56472f9c1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9c1f30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f73c40f2c70 tx=0x7f73c40f2d60 comp rx=0 tx=0).stop 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 -- 172.21.15.64:0/2642329894 shutdown_connections 2024-05-09T09:18:39.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9c1f30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2642329894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 -- 172.21.15.64:0/2642329894 >> 172.21.15.64:0/2642329894 conn(0x56472f9b55b0 msgr2=0x56472f9b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 -- 172.21.15.64:0/2642329894 shutdown_connections 2024-05-09T09:18:39.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.447+0000 7f73d89d4740 1 -- 172.21.15.64:0/2642329894 wait complete. 2024-05-09T09:18:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 -- start start 2024-05-09T09:18:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 Processor -- start 2024-05-09T09:18:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9c4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56472f9c2810 con 0x56472f9bfaa0 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56472f8d9c40 con 0x56472f9b9f40 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56472f9c2990 con 0x56472f9ba930 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.448+0000 7f73d89d4740 10 client.0 _ll_register_callbacks cb 0x56472f631be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.449+0000 7f73cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.449+0000 7f73cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42648/0 (socket says 172.21.15.64:42648) 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.449+0000 7f73cffff640 1 -- 172.21.15.64:0/1314691089 learned_addr learned my addr 172.21.15.64:0/1314691089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.449+0000 7f73ceffd640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cffff640 1 -- 172.21.15.64:0/1314691089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 msgr2=0x56472f9c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cffff640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cffff640 1 -- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 msgr2=0x56472f9c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cffff640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cffff640 1 -- 172.21.15.64:0/1314691089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472f6fc7d0 con 0x56472f9bfaa0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.450+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.451+0000 7f73ceffd640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.451+0000 7f73cffff640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9cb690 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f73c800ded0 tx=0x7f73c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0957530062f38ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.451+0000 7f73aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.452+0000 7f73aeffd640 1 -- 172.21.15.64:0/1314691089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c801d110 con 0x56472f9bfaa0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.452+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c40bb880 con 0x56472f9bfaa0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.452+0000 7f73aeffd640 1 -- 172.21.15.64:0/1314691089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f73c8003970 con 0x56472f9bfaa0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.452+0000 7f73aeffd640 1 -- 172.21.15.64:0/1314691089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f73c8003b70 con 0x56472f9bfaa0 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.454+0000 7f73d89d4740 10 client.17847 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:39.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.454+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56472f9ce9e0 con 0x56472f9bfaa0 2024-05-09T09:18:39.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.455+0000 7f73aeffd640 1 -- 172.21.15.64:0/1314691089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c40bb880 con 0x56472f9bfaa0 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.455+0000 7f73aeffd640 1 -- 172.21.15.64:0/1314691089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f73c800b040 con 0x56472f9bfaa0 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73d89d4740 10 client.17847 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73d89d4740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73d89d4740 20 client.17847 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73d89d4740 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472f9d05f0 0x56472f9d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73d89d4740 4 client.17847 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:39.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472f92c370 con 0x56472f9d05f0 2024-05-09T09:18:39.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.456+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472f9d05f0 0x56472f9d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.457+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472f9d05f0 0x56472f9d2a70 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5c25d72402a3c0 server_cookie=80c158462811a5d9 in_seq=0 out_seq=0 2024-05-09T09:18:39.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.457+0000 7f73aeffd640 10 client.17847 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:39.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.459+0000 7f73aeffd640 1 -- 172.21.15.64:0/1314691089 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56472f92c370 con 0x56472f9d05f0 2024-05-09T09:18:39.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.460+0000 7f73aeffd640 10 client.17847 handle_command_reply: tid=0 2024-05-09T09:18:39.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.460+0000 7f73d89d4740 1 client.17847 shutdown 2024-05-09T09:18:39.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 msgr2=0x56472f9cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9cb690 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f73c800ded0 tx=0x7f73c8004ab0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 shutdown_connections 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472f9d05f0 0x56472f9d2a70 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9cb690 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 --2- 172.21.15.64:0/1314691089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 >> 172.21.15.64:0/1314691089 conn(0x56472f9b55b0 msgr2=0x56472f9b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.465+0000 7f73aeffd640 0 client.17847 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.466+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 shutdown_connections 2024-05-09T09:18:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.466+0000 7f73d89d4740 1 -- 172.21.15.64:0/1314691089 wait complete. 2024-05-09T09:18:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.466+0000 7f73d89d4740 20 client.17847 trim_cache size 0 max 16384 2024-05-09T09:18:39.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.519+0000 7f73d89d4740 1 Processor -- start 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.521+0000 7f73d89d4740 1 -- start start 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.521+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9d2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.521+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9d2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54570/0 (socket says 172.21.15.64:54570) 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73cf7fe640 1 -- 172.21.15.64:0/145846176 learned_addr learned my addr 172.21.15.64:0/145846176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73ceffd640 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73d89d4740 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc012550 con 0x56472f9bfaa0 2024-05-09T09:18:39.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc00fe20 con 0x56472f9b9f40 2024-05-09T09:18:39.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73cffff640 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.522+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc00dc70 con 0x56472f9ba930 2024-05-09T09:18:39.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.523+0000 7f73cffff640 1 -- 172.21.15.64:0/145846176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 msgr2=0x56472f9cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.523+0000 7f73cffff640 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.523+0000 7f73cffff640 1 -- 172.21.15.64:0/145846176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 msgr2=0x56472f9d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.523+0000 7f73cffff640 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9d2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.523+0000 7f73cffff640 1 -- 172.21.15.64:0/145846176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472f9a6d20 con 0x56472f9bfaa0 2024-05-09T09:18:39.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.523+0000 7f73cffff640 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9b4630 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f73c8027660 tx=0x7f73c800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c881b4a1d4f03957 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.529+0000 7f73b57fa640 1 -- 172.21.15.64:0/145846176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c801d110 con 0x56472f9bfaa0 2024-05-09T09:18:39.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.529+0000 7f73b57fa640 1 -- 172.21.15.64:0/145846176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c80276e0 con 0x56472f9bfaa0 2024-05-09T09:18:39.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.530+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 msgr2=0x56472f9b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.530+0000 7f73d89d4740 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9b4630 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f73c8027660 tx=0x7f73c800db60 comp rx=0 tx=0).stop 2024-05-09T09:18:39.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.537+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 shutdown_connections 2024-05-09T09:18:39.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.537+0000 7f73d89d4740 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9bfaa0 0x56472f9b4630 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.537+0000 7f73d89d4740 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9ba930 0x56472f9cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.537+0000 7f73d89d4740 1 --2- 172.21.15.64:0/145846176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9b9f40 0x56472f9d2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.537+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 >> 172.21.15.64:0/145846176 conn(0x56472f9b55b0 msgr2=0x56472f9b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.540+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 shutdown_connections 2024-05-09T09:18:39.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.540+0000 7f73d89d4740 1 -- 172.21.15.64:0/145846176 wait complete. 2024-05-09T09:18:39.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.540+0000 7f73d89d4740 1 -- start start 2024-05-09T09:18:39.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 Processor -- start 2024-05-09T09:18:39.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9b9f40 0x56472f9c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 0x56472f9c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9bfaa0 0x56472fa34ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc00e2e0 con 0x56472f9b9f40 2024-05-09T09:18:39.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc00f750 con 0x56472f9ba930 2024-05-09T09:18:39.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc00e9b0 con 0x56472f9bfaa0 2024-05-09T09:18:39.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73d89d4740 10 client.0 _ll_register_callbacks cb 0x7f73c8009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 0x56472f9c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.541+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 0x56472f9c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54582/0 (socket says 172.21.15.64:54582) 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73ceffd640 1 -- 172.21.15.64:0/2735382194 learned_addr learned my addr 172.21.15.64:0/2735382194 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9b9f40 0x56472f9c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73ceffd640 1 -- 172.21.15.64:0/2735382194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9bfaa0 msgr2=0x56472fa34ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73ceffd640 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9bfaa0 0x56472fa34ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73ceffd640 1 -- 172.21.15.64:0/2735382194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9b9f40 msgr2=0x56472f9c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73ceffd640 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9b9f40 0x56472f9c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.542+0000 7f73ceffd640 1 -- 172.21.15.64:0/2735382194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472fa350e0 con 0x56472f9ba930 2024-05-09T09:18:39.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.544+0000 7f73ceffd640 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 0x56472f9c88e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73c40bae80 tx=0x7f73c40b9190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cf3304c30155e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:39.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.544+0000 7f73aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:39.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.545+0000 7f73aeffd640 1 -- 172.21.15.64:0/2735382194 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c40baae0 con 0x56472f9ba930 2024-05-09T09:18:39.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.545+0000 7f73aeffd640 1 -- 172.21.15.64:0/2735382194 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f73c40bf130 con 0x56472f9ba930 2024-05-09T09:18:39.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.548+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c80276e0 con 0x56472f9ba930 2024-05-09T09:18:39.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.548+0000 7f73aeffd640 1 -- 172.21.15.64:0/2735382194 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f73c40bacc0 con 0x56472f9ba930 2024-05-09T09:18:39.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 10 client.27359 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:39.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56472fa36220 con 0x56472f9ba930 2024-05-09T09:18:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73aeffd640 1 -- 172.21.15.64:0/2735382194 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c80276e0 con 0x56472f9ba930 2024-05-09T09:18:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73aeffd640 1 -- 172.21.15.64:0/2735382194 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f73c40c3960 con 0x56472f9ba930 2024-05-09T09:18:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 10 client.27359 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:39.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:39.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 20 client.27359 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:39.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472fa36940 0x56472fa38d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:39.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 4 client.27359 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:18:39.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.549+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56472f9266c0 con 0x56472fa36940 2024-05-09T09:18:39.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.551+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472fa36940 0x56472fa38d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:39.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.554+0000 7f73cf7fe640 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472fa36940 0x56472fa38d60 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5b8ff0b722098f server_cookie=480a9fa442c461e6 in_seq=0 out_seq=0 2024-05-09T09:18:39.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.554+0000 7f73aeffd640 10 client.27359 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:39.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.554+0000 7f73aeffd640 1 -- 172.21.15.64:0/2735382194 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56472f9266c0 con 0x56472fa36940 2024-05-09T09:18:39.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.554+0000 7f73aeffd640 10 client.27359 handle_command_reply: tid=0 2024-05-09T09:18:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.554+0000 7f73d89d4740 1 client.27359 shutdown 2024-05-09T09:18:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 msgr2=0x56472f9c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 0x56472f9c88e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73c40bae80 tx=0x7f73c40b9190 comp rx=0 tx=0).stop 2024-05-09T09:18:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 shutdown_connections 2024-05-09T09:18:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56472fa36940 0x56472fa38d60 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56472f9bfaa0 0x56472fa34ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56472f9ba930 0x56472f9c88e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 --2- 172.21.15.64:0/2735382194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56472f9b9f40 0x56472f9c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 >> 172.21.15.64:0/2735382194 conn(0x56472f9b55b0 msgr2=0x56472f9b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73aeffd640 0 client.27359 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.557+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 shutdown_connections 2024-05-09T09:18:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.558+0000 7f73d89d4740 1 -- 172.21.15.64:0/2735382194 wait complete. 2024-05-09T09:18:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.558+0000 7f73d89d4740 20 client.27359 trim_cache size 0 max 16384 2024-05-09T09:18:39.560 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:18:39.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.563+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73b80a2d90 msgr2=0x7f73b80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.563+0000 7f73d5be7640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73b80a2d90 0x7f73b80a5250 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009fd0 tx=0x7f73c80095c0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.563+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 msgr2=0x7f73d007f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:39.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.563+0000 7f73d5be7640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 0x7f73d007f560 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009780 tx=0x7f73c4032790 comp rx=0 tx=0).stop 2024-05-09T09:18:39.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 shutdown_connections 2024-05-09T09:18:39.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73b80a2d90 0x7f73b80a5250 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01099a0 0x7f73d019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73d0108fe0 0x7f73d007f560 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 --2- 172.21.15.64:0/147540270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0107e90 0x7f73d007f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:39.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 >> 172.21.15.64:0/147540270 conn(0x7f73d0076e30 msgr2=0x7f73d010c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:39.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.568+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 shutdown_connections 2024-05-09T09:18:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:39.570+0000 7f73d5be7640 1 -- 172.21.15.64:0/147540270 wait complete. 2024-05-09T09:18:40.609 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:18:40.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:18:40.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:18:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.815+0000 7ff376bc8640 1 -- 172.21.15.64:0/1724157571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 msgr2=0x7ff370100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.815+0000 7ff376bc8640 1 --2- 172.21.15.64:0/1724157571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370100b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff364009900 tx=0x7ff36402a440 comp rx=0 tx=0).stop 2024-05-09T09:18:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.818+0000 7ff376bc8640 1 -- 172.21.15.64:0/1724157571 shutdown_connections 2024-05-09T09:18:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.818+0000 7ff376bc8640 1 --2- 172.21.15.64:0/1724157571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff370102320 0x7ff370108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.818+0000 7ff376bc8640 1 --2- 172.21.15.64:0/1724157571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff370101960 0x7ff370101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.818+0000 7ff376bc8640 1 --2- 172.21.15.64:0/1724157571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370100b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.818+0000 7ff376bc8640 1 -- 172.21.15.64:0/1724157571 >> 172.21.15.64:0/1724157571 conn(0x7ff3700fbce0 msgr2=0x7ff3700fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.819+0000 7ff376bc8640 1 -- 172.21.15.64:0/1724157571 shutdown_connections 2024-05-09T09:18:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.820+0000 7ff376bc8640 1 -- 172.21.15.64:0/1724157571 wait complete. 2024-05-09T09:18:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.821+0000 7ff376bc8640 1 Processor -- start 2024-05-09T09:18:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.821+0000 7ff376bc8640 1 -- start start 2024-05-09T09:18:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff376bc8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff376bc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff370101960 0x7ff370197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff374964640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff374964640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35266/0 (socket says 172.21.15.64:35266) 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff374964640 1 -- 172.21.15.64:0/3615599916 learned_addr learned my addr 172.21.15.64:0/3615599916 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff370102320 0x7ff37019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010c2c0 con 0x7ff370101960 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.822+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010c140 con 0x7ff370102320 2024-05-09T09:18:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.823+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010c440 con 0x7ff370100770 2024-05-09T09:18:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:40 smithi139 ceph-mon[33599]: pgmap v656: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-05-09T09:18:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:40.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:18:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.823+0000 7ff374964640 1 -- 172.21.15.64:0/3615599916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff370102320 msgr2=0x7ff37019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.823+0000 7ff374964640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff370102320 0x7ff37019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.823+0000 7ff374964640 1 -- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff370101960 msgr2=0x7ff370197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.823+0000 7ff374964640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff370101960 0x7ff370197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.823+0000 7ff374964640 1 -- 172.21.15.64:0/3615599916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37019f6b0 con 0x7ff370100770 2024-05-09T09:18:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.824+0000 7ff374964640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370197a90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff364009230 tx=0x7ff3640036c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.825+0000 7ff36dffb640 1 -- 172.21.15.64:0/3615599916 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364004110 con 0x7ff370100770 2024-05-09T09:18:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.825+0000 7ff36dffb640 1 -- 172.21.15.64:0/3615599916 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff36403ebf0 con 0x7ff370100770 2024-05-09T09:18:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.825+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff370102b80 con 0x7ff370100770 2024-05-09T09:18:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.826+0000 7ff36dffb640 1 -- 172.21.15.64:0/3615599916 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3640054e0 con 0x7ff370100770 2024-05-09T09:18:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.826+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff370103010 con 0x7ff370100770 2024-05-09T09:18:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.829+0000 7ff36dffb640 1 -- 172.21.15.64:0/3615599916 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff36403a040 con 0x7ff370100770 2024-05-09T09:18:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.830+0000 7ff3799b2740 1 Processor -- start 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.830+0000 7ff3799b2740 1 -- start start 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc0cd0 0x55f828bc10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc16c0 0x55f828bc61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bc8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff3799b2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f828b1c990 con 0x55f828bc16c0 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff3799b2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f828ac45b0 con 0x55f828bc0cd0 2024-05-09T09:18:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff3799b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f828b18950 con 0x55f828bc6830 2024-05-09T09:18:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff375165640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc0cd0 0x55f828bc10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bc8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff375165640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc0cd0 0x55f828bc10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54610/0 (socket says 172.21.15.64:54610) 2024-05-09T09:18:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.833+0000 7ff375165640 1 -- 172.21.15.64:0/2027579506 learned_addr learned my addr 172.21.15.64:0/2027579506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff374964640 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc16c0 0x55f828bc61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff36ffff640 1 -- 172.21.15.64:0/2027579506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc0cd0 msgr2=0x55f828bc10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc0cd0 0x55f828bc10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff36ffff640 1 -- 172.21.15.64:0/2027579506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc16c0 msgr2=0x55f828bc61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc16c0 0x55f828bc61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff36ffff640 1 -- 172.21.15.64:0/2027579506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8288f87e0 con 0x55f828bc6830 2024-05-09T09:18:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.834+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bc8cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00a9b0 tx=0x7ff35c00ae70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92bc11c5a77795a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.835+0000 7ff36dffb640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff3600a2d90 0x7ff3600a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.836+0000 7ff35affd640 1 -- 172.21.15.64:0/2027579506 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00cd80 con 0x55f828bc6830 2024-05-09T09:18:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.836+0000 7ff35affd640 1 -- 172.21.15.64:0/2027579506 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff35c00f900 con 0x55f828bc6830 2024-05-09T09:18:40.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 -- 172.21.15.64:0/2027579506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 msgr2=0x55f828bc8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bc8cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00a9b0 tx=0x7ff35c00ae70 comp rx=0 tx=0).stop 2024-05-09T09:18:40.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 -- 172.21.15.64:0/2027579506 shutdown_connections 2024-05-09T09:18:40.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bc8cc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc16c0 0x55f828bc61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2027579506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc0cd0 0x55f828bc10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff3799b2740 1 -- 172.21.15.64:0/2027579506 >> 172.21.15.64:0/2027579506 conn(0x55f828b800a0 msgr2=0x55f828bb3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.837+0000 7ff36dffb640 1 -- 172.21.15.64:0/3615599916 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff3640f3f00 con 0x7ff370100770 2024-05-09T09:18:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.838+0000 7ff375165640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff3600a2d90 0x7ff3600a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.839+0000 7ff375165640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff3600a2d90 0x7ff3600a5250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55f828bc1f20 tx=0x7ff3680099b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.840+0000 7ff3799b2740 1 -- 172.21.15.64:0/2027579506 shutdown_connections 2024-05-09T09:18:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.840+0000 7ff3799b2740 1 -- 172.21.15.64:0/2027579506 wait complete. 2024-05-09T09:18:40.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.841+0000 7ff3799b2740 1 -- start start 2024-05-09T09:18:40.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.841+0000 7ff3799b2740 1 Processor -- start 2024-05-09T09:18:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff374964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff374964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42696/0 (socket says 172.21.15.64:42696) 2024-05-09T09:18:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff374964640 1 -- 172.21.15.64:0/1375163360 learned_addr learned my addr 172.21.15.64:0/1375163360 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bbf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.842+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f828bb8630 con 0x55f828bc0cd0 2024-05-09T09:18:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.843+0000 7ff375165640 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bbf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.843+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f828ad7b40 con 0x55f828bc16c0 2024-05-09T09:18:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.843+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f828bb87b0 con 0x55f828bc6830 2024-05-09T09:18:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.843+0000 7ff3799b2740 10 client.0 _ll_register_callbacks cb 0x55f828acc0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.846+0000 7ff374964640 1 -- 172.21.15.64:0/1375163360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 msgr2=0x55f828bbf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.846+0000 7ff374964640 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bbf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.846+0000 7ff374964640 1 -- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 msgr2=0x55f828bba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:40.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.846+0000 7ff374964640 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.846+0000 7ff374964640 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f828958f70 con 0x55f828bc0cd0 2024-05-09T09:18:40.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.846+0000 7ff374964640 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9f90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff3640c8ab0 tx=0x7ff3640bfeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a569fe5f0e433b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:40.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff34ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff34ffff640 1 -- 172.21.15.64:0/1375163360 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364005680 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35c00f900 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff34ffff640 1 -- 172.21.15.64:0/1375163360 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff3640bc9e0 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff34ffff640 1 -- 172.21.15.64:0/1375163360 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff3640c0a70 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff3799b2740 10 client.17859 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.847+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f8287f0170 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff34ffff640 1 -- 172.21.15.64:0/1375163360 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff35c00f900 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff34ffff640 1 -- 172.21.15.64:0/1375163360 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff3640c05d0 con 0x55f828bc0cd0 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff3799b2740 10 client.17859 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff3799b2740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff3799b2740 20 client.17859 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828bdf630 0x55f828be1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff3799b2740 4 client.17859 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:40.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f828b2a0c0 con 0x55f828bdf630 2024-05-09T09:18:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.848+0000 7ff36ffff640 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828bdf630 0x55f828be1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.849+0000 7ff36ffff640 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828bdf630 0x55f828be1aa0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecbbbefc8146dd server_cookie=4a5e33e933cc8f3b in_seq=0 out_seq=0 2024-05-09T09:18:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.849+0000 7ff34ffff640 10 client.17859 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.850+0000 7ff34ffff640 1 -- 172.21.15.64:0/1375163360 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f828b2a0c0 con 0x55f828bdf630 2024-05-09T09:18:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.850+0000 7ff34ffff640 10 client.17859 handle_command_reply: tid=0 2024-05-09T09:18:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.850+0000 7ff3799b2740 1 client.17859 shutdown 2024-05-09T09:18:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.850+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 msgr2=0x55f828bb9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.850+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9f90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff3640c8ab0 tx=0x7ff3640bfeb0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.853+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 shutdown_connections 2024-05-09T09:18:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.853+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828bdf630 0x55f828be1aa0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.853+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bbf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.853+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.853+0000 7ff3799b2740 1 --2- 172.21.15.64:0/1375163360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9f90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.853+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 >> 172.21.15.64:0/1375163360 conn(0x55f828b800a0 msgr2=0x55f828bb3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.854+0000 7ff34ffff640 0 client.17859 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.854+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 shutdown_connections 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.854+0000 7ff3799b2740 1 -- 172.21.15.64:0/1375163360 wait complete. 2024-05-09T09:18:40.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.854+0000 7ff3799b2740 20 client.17859 trim_cache size 0 max 16384 2024-05-09T09:18:40.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.894+0000 7ff3799b2740 1 Processor -- start 2024-05-09T09:18:40.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.895+0000 7ff3799b2740 1 -- start start 2024-05-09T09:18:40.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.895+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.895+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.895+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bb6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.896+0000 7ff3799b2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701aaac0 con 0x55f828bc0cd0 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.896+0000 7ff3799b2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a8390 con 0x55f828bc16c0 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.896+0000 7ff3799b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a61e0 con 0x55f828bc6830 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.896+0000 7ff374964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.896+0000 7ff374964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42702/0 (socket says 172.21.15.64:42702) 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.896+0000 7ff374964640 1 -- 172.21.15.64:0/413162592 learned_addr learned my addr 172.21.15.64:0/413162592 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.897+0000 7ff374964640 1 -- 172.21.15.64:0/413162592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 msgr2=0x55f828bb6e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.897+0000 7ff374964640 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bb6e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.897+0000 7ff374964640 1 -- 172.21.15.64:0/413162592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 msgr2=0x55f828bba0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:40.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.897+0000 7ff374964640 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.897+0000 7ff374964640 1 -- 172.21.15.64:0/413162592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f828bc1aa0 con 0x55f828bc0cd0 2024-05-09T09:18:40.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.897+0000 7ff374964640 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9ba0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff364004a70 tx=0x7ff3640bffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cb79c3022355b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:40.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.898+0000 7ff35a7fc640 1 -- 172.21.15.64:0/413162592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3640bc610 con 0x55f828bc0cd0 2024-05-09T09:18:40.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.898+0000 7ff35a7fc640 1 -- 172.21.15.64:0/413162592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff364005680 con 0x55f828bc0cd0 2024-05-09T09:18:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.899+0000 7ff35a7fc640 1 -- 172.21.15.64:0/413162592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3640c05d0 con 0x55f828bc0cd0 2024-05-09T09:18:40.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.900+0000 7ff3799b2740 1 -- 172.21.15.64:0/413162592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 msgr2=0x55f828bb9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9ba0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff364004a70 tx=0x7ff3640bffe0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 -- 172.21.15.64:0/413162592 shutdown_connections 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bb6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 --2- 172.21.15.64:0/413162592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bb9ba0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 -- 172.21.15.64:0/413162592 >> 172.21.15.64:0/413162592 conn(0x55f828b800a0 msgr2=0x55f828bc2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 -- 172.21.15.64:0/413162592 shutdown_connections 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.901+0000 7ff3799b2740 1 -- 172.21.15.64:0/413162592 wait complete. 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 -- start start 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 Processor -- start 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bf0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bf0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bf1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a6850 con 0x55f828bc0cd0 2024-05-09T09:18:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a7cc0 con 0x55f828bc16c0 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a6f20 con 0x55f828bc6830 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.902+0000 7ff3799b2740 10 client.0 _ll_register_callbacks cb 0x7ff3640042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff375165640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bf1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff375165640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bf1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35310/0 (socket says 172.21.15.64:35310) 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff375165640 1 -- 172.21.15.64:0/2009796483 learned_addr learned my addr 172.21.15.64:0/2009796483 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff374964640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bf0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bf0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:40 smithi064 ceph-mon[30149]: pgmap v656: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-05-09T09:18:40.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:40.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:40.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff36ffff640 1 -- 172.21.15.64:0/2009796483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 msgr2=0x55f828bf1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bf1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff36ffff640 1 -- 172.21.15.64:0/2009796483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 msgr2=0x55f828bf0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bf0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.903+0000 7ff36ffff640 1 -- 172.21.15.64:0/2009796483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f828bf24b0 con 0x55f828bc16c0 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff36ffff640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bf0eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff35c018460 tx=0x7ff35c00a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f4b1c4f7b3f0b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff375165640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bf1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:18:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff34ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff34ffff640 1 -- 172.21.15.64:0/2009796483 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00cd80 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff34ffff640 1 -- 172.21.15.64:0/2009796483 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff35c005020 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff364005680 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff34ffff640 1 -- 172.21.15.64:0/2009796483 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff35c018b00 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff3799b2740 10 client.27371 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.904+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f828c45af0 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.905+0000 7ff34ffff640 1 -- 172.21.15.64:0/2009796483 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff364005680 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.905+0000 7ff34ffff640 1 -- 172.21.15.64:0/2009796483 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff35c027170 con 0x55f828bc16c0 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff3799b2740 10 client.27371 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:40.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff3799b2740 10 client.27371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff3799b2740 20 client.27371 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828c46180 0x55f828c48570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff3799b2740 4 client.27371 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f828b24410 con 0x55f828c46180 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.906+0000 7ff374964640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828c46180 0x55f828c48570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.907+0000 7ff34ffff640 10 client.27371 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.907+0000 7ff374964640 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828c46180 0x55f828c48570 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabb5a79b97899e server_cookie=de8d489220fd669f in_seq=0 out_seq=0 2024-05-09T09:18:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.908+0000 7ff34ffff640 1 -- 172.21.15.64:0/2009796483 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f828b24410 con 0x55f828c46180 2024-05-09T09:18:40.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.908+0000 7ff34ffff640 10 client.27371 handle_command_reply: tid=0 2024-05-09T09:18:40.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.908+0000 7ff3799b2740 1 client.27371 shutdown 2024-05-09T09:18:40.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 msgr2=0x55f828bf0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bf0eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff35c018460 tx=0x7ff35c00a510 comp rx=0 tx=0).stop 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 shutdown_connections 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55f828c46180 0x55f828c48570 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff34ffff640 0 client.27371 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f828bc6830 0x55f828bf1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f828bc16c0 0x55f828bf0eb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 --2- 172.21.15.64:0/2009796483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f828bc0cd0 0x55f828bf0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 >> 172.21.15.64:0/2009796483 conn(0x55f828b800a0 msgr2=0x55f828bb9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 shutdown_connections 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 1 -- 172.21.15.64:0/2009796483 wait complete. 2024-05-09T09:18:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.909+0000 7ff3799b2740 20 client.27371 trim_cache size 0 max 16384 2024-05-09T09:18:40.920 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:18:40.920 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:18:40.920 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "41952077-0cf0-45de-87a2-055b0135c058", 2024-05-09T09:18:40.920 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:18:40.920 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:18:40.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.912+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff3600a2d90 msgr2=0x7ff3600a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.912+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff3600a2d90 0x7ff3600a5250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55f828bc1f20 tx=0x7ff3680099b0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.912+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 msgr2=0x7ff370197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.912+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370197a90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff364009230 tx=0x7ff3640036c0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 shutdown_connections 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff3600a2d90 0x7ff3600a5250 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff370102320 0x7ff37019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff370101960 0x7ff370197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 --2- 172.21.15.64:0/3615599916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff370100770 0x7ff370197a90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 >> 172.21.15.64:0/3615599916 conn(0x7ff3700fbce0 msgr2=0x7ff3700fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 shutdown_connections 2024-05-09T09:18:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:40.913+0000 7ff376bc8640 1 -- 172.21.15.64:0/3615599916 wait complete. 2024-05-09T09:18:40.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:18:40.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:18:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:40 smithi179 ceph-mon[34086]: pgmap v656: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-05-09T09:18:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:18:40.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:18:40.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:18:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.150+0000 7fa59a373640 1 -- 172.21.15.64:0/87598121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 msgr2=0x7fa59410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.150+0000 7fa59a373640 1 --2- 172.21.15.64:0/87598121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa59410d750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa584009860 tx=0x7fa58402a360 comp rx=0 tx=0).stop 2024-05-09T09:18:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.150+0000 7fa59a373640 1 -- 172.21.15.64:0/87598121 shutdown_connections 2024-05-09T09:18:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.150+0000 7fa59a373640 1 --2- 172.21.15.64:0/87598121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa59410dde0 0x7fa5941102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.150+0000 7fa59a373640 1 --2- 172.21.15.64:0/87598121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa59410d750 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.150+0000 7fa59a373640 1 --2- 172.21.15.64:0/87598121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa594108a10 0x7fa59410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.151+0000 7fa59a373640 1 -- 172.21.15.64:0/87598121 >> 172.21.15.64:0/87598121 conn(0x7fa59406e930 msgr2=0x7fa594070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.151+0000 7fa59a373640 1 -- 172.21.15.64:0/87598121 shutdown_connections 2024-05-09T09:18:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.151+0000 7fa59a373640 1 -- 172.21.15.64:0/87598121 wait complete. 2024-05-09T09:18:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.152+0000 7fa59a373640 1 Processor -- start 2024-05-09T09:18:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.152+0000 7fa59a373640 1 -- start start 2024-05-09T09:18:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa59a373640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa594108a10 0x7fa5941a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa59a373640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa5941a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa59a373640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59410dde0 0x7fa5941add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa59a373640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa594113820 con 0x7fa59410dde0 2024-05-09T09:18:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa599371640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa594108a10 0x7fa5941a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa598b70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa5941a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa598b70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa5941a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35330/0 (socket says 172.21.15.64:35330) 2024-05-09T09:18:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.153+0000 7fa598b70640 1 -- 172.21.15.64:0/1950457551 learned_addr learned my addr 172.21.15.64:0/1950457551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.154+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941136a0 con 0x7fa594108a10 2024-05-09T09:18:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.154+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941139a0 con 0x7fa59410b340 2024-05-09T09:18:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.154+0000 7fa599b72640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59410dde0 0x7fa5941add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.155+0000 7fa599b72640 1 -- 172.21.15.64:0/1950457551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 msgr2=0x7fa5941a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.155+0000 7fa599b72640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa5941a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.155+0000 7fa599b72640 1 -- 172.21.15.64:0/1950457551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa594108a10 msgr2=0x7fa5941a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.155+0000 7fa599b72640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa594108a10 0x7fa5941a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.155+0000 7fa599b72640 1 -- 172.21.15.64:0/1950457551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5941ae390 con 0x7fa59410dde0 2024-05-09T09:18:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.155+0000 7fa599b72640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59410dde0 0x7fa5941add20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa59000e970 tx=0x7fa59000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.156+0000 7fa58a7fc640 1 -- 172.21.15.64:0/1950457551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000ccc0 con 0x7fa59410dde0 2024-05-09T09:18:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.156+0000 7fa58a7fc640 1 -- 172.21.15.64:0/1950457551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5900035f0 con 0x7fa59410dde0 2024-05-09T09:18:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.156+0000 7fa58a7fc640 1 -- 172.21.15.64:0/1950457551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000ce60 con 0x7fa59410dde0 2024-05-09T09:18:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.157+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5941ae680 con 0x7fa59410dde0 2024-05-09T09:18:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.157+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5941aeb20 con 0x7fa59410dde0 2024-05-09T09:18:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.161+0000 7fa58a7fc640 1 -- 172.21.15.64:0/1950457551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5900037a0 con 0x7fa59410dde0 2024-05-09T09:18:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.165+0000 7fa58a7fc640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5800a2ce0 0x7fa5800a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.165+0000 7fa59e3c4740 1 Processor -- start 2024-05-09T09:18:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.165+0000 7fa58a7fc640 1 -- 172.21.15.64:0/1950457551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5900cf660 con 0x7fa59410dde0 2024-05-09T09:18:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 -- start start 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca1fad0 0x558e1ca245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca24c40 0x558e1ca270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e1c983d00 con 0x558e1ca1f0e0 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e1c92b4c0 con 0x558e1ca1fad0 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e1c97fcc0 con 0x558e1ca24c40 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa599b72640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca24c40 0x558e1ca270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa599b72640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca24c40 0x558e1ca270d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35332/0 (socket says 172.21.15.64:35332) 2024-05-09T09:18:41.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.166+0000 7fa599b72640 1 -- 172.21.15.64:0/4018407124 learned_addr learned my addr 172.21.15.64:0/4018407124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:41.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.167+0000 7fa599371640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5800a2ce0 0x7fa5800a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.167+0000 7fa598b70640 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca1fad0 0x558e1ca245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.167+0000 7fa599371640 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa598b70640 1 -- 172.21.15.64:0/4018407124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca24c40 msgr2=0x558e1ca270d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa598b70640 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca24c40 0x558e1ca270d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa598b70640 1 -- 172.21.15.64:0/4018407124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 msgr2=0x558e1ca1f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa598b70640 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca1f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa598b70640 1 -- 172.21.15.64:0/4018407124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e1c7617d0 con 0x558e1ca1fad0 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa599371640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5800a2ce0 0x7fa5800a51a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c00c650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.168+0000 7fa598b70640 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca1fad0 0x558e1ca245d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa584000c00 tx=0x7fa584009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27275b7a755b1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:41.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.169+0000 7fa57f7fe640 1 -- 172.21.15.64:0/4018407124 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584037040 con 0x558e1ca1fad0 2024-05-09T09:18:41.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.169+0000 7fa57f7fe640 1 -- 172.21.15.64:0/4018407124 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa584030e50 con 0x558e1ca1fad0 2024-05-09T09:18:41.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.169+0000 7fa57f7fe640 1 -- 172.21.15.64:0/4018407124 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584005560 con 0x558e1ca1fad0 2024-05-09T09:18:41.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.172+0000 7fa59e3c4740 1 -- 172.21.15.64:0/4018407124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca1fad0 msgr2=0x558e1ca245d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.172+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca1fad0 0x558e1ca245d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa584000c00 tx=0x7fa584009560 comp rx=0 tx=0).stop 2024-05-09T09:18:41.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.176+0000 7fa59e3c4740 1 -- 172.21.15.64:0/4018407124 shutdown_connections 2024-05-09T09:18:41.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.176+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca24c40 0x558e1ca270d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.176+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca1fad0 0x558e1ca245d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.176+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/4018407124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.176+0000 7fa59e3c4740 1 -- 172.21.15.64:0/4018407124 >> 172.21.15.64:0/4018407124 conn(0x558e1ca1a750 msgr2=0x558e1ca1ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:41.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.177+0000 7fa59e3c4740 1 -- 172.21.15.64:0/4018407124 shutdown_connections 2024-05-09T09:18:41.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.180+0000 7fa59e3c4740 1 -- 172.21.15.64:0/4018407124 wait complete. 2024-05-09T09:18:41.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.180+0000 7fa59e3c4740 1 -- start start 2024-05-09T09:18:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.180+0000 7fa59e3c4740 1 Processor -- start 2024-05-09T09:18:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca29b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e1ca27b90 con 0x558e1ca1fad0 2024-05-09T09:18:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e1c93eb90 con 0x558e1ca24c40 2024-05-09T09:18:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e1ca27d10 con 0x558e1ca1f0e0 2024-05-09T09:18:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa59e3c4740 10 client.0 _ll_register_callbacks cb 0x558e1c97ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa599b72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca30b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa599b72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca30b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54664/0 (socket says 172.21.15.64:54664) 2024-05-09T09:18:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.181+0000 7fa599b72640 1 -- 172.21.15.64:0/755054354 learned_addr learned my addr 172.21.15.64:0/755054354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.183+0000 7fa598b70640 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca29b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.183+0000 7fa599b72640 1 -- 172.21.15.64:0/755054354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 msgr2=0x558e1ca295c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.183+0000 7fa599b72640 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca295c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.183+0000 7fa599b72640 1 -- 172.21.15.64:0/755054354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 msgr2=0x558e1ca29b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.183+0000 7fa599b72640 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca29b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.183+0000 7fa599b72640 1 -- 172.21.15.64:0/755054354 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e1c6973f0 con 0x558e1ca24c40 2024-05-09T09:18:41.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.184+0000 7fa599b72640 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca30b40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa5900cce20 tx=0x7fa590003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20422ac32f02c9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:41.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.184+0000 7fa57cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:41.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.184+0000 7fa57cff9640 1 -- 172.21.15.64:0/755054354 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900b8020 con 0x558e1ca24c40 2024-05-09T09:18:41.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.184+0000 7fa57cff9640 1 -- 172.21.15.64:0/755054354 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5900cf370 con 0x558e1ca24c40 2024-05-09T09:18:41.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.185+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa584030e50 con 0x558e1ca24c40 2024-05-09T09:18:41.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.188+0000 7fa57cff9640 1 -- 172.21.15.64:0/755054354 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa59009fee0 con 0x558e1ca24c40 2024-05-09T09:18:41.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.188+0000 7fa59e3c4740 10 client.27380 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:41.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.188+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558e1c6570f0 con 0x558e1ca24c40 2024-05-09T09:18:41.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.188+0000 7fa57cff9640 1 -- 172.21.15.64:0/755054354 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa584030e50 con 0x558e1ca24c40 2024-05-09T09:18:41.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.188+0000 7fa57cff9640 1 -- 172.21.15.64:0/755054354 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa59009b010 con 0x558e1ca24c40 2024-05-09T09:18:41.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa59e3c4740 10 client.27380 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa59e3c4740 10 client.27380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa59e3c4740 20 client.27380 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca354a0 0x558e1ca37960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa59e3c4740 4 client.27380 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e1c991430 con 0x558e1ca354a0 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.189+0000 7fa599371640 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca354a0 0x558e1ca37960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.190+0000 7fa599371640 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca354a0 0x558e1ca37960 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9856bb186ad413a server_cookie=2efe2362219213d4 in_seq=0 out_seq=0 2024-05-09T09:18:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.190+0000 7fa57cff9640 10 client.27380 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:41.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.191+0000 7fa57cff9640 1 -- 172.21.15.64:0/755054354 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e1c991430 con 0x558e1ca354a0 2024-05-09T09:18:41.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.191+0000 7fa57cff9640 10 client.27380 handle_command_reply: tid=0 2024-05-09T09:18:41.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.191+0000 7fa59e3c4740 1 client.27380 shutdown 2024-05-09T09:18:41.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 msgr2=0x558e1ca30b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca30b40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa5900cce20 tx=0x7fa590003bd0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 shutdown_connections 2024-05-09T09:18:41.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca354a0 0x558e1ca37960 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca30b40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca29b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/755054354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 >> 172.21.15.64:0/755054354 conn(0x558e1ca1a750 msgr2=0x558e1ca19a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa57cff9640 0 client.27380 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.192+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 shutdown_connections 2024-05-09T09:18:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.193+0000 7fa59e3c4740 1 -- 172.21.15.64:0/755054354 wait complete. 2024-05-09T09:18:41.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.193+0000 7fa59e3c4740 20 client.27380 trim_cache size 0 max 16384 2024-05-09T09:18:41.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.235+0000 7fa59e3c4740 1 Processor -- start 2024-05-09T09:18:41.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.236+0000 7fa59e3c4740 1 -- start start 2024-05-09T09:18:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.236+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.236+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1fad0 0x558e1ca30b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.236+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca1d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941c4150 con 0x558e1ca1f0e0 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941c1a20 con 0x558e1ca24c40 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa594120b80 con 0x558e1ca1fad0 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42750/0 (socket says 172.21.15.64:42750) 2024-05-09T09:18:41.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 -- 172.21.15.64:0/3593443141 learned_addr learned my addr 172.21.15.64:0/3593443141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599b72640 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca1d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 -- 172.21.15.64:0/3593443141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1fad0 msgr2=0x558e1ca30b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1fad0 0x558e1ca30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 -- 172.21.15.64:0/3593443141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 msgr2=0x558e1ca1d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca1d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.237+0000 7fa599371640 1 -- 172.21.15.64:0/3593443141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e1ca1ec40 con 0x558e1ca1f0e0 2024-05-09T09:18:41.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.238+0000 7fa599371640 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca374b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa58c01d000 tx=0x7fa58c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e33692169aae597 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:41.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.240+0000 7fa57effd640 1 -- 172.21.15.64:0/3593443141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003970 con 0x558e1ca1f0e0 2024-05-09T09:18:41.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.240+0000 7fa57effd640 1 -- 172.21.15.64:0/3593443141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa58c019040 con 0x558e1ca1f0e0 2024-05-09T09:18:41.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.241+0000 7fa57effd640 1 -- 172.21.15.64:0/3593443141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0047d0 con 0x558e1ca1f0e0 2024-05-09T09:18:41.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.241+0000 7fa59e3c4740 1 -- 172.21.15.64:0/3593443141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 msgr2=0x558e1ca374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.241+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca374b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa58c01d000 tx=0x7fa58c01f000 comp rx=0 tx=0).stop 2024-05-09T09:18:41.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.242+0000 7fa59e3c4740 1 -- 172.21.15.64:0/3593443141 shutdown_connections 2024-05-09T09:18:41.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.242+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca1d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.242+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1fad0 0x558e1ca30b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.242+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/3593443141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1f0e0 0x558e1ca374b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.242+0000 7fa59e3c4740 1 -- 172.21.15.64:0/3593443141 >> 172.21.15.64:0/3593443141 conn(0x558e1ca1a750 msgr2=0x558e1ca21010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:41.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.244+0000 7fa59e3c4740 1 -- 172.21.15.64:0/3593443141 shutdown_connections 2024-05-09T09:18:41.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.244+0000 7fa59e3c4740 1 -- 172.21.15.64:0/3593443141 wait complete. 2024-05-09T09:18:41.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.244+0000 7fa59e3c4740 1 -- start start 2024-05-09T09:18:41.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.245+0000 7fa59e3c4740 1 Processor -- start 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.245+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca2e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca2e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599371640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca2e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599371640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca2e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35360/0 (socket says 172.21.15.64:35360) 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa59e3c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941bfee0 con 0x558e1ca1fad0 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941c1350 con 0x558e1ca24c40 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa59e3c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941c05b0 con 0x558e1ca1f0e0 2024-05-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca2f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54676/0 (socket says 172.21.15.64:54676) 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 -- 172.21.15.64:0/1645751377 learned_addr learned my addr 172.21.15.64:0/1645751377 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa59e3c4740 10 client.0 _ll_register_callbacks cb 0x7fa59009b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa598b70640 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca2e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 -- 172.21.15.64:0/1645751377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 msgr2=0x558e1ca2e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca2e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 -- 172.21.15.64:0/1645751377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 msgr2=0x558e1ca2e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.246+0000 7fa599b72640 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca2e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.247+0000 7fa599b72640 1 -- 172.21.15.64:0/1645751377 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e1ca2fc60 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.247+0000 7fa599b72640 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca2f670 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa590097f90 tx=0x7fa5900cce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84618014a8bbd6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.247+0000 7fa57cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.248+0000 7fa57cff9640 1 -- 172.21.15.64:0/1645751377 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59007e070 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.248+0000 7fa57cff9640 1 -- 172.21.15.64:0/1645751377 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa5900ba4c0 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.248+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c019040 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.248+0000 7fa57cff9640 1 -- 172.21.15.64:0/1645751377 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa5900b8070 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.248+0000 7fa59e3c4740 10 client.27386 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.248+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558e1ca9baa0 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.249+0000 7fa57cff9640 1 -- 172.21.15.64:0/1645751377 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5900a47a0 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.249+0000 7fa57cff9640 1 -- 172.21.15.64:0/1645751377 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa590081020 con 0x558e1ca24c40 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.250+0000 7fa59e3c4740 10 client.27386 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.250+0000 7fa59e3c4740 10 client.27386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.250+0000 7fa59e3c4740 20 client.27386 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.250+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca9c300 0x558e1ca9e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.250+0000 7fa59e3c4740 4 client.27386 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:18:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.250+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa58c019040 con 0x558e1ca9c300 2024-05-09T09:18:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.252+0000 7fa599371640 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca9c300 0x558e1ca9e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.253+0000 7fa599371640 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca9c300 0x558e1ca9e720 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2569d68ff99932f server_cookie=a4798c3c3de7f33e in_seq=0 out_seq=0 2024-05-09T09:18:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.253+0000 7fa57cff9640 10 client.27386 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.254+0000 7fa57cff9640 1 -- 172.21.15.64:0/1645751377 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fa58c019040 con 0x558e1ca9c300 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.254+0000 7fa57cff9640 10 client.27386 handle_command_reply: tid=0 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.254+0000 7fa59e3c4740 1 client.27386 shutdown 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.254+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 msgr2=0x558e1ca2f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca2f670 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa590097f90 tx=0x7fa5900cce50 comp rx=0 tx=0).stop 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 shutdown_connections 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e1ca9c300 0x558e1ca9e720 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1ca24c40 0x558e1ca2f670 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa57cff9640 0 client.27386 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e1ca1fad0 0x558e1ca2e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 --2- 172.21.15.64:0/1645751377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1ca1f0e0 0x558e1ca2e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.255+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 >> 172.21.15.64:0/1645751377 conn(0x558e1ca1a750 msgr2=0x558e1ca191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.256+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 shutdown_connections 2024-05-09T09:18:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.256+0000 7fa59e3c4740 1 -- 172.21.15.64:0/1645751377 wait complete. 2024-05-09T09:18:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.256+0000 7fa59e3c4740 20 client.27386 trim_cache size 0 max 16384 2024-05-09T09:18:41.259 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "41952077-0cf0-45de-87a2-055b0135c058": { 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "41952077-0cf0-45de-87a2-055b0135c058", 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:18:41.260 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.260+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5800a2ce0 msgr2=0x7fa5800a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.260+0000 7fa59a373640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5800a2ce0 0x7fa5800a51a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c00c650 comp rx=0 tx=0).stop 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.260+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59410dde0 msgr2=0x7fa5941add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59410dde0 0x7fa5941add20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa59000e970 tx=0x7fa59000ee30 comp rx=0 tx=0).stop 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 shutdown_connections 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa5800a2ce0 0x7fa5800a51a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59410dde0 0x7fa5941add20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa59410b340 0x7fa5941a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 --2- 172.21.15.64:0/1950457551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa594108a10 0x7fa5941a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 >> 172.21.15.64:0/1950457551 conn(0x7fa59406e930 msgr2=0x7fa594109430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.261+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 shutdown_connections 2024-05-09T09:18:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:18:41.262+0000 7fa59a373640 1 -- 172.21.15.64:0/1950457551 wait complete. 2024-05-09T09:18:41.284 INFO:tasks.cephfs.filesystem:scrub status for tag:41952077-0cf0-45de-87a2-055b0135c058 - {'path': '/', 'tag': '41952077-0cf0-45de-87a2-055b0135c058', 'options': 'recursive,force'} 2024-05-09T09:18:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 500.0 s, 501.8 tps, lat 238.379 ms stddev 869.673 2024-05-09T09:18:41.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:41 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:18:41.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:41 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:41 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:18:41.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:41 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:18:41.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:41 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:41 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:41 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:18:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:41 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:41 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:18:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:41 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:18:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:41 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:41 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:41 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:18:41.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:41 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:41 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:18:41.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:41 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:18:41.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:41 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:41.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:41 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:42.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:42 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:18:42.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:42 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:42.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:42 smithi139 ceph-mon[33599]: pgmap v657: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.2 MiB/s wr, 850 op/s 2024-05-09T09:18:42.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:42 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:18:42.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:42 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:42.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:42 smithi064 ceph-mon[30149]: pgmap v657: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.2 MiB/s wr, 850 op/s 2024-05-09T09:18:42.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:42 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:18:42.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:42 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:18:42.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:42 smithi179 ceph-mon[34086]: pgmap v657: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.2 MiB/s wr, 850 op/s 2024-05-09T09:18:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.447+0000 7f9a21ad4640 1 -- 172.21.15.64:0/428938457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 msgr2=0x7f9a1c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.447+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/428938457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c110330 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009ca0 tx=0x7f9a1402a8f0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.450+0000 7f9a21ad4640 1 -- 172.21.15.64:0/428938457 shutdown_connections 2024-05-09T09:18:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.450+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/428938457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c110330 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.450+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/428938457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a1c109050 0x7f9a1c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.450+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/428938457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a1c107e70 0x7f9a1c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.450+0000 7f9a21ad4640 1 -- 172.21.15.64:0/428938457 >> 172.21.15.64:0/428938457 conn(0x7f9a1c076e80 msgr2=0x7f9a1c0792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.456+0000 7f9a21ad4640 1 -- 172.21.15.64:0/428938457 shutdown_connections 2024-05-09T09:18:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.458+0000 7f9a21ad4640 1 -- 172.21.15.64:0/428938457 wait complete. 2024-05-09T09:18:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.459+0000 7f9a21ad4640 1 Processor -- start 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.460+0000 7f9a21ad4640 1 -- start start 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a21ad4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a1c107e70 0x7f9a1c07fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a21ad4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a1c109050 0x7f9a1c080300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a21ad4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c0812f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a21ad4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c07a550 con 0x7f9a1c107e70 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a21ad4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c07a3d0 con 0x7f9a1c109a10 2024-05-09T09:18:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a21ad4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c07a6d0 con 0x7f9a1c109050 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c0812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c0812f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54690/0 (socket says 172.21.15.64:54690) 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a1bfff640 1 -- 172.21.15.64:0/1008659057 learned_addr learned my addr 172.21.15.64:0/1008659057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.461+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a1c107e70 0x7f9a1c07fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a1bfff640 1 -- 172.21.15.64:0/1008659057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a1c109050 msgr2=0x7f9a1c080300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a1bfff640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a1c109050 0x7f9a1c080300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a1bfff640 1 -- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a1c107e70 msgr2=0x7f9a1c07fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a1bfff640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a1c107e70 0x7f9a1c07fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a1bfff640 1 -- 172.21.15.64:0/1008659057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c081960 con 0x7f9a1c109a10 2024-05-09T09:18:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a1bfff640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c0812f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a14031040 tx=0x7f9a14030c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a18ff9640 1 -- 172.21.15.64:0/1008659057 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14003970 con 0x7f9a1c109a10 2024-05-09T09:18:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.462+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c081bf0 con 0x7f9a1c109a10 2024-05-09T09:18:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.463+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c082140 con 0x7f9a1c109a10 2024-05-09T09:18:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 Processor -- start 2024-05-09T09:18:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 -- start start 2024-05-09T09:18:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 0x5570de910670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910c40 0x5570de915740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de918240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5570de874340 con 0x5570de910c40 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5570de81c290 con 0x5570de915db0 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a247e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5570de870300 con 0x5570de910250 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a18ff9640 1 -- 172.21.15.64:0/1008659057 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a1402d070 con 0x7f9a1c109a10 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.466+0000 7f9a18ff9640 1 -- 172.21.15.64:0/1008659057 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1403a420 con 0x7f9a1c109a10 2024-05-09T09:18:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.467+0000 7f9a1affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 0x5570de910670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.467+0000 7f9a1affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 0x5570de910670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35398/0 (socket says 172.21.15.64:35398) 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.467+0000 7f9a1affd640 1 -- 172.21.15.64:0/2698779324 learned_addr learned my addr 172.21.15.64:0/2698779324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.467+0000 7f9a1affd640 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de918240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.468+0000 7f9a1affd640 1 -- 172.21.15.64:0/2698779324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 msgr2=0x5570de918240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.468+0000 7f9a1affd640 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de918240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.468+0000 7f9a1affd640 1 -- 172.21.15.64:0/2698779324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910c40 msgr2=0x5570de915740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.468+0000 7f9a1affd640 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910c40 0x5570de915740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.468+0000 7f9a1affd640 1 -- 172.21.15.64:0/2698779324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570de8408e0 con 0x5570de910250 2024-05-09T09:18:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.468+0000 7f9a1affd640 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 0x5570de910670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0127a0 tx=0x7f9a0c00d3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46627abe3d4803b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.470+0000 7f9a01ffb640 1 -- 172.21.15.64:0/2698779324 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00a650 con 0x5570de910250 2024-05-09T09:18:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.470+0000 7f9a01ffb640 1 -- 172.21.15.64:0/2698779324 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a0c00aca0 con 0x5570de910250 2024-05-09T09:18:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.470+0000 7f9a18ff9640 1 -- 172.21.15.64:0/1008659057 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9a14003b10 con 0x7f9a1c109a10 2024-05-09T09:18:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.471+0000 7f9a01ffb640 1 -- 172.21.15.64:0/2698779324 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c010b20 con 0x5570de910250 2024-05-09T09:18:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.472+0000 7f9a18ff9640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a040a2e60 0x7f9a040a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.472+0000 7f9a18ff9640 1 -- 172.21.15.64:0/1008659057 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a140f0550 con 0x7f9a1c109a10 2024-05-09T09:18:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.473+0000 7f9a247e7740 1 -- 172.21.15.64:0/2698779324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 msgr2=0x5570de910670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.473+0000 7f9a247e7740 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 0x5570de910670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0127a0 tx=0x7f9a0c00d3a0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.475+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a040a2e60 0x7f9a040a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.477+0000 7f9a247e7740 1 -- 172.21.15.64:0/2698779324 shutdown_connections 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.477+0000 7f9a247e7740 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de918240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.477+0000 7f9a247e7740 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910c40 0x5570de915740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.477+0000 7f9a247e7740 1 --2- 172.21.15.64:0/2698779324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910250 0x5570de910670 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.477+0000 7f9a247e7740 1 -- 172.21.15.64:0/2698779324 >> 172.21.15.64:0/2698779324 conn(0x5570de90b8c0 msgr2=0x5570de90bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.478+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a040a2e60 0x7f9a040a5320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5570de90c830 tx=0x7f9a100046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.478+0000 7f9a247e7740 1 -- 172.21.15.64:0/2698779324 shutdown_connections 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.478+0000 7f9a247e7740 1 -- 172.21.15.64:0/2698779324 wait complete. 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.479+0000 7f9a247e7740 1 -- start start 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.479+0000 7f9a247e7740 1 Processor -- start 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.479+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de91a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.479+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 0x5570de91aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.479+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de915db0 0x5570de921aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a247e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5570de918d60 con 0x5570de910250 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a247e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5570de82f790 con 0x5570de910c40 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 0x5570de91aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 0x5570de91aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54706/0 (socket says 172.21.15.64:54706) 2024-05-09T09:18:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 -- 172.21.15.64:0/810670797 learned_addr learned my addr 172.21.15.64:0/810670797 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1bfff640 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de915db0 0x5570de921aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5570de918ee0 con 0x5570de915db0 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a247e7740 10 client.0 _ll_register_callbacks cb 0x5570de86aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 -- 172.21.15.64:0/810670797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de915db0 msgr2=0x5570de921aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de915db0 0x5570de921aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 -- 172.21.15.64:0/810670797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 msgr2=0x5570de91a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de91a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.480+0000 7f9a1affd640 1 -- 172.21.15.64:0/810670797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570de588210 con 0x5570de910c40 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.481+0000 7f9a1affd640 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 0x5570de91aa60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ae50 tx=0x7f9a0c00ae80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98ea6a14283ecc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.481+0000 7f99faffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.481+0000 7f99faffd640 1 -- 172.21.15.64:0/810670797 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c04d060 con 0x5570de910c40 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.481+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0c00aca0 con 0x5570de910c40 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.482+0000 7f99faffd640 1 -- 172.21.15.64:0/810670797 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a0c00a650 con 0x5570de910c40 2024-05-09T09:18:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.482+0000 7f99faffd640 1 -- 172.21.15.64:0/810670797 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9a0c054050 con 0x5570de910c40 2024-05-09T09:18:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.482+0000 7f9a247e7740 10 client.27398 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.482+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5570de654620 con 0x5570de910c40 2024-05-09T09:18:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.482+0000 7f99faffd640 1 -- 172.21.15.64:0/810670797 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a0c0104d0 con 0x5570de910c40 2024-05-09T09:18:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.482+0000 7f99faffd640 1 -- 172.21.15.64:0/810670797 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9a0c010a30 con 0x5570de910c40 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.483+0000 7f9a247e7740 10 client.27398 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.483+0000 7f9a247e7740 10 client.27398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.483+0000 7f9a247e7740 20 client.27398 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.483+0000 7f9a247e7740 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de9262b0 0x5570de928730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.483+0000 7f9a247e7740 4 client.27398 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.483+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a0c00aca0 con 0x5570de9262b0 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.484+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de9262b0 0x5570de928730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.485+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de9262b0 0x5570de928730 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91e33f83f5334958 server_cookie=98db6d52786139af in_seq=0 out_seq=0 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.485+0000 7f99faffd640 10 client.27398 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.485+0000 7f99faffd640 1 -- 172.21.15.64:0/810670797 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f9a0c00aca0 con 0x5570de9262b0 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.485+0000 7f99faffd640 10 client.27398 handle_command_reply: tid=0 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.485+0000 7f9a247e7740 1 client.27398 shutdown 2024-05-09T09:18:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 msgr2=0x5570de91aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 0x5570de91aa60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ae50 tx=0x7f9a0c00ae80 comp rx=0 tx=0).stop 2024-05-09T09:18:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 shutdown_connections 2024-05-09T09:18:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de9262b0 0x5570de928730 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de915db0 0x5570de921aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910c40 0x5570de91aa60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 --2- 172.21.15.64:0/810670797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de91a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.486+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 >> 172.21.15.64:0/810670797 conn(0x5570de90b8c0 msgr2=0x5570de90aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.487+0000 7f99faffd640 0 client.27398 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.487+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 shutdown_connections 2024-05-09T09:18:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.487+0000 7f9a247e7740 1 -- 172.21.15.64:0/810670797 wait complete. 2024-05-09T09:18:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.487+0000 7f9a247e7740 20 client.27398 trim_cache size 0 max 16384 2024-05-09T09:18:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.527+0000 7f9a247e7740 1 Processor -- start 2024-05-09T09:18:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.529+0000 7f9a247e7740 1 -- start start 2024-05-09T09:18:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de92e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de92e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de921a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a247e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a080125b0 con 0x5570de910250 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a247e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0800fe80 con 0x5570de915db0 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a247e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0800dcd0 con 0x5570de910c40 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de921a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de921a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54710/0 (socket says 172.21.15.64:54710) 2024-05-09T09:18:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a1bfff640 1 -- 172.21.15.64:0/1375976675 learned_addr learned my addr 172.21.15.64:0/1375976675 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.530+0000 7f9a1affd640 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de92e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.532+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de92e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.533+0000 7f9a1affd640 1 -- 172.21.15.64:0/1375976675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 msgr2=0x5570de921a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.533+0000 7f9a1affd640 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de921a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.533+0000 7f9a1affd640 1 -- 172.21.15.64:0/1375976675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 msgr2=0x5570de92e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.533+0000 7f9a1affd640 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de92e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.533+0000 7f9a1affd640 1 -- 172.21.15.64:0/1375976675 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570de90c630 con 0x5570de910c40 2024-05-09T09:18:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.535+0000 7f9a1affd640 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de92e580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c010fa0 tx=0x7f9a0c00d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb7af99dff6fedd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.538+0000 7f9a017fa640 1 -- 172.21.15.64:0/1375976675 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c04d070 con 0x5570de910c40 2024-05-09T09:18:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.538+0000 7f9a017fa640 1 -- 172.21.15.64:0/1375976675 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a0c00d7e0 con 0x5570de910c40 2024-05-09T09:18:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.538+0000 7f9a017fa640 1 -- 172.21.15.64:0/1375976675 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0052a0 con 0x5570de910c40 2024-05-09T09:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.543+0000 7f9a247e7740 1 -- 172.21.15.64:0/1375976675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 msgr2=0x5570de92e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.543+0000 7f9a247e7740 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de92e580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c010fa0 tx=0x7f9a0c00d9e0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.544+0000 7f9a247e7740 1 -- 172.21.15.64:0/1375976675 shutdown_connections 2024-05-09T09:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.544+0000 7f9a247e7740 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de915db0 0x5570de921a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.544+0000 7f9a247e7740 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de92e580 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.544+0000 7f9a247e7740 1 --2- 172.21.15.64:0/1375976675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de910250 0x5570de92e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.544+0000 7f9a247e7740 1 -- 172.21.15.64:0/1375976675 >> 172.21.15.64:0/1375976675 conn(0x5570de90b8c0 msgr2=0x5570de91a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.548+0000 7f9a247e7740 1 -- 172.21.15.64:0/1375976675 shutdown_connections 2024-05-09T09:18:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.548+0000 7f9a247e7740 1 -- 172.21.15.64:0/1375976675 wait complete. 2024-05-09T09:18:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.548+0000 7f9a247e7740 1 -- start start 2024-05-09T09:18:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.549+0000 7f9a247e7740 1 Processor -- start 2024-05-09T09:18:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.549+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 0x5570de91ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.549+0000 7f9a247e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de91f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a1b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 0x5570de91ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a1b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 0x5570de91ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:54722/0 (socket says 172.21.15.64:54722) 2024-05-09T09:18:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a1affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de91f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a1affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de91f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35430/0 (socket says 172.21.15.64:35430) 2024-05-09T09:18:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a1b7fe640 1 -- 172.21.15.64:0/623140274 learned_addr learned my addr 172.21.15.64:0/623140274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de915db0 0x5570de98a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0800e340 con 0x5570de915db0 2024-05-09T09:18:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0800f7b0 con 0x5570de910250 2024-05-09T09:18:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a1bfff640 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de915db0 0x5570de98a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0800ea10 con 0x5570de910c40 2024-05-09T09:18:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.550+0000 7f9a247e7740 10 client.0 _ll_register_callbacks cb 0x7f9a0c00dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.552+0000 7f9a1b7fe640 1 -- 172.21.15.64:0/623140274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 msgr2=0x5570de91f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.552+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de91f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.552+0000 7f9a1b7fe640 1 -- 172.21.15.64:0/623140274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de915db0 msgr2=0x5570de98a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.552+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de915db0 0x5570de98a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.553+0000 7f9a1b7fe640 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570de98ade0 con 0x5570de910250 2024-05-09T09:18:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.556+0000 7f9a1b7fe640 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 0x5570de91ece0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c108bc0 tx=0x7f9a10004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821ac6d5cf7f0b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.558+0000 7f99faffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:18:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.558+0000 7f99faffd640 1 -- 172.21.15.64:0/623140274 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10004460 con 0x5570de910250 2024-05-09T09:18:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.558+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0c00d7e0 con 0x5570de910250 2024-05-09T09:18:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.558+0000 7f99faffd640 1 -- 172.21.15.64:0/623140274 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9a10026510 con 0x5570de910250 2024-05-09T09:18:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.559+0000 7f99faffd640 1 -- 172.21.15.64:0/623140274 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9a10026710 con 0x5570de910250 2024-05-09T09:18:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.559+0000 7f9a247e7740 10 client.27404 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.559+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5570de98c080 con 0x5570de910250 2024-05-09T09:18:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f99faffd640 1 -- 172.21.15.64:0/623140274 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a0c00d7e0 con 0x5570de910250 2024-05-09T09:18:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f99faffd640 1 -- 172.21.15.64:0/623140274 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9a1002d070 con 0x5570de910250 2024-05-09T09:18:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f9a247e7740 10 client.27404 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f9a247e7740 10 client.27404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f9a247e7740 20 client.27404 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de98ca00 0x5570de98ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f9a247e7740 4 client.27404 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:18:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.562+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570de881a70 con 0x5570de98ca00 2024-05-09T09:18:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.565+0000 7f9a1affd640 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de98ca00 0x5570de98ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.576+0000 7f9a1affd640 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de98ca00 0x5570de98ee20 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535b328fc340bd9 server_cookie=d17bf0e07a19e9c3 in_seq=0 out_seq=0 2024-05-09T09:18:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.577+0000 7f99faffd640 10 client.27404 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.836+0000 7f99faffd640 1 -- 172.21.15.64:0/623140274 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570de881a70 con 0x5570de98ca00 2024-05-09T09:18:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.836+0000 7f99faffd640 10 client.27404 handle_command_reply: tid=0 2024-05-09T09:18:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.836+0000 7f9a247e7740 1 client.27404 shutdown 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.837+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 msgr2=0x5570de91ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.837+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 0x5570de91ece0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c108bc0 tx=0x7f9a10004730 comp rx=0 tx=0).stop 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 shutdown_connections 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5570de98ca00 0x5570de98ee20 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5570de915db0 0x5570de98a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570de910c40 0x5570de91f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 --2- 172.21.15.64:0/623140274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570de910250 0x5570de91ece0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 >> 172.21.15.64:0/623140274 conn(0x5570de90b8c0 msgr2=0x5570de911ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f99faffd640 0 client.27404 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 shutdown_connections 2024-05-09T09:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 1 -- 172.21.15.64:0/623140274 wait complete. 2024-05-09T09:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.838+0000 7f9a247e7740 20 client.27404 trim_cache size 0 max 16384 2024-05-09T09:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.842+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a040a2e60 msgr2=0x7f9a040a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.842+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a040a2e60 0x7f9a040a5320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5570de90c830 tx=0x7f9a100046c0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.842+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 msgr2=0x7f9a1c0812f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.842+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c0812f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a14031040 tx=0x7f9a14030c60 comp rx=0 tx=0).stop 2024-05-09T09:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.844+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 shutdown_connections 2024-05-09T09:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.844+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9a040a2e60 0x7f9a040a5320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.844+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a1c109a10 0x7f9a1c0812f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.844+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a1c109050 0x7f9a1c080300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.844+0000 7f9a21ad4640 1 --2- 172.21.15.64:0/1008659057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a1c107e70 0x7f9a1c07fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.844+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 >> 172.21.15.64:0/1008659057 conn(0x7f9a1c076e80 msgr2=0x7f9a1c10c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.848+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 shutdown_connections 2024-05-09T09:18:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:44.849+0000 7f9a21ad4640 1 -- 172.21.15.64:0/1008659057 wait complete. 2024-05-09T09:18:44.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:44 smithi064 ceph-mon[30149]: pgmap v658: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 8.9 MiB/s wr, 955 op/s 2024-05-09T09:18:44.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:44 smithi179 ceph-mon[34086]: pgmap v658: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 8.9 MiB/s wr, 955 op/s 2024-05-09T09:18:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:44 smithi139 ceph-mon[33599]: pgmap v658: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 8.9 MiB/s wr, 955 op/s 2024-05-09T09:18:46.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 505.0 s, 582.8 tps, lat 170.389 ms stddev 209.841 2024-05-09T09:18:46.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:46 smithi064 ceph-mon[30149]: pgmap v659: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.7 MiB/s wr, 955 op/s 2024-05-09T09:18:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:46 smithi179 ceph-mon[34086]: pgmap v659: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.7 MiB/s wr, 955 op/s 2024-05-09T09:18:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:46 smithi139 ceph-mon[33599]: pgmap v659: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.7 MiB/s wr, 955 op/s 2024-05-09T09:18:48.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:48 smithi064 ceph-mon[30149]: pgmap v660: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.8 MiB/s wr, 920 op/s 2024-05-09T09:18:48.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:48 smithi179 ceph-mon[34086]: pgmap v660: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.8 MiB/s wr, 920 op/s 2024-05-09T09:18:49.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:48 smithi139 ceph-mon[33599]: pgmap v660: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.8 MiB/s wr, 920 op/s 2024-05-09T09:18:50.874 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:50 smithi139 ceph-mon[33599]: pgmap v661: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:18:50.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:50 smithi064 ceph-mon[30149]: pgmap v661: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:18:50.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:50 smithi179 ceph-mon[34086]: pgmap v661: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-05-09T09:18:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 510.0 s, 792.6 tps, lat 127.112 ms stddev 60.296 2024-05-09T09:18:52.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:52 smithi064 ceph-mon[30149]: pgmap v662: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.6 MiB/s wr, 829 op/s 2024-05-09T09:18:52.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:52 smithi179 ceph-mon[34086]: pgmap v662: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.6 MiB/s wr, 829 op/s 2024-05-09T09:18:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:52 smithi139 ceph-mon[33599]: pgmap v662: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.6 MiB/s wr, 829 op/s 2024-05-09T09:18:54.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:54 smithi064 ceph-mon[30149]: pgmap v663: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.3 MiB/s wr, 898 op/s 2024-05-09T09:18:54.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:54 smithi179 ceph-mon[34086]: pgmap v663: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.3 MiB/s wr, 898 op/s 2024-05-09T09:18:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:54 smithi139 ceph-mon[33599]: pgmap v663: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.3 MiB/s wr, 898 op/s 2024-05-09T09:18:55.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:55 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:18:55.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:55 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:18:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:55 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:18:55.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:55 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:18:56.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:55 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:18:56.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:55 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:18:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 515.0 s, 688.0 tps, lat 145.014 ms stddev 45.519 2024-05-09T09:18:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:56 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:18:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:56 smithi064 ceph-mon[30149]: pgmap v664: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.2 MiB/s wr, 873 op/s 2024-05-09T09:18:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:56 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:18:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:56 smithi179 ceph-mon[34086]: pgmap v664: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.2 MiB/s wr, 873 op/s 2024-05-09T09:18:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:56 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:18:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:56 smithi139 ceph-mon[33599]: pgmap v664: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.2 MiB/s wr, 873 op/s 2024-05-09T09:18:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 -- 172.21.15.64:0/172154859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8075870 msgr2=0x7f05d8073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/172154859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8075870 0x7f05d8073cc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05d401a910 tx=0x7f05d403b470 comp rx=0 tx=0).stop 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 -- 172.21.15.64:0/172154859 shutdown_connections 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/172154859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d810de60 0x7f05d8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/172154859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d8074290 0x7f05d8074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/172154859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8075870 0x7f05d8073cc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.013+0000 7f05dfd5a640 1 -- 172.21.15.64:0/172154859 >> 172.21.15.64:0/172154859 conn(0x7f05d80fce00 msgr2=0x7f05d80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.014+0000 7f05dfd5a640 1 -- 172.21.15.64:0/172154859 shutdown_connections 2024-05-09T09:18:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.015+0000 7f05dfd5a640 1 -- 172.21.15.64:0/172154859 wait complete. 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 Processor -- start 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 -- start start 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8074290 0x7f05d8195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d8075870 0x7f05d8196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 0x7f05d819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8108260 con 0x7f05d810de60 2024-05-09T09:18:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81080e0 con 0x7f05d8075870 2024-05-09T09:18:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.016+0000 7f05dfd5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05d81083e0 con 0x7f05d8074290 2024-05-09T09:18:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 0x7f05d819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 0x7f05d819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52712/0 (socket says 172.21.15.64:52712) 2024-05-09T09:18:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 -- 172.21.15.64:0/1984508775 learned_addr learned my addr 172.21.15.64:0/1984508775 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 -- 172.21.15.64:0/1984508775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8074290 msgr2=0x7f05d8195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d8075870 0x7f05d8196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8074290 0x7f05d8195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 -- 172.21.15.64:0/1984508775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d8075870 msgr2=0x7f05d8196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d8075870 0x7f05d8196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05de2f7640 1 -- 172.21.15.64:0/1984508775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d806b380 con 0x7f05d810de60 2024-05-09T09:18:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.017+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d8075870 0x7f05d8196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:18:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.019+0000 7f05de2f7640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 0x7f05d819cf40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f05d000b970 tx=0x7f05d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.019+0000 7f05ceffd640 1 -- 172.21.15.64:0/1984508775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0019070 con 0x7f05d810de60 2024-05-09T09:18:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.019+0000 7f05ceffd640 1 -- 172.21.15.64:0/1984508775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05d0014440 con 0x7f05d810de60 2024-05-09T09:18:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.019+0000 7f05ceffd640 1 -- 172.21.15.64:0/1984508775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0007630 con 0x7f05d810de60 2024-05-09T09:18:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.019+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d806b670 con 0x7f05d810de60 2024-05-09T09:18:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.020+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d806bbf0 con 0x7f05d810de60 2024-05-09T09:18:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.025+0000 7f05ceffd640 1 -- 172.21.15.64:0/1984508775 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05d001c070 con 0x7f05d810de60 2024-05-09T09:18:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.027+0000 7f05e2a62740 1 Processor -- start 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.027+0000 7f05e2a62740 1 -- start start 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 0x55ee0f2c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05e2a62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f229340 con 0x55ee0f2cadb0 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05e2a62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f1d1290 con 0x55ee0f2c5c40 2024-05-09T09:18:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05e2a62740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f225300 con 0x55ee0f2c5250 2024-05-09T09:18:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 0x55ee0f2c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 0x55ee0f2c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54966/0 (socket says 172.21.15.64:54966) 2024-05-09T09:18:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.028+0000 7f05dd2f5640 1 -- 172.21.15.64:0/2024162250 learned_addr learned my addr 172.21.15.64:0/2024162250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05ceffd640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05c40a2e70 0x7f05c40a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05dd2f5640 1 -- 172.21.15.64:0/2024162250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 msgr2=0x55ee0f2ca740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2ca740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05dd2f5640 1 -- 172.21.15.64:0/2024162250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 msgr2=0x55ee0f2cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05dd2f5640 1 -- 172.21.15.64:0/2024162250 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f1f58e0 con 0x55ee0f2c5250 2024-05-09T09:18:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.029+0000 7f05ceffd640 1 -- 172.21.15.64:0/1984508775 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f05d00cf920 con 0x7f05d810de60 2024-05-09T09:18:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.030+0000 7f05de2f7640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05c40a2e70 0x7f05c40a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.030+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 0x55ee0f2c5670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05c8006fd0 tx=0x7f05c800c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcf74bcdab5de82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.031+0000 7f05de2f7640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05c40a2e70 0x7f05c40a5330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f05d00a44d0 tx=0x7f05d00098d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.031+0000 7f05c3fff640 1 -- 172.21.15.64:0/2024162250 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c800f040 con 0x55ee0f2c5250 2024-05-09T09:18:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.031+0000 7f05c3fff640 1 -- 172.21.15.64:0/2024162250 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c8017870 con 0x55ee0f2c5250 2024-05-09T09:18:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.032+0000 7f05e2a62740 1 -- 172.21.15.64:0/2024162250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 msgr2=0x55ee0f2c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.032+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 0x55ee0f2c5670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05c8006fd0 tx=0x7f05c800c410 comp rx=0 tx=0).stop 2024-05-09T09:18:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.032+0000 7f05c3fff640 1 -- 172.21.15.64:0/2024162250 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8013700 con 0x55ee0f2c5250 2024-05-09T09:18:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.032+0000 7f05e2a62740 1 -- 172.21.15.64:0/2024162250 shutdown_connections 2024-05-09T09:18:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.033+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.033+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.033+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2024162250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5250 0x55ee0f2c5670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.033+0000 7f05e2a62740 1 -- 172.21.15.64:0/2024162250 >> 172.21.15.64:0/2024162250 conn(0x55ee0f2c08c0 msgr2=0x55ee0f2c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.033+0000 7f05e2a62740 1 -- 172.21.15.64:0/2024162250 shutdown_connections 2024-05-09T09:18:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.038+0000 7f05e2a62740 1 -- 172.21.15.64:0/2024162250 wait complete. 2024-05-09T09:18:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.039+0000 7f05e2a62740 1 -- start start 2024-05-09T09:18:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.039+0000 7f05e2a62740 1 Processor -- start 2024-05-09T09:18:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2d4890 0x55ee0f2d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f2cddb0 con 0x55ee0f2cadb0 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f1e4790 con 0x55ee0f2d4890 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f2cdf30 con 0x55ee0f2c5c40 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.040+0000 7f05e2a62740 10 client.0 _ll_register_callbacks cb 0x55ee0f21fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.041+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2d4890 0x55ee0f2d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.041+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.041+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54980/0 (socket says 172.21.15.64:54980) 2024-05-09T09:18:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.041+0000 7f05dd2f5640 1 -- 172.21.15.64:0/1615323378 learned_addr learned my addr 172.21.15.64:0/1615323378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.042+0000 7f05dd2f5640 1 -- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2d4890 msgr2=0x55ee0f2d6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.042+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2d4890 0x55ee0f2d6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.042+0000 7f05dd2f5640 1 -- 172.21.15.64:0/1615323378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 msgr2=0x55ee0f2cf860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.042+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cf860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.042+0000 7f05dd2f5640 1 -- 172.21.15.64:0/1615323378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0ef3d210 con 0x55ee0f2c5c40 2024-05-09T09:18:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.043+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2cf320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f05c800cfd0 tx=0x7f05c80047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa9bdb8f129e2d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.043+0000 7f05c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.043+0000 7f05c17fa640 1 -- 172.21.15.64:0/1615323378 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c802a060 con 0x55ee0f2c5c40 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.043+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c8017870 con 0x55ee0f2c5c40 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.043+0000 7f05c17fa640 1 -- 172.21.15.64:0/1615323378 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f05c8017c20 con 0x55ee0f2c5c40 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.044+0000 7f05c17fa640 1 -- 172.21.15.64:0/1615323378 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f05c8031050 con 0x55ee0f2c5c40 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.044+0000 7f05e2a62740 10 client.27547 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.044+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ee0f009620 con 0x55ee0f2c5c40 2024-05-09T09:18:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.044+0000 7f05c17fa640 1 -- 172.21.15.64:0/1615323378 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c8017870 con 0x55ee0f2c5c40 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05c17fa640 1 -- 172.21.15.64:0/1615323378 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f05c8023040 con 0x55ee0f2c5c40 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05e2a62740 10 client.27547 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05e2a62740 10 client.27547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05e2a62740 20 client.27547 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05e2a62740 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f2db5d0 0x55ee0f2dda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05e2a62740 4 client.27547 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:18:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.045+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0f236a70 con 0x55ee0f2db5d0 2024-05-09T09:18:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.046+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f2db5d0 0x55ee0f2dda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.046+0000 7f05c17fa640 10 client.27547 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.046+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f2db5d0 0x55ee0f2dda30 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5767d5df66a0e16 server_cookie=ba41734554eaaed5 in_seq=0 out_seq=0 2024-05-09T09:18:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.047+0000 7f05c17fa640 1 -- 172.21.15.64:0/1615323378 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee0f236a70 con 0x55ee0f2db5d0 2024-05-09T09:18:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.047+0000 7f05c17fa640 10 client.27547 handle_command_reply: tid=0 2024-05-09T09:18:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.047+0000 7f05e2a62740 1 client.27547 shutdown 2024-05-09T09:18:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.049+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 msgr2=0x55ee0f2cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.049+0000 7f05e2a62740 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2cf320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f05c800cfd0 tx=0x7f05c80047d0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 shutdown_connections 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f2db5d0 0x55ee0f2dda30 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2d4890 0x55ee0f2d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 --2- 172.21.15.64:0/1615323378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2cf320 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 >> 172.21.15.64:0/1615323378 conn(0x55ee0f2c08c0 msgr2=0x55ee0f2bf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05c17fa640 0 client.27547 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 shutdown_connections 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 1 -- 172.21.15.64:0/1615323378 wait complete. 2024-05-09T09:18:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.050+0000 7f05e2a62740 20 client.27547 trim_cache size 0 max 16384 2024-05-09T09:18:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.081+0000 7f05e2a62740 1 Processor -- start 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 -- start start 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 0x55ee0f2ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0125a0 con 0x55ee0f2d4890 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc00fe70 con 0x55ee0f2c5c40 2024-05-09T09:18:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.082+0000 7f05e2a62740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc00dcc0 con 0x55ee0f2cadb0 2024-05-09T09:18:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.083+0000 7f05ddaf6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.083+0000 7f05ddaf6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54992/0 (socket says 172.21.15.64:54992) 2024-05-09T09:18:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.083+0000 7f05ddaf6640 1 -- 172.21.15.64:0/2086599917 learned_addr learned my addr 172.21.15.64:0/2086599917 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.084+0000 7f05ddaf6640 1 -- 172.21.15.64:0/2086599917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 msgr2=0x55ee0f2c3870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.084+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2c3870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.084+0000 7f05ddaf6640 1 -- 172.21.15.64:0/2086599917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 msgr2=0x55ee0f2ca760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:18:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.084+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 0x55ee0f2ca760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.084+0000 7f05ddaf6640 1 -- 172.21.15.64:0/2086599917 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f2bbbc0 con 0x55ee0f2cadb0 2024-05-09T09:18:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.084+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2ca220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f05d401a860 tx=0x7f05d4004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7de305658f9064c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.085+0000 7f05c37fe640 1 -- 172.21.15.64:0/2086599917 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4046030 con 0x55ee0f2cadb0 2024-05-09T09:18:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.085+0000 7f05c37fe640 1 -- 172.21.15.64:0/2086599917 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05d4004ef0 con 0x55ee0f2cadb0 2024-05-09T09:18:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.085+0000 7f05c37fe640 1 -- 172.21.15.64:0/2086599917 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4003850 con 0x55ee0f2cadb0 2024-05-09T09:18:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 -- 172.21.15.64:0/2086599917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 msgr2=0x55ee0f2ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2ca220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f05d401a860 tx=0x7f05d4004b00 comp rx=0 tx=0).stop 2024-05-09T09:18:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 -- 172.21.15.64:0/2086599917 shutdown_connections 2024-05-09T09:18:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 0x55ee0f2ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2ca220 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 --2- 172.21.15.64:0/2086599917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.086+0000 7f05e2a62740 1 -- 172.21.15.64:0/2086599917 >> 172.21.15.64:0/2086599917 conn(0x55ee0f2c08c0 msgr2=0x55ee0f2dd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.087+0000 7f05e2a62740 1 -- 172.21.15.64:0/2086599917 shutdown_connections 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.087+0000 7f05e2a62740 1 -- 172.21.15.64:0/2086599917 wait complete. 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.088+0000 7f05e2a62740 1 -- start start 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.088+0000 7f05e2a62740 1 Processor -- start 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.088+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05e2a62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 0x55ee0f2eba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05e2a62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc00e330 con 0x55ee0f2d4890 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05e2a62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc00f7a0 con 0x55ee0f2cadb0 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05e2a62740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc00ea00 con 0x55ee0f2c5c40 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05e2a62740 10 client.0 _ll_register_callbacks cb 0x7f05c800cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05dd2f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2eeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55000/0 (socket says 172.21.15.64:55000) 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05dd2f5640 1 -- 172.21.15.64:0/141098153 learned_addr learned my addr 172.21.15.64:0/141098153 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.089+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.090+0000 7f05dd2f5640 1 -- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2cadb0 msgr2=0x55ee0f2eb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.090+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2eb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.090+0000 7f05dd2f5640 1 -- 172.21.15.64:0/141098153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 msgr2=0x55ee0f2eba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.090+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 0x55ee0f2eba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.090+0000 7f05dd2f5640 1 -- 172.21.15.64:0/141098153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0f2ebfd0 con 0x55ee0f2c5c40 2024-05-09T09:18:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.090+0000 7f05dd2f5640 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2eeab0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05c8006d50 tx=0x7f05c8013e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a09f7d480cc098 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05c17fa640 1 -- 172.21.15.64:0/141098153 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c802a030 con 0x55ee0f2c5c40 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d4004ef0 con 0x55ee0f2c5c40 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05c17fa640 1 -- 172.21.15.64:0/141098153 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f05c80059c0 con 0x55ee0f2c5c40 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05c17fa640 1 -- 172.21.15.64:0/141098153 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f05c8042050 con 0x55ee0f2c5c40 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05e2a62740 10 client.27553 fetch_fsmap learned FSMap version 45 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.091+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ee0f340620 con 0x55ee0f2c5c40 2024-05-09T09:18:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05c17fa640 1 -- 172.21.15.64:0/141098153 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05d4004ef0 con 0x55ee0f2c5c40 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05c17fa640 1 -- 172.21.15.64:0/141098153 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f05c803d030 con 0x55ee0f2c5c40 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05e2a62740 10 client.27553 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05e2a62740 10 client.27553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05e2a62740 20 client.27553 populate_metadata read hostname 'smithi064' 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05e2a62740 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f340f10 0x55ee0f343330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05e2a62740 4 client.27553 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.092+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee0f230dc0 con 0x55ee0f340f10 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.093+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f340f10 0x55ee0f343330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:18:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.093+0000 7f05ddaf6640 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f340f10 0x55ee0f343330 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b244af3cada9f27 server_cookie=98f2b322e7d5198c in_seq=0 out_seq=0 2024-05-09T09:18:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.093+0000 7f05c17fa640 10 client.27553 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.127+0000 7f05c17fa640 1 -- 172.21.15.64:0/141098153 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee0f230dc0 con 0x55ee0f340f10 2024-05-09T09:18:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.128+0000 7f05c17fa640 10 client.27553 handle_command_reply: tid=0 2024-05-09T09:18:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.128+0000 7f05e2a62740 1 client.27553 shutdown 2024-05-09T09:18:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.129+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 msgr2=0x55ee0f2eeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.129+0000 7f05e2a62740 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2eeab0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05c8006d50 tx=0x7f05c8013e70 comp rx=0 tx=0).stop 2024-05-09T09:18:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 shutdown_connections 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ee0f340f10 0x55ee0f343330 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2d4890 0x55ee0f2eba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ee0f2cadb0 0x55ee0f2eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 --2- 172.21.15.64:0/141098153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee0f2c5c40 0x55ee0f2eeab0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 >> 172.21.15.64:0/141098153 conn(0x55ee0f2c08c0 msgr2=0x55ee0f2c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05c17fa640 0 client.27553 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:18:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.130+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 shutdown_connections 2024-05-09T09:18:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.131+0000 7f05e2a62740 1 -- 172.21.15.64:0/141098153 wait complete. 2024-05-09T09:18:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.131+0000 7f05e2a62740 20 client.27553 trim_cache size 0 max 16384 2024-05-09T09:18:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:18:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:18:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:18:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:18:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.136+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05c40a2e70 msgr2=0x7f05c40a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.136+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05c40a2e70 0x7f05c40a5330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f05d00a44d0 tx=0x7f05d00098d0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.136+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 msgr2=0x7f05d819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:18:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.136+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 0x7f05d819cf40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f05d000b970 tx=0x7f05d000be30 comp rx=0 tx=0).stop 2024-05-09T09:18:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.136+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 shutdown_connections 2024-05-09T09:18:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.137+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f05c40a2e70 0x7f05c40a5330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.137+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d810de60 0x7f05d819cf40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.137+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05d8075870 0x7f05d8196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.137+0000 7f05dfd5a640 1 --2- 172.21.15.64:0/1984508775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05d8074290 0x7f05d8195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:18:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.137+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 >> 172.21.15.64:0/1984508775 conn(0x7f05d80fce00 msgr2=0x7f05d8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:18:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.138+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 shutdown_connections 2024-05-09T09:18:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:18:58.138+0000 7f05dfd5a640 1 -- 172.21.15.64:0/1984508775 wait complete. 2024-05-09T09:18:58.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:18:58 smithi064 ceph-mon[30149]: pgmap v665: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-05-09T09:18:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:18:58 smithi179 ceph-mon[34086]: pgmap v665: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-05-09T09:18:59.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:18:58 smithi139 ceph-mon[33599]: pgmap v665: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-05-09T09:19:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:00 smithi064 ceph-mon[30149]: pgmap v666: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.6 MiB/s wr, 931 op/s 2024-05-09T09:19:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:00 smithi179 ceph-mon[34086]: pgmap v666: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.6 MiB/s wr, 931 op/s 2024-05-09T09:19:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:00 smithi139 ceph-mon[33599]: pgmap v666: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.6 MiB/s wr, 931 op/s 2024-05-09T09:19:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 520.0 s, 642.0 tps, lat 155.851 ms stddev 57.412 2024-05-09T09:19:02.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:02 smithi064 ceph-mon[30149]: pgmap v667: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.0 MiB/s wr, 787 op/s 2024-05-09T09:19:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:02 smithi179 ceph-mon[34086]: pgmap v667: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.0 MiB/s wr, 787 op/s 2024-05-09T09:19:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:02 smithi139 ceph-mon[33599]: pgmap v667: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.0 MiB/s wr, 787 op/s 2024-05-09T09:19:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:04 smithi064 ceph-mon[30149]: pgmap v668: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.6 MiB/s wr, 864 op/s 2024-05-09T09:19:04.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:04 smithi179 ceph-mon[34086]: pgmap v668: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.6 MiB/s wr, 864 op/s 2024-05-09T09:19:05.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:04 smithi139 ceph-mon[33599]: pgmap v668: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.6 MiB/s wr, 864 op/s 2024-05-09T09:19:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 525.0 s, 568.6 tps, lat 159.850 ms stddev 69.768 2024-05-09T09:19:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:06 smithi064 ceph-mon[30149]: pgmap v669: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.4 MiB/s wr, 845 op/s 2024-05-09T09:19:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:06 smithi179 ceph-mon[34086]: pgmap v669: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.4 MiB/s wr, 845 op/s 2024-05-09T09:19:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:06 smithi139 ceph-mon[33599]: pgmap v669: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.4 MiB/s wr, 845 op/s 2024-05-09T09:19:08.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:08 smithi064 ceph-mon[30149]: pgmap v670: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-05-09T09:19:08.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:08 smithi179 ceph-mon[34086]: pgmap v670: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-05-09T09:19:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:08 smithi139 ceph-mon[33599]: pgmap v670: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-05-09T09:19:10.982 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:10 smithi179 ceph-mon[34086]: pgmap v671: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 965 op/s 2024-05-09T09:19:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:10 smithi139 ceph-mon[33599]: pgmap v671: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 965 op/s 2024-05-09T09:19:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:10 smithi064 ceph-mon[30149]: pgmap v671: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 965 op/s 2024-05-09T09:19:11.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:19:11.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:19:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 -- 172.21.15.64:0/3894680062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 msgr2=0x7f40dc10d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 --2- 172.21.15.64:0/3894680062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc10d610 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009ca0 tx=0x7f40d402a8d0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 -- 172.21.15.64:0/3894680062 shutdown_connections 2024-05-09T09:19:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 --2- 172.21.15.64:0/3894680062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40dc10db50 0x7f40dc10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 --2- 172.21.15.64:0/3894680062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc10d610 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 --2- 172.21.15.64:0/3894680062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 0x7f40dc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.304+0000 7f40e3977640 1 -- 172.21.15.64:0/3894680062 >> 172.21.15.64:0/3894680062 conn(0x7f40dc06e8a0 msgr2=0x7f40dc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.305+0000 7f40e3977640 1 -- 172.21.15.64:0/3894680062 shutdown_connections 2024-05-09T09:19:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.305+0000 7f40e3977640 1 -- 172.21.15.64:0/3894680062 wait complete. 2024-05-09T09:19:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.306+0000 7f40e3977640 1 Processor -- start 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.306+0000 7f40e3977640 1 -- start start 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e3977640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 0x7f40dc1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e3977640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc1a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e3977640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40dc10db50 0x7f40dc1ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e3977640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc11c260 con 0x7f40dc10b1a0 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e3977640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc11c0e0 con 0x7f40dc10db50 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e3977640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc11c3e0 con 0x7f40dc108830 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e1713640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 0x7f40dc1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e0f12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc1a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e0f12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc1a6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44356/0 (socket says 172.21.15.64:44356) 2024-05-09T09:19:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e0f12640 1 -- 172.21.15.64:0/191838561 learned_addr learned my addr 172.21.15.64:0/191838561 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e1713640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 0x7f40dc1a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55336/0 (socket says 172.21.15.64:55336) 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.307+0000 7f40e1f14640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40dc10db50 0x7f40dc1ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.308+0000 7f40e0f12640 1 -- 172.21.15.64:0/191838561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 msgr2=0x7f40dc1a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.308+0000 7f40e0f12640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 0x7f40dc1a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.308+0000 7f40e0f12640 1 -- 172.21.15.64:0/191838561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40dc10db50 msgr2=0x7f40dc1ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.308+0000 7f40e0f12640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40dc10db50 0x7f40dc1ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.308+0000 7f40e0f12640 1 -- 172.21.15.64:0/191838561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc1ae4c0 con 0x7f40dc10b1a0 2024-05-09T09:19:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.308+0000 7f40e0f12640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc1a6de0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f40d4031980 tx=0x7f40d4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.309+0000 7f40d27fc640 1 -- 172.21.15.64:0/191838561 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4032ed0 con 0x7f40dc10b1a0 2024-05-09T09:19:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.309+0000 7f40d27fc640 1 -- 172.21.15.64:0/191838561 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40d4003e10 con 0x7f40dc10b1a0 2024-05-09T09:19:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.309+0000 7f40d27fc640 1 -- 172.21.15.64:0/191838561 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4030c80 con 0x7f40dc10b1a0 2024-05-09T09:19:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.311+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1ae750 con 0x7f40dc10b1a0 2024-05-09T09:19:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.311+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc1aeca0 con 0x7f40dc10b1a0 2024-05-09T09:19:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 Processor -- start 2024-05-09T09:19:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 -- start start 2024-05-09T09:19:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54c40 0x55b26fe59740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b26fdb8340 con 0x55b26fe54250 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b26fd60290 con 0x55b26fe54c40 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e6765740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b26fdb4300 con 0x55b26fe59db0 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe54670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44360/0 (socket says 172.21.15.64:44360) 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40e1f14640 1 -- 172.21.15.64:0/4077533674 learned_addr learned my addr 172.21.15.64:0/4077533674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.317+0000 7f40d27fc640 1 -- 172.21.15.64:0/191838561 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40d4004240 con 0x7f40dc10b1a0 2024-05-09T09:19:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40e1f14640 1 -- 172.21.15.64:0/4077533674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 msgr2=0x55b26fe5c240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40e1f14640 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe5c240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40e1f14640 1 -- 172.21.15.64:0/4077533674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54c40 msgr2=0x55b26fe59740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40e1f14640 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54c40 0x55b26fe59740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40e1f14640 1 -- 172.21.15.64:0/4077533674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26fd848e0 con 0x55b26fe54250 2024-05-09T09:19:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40e1f14640 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe54670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0099c0 tx=0x7f40cc009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d9a93aa8859a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.318+0000 7f40c77fe640 1 -- 172.21.15.64:0/4077533674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc017070 con 0x55b26fe54250 2024-05-09T09:19:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.319+0000 7f40d27fc640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f40c80a2da0 0x7f40c80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.319+0000 7f40c77fe640 1 -- 172.21.15.64:0/4077533674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40cc013470 con 0x55b26fe54250 2024-05-09T09:19:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.319+0000 7f40c77fe640 1 -- 172.21.15.64:0/4077533674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc012630 con 0x55b26fe54250 2024-05-09T09:19:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.319+0000 7f40d27fc640 1 -- 172.21.15.64:0/191838561 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40d40f07e0 con 0x7f40dc10b1a0 2024-05-09T09:19:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.320+0000 7f40e1713640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f40c80a2da0 0x7f40c80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.320+0000 7f40e6765740 1 -- 172.21.15.64:0/4077533674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 msgr2=0x55b26fe54670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.320+0000 7f40e6765740 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe54670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0099c0 tx=0x7f40cc009e80 comp rx=0 tx=0).stop 2024-05-09T09:19:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.321+0000 7f40e1713640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f40c80a2da0 0x7f40c80a5260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f40d80095f0 tx=0x7f40d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.321+0000 7f40e6765740 1 -- 172.21.15.64:0/4077533674 shutdown_connections 2024-05-09T09:19:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.321+0000 7f40e6765740 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe5c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.321+0000 7f40e6765740 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54c40 0x55b26fe59740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.321+0000 7f40e6765740 1 --2- 172.21.15.64:0/4077533674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe54670 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.321+0000 7f40e6765740 1 -- 172.21.15.64:0/4077533674 >> 172.21.15.64:0/4077533674 conn(0x55b26fe4f8c0 msgr2=0x55b26fe4fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.324+0000 7f40e6765740 1 -- 172.21.15.64:0/4077533674 shutdown_connections 2024-05-09T09:19:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.324+0000 7f40e6765740 1 -- 172.21.15.64:0/4077533674 wait complete. 2024-05-09T09:19:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.325+0000 7f40e6765740 1 -- start start 2024-05-09T09:19:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.325+0000 7f40e6765740 1 Processor -- start 2024-05-09T09:19:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.326+0000 7f40e6765740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.327+0000 7f40e6765740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe5ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.327+0000 7f40e6765740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fe65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.327+0000 7f40e6765740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b26fe5cd60 con 0x55b26fe54250 2024-05-09T09:19:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.327+0000 7f40e6765740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b26fd73790 con 0x55b26fe59db0 2024-05-09T09:19:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.327+0000 7f40e6765740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b26fe5cee0 con 0x55b26fe54c40 2024-05-09T09:19:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.327+0000 7f40e6765740 10 client.0 _ll_register_callbacks cb 0x55b26fdaea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.328+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fe65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.328+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fe65aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57920/0 (socket says 172.21.15.64:57920) 2024-05-09T09:19:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.328+0000 7f40e1f14640 1 -- 172.21.15.64:0/2656548119 learned_addr learned my addr 172.21.15.64:0/2656548119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.328+0000 7f40e0f12640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe5ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.329+0000 7f40e1f14640 1 -- 172.21.15.64:0/2656548119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 msgr2=0x55b26fe5ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.329+0000 7f40e1f14640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe5ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.329+0000 7f40e1f14640 1 -- 172.21.15.64:0/2656548119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 msgr2=0x55b26fe5e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.329+0000 7f40e1f14640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.329+0000 7f40e1f14640 1 -- 172.21.15.64:0/2656548119 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26facc210 con 0x55b26fe59db0 2024-05-09T09:19:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.329+0000 7f40e0f12640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe5ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:19:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.330+0000 7f40e1f14640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fe65aa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f40cc006a00 tx=0x7f40cc00ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212e0899a5b5ae34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.330+0000 7f40c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.330+0000 7f40c4ff9640 1 -- 172.21.15.64:0/2656548119 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc017050 con 0x55b26fe59db0 2024-05-09T09:19:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.331+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40cc013470 con 0x55b26fe59db0 2024-05-09T09:19:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.331+0000 7f40c4ff9640 1 -- 172.21.15.64:0/2656548119 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40cc01c860 con 0x55b26fe59db0 2024-05-09T09:19:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40c4ff9640 1 -- 172.21.15.64:0/2656548119 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f40cc02f050 con 0x55b26fe59db0 2024-05-09T09:19:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 10 client.27416 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b26fb98620 con 0x55b26fe59db0 2024-05-09T09:19:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40c4ff9640 1 -- 172.21.15.64:0/2656548119 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40cc013470 con 0x55b26fe59db0 2024-05-09T09:19:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40c4ff9640 1 -- 172.21.15.64:0/2656548119 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f40cc012c40 con 0x55b26fe59db0 2024-05-09T09:19:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 10 client.27416 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 10 client.27416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 20 client.27416 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fe69a50 0x55b26fe6be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 4 client.27416 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.332+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b26fdc5a70 con 0x55b26fe69a50 2024-05-09T09:19:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.333+0000 7f40e1713640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fe69a50 0x55b26fe6be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.334+0000 7f40e1713640 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fe69a50 0x55b26fe6be40 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8526a988c5f02f14 server_cookie=bf2a56636c74b48a in_seq=0 out_seq=0 2024-05-09T09:19:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.334+0000 7f40c4ff9640 10 client.27416 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.335+0000 7f40c4ff9640 1 -- 172.21.15.64:0/2656548119 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b26fdc5a70 con 0x55b26fe69a50 2024-05-09T09:19:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.336+0000 7f40c4ff9640 10 client.27416 handle_command_reply: tid=0 2024-05-09T09:19:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.337+0000 7f40e6765740 1 client.27416 shutdown 2024-05-09T09:19:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 msgr2=0x55b26fe65aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fe65aa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f40cc006a00 tx=0x7f40cc00ede0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 shutdown_connections 2024-05-09T09:19:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fe69a50 0x55b26fe6be40 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fe65aa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe5ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.340+0000 7f40e6765740 1 --2- 172.21.15.64:0/2656548119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe5e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.341+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 >> 172.21.15.64:0/2656548119 conn(0x55b26fe4f8c0 msgr2=0x55b26fe4ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.343+0000 7f40c4ff9640 0 client.27416 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.344+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 shutdown_connections 2024-05-09T09:19:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.346+0000 7f40e6765740 1 -- 172.21.15.64:0/2656548119 wait complete. 2024-05-09T09:19:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.346+0000 7f40e6765740 20 client.27416 trim_cache size 0 max 16384 2024-05-09T09:19:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 530.0 s, 470.8 tps, lat 230.426 ms stddev 297.711 2024-05-09T09:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.384+0000 7f40e6765740 1 Processor -- start 2024-05-09T09:19:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 -- start start 2024-05-09T09:19:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54250 0x55b26fe59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54c40 0x55b26fe5df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc1bbba0 con 0x55b26fe54c40 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc1b9470 con 0x55b26fe54250 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.386+0000 7f40e6765740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc1b72c0 con 0x55b26fe59db0 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe65a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55358/0 (socket says 172.21.15.64:55358) 2024-05-09T09:19:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 -- 172.21.15.64:0/3924636437 learned_addr learned my addr 172.21.15.64:0/3924636437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 -- 172.21.15.64:0/3924636437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54250 msgr2=0x55b26fe59750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54250 0x55b26fe59750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 -- 172.21.15.64:0/3924636437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54c40 msgr2=0x55b26fe5df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54c40 0x55b26fe5df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.387+0000 7f40e1f14640 1 -- 172.21.15.64:0/3924636437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26fe53e90 con 0x55b26fe59db0 2024-05-09T09:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.388+0000 7f40e1f14640 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe65a60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0139b0 tx=0x7f40cc009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d227425ad935d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.388+0000 7f40c6ffd640 1 -- 172.21.15.64:0/3924636437 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc017040 con 0x55b26fe59db0 2024-05-09T09:19:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.388+0000 7f40c6ffd640 1 -- 172.21.15.64:0/3924636437 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40cc013d70 con 0x55b26fe59db0 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.389+0000 7f40e6765740 1 -- 172.21.15.64:0/3924636437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 msgr2=0x55b26fe65a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.390+0000 7f40e6765740 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe65a60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0139b0 tx=0x7f40cc009e80 comp rx=0 tx=0).stop 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.390+0000 7f40e6765740 1 -- 172.21.15.64:0/3924636437 shutdown_connections 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.390+0000 7f40e6765740 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe59db0 0x55b26fe65a60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.390+0000 7f40e6765740 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54c40 0x55b26fe5df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.390+0000 7f40e6765740 1 --2- 172.21.15.64:0/3924636437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe54250 0x55b26fe59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.390+0000 7f40e6765740 1 -- 172.21.15.64:0/3924636437 >> 172.21.15.64:0/3924636437 conn(0x55b26fe4f8c0 msgr2=0x55b26fe5d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.392+0000 7f40e6765740 1 -- 172.21.15.64:0/3924636437 shutdown_connections 2024-05-09T09:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.396+0000 7f40e6765740 1 -- 172.21.15.64:0/3924636437 wait complete. 2024-05-09T09:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.397+0000 7f40e6765740 1 -- start start 2024-05-09T09:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.397+0000 7f40e6765740 1 Processor -- start 2024-05-09T09:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe60700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fedee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc1b7930 con 0x55b26fe54250 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc1b8da0 con 0x55b26fe59db0 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc1b8000 con 0x55b26fe54c40 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.398+0000 7f40e6765740 10 client.0 _ll_register_callbacks cb 0x7f40cc009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fedee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fedee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57930/0 (socket says 172.21.15.64:57930) 2024-05-09T09:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 -- 172.21.15.64:0/833367698 learned_addr learned my addr 172.21.15.64:0/833367698 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1713640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe60700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 -- 172.21.15.64:0/833367698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 msgr2=0x55b26fe60c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe60c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 -- 172.21.15.64:0/833367698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 msgr2=0x55b26fe60700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe60700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 -- 172.21.15.64:0/833367698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26fe5d5e0 con 0x55b26fe59db0 2024-05-09T09:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.399+0000 7f40e1f14640 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fedee40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f40cc013a90 tx=0x7f40cc011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78536bfcf86171cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.400+0000 7f40c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.400+0000 7f40c4ff9640 1 -- 172.21.15.64:0/833367698 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc041060 con 0x55b26fe59db0 2024-05-09T09:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.400+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40cc013d70 con 0x55b26fe59db0 2024-05-09T09:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.401+0000 7f40c4ff9640 1 -- 172.21.15.64:0/833367698 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f40cc01c860 con 0x55b26fe59db0 2024-05-09T09:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.401+0000 7f40c4ff9640 1 -- 172.21.15.64:0/833367698 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f40cc048050 con 0x55b26fe59db0 2024-05-09T09:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.401+0000 7f40e6765740 10 client.27419 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.401+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b26fee00c0 con 0x55b26fe59db0 2024-05-09T09:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.402+0000 7f40c4ff9640 1 -- 172.21.15.64:0/833367698 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40cc013d70 con 0x55b26fe59db0 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.402+0000 7f40c4ff9640 1 -- 172.21.15.64:0/833367698 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f40cc0037b0 con 0x55b26fe59db0 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e6765740 10 client.27419 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e6765740 10 client.27419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e6765740 20 client.27419 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e6765740 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fee0920 0x55b26fee2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e6765740 4 client.27419 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b26fdbfdc0 con 0x55b26fee0920 2024-05-09T09:19:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.403+0000 7f40e1713640 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fee0920 0x55b26fee2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.404+0000 7f40e1713640 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fee0920 0x55b26fee2d40 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5fd7e2bd3116a5 server_cookie=a15f5931f036cf86 in_seq=0 out_seq=0 2024-05-09T09:19:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.404+0000 7f40c4ff9640 10 client.27419 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.428+0000 7f40c4ff9640 1 -- 172.21.15.64:0/833367698 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b26fdbfdc0 con 0x55b26fee0920 2024-05-09T09:19:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.428+0000 7f40c4ff9640 10 client.27419 handle_command_reply: tid=0 2024-05-09T09:19:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.428+0000 7f40e6765740 1 client.27419 shutdown 2024-05-09T09:19:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.431+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 msgr2=0x55b26fedee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.431+0000 7f40e6765740 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fedee40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f40cc013a90 tx=0x7f40cc011af0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 shutdown_connections 2024-05-09T09:19:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40e6765740 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b26fee0920 0x55b26fee2d40 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40e6765740 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b26fe59db0 0x55b26fedee40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40e6765740 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b26fe54c40 0x55b26fe60c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40e6765740 1 --2- 172.21.15.64:0/833367698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b26fe54250 0x55b26fe60700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 >> 172.21.15.64:0/833367698 conn(0x55b26fe4f8c0 msgr2=0x55b26fe52640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.432+0000 7f40c4ff9640 0 client.27419 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.435+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 shutdown_connections 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.437+0000 7f40e6765740 1 -- 172.21.15.64:0/833367698 wait complete. 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.437+0000 7f40e6765740 20 client.27419 trim_cache size 0 max 16384 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.438+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f40c80a2da0 msgr2=0x7f40c80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.438+0000 7f40e3977640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f40c80a2da0 0x7f40c80a5260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f40d80095f0 tx=0x7f40d8005f90 comp rx=0 tx=0).stop 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.438+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 msgr2=0x7f40dc1a6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.438+0000 7f40e3977640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc1a6de0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f40d4031980 tx=0x7f40d4003c20 comp rx=0 tx=0).stop 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.439+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 shutdown_connections 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.439+0000 7f40e3977640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f40c80a2da0 0x7f40c80a5260 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.439+0000 7f40e3977640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40dc10db50 0x7f40dc1ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.439+0000 7f40e3977640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40dc10b1a0 0x7f40dc1a6de0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.439+0000 7f40e3977640 1 --2- 172.21.15.64:0/191838561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40dc108830 0x7f40dc1a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.439+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 >> 172.21.15.64:0/191838561 conn(0x7f40dc06e8a0 msgr2=0x7f40dc108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.443+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 shutdown_connections 2024-05-09T09:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:11.444+0000 7f40e3977640 1 -- 172.21.15.64:0/191838561 wait complete. 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 -- 172.21.15.64:0/851966534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09df20 msgr2=0x7f1a7c0a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/851966534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09df20 0x7f1a7c0a47b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1a8404f8c0 tx=0x7f1a84096020 comp rx=0 tx=0).stop 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 -- 172.21.15.64:0/851966534 shutdown_connections 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/851966534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09df20 0x7f1a7c0a47b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/851966534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a7c09d560 0x7f1a7c09d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/851966534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09c370 0x7f1a7c09c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 -- 172.21.15.64:0/851966534 >> 172.21.15.64:0/851966534 conn(0x7f1a7c00b310 msgr2=0x7f1a7c00b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.510+0000 7f1a8b29a640 1 -- 172.21.15.64:0/851966534 shutdown_connections 2024-05-09T09:19:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.511+0000 7f1a8b29a640 1 -- 172.21.15.64:0/851966534 wait complete. 2024-05-09T09:19:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.512+0000 7f1a8b29a640 1 Processor -- start 2024-05-09T09:19:11.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.512+0000 7f1a8b29a640 1 -- start start 2024-05-09T09:19:11.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a8b29a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09c370 0x7f1a7c132380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a8b29a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a7c09d560 0x7f1a7c1328c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a8b29a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09df20 0x7f1a7c139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a89036640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09c370 0x7f1a7c132380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a89036640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09c370 0x7f1a7c132380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44400/0 (socket says 172.21.15.64:44400) 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a88835640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a7c09d560 0x7f1a7c1328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a89036640 1 -- 172.21.15.64:0/4203549515 learned_addr learned my addr 172.21.15.64:0/4203549515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a8b29a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c0a7e40 con 0x7f1a7c09c370 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c0a7cc0 con 0x7f1a7c09d560 2024-05-09T09:19:11.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.513+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c0a7fc0 con 0x7f1a7c09df20 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.514+0000 7f1a89837640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09df20 0x7f1a7c139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.514+0000 7f1a89837640 1 -- 172.21.15.64:0/4203549515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a7c09d560 msgr2=0x7f1a7c1328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.514+0000 7f1a89837640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a7c09d560 0x7f1a7c1328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.514+0000 7f1a89837640 1 -- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09c370 msgr2=0x7f1a7c132380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.514+0000 7f1a89837640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09c370 0x7f1a7c132380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.514+0000 7f1a89837640 1 -- 172.21.15.64:0/4203549515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c139fa0 con 0x7f1a7c09df20 2024-05-09T09:19:11.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.515+0000 7f1a89837640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09df20 0x7f1a7c139930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a84050dd0 tx=0x7f1a84098cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.515+0000 7f1a7a7fc640 1 -- 172.21.15.64:0/4203549515 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84096920 con 0x7f1a7c09df20 2024-05-09T09:19:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.515+0000 7f1a7a7fc640 1 -- 172.21.15.64:0/4203549515 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a84098110 con 0x7f1a7c09df20 2024-05-09T09:19:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.515+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c13a230 con 0x7f1a7c09df20 2024-05-09T09:19:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.515+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c13a750 con 0x7f1a7c09df20 2024-05-09T09:19:11.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.515+0000 7f1a7a7fc640 1 -- 172.21.15.64:0/4203549515 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84075440 con 0x7f1a7c09df20 2024-05-09T09:19:11.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.520+0000 7f1a8dfa7740 1 Processor -- start 2024-05-09T09:19:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a7a7fc640 1 -- 172.21.15.64:0/4203549515 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a840755e0 con 0x7f1a7c09df20 2024-05-09T09:19:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 -- start start 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c058500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c05d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c0600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d78bfbcd00 con 0x55d78c0580e0 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d78bf644c0 con 0x55d78c05dc40 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.521+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d78bfb8cc0 con 0x55d78c058ad0 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c058500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a89036640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c05d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c058500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44406/0 (socket says 172.21.15.64:44406) 2024-05-09T09:19:11.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 -- 172.21.15.64:0/3342579809 learned_addr learned my addr 172.21.15.64:0/3342579809 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c0600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 -- 172.21.15.64:0/3342579809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 msgr2=0x55d78c05d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c05d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 -- 172.21.15.64:0/3342579809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 msgr2=0x55d78c0600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c0600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.522+0000 7f1a88835640 1 -- 172.21.15.64:0/3342579809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78bd9a7d0 con 0x55d78c0580e0 2024-05-09T09:19:11.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.523+0000 7f1a7a7fc640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1a700a2ce0 0x7f1a700a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.523+0000 7f1a89036640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1a700a2ce0 0x7f1a700a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.523+0000 7f1a88835640 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c058500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1a74014630 tx=0x7f1a7400a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907ecdc9a7f7f881 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.523+0000 7f1a7a7fc640 1 -- 172.21.15.64:0/4203549515 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1a8415afe0 con 0x7f1a7c09df20 2024-05-09T09:19:11.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.524+0000 7f1a6f7fe640 1 -- 172.21.15.64:0/3342579809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74004220 con 0x55d78c0580e0 2024-05-09T09:19:11.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.524+0000 7f1a6f7fe640 1 -- 172.21.15.64:0/3342579809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a740043c0 con 0x55d78c0580e0 2024-05-09T09:19:11.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.524+0000 7f1a6f7fe640 1 -- 172.21.15.64:0/3342579809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74012930 con 0x55d78c0580e0 2024-05-09T09:19:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.526+0000 7f1a89036640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1a700a2ce0 0x7f1a700a51a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55d78c058ed0 tx=0x7f1a80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/3342579809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 msgr2=0x55d78c058500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c058500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1a74014630 tx=0x7f1a7400a6b0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/3342579809 shutdown_connections 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c0600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c05d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/3342579809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c058500 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/3342579809 >> 172.21.15.64:0/3342579809 conn(0x55d78c053750 msgr2=0x55d78c053b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/3342579809 shutdown_connections 2024-05-09T09:19:11.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.527+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/3342579809 wait complete. 2024-05-09T09:19:11.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.528+0000 7f1a8dfa7740 1 -- start start 2024-05-09T09:19:11.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.528+0000 7f1a8dfa7740 1 Processor -- start 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c0580e0 0x55d78c0625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c062af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c05dc40 0x55d78c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d78c060c40 con 0x55d78c05dc40 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a88835640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a88835640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c062af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57972/0 (socket says 172.21.15.64:57972) 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a88835640 1 -- 172.21.15.64:0/2594112569 learned_addr learned my addr 172.21.15.64:0/2594112569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.529+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d78bf77b90 con 0x55d78c058ad0 2024-05-09T09:19:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.530+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d78c060dc0 con 0x55d78c0580e0 2024-05-09T09:19:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.530+0000 7f1a8dfa7740 10 client.0 _ll_register_callbacks cb 0x55d78bfb3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.530+0000 7f1a89036640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c0580e0 0x55d78c0625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.530+0000 7f1a89837640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c05dc40 0x55d78c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a89837640 1 -- 172.21.15.64:0/2594112569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c0580e0 msgr2=0x55d78c0625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a89837640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c0580e0 0x55d78c0625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a89837640 1 -- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 msgr2=0x55d78c062af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a89837640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c062af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a89837640 1 -- 172.21.15.64:0/2594112569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78bcd03f0 con 0x55d78c05dc40 2024-05-09T09:19:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a88835640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c062af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a89837640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c05dc40 0x55d78c069b30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1a84050ef0 tx=0x7f1a84067600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74077d615f54e1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.531+0000 7f1a6cff9640 1 -- 172.21.15.64:0/2594112569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8413f020 con 0x55d78c05dc40 2024-05-09T09:19:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.532+0000 7f1a6cff9640 1 -- 172.21.15.64:0/2594112569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1a84140250 con 0x55d78c05dc40 2024-05-09T09:19:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.532+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a740043c0 con 0x55d78c05dc40 2024-05-09T09:19:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.532+0000 7f1a6cff9640 1 -- 172.21.15.64:0/2594112569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1a840982c0 con 0x55d78c05dc40 2024-05-09T09:19:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a8dfa7740 10 client.17925 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d78bc900f0 con 0x55d78c05dc40 2024-05-09T09:19:11.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a6cff9640 1 -- 172.21.15.64:0/2594112569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a8409a040 con 0x55d78c05dc40 2024-05-09T09:19:11.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a6cff9640 1 -- 172.21.15.64:0/2594112569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f1a840b1030 con 0x55d78c05dc40 2024-05-09T09:19:11.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a8dfa7740 10 client.17925 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:11.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a8dfa7740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:11.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.533+0000 7f1a8dfa7740 20 client.17925 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:11.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.534+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.534+0000 7f1a8dfa7740 4 client.17925 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:11.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.534+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1a740043c0 con 0x55d78c06e310 2024-05-09T09:19:11.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.534+0000 7f1a89036640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.534+0000 7f1a6cff9640 10 client.17925 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.534+0000 7f1a89036640 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c070770 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1970d280168ff19 server_cookie=28f0237cb793a4c7 in_seq=0 out_seq=0 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.535+0000 7f1a6cff9640 1 -- 172.21.15.64:0/2594112569 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f1a740043c0 con 0x55d78c06e310 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.535+0000 7f1a6cff9640 10 client.17925 handle_command_reply: tid=0 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.535+0000 7f1a8dfa7740 1 client.17925 shutdown 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c05dc40 msgr2=0x55d78c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c05dc40 0x55d78c069b30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1a84050ef0 tx=0x7f1a84067600 comp rx=0 tx=0).stop 2024-05-09T09:19:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 shutdown_connections 2024-05-09T09:19:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c070770 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c05dc40 0x55d78c069b30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a6cff9640 0 client.17925 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c062af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.536+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/2594112569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c0580e0 0x55d78c0625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.537+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 >> 172.21.15.64:0/2594112569 conn(0x55d78c053750 msgr2=0x55d78c052a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.537+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 shutdown_connections 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.537+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/2594112569 wait complete. 2024-05-09T09:19:11.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.537+0000 7f1a8dfa7740 20 client.17925 trim_cache size 0 max 16384 2024-05-09T09:19:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 Processor -- start 2024-05-09T09:19:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 -- start start 2024-05-09T09:19:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c056660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c069af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c05dc40 0x55d78c056ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c1473a0 con 0x55d78c0580e0 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c144c70 con 0x55d78c058ad0 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.570+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c142ac0 con 0x55d78c05dc40 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89837640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c05dc40 0x55d78c056ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a88835640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c069af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a88835640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c069af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57978/0 (socket says 172.21.15.64:57978) 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a88835640 1 -- 172.21.15.64:0/1999212485 learned_addr learned my addr 172.21.15.64:0/1999212485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89036640 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c056660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89837640 1 -- 172.21.15.64:0/1999212485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 msgr2=0x55d78c069af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89837640 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c069af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89837640 1 -- 172.21.15.64:0/1999212485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 msgr2=0x55d78c056660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89837640 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c056660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.571+0000 7f1a89837640 1 -- 172.21.15.64:0/1999212485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78c05d180 con 0x55d78c05dc40 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.572+0000 7f1a89837640 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c05dc40 0x55d78c056ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1a8404f8c0 tx=0x7f1a84126d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4060f837bb0d6f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.572+0000 7f1a6effd640 1 -- 172.21.15.64:0/1999212485 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8413f070 con 0x55d78c05dc40 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.572+0000 7f1a6effd640 1 -- 172.21.15.64:0/1999212485 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a8409a030 con 0x55d78c05dc40 2024-05-09T09:19:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.572+0000 7f1a6effd640 1 -- 172.21.15.64:0/1999212485 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84123320 con 0x55d78c05dc40 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.573+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/1999212485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c05dc40 msgr2=0x55d78c056ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.573+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c05dc40 0x55d78c056ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1a8404f8c0 tx=0x7f1a84126d20 comp rx=0 tx=0).stop 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/1999212485 shutdown_connections 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c05dc40 0x55d78c056ba0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c058ad0 0x55d78c069af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/1999212485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c056660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/1999212485 >> 172.21.15.64:0/1999212485 conn(0x55d78c053750 msgr2=0x55d78c0625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/1999212485 shutdown_connections 2024-05-09T09:19:11.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/1999212485 wait complete. 2024-05-09T09:19:11.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.574+0000 7f1a8dfa7740 1 -- start start 2024-05-09T09:19:11.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.575+0000 7f1a8dfa7740 1 Processor -- start 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.575+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c0676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c067bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a8dfa7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a89036640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c0676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a89036640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c0676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44432/0 (socket says 172.21.15.64:44432) 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a8dfa7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c143130 con 0x55d78c0580e0 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a89036640 1 -- 172.21.15.64:0/614789189 learned_addr learned my addr 172.21.15.64:0/614789189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c1445a0 con 0x55d78c05dc40 2024-05-09T09:19:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c143800 con 0x55d78c058ad0 2024-05-09T09:19:11.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a8dfa7740 10 client.0 _ll_register_callbacks cb 0x7f1a84123820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:11.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.576+0000 7f1a89837640 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a89036640 1 -- 172.21.15.64:0/614789189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 msgr2=0x55d78c067bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:11.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a89036640 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c067bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a89036640 1 -- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 msgr2=0x55d78c068c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:11.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a89036640 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c068c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a89036640 1 -- 172.21.15.64:0/614789189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78c0691f0 con 0x55d78c0580e0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a89036640 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c0676b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55d78c054620 tx=0x7f1a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b12a1415c0cef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.577+0000 7f1a6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.578+0000 7f1a6cff9640 1 -- 172.21.15.64:0/614789189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004020 con 0x55d78c0580e0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.578+0000 7f1a6cff9640 1 -- 172.21.15.64:0/614789189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1a800041c0 con 0x55d78c0580e0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.578+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8409a030 con 0x55d78c0580e0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.578+0000 7f1a6cff9640 1 -- 172.21.15.64:0/614789189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1a80039050 con 0x55d78c0580e0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.578+0000 7f1a8dfa7740 10 client.17931 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.578+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d78c0d4e40 con 0x55d78c0580e0 2024-05-09T09:19:11.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.579+0000 7f1a6cff9640 1 -- 172.21.15.64:0/614789189 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a8409a030 con 0x55d78c0580e0 2024-05-09T09:19:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.579+0000 7f1a6cff9640 1 -- 172.21.15.64:0/614789189 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f1a8002b050 con 0x55d78c0580e0 2024-05-09T09:19:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.579+0000 7f1a8dfa7740 10 client.17931 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.579+0000 7f1a8dfa7740 10 client.17931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.579+0000 7f1a8dfa7740 20 client.17931 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.579+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c0d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.580+0000 7f1a8dfa7740 4 client.17931 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.580+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d78bfca430 con 0x55d78c06e310 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.580+0000 7f1a88835640 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c0d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.580+0000 7f1a88835640 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c0d78e0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b737faea9f6aa8 server_cookie=6a25b5b0fae1cd4a in_seq=0 out_seq=0 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.580+0000 7f1a6cff9640 10 client.17931 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.581+0000 7f1a6cff9640 1 -- 172.21.15.64:0/614789189 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d78bfca430 con 0x55d78c06e310 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.581+0000 7f1a6cff9640 10 client.17931 handle_command_reply: tid=0 2024-05-09T09:19:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.581+0000 7f1a8dfa7740 1 client.17931 shutdown 2024-05-09T09:19:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 msgr2=0x55d78c0676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c0676b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55d78c054620 tx=0x7f1a800046c0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 shutdown_connections 2024-05-09T09:19:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d78c06e310 0x55d78c0d78e0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78c05dc40 0x55d78c068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d78c058ad0 0x55d78c067bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 --2- 172.21.15.64:0/614789189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d78c0580e0 0x55d78c0676b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 >> 172.21.15.64:0/614789189 conn(0x55d78c053750 msgr2=0x55d78c0521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.583+0000 7f1a6cff9640 0 client.17931 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.584+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 shutdown_connections 2024-05-09T09:19:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.584+0000 7f1a8dfa7740 1 -- 172.21.15.64:0/614789189 wait complete. 2024-05-09T09:19:11.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.584+0000 7f1a8dfa7740 20 client.17931 trim_cache size 0 max 16384 2024-05-09T09:19:11.584 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:19:11.584 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:19:11.584 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:19:11.584 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:19:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1a700a2ce0 msgr2=0x7f1a700a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1a700a2ce0 0x7f1a700a51a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55d78c058ed0 tx=0x7f1a80009260 comp rx=0 tx=0).stop 2024-05-09T09:19:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09df20 msgr2=0x7f1a7c139930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09df20 0x7f1a7c139930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a84050dd0 tx=0x7f1a84098cb0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 shutdown_connections 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1a700a2ce0 0x7f1a700a51a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a7c09df20 0x7f1a7c139930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a7c09d560 0x7f1a7c1328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 --2- 172.21.15.64:0/4203549515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a7c09c370 0x7f1a7c132380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 >> 172.21.15.64:0/4203549515 conn(0x7f1a7c00b310 msgr2=0x7f1a7c0a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.586+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 shutdown_connections 2024-05-09T09:19:11.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.587+0000 7f1a8b29a640 1 -- 172.21.15.64:0/4203549515 wait complete. 2024-05-09T09:19:11.608 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:19:11.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:19:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.856+0000 7feb98f63640 1 -- 172.21.15.64:0/243540079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb941011f0 msgr2=0x7feb94105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.856+0000 7feb98f63640 1 --2- 172.21.15.64:0/243540079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb941011f0 0x7feb94105ab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb8c01acb0 tx=0x7feb8c03b8e0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.858+0000 7feb98f63640 1 -- 172.21.15.64:0/243540079 shutdown_connections 2024-05-09T09:19:11.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.858+0000 7feb98f63640 1 --2- 172.21.15.64:0/243540079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb941011f0 0x7feb94105ab0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.858+0000 7feb98f63640 1 --2- 172.21.15.64:0/243540079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.858+0000 7feb98f63640 1 --2- 172.21.15.64:0/243540079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb940ff720 0x7feb940ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.858+0000 7feb98f63640 1 -- 172.21.15.64:0/243540079 >> 172.21.15.64:0/243540079 conn(0x7feb940fb020 msgr2=0x7feb940fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:11.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.861+0000 7feb98f63640 1 -- 172.21.15.64:0/243540079 shutdown_connections 2024-05-09T09:19:11.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.862+0000 7feb98f63640 1 -- 172.21.15.64:0/243540079 wait complete. 2024-05-09T09:19:11.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 Processor -- start 2024-05-09T09:19:11.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 -- start start 2024-05-09T09:19:11.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb940ff720 0x7feb94195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb941011f0 0x7feb9419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feb941092b0 con 0x7feb941011f0 2024-05-09T09:19:11.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb94109130 con 0x7feb940ff720 2024-05-09T09:19:11.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.863+0000 7feb98f63640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feb94109430 con 0x7feb94100850 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55442/0 (socket says 172.21.15.64:55442) 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 -- 172.21.15.64:0/2227648485 learned_addr learned my addr 172.21.15.64:0/2227648485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb92d9d640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb941011f0 0x7feb9419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 -- 172.21.15.64:0/2227648485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb940ff720 msgr2=0x7feb94195840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb940ff720 0x7feb94195840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 -- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb941011f0 msgr2=0x7feb9419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb941011f0 0x7feb9419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:11.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.864+0000 7feb91d9b640 1 -- 172.21.15.64:0/2227648485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9419d460 con 0x7feb94100850 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.865+0000 7feb91d9b640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94195d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feb8800c890 tx=0x7feb8800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.865+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004220 con 0x7feb94100850 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.865+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb880043c0 con 0x7feb94100850 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.865+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004a00 con 0x7feb94100850 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.865+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9419d750 con 0x7feb94100850 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.866+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9419dc50 con 0x7feb94100850 2024-05-09T09:19:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.867+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9419df70 con 0x7feb94100850 2024-05-09T09:19:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.869+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb88004ba0 con 0x7feb94100850 2024-05-09T09:19:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.870+0000 7feb837fe640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb7c0a2d10 0x7feb7c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.870+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7feb880cf070 con 0x7feb94100850 2024-05-09T09:19:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.871+0000 7feb9259c640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb7c0a2d10 0x7feb7c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.872+0000 7feb9259c640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb7c0a2d10 0x7feb7c0a51d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feb8400b990 tx=0x7feb8400b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:11.878+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feb88014030 con 0x7feb94100850 2024-05-09T09:19:12.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.333+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb9419e140 con 0x7feb94100850 2024-05-09T09:19:12.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.335+0000 7feb837fe640 1 -- 172.21.15.64:0/2227648485 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11879 (secure 0 0 0) 0x7feb8809bc40 con 0x7feb94100850 2024-05-09T09:19:12.334 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:19:12.335 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:17:16.247105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:19:12.335 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 45 2024-05-09T09:19:12.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.339+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb7c0a2d10 msgr2=0x7feb7c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.339+0000 7feb98f63640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb7c0a2d10 0x7feb7c0a51d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feb8400b990 tx=0x7feb8400b370 comp rx=0 tx=0).stop 2024-05-09T09:19:12.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.339+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 msgr2=0x7feb94195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.339+0000 7feb98f63640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94195d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feb8800c890 tx=0x7feb8800cd50 comp rx=0 tx=0).stop 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 shutdown_connections 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feb7c0a2d10 0x7feb7c0a51d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb941011f0 0x7feb9419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb94100850 0x7feb94195d80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 --2- 172.21.15.64:0/2227648485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb940ff720 0x7feb94195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 >> 172.21.15.64:0/2227648485 conn(0x7feb940fb020 msgr2=0x7feb94103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.340+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 shutdown_connections 2024-05-09T09:19:12.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.341+0000 7feb98f63640 1 -- 172.21.15.64:0/2227648485 wait complete. 2024-05-09T09:19:12.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 -- 172.21.15.64:0/3113195313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 msgr2=0x7f46180ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 --2- 172.21.15.64:0/3113195313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f46180ffe10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4608009900 tx=0x7f460802a410 comp rx=0 tx=0).stop 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 -- 172.21.15.64:0/3113195313 shutdown_connections 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 --2- 172.21.15.64:0/3113195313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 0x7f4618101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 --2- 172.21.15.64:0/3113195313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46180685e0 0x7f4618100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 --2- 172.21.15.64:0/3113195313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f46180ffe10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 -- 172.21.15.64:0/3113195313 >> 172.21.15.64:0/3113195313 conn(0x7f46180fb460 msgr2=0x7f46180fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 -- 172.21.15.64:0/3113195313 shutdown_connections 2024-05-09T09:19:12.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.560+0000 7f461dea5640 1 -- 172.21.15.64:0/3113195313 wait complete. 2024-05-09T09:19:12.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.561+0000 7f461dea5640 1 Processor -- start 2024-05-09T09:19:12.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.561+0000 7f461dea5640 1 -- start start 2024-05-09T09:19:12.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f461dea5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f4618070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f461dea5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46180685e0 0x7f46180707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f461dea5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 0x7f46181ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f461dea5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f461810b210 con 0x7f46180685e0 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f461dea5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f461810b090 con 0x7f4618068130 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f461dea5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f461810b390 con 0x7f4618101080 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f46177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f4618070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f4617fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 0x7f46181ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f460ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46180685e0 0x7f46180707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f46177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f4618070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58034/0 (socket says 172.21.15.64:58034) 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f46177fe640 1 -- 172.21.15.64:0/2579884825 learned_addr learned my addr 172.21.15.64:0/2579884825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f4617fff640 1 -- 172.21.15.64:0/2579884825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 msgr2=0x7f4618070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f4617fff640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f4618070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f4617fff640 1 -- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46180685e0 msgr2=0x7f46180707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f4617fff640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46180685e0 0x7f46180707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.562+0000 7f4617fff640 1 -- 172.21.15.64:0/2579884825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46181a1900 con 0x7f4618101080 2024-05-09T09:19:12.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.563+0000 7f4617fff640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 0x7f46181ae130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f461000ea80 tx=0x7f461000ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.563+0000 7f46157fa640 1 -- 172.21.15.64:0/2579884825 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000cea0 con 0x7f4618101080 2024-05-09T09:19:12.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.563+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181a1bf0 con 0x7f4618101080 2024-05-09T09:19:12.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.563+0000 7f46157fa640 1 -- 172.21.15.64:0/2579884825 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4610005020 con 0x7f4618101080 2024-05-09T09:19:12.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.563+0000 7f46157fa640 1 -- 172.21.15.64:0/2579884825 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610010690 con 0x7f4618101080 2024-05-09T09:19:12.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.563+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a2190 con 0x7f4618101080 2024-05-09T09:19:12.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.567+0000 7f4620bbb740 1 Processor -- start 2024-05-09T09:19:12.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 -- start start 2024-05-09T09:19:12.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f46157fa640 1 -- 172.21.15.64:0/2579884825 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46100051d0 con 0x7f4618101080 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6126360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe612b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe612df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fe608ac40 con 0x564fe612baa0 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564fe6032580 con 0x564fe6126930 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.568+0000 7f4620bbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564fe6086c00 con 0x564fe6125f40 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.569+0000 7f460ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6126360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.569+0000 7f460ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6126360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55480/0 (socket says 172.21.15.64:55480) 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.569+0000 7f460ffff640 1 -- 172.21.15.64:0/1269361448 learned_addr learned my addr 172.21.15.64:0/1269361448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.569+0000 7f46177fe640 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe612b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.569+0000 7f460ffff640 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe612df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f460ffff640 1 -- 172.21.15.64:0/1269361448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 msgr2=0x564fe612b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f460ffff640 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe612b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f460ffff640 1 -- 172.21.15.64:0/1269361448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 msgr2=0x564fe612df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f460ffff640 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe612df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f460ffff640 1 -- 172.21.15.64:0/1269361448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe5d9e3f0 con 0x564fe6125f40 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f46157fa640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46040a2c80 0x7f46040a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f46157fa640 1 -- 172.21.15.64:0/2579884825 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46100cf9a0 con 0x7f4618101080 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.570+0000 7f46177fe640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46040a2c80 0x7f46040a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.571+0000 7f460ffff640 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6126360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46000093e0 tx=0x7f4600009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ba51fbf5d15669 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.571+0000 7f460dffb640 1 -- 172.21.15.64:0/1269361448 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460000f790 con 0x564fe6125f40 2024-05-09T09:19:12.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.571+0000 7f460dffb640 1 -- 172.21.15.64:0/1269361448 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f460000fde0 con 0x564fe6125f40 2024-05-09T09:19:12.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.571+0000 7f460dffb640 1 -- 172.21.15.64:0/1269361448 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460001fdf0 con 0x564fe6125f40 2024-05-09T09:19:12.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.573+0000 7f4620bbb740 1 -- 172.21.15.64:0/1269361448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 msgr2=0x564fe6126360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.573+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6126360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46000093e0 tx=0x7f4600009370 comp rx=0 tx=0).stop 2024-05-09T09:19:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.576+0000 7f46177fe640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46040a2c80 0x7f46040a5140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4608000c00 tx=0x7f4608005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 -- 172.21.15.64:0/1269361448 shutdown_connections 2024-05-09T09:19:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe612df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe612b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1269361448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6126360 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 -- 172.21.15.64:0/1269361448 >> 172.21.15.64:0/1269361448 conn(0x564fe61215b0 msgr2=0x564fe61219b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 -- 172.21.15.64:0/1269361448 shutdown_connections 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.577+0000 7f4620bbb740 1 -- 172.21.15.64:0/1269361448 wait complete. 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.578+0000 7f4620bbb740 1 -- start start 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.578+0000 7f4620bbb740 1 Processor -- start 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6130260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe61307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f46177fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6130260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f46177fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6130260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55484/0 (socket says 172.21.15.64:55484) 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f46177fe640 1 -- 172.21.15.64:0/3454778623 learned_addr learned my addr 172.21.15.64:0/3454778623 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe61377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.579+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fe612eaa0 con 0x564fe612baa0 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564fe6045c40 con 0x564fe6126930 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f460ffff640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe61307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564fe612ec20 con 0x564fe6125f40 2024-05-09T09:19:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f4620bbb740 10 client.0 _ll_register_callbacks cb 0x564fe5d9dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:12.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f4617fff640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe61377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f460ffff640 1 -- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 msgr2=0x564fe6130260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f460ffff640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6130260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f460ffff640 1 -- 172.21.15.64:0/3454778623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 msgr2=0x564fe61377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f460ffff640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe61377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.580+0000 7f460ffff640 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe5e687d0 con 0x564fe6126930 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f46177fe640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6130260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f460ffff640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe61307a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f46000097e0 tx=0x7f460001f8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7367eea70b2c2fe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f4617fff640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe61377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f45f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460000fde0 con 0x564fe6126930 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f45f6ffd640 1 -- 172.21.15.64:0/3454778623 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600004020 con 0x564fe6126930 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f45f6ffd640 1 -- 172.21.15.64:0/3454778623 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f46000041c0 con 0x564fe6126930 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f45f6ffd640 1 -- 172.21.15.64:0/3454778623 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4600030050 con 0x564fe6126930 2024-05-09T09:19:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f4620bbb740 10 client.27449 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.581+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=45}) -- 0x564fe613a9b0 con 0x564fe6126930 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f45f6ffd640 1 -- 172.21.15.64:0/3454778623 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4600026030 con 0x564fe6126930 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f4620bbb740 10 client.27449 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f4620bbb740 10 client.27449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f4620bbb740 20 client.27449 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe613bd00 0x564fe613e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f4620bbb740 4 client.27449 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.582+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe6098370 con 0x564fe613bd00 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.583+0000 7f46177fe640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe613bd00 0x564fe613e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.583+0000 7f46177fe640 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe613bd00 0x564fe613e0f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6a1a64ee2d6f64d server_cookie=47e8ccc0ea134326 in_seq=0 out_seq=0 2024-05-09T09:19:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.583+0000 7f45f6ffd640 10 client.27449 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.587+0000 7f45f6ffd640 1 -- 172.21.15.64:0/3454778623 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fe6098370 con 0x564fe613bd00 2024-05-09T09:19:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.587+0000 7f45f6ffd640 10 client.27449 handle_command_reply: tid=0 2024-05-09T09:19:12.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.589+0000 7f4620bbb740 1 client.27449 shutdown 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 msgr2=0x564fe61307a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe61307a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f46000097e0 tx=0x7f460001f8c0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 shutdown_connections 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe613bd00 0x564fe613e0f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe612baa0 0x564fe61377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6126930 0x564fe61307a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 --2- 172.21.15.64:0/3454778623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6125f40 0x564fe6130260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 >> 172.21.15.64:0/3454778623 conn(0x564fe61215b0 msgr2=0x564fe61207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f45f6ffd640 0 client.27449 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:12.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.590+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 shutdown_connections 2024-05-09T09:19:12.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.591+0000 7f4620bbb740 1 -- 172.21.15.64:0/3454778623 wait complete. 2024-05-09T09:19:12.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.591+0000 7f4620bbb740 20 client.27449 trim_cache size 0 max 16384 2024-05-09T09:19:12.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.618+0000 7f4620bbb740 1 Processor -- start 2024-05-09T09:19:12.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.619+0000 7f4620bbb740 1 -- start start 2024-05-09T09:19:12.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.619+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6125f40 0x564fe6124a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.619+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6126930 0x564fe6124f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.619+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 0x564fe61377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.620+0000 7f4620bbb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f46181c4420 con 0x564fe6125f40 2024-05-09T09:19:12.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.620+0000 7f4620bbb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46181c1cf0 con 0x564fe612baa0 2024-05-09T09:19:12.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.620+0000 7f4620bbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46181a9cc0 con 0x564fe6126930 2024-05-09T09:19:12.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.620+0000 7f4617fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 0x564fe61377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f460ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6126930 0x564fe6124f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 0x564fe61377a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58060/0 (socket says 172.21.15.64:58060) 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 -- 172.21.15.64:0/596167802 learned_addr learned my addr 172.21.15.64:0/596167802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 -- 172.21.15.64:0/596167802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6126930 msgr2=0x564fe6124f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6126930 0x564fe6124f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 -- 172.21.15.64:0/596167802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6125f40 msgr2=0x564fe6124a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6125f40 0x564fe6124a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.621+0000 7f4617fff640 1 -- 172.21.15.64:0/596167802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe611d1a0 con 0x564fe612baa0 2024-05-09T09:19:12.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.622+0000 7f4617fff640 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 0x564fe61377a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f461000c9a0 tx=0x7f461009bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a7de7d5161eb57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.622+0000 7f460d7fa640 1 -- 172.21.15.64:0/596167802 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46100a4a80 con 0x564fe612baa0 2024-05-09T09:19:12.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.622+0000 7f460d7fa640 1 -- 172.21.15.64:0/596167802 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461009a110 con 0x564fe612baa0 2024-05-09T09:19:12.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.623+0000 7f4620bbb740 1 -- 172.21.15.64:0/596167802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 msgr2=0x564fe61377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.623+0000 7f4620bbb740 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 0x564fe61377a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f461000c9a0 tx=0x7f461009bd10 comp rx=0 tx=0).stop 2024-05-09T09:19:12.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.624+0000 7f4620bbb740 1 -- 172.21.15.64:0/596167802 shutdown_connections 2024-05-09T09:19:12.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.624+0000 7f4620bbb740 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe612baa0 0x564fe61377a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.624+0000 7f4620bbb740 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe6126930 0x564fe6124f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.624+0000 7f4620bbb740 1 --2- 172.21.15.64:0/596167802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6125f40 0x564fe6124a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.624+0000 7f4620bbb740 1 -- 172.21.15.64:0/596167802 >> 172.21.15.64:0/596167802 conn(0x564fe61215b0 msgr2=0x564fe6124330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.626+0000 7f4620bbb740 1 -- 172.21.15.64:0/596167802 shutdown_connections 2024-05-09T09:19:12.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.626+0000 7f4620bbb740 1 -- 172.21.15.64:0/596167802 wait complete. 2024-05-09T09:19:12.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.626+0000 7f4620bbb740 1 -- start start 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.626+0000 7f4620bbb740 1 Processor -- start 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 0x564fe6134c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6126930 0x564fe6135170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe612baa0 0x564fe6136180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f46181c01b0 con 0x564fe6126930 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46181c1620 con 0x564fe6125f40 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46181c0880 con 0x564fe612baa0 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4620bbb740 10 client.0 _ll_register_callbacks cb 0x7f4600009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f46177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 0x564fe6134c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f46177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 0x564fe6134c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58062/0 (socket says 172.21.15.64:58062) 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f46177fe640 1 -- 172.21.15.64:0/1243141449 learned_addr learned my addr 172.21.15.64:0/1243141449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.627+0000 7f4617fff640 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe612baa0 0x564fe6136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f46177fe640 1 -- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe612baa0 msgr2=0x564fe6136180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f46177fe640 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe612baa0 0x564fe6136180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f46177fe640 1 -- 172.21.15.64:0/1243141449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6126930 msgr2=0x564fe6135170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f46177fe640 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6126930 0x564fe6135170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f46177fe640 1 -- 172.21.15.64:0/1243141449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe6136770 con 0x564fe6125f40 2024-05-09T09:19:12.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f46177fe640 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 0x564fe6134c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4608005ab0 tx=0x7f460802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26577a38d8d4e875 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f45f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:12.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.628+0000 7f45f6ffd640 1 -- 172.21.15.64:0/1243141449 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608059060 con 0x564fe6125f40 2024-05-09T09:19:12.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.629+0000 7f45f6ffd640 1 -- 172.21.15.64:0/1243141449 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4608003730 con 0x564fe6125f40 2024-05-09T09:19:12.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.629+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461009a110 con 0x564fe6125f40 2024-05-09T09:19:12.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.629+0000 7f45f6ffd640 1 -- 172.21.15.64:0/1243141449 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4608060050 con 0x564fe6125f40 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.629+0000 7f4620bbb740 10 client.27455 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.629+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564fe6137ae0 con 0x564fe6125f40 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f45f6ffd640 1 -- 172.21.15.64:0/1243141449 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461009a110 con 0x564fe6125f40 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f45f6ffd640 1 -- 172.21.15.64:0/1243141449 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4608043030 con 0x564fe6125f40 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f4620bbb740 10 client.27455 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f4620bbb740 10 client.27455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f4620bbb740 20 client.27455 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe61a28b0 0x564fe61a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f4620bbb740 4 client.27455 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:19:12.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.630+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fe60926c0 con 0x564fe61a28b0 2024-05-09T09:19:12.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.631+0000 7f460ffff640 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe61a28b0 0x564fe61a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.632+0000 7f460ffff640 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe61a28b0 0x564fe61a4ca0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65acb5bec4eeb51a server_cookie=53a35eca46bc626d in_seq=0 out_seq=0 2024-05-09T09:19:12.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.632+0000 7f45f6ffd640 10 client.27455 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:12.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.632+0000 7f45f6ffd640 1 -- 172.21.15.64:0/1243141449 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fe60926c0 con 0x564fe61a28b0 2024-05-09T09:19:12.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.632+0000 7f45f6ffd640 10 client.27455 handle_command_reply: tid=0 2024-05-09T09:19:12.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.632+0000 7f4620bbb740 1 client.27455 shutdown 2024-05-09T09:19:12.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 msgr2=0x564fe6134c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 0x564fe6134c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4608005ab0 tx=0x7f460802a410 comp rx=0 tx=0).stop 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 shutdown_connections 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x564fe61a28b0 0x564fe61a4ca0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fe612baa0 0x564fe6136180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe6126930 0x564fe6135170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f45f6ffd640 0 client.27455 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 --2- 172.21.15.64:0/1243141449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fe6125f40 0x564fe6134c30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 >> 172.21.15.64:0/1243141449 conn(0x564fe61215b0 msgr2=0x564fe61207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 shutdown_connections 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 1 -- 172.21.15.64:0/1243141449 wait complete. 2024-05-09T09:19:12.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.633+0000 7f4620bbb740 20 client.27455 trim_cache size 0 max 16384 2024-05-09T09:19:12.634 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46040a2c80 msgr2=0x7f46040a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46040a2c80 0x7f46040a5140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4608000c00 tx=0x7f4608005a20 comp rx=0 tx=0).stop 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 msgr2=0x7f46181ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 0x7f46181ae130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f461000ea80 tx=0x7f461000ef40 comp rx=0 tx=0).stop 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 shutdown_connections 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46040a2c80 0x7f46040a5140 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4618101080 0x7f46181ae130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46180685e0 0x7f46180707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 --2- 172.21.15.64:0/2579884825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4618068130 0x7f4618070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.635+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 >> 172.21.15.64:0/2579884825 conn(0x7f46180fb460 msgr2=0x7f46180fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.636+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 shutdown_connections 2024-05-09T09:19:12.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.636+0000 7f461dea5640 1 -- 172.21.15.64:0/2579884825 wait complete. 2024-05-09T09:19:12.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:19:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.815+0000 7fe612c5d640 1 -- 172.21.15.64:0/4073987493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 msgr2=0x7fe60c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.815+0000 7fe612c5d640 1 --2- 172.21.15.64:0/4073987493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 0x7fe60c100ca0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009900 tx=0x7fe5f402a400 comp rx=0 tx=0).stop 2024-05-09T09:19:12.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.818+0000 7fe612c5d640 1 -- 172.21.15.64:0/4073987493 shutdown_connections 2024-05-09T09:19:12.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.818+0000 7fe612c5d640 1 --2- 172.21.15.64:0/4073987493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.818+0000 7fe612c5d640 1 --2- 172.21.15.64:0/4073987493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 0x7fe60c100ca0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.818+0000 7fe612c5d640 1 --2- 172.21.15.64:0/4073987493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c0ff710 0x7fe60c0ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.818+0000 7fe612c5d640 1 -- 172.21.15.64:0/4073987493 >> 172.21.15.64:0/4073987493 conn(0x7fe60c0fb030 msgr2=0x7fe60c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.818+0000 7fe612c5d640 1 -- 172.21.15.64:0/4073987493 shutdown_connections 2024-05-09T09:19:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.819+0000 7fe612c5d640 1 -- 172.21.15.64:0/4073987493 wait complete. 2024-05-09T09:19:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.820+0000 7fe612c5d640 1 Processor -- start 2024-05-09T09:19:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 -- start start 2024-05-09T09:19:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c0ff710 0x7fe60c10c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 0x7fe60c10c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c1ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1139c0 con 0x7fe60c100840 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c113840 con 0x7fe60c1011e0 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.821+0000 7fe612c5d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c113b40 con 0x7fe60c0ff710 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.823+0000 7fe6111fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c1ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.823+0000 7fe6111fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c1ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58088/0 (socket says 172.21.15.64:58088) 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.823+0000 7fe6111fa640 1 -- 172.21.15.64:0/2672284383 learned_addr learned my addr 172.21.15.64:0/2672284383 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe6111fa640 1 -- 172.21.15.64:0/2672284383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c0ff710 msgr2=0x7fe60c10c370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe6111fa640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c0ff710 0x7fe60c10c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe6111fa640 1 -- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 msgr2=0x7fe60c10c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe6111fa640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 0x7fe60c10c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe6111fa640 1 -- 172.21.15.64:0/2672284383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c06b250 con 0x7fe60c1011e0 2024-05-09T09:19:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe6111fa640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c1ae0b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00eaa0 tx=0x7fe5fc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.824+0000 7fe609ffb640 1 -- 172.21.15.64:0/2672284383 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00cdb0 con 0x7fe60c1011e0 2024-05-09T09:19:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.825+0000 7fe609ffb640 1 -- 172.21.15.64:0/2672284383 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5fc005020 con 0x7fe60c1011e0 2024-05-09T09:19:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.825+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c06b540 con 0x7fe60c1011e0 2024-05-09T09:19:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.825+0000 7fe609ffb640 1 -- 172.21.15.64:0/2672284383 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0116d0 con 0x7fe60c1011e0 2024-05-09T09:19:12.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.826+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c06ba30 con 0x7fe60c1011e0 2024-05-09T09:19:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.829+0000 7fe61596b740 1 Processor -- start 2024-05-09T09:19:12.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.833+0000 7fe61596b740 1 -- start start 2024-05-09T09:19:12.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe61596b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe61596b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 0x563963a37430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe61596b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe61596b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563963996c40 con 0x563963a32930 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe61596b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56396393e580 con 0x563963a31f40 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe61596b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563963992c00 con 0x563963a37aa0 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe6109f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 0x563963a37430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe6109f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 0x563963a37430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44522/0 (socket says 172.21.15.64:44522) 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.834+0000 7fe6109f9640 1 -- 172.21.15.64:0/2024975736 learned_addr learned my addr 172.21.15.64:0/2024975736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.835+0000 7fe6109f9640 1 -- 172.21.15.64:0/2024975736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 msgr2=0x563963a39f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.835+0000 7fe6109f9640 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a39f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.835+0000 7fe6109f9640 1 -- 172.21.15.64:0/2024975736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 msgr2=0x563963a32360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.835+0000 7fe6109f9640 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a32360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.835+0000 7fe6109f9640 1 -- 172.21.15.64:0/2024975736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639636aa3f0 con 0x563963a32930 2024-05-09T09:19:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.836+0000 7fe609ffb640 1 -- 172.21.15.64:0/2672284383 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe5fc0051d0 con 0x7fe60c1011e0 2024-05-09T09:19:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.838+0000 7fe609ffb640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5f80a2ce0 0x7fe5f80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.838+0000 7fe6109f9640 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 0x563963a37430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe60400a980 tx=0x7fe60400ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbff18208118fca2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.838+0000 7fe609ffb640 1 -- 172.21.15.64:0/2672284383 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5fc0cf710 con 0x7fe60c1011e0 2024-05-09T09:19:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.839+0000 7fe602ffd640 1 -- 172.21.15.64:0/2024975736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60400cd80 con 0x563963a32930 2024-05-09T09:19:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.839+0000 7fe602ffd640 1 -- 172.21.15.64:0/2024975736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe60400f900 con 0x563963a32930 2024-05-09T09:19:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.839+0000 7fe60bfff640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5f80a2ce0 0x7fe5f80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 -- 172.21.15.64:0/2024975736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 msgr2=0x563963a37430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 0x563963a37430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe60400a980 tx=0x7fe60400ae40 comp rx=0 tx=0).stop 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 -- 172.21.15.64:0/2024975736 shutdown_connections 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a32930 0x563963a37430 secure :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe60400a980 tx=0x7fe60400ae40 comp rx=0 tx=0).stop 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 --2- 172.21.15.64:0/2024975736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a32360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.840+0000 7fe61596b740 1 -- 172.21.15.64:0/2024975736 >> 172.21.15.64:0/2024975736 conn(0x563963a2d5b0 msgr2=0x563963a2d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.841+0000 7fe61596b740 1 -- 172.21.15.64:0/2024975736 shutdown_connections 2024-05-09T09:19:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.841+0000 7fe61596b740 1 -- 172.21.15.64:0/2024975736 wait complete. 2024-05-09T09:19:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.841+0000 7fe61596b740 1 -- start start 2024-05-09T09:19:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 Processor -- start 2024-05-09T09:19:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe60bfff640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5f80a2ce0 0x7fe5f80a51a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009230 tx=0x7fe5f40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a31f40 0x563963a3c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a3caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563963a3aaa0 con 0x563963a31f40 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563963951c40 con 0x563963a41ad0 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563963a3ac20 con 0x563963a37aa0 2024-05-09T09:19:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.842+0000 7fe61596b740 10 client.0 _ll_register_callbacks cb 0x5639636a9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.843+0000 7fe60bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a3caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.843+0000 7fe60bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a3caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55528/0 (socket says 172.21.15.64:55528) 2024-05-09T09:19:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.843+0000 7fe60bfff640 1 -- 172.21.15.64:0/1511582696 learned_addr learned my addr 172.21.15.64:0/1511582696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.843+0000 7fe6109f9640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a31f40 0x563963a3c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.846+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.846+0000 7fe6111fa640 1 -- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 msgr2=0x563963a3caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.846+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a3caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.846+0000 7fe6111fa640 1 -- 172.21.15.64:0/1511582696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a31f40 msgr2=0x563963a3c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.846+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a31f40 0x563963a3c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.846+0000 7fe6111fa640 1 -- 172.21.15.64:0/1511582696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639637747d0 con 0x563963a41ad0 2024-05-09T09:19:12.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.847+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a43ec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00efd0 tx=0x7fe5fc09bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58ba9853221bca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.848+0000 7fe5ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:12.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.848+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1511582696 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc01f070 con 0x563963a41ad0 2024-05-09T09:19:12.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.848+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1511582696 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5fc0981f0 con 0x563963a41ad0 2024-05-09T09:19:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.848+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60400f900 con 0x563963a41ad0 2024-05-09T09:19:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.849+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1511582696 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe5fc0b9050 con 0x563963a41ad0 2024-05-09T09:19:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.849+0000 7fe61596b740 10 client.27464 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.849+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563963a472a0 con 0x563963a41ad0 2024-05-09T09:19:12.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.849+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1511582696 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe60400f900 con 0x563963a41ad0 2024-05-09T09:19:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1511582696 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fe5fc00f040 con 0x563963a41ad0 2024-05-09T09:19:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe61596b740 10 client.27464 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe61596b740 10 client.27464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:19:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe61596b740 20 client.27464 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe61596b740 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963a48da0 0x563963a4b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe61596b740 4 client.27464 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.850+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639639a4370 con 0x563963a48da0 2024-05-09T09:19:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.851+0000 7fe6109f9640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963a48da0 0x563963a4b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.852+0000 7fe5ebfff640 10 client.27464 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.852+0000 7fe6109f9640 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963a48da0 0x563963a4b240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bfb313c20ce8f58 server_cookie=5c71a2b15f1b413b in_seq=0 out_seq=0 2024-05-09T09:19:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.853+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1511582696 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639639a4370 con 0x563963a48da0 2024-05-09T09:19:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.853+0000 7fe5ebfff640 10 client.27464 handle_command_reply: tid=0 2024-05-09T09:19:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.853+0000 7fe61596b740 1 client.27464 shutdown 2024-05-09T09:19:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.855+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 msgr2=0x563963a43ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.855+0000 7fe61596b740 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a43ec0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00efd0 tx=0x7fe5fc09bcc0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 shutdown_connections 2024-05-09T09:19:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe61596b740 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a43ec0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe61596b740 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a37aa0 0x563963a3caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe61596b740 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963a48da0 0x563963a4b240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe61596b740 1 --2- 172.21.15.64:0/1511582696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a31f40 0x563963a3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 >> 172.21.15.64:0/1511582696 conn(0x563963a2d5b0 msgr2=0x563963a2c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.856+0000 7fe5ebfff640 0 client.27464 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.857+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 shutdown_connections 2024-05-09T09:19:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.858+0000 7fe61596b740 1 -- 172.21.15.64:0/1511582696 wait complete. 2024-05-09T09:19:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.859+0000 7fe61596b740 20 client.27464 trim_cache size 0 max 16384 2024-05-09T09:19:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.885+0000 7fe61596b740 1 Processor -- start 2024-05-09T09:19:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.885+0000 7fe61596b740 1 -- start start 2024-05-09T09:19:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.885+0000 7fe61596b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a28780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.885+0000 7fe61596b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a3b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe6109f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a28780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe6109f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a28780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58114/0 (socket says 172.21.15.64:58114) 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe6109f9640 1 -- 172.21.15.64:0/4018705104 learned_addr learned my addr 172.21.15.64:0/4018705104 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe61596b740 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a41ad0 0x563963a2d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1c44d0 con 0x563963a37aa0 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1c1da0 con 0x563963a31f40 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe6111fa640 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a41ad0 0x563963a2d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.886+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c073740 con 0x563963a41ad0 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe60bfff640 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a3b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe6109f9640 1 -- 172.21.15.64:0/4018705104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a41ad0 msgr2=0x563963a2d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe6109f9640 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a41ad0 0x563963a2d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe6109f9640 1 -- 172.21.15.64:0/4018705104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 msgr2=0x563963a3b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe6109f9640 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a3b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe6109f9640 1 -- 172.21.15.64:0/4018705104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563963a28cc0 con 0x563963a31f40 2024-05-09T09:19:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.887+0000 7fe6109f9640 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a28780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe60400a950 tx=0x7fe60400a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7485d62d2b4aafc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.888+0000 7fe6027fc640 1 -- 172.21.15.64:0/4018705104 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604004020 con 0x563963a31f40 2024-05-09T09:19:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.888+0000 7fe6027fc640 1 -- 172.21.15.64:0/4018705104 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe60402a030 con 0x563963a31f40 2024-05-09T09:19:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.888+0000 7fe6027fc640 1 -- 172.21.15.64:0/4018705104 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6040052a0 con 0x563963a31f40 2024-05-09T09:19:12.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 msgr2=0x563963a28780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a28780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe60400a950 tx=0x7fe60400a630 comp rx=0 tx=0).stop 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 shutdown_connections 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a41ad0 0x563963a2d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a3b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 --2- 172.21.15.64:0/4018705104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a31f40 0x563963a28780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 >> 172.21.15.64:0/4018705104 conn(0x563963a2d5b0 msgr2=0x563963a333d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 shutdown_connections 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.889+0000 7fe61596b740 1 -- 172.21.15.64:0/4018705104 wait complete. 2024-05-09T09:19:12.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 -- start start 2024-05-09T09:19:12.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 Processor -- start 2024-05-09T09:19:12.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a31f40 0x563963a5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a5a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a5b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1c0260 con 0x563963a37aa0 2024-05-09T09:19:12.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1c16d0 con 0x563963a41ad0 2024-05-09T09:19:12.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1c0930 con 0x563963a31f40 2024-05-09T09:19:12.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.890+0000 7fe61596b740 10 client.0 _ll_register_callbacks cb 0x7fe5fc09bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:12.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.891+0000 7fe6109f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a31f40 0x563963a5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.891+0000 7fe6111fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a5b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.891+0000 7fe6109f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a31f40 0x563963a5a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55540/0 (socket says 172.21.15.64:55540) 2024-05-09T09:19:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.891+0000 7fe6109f9640 1 -- 172.21.15.64:0/1868100188 learned_addr learned my addr 172.21.15.64:0/1868100188 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.891+0000 7fe60bfff640 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a5a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.892+0000 7fe6111fa640 1 -- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a31f40 msgr2=0x563963a5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.892+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a31f40 0x563963a5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.892+0000 7fe6111fa640 1 -- 172.21.15.64:0/1868100188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 msgr2=0x563963a5a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.892+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a5a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.892+0000 7fe6111fa640 1 -- 172.21.15.64:0/1868100188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563963a5bbd0 con 0x563963a41ad0 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.893+0000 7fe6111fa640 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a5b5e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009270 tx=0x7fe5fc0ba470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0fe38afe0ab546 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.893+0000 7fe5ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.894+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1868100188 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc01f070 con 0x563963a41ad0 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.894+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1868100188 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5fc0979a0 con 0x563963a41ad0 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.894+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60402a030 con 0x563963a41ad0 2024-05-09T09:19:12.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.895+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1868100188 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe5fc097ba0 con 0x563963a41ad0 2024-05-09T09:19:12.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.895+0000 7fe61596b740 10 client.27470 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:12.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.895+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563963aaf1d0 con 0x563963a41ad0 2024-05-09T09:19:12.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1868100188 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe60402a030 con 0x563963a41ad0 2024-05-09T09:19:12.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1868100188 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fe5fc0a10c0 con 0x563963a41ad0 2024-05-09T09:19:12.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe61596b740 10 client.27470 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:12.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe61596b740 10 client.27470 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:19:12.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe61596b740 20 client.27470 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:12.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe61596b740 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963aaf960 0x563963ab1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:12.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe61596b740 4 client.27470 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:19:12.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.896+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56396399e6c0 con 0x563963aaf960 2024-05-09T09:19:12.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.897+0000 7fe6109f9640 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963aaf960 0x563963ab1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:12.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.903+0000 7fe6109f9640 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963aaf960 0x563963ab1d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adea1b0e1186bba1 server_cookie=19cde579d1f21f63 in_seq=0 out_seq=0 2024-05-09T09:19:12.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.903+0000 7fe5ebfff640 10 client.27470 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:12.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.904+0000 7fe5ebfff640 1 -- 172.21.15.64:0/1868100188 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56396399e6c0 con 0x563963aaf960 2024-05-09T09:19:12.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.904+0000 7fe5ebfff640 10 client.27470 handle_command_reply: tid=0 2024-05-09T09:19:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.904+0000 7fe61596b740 1 client.27470 shutdown 2024-05-09T09:19:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.904+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 msgr2=0x563963a5b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.904+0000 7fe61596b740 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a5b5e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009270 tx=0x7fe5fc0ba470 comp rx=0 tx=0).stop 2024-05-09T09:19:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 shutdown_connections 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563963a41ad0 0x563963a5b5e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x563963aaf960 0x563963ab1d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563963a37aa0 0x563963a5a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 --2- 172.21.15.64:0/1868100188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563963a31f40 0x563963a5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 >> 172.21.15.64:0/1868100188 conn(0x563963a2d5b0 msgr2=0x563963a37330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe5ebfff640 0 client.27470 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 shutdown_connections 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.905+0000 7fe61596b740 1 -- 172.21.15.64:0/1868100188 wait complete. 2024-05-09T09:19:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.906+0000 7fe61596b740 20 client.27470 trim_cache size 0 max 16384 2024-05-09T09:19:12.906 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:19:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.909+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5f80a2ce0 msgr2=0x7fe5f80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.909+0000 7fe612c5d640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5f80a2ce0 0x7fe5f80a51a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009230 tx=0x7fe5f40314a0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.909+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 msgr2=0x7fe60c1ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.909+0000 7fe612c5d640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c1ae0b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00eaa0 tx=0x7fe5fc00ef60 comp rx=0 tx=0).stop 2024-05-09T09:19:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.910+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 shutdown_connections 2024-05-09T09:19:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.910+0000 7fe612c5d640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5f80a2ce0 0x7fe5f80a51a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.910+0000 7fe612c5d640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c1011e0 0x7fe60c1ae0b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.910+0000 7fe612c5d640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe60c100840 0x7fe60c10c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.910+0000 7fe612c5d640 1 --2- 172.21.15.64:0/2672284383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c0ff710 0x7fe60c10c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.910+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 >> 172.21.15.64:0/2672284383 conn(0x7fe60c0fb030 msgr2=0x7fe60c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:12.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.911+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 shutdown_connections 2024-05-09T09:19:12.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:12.911+0000 7fe612c5d640 1 -- 172.21.15.64:0/2672284383 wait complete. 2024-05-09T09:19:12.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:12 smithi064 ceph-mon[30149]: pgmap v672: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.0 MiB/s wr, 755 op/s 2024-05-09T09:19:12.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:12 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2227648485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:19:12.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:19:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:12 smithi179 ceph-mon[34086]: pgmap v672: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.0 MiB/s wr, 755 op/s 2024-05-09T09:19:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:12 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2227648485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:19:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:12 smithi139 ceph-mon[33599]: pgmap v672: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.0 MiB/s wr, 755 op/s 2024-05-09T09:19:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:12 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2227648485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:19:13.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.134+0000 7f2623fff640 1 -- 172.21.15.64:0/3162203061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2624108a10 msgr2=0x7f262410ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.134+0000 7f2623fff640 1 --2- 172.21.15.64:0/3162203061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2624108a10 0x7f262410ae00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2614009900 tx=0x7f261402a440 comp rx=0 tx=0).stop 2024-05-09T09:19:13.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.136+0000 7f2623fff640 1 -- 172.21.15.64:0/3162203061 shutdown_connections 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.136+0000 7f2623fff640 1 --2- 172.21.15.64:0/3162203061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 0x7f26241102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.136+0000 7f2623fff640 1 --2- 172.21.15.64:0/3162203061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f262410b340 0x7f262410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.136+0000 7f2623fff640 1 --2- 172.21.15.64:0/3162203061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2624108a10 0x7f262410ae00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.136+0000 7f2623fff640 1 -- 172.21.15.64:0/3162203061 >> 172.21.15.64:0/3162203061 conn(0x7f262406e930 msgr2=0x7f2624070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.137+0000 7f2623fff640 1 -- 172.21.15.64:0/3162203061 shutdown_connections 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.137+0000 7f2623fff640 1 -- 172.21.15.64:0/3162203061 wait complete. 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.138+0000 7f2623fff640 1 Processor -- start 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.138+0000 7f2623fff640 1 -- start start 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2623fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2624108a10 0x7f262419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2623fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262410b340 0x7f262419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2623fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 0x7f26241a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2623fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2624113d00 con 0x7f262410b340 2024-05-09T09:19:13.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2623fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2624113b80 con 0x7f262410dde0 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2623fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2624113e80 con 0x7f2624108a10 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.139+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2624108a10 0x7f262419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.140+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2624108a10 0x7f262419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55566/0 (socket says 172.21.15.64:55566) 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.140+0000 7f2622ffd640 1 -- 172.21.15.64:0/899678570 learned_addr learned my addr 172.21.15.64:0/899678570 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.140+0000 7f261bfff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262410b340 0x7f262419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.144+0000 7f261bfff640 1 -- 172.21.15.64:0/899678570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2624108a10 msgr2=0x7f262419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.144+0000 7f261bfff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2624108a10 0x7f262419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.144+0000 7f261bfff640 1 -- 172.21.15.64:0/899678570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 msgr2=0x7f26241a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.140+0000 7f26237fe640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 0x7f26241a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.145+0000 7f261bfff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 0x7f26241a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.145+0000 7f261bfff640 1 -- 172.21.15.64:0/899678570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26241a5b60 con 0x7f262410b340 2024-05-09T09:19:13.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.145+0000 7f26237fe640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 0x7f26241a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:19:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.146+0000 7f261bfff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262410b340 0x7f262419e480 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c940 tx=0x7f260c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:13.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.147+0000 7f2620ff9640 1 -- 172.21.15.64:0/899678570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c007bb0 con 0x7f262410b340 2024-05-09T09:19:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.147+0000 7f2620ff9640 1 -- 172.21.15.64:0/899678570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c007d50 con 0x7f262410b340 2024-05-09T09:19:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.147+0000 7f2620ff9640 1 -- 172.21.15.64:0/899678570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00e6e0 con 0x7f262410b340 2024-05-09T09:19:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.148+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26241a5e50 con 0x7f262410b340 2024-05-09T09:19:13.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.148+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241a6350 con 0x7f262410b340 2024-05-09T09:19:13.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.152+0000 7f262c41e740 1 Processor -- start 2024-05-09T09:19:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.153+0000 7f262c41e740 1 -- start start 2024-05-09T09:19:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.154+0000 7f262c41e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655989550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.154+0000 7f262c41e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x55865598e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.154+0000 7f262c41e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655991120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.154+0000 7f262c41e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5586558edc40 con 0x558655989130 2024-05-09T09:19:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.154+0000 7f262c41e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558655895580 con 0x558655989b20 2024-05-09T09:19:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.154+0000 7f262c41e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5586558e9c00 con 0x55865598ec90 2024-05-09T09:19:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.155+0000 7f26237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655989550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.155+0000 7f26237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655989550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44596/0 (socket says 172.21.15.64:44596) 2024-05-09T09:19:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.155+0000 7f26237fe640 1 -- 172.21.15.64:0/489130943 learned_addr learned my addr 172.21.15.64:0/489130943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:13.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.155+0000 7f261bfff640 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655991120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.155+0000 7f2622ffd640 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x55865598e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f261bfff640 1 -- 172.21.15.64:0/489130943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 msgr2=0x55865598e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:13.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f261bfff640 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x55865598e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f261bfff640 1 -- 172.21.15.64:0/489130943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 msgr2=0x558655989550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:13.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f261bfff640 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655989550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f261bfff640 1 -- 172.21.15.64:0/489130943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586556013f0 con 0x55865598ec90 2024-05-09T09:19:13.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f2620ff9640 1 -- 172.21.15.64:0/899678570 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f260c00f450 con 0x7f262410b340 2024-05-09T09:19:13.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.156+0000 7f261bfff640 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655991120 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f260c0ccef0 tx=0x7f260c0d5ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b5cad78758f7e1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:13.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.157+0000 7f26197fa640 1 -- 172.21.15.64:0/489130943 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c0de070 con 0x55865598ec90 2024-05-09T09:19:13.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.158+0000 7f26197fa640 1 -- 172.21.15.64:0/489130943 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c0d7bd0 con 0x55865598ec90 2024-05-09T09:19:13.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.158+0000 7f2620ff9640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26100a2d90 0x7f26100a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.158+0000 7f26197fa640 1 -- 172.21.15.64:0/489130943 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c0da680 con 0x55865598ec90 2024-05-09T09:19:13.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.158+0000 7f2620ff9640 1 -- 172.21.15.64:0/899678570 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f260c0cf840 con 0x7f262410b340 2024-05-09T09:19:13.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 -- 172.21.15.64:0/489130943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 msgr2=0x558655991120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655991120 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f260c0ccef0 tx=0x7f260c0d5ef0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f26237fe640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26100a2d90 0x7f26100a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 -- 172.21.15.64:0/489130943 shutdown_connections 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655991120 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x55865598e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 --2- 172.21.15.64:0/489130943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655989550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.159+0000 7f262c41e740 1 -- 172.21.15.64:0/489130943 >> 172.21.15.64:0/489130943 conn(0x5586559847a0 msgr2=0x558655984ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:13.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.160+0000 7f26237fe640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26100a2d90 0x7f26100a5250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55865598a380 tx=0x7f261c0086e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:13.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.160+0000 7f262c41e740 1 -- 172.21.15.64:0/489130943 shutdown_connections 2024-05-09T09:19:13.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.160+0000 7f262c41e740 1 -- 172.21.15.64:0/489130943 wait complete. 2024-05-09T09:19:13.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 -- start start 2024-05-09T09:19:13.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 Processor -- start 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655993590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x558655993ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x55865599ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558655991c90 con 0x558655989130 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5586558a8c40 con 0x55865598ec90 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558655991e10 con 0x558655989b20 2024-05-09T09:19:13.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.161+0000 7f262c41e740 10 client.0 _ll_register_callbacks cb 0x558655600be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.162+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655993590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.162+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655993590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44610/0 (socket says 172.21.15.64:44610) 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.162+0000 7f2622ffd640 1 -- 172.21.15.64:0/2511234608 learned_addr learned my addr 172.21.15.64:0/2511234608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.163+0000 7f2622ffd640 1 -- 172.21.15.64:0/2511234608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 msgr2=0x558655993ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.163+0000 7f2622ffd640 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x558655993ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.163+0000 7f2622ffd640 1 -- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 msgr2=0x55865599ab10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.163+0000 7f2622ffd640 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x55865599ab10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.163+0000 7f2622ffd640 1 -- 172.21.15.64:0/2511234608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586556cb7d0 con 0x558655989130 2024-05-09T09:19:13.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.163+0000 7f2622ffd640 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655993590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f261402aa30 tx=0x7f2614004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e52309c12e0b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:13.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.164+0000 7f260a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:13.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.164+0000 7f260a7fc640 1 -- 172.21.15.64:0/2511234608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614035070 con 0x558655989130 2024-05-09T09:19:13.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.165+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260c0d7bd0 con 0x558655989130 2024-05-09T09:19:13.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.165+0000 7f260a7fc640 1 -- 172.21.15.64:0/2511234608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2614003a80 con 0x558655989130 2024-05-09T09:19:13.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.165+0000 7f260a7fc640 1 -- 172.21.15.64:0/2511234608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f261404e050 con 0x558655989130 2024-05-09T09:19:13.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.165+0000 7f262c41e740 10 client.17973 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:13.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.165+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55865599df40 con 0x558655989130 2024-05-09T09:19:13.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f260a7fc640 1 -- 172.21.15.64:0/2511234608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c0d7bd0 con 0x558655989130 2024-05-09T09:19:13.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f260a7fc640 1 -- 172.21.15.64:0/2511234608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f26140484e0 con 0x558655989130 2024-05-09T09:19:13.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f262c41e740 10 client.17973 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:13.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f262c41e740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:13.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f262c41e740 20 client.17973 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:13.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f262c41e740 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55865599f680 0x5586559a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f262c41e740 4 client.17973 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:13.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586558fb370 con 0x55865599f680 2024-05-09T09:19:13.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.167+0000 7f261bfff640 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55865599f680 0x5586559a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.169+0000 7f261bfff640 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55865599f680 0x5586559a1b10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8bae91f8b5652e0 server_cookie=9390b8aa365703b2 in_seq=0 out_seq=0 2024-05-09T09:19:13.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.170+0000 7f260a7fc640 10 client.17973 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:13.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.170+0000 7f260a7fc640 1 -- 172.21.15.64:0/2511234608 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586558fb370 con 0x55865599f680 2024-05-09T09:19:13.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.170+0000 7f260a7fc640 10 client.17973 handle_command_reply: tid=0 2024-05-09T09:19:13.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.170+0000 7f262c41e740 1 client.17973 shutdown 2024-05-09T09:19:13.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.173+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 msgr2=0x558655993590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.173+0000 7f262c41e740 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655993590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f261402aa30 tx=0x7f2614004580 comp rx=0 tx=0).stop 2024-05-09T09:19:13.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 shutdown_connections 2024-05-09T09:19:13.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f262c41e740 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55865599f680 0x5586559a1b10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f262c41e740 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x55865599ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f262c41e740 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x558655993ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f262c41e740 1 --2- 172.21.15.64:0/2511234608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655993590 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 >> 172.21.15.64:0/2511234608 conn(0x5586559847a0 msgr2=0x5586559839c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:13.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.174+0000 7f260a7fc640 0 client.17973 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:13.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.176+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 shutdown_connections 2024-05-09T09:19:13.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.176+0000 7f262c41e740 1 -- 172.21.15.64:0/2511234608 wait complete. 2024-05-09T09:19:13.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.177+0000 7f262c41e740 20 client.17973 trim_cache size 0 max 16384 2024-05-09T09:19:13.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.204+0000 7f262c41e740 1 Processor -- start 2024-05-09T09:19:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.204+0000 7f262c41e740 1 -- start start 2024-05-09T09:19:13.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.205+0000 7f262c41e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655987ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.205+0000 7f262c41e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x55865599aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.205+0000 7f262c41e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x5586559a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.205+0000 7f262c41e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26241b3140 con 0x558655989130 2024-05-09T09:19:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f2622ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655987ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f261bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x55865599aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x5586559a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x5586559a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58202/0 (socket says 172.21.15.64:58202) 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 -- 172.21.15.64:0/1661077362 learned_addr learned my addr 172.21.15.64:0/1661077362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26241b0a10 con 0x55865598ec90 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f26241ae8c0 con 0x558655989b20 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 -- 172.21.15.64:0/1661077362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 msgr2=0x55865599aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x55865599aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 -- 172.21.15.64:0/1661077362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 msgr2=0x558655987ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655987ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.206+0000 7f26237fe640 1 -- 172.21.15.64:0/1661077362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558655989f00 con 0x55865598ec90 2024-05-09T09:19:13.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.207+0000 7f26237fe640 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x5586559a63d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f261c008fd0 tx=0x7f261c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182c120809f7a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:13.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.207+0000 7f261bfff640 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x55865599aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:19:13.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.207+0000 7f2618ff9640 1 -- 172.21.15.64:0/1661077362 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c028070 con 0x55865598ec90 2024-05-09T09:19:13.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.207+0000 7f2618ff9640 1 -- 172.21.15.64:0/1661077362 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c019c00 con 0x55865598ec90 2024-05-09T09:19:13.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.207+0000 7f2618ff9640 1 -- 172.21.15.64:0/1661077362 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c023d50 con 0x55865598ec90 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 msgr2=0x5586559a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x5586559a63d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f261c008fd0 tx=0x7f261c013000 comp rx=0 tx=0).stop 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 shutdown_connections 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55865598ec90 0x5586559a63d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558655989b20 0x55865599aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 --2- 172.21.15.64:0/1661077362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x558655987ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.209+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 >> 172.21.15.64:0/1661077362 conn(0x5586559847a0 msgr2=0x5586559877b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:13.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.210+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 shutdown_connections 2024-05-09T09:19:13.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.211+0000 7f262c41e740 1 -- 172.21.15.64:0/1661077362 wait complete. 2024-05-09T09:19:13.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.212+0000 7f262c41e740 1 -- start start 2024-05-09T09:19:13.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.212+0000 7f262c41e740 1 Processor -- start 2024-05-09T09:19:13.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x55865599aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x558655997560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655997b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26241aef10 con 0x558655989130 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26241b0340 con 0x558655989b20 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f26241af5a0 con 0x55865598ec90 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.213+0000 7f262c41e740 10 client.0 _ll_register_callbacks cb 0x7f2614005a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f261bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x558655997560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655997b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655997b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55618/0 (socket says 172.21.15.64:55618) 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 -- 172.21.15.64:0/81754441 learned_addr learned my addr 172.21.15.64:0/81754441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f2622ffd640 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x55865599aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 -- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 msgr2=0x558655997560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x558655997560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 -- 172.21.15.64:0/81754441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 msgr2=0x55865599aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x55865599aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.214+0000 7f26237fe640 1 -- 172.21.15.64:0/81754441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558655a03cf0 con 0x55865598ec90 2024-05-09T09:19:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f26237fe640 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655997b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f261c026040 tx=0x7f261c023620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94742f725de0b91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f260a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f260a7fc640 1 -- 172.21.15.64:0/81754441 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c028040 con 0x55865598ec90 2024-05-09T09:19:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261c019c00 con 0x55865598ec90 2024-05-09T09:19:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f260a7fc640 1 -- 172.21.15.64:0/81754441 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f261c023ad0 con 0x55865598ec90 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f260a7fc640 1 -- 172.21.15.64:0/81754441 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f261c03a050 con 0x55865598ec90 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f262c41e740 10 client.27619 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.215+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558655a04f60 con 0x55865598ec90 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f260a7fc640 1 -- 172.21.15.64:0/81754441 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c031070 con 0x55865598ec90 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f260a7fc640 1 -- 172.21.15.64:0/81754441 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f261c02b070 con 0x55865598ec90 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f262c41e740 10 client.27619 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f262c41e740 10 client.27619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f262c41e740 20 client.27619 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f262c41e740 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558655a057e0 0x558655a07c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f262c41e740 4 client.27619 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:19:13.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f261c019c00 con 0x558655a057e0 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.216+0000 7f2622ffd640 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558655a057e0 0x558655a07c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.217+0000 7f2622ffd640 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558655a057e0 0x558655a07c00 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f012dc4f8c3425df server_cookie=5a3be2d6ba68917 in_seq=0 out_seq=0 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.217+0000 7f260a7fc640 10 client.27619 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.217+0000 7f260a7fc640 1 -- 172.21.15.64:0/81754441 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f261c019c00 con 0x558655a057e0 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.217+0000 7f260a7fc640 10 client.27619 handle_command_reply: tid=0 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 client.27619 shutdown 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 msgr2=0x558655997b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655997b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f261c026040 tx=0x7f261c023620 comp rx=0 tx=0).stop 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 shutdown_connections 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558655a057e0 0x558655a07c00 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55865598ec90 0x558655997b90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558655989b20 0x558655997560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f260a7fc640 0 client.27619 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.218+0000 7f262c41e740 1 --2- 172.21.15.64:0/81754441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558655989130 0x55865599aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.219+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 >> 172.21.15.64:0/81754441 conn(0x5586559847a0 msgr2=0x558655988040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.219+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 shutdown_connections 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.219+0000 7f262c41e740 1 -- 172.21.15.64:0/81754441 wait complete. 2024-05-09T09:19:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.220+0000 7f262c41e740 20 client.27619 trim_cache size 0 max 16384 2024-05-09T09:19:13.222 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.222+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26100a2d90 msgr2=0x7f26100a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.222+0000 7f2623fff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26100a2d90 0x7f26100a5250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55865598a380 tx=0x7f261c0086e0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.222+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262410b340 msgr2=0x7f262419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.222+0000 7f2623fff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262410b340 0x7f262419e480 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c940 tx=0x7f260c00ce00 comp rx=0 tx=0).stop 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 shutdown_connections 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26100a2d90 0x7f26100a5250 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f262410dde0 0x7f26241a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262410b340 0x7f262419e480 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 --2- 172.21.15.64:0/899678570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2624108a10 0x7f262419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 >> 172.21.15.64:0/899678570 conn(0x7f262406e930 msgr2=0x7f262410e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 shutdown_connections 2024-05-09T09:19:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:13.223+0000 7f2623fff640 1 -- 172.21.15.64:0/899678570 wait complete. 2024-05-09T09:19:14.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:19:14.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:19:14.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:19:14.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.453+0000 7fb03ffff640 1 -- 172.21.15.64:0/1717306985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809c360 msgr2=0x7fb03809c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.453+0000 7fb03ffff640 1 --2- 172.21.15.64:0/1717306985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809c360 0x7fb03809c780 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb030009860 tx=0x7fb03002a360 comp rx=0 tx=0).stop 2024-05-09T09:19:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.454+0000 7fb03ffff640 1 -- 172.21.15.64:0/1717306985 shutdown_connections 2024-05-09T09:19:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.454+0000 7fb03ffff640 1 --2- 172.21.15.64:0/1717306985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809df10 0x7fb0380a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.454+0000 7fb03ffff640 1 --2- 172.21.15.64:0/1717306985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb03809d550 0x7fb03809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.454+0000 7fb03ffff640 1 --2- 172.21.15.64:0/1717306985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809c360 0x7fb03809c780 secure :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb030009860 tx=0x7fb03002a360 comp rx=0 tx=0).stop 2024-05-09T09:19:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.455+0000 7fb03ffff640 1 -- 172.21.15.64:0/1717306985 >> 172.21.15.64:0/1717306985 conn(0x7fb03800b300 msgr2=0x7fb03800b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.456+0000 7fb03ffff640 1 -- 172.21.15.64:0/1717306985 shutdown_connections 2024-05-09T09:19:14.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.457+0000 7fb03ffff640 1 -- 172.21.15.64:0/1717306985 wait complete. 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.458+0000 7fb03ffff640 1 Processor -- start 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.458+0000 7fb03ffff640 1 -- start start 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 0x7fb0381437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809df10 0x7fb038143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 0x7fb0381437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 0x7fb0381437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58236/0 (socket says 172.21.15.64:58236) 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb038148d90 0x7fb03814b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03effd640 1 -- 172.21.15.64:0/3096435470 learned_addr learned my addr 172.21.15.64:0/3096435470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380a7ea0 con 0x7fb03809df10 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380a7d20 con 0x7fb03809d550 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03e7fc640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809df10 0x7fb038143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.459+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb038148d90 0x7fb03814b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.460+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380a8020 con 0x7fb038148d90 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.460+0000 7fb03effd640 1 -- 172.21.15.64:0/3096435470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb038148d90 msgr2=0x7fb03814b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.460+0000 7fb03effd640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb038148d90 0x7fb03814b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.460+0000 7fb03effd640 1 -- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809df10 msgr2=0x7fb038143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.460+0000 7fb03effd640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809df10 0x7fb038143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.460+0000 7fb03effd640 1 -- 172.21.15.64:0/3096435470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03814b7f0 con 0x7fb03809d550 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.461+0000 7fb03effd640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 0x7fb0381437f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb030035040 tx=0x7fb030004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.461+0000 7fb027fff640 1 -- 172.21.15.64:0/3096435470 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030042020 con 0x7fb03809d550 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.461+0000 7fb027fff640 1 -- 172.21.15.64:0/3096435470 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb030038040 con 0x7fb03809d550 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.461+0000 7fb027fff640 1 -- 172.21.15.64:0/3096435470 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030037050 con 0x7fb03809d550 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.461+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03814ba80 con 0x7fb03809d550 2024-05-09T09:19:14.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.461+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03814bef0 con 0x7fb03809d550 2024-05-09T09:19:14.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.465+0000 7fb027fff640 1 -- 172.21.15.64:0/3096435470 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb030005220 con 0x7fb03809d550 2024-05-09T09:19:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.465+0000 7fb048400740 1 Processor -- start 2024-05-09T09:19:14.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 -- start start 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baef6d0 0x55ca4baefaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf00c0 0x55ca4baf4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baf76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ba53990 con 0x55ca4baf00c0 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ca4b9fb5b0 con 0x55ca4baef6d0 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb048400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ba4f950 con 0x55ca4baf5230 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb03e7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baf76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb03e7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baf76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55632/0 (socket says 172.21.15.64:55632) 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb03e7fc640 1 -- 172.21.15.64:0/943909929 learned_addr learned my addr 172.21.15.64:0/943909929 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb03effd640 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf00c0 0x55ca4baf4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.466+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baef6d0 0x55ca4baefaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb03e7fc640 1 -- 172.21.15.64:0/943909929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baef6d0 msgr2=0x55ca4baefaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb03e7fc640 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baef6d0 0x55ca4baefaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb03e7fc640 1 -- 172.21.15.64:0/943909929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf00c0 msgr2=0x55ca4baf4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb03e7fc640 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf00c0 0x55ca4baf4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb03e7fc640 1 -- 172.21.15.64:0/943909929 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4b82f7e0 con 0x55ca4baf5230 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb027fff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0280a2c90 0x7fb0280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.467+0000 7fb03e7fc640 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baf76c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb04002d780 tx=0x7fb040073cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f17d82370b270e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.468+0000 7fb027fff640 1 -- 172.21.15.64:0/3096435470 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0300ef310 con 0x7fb03809d550 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.468+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0280a2c90 0x7fb0280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.468+0000 7fb0257fa640 1 -- 172.21.15.64:0/943909929 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040071980 con 0x55ca4baf5230 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.468+0000 7fb0257fa640 1 -- 172.21.15.64:0/943909929 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb040072440 con 0x55ca4baf5230 2024-05-09T09:19:14.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.468+0000 7fb0257fa640 1 -- 172.21.15.64:0/943909929 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400703f0 con 0x55ca4baf5230 2024-05-09T09:19:14.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.469+0000 7fb048400740 1 -- 172.21.15.64:0/943909929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 msgr2=0x55ca4baf76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.469+0000 7fb048400740 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baf76c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb04002d780 tx=0x7fb040073cf0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0280a2c90 0x7fb0280a5150 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55ca4baf0920 tx=0x7fb034009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 -- 172.21.15.64:0/943909929 shutdown_connections 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baf76c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf00c0 0x55ca4baf4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 --2- 172.21.15.64:0/943909929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baef6d0 0x55ca4baefaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 -- 172.21.15.64:0/943909929 >> 172.21.15.64:0/943909929 conn(0x55ca4bab70a0 msgr2=0x55ca4baeb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 -- 172.21.15.64:0/943909929 shutdown_connections 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.470+0000 7fb048400740 1 -- 172.21.15.64:0/943909929 wait complete. 2024-05-09T09:19:14.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.471+0000 7fb048400740 1 -- start start 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.471+0000 7fb048400740 1 Processor -- start 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baf9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf00c0 0x55ca4baf9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 0x55ca4bb00fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ca4baf81e0 con 0x55ca4baef6d0 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ba0eb40 con 0x55ca4baf5230 2024-05-09T09:19:14.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ca4baf8360 con 0x55ca4baf00c0 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb048400740 10 client.0 _ll_register_callbacks cb 0x55ca4ba030f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baf9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 0x55ca4bb00fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03e7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf00c0 0x55ca4baf9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baf9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44652/0 (socket says 172.21.15.64:44652) 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 0x55ca4bb00fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58252/0 (socket says 172.21.15.64:58252) 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03effd640 1 -- 172.21.15.64:0/2629997541 learned_addr learned my addr 172.21.15.64:0/2629997541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 -- 172.21.15.64:0/2629997541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf00c0 msgr2=0x55ca4baf9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf00c0 0x55ca4baf9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 -- 172.21.15.64:0/2629997541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 msgr2=0x55ca4baf9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baf9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.472+0000 7fb03f7fe640 1 -- 172.21.15.64:0/2629997541 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4b88ff70 con 0x55ca4baf5230 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 0x55ca4bb00fb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb03401e9d0 tx=0x7fb034004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e164c767ba8986 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb022ff5640 1 -- 172.21.15.64:0/2629997541 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03402e060 con 0x55ca4baf5230 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb040072440 con 0x55ca4baf5230 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb022ff5640 1 -- 172.21.15.64:0/2629997541 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb034018030 con 0x55ca4baf5230 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb022ff5640 1 -- 172.21.15.64:0/2629997541 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb034035050 con 0x55ca4baf5230 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.474+0000 7fb048400740 10 client.27491 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:14.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.475+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ca4b727170 con 0x55ca4baf5230 2024-05-09T09:19:14.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.475+0000 7fb022ff5640 1 -- 172.21.15.64:0/2629997541 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb040072440 con 0x55ca4baf5230 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.476+0000 7fb022ff5640 1 -- 172.21.15.64:0/2629997541 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fb03400f3c0 con 0x55ca4baf5230 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.476+0000 7fb048400740 10 client.27491 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.476+0000 7fb048400740 10 client.27491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.476+0000 7fb048400740 20 client.27491 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.476+0000 7fb048400740 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb05590 0x55ca4bb07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.476+0000 7fb048400740 4 client.27491 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.477+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4ba610c0 con 0x55ca4bb05590 2024-05-09T09:19:14.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.477+0000 7fb03effd640 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb05590 0x55ca4bb07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.478+0000 7fb03effd640 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb05590 0x55ca4bb07a30 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db55deaa57d2987 server_cookie=2b80357f2e7023b in_seq=0 out_seq=0 2024-05-09T09:19:14.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.478+0000 7fb022ff5640 10 client.27491 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb022ff5640 1 -- 172.21.15.64:0/2629997541 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca4ba610c0 con 0x55ca4bb05590 2024-05-09T09:19:14.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb022ff5640 10 client.27491 handle_command_reply: tid=0 2024-05-09T09:19:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 client.27491 shutdown 2024-05-09T09:19:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 msgr2=0x55ca4bb00fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 0x55ca4bb00fb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb03401e9d0 tx=0x7fb034004730 comp rx=0 tx=0).stop 2024-05-09T09:19:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 shutdown_connections 2024-05-09T09:19:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb05590 0x55ca4bb07a30 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf5230 0x55ca4bb00fb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf00c0 0x55ca4baf9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.479+0000 7fb048400740 1 --2- 172.21.15.64:0/2629997541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baf9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.480+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 >> 172.21.15.64:0/2629997541 conn(0x55ca4bab70a0 msgr2=0x55ca4baea290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.480+0000 7fb022ff5640 0 client.27491 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.480+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 shutdown_connections 2024-05-09T09:19:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.480+0000 7fb048400740 1 -- 172.21.15.64:0/2629997541 wait complete. 2024-05-09T09:19:14.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.480+0000 7fb048400740 20 client.27491 trim_cache size 0 max 16384 2024-05-09T09:19:14.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.514+0000 7fb048400740 1 Processor -- start 2024-05-09T09:19:14.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.514+0000 7fb048400740 1 -- start start 2024-05-09T09:19:14.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb048400740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb048400740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf00c0 0x55ca4bb00ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb048400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb048400740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb038158d50 con 0x55ca4baef6d0 2024-05-09T09:19:14.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb048400740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb038156620 con 0x55ca4baf00c0 2024-05-09T09:19:14.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb048400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb038154470 con 0x55ca4baf5230 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb03e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf00c0 0x55ca4bb00ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb03e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf00c0 0x55ca4bb00ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58268/0 (socket says 172.21.15.64:58268) 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb03e7fc640 1 -- 172.21.15.64:0/1267010189 learned_addr learned my addr 172.21.15.64:0/1267010189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.515+0000 7fb03effd640 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb03f7fe640 1 -- 172.21.15.64:0/1267010189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf00c0 msgr2=0x55ca4bb00ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf00c0 0x55ca4bb00ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb03f7fe640 1 -- 172.21.15.64:0/1267010189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 msgr2=0x55ca4baee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb03f7fe640 1 -- 172.21.15.64:0/1267010189 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4baf04a0 con 0x55ca4baf5230 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baee580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb03400f860 tx=0x7fb034012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569889981681ed5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb024ff9640 1 -- 172.21.15.64:0/1267010189 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03400fe20 con 0x55ca4baf5230 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb024ff9640 1 -- 172.21.15.64:0/1267010189 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb034009de0 con 0x55ca4baf5230 2024-05-09T09:19:14.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.516+0000 7fb024ff9640 1 -- 172.21.15.64:0/1267010189 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034005280 con 0x55ca4baf5230 2024-05-09T09:19:14.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.518+0000 7fb048400740 1 -- 172.21.15.64:0/1267010189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 msgr2=0x55ca4baee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.518+0000 7fb048400740 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baee580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb03400f860 tx=0x7fb034012d00 comp rx=0 tx=0).stop 2024-05-09T09:19:14.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.518+0000 7fb048400740 1 -- 172.21.15.64:0/1267010189 shutdown_connections 2024-05-09T09:19:14.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.518+0000 7fb048400740 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baf5230 0x55ca4baee580 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.518+0000 7fb048400740 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4baf00c0 0x55ca4bb00ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.518+0000 7fb048400740 1 --2- 172.21.15.64:0/1267010189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baef6d0 0x55ca4baee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.519+0000 7fb048400740 1 -- 172.21.15.64:0/1267010189 >> 172.21.15.64:0/1267010189 conn(0x55ca4bab70a0 msgr2=0x55ca4baf1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.520+0000 7fb048400740 1 -- 172.21.15.64:0/1267010189 shutdown_connections 2024-05-09T09:19:14.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.520+0000 7fb048400740 1 -- 172.21.15.64:0/1267010189 wait complete. 2024-05-09T09:19:14.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.521+0000 7fb048400740 1 -- start start 2024-05-09T09:19:14.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.521+0000 7fb048400740 1 Processor -- start 2024-05-09T09:19:14.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.522+0000 7fb048400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 0x55ca4bafecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.522+0000 7fb048400740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf5230 0x55ca4baff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.522+0000 7fb03effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 0x55ca4bafecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.522+0000 7fb03effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 0x55ca4bafecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55668/0 (socket says 172.21.15.64:55668) 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.522+0000 7fb03effd640 1 -- 172.21.15.64:0/2755837956 learned_addr learned my addr 172.21.15.64:0/2755837956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.522+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4bb001f0 0x55ca4bb00670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb038154ae0 con 0x55ca4baf5230 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb03f7fe640 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4bb001f0 0x55ca4bb00670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb038155f50 con 0x55ca4bb001f0 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381551b0 con 0x55ca4baef6d0 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb048400740 10 client.0 _ll_register_callbacks cb 0x7fb03401e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb03effd640 1 -- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4bb001f0 msgr2=0x55ca4bb00670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb03effd640 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4bb001f0 0x55ca4bb00670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb03effd640 1 -- 172.21.15.64:0/2755837956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf5230 msgr2=0x55ca4baff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb03effd640 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf5230 0x55ca4baff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.523+0000 7fb03effd640 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4bb00bb0 con 0x55ca4baef6d0 2024-05-09T09:19:14.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.524+0000 7fb03effd640 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 0x55ca4bafecf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb0300bb710 tx=0x7fb0300ec8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c418a9934dea9720 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.524+0000 7fb022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:14.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.524+0000 7fb022ff5640 1 -- 172.21.15.64:0/2755837956 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0300b7a20 con 0x55ca4baef6d0 2024-05-09T09:19:14.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.525+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb034009de0 con 0x55ca4baef6d0 2024-05-09T09:19:14.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.525+0000 7fb022ff5640 1 -- 172.21.15.64:0/2755837956 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb0300b9a80 con 0x55ca4baef6d0 2024-05-09T09:19:14.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb022ff5640 1 -- 172.21.15.64:0/2755837956 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb0300bb0e0 con 0x55ca4baef6d0 2024-05-09T09:19:14.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 10 client.27637 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:14.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ca4bb6c720 con 0x55ca4baef6d0 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb022ff5640 1 -- 172.21.15.64:0/2755837956 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb030005510 con 0x55ca4baef6d0 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb022ff5640 1 -- 172.21.15.64:0/2755837956 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fb0300b7040 con 0x55ca4baef6d0 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 10 client.27637 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 10 client.27637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 20 client.27637 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb6d080 0x55ca4bb6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.526+0000 7fb048400740 4 client.27637 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:19:14.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.527+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb034009de0 con 0x55ca4bb6d080 2024-05-09T09:19:14.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.528+0000 7fb03e7fc640 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb6d080 0x55ca4bb6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.528+0000 7fb03e7fc640 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb6d080 0x55ca4bb6f4a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647f28771e7846ba server_cookie=162fded9e8e0516d in_seq=0 out_seq=0 2024-05-09T09:19:14.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.529+0000 7fb022ff5640 10 client.27637 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.534+0000 7fb022ff5640 1 -- 172.21.15.64:0/2755837956 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb034009de0 con 0x55ca4bb6d080 2024-05-09T09:19:14.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.534+0000 7fb022ff5640 10 client.27637 handle_command_reply: tid=0 2024-05-09T09:19:14.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.535+0000 7fb048400740 1 client.27637 shutdown 2024-05-09T09:19:14.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.537+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 msgr2=0x55ca4bafecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.537+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 0x55ca4bafecf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb0300bb710 tx=0x7fb0300ec8f0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 shutdown_connections 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ca4bb6d080 0x55ca4bb6f4a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ca4bb001f0 0x55ca4bb00670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb022ff5640 0 client.27637 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca4baf5230 0x55ca4baff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb048400740 1 --2- 172.21.15.64:0/2755837956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ca4baef6d0 0x55ca4bafecf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.540+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 >> 172.21.15.64:0/2755837956 conn(0x55ca4bab70a0 msgr2=0x55ca4bae9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.546+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 shutdown_connections 2024-05-09T09:19:14.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.546+0000 7fb048400740 1 -- 172.21.15.64:0/2755837956 wait complete. 2024-05-09T09:19:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.548+0000 7fb048400740 20 client.27637 trim_cache size 0 max 16384 2024-05-09T09:19:14.550 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:19:14.550 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:19:14.550 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "85bd4d63-eb44-4e15-aade-309eee543ff4", 2024-05-09T09:19:14.550 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:19:14.550 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:19:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.552+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0280a2c90 msgr2=0x7fb0280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.552+0000 7fb03ffff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0280a2c90 0x7fb0280a5150 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55ca4baf0920 tx=0x7fb034009260 comp rx=0 tx=0).stop 2024-05-09T09:19:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.552+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 msgr2=0x7fb0381437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.553+0000 7fb03ffff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 0x7fb0381437f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb030035040 tx=0x7fb030004360 comp rx=0 tx=0).stop 2024-05-09T09:19:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.553+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 shutdown_connections 2024-05-09T09:19:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.553+0000 7fb03ffff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb0280a2c90 0x7fb0280a5150 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.554+0000 7fb03ffff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb038148d90 0x7fb03814b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.554+0000 7fb03ffff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb03809df10 0x7fb038143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.554+0000 7fb03ffff640 1 --2- 172.21.15.64:0/3096435470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03809d550 0x7fb0381437f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.554+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 >> 172.21.15.64:0/3096435470 conn(0x7fb03800b300 msgr2=0x7fb0380a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.555+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 shutdown_connections 2024-05-09T09:19:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.555+0000 7fb03ffff640 1 -- 172.21.15.64:0/3096435470 wait complete. 2024-05-09T09:19:14.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:19:14.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:19:14.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 -- 172.21.15.64:0/2308272168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0948077430 msgr2=0x7f0948077890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 --2- 172.21.15.64:0/2308272168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0948077430 0x7f0948077890 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0938009860 tx=0x7f093802a340 comp rx=0 tx=0).stop 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 -- 172.21.15.64:0/2308272168 shutdown_connections 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 --2- 172.21.15.64:0/2308272168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0948077dd0 0x7f0948078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 --2- 172.21.15.64:0/2308272168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0948077430 0x7f0948077890 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 --2- 172.21.15.64:0/2308272168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09480761f0 0x7f0948076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 -- 172.21.15.64:0/2308272168 >> 172.21.15.64:0/2308272168 conn(0x7f09480ff020 msgr2=0x7f0948101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 -- 172.21.15.64:0/2308272168 shutdown_connections 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.775+0000 7f094ec4d640 1 -- 172.21.15.64:0/2308272168 wait complete. 2024-05-09T09:19:14.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 Processor -- start 2024-05-09T09:19:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 -- start start 2024-05-09T09:19:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09480761f0 0x7f094819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 0x7f094819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09481a38f0 0x7f09481a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f09481139d0 con 0x7f09480761f0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0948113850 con 0x7f09481a38f0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.776+0000 7f094ec4d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0948113b50 con 0x7f0948077dd0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 0x7f094819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 0x7f094819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36334/0 (socket says 172.21.15.64:36334) 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 -- 172.21.15.64:0/4175041365 learned_addr learned my addr 172.21.15.64:0/4175041365 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f094d1ea640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09481a38f0 0x7f09481a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09480761f0 0x7f094819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 -- 172.21.15.64:0/4175041365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09481a38f0 msgr2=0x7f09481a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09481a38f0 0x7f09481a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 -- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09480761f0 msgr2=0x7f094819e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09480761f0 0x7f094819e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.777+0000 7f0947fff640 1 -- 172.21.15.64:0/4175041365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094806b380 con 0x7f0948077dd0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.778+0000 7f0947fff640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 0x7f094819e890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0938034000 tx=0x7f0938005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.778+0000 7f0945ffb640 1 -- 172.21.15.64:0/4175041365 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938040070 con 0x7f0948077dd0 2024-05-09T09:19:14.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.778+0000 7f0945ffb640 1 -- 172.21.15.64:0/4175041365 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0938005440 con 0x7f0948077dd0 2024-05-09T09:19:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.778+0000 7f0945ffb640 1 -- 172.21.15.64:0/4175041365 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938037070 con 0x7f0948077dd0 2024-05-09T09:19:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.778+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094806b610 con 0x7f0948077dd0 2024-05-09T09:19:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.778+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094806bab0 con 0x7f0948077dd0 2024-05-09T09:19:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 Processor -- start 2024-05-09T09:19:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 -- start start 2024-05-09T09:19:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d17350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d1ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5634e0c7bd00 con 0x5634e0d16f30 2024-05-09T09:19:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5634e0c234c0 con 0x5634e0d17920 2024-05-09T09:19:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.781+0000 7f09515fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5634e0c77cc0 con 0x5634e0d1ca90 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d17350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d17350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54282/0 (socket says 172.21.15.64:54282) 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 -- 172.21.15.64:0/713877683 learned_addr learned my addr 172.21.15.64:0/713877683 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 -- 172.21.15.64:0/713877683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 msgr2=0x5634e0d1ef20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d1ef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 -- 172.21.15.64:0/713877683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 msgr2=0x5634e0d1c420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d1c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f094d1ea640 1 -- 172.21.15.64:0/713877683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e0a597d0 con 0x5634e0d16f30 2024-05-09T09:19:14.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.782+0000 7f0945ffb640 1 -- 172.21.15.64:0/4175041365 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0938043070 con 0x7f0948077dd0 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.783+0000 7f094d1ea640 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d17350 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f094000e450 tx=0x7f09400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8356041cfc3f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.783+0000 7f092effd640 1 -- 172.21.15.64:0/713877683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094000d910 con 0x5634e0d16f30 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.783+0000 7f092effd640 1 -- 172.21.15.64:0/713877683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f094001c070 con 0x5634e0d16f30 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.783+0000 7f092effd640 1 -- 172.21.15.64:0/713877683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940017e30 con 0x5634e0d16f30 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.783+0000 7f0945ffb640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f09300a2c10 0x7f09300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f0945ffb640 1 -- 172.21.15.64:0/4175041365 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f09380eedb0 con 0x7f0948077dd0 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 -- 172.21.15.64:0/713877683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 msgr2=0x5634e0d17350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d17350 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f094000e450 tx=0x7f09400095c0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 -- 172.21.15.64:0/713877683 shutdown_connections 2024-05-09T09:19:14.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d1ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d1c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 --2- 172.21.15.64:0/713877683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d17350 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 -- 172.21.15.64:0/713877683 >> 172.21.15.64:0/713877683 conn(0x5634e0d125a0 msgr2=0x5634e0d129a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f09300a2c10 0x7f09300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.784+0000 7f09515fd740 1 -- 172.21.15.64:0/713877683 shutdown_connections 2024-05-09T09:19:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.785+0000 7f09515fd740 1 -- 172.21.15.64:0/713877683 wait complete. 2024-05-09T09:19:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.785+0000 7f09515fd740 1 -- start start 2024-05-09T09:19:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.785+0000 7f09515fd740 1 Processor -- start 2024-05-09T09:19:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d213b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5634e0d1fa40 con 0x5634e0d16f30 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5634e0c36b90 con 0x5634e0d17920 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5634e0d1fbc0 con 0x5634e0d1ca90 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f09515fd740 10 client.0 _ll_register_callbacks cb 0x5634e0c72da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f094c9e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d213b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f094c9e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d213b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54286/0 (socket says 172.21.15.64:54286) 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f0947fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f094c9e9640 1 -- 172.21.15.64:0/4113405014 learned_addr learned my addr 172.21.15.64:0/4113405014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f094d1ea640 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f0947fff640 1 -- 172.21.15.64:0/4113405014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 msgr2=0x5634e0d288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f0947fff640 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f0947fff640 1 -- 172.21.15.64:0/4113405014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 msgr2=0x5634e0d213b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.786+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f09300a2c10 0x7f09300a50d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5634e0d13470 tx=0x7f093c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0947fff640 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d213b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0947fff640 1 -- 172.21.15.64:0/4113405014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e098f3f0 con 0x5634e0d17920 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0947fff640 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d218f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f093802a930 tx=0x7f09380bb0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9904c3ff2baecedc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0927fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0927fff640 1 -- 172.21.15.64:0/4113405014 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380d9e60 con 0x5634e0d17920 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0927fff640 1 -- 172.21.15.64:0/4113405014 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f09380db0d0 con 0x5634e0d17920 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094001c070 con 0x5634e0d17920 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f0927fff640 1 -- 172.21.15.64:0/4113405014 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f09380d4200 con 0x5634e0d17920 2024-05-09T09:19:14.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f09515fd740 10 client.27506 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:14.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.787+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5634e094f0f0 con 0x5634e0d17920 2024-05-09T09:19:14.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f0927fff640 1 -- 172.21.15.64:0/4113405014 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f094001c070 con 0x5634e0d17920 2024-05-09T09:19:14.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f0927fff640 1 -- 172.21.15.64:0/4113405014 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f09380c3a00 con 0x5634e0d17920 2024-05-09T09:19:14.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f09515fd740 10 client.27506 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:14.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f09515fd740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:14.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f09515fd740 20 client.27506 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:14.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f09515fd740 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d2d110 0x5634e0d2f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f09515fd740 4 client.27506 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e0c89430 con 0x5634e0d2d110 2024-05-09T09:19:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.788+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d2d110 0x5634e0d2f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.789+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d2d110 0x5634e0d2f550 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237f01e040acd275 server_cookie=c743f2badedcf64b in_seq=0 out_seq=0 2024-05-09T09:19:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.789+0000 7f0927fff640 10 client.27506 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.789+0000 7f0927fff640 1 -- 172.21.15.64:0/4113405014 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634e0c89430 con 0x5634e0d2d110 2024-05-09T09:19:14.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f0927fff640 10 client.27506 handle_command_reply: tid=0 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 client.27506 shutdown 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 msgr2=0x5634e0d218f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d218f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f093802a930 tx=0x7f09380bb0f0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 shutdown_connections 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d2d110 0x5634e0d2f550 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d1ca90 0x5634e0d288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d17920 0x5634e0d218f0 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f093802a930 tx=0x7f09380bb0f0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 --2- 172.21.15.64:0/4113405014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d16f30 0x5634e0d213b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 >> 172.21.15.64:0/4113405014 conn(0x5634e0d125a0 msgr2=0x5634e0d117e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f0927fff640 0 client.27506 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 shutdown_connections 2024-05-09T09:19:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 1 -- 172.21.15.64:0/4113405014 wait complete. 2024-05-09T09:19:14.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.790+0000 7f09515fd740 20 client.27506 trim_cache size 0 max 16384 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.824+0000 7f09515fd740 1 Processor -- start 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.824+0000 7f09515fd740 1 -- start start 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.824+0000 7f09515fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f09515fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d28850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f09515fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d10f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d20d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:44200/0 (socket says 172.21.15.64:44200) 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 -- 172.21.15.64:0/4157754338 learned_addr learned my addr 172.21.15.64:0/4157754338 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f09481bc040 con 0x5634e0d1ca90 2024-05-09T09:19:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b9910 con 0x5634e0d16f30 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0948073680 con 0x5634e0d18610 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f0947fff640 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d28850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094d1ea640 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d10f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 -- 172.21.15.64:0/4157754338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 msgr2=0x5634e0d10f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d10f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 -- 172.21.15.64:0/4157754338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 msgr2=0x5634e0d28850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d28850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.825+0000 7f094c9e9640 1 -- 172.21.15.64:0/4157754338 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e0d1c7e0 con 0x5634e0d16f30 2024-05-09T09:19:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.826+0000 7f094c9e9640 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d20d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f093c01b000 tx=0x7f093c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11dec0d0a70ec1c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.826+0000 7f092e7fc640 1 -- 172.21.15.64:0/4157754338 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c027d50 con 0x5634e0d16f30 2024-05-09T09:19:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.826+0000 7f092e7fc640 1 -- 172.21.15.64:0/4157754338 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f093c0044d0 con 0x5634e0d16f30 2024-05-09T09:19:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.826+0000 7f092e7fc640 1 -- 172.21.15.64:0/4157754338 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c012880 con 0x5634e0d16f30 2024-05-09T09:19:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 msgr2=0x5634e0d20d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d20d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f093c01b000 tx=0x7f093c01d000 comp rx=0 tx=0).stop 2024-05-09T09:19:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 shutdown_connections 2024-05-09T09:19:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d10f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d28850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 --2- 172.21.15.64:0/4157754338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d20d90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 >> 172.21.15.64:0/4157754338 conn(0x5634e0d125a0 msgr2=0x5634e0d20680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 shutdown_connections 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 -- 172.21.15.64:0/4157754338 wait complete. 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.827+0000 7f09515fd740 1 -- start start 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 Processor -- start 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d26ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b7dd0 con 0x5634e0d1ca90 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b9240 con 0x5634e0d16f30 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f09481b84a0 con 0x5634e0d18610 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f09515fd740 10 client.0 _ll_register_callbacks cb 0x7f0938009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.828+0000 7f094c9e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d26ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d26ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36356/0 (socket says 172.21.15.64:36356) 2024-05-09T09:19:14.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 -- 172.21.15.64:0/2373019289 learned_addr learned my addr 172.21.15.64:0/2373019289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:14.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f094c9e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d26590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:44202/0 (socket says 172.21.15.64:44202) 2024-05-09T09:19:14.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f094d1ea640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 -- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 msgr2=0x5634e0d26590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d26590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 -- 172.21.15.64:0/2373019289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 msgr2=0x5634e0d27a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:14.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d27a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 -- 172.21.15.64:0/2373019289 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e0d28060 con 0x5634e0d18610 2024-05-09T09:19:14.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f094d1ea640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:19:14.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f094c9e9640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:19:14.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.829+0000 7f0947fff640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d26ad0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f09380b94d0 tx=0x7f09380b74c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9527000a124d856d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.830+0000 7f0927fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.830+0000 7f0927fff640 1 -- 172.21.15.64:0/2373019289 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380d9e60 con 0x5634e0d18610 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.830+0000 7f0927fff640 1 -- 172.21.15.64:0/2373019289 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f09380db070 con 0x5634e0d18610 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.830+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f093c0044d0 con 0x5634e0d18610 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.830+0000 7f0927fff640 1 -- 172.21.15.64:0/2373019289 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f09380bab80 con 0x5634e0d18610 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.830+0000 7f09515fd740 10 client.27652 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:14.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.831+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5634e0d93ce0 con 0x5634e0d18610 2024-05-09T09:19:14.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.831+0000 7f0927fff640 1 -- 172.21.15.64:0/2373019289 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09380d8030 con 0x5634e0d18610 2024-05-09T09:19:14.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f0927fff640 1 -- 172.21.15.64:0/2373019289 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f0938030ce0 con 0x5634e0d18610 2024-05-09T09:19:14.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f09515fd740 10 client.27652 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f09515fd740 10 client.27652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f09515fd740 20 client.27652 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f09515fd740 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d94580 0x5634e0d969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f09515fd740 4 client.27652 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5634e0c83780 con 0x5634e0d94580 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.832+0000 7f094c9e9640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d94580 0x5634e0d969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.833+0000 7f094c9e9640 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d94580 0x5634e0d969a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1245925e6202ff36 server_cookie=1d92865a871e163 in_seq=0 out_seq=0 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.833+0000 7f0927fff640 10 client.27652 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.833+0000 7f0927fff640 1 -- 172.21.15.64:0/2373019289 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5634e0c83780 con 0x5634e0d94580 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.834+0000 7f0927fff640 10 client.27652 handle_command_reply: tid=0 2024-05-09T09:19:14.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.834+0000 7f09515fd740 1 client.27652 shutdown 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.834+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 msgr2=0x5634e0d26ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.834+0000 7f09515fd740 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d26ad0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f09380b94d0 tx=0x7f09380b74c0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 shutdown_connections 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f09515fd740 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5634e0d94580 0x5634e0d969a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f09515fd740 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5634e0d1ca90 0x5634e0d27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f09515fd740 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634e0d18610 0x5634e0d26ad0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f09515fd740 1 --2- 172.21.15.64:0/2373019289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5634e0d16f30 0x5634e0d26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 >> 172.21.15.64:0/2373019289 conn(0x5634e0d125a0 msgr2=0x5634e0d154b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.838+0000 7f0927fff640 0 client.27652 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.840+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 shutdown_connections 2024-05-09T09:19:14.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.840+0000 7f09515fd740 1 -- 172.21.15.64:0/2373019289 wait complete. 2024-05-09T09:19:14.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.840+0000 7f09515fd740 20 client.27652 trim_cache size 0 max 16384 2024-05-09T09:19:14.846 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:19:14.846 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:19:14.846 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:19:14.846 INFO:teuthology.orchestra.run.smithi064.stdout: "85bd4d63-eb44-4e15-aade-309eee543ff4": { 2024-05-09T09:19:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:19:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "85bd4d63-eb44-4e15-aade-309eee543ff4", 2024-05-09T09:19:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:19:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:19:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:19:14.847 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:19:14.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.842+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f09300a2c10 msgr2=0x7f09300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.842+0000 7f094ec4d640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f09300a2c10 0x7f09300a50d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5634e0d13470 tx=0x7f093c005f90 comp rx=0 tx=0).stop 2024-05-09T09:19:14.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.842+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 msgr2=0x7f094819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:14.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.842+0000 7f094ec4d640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 0x7f094819e890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0938034000 tx=0x7f0938005040 comp rx=0 tx=0).stop 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.844+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 shutdown_connections 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.845+0000 7f094ec4d640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f09300a2c10 0x7f09300a50d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.845+0000 7f094ec4d640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09481a38f0 0x7f09481a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.845+0000 7f094ec4d640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0948077dd0 0x7f094819e890 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.845+0000 7f094ec4d640 1 --2- 172.21.15.64:0/4175041365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09480761f0 0x7f094819e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.845+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 >> 172.21.15.64:0/4175041365 conn(0x7f09480ff020 msgr2=0x7f0948102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.846+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 shutdown_connections 2024-05-09T09:19:14.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:14.848+0000 7f094ec4d640 1 -- 172.21.15.64:0/4175041365 wait complete. 2024-05-09T09:19:14.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:14 smithi064 ceph-mon[30149]: pgmap v673: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.3 MiB/s wr, 865 op/s 2024-05-09T09:19:14.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:14 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:19:14.871 INFO:tasks.cephfs.filesystem:scrub status for tag:85bd4d63-eb44-4e15-aade-309eee543ff4 - {'path': '/', 'tag': '85bd4d63-eb44-4e15-aade-309eee543ff4', 'options': 'recursive,force'} 2024-05-09T09:19:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:14 smithi179 ceph-mon[34086]: pgmap v673: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.3 MiB/s wr, 865 op/s 2024-05-09T09:19:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:14 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:19:15.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:14 smithi139 ceph-mon[33599]: pgmap v673: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.3 MiB/s wr, 865 op/s 2024-05-09T09:19:15.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:14 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:19:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:19:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:19:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:19:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:15.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:15.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:15 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:15 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:15 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:16.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 535.0 s, 640.8 tps, lat 156.283 ms stddev 65.946 2024-05-09T09:19:16.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:16 smithi179 ceph-mon[34086]: pgmap v674: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 889 op/s 2024-05-09T09:19:17.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:16 smithi139 ceph-mon[33599]: pgmap v674: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 889 op/s 2024-05-09T09:19:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:16 smithi064 ceph-mon[30149]: pgmap v674: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 889 op/s 2024-05-09T09:19:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:18 smithi179 ceph-mon[34086]: pgmap v675: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.5 MiB/s wr, 849 op/s 2024-05-09T09:19:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:18 smithi139 ceph-mon[33599]: pgmap v675: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.5 MiB/s wr, 849 op/s 2024-05-09T09:19:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:18 smithi064 ceph-mon[30149]: pgmap v675: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.5 MiB/s wr, 849 op/s 2024-05-09T09:19:20.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:20 smithi064 ceph-mon[30149]: pgmap v676: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-05-09T09:19:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:20 smithi179 ceph-mon[34086]: pgmap v676: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-05-09T09:19:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:20 smithi139 ceph-mon[33599]: pgmap v676: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-05-09T09:19:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 540.0 s, 652.2 tps, lat 153.593 ms stddev 60.468 2024-05-09T09:19:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:22 smithi179 ceph-mon[34086]: pgmap v677: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.4 MiB/s wr, 815 op/s 2024-05-09T09:19:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:22 smithi139 ceph-mon[33599]: pgmap v677: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.4 MiB/s wr, 815 op/s 2024-05-09T09:19:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:22 smithi064 ceph-mon[30149]: pgmap v677: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.4 MiB/s wr, 815 op/s 2024-05-09T09:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 -- 172.21.15.64:0/2870781717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c1018b0 msgr2=0x7fdd0c101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 --2- 172.21.15.64:0/2870781717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c101cd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009900 tx=0x7fdd0802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 -- 172.21.15.64:0/2870781717 shutdown_connections 2024-05-09T09:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 --2- 172.21.15.64:0/2870781717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c103460 0x7fdd0c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 --2- 172.21.15.64:0/2870781717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c102aa0 0x7fdd0c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 --2- 172.21.15.64:0/2870781717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c101cd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 -- 172.21.15.64:0/2870781717 >> 172.21.15.64:0/2870781717 conn(0x7fdd0c0fce20 msgr2=0x7fdd0c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.634+0000 7fdd1464d640 1 -- 172.21.15.64:0/2870781717 shutdown_connections 2024-05-09T09:19:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.635+0000 7fdd1464d640 1 -- 172.21.15.64:0/2870781717 wait complete. 2024-05-09T09:19:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.635+0000 7fdd1464d640 1 Processor -- start 2024-05-09T09:19:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 -- start start 2024-05-09T09:19:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c102aa0 0x7fdd0c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c103460 0x7fdd0c1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10b250 con 0x7fdd0c1018b0 2024-05-09T09:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10b0d0 con 0x7fdd0c103460 2024-05-09T09:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd1464d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10b3d0 con 0x7fdd0c102aa0 2024-05-09T09:19:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36834/0 (socket says 172.21.15.64:36834) 2024-05-09T09:19:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd123e9640 1 -- 172.21.15.64:0/4157868602 learned_addr learned my addr 172.21.15.64:0/4157868602 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.636+0000 7fdd12bea640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c103460 0x7fdd0c1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd11be8640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c102aa0 0x7fdd0c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd12bea640 1 -- 172.21.15.64:0/4157868602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c102aa0 msgr2=0x7fdd0c19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd12bea640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c102aa0 0x7fdd0c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd12bea640 1 -- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c1018b0 msgr2=0x7fdd0c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd12bea640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c19e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd12bea640 1 -- 172.21.15.64:0/4157868602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c1a5c50 con 0x7fdd0c103460 2024-05-09T09:19:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd12bea640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c103460 0x7fdd0c1a55e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009fd0 tx=0x7fdd0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.637+0000 7fdd037fe640 1 -- 172.21.15.64:0/4157868602 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400ce20 con 0x7fdd0c103460 2024-05-09T09:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.638+0000 7fdd037fe640 1 -- 172.21.15.64:0/4157868602 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd04005020 con 0x7fdd0c103460 2024-05-09T09:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.638+0000 7fdd037fe640 1 -- 172.21.15.64:0/4157868602 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04010690 con 0x7fdd0c103460 2024-05-09T09:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.638+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1a5f40 con 0x7fdd0c103460 2024-05-09T09:19:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.638+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c1a6410 con 0x7fdd0c103460 2024-05-09T09:19:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.641+0000 7fdd037fe640 1 -- 172.21.15.64:0/4157868602 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdd04010830 con 0x7fdd0c103460 2024-05-09T09:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.642+0000 7fdd037fe640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcf80a2d10 0x7fdcf80a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.643+0000 7fdd037fe640 1 -- 172.21.15.64:0/4157868602 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd040d0810 con 0x7fdd0c103460 2024-05-09T09:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.644+0000 7fdd123e9640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcf80a2d10 0x7fdcf80a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.646+0000 7fdd123e9640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcf80a2d10 0x7fdcf80a51d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000c00 tx=0x7fdd08005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.646+0000 7fdd1743a740 1 Processor -- start 2024-05-09T09:19:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.647+0000 7fdd1743a740 1 -- start start 2024-05-09T09:19:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.647+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93e9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.647+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9fc40 0x563d93ea4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.647+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93ea4db0 0x563d93ea7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd1743a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d93e03340 con 0x563d93e9fc40 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd1743a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563d93dab290 con 0x563d93e9f250 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93e9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93e9f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36688/0 (socket says 172.21.15.64:36688) 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd123e9640 1 -- 172.21.15.64:0/2799768860 learned_addr learned my addr 172.21.15.64:0/2799768860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd12bea640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93ea4db0 0x563d93ea7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd1743a740 1 -- 172.21.15.64:0/2799768860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d93dff300 con 0x563d93ea4db0 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.648+0000 7fdd11be8640 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9fc40 0x563d93ea4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd12bea640 1 -- 172.21.15.64:0/2799768860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 msgr2=0x563d93e9f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd12bea640 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93e9f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd12bea640 1 -- 172.21.15.64:0/2799768860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9fc40 msgr2=0x563d93ea4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd12bea640 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9fc40 0x563d93ea4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd12bea640 1 -- 172.21.15.64:0/2799768860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d93dcf8e0 con 0x563d93ea4db0 2024-05-09T09:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd12bea640 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93ea4db0 0x563d93ea7240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd040cdf80 tx=0x7fdd0409d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987e3d1a2dc4a120 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.649+0000 7fdd00bfe640 1 -- 172.21.15.64:0/2799768860 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5950 con 0x563d93ea4db0 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.650+0000 7fdd00bfe640 1 -- 172.21.15.64:0/2799768860 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd0409afe0 con 0x563d93ea4db0 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.650+0000 7fdd00bfe640 1 -- 172.21.15.64:0/2799768860 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5720 con 0x563d93ea4db0 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 -- 172.21.15.64:0/2799768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93ea4db0 msgr2=0x563d93ea7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93ea4db0 0x563d93ea7240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd040cdf80 tx=0x7fdd0409d4a0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 -- 172.21.15.64:0/2799768860 shutdown_connections 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93ea4db0 0x563d93ea7240 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9fc40 0x563d93ea4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 --2- 172.21.15.64:0/2799768860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93e9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 -- 172.21.15.64:0/2799768860 >> 172.21.15.64:0/2799768860 conn(0x563d93e9a8c0 msgr2=0x563d93e9acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 -- 172.21.15.64:0/2799768860 shutdown_connections 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.651+0000 7fdd1743a740 1 -- 172.21.15.64:0/2799768860 wait complete. 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.652+0000 7fdd1743a740 1 -- start start 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.652+0000 7fdd1743a740 1 Processor -- start 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93eba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93ebaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93ea4db0 0x563d93ec1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d93eb8d70 con 0x563d93ea4db0 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563d93dbe790 con 0x563d93e9f250 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d93eb8ef0 con 0x563d93e9fc40 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd1743a740 10 client.0 _ll_register_callbacks cb 0x563d93df9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd11be8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93ebaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd11be8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93ebaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35982/0 (socket says 172.21.15.64:35982) 2024-05-09T09:19:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd11be8640 1 -- 172.21.15.64:0/521159121 learned_addr learned my addr 172.21.15.64:0/521159121 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd123e9640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93eba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd12bea640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93ea4db0 0x563d93ec1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd123e9640 1 -- 172.21.15.64:0/521159121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 msgr2=0x563d93ebaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd123e9640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93ebaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd123e9640 1 -- 172.21.15.64:0/521159121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93ea4db0 msgr2=0x563d93ec1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd123e9640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93ea4db0 0x563d93ec1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.653+0000 7fdd123e9640 1 -- 172.21.15.64:0/521159121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d93b17210 con 0x563d93e9f250 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.654+0000 7fdd11be8640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93ebaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.654+0000 7fdd123e9640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93eba560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd08005ab0 tx=0x7fdd0802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca44dc5fb4b7e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.654+0000 7fdcf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.654+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/521159121 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08035050 con 0x563d93e9f250 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.654+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/521159121 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdd08031990 con 0x563d93e9f250 2024-05-09T09:19:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.654+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd0409afe0 con 0x563d93e9f250 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.656+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/521159121 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdd08031b90 con 0x563d93e9f250 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.657+0000 7fdd1743a740 10 client.27524 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.657+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563d93be3620 con 0x563d93e9f250 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/521159121 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd0804e070 con 0x563d93e9f250 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/521159121 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdd0803c440 con 0x563d93e9f250 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdd1743a740 10 client.27524 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdd1743a740 10 client.27524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdd1743a740 20 client.27524 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdd1743a740 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ec5dc0 0x563d93ec8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdd1743a740 4 client.27524 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.658+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd0409afe0 con 0x563d93ec5dc0 2024-05-09T09:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.659+0000 7fdd11be8640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ec5dc0 0x563d93ec8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.660+0000 7fdd11be8640 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ec5dc0 0x563d93ec8210 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba920c3e2a7e5222 server_cookie=5a874bb666dba9b6 in_seq=0 out_seq=0 2024-05-09T09:19:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.660+0000 7fdcf5ffb640 10 client.27524 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.661+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/521159121 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fdd0409afe0 con 0x563d93ec5dc0 2024-05-09T09:19:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.661+0000 7fdcf5ffb640 10 client.27524 handle_command_reply: tid=0 2024-05-09T09:19:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.661+0000 7fdd1743a740 1 client.27524 shutdown 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 msgr2=0x563d93eba560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93eba560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd08005ab0 tx=0x7fdd0802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 shutdown_connections 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ec5dc0 0x563d93ec8210 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93ea4db0 0x563d93ec1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93ebaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdcf5ffb640 0 client.27524 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 --2- 172.21.15.64:0/521159121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9f250 0x563d93eba560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 >> 172.21.15.64:0/521159121 conn(0x563d93e9a8c0 msgr2=0x563d93e99a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 shutdown_connections 2024-05-09T09:19:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 1 -- 172.21.15.64:0/521159121 wait complete. 2024-05-09T09:19:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.662+0000 7fdd1743a740 20 client.27524 trim_cache size 0 max 16384 2024-05-09T09:19:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.695+0000 7fdd1743a740 1 Processor -- start 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.695+0000 7fdd1743a740 1 -- start start 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9a8c0 0x563d93eb9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ec1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9a8c0 0x563d93eb9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9a8c0 0x563d93eb9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36708/0 (socket says 172.21.15.64:36708) 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd123e9640 1 -- 172.21.15.64:0/1478720958 learned_addr learned my addr 172.21.15.64:0/1478720958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd1743a740 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93eba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1aab30 con 0x563d93e9f250 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1a8400 con 0x563d93e9a8c0 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.696+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c198c80 con 0x563d93e9fc40 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd11be8640 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ec1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd12bea640 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93eba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd11be8640 1 -- 172.21.15.64:0/1478720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 msgr2=0x563d93eba520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd11be8640 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93eba520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd11be8640 1 -- 172.21.15.64:0/1478720958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9a8c0 msgr2=0x563d93eb9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd11be8640 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9a8c0 0x563d93eb9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.697+0000 7fdd11be8640 1 -- 172.21.15.64:0/1478720958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d93ea0020 con 0x563d93e9f250 2024-05-09T09:19:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.698+0000 7fdd11be8640 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ec1a70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0096b0 tx=0x7fdcfc013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7af89288bf204d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.700+0000 7fdcf7fff640 1 -- 172.21.15.64:0/1478720958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc01f700 con 0x563d93e9f250 2024-05-09T09:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.701+0000 7fdcf7fff640 1 -- 172.21.15.64:0/1478720958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcfc023030 con 0x563d93e9f250 2024-05-09T09:19:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.701+0000 7fdcf7fff640 1 -- 172.21.15.64:0/1478720958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc003d80 con 0x563d93e9f250 2024-05-09T09:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.702+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 msgr2=0x563d93ec1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.702+0000 7fdd1743a740 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ec1a70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0096b0 tx=0x7fdcfc013a70 comp rx=0 tx=0).stop 2024-05-09T09:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.703+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 shutdown_connections 2024-05-09T09:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.703+0000 7fdd1743a740 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9fc40 0x563d93eba520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.703+0000 7fdd1743a740 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ec1a70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.703+0000 7fdd1743a740 1 --2- 172.21.15.64:0/1478720958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9a8c0 0x563d93eb9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.703+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 >> 172.21.15.64:0/1478720958 conn(0x563d93ea48f0 msgr2=0x563d93e9aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.703+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 shutdown_connections 2024-05-09T09:19:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.704+0000 7fdd1743a740 1 -- 172.21.15.64:0/1478720958 wait complete. 2024-05-09T09:19:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.704+0000 7fdd1743a740 1 -- start start 2024-05-09T09:19:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.704+0000 7fdd1743a740 1 Processor -- start 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 0x563d93ebee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ebf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9fc40 0x563d93ec03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1a68c0 con 0x563d93e9f250 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1a7d30 con 0x563d93e9fc40 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1a6f90 con 0x563d93e9a8c0 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd1743a740 10 client.0 _ll_register_callbacks cb 0x7fdd08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 0x563d93ebee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd123e9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 0x563d93ebee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36002/0 (socket says 172.21.15.64:36002) 2024-05-09T09:19:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd123e9640 1 -- 172.21.15.64:0/3472268341 learned_addr learned my addr 172.21.15.64:0/3472268341 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.705+0000 7fdd11be8640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ebf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd12bea640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9fc40 0x563d93ec03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd11be8640 1 -- 172.21.15.64:0/3472268341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 msgr2=0x563d93ebee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd11be8640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 0x563d93ebee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd11be8640 1 -- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9fc40 msgr2=0x563d93ec03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd11be8640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9fc40 0x563d93ec03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd11be8640 1 -- 172.21.15.64:0/3472268341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d93ec09b0 con 0x563d93e9f250 2024-05-09T09:19:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd123e9640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 0x563d93ebee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:19:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdd11be8640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ebf3b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc015010 tx=0x7fdcfc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ecf24b8e7d5f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.706+0000 7fdcf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.707+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/3472268341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0037f0 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.707+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/3472268341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdcfc01f500 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.707+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcfc023030 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.707+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/3472268341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdcfc01f700 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.707+0000 7fdd1743a740 10 client.18027 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.707+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563d93ec1d60 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/3472268341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcfc01f8a0 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/3472268341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdcfc00f040 con 0x563d93e9f250 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd1743a740 10 client.18027 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd1743a740 10 client.18027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd1743a740 20 client.18027 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd1743a740 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ea4db0 0x563d93f2ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd1743a740 4 client.18027 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdcfc023030 con 0x563d93ea4db0 2024-05-09T09:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.708+0000 7fdd123e9640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ea4db0 0x563d93f2ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.709+0000 7fdcf5ffb640 10 client.18027 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.709+0000 7fdd123e9640 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ea4db0 0x563d93f2ea80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd875285a88661b server_cookie=38da05db8005c190 in_seq=0 out_seq=0 2024-05-09T09:19:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.737+0000 7fdcf5ffb640 1 -- 172.21.15.64:0/3472268341 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdcfc023030 con 0x563d93ea4db0 2024-05-09T09:19:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.737+0000 7fdcf5ffb640 10 client.18027 handle_command_reply: tid=0 2024-05-09T09:19:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.737+0000 7fdd1743a740 1 client.18027 shutdown 2024-05-09T09:19:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.737+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 msgr2=0x563d93ebf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.737+0000 7fdd1743a740 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ebf3b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc015010 tx=0x7fdcfc003680 comp rx=0 tx=0).stop 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 shutdown_connections 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563d93ea4db0 0x563d93f2ea80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563d93e9fc40 0x563d93ec03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d93e9f250 0x563d93ebf3b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 --2- 172.21.15.64:0/3472268341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d93e9a8c0 0x563d93ebee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 >> 172.21.15.64:0/3472268341 conn(0x563d93ea48f0 msgr2=0x563d93eba4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdcf5ffb640 0 client.18027 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.738+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 shutdown_connections 2024-05-09T09:19:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.739+0000 7fdd1743a740 1 -- 172.21.15.64:0/3472268341 wait complete. 2024-05-09T09:19:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.739+0000 7fdd1743a740 20 client.18027 trim_cache size 0 max 16384 2024-05-09T09:19:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:19:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:19:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:19:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:19:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.740+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcf80a2d10 msgr2=0x7fdcf80a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.740+0000 7fdd1464d640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcf80a2d10 0x7fdcf80a51d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000c00 tx=0x7fdd08005b50 comp rx=0 tx=0).stop 2024-05-09T09:19:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c103460 msgr2=0x7fdd0c1a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c103460 0x7fdd0c1a55e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009fd0 tx=0x7fdd0400ef10 comp rx=0 tx=0).stop 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 shutdown_connections 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdcf80a2d10 0x7fdcf80a51d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0c103460 0x7fdd0c1a55e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c102aa0 0x7fdd0c19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 --2- 172.21.15.64:0/4157868602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd0c1018b0 0x7fdd0c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.741+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 >> 172.21.15.64:0/4157868602 conn(0x7fdd0c0fce20 msgr2=0x7fdd0c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.742+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 shutdown_connections 2024-05-09T09:19:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:24.743+0000 7fdd1464d640 1 -- 172.21.15.64:0/4157868602 wait complete. 2024-05-09T09:19:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:24 smithi179 ceph-mon[34086]: pgmap v678: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.8 MiB/s wr, 974 op/s 2024-05-09T09:19:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:24 smithi139 ceph-mon[33599]: pgmap v678: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.8 MiB/s wr, 974 op/s 2024-05-09T09:19:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:24 smithi064 ceph-mon[30149]: pgmap v678: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.8 MiB/s wr, 974 op/s 2024-05-09T09:19:25.984 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:19:25.984 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:19:25.984 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:25 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:19:26.079 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:19:26.079 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:19:26.079 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:25 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:19:26.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:19:26.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:19:26.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:25 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:19:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 545.0 s, 681.8 tps, lat 147.422 ms stddev 73.212 2024-05-09T09:19:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:26 smithi179 ceph-mon[34086]: pgmap v679: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.7 MiB/s wr, 906 op/s 2024-05-09T09:19:27.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:26 smithi139 ceph-mon[33599]: pgmap v679: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.7 MiB/s wr, 906 op/s 2024-05-09T09:19:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:26 smithi064 ceph-mon[30149]: pgmap v679: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.7 MiB/s wr, 906 op/s 2024-05-09T09:19:28.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:28 smithi179 ceph-mon[34086]: pgmap v680: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 818 op/s 2024-05-09T09:19:29.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:28 smithi139 ceph-mon[33599]: pgmap v680: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 818 op/s 2024-05-09T09:19:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:28 smithi064 ceph-mon[30149]: pgmap v680: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 818 op/s 2024-05-09T09:19:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:30 smithi179 ceph-mon[34086]: pgmap v681: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.5 MiB/s wr, 1.08k op/s 2024-05-09T09:19:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:30 smithi139 ceph-mon[33599]: pgmap v681: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.5 MiB/s wr, 1.08k op/s 2024-05-09T09:19:31.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:30 smithi064 ceph-mon[30149]: pgmap v681: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.5 MiB/s wr, 1.08k op/s 2024-05-09T09:19:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 550.0 s, 467.2 tps, lat 214.700 ms stddev 453.057 2024-05-09T09:19:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:32 smithi179 ceph-mon[34086]: pgmap v682: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-05-09T09:19:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:32 smithi139 ceph-mon[33599]: pgmap v682: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-05-09T09:19:33.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:32 smithi064 ceph-mon[30149]: pgmap v682: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-05-09T09:19:34.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:34 smithi179 ceph-mon[34086]: pgmap v683: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 973 op/s 2024-05-09T09:19:35.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:34 smithi139 ceph-mon[33599]: pgmap v683: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 973 op/s 2024-05-09T09:19:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:34 smithi064 ceph-mon[30149]: pgmap v683: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 973 op/s 2024-05-09T09:19:36.354 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 555.0 s, 706.4 tps, lat 140.294 ms stddev 89.585 2024-05-09T09:19:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:36 smithi179 ceph-mon[34086]: pgmap v684: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 943 op/s 2024-05-09T09:19:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:36 smithi139 ceph-mon[33599]: pgmap v684: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 943 op/s 2024-05-09T09:19:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:36 smithi064 ceph-mon[30149]: pgmap v684: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 943 op/s 2024-05-09T09:19:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 -- 172.21.15.64:0/2235285053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 msgr2=0x7fdc901146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 --2- 172.21.15.64:0/2235285053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901146e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009860 tx=0x7fdc8802a360 comp rx=0 tx=0).stop 2024-05-09T09:19:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 -- 172.21.15.64:0/2235285053 shutdown_connections 2024-05-09T09:19:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 --2- 172.21.15.64:0/2235285053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901146e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 --2- 172.21.15.64:0/2235285053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc9010d490 0x7fdc9010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 --2- 172.21.15.64:0/2235285053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 0x7fdc9010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.894+0000 7fdc964fc640 1 -- 172.21.15.64:0/2235285053 >> 172.21.15.64:0/2235285053 conn(0x7fdc9006c470 msgr2=0x7fdc9006c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.897+0000 7fdc964fc640 1 -- 172.21.15.64:0/2235285053 shutdown_connections 2024-05-09T09:19:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.897+0000 7fdc964fc640 1 -- 172.21.15.64:0/2235285053 wait complete. 2024-05-09T09:19:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.900+0000 7fdc964fc640 1 Processor -- start 2024-05-09T09:19:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.900+0000 7fdc964fc640 1 -- start start 2024-05-09T09:19:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.900+0000 7fdc964fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 0x7fdc90119340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.900+0000 7fdc964fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc9010d490 0x7fdc90119880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.900+0000 7fdc964fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc964fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9006efe0 con 0x7fdc9010d490 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 0x7fdc90119340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 0x7fdc90119340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45766/0 (socket says 172.21.15.64:45766) 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc8ffff640 1 -- 172.21.15.64:0/817267153 learned_addr learned my addr 172.21.15.64:0/817267153 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9006ee60 con 0x7fdc9010de50 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9006f160 con 0x7fdc9010c2a0 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc94a99640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc94a99640 1 -- 172.21.15.64:0/817267153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 msgr2=0x7fdc90119340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc94a99640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 0x7fdc90119340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc94a99640 1 -- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc9010d490 msgr2=0x7fdc90119880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.901+0000 7fdc94a99640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc9010d490 0x7fdc90119880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.902+0000 7fdc94a99640 1 -- 172.21.15.64:0/817267153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc901b6ca0 con 0x7fdc9010de50 2024-05-09T09:19:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.902+0000 7fdc94a99640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901b66e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009780 tx=0x7fdc88032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.903+0000 7fdc8dffb640 1 -- 172.21.15.64:0/817267153 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88042070 con 0x7fdc9010de50 2024-05-09T09:19:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.903+0000 7fdc8dffb640 1 -- 172.21.15.64:0/817267153 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc88030780 con 0x7fdc9010de50 2024-05-09T09:19:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.903+0000 7fdc8dffb640 1 -- 172.21.15.64:0/817267153 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88037040 con 0x7fdc9010de50 2024-05-09T09:19:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.903+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc901b6f30 con 0x7fdc9010de50 2024-05-09T09:19:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.903+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc901b73a0 con 0x7fdc9010de50 2024-05-09T09:19:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.911+0000 7fdc8dffb640 1 -- 172.21.15.64:0/817267153 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc88045070 con 0x7fdc9010de50 2024-05-09T09:19:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 Processor -- start 2024-05-09T09:19:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 -- start start 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33dcc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 0x564f33dd1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f33d30340 con 0x564f33dcc250 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564f33cd8290 con 0x564f33dd1db0 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.912+0000 7fdc99206740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564f33d2c300 con 0x564f33dccc40 2024-05-09T09:19:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8dffb640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc7c0a2df0 0x7fdc7c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 0x564f33dd1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 0x564f33dd1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45782/0 (socket says 172.21.15.64:45782) 2024-05-09T09:19:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 -- 172.21.15.64:0/2214180084 learned_addr learned my addr 172.21.15.64:0/2214180084 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc877fe640 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc877fe640 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33dcc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc94a99640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc7c0a2df0 0x7fdc7c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8dffb640 1 -- 172.21.15.64:0/817267153 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc880eebd0 con 0x7fdc9010de50 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 -- 172.21.15.64:0/2214180084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 msgr2=0x564f33dd4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 -- 172.21.15.64:0/2214180084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 msgr2=0x564f33dcc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33dcc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 -- 172.21.15.64:0/2214180084 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f33cfc8e0 con 0x564f33dccc40 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.913+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 0x564f33dd1740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000f480 tx=0x7fdc8000f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947f7a51a87373ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.914+0000 7fdc867fc640 1 -- 172.21.15.64:0/2214180084 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80003970 con 0x564f33dccc40 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.914+0000 7fdc94a99640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc7c0a2df0 0x7fdc7c0a52b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b75b0 tx=0x7fdc88085e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.914+0000 7fdc867fc640 1 -- 172.21.15.64:0/2214180084 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc8000e780 con 0x564f33dccc40 2024-05-09T09:19:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc867fc640 1 -- 172.21.15.64:0/2214180084 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004450 con 0x564f33dccc40 2024-05-09T09:19:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 -- 172.21.15.64:0/2214180084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 msgr2=0x564f33dd1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 0x564f33dd1740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000f480 tx=0x7fdc8000f940 comp rx=0 tx=0).stop 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 -- 172.21.15.64:0/2214180084 shutdown_connections 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dccc40 0x564f33dd1740 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 --2- 172.21.15.64:0/2214180084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33dcc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 -- 172.21.15.64:0/2214180084 >> 172.21.15.64:0/2214180084 conn(0x564f33dc78c0 msgr2=0x564f33dc7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.915+0000 7fdc99206740 1 -- 172.21.15.64:0/2214180084 shutdown_connections 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.916+0000 7fdc99206740 1 -- 172.21.15.64:0/2214180084 wait complete. 2024-05-09T09:19:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.916+0000 7fdc99206740 1 -- start start 2024-05-09T09:19:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.916+0000 7fdc99206740 1 Processor -- start 2024-05-09T09:19:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 0x564f33dd64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33dd6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 0x564f33dd64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 0x564f33dd64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43844/0 (socket says 172.21.15.64:43844) 2024-05-09T09:19:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f33dd4db0 con 0x564f33dccc40 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564f33ceb790 con 0x564f33dcc250 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564f33dd4f30 con 0x564f33dd1db0 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.917+0000 7fdc99206740 10 client.0 _ll_register_callbacks cb 0x564f33d26a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 -- 172.21.15.64:0/933223179 learned_addr learned my addr 172.21.15.64:0/933223179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33dd6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 -- 172.21.15.64:0/933223179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 msgr2=0x564f33ddda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 -- 172.21.15.64:0/933223179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 msgr2=0x564f33dd6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33dd6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 -- 172.21.15.64:0/933223179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f33a44210 con 0x564f33dcc250 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc877fe640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 0x564f33dd64d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdc78004e50 tx=0x7fdc780068f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ebba672089f34e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.918+0000 7fdc6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.919+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/933223179 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801b070 con 0x564f33dcc250 2024-05-09T09:19:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.919+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc8000e780 con 0x564f33dcc250 2024-05-09T09:19:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.919+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/933223179 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc78004020 con 0x564f33dcc250 2024-05-09T09:19:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.920+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/933223179 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc78027050 con 0x564f33dcc250 2024-05-09T09:19:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.920+0000 7fdc99206740 10 client.27539 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.920+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564f33b10620 con 0x564f33dcc250 2024-05-09T09:19:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.921+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/933223179 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc78004220 con 0x564f33dcc250 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.921+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/933223179 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdc78003c30 con 0x564f33dcc250 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.921+0000 7fdc99206740 10 client.27539 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.921+0000 7fdc99206740 10 client.27539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.921+0000 7fdc99206740 20 client.27539 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.921+0000 7fdc99206740 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33de1c90 0x564f33de40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.922+0000 7fdc99206740 4 client.27539 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.922+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdc8000e780 con 0x564f33de1c90 2024-05-09T09:19:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.922+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33de1c90 0x564f33de40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.923+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33de1c90 0x564f33de40c0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c45caf2e308687d server_cookie=b853b6c21ebb25fd in_seq=0 out_seq=0 2024-05-09T09:19:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.923+0000 7fdc6f7fe640 10 client.27539 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.923+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/933223179 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fdc8000e780 con 0x564f33de1c90 2024-05-09T09:19:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.923+0000 7fdc6f7fe640 10 client.27539 handle_command_reply: tid=0 2024-05-09T09:19:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.924+0000 7fdc99206740 1 client.27539 shutdown 2024-05-09T09:19:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.926+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 msgr2=0x564f33dd64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.926+0000 7fdc99206740 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 0x564f33dd64d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdc78004e50 tx=0x7fdc780068f0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 shutdown_connections 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33de1c90 0x564f33de40c0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33dd6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 --2- 172.21.15.64:0/933223179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dcc250 0x564f33dd64d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 >> 172.21.15.64:0/933223179 conn(0x564f33dc78c0 msgr2=0x564f33dc6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc6f7fe640 0 client.27539 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 shutdown_connections 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.927+0000 7fdc99206740 1 -- 172.21.15.64:0/933223179 wait complete. 2024-05-09T09:19:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.928+0000 7fdc99206740 20 client.27539 trim_cache size 0 max 16384 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.959+0000 7fdc99206740 1 Processor -- start 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.959+0000 7fdc99206740 1 -- start start 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.959+0000 7fdc99206740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dcc250 0x564f33dcb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.959+0000 7fdc99206740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33ddda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc99206740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc99206740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901c4350 con 0x564f33dccc40 2024-05-09T09:19:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc99206740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901c1c20 con 0x564f33dd1db0 2024-05-09T09:19:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc99206740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901bfa70 con 0x564f33dcc250 2024-05-09T09:19:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33ddda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33ddda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58068/0 (socket says 172.21.15.64:58068) 2024-05-09T09:19:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 -- 172.21.15.64:0/627312071 learned_addr learned my addr 172.21.15.64:0/627312071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 -- 172.21.15.64:0/627312071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dcc250 msgr2=0x564f33dcb030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dcc250 0x564f33dcb030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 -- 172.21.15.64:0/627312071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 msgr2=0x564f33dd60e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd60e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.960+0000 7fdc8ffff640 1 -- 172.21.15.64:0/627312071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f33dc5f00 con 0x564f33dccc40 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.961+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33ddda10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005c80 tx=0x7fdc8000e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74d66c4f7f93cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.961+0000 7fdc85ffb640 1 -- 172.21.15.64:0/627312071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80009700 con 0x564f33dccc40 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.961+0000 7fdc85ffb640 1 -- 172.21.15.64:0/627312071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc80028200 con 0x564f33dccc40 2024-05-09T09:19:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.961+0000 7fdc85ffb640 1 -- 172.21.15.64:0/627312071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800052d0 con 0x564f33dccc40 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 -- 172.21.15.64:0/627312071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 msgr2=0x564f33ddda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33ddda10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005c80 tx=0x7fdc8000e910 comp rx=0 tx=0).stop 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 -- 172.21.15.64:0/627312071 shutdown_connections 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dd1db0 0x564f33dd60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dccc40 0x564f33ddda10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 --2- 172.21.15.64:0/627312071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dcc250 0x564f33dcb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 -- 172.21.15.64:0/627312071 >> 172.21.15.64:0/627312071 conn(0x564f33dc78c0 msgr2=0x564f33dca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 -- 172.21.15.64:0/627312071 shutdown_connections 2024-05-09T09:19:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.962+0000 7fdc99206740 1 -- 172.21.15.64:0/627312071 wait complete. 2024-05-09T09:19:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.963+0000 7fdc99206740 1 -- start start 2024-05-09T09:19:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.963+0000 7fdc99206740 1 Processor -- start 2024-05-09T09:19:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc99206740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33ddaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc99206740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dccc40 0x564f33ddb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33ddaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc99206740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33ddaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58076/0 (socket says 172.21.15.64:58076) 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc877fe640 1 -- 172.21.15.64:0/1603659079 learned_addr learned my addr 172.21.15.64:0/1603659079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901c00e0 con 0x564f33dcc250 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901c1550 con 0x564f33dccc40 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901c07b0 con 0x564f33dd1db0 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.964+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dccc40 0x564f33ddb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc877fe640 1 -- 172.21.15.64:0/1603659079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 msgr2=0x564f33ddc410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc877fe640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc877fe640 1 -- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dccc40 msgr2=0x564f33ddb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc877fe640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dccc40 0x564f33ddb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc877fe640 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f33ddca00 con 0x564f33dcc250 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dccc40 0x564f33ddb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc877fe640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33ddaf10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdc78021680 tx=0x7fdc78004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7effd4311f8a7553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc99206740 10 client.0 _ll_register_callbacks cb 0x7fdc78004ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc80028200 con 0x564f33dcc250 2024-05-09T09:19:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc6f7fe640 10 client.18039 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/1603659079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801b050 con 0x564f33dcc250 2024-05-09T09:19:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/1603659079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc78021230 con 0x564f33dcc250 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.965+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/1603659079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc7800a160 con 0x564f33dcc250 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.966+0000 7fdc99206740 10 client.18039 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.966+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=45}) -- 0x564f33dddbc0 con 0x564f33dcc250 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.966+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/1603659079 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdc78004020 con 0x564f33dcc250 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.966+0000 7fdc99206740 10 client.18039 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.966+0000 7fdc99206740 10 client.18039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.966+0000 7fdc99206740 20 client.18039 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.967+0000 7fdc99206740 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33e48720 0x564f33e4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.967+0000 7fdc99206740 4 client.18039 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.967+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdc80028200 con 0x564f33e48720 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.967+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33e48720 0x564f33e4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.968+0000 7fdc8ffff640 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33e48720 0x564f33e4ab60 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c96b040bfda6ae server_cookie=20f015efe0a08ad0 in_seq=0 out_seq=0 2024-05-09T09:19:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:37.968+0000 7fdc6f7fe640 10 client.18039 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.001+0000 7fdc6f7fe640 1 -- 172.21.15.64:0/1603659079 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdc80028200 con 0x564f33e48720 2024-05-09T09:19:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.001+0000 7fdc6f7fe640 10 client.18039 handle_command_reply: tid=0 2024-05-09T09:19:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.002+0000 7fdc99206740 1 client.18039 shutdown 2024-05-09T09:19:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.002+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 msgr2=0x564f33ddaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33ddaf10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdc78021680 tx=0x7fdc78004d90 comp rx=0 tx=0).stop 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 shutdown_connections 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564f33e48720 0x564f33e4ab60 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f33dd1db0 0x564f33ddc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc6f7fe640 0 client.18039 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564f33dccc40 0x564f33ddb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 --2- 172.21.15.64:0/1603659079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f33dcc250 0x564f33ddaf10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.003+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 >> 172.21.15.64:0/1603659079 conn(0x564f33dc78c0 msgr2=0x564f33dd60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.004+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 shutdown_connections 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.004+0000 7fdc99206740 1 -- 172.21.15.64:0/1603659079 wait complete. 2024-05-09T09:19:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.004+0000 7fdc99206740 20 client.18039 trim_cache size 0 max 16384 2024-05-09T09:19:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:19:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:19:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:19:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc7c0a2df0 msgr2=0x7fdc7c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc7c0a2df0 0x7fdc7c0a52b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b75b0 tx=0x7fdc88085e50 comp rx=0 tx=0).stop 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 msgr2=0x7fdc901b66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901b66e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009780 tx=0x7fdc88032cd0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 shutdown_connections 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc7c0a2df0 0x7fdc7c0a52b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc9010de50 0x7fdc901b66e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc9010d490 0x7fdc90119880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 --2- 172.21.15.64:0/817267153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9010c2a0 0x7fdc90119340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 >> 172.21.15.64:0/817267153 conn(0x7fdc9006c470 msgr2=0x7fdc90108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.006+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 shutdown_connections 2024-05-09T09:19:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:38.007+0000 7fdc964fc640 1 -- 172.21.15.64:0/817267153 wait complete. 2024-05-09T09:19:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:38 smithi179 ceph-mon[34086]: pgmap v685: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-05-09T09:19:39.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:38 smithi139 ceph-mon[33599]: pgmap v685: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-05-09T09:19:39.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:38 smithi064 ceph-mon[30149]: pgmap v685: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-05-09T09:19:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:40 smithi139 ceph-mon[33599]: pgmap v686: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.11k op/s 2024-05-09T09:19:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:40 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:40 smithi064 ceph-mon[30149]: pgmap v686: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.11k op/s 2024-05-09T09:19:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:40 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:41.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:40 smithi179 ceph-mon[34086]: pgmap v686: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.11k op/s 2024-05-09T09:19:41.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:40 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:19:41.354 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 560.0 s, 850.6 tps, lat 118.482 ms stddev 92.102 2024-05-09T09:19:43.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:42 smithi139 ceph-mon[33599]: pgmap v687: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 818 op/s 2024-05-09T09:19:43.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:42 smithi064 ceph-mon[30149]: pgmap v687: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 818 op/s 2024-05-09T09:19:43.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:42 smithi179 ceph-mon[34086]: pgmap v687: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 818 op/s 2024-05-09T09:19:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:43 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:43 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:43 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:43 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:43 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:43 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:43 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:43 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:43 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:43 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:43 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:43 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:44.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:19:44.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:19:45.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.080+0000 7fdb6689e640 1 -- 172.21.15.64:0/3342341197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60108fe0 msgr2=0x7fdb60109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.080+0000 7fdb6689e640 1 --2- 172.21.15.64:0/3342341197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60108fe0 0x7fdb60109460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.081+0000 7fdb6689e640 1 -- 172.21.15.64:0/3342341197 shutdown_connections 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.081+0000 7fdb6689e640 1 --2- 172.21.15.64:0/3342341197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb601099a0 0x7fdb601102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.081+0000 7fdb6689e640 1 --2- 172.21.15.64:0/3342341197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60108fe0 0x7fdb60109460 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.081+0000 7fdb6689e640 1 --2- 172.21.15.64:0/3342341197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb60107e90 0x7fdb601082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.081+0000 7fdb6689e640 1 -- 172.21.15.64:0/3342341197 >> 172.21.15.64:0/3342341197 conn(0x7fdb60076e40 msgr2=0x7fdb60079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.081+0000 7fdb6689e640 1 -- 172.21.15.64:0/3342341197 shutdown_connections 2024-05-09T09:19:45.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.082+0000 7fdb6689e640 1 -- 172.21.15.64:0/3342341197 wait complete. 2024-05-09T09:19:45.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.082+0000 7fdb6689e640 1 Processor -- start 2024-05-09T09:19:45.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 -- start start 2024-05-09T09:19:45.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 0x7fdb6019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb60108fe0 0x7fdb6019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb601099a0 0x7fdb601a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60113a20 con 0x7fdb60108fe0 2024-05-09T09:19:45.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601138a0 con 0x7fdb601099a0 2024-05-09T09:19:45.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb6689e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60113ba0 con 0x7fdb60107e90 2024-05-09T09:19:45.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb5f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb60108fe0 0x7fdb6019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 0x7fdb6019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 0x7fdb6019e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43400/0 (socket says 172.21.15.64:43400) 2024-05-09T09:19:45.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb5ffff640 1 -- 172.21.15.64:0/162029 learned_addr learned my addr 172.21.15.64:0/162029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:45.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.083+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb601099a0 0x7fdb601a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5ffff640 1 -- 172.21.15.64:0/162029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb601099a0 msgr2=0x7fdb601a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb601099a0 0x7fdb601a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5ffff640 1 -- 172.21.15.64:0/162029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb60108fe0 msgr2=0x7fdb6019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb60108fe0 0x7fdb6019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5ffff640 1 -- 172.21.15.64:0/162029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb601a5cf0 con 0x7fdb60107e90 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb60108fe0 0x7fdb6019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 0x7fdb6019e120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400d950 tx=0x7fdb5400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb5d7fa640 1 -- 172.21.15.64:0/162029 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54017030 con 0x7fdb60107e90 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.084+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb601a5fe0 con 0x7fdb60107e90 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.085+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb601a6450 con 0x7fdb60107e90 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.085+0000 7fdb5d7fa640 1 -- 172.21.15.64:0/162029 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb54004020 con 0x7fdb60107e90 2024-05-09T09:19:45.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.085+0000 7fdb5d7fa640 1 -- 172.21.15.64:0/162029 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540047a0 con 0x7fdb60107e90 2024-05-09T09:19:45.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.088+0000 7fdb695aa740 1 Processor -- start 2024-05-09T09:19:45.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 -- start start 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa94350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa94920 0x562d8aa99420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d8a9f8d00 con 0x562d8aa94920 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562d8a9a04c0 con 0x562d8aa99a90 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.089+0000 7fdb695aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562d8a9f4cc0 con 0x562d8aa93f30 2024-05-09T09:19:45.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.090+0000 7fdb64e3b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa94350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.090+0000 7fdb64e3b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa94350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43404/0 (socket says 172.21.15.64:43404) 2024-05-09T09:19:45.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.090+0000 7fdb64e3b640 1 -- 172.21.15.64:0/3036574541 learned_addr learned my addr 172.21.15.64:0/3036574541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:45.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa94920 0x562d8aa99420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5d7fa640 1 -- 172.21.15.64:0/162029 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb54004940 con 0x7fdb60107e90 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5ffff640 1 -- 172.21.15.64:0/3036574541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 msgr2=0x562d8aa94350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa94350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5ffff640 1 -- 172.21.15.64:0/3036574541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 msgr2=0x562d8aa9bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.091+0000 7fdb5ffff640 1 -- 172.21.15.64:0/3036574541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8a7d67d0 con 0x562d8aa94920 2024-05-09T09:19:45.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.092+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa94920 0x562d8aa99420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdb540cc4a0 tx=0x7fdb540d2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4303d420d4a93e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.093+0000 7fdb5d7fa640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdb480a2c90 0x7fdb480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.093+0000 7fdb5d7fa640 1 -- 172.21.15.64:0/162029 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb540cedf0 con 0x7fdb60107e90 2024-05-09T09:19:45.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.093+0000 7fdb467fc640 1 -- 172.21.15.64:0/3036574541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5409abc0 con 0x562d8aa94920 2024-05-09T09:19:45.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.093+0000 7fdb467fc640 1 -- 172.21.15.64:0/3036574541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5409b960 con 0x562d8aa94920 2024-05-09T09:19:45.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.093+0000 7fdb467fc640 1 -- 172.21.15.64:0/3036574541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5409ada0 con 0x562d8aa94920 2024-05-09T09:19:45.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.094+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdb480a2c90 0x7fdb480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 -- 172.21.15.64:0/3036574541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa94920 msgr2=0x562d8aa99420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa94920 0x562d8aa99420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdb540cc4a0 tx=0x7fdb540d2de0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 -- 172.21.15.64:0/3036574541 shutdown_connections 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa94920 0x562d8aa99420 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 --2- 172.21.15.64:0/3036574541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa94350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 -- 172.21.15.64:0/3036574541 >> 172.21.15.64:0/3036574541 conn(0x562d8aa8f5a0 msgr2=0x562d8aa8f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdb480a2c90 0x7fdb480a5150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x562d8aa95180 tx=0x7fdb58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 -- 172.21.15.64:0/3036574541 shutdown_connections 2024-05-09T09:19:45.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.095+0000 7fdb695aa740 1 -- 172.21.15.64:0/3036574541 wait complete. 2024-05-09T09:19:45.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.096+0000 7fdb695aa740 1 -- start start 2024-05-09T09:19:45.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.096+0000 7fdb695aa740 1 Processor -- start 2024-05-09T09:19:45.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa93f30 0x562d8aa9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aaa5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d8aa9ca40 con 0x562d8aa93f30 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562d8a9b3b90 con 0x562d8aa99a90 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562d8aa9cbc0 con 0x562d8aaa3b30 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aaa5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aaa5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43412/0 (socket says 172.21.15.64:43412) 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 -- 172.21.15.64:0/332267626 learned_addr learned my addr 172.21.15.64:0/332267626 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb695aa740 10 client.0 _ll_register_callbacks cb 0x562d8a9efda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa93f30 0x562d8aa9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 -- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 msgr2=0x562d8aa9eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 -- 172.21.15.64:0/332267626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa93f30 msgr2=0x562d8aa9e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa93f30 0x562d8aa9e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.097+0000 7fdb64e3b640 1 -- 172.21.15.64:0/332267626 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8a70c3f0 con 0x562d8aaa3b30 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aaa5f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb58003f20 tx=0x7fdb58004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8badc081c9be44c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/332267626 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800f810 con 0x562d8aaa3b30 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5409b960 con 0x562d8aaa3b30 2024-05-09T09:19:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/332267626 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb5800f9b0 con 0x562d8aaa3b30 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/332267626 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdb5800fbb0 con 0x562d8aaa3b30 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb695aa740 10 client.27691 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.098+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562d8a6cc0f0 con 0x562d8aaa3b30 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/332267626 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5409b960 con 0x562d8aaa3b30 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/332267626 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdb58026070 con 0x562d8aaa3b30 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb695aa740 10 client.27691 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb695aa740 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb695aa740 20 client.27691 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb695aa740 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8aabb900 0x562d8aabdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb695aa740 4 client.27691 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d8aa06430 con 0x562d8aabb900 2024-05-09T09:19:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.099+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8aabb900 0x562d8aabdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.101+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8aabb900 0x562d8aabdd60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79cad521ffdc8d2 server_cookie=320f0309a6e573e4 in_seq=0 out_seq=0 2024-05-09T09:19:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.101+0000 7fdb3f7fe640 10 client.27691 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.103+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/332267626 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d8aa06430 con 0x562d8aabb900 2024-05-09T09:19:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.103+0000 7fdb3f7fe640 10 client.27691 handle_command_reply: tid=0 2024-05-09T09:19:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.104+0000 7fdb695aa740 1 client.27691 shutdown 2024-05-09T09:19:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.104+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 msgr2=0x562d8aaa5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.104+0000 7fdb695aa740 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aaa5f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb58003f20 tx=0x7fdb58004ab0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.104+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 shutdown_connections 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.104+0000 7fdb695aa740 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8aabb900 0x562d8aabdd60 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb3f7fe640 0 client.27691 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aaa5f20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa99a90 0x562d8aa9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 1 --2- 172.21.15.64:0/332267626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa93f30 0x562d8aa9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 >> 172.21.15.64:0/332267626 conn(0x562d8aa8f5a0 msgr2=0x562d8aa8e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 shutdown_connections 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 1 -- 172.21.15.64:0/332267626 wait complete. 2024-05-09T09:19:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.105+0000 7fdb695aa740 20 client.27691 trim_cache size 0 max 16384 2024-05-09T09:19:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.137+0000 7fdb695aa740 1 Processor -- start 2024-05-09T09:19:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.137+0000 7fdb695aa740 1 -- start start 2024-05-09T09:19:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.137+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.137+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aa96470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aaa3b30 0x562d8aa92280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb695aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c012480 con 0x562d8aa99a90 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb695aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c00fd50 con 0x562d8aaa3b30 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb695aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c00dba0 con 0x562d8aa93f30 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa95f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43424/0 (socket says 172.21.15.64:43424) 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 -- 172.21.15.64:0/2123681859 learned_addr learned my addr 172.21.15.64:0/2123681859 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aa96470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aaa3b30 0x562d8aa92280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 -- 172.21.15.64:0/2123681859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aaa3b30 msgr2=0x562d8aa92280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aaa3b30 0x562d8aa92280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 -- 172.21.15.64:0/2123681859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 msgr2=0x562d8aa96470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aa96470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.138+0000 7fdb5ffff640 1 -- 172.21.15.64:0/2123681859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8aaba1c0 con 0x562d8aa93f30 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.139+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa95f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdb540a39a0 tx=0x7fdb540cc490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3343312cb396f5e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.139+0000 7fdb45ffb640 1 -- 172.21.15.64:0/2123681859 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5409b030 con 0x562d8aa93f30 2024-05-09T09:19:45.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.139+0000 7fdb45ffb640 1 -- 172.21.15.64:0/2123681859 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5409b960 con 0x562d8aa93f30 2024-05-09T09:19:45.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.139+0000 7fdb45ffb640 1 -- 172.21.15.64:0/2123681859 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54097500 con 0x562d8aa93f30 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.140+0000 7fdb695aa740 1 -- 172.21.15.64:0/2123681859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 msgr2=0x562d8aa95f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.140+0000 7fdb695aa740 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa95f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdb540a39a0 tx=0x7fdb540cc490 comp rx=0 tx=0).stop 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.140+0000 7fdb695aa740 1 -- 172.21.15.64:0/2123681859 shutdown_connections 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.141+0000 7fdb695aa740 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aaa3b30 0x562d8aa92280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.141+0000 7fdb695aa740 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aa96470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.141+0000 7fdb695aa740 1 --2- 172.21.15.64:0/2123681859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aa93f30 0x562d8aa95f30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.141+0000 7fdb695aa740 1 -- 172.21.15.64:0/2123681859 >> 172.21.15.64:0/2123681859 conn(0x562d8aa8f5a0 msgr2=0x562d8aa95390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.141+0000 7fdb695aa740 1 -- 172.21.15.64:0/2123681859 shutdown_connections 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.141+0000 7fdb695aa740 1 -- 172.21.15.64:0/2123681859 wait complete. 2024-05-09T09:19:45.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.142+0000 7fdb695aa740 1 -- start start 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.142+0000 7fdb695aa740 1 Processor -- start 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 0x562d8aaccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb695aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aacd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 0x562d8aaccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 0x562d8aaccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43514/0 (socket says 172.21.15.64:43514) 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb5ffff640 1 -- 172.21.15.64:0/1666039710 learned_addr learned my addr 172.21.15.64:0/1666039710 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aace440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c00e210 con 0x562d8aa99a90 2024-05-09T09:19:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.143+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c00f680 con 0x562d8aa93f30 2024-05-09T09:19:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aace440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c00e8e0 con 0x562d8aaa3b30 2024-05-09T09:19:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aacd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb5ffff640 1 -- 172.21.15.64:0/1666039710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 msgr2=0x562d8aace440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aace440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb5ffff640 1 -- 172.21.15.64:0/1666039710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 msgr2=0x562d8aacd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aacd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb5ffff640 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8aacea30 con 0x562d8aa93f30 2024-05-09T09:19:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.144+0000 7fdb64e3b640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aace440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.145+0000 7fdb695aa740 10 client.0 _ll_register_callbacks cb 0x7fdb58009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:45.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.145+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5409b960 con 0x562d8aa93f30 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb5ffff640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 0x562d8aaccef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb54097c30 tx=0x7fdb540cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e5848e2d2d29194 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb3f7fe640 10 client.27557 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/1666039710 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54097500 con 0x562d8aa93f30 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/1666039710 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb540b4200 con 0x562d8aa93f30 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/1666039710 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdb5409b030 con 0x562d8aa93f30 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb695aa740 10 client.27557 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:45.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.146+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=45}) -- 0x562d8ab21de0 con 0x562d8aa93f30 2024-05-09T09:19:45.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/1666039710 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdb540978d0 con 0x562d8aa93f30 2024-05-09T09:19:45.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb695aa740 10 client.27557 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:45.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb695aa740 10 client.27557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb695aa740 20 client.27557 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8ab22100 0x562d8ab24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb695aa740 4 client.27557 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d8aa00780 con 0x562d8ab22100 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.147+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8ab22100 0x562d8ab24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.148+0000 7fdb5f7fe640 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8ab22100 0x562d8ab24540 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c792aaed3de00cb5 server_cookie=719393d75954d36 in_seq=0 out_seq=0 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.148+0000 7fdb3f7fe640 10 client.27557 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.148+0000 7fdb3f7fe640 1 -- 172.21.15.64:0/1666039710 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d8aa00780 con 0x562d8ab22100 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.148+0000 7fdb3f7fe640 10 client.27557 handle_command_reply: tid=0 2024-05-09T09:19:45.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.148+0000 7fdb695aa740 1 client.27557 shutdown 2024-05-09T09:19:45.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.150+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 msgr2=0x562d8aaccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.150+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 0x562d8aaccef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb54097c30 tx=0x7fdb540cf040 comp rx=0 tx=0).stop 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 shutdown_connections 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562d8ab22100 0x562d8ab24540 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d8aaa3b30 0x562d8aace440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb3f7fe640 0 client.27557 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d8aa99a90 0x562d8aacd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb695aa740 1 --2- 172.21.15.64:0/1666039710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d8aa93f30 0x562d8aaccef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.151+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 >> 172.21.15.64:0/1666039710 conn(0x562d8aa8f5a0 msgr2=0x562d8aa9e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.153+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 shutdown_connections 2024-05-09T09:19:45.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.153+0000 7fdb695aa740 1 -- 172.21.15.64:0/1666039710 wait complete. 2024-05-09T09:19:45.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.154+0000 7fdb695aa740 20 client.27557 trim_cache size 0 max 16384 2024-05-09T09:19:45.154 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:19:45.154 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:19:45.154 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:19:45.154 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:19:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.156+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdb480a2c90 msgr2=0x7fdb480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.156+0000 7fdb6689e640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdb480a2c90 0x7fdb480a5150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x562d8aa95180 tx=0x7fdb58009500 comp rx=0 tx=0).stop 2024-05-09T09:19:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.156+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 msgr2=0x7fdb6019e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.156+0000 7fdb6689e640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 0x7fdb6019e120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400d950 tx=0x7fdb5400de10 comp rx=0 tx=0).stop 2024-05-09T09:19:45.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.158+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 shutdown_connections 2024-05-09T09:19:45.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.158+0000 7fdb6689e640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdb480a2c90 0x7fdb480a5150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.158+0000 7fdb6689e640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb601099a0 0x7fdb601a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.158+0000 7fdb6689e640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb60108fe0 0x7fdb6019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.158+0000 7fdb6689e640 1 --2- 172.21.15.64:0/162029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb60107e90 0x7fdb6019e120 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.158+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 >> 172.21.15.64:0/162029 conn(0x7fdb60076e40 msgr2=0x7fdb6010c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.160+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 shutdown_connections 2024-05-09T09:19:45.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.160+0000 7fdb6689e640 1 -- 172.21.15.64:0/162029 wait complete. 2024-05-09T09:19:45.194 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:19:45.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:19:45.195 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:44 smithi064 ceph-mon[30149]: pgmap v688: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 972 op/s 2024-05-09T09:19:45.195 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:44 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:45.195 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:44 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:44 smithi179 ceph-mon[34086]: pgmap v688: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 972 op/s 2024-05-09T09:19:45.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:44 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:45.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:44 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:44 smithi139 ceph-mon[33599]: pgmap v688: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 972 op/s 2024-05-09T09:19:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:44 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:44 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:45.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.390+0000 7fe172129640 1 -- 172.21.15.64:0/249455645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 msgr2=0x7fe16c1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.390+0000 7fe172129640 1 --2- 172.21.15.64:0/249455645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 0x7fe16c1082b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe15c01a910 tx=0x7fe15c03b430 comp rx=0 tx=0).stop 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 -- 172.21.15.64:0/249455645 shutdown_connections 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 --2- 172.21.15.64:0/249455645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c1099a0 0x7fe16c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 --2- 172.21.15.64:0/249455645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe16c108fe0 0x7fe16c109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 --2- 172.21.15.64:0/249455645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 0x7fe16c1082b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 -- 172.21.15.64:0/249455645 >> 172.21.15.64:0/249455645 conn(0x7fe16c076e50 msgr2=0x7fe16c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 -- 172.21.15.64:0/249455645 shutdown_connections 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.391+0000 7fe172129640 1 -- 172.21.15.64:0/249455645 wait complete. 2024-05-09T09:19:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.392+0000 7fe172129640 1 Processor -- start 2024-05-09T09:19:45.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 -- start start 2024-05-09T09:19:45.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 0x7fe16c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 0x7fe16c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe16c1099a0 0x7fe16c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c07cbb0 con 0x7fe16c107e90 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c07ca30 con 0x7fe16c1099a0 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.393+0000 7fe172129640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c07cd30 con 0x7fe16c108fe0 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.394+0000 7fe16affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 0x7fe16c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.394+0000 7fe16affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 0x7fe16c1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43450/0 (socket says 172.21.15.64:43450) 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.394+0000 7fe16affd640 1 -- 172.21.15.64:0/3130046783 learned_addr learned my addr 172.21.15.64:0/3130046783 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.394+0000 7fe16b7fe640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 0x7fe16c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.394+0000 7fe16bfff640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe16c1099a0 0x7fe16c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe16affd640 1 -- 172.21.15.64:0/3130046783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe16c1099a0 msgr2=0x7fe16c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe16affd640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe16c1099a0 0x7fe16c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe16affd640 1 -- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 msgr2=0x7fe16c195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe16affd640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 0x7fe16c195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe16affd640 1 -- 172.21.15.64:0/3130046783 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c19d500 con 0x7fe16c108fe0 2024-05-09T09:19:45.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe16affd640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 0x7fe16c1963b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e9f0 tx=0x7fe16000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16000cd80 con 0x7fe16c108fe0 2024-05-09T09:19:45.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.395+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1600035f0 con 0x7fe16c108fe0 2024-05-09T09:19:45.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.396+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c19d790 con 0x7fe16c108fe0 2024-05-09T09:19:45.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.396+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160010690 con 0x7fe16c108fe0 2024-05-09T09:19:45.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.396+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c19dcb0 con 0x7fe16c108fe0 2024-05-09T09:19:45.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.399+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16c068aa0 con 0x7fe16c108fe0 2024-05-09T09:19:45.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.400+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe1600037a0 con 0x7fe16c108fe0 2024-05-09T09:19:45.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.402+0000 7fe168ff9640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1540a2d60 0x7fe1540a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:45.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.403+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1600cf770 con 0x7fe16c108fe0 2024-05-09T09:19:45.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.404+0000 7fe16b7fe640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1540a2d60 0x7fe1540a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:45.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.405+0000 7fe16b7fe640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1540a2d60 0x7fe1540a5220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c004e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:45.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.408+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe16009c2e0 con 0x7fe16c108fe0 2024-05-09T09:19:45.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.863+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe16c062430 con 0x7fe16c108fe0 2024-05-09T09:19:45.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.866+0000 7fe168ff9640 1 -- 172.21.15.64:0/3130046783 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11879 (secure 0 0 0) 0x7fe16009bbe0 con 0x7fe16c108fe0 2024-05-09T09:19:45.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:19:45.867 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:17:16.247105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:19:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 45 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1540a2d60 msgr2=0x7fe1540a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1540a2d60 0x7fe1540a5220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c004e20 comp rx=0 tx=0).stop 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 msgr2=0x7fe16c1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 0x7fe16c1963b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e9f0 tx=0x7fe16000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 shutdown_connections 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1540a2d60 0x7fe1540a5220 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe16c1099a0 0x7fe16c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe16c108fe0 0x7fe16c1963b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 --2- 172.21.15.64:0/3130046783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe16c107e90 0x7fe16c195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.870+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 >> 172.21.15.64:0/3130046783 conn(0x7fe16c076e50 msgr2=0x7fe16c07f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.871+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 shutdown_connections 2024-05-09T09:19:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:45.871+0000 7fe172129640 1 -- 172.21.15.64:0/3130046783 wait complete. 2024-05-09T09:19:45.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:19:46.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.074+0000 7efed4210640 1 -- 172.21.15.64:0/845983541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc102a50 msgr2=0x7efecc104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.074+0000 7efed4210640 1 --2- 172.21.15.64:0/845983541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc102a50 0x7efecc104ec0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efec801a910 tx=0x7efec803b430 comp rx=0 tx=0).stop 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.074+0000 7efed4210640 1 -- 172.21.15.64:0/845983541 shutdown_connections 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.074+0000 7efed4210640 1 --2- 172.21.15.64:0/845983541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc105400 0x7efecc105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.075+0000 7efed4210640 1 --2- 172.21.15.64:0/845983541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc102a50 0x7efecc104ec0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.075+0000 7efed4210640 1 --2- 172.21.15.64:0/845983541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc1000e0 0x7efecc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.075+0000 7efed4210640 1 -- 172.21.15.64:0/845983541 >> 172.21.15.64:0/845983541 conn(0x7efecc0fbce0 msgr2=0x7efecc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.075+0000 7efed4210640 1 -- 172.21.15.64:0/845983541 shutdown_connections 2024-05-09T09:19:46.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.075+0000 7efed4210640 1 -- 172.21.15.64:0/845983541 wait complete. 2024-05-09T09:19:46.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.077+0000 7efed4210640 1 Processor -- start 2024-05-09T09:19:46.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 -- start start 2024-05-09T09:19:46.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc1000e0 0x7efecc195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc102a50 0x7efecc195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc105400 0x7efecc19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1091c0 con 0x7efecc102a50 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efecc109040 con 0x7efecc1000e0 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.078+0000 7efed4210640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efecc109340 con 0x7efecc105400 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed27ad640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc105400 0x7efecc19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed27ad640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc105400 0x7efecc19cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43462/0 (socket says 172.21.15.64:43462) 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed17ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc102a50 0x7efecc195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed27ad640 1 -- 172.21.15.64:0/796498265 learned_addr learned my addr 172.21.15.64:0/796498265 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed1fac640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc1000e0 0x7efecc195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed17ab640 1 -- 172.21.15.64:0/796498265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc105400 msgr2=0x7efecc19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed17ab640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc105400 0x7efecc19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed17ab640 1 -- 172.21.15.64:0/796498265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc1000e0 msgr2=0x7efecc195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed17ab640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc1000e0 0x7efecc195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.079+0000 7efed17ab640 1 -- 172.21.15.64:0/796498265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc19d570 con 0x7efecc102a50 2024-05-09T09:19:46.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.080+0000 7efed17ab640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc102a50 0x7efecc195e90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7efec803b950 tx=0x7efec8005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.080+0000 7efec2ffd640 1 -- 172.21.15.64:0/796498265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec804f070 con 0x7efecc102a50 2024-05-09T09:19:46.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.080+0000 7efec2ffd640 1 -- 172.21.15.64:0/796498265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec8040e50 con 0x7efecc102a50 2024-05-09T09:19:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.080+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc19d800 con 0x7efecc102a50 2024-05-09T09:19:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.080+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc19ddc0 con 0x7efecc102a50 2024-05-09T09:19:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.081+0000 7efec2ffd640 1 -- 172.21.15.64:0/796498265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8046030 con 0x7efecc102a50 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.084+0000 7efed6ff3740 1 Processor -- start 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.085+0000 7efec2ffd640 1 -- 172.21.15.64:0/796498265 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efec8052070 con 0x7efecc102a50 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.085+0000 7efed6ff3740 1 -- start start 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f31c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31c930 0x56400f321430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f323f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed6ff3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56400f280c40 con 0x56400f31bf40 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed6ff3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56400f228580 con 0x56400f31c930 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed6ff3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56400f27cc00 con 0x56400f321aa0 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed27ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f31c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed27ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f31c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51850/0 (socket says 172.21.15.64:51850) 2024-05-09T09:19:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.086+0000 7efed27ad640 1 -- 172.21.15.64:0/3749329974 learned_addr learned my addr 172.21.15.64:0/3749329974 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efed27ad640 1 -- 172.21.15.64:0/3749329974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 msgr2=0x56400f323f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efed27ad640 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f323f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efed27ad640 1 -- 172.21.15.64:0/3749329974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31c930 msgr2=0x56400f321430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efed27ad640 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31c930 0x56400f321430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efed27ad640 1 -- 172.21.15.64:0/3749329974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400ef943f0 con 0x56400f31bf40 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efed27ad640 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f31c360 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efec400e1c0 tx=0x7efec4009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f737cfbd07504c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.087+0000 7efec2ffd640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efeb80a2c90 0x7efeb80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.088+0000 7efeb7fff640 1 -- 172.21.15.64:0/3749329974 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4018020 con 0x56400f31bf40 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.088+0000 7efed1fac640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efeb80a2c90 0x7efeb80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.088+0000 7efeb7fff640 1 -- 172.21.15.64:0/3749329974 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec4014b30 con 0x56400f31bf40 2024-05-09T09:19:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.088+0000 7efec2ffd640 1 -- 172.21.15.64:0/796498265 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efec81006d0 con 0x7efecc102a50 2024-05-09T09:19:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.089+0000 7efeb7fff640 1 -- 172.21.15.64:0/3749329974 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4013c10 con 0x56400f31bf40 2024-05-09T09:19:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.089+0000 7efed1fac640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efeb80a2c90 0x7efeb80a5150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x56400f31cd30 tx=0x7efebc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 -- 172.21.15.64:0/3749329974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 msgr2=0x56400f31c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f31c360 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efec400e1c0 tx=0x7efec4009710 comp rx=0 tx=0).stop 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 -- 172.21.15.64:0/3749329974 shutdown_connections 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f323f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31c930 0x56400f321430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 --2- 172.21.15.64:0/3749329974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f31c360 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.090+0000 7efed6ff3740 1 -- 172.21.15.64:0/3749329974 >> 172.21.15.64:0/3749329974 conn(0x56400f3175b0 msgr2=0x56400f3179b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.091+0000 7efed6ff3740 1 -- 172.21.15.64:0/3749329974 shutdown_connections 2024-05-09T09:19:46.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.091+0000 7efed6ff3740 1 -- 172.21.15.64:0/3749329974 wait complete. 2024-05-09T09:19:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.092+0000 7efed6ff3740 1 -- start start 2024-05-09T09:19:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.092+0000 7efed6ff3740 1 Processor -- start 2024-05-09T09:19:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.093+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 0x56400f326270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.093+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f3267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed1fac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 0x56400f326270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed1fac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 0x56400f326270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43482/0 (socket says 172.21.15.64:43482) 2024-05-09T09:19:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed17ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f3267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed17ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f3267b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51864/0 (socket says 172.21.15.64:51864) 2024-05-09T09:19:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed1fac640 1 -- 172.21.15.64:0/2141780507 learned_addr learned my addr 172.21.15.64:0/2141780507 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56400f324aa0 con 0x56400f31c930 2024-05-09T09:19:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56400f23bc40 con 0x56400f321aa0 2024-05-09T09:19:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56400f324c20 con 0x56400f31bf40 2024-05-09T09:19:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.094+0000 7efed6ff3740 10 client.0 _ll_register_callbacks cb 0x56400ef93be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed27ad640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed1fac640 1 -- 172.21.15.64:0/2141780507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 msgr2=0x56400f32d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed1fac640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed1fac640 1 -- 172.21.15.64:0/2141780507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 msgr2=0x56400f3267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed1fac640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f3267b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed1fac640 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400f05e7d0 con 0x56400f31bf40 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed27ad640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed17ab640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f3267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efed1fac640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 0x56400f326270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efebc011490 tx=0x7efebc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e14ef55488745e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efeb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.095+0000 7efeb57fa640 1 -- 172.21.15.64:0/2141780507 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc01e830 con 0x56400f31bf40 2024-05-09T09:19:46.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.096+0000 7efeb57fa640 1 -- 172.21.15.64:0/2141780507 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efebc01e9d0 con 0x56400f31bf40 2024-05-09T09:19:46.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.096+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec4014b30 con 0x56400f31bf40 2024-05-09T09:19:46.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efeb57fa640 1 -- 172.21.15.64:0/2141780507 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efebc02f050 con 0x56400f31bf40 2024-05-09T09:19:46.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efed6ff3740 10 client.27709 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:46.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56400f330c10 con 0x56400f31bf40 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efeb57fa640 1 -- 172.21.15.64:0/2141780507 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec4014b30 con 0x56400f31bf40 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efeb57fa640 1 -- 172.21.15.64:0/2141780507 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7efebc01cc00 con 0x56400f31bf40 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efed6ff3740 10 client.27709 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efed6ff3740 10 client.27709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efed6ff3740 20 client.27709 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.097+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f3347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.098+0000 7efed6ff3740 4 client.27709 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.098+0000 7efed17ab640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f3347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.098+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56400f28e370 con 0x56400f332360 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.099+0000 7efed17ab640 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f3347c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d8b0464ad705abf server_cookie=5c8eccf25428bb23 in_seq=0 out_seq=0 2024-05-09T09:19:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.099+0000 7efeb57fa640 10 client.27709 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.100+0000 7efeb57fa640 1 -- 172.21.15.64:0/2141780507 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56400f28e370 con 0x56400f332360 2024-05-09T09:19:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.100+0000 7efeb57fa640 10 client.27709 handle_command_reply: tid=0 2024-05-09T09:19:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.100+0000 7efed6ff3740 1 client.27709 shutdown 2024-05-09T09:19:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.101+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 msgr2=0x56400f326270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.101+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 0x56400f326270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efebc011490 tx=0x7efebc004730 comp rx=0 tx=0).stop 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 shutdown_connections 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f3347c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efeb57fa640 0 client.27709 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f3267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 --2- 172.21.15.64:0/2141780507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31bf40 0x56400f326270 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 >> 172.21.15.64:0/2141780507 conn(0x56400f3175b0 msgr2=0x56400f3167b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 shutdown_connections 2024-05-09T09:19:46.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.102+0000 7efed6ff3740 1 -- 172.21.15.64:0/2141780507 wait complete. 2024-05-09T09:19:46.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.103+0000 7efed6ff3740 20 client.27709 trim_cache size 0 max 16384 2024-05-09T09:19:46.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.126+0000 7efed6ff3740 1 Processor -- start 2024-05-09T09:19:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.126+0000 7efed6ff3740 1 -- start start 2024-05-09T09:19:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31bf40 0x56400f31db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f31e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f32d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed6ff3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0012490 con 0x56400f31c930 2024-05-09T09:19:46.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed6ff3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efeb000fd60 con 0x56400f31bf40 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed6ff3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efeb000dbb0 con 0x56400f321aa0 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed17ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f31e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed17ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f31e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51876/0 (socket says 172.21.15.64:51876) 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed17ab640 1 -- 172.21.15.64:0/1088180209 learned_addr learned my addr 172.21.15.64:0/1088180209 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed1fac640 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31bf40 0x56400f31db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.127+0000 7efed27ad640 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f32d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.128+0000 7efed1fac640 1 -- 172.21.15.64:0/1088180209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 msgr2=0x56400f32d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.128+0000 7efed1fac640 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f32d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.128+0000 7efed1fac640 1 -- 172.21.15.64:0/1088180209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 msgr2=0x56400f31e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.128+0000 7efed1fac640 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f31e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.128+0000 7efed1fac640 1 -- 172.21.15.64:0/1088180209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400f318320 con 0x56400f31bf40 2024-05-09T09:19:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.128+0000 7efed1fac640 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31bf40 0x56400f31db50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efebc01d5c0 tx=0x7efebc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53514225aa57ab66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.129+0000 7efeb77fe640 1 -- 172.21.15.64:0/1088180209 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc038070 con 0x56400f31bf40 2024-05-09T09:19:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.129+0000 7efeb77fe640 1 -- 172.21.15.64:0/1088180209 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efebc010c10 con 0x56400f31bf40 2024-05-09T09:19:46.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.129+0000 7efeb77fe640 1 -- 172.21.15.64:0/1088180209 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc01ddc0 con 0x56400f31bf40 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.130+0000 7efed6ff3740 1 -- 172.21.15.64:0/1088180209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31bf40 msgr2=0x56400f31db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.130+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31bf40 0x56400f31db50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efebc01d5c0 tx=0x7efebc0046c0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 -- 172.21.15.64:0/1088180209 shutdown_connections 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f321aa0 0x56400f32d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31c930 0x56400f31e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1088180209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f31bf40 0x56400f31db50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 -- 172.21.15.64:0/1088180209 >> 172.21.15.64:0/1088180209 conn(0x56400f3175b0 msgr2=0x56400f317990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 -- 172.21.15.64:0/1088180209 shutdown_connections 2024-05-09T09:19:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.131+0000 7efed6ff3740 1 -- 172.21.15.64:0/1088180209 wait complete. 2024-05-09T09:19:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.132+0000 7efed6ff3740 1 -- start start 2024-05-09T09:19:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.132+0000 7efed6ff3740 1 Processor -- start 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f32ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31c930 0x56400f32b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efeb000e220 con 0x56400f31bf40 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efeb000f690 con 0x56400f321aa0 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efeb000e8f0 con 0x56400f31c930 2024-05-09T09:19:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed6ff3740 10 client.0 _ll_register_callbacks cb 0x7efebc00b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed1fac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f32ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed1fac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f32ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51878/0 (socket says 172.21.15.64:51878) 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed1fac640 1 -- 172.21.15.64:0/1481747506 learned_addr learned my addr 172.21.15.64:0/1481747506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed17ab640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31c930 0x56400f32b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.133+0000 7efed27ad640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efed27ad640 1 -- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31c930 msgr2=0x56400f32b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efed27ad640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31c930 0x56400f32b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efed27ad640 1 -- 172.21.15.64:0/1481747506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 msgr2=0x56400f32ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efed27ad640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f32ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efed27ad640 1 -- 172.21.15.64:0/1481747506 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400f32c660 con 0x56400f321aa0 2024-05-09T09:19:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efed27ad640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32c070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efec400f010 tx=0x7efec4013870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1202699670f1bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efeb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efeb57fa640 1 -- 172.21.15.64:0/1481747506 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4018040 con 0x56400f321aa0 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.134+0000 7efeb57fa640 1 -- 172.21.15.64:0/1481747506 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efec4013df0 con 0x56400f321aa0 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.135+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56400f2886c0 con 0x56400f321aa0 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.135+0000 7efeb57fa640 1 -- 172.21.15.64:0/1481747506 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efec4009ce0 con 0x56400f321aa0 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.135+0000 7efed6ff3740 10 client.27578 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.135+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56400f32d9d0 con 0x56400f321aa0 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.136+0000 7efeb57fa640 1 -- 172.21.15.64:0/1481747506 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec4027920 con 0x56400f321aa0 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.136+0000 7efeb57fa640 1 -- 172.21.15.64:0/1481747506 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7efec4014ce0 con 0x56400f321aa0 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.136+0000 7efed6ff3740 10 client.27578 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.136+0000 7efed6ff3740 10 client.27578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.136+0000 7efed6ff3740 20 client.27578 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.137+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f39abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.137+0000 7efed6ff3740 4 client.27578 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:19:46.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.137+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56400f2886c0 con 0x56400f332360 2024-05-09T09:19:46.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.137+0000 7efed1fac640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f39abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.138+0000 7efed1fac640 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f39abc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0d834ffc1a11497 server_cookie=9f776c1934d8bafd in_seq=0 out_seq=0 2024-05-09T09:19:46.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.138+0000 7efeb57fa640 10 client.27578 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:46.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.141+0000 7efeb57fa640 1 -- 172.21.15.64:0/1481747506 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56400f2886c0 con 0x56400f332360 2024-05-09T09:19:46.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.141+0000 7efeb57fa640 10 client.27578 handle_command_reply: tid=0 2024-05-09T09:19:46.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.141+0000 7efed6ff3740 1 client.27578 shutdown 2024-05-09T09:19:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.145+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 msgr2=0x56400f32c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.145+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32c070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efec400f010 tx=0x7efec4013870 comp rx=0 tx=0).stop 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 shutdown_connections 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x56400f332360 0x56400f39abc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56400f321aa0 0x56400f32c070 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56400f31c930 0x56400f32b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efed6ff3740 1 --2- 172.21.15.64:0/1481747506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56400f31bf40 0x56400f32ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 >> 172.21.15.64:0/1481747506 conn(0x56400f3175b0 msgr2=0x56400f315f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.146+0000 7efeb57fa640 0 client.27578 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:19:46.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.147+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 shutdown_connections 2024-05-09T09:19:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.148+0000 7efed6ff3740 1 -- 172.21.15.64:0/1481747506 wait complete. 2024-05-09T09:19:46.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.149+0000 7efed6ff3740 20 client.27578 trim_cache size 0 max 16384 2024-05-09T09:19:46.151 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:19:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.152+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efeb80a2c90 msgr2=0x7efeb80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.152+0000 7efed4210640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efeb80a2c90 0x7efeb80a5150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x56400f31cd30 tx=0x7efebc00b390 comp rx=0 tx=0).stop 2024-05-09T09:19:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.152+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc102a50 msgr2=0x7efecc195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.152+0000 7efed4210640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc102a50 0x7efecc195e90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7efec803b950 tx=0x7efec8005150 comp rx=0 tx=0).stop 2024-05-09T09:19:46.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:46.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:46.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:45 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:46.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:45 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3130046783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:19:46.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.154+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 shutdown_connections 2024-05-09T09:19:46.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.154+0000 7efed4210640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efeb80a2c90 0x7efeb80a5150 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.154+0000 7efed4210640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efecc105400 0x7efecc19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.154+0000 7efed4210640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efecc102a50 0x7efecc195e90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.154+0000 7efed4210640 1 --2- 172.21.15.64:0/796498265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc1000e0 0x7efecc195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.154+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 >> 172.21.15.64:0/796498265 conn(0x7efecc0fbce0 msgr2=0x7efecc0fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.156+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 shutdown_connections 2024-05-09T09:19:46.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.157+0000 7efed4210640 1 -- 172.21.15.64:0/796498265 wait complete. 2024-05-09T09:19:46.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:19:46.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:46.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:46.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:45 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:46.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:45 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3130046783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:19:46.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:19:46.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:19:46.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:45 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:19:46.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:45 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3130046783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:19:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 565.0 s, 836.8 tps, lat 119.333 ms stddev 74.857 2024-05-09T09:19:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.381+0000 7f9238522640 1 -- 172.21.15.64:0/126806645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010db70 msgr2=0x7f923010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.381+0000 7f9238522640 1 --2- 172.21.15.64:0/126806645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010db70 0x7f923010e020 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9228009ca0 tx=0x7f922802a8e0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.381+0000 7f9238522640 1 -- 172.21.15.64:0/126806645 shutdown_connections 2024-05-09T09:19:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.381+0000 7f9238522640 1 --2- 172.21.15.64:0/126806645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010db70 0x7f923010e020 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.381+0000 7f9238522640 1 --2- 172.21.15.64:0/126806645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f923010b1a0 0x7f923010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.381+0000 7f9238522640 1 --2- 172.21.15.64:0/126806645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9230108830 0x7f923010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.382+0000 7f9238522640 1 -- 172.21.15.64:0/126806645 >> 172.21.15.64:0/126806645 conn(0x7f923006e6c0 msgr2=0x7f9230070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.382+0000 7f9238522640 1 -- 172.21.15.64:0/126806645 shutdown_connections 2024-05-09T09:19:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.382+0000 7f9238522640 1 -- 172.21.15.64:0/126806645 wait complete. 2024-05-09T09:19:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.384+0000 7f9238522640 1 Processor -- start 2024-05-09T09:19:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.385+0000 7f9238522640 1 -- start start 2024-05-09T09:19:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.385+0000 7f9238522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 0x7f92301a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.385+0000 7f9238522640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010b1a0 0x7f92301a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f9238522640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f923010db70 0x7f92301ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 0x7f92301a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 0x7f92301a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43630/0 (socket says 172.21.15.64:43630) 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f9238522640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f923011c270 con 0x7f923010b1a0 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f9238522640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f923011c0f0 con 0x7f9230108830 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f9238522640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f923011c3f0 con 0x7f923010db70 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 -- 172.21.15.64:0/2734222769 learned_addr learned my addr 172.21.15.64:0/2734222769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f9235abd640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010b1a0 0x7f92301a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f9236abf640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f923010db70 0x7f92301ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 -- 172.21.15.64:0/2734222769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f923010db70 msgr2=0x7f92301ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f923010db70 0x7f92301ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 -- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010b1a0 msgr2=0x7f92301a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010b1a0 0x7f92301a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.386+0000 7f92362be640 1 -- 172.21.15.64:0/2734222769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301ae4d0 con 0x7f9230108830 2024-05-09T09:19:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.387+0000 7f92362be640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 0x7f92301a68b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f922000b450 tx=0x7f922000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.388+0000 7f92277fe640 1 -- 172.21.15.64:0/2734222769 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220015020 con 0x7f9230108830 2024-05-09T09:19:46.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.388+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301ae7c0 con 0x7f9230108830 2024-05-09T09:19:46.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.388+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301aed10 con 0x7f9230108830 2024-05-09T09:19:46.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.388+0000 7f92277fe640 1 -- 172.21.15.64:0/2734222769 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9220011d40 con 0x7f9230108830 2024-05-09T09:19:46.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.388+0000 7f92277fe640 1 -- 172.21.15.64:0/2734222769 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922000f340 con 0x7f9230108830 2024-05-09T09:19:46.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.394+0000 7f923b233740 1 Processor -- start 2024-05-09T09:19:46.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.394+0000 7f923b233740 1 -- start start 2024-05-09T09:19:46.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f923b233740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 0x5638bf3f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f923b233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3f5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f923b233740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf3f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f923b233740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638bf354c40 con 0x5638bf3f0930 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f923b233740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5638bf2fc580 con 0x5638bf3eff40 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f923b233740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5638bf350c00 con 0x5638bf3f5aa0 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 0x5638bf3f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 0x5638bf3f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43638/0 (socket says 172.21.15.64:43638) 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 -- 172.21.15.64:0/752554012 learned_addr learned my addr 172.21.15.64:0/752554012 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f92277fe640 1 -- 172.21.15.64:0/2734222769 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9220018020 con 0x7f9230108830 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9235abd640 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf3f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 -- 172.21.15.64:0/752554012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 msgr2=0x5638bf3f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf3f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 -- 172.21.15.64:0/752554012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 msgr2=0x5638bf3f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.395+0000 7f9236abf640 1 -- 172.21.15.64:0/752554012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638bf0683f0 con 0x5638bf3eff40 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.396+0000 7f9236abf640 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 0x5638bf3f0360 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9228031ea0 tx=0x7f9228031ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef52231635b9837 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.396+0000 7f9224bfe640 1 -- 172.21.15.64:0/752554012 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228031470 con 0x5638bf3eff40 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.396+0000 7f9224bfe640 1 -- 172.21.15.64:0/752554012 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9228036040 con 0x5638bf3eff40 2024-05-09T09:19:46.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.396+0000 7f9224bfe640 1 -- 172.21.15.64:0/752554012 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228004720 con 0x5638bf3eff40 2024-05-09T09:19:46.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.397+0000 7f923b233740 1 -- 172.21.15.64:0/752554012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 msgr2=0x5638bf3f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.397+0000 7f923b233740 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 0x5638bf3f0360 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9228031ea0 tx=0x7f9228031ac0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.398+0000 7f92277fe640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f921c0a2da0 0x7f921c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.398+0000 7f92277fe640 1 -- 172.21.15.64:0/2734222769 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f92200ced60 con 0x7f9230108830 2024-05-09T09:19:46.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.399+0000 7f9236abf640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f921c0a2da0 0x7f921c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.400+0000 7f9236abf640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f921c0a2da0 0x7f921c0a5260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9228031e70 tx=0x7f9228004960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.400+0000 7f923b233740 1 -- 172.21.15.64:0/752554012 shutdown_connections 2024-05-09T09:19:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.400+0000 7f923b233740 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf3f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.400+0000 7f923b233740 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.400+0000 7f923b233740 1 --2- 172.21.15.64:0/752554012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3eff40 0x5638bf3f0360 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.400+0000 7f923b233740 1 -- 172.21.15.64:0/752554012 >> 172.21.15.64:0/752554012 conn(0x5638bf3eb5b0 msgr2=0x5638bf3eb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.401+0000 7f923b233740 1 -- 172.21.15.64:0/752554012 shutdown_connections 2024-05-09T09:19:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.401+0000 7f923b233740 1 -- 172.21.15.64:0/752554012 wait complete. 2024-05-09T09:19:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.402+0000 7f923b233740 1 -- start start 2024-05-09T09:19:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.402+0000 7f923b233740 1 Processor -- start 2024-05-09T09:19:46.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.403+0000 7f923b233740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f923b233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f923b233740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf4016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f9235abd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f9235abd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3fa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51916/0 (socket says 172.21.15.64:51916) 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f9235abd640 1 -- 172.21.15.64:0/3043387303 learned_addr learned my addr 172.21.15.64:0/3043387303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f92362be640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f9236abf640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf4016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638bf3f89f0 con 0x5638bf3f0930 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5638bf30fc40 con 0x5638bf3f5aa0 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.404+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5638bf3f8b70 con 0x5638bf3eff40 2024-05-09T09:19:46.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f923b233740 10 client.0 _ll_register_callbacks cb 0x5638bf067be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f92362be640 1 -- 172.21.15.64:0/3043387303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 msgr2=0x5638bf4016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f92362be640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf4016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f92362be640 1 -- 172.21.15.64:0/3043387303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 msgr2=0x5638bf3fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f92362be640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f92362be640 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638bf1327d0 con 0x5638bf3eff40 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f9235abd640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.405+0000 7f92362be640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x5638bf3f13a0 tx=0x7f922009aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25685e9338d2e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f9219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f9219ffb640 1 -- 172.21.15.64:0/3043387303 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92200afdb0 con 0x5638bf3eff40 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f9219ffb640 1 -- 172.21.15.64:0/3043387303 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9220011890 con 0x5638bf3eff40 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9228036040 con 0x5638bf3eff40 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f9219ffb640 1 -- 172.21.15.64:0/3043387303 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f922009fa40 con 0x5638bf3eff40 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f923b233740 10 client.27721 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.406+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5638bf404aa0 con 0x5638bf3eff40 2024-05-09T09:19:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.407+0000 7f9219ffb640 1 -- 172.21.15.64:0/3043387303 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9228036040 con 0x5638bf3eff40 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.408+0000 7f9219ffb640 1 -- 172.21.15.64:0/3043387303 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f92200973e0 con 0x5638bf3eff40 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.408+0000 7f923b233740 10 client.27721 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.408+0000 7f923b233740 10 client.27721 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.408+0000 7f923b233740 20 client.27721 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.408+0000 7f923b233740 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf4085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.409+0000 7f923b233740 4 client.27721 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.409+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638bf362370 con 0x5638bf406170 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.409+0000 7f9235abd640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf4085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.409+0000 7f9219ffb640 10 client.27721 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.409+0000 7f9235abd640 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf4085b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67ce6a7a2228e7fd server_cookie=c3c6c622cedc46c7 in_seq=0 out_seq=0 2024-05-09T09:19:46.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.412+0000 7f9219ffb640 1 -- 172.21.15.64:0/3043387303 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638bf362370 con 0x5638bf406170 2024-05-09T09:19:46.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.412+0000 7f9219ffb640 10 client.27721 handle_command_reply: tid=0 2024-05-09T09:19:46.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.414+0000 7f923b233740 1 client.27721 shutdown 2024-05-09T09:19:46.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.418+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 msgr2=0x5638bf3fa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.418+0000 7f923b233740 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x5638bf3f13a0 tx=0x7f922009aff0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.421+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 shutdown_connections 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.421+0000 7f923b233740 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf4016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.421+0000 7f923b233740 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf4085b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.422+0000 7f9219ffb640 0 client.27721 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.422+0000 7f923b233740 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf3fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.422+0000 7f923b233740 1 --2- 172.21.15.64:0/3043387303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa160 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.422+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 >> 172.21.15.64:0/3043387303 conn(0x5638bf3eb5b0 msgr2=0x5638bf3ea7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.424+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 shutdown_connections 2024-05-09T09:19:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.424+0000 7f923b233740 1 -- 172.21.15.64:0/3043387303 wait complete. 2024-05-09T09:19:46.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.425+0000 7f923b233740 20 client.27721 trim_cache size 0 max 16384 2024-05-09T09:19:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.457+0000 7f923b233740 1 Processor -- start 2024-05-09T09:19:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.457+0000 7f923b233740 1 -- start start 2024-05-09T09:19:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.457+0000 7f923b233740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f923b233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf401670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f92362be640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f92362be640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43540/0 (socket says 172.21.15.64:43540) 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f923b233740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf3ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f923b233740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9214012320 con 0x5638bf3f0930 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f923b233740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f921400fbf0 con 0x5638bf3f5aa0 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f923b233740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f921400daf0 con 0x5638bf3eff40 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f92362be640 1 -- 172.21.15.64:0/3441870051 learned_addr learned my addr 172.21.15.64:0/3441870051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f9235abd640 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf401670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.458+0000 7f9236abf640 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf3ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f92362be640 1 -- 172.21.15.64:0/3441870051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 msgr2=0x5638bf3ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f92362be640 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf3ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f92362be640 1 -- 172.21.15.64:0/3441870051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 msgr2=0x5638bf401670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f92362be640 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf401670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f92362be640 1 -- 172.21.15.64:0/3441870051 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638bf3dcd20 con 0x5638bf3eff40 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f92362be640 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa480 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f922000be10 tx=0x7f922009b120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa72f15d36d9922 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f921bfff640 1 -- 172.21.15.64:0/3441870051 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922009a550 con 0x5638bf3eff40 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f921bfff640 1 -- 172.21.15.64:0/3441870051 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92200afbb0 con 0x5638bf3eff40 2024-05-09T09:19:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.459+0000 7f921bfff640 1 -- 172.21.15.64:0/3441870051 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922009ad20 con 0x5638bf3eff40 2024-05-09T09:19:46.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 -- 172.21.15.64:0/3441870051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 msgr2=0x5638bf3fa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa480 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f922000be10 tx=0x7f922009b120 comp rx=0 tx=0).stop 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 -- 172.21.15.64:0/3441870051 shutdown_connections 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f5aa0 0x5638bf3ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3f0930 0x5638bf401670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 --2- 172.21.15.64:0/3441870051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3eff40 0x5638bf3fa480 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 -- 172.21.15.64:0/3441870051 >> 172.21.15.64:0/3441870051 conn(0x5638bf3eb5b0 msgr2=0x5638bf3f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 -- 172.21.15.64:0/3441870051 shutdown_connections 2024-05-09T09:19:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 -- 172.21.15.64:0/3441870051 wait complete. 2024-05-09T09:19:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.461+0000 7f923b233740 1 -- start start 2024-05-09T09:19:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.462+0000 7f923b233740 1 Processor -- start 2024-05-09T09:19:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.462+0000 7f923b233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 0x5638bf3feb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.462+0000 7f923b233740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f0930 0x5638bf3ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f9235abd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f0930 0x5638bf3ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f92362be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 0x5638bf3feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f92362be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 0x5638bf3feb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51934/0 (socket says 172.21.15.64:51934) 2024-05-09T09:19:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f92362be640 1 -- 172.21.15.64:0/1706063811 learned_addr learned my addr 172.21.15.64:0/1706063811 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf4000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f921400e0b0 con 0x5638bf3eff40 2024-05-09T09:19:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f921400f520 con 0x5638bf3f0930 2024-05-09T09:19:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f921400e780 con 0x5638bf3f5aa0 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f9236abf640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf4000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f9235abd640 1 -- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 msgr2=0x5638bf4000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f9235abd640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf4000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f9235abd640 1 -- 172.21.15.64:0/1706063811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 msgr2=0x5638bf3feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.463+0000 7f9235abd640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 0x5638bf3feb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f9235abd640 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638bf4006a0 con 0x5638bf3f0930 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f92362be640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 0x5638bf3feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f9235abd640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f0930 0x5638bf3ff0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f922c004700 tx=0x7f922c004eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4421ce5d01418c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f9219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f9219ffb640 1 -- 172.21.15.64:0/1706063811 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c0205a0 con 0x5638bf3f0930 2024-05-09T09:19:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f9219ffb640 1 -- 172.21.15.64:0/1706063811 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f922c020740 con 0x5638bf3f0930 2024-05-09T09:19:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.464+0000 7f9219ffb640 10 client.0 inactive, discarding osd_map(94..94 src has 1..94) 2024-05-09T09:19:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.465+0000 7f923b233740 10 client.0 _ll_register_callbacks cb 0x7f9220012f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.465+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92200afbb0 con 0x5638bf3f0930 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.465+0000 7f9219ffb640 1 -- 172.21.15.64:0/1706063811 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f922c02e050 con 0x5638bf3f0930 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.465+0000 7f923b233740 10 client.27596 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.465+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5638bf4017b0 con 0x5638bf3f0930 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f9219ffb640 1 -- 172.21.15.64:0/1706063811 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f922c004030 con 0x5638bf3f0930 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f923b233740 10 client.27596 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f923b233740 10 client.27596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f923b233740 20 client.27596 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf46ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f923b233740 4 client.27596 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:19:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.466+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638bf35c6c0 con 0x5638bf406170 2024-05-09T09:19:46.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.469+0000 7f92362be640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf46ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.471+0000 7f92362be640 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf46ebf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9940c17c6fe406d server_cookie=4acb04a602119d4d in_seq=0 out_seq=0 2024-05-09T09:19:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f9219ffb640 10 client.27596 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f9219ffb640 1 -- 172.21.15.64:0/1706063811 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638bf35c6c0 con 0x5638bf406170 2024-05-09T09:19:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f9219ffb640 10 client.27596 handle_command_reply: tid=0 2024-05-09T09:19:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 client.27596 shutdown 2024-05-09T09:19:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f0930 msgr2=0x5638bf3ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f0930 0x5638bf3ff0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f922c004700 tx=0x7f922c004eb0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 shutdown_connections 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638bf3f5aa0 0x5638bf4000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5638bf3f0930 0x5638bf3ff0a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5638bf406170 0x5638bf46ebf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.475+0000 7f923b233740 1 --2- 172.21.15.64:0/1706063811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bf3eff40 0x5638bf3feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.476+0000 7f9219ffb640 0 client.27596 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:19:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.476+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 >> 172.21.15.64:0/1706063811 conn(0x5638bf3eb5b0 msgr2=0x5638bf3ea7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.476+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 shutdown_connections 2024-05-09T09:19:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.477+0000 7f923b233740 1 -- 172.21.15.64:0/1706063811 wait complete. 2024-05-09T09:19:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.477+0000 7f923b233740 20 client.27596 trim_cache size 0 max 16384 2024-05-09T09:19:46.481 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:19:46.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.482+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f921c0a2da0 msgr2=0x7f921c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.482+0000 7f9238522640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f921c0a2da0 0x7f921c0a5260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9228031e70 tx=0x7f9228004960 comp rx=0 tx=0).stop 2024-05-09T09:19:46.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.482+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 msgr2=0x7f92301a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.482+0000 7f9238522640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 0x7f92301a68b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f922000b450 tx=0x7f922000b910 comp rx=0 tx=0).stop 2024-05-09T09:19:46.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.483+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 shutdown_connections 2024-05-09T09:19:46.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.483+0000 7f9238522640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f921c0a2da0 0x7f921c0a5260 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.484+0000 7f9238522640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f923010db70 0x7f92301ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.484+0000 7f9238522640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923010b1a0 0x7f92301a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.484+0000 7f9238522640 1 --2- 172.21.15.64:0/2734222769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9230108830 0x7f92301a68b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.484+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 >> 172.21.15.64:0/2734222769 conn(0x7f923006e6c0 msgr2=0x7f923010a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.484+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 shutdown_connections 2024-05-09T09:19:46.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.484+0000 7f9238522640 1 -- 172.21.15.64:0/2734222769 wait complete. 2024-05-09T09:19:46.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 -- 172.21.15.64:0/804867208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc1098b0 msgr2=0x7fc8fc10e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 --2- 172.21.15.64:0/804867208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc1098b0 0x7fc8fc10e170 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009900 tx=0x7fc8f402a440 comp rx=0 tx=0).stop 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 -- 172.21.15.64:0/804867208 shutdown_connections 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 --2- 172.21.15.64:0/804867208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc1098b0 0x7fc8fc10e170 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 --2- 172.21.15.64:0/804867208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 0x7fc8fc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 --2- 172.21.15.64:0/804867208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8fc072c70 0x7fc8fc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 -- 172.21.15.64:0/804867208 >> 172.21.15.64:0/804867208 conn(0x7fc8fc06e740 msgr2=0x7fc8fc070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 -- 172.21.15.64:0/804867208 shutdown_connections 2024-05-09T09:19:46.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.703+0000 7fc902051640 1 -- 172.21.15.64:0/804867208 wait complete. 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.704+0000 7fc902051640 1 Processor -- start 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.704+0000 7fc902051640 1 -- start start 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc902051640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc072c70 0x7fc8fc1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc902051640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 0x7fc8fc1a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc902051640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8fc1098b0 0x7fc8fc1ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc902051640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc11c170 con 0x7fc8fc072c70 2024-05-09T09:19:46.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc902051640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc11bff0 con 0x7fc8fc108f10 2024-05-09T09:19:46.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc902051640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc11c2f0 con 0x7fc8fc1098b0 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc90084e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 0x7fc8fc1a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc90084e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 0x7fc8fc1a6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43690/0 (socket says 172.21.15.64:43690) 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc90084e640 1 -- 172.21.15.64:0/3775793498 learned_addr learned my addr 172.21.15.64:0/3775793498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc901850640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8fc1098b0 0x7fc8fc1ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.705+0000 7fc90104f640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc072c70 0x7fc8fc1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.706+0000 7fc90104f640 1 -- 172.21.15.64:0/3775793498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8fc1098b0 msgr2=0x7fc8fc1ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.706+0000 7fc90104f640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8fc1098b0 0x7fc8fc1ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.706+0000 7fc90104f640 1 -- 172.21.15.64:0/3775793498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 msgr2=0x7fc8fc1a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.706+0000 7fc90104f640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 0x7fc8fc1a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.706+0000 7fc90104f640 1 -- 172.21.15.64:0/3775793498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1ae4c0 con 0x7fc8fc072c70 2024-05-09T09:19:46.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.707+0000 7fc90104f640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc072c70 0x7fc8fc1a68a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800be20 tx=0x7fc8f8009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.707+0000 7fc8f27fc640 1 -- 172.21.15.64:0/3775793498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8019070 con 0x7fc8fc072c70 2024-05-09T09:19:46.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.708+0000 7fc8f27fc640 1 -- 172.21.15.64:0/3775793498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f80148f0 con 0x7fc8fc072c70 2024-05-09T09:19:46.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.708+0000 7fc8f27fc640 1 -- 172.21.15.64:0/3775793498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80128d0 con 0x7fc8fc072c70 2024-05-09T09:19:46.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.709+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc1ae7b0 con 0x7fc8fc072c70 2024-05-09T09:19:46.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.709+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1aed00 con 0x7fc8fc072c70 2024-05-09T09:19:46.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.713+0000 7fc905fbe740 1 Processor -- start 2024-05-09T09:19:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.714+0000 7fc905fbe740 1 -- start start 2024-05-09T09:19:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.715+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb114ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.715+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb114f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.715+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.715+0000 7fc905fbe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cb11453c40 con 0x55cb114f4c90 2024-05-09T09:19:46.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.715+0000 7fc905fbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cb113fb580 con 0x55cb114efb20 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.715+0000 7fc905fbe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cb1144fc00 con 0x55cb114ef130 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.716+0000 7fc8f27fc640 1 -- 172.21.15.64:0/3775793498 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8f8014aa0 con 0x7fc8fc072c70 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.716+0000 7fc90084e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.716+0000 7fc90084e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51962/0 (socket says 172.21.15.64:51962) 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.716+0000 7fc90084e640 1 -- 172.21.15.64:0/3807762655 learned_addr learned my addr 172.21.15.64:0/3807762655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc8f27fc640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8e80a2d90 0x7fc8e80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc90084e640 1 -- 172.21.15.64:0/3807762655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 msgr2=0x55cb114ef550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc90084e640 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb114ef550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc90084e640 1 -- 172.21.15.64:0/3807762655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 msgr2=0x55cb114f4620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc90084e640 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb114f4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc90084e640 1 -- 172.21.15.64:0/3807762655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb111673f0 con 0x55cb114f4c90 2024-05-09T09:19:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc8f27fc640 1 -- 172.21.15.64:0/3775793498 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f80cebc0 con 0x7fc8fc072c70 2024-05-09T09:19:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.717+0000 7fc901850640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8e80a2d90 0x7fc8e80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.718+0000 7fc90084e640 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114f7120 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ae0 tx=0x7fc8ec00e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbda8eb52441727 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.718+0000 7fc901850640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8e80a2d90 0x7fc8e80a5250 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55cb114f0380 tx=0x7fc8f40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.719+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3807762655 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec017070 con 0x55cb114f4c90 2024-05-09T09:19:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.719+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3807762655 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8ec00ebb0 con 0x55cb114f4c90 2024-05-09T09:19:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.719+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3807762655 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec012630 con 0x55cb114f4c90 2024-05-09T09:19:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 -- 172.21.15.64:0/3807762655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 msgr2=0x55cb114f7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114f7120 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ae0 tx=0x7fc8ec00e1d0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 -- 172.21.15.64:0/3807762655 shutdown_connections 2024-05-09T09:19:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114f7120 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb114f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 --2- 172.21.15.64:0/3807762655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb114ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 -- 172.21.15.64:0/3807762655 >> 172.21.15.64:0/3807762655 conn(0x55cb114ea7a0 msgr2=0x55cb114eaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 -- 172.21.15.64:0/3807762655 shutdown_connections 2024-05-09T09:19:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 -- 172.21.15.64:0/3807762655 wait complete. 2024-05-09T09:19:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.720+0000 7fc905fbe740 1 -- start start 2024-05-09T09:19:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 Processor -- start 2024-05-09T09:19:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114efb20 0x55cb1150ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 0x55cb11511bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cb11508c50 con 0x55cb114ef130 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cb1140ec40 con 0x55cb114f4c90 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cb11508dd0 con 0x55cb114efb20 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc905fbe740 10 client.0 _ll_register_callbacks cb 0x55cb11166be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.721+0000 7fc901850640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 0x55cb11511bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 0x55cb11511bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43712/0 (socket says 172.21.15.64:43712) 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 -- 172.21.15.64:0/1564758754 learned_addr learned my addr 172.21.15.64:0/1564758754 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc90104f640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc90084e640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114efb20 0x55cb1150ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 -- 172.21.15.64:0/1564758754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114efb20 msgr2=0x55cb1150ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114efb20 0x55cb1150ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 -- 172.21.15.64:0/1564758754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 msgr2=0x55cb1150a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 -- 172.21.15.64:0/1564758754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb112317d0 con 0x55cb114f4c90 2024-05-09T09:19:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc901850640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 0x55cb11511bd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4033000 tx=0x7fc8f4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=325b7caf5f40902f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.722+0000 7fc8e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:19:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.723+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/1564758754 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f404d020 con 0x55cb114f4c90 2024-05-09T09:19:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.723+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec00ebb0 con 0x55cb114f4c90 2024-05-09T09:19:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.723+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/1564758754 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f4031580 con 0x55cb114f4c90 2024-05-09T09:19:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.723+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/1564758754 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc8f4058050 con 0x55cb114f4c90 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.724+0000 7fc905fbe740 10 client.27602 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.724+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cb11514fb0 con 0x55cb114f4c90 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.724+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/1564758754 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8ec00ebb0 con 0x55cb114f4c90 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.724+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/1564758754 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f4049910 con 0x55cb114f4c90 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc905fbe740 10 client.27602 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc905fbe740 10 client.27602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc905fbe740 20 client.27602 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc905fbe740 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb11516670 0x55cb11518af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc905fbe740 4 client.27602 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb11461370 con 0x55cb11516670 2024-05-09T09:19:46.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.725+0000 7fc90104f640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb11516670 0x55cb11518af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.727+0000 7fc8e4ff9640 10 client.27602 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.727+0000 7fc90104f640 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb11516670 0x55cb11518af0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d07f39bd629198 server_cookie=a786481d23ba6869 in_seq=0 out_seq=0 2024-05-09T09:19:46.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.728+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/1564758754 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb11461370 con 0x55cb11516670 2024-05-09T09:19:46.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.728+0000 7fc8e4ff9640 10 client.27602 handle_command_reply: tid=0 2024-05-09T09:19:46.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.728+0000 7fc905fbe740 1 client.27602 shutdown 2024-05-09T09:19:46.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.729+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 msgr2=0x55cb11511bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.729+0000 7fc905fbe740 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 0x55cb11511bd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4033000 tx=0x7fc8f4009260 comp rx=0 tx=0).stop 2024-05-09T09:19:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 shutdown_connections 2024-05-09T09:19:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb11516670 0x55cb11518af0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114f4c90 0x55cb11511bd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114efb20 0x55cb1150ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 --2- 172.21.15.64:0/1564758754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 >> 172.21.15.64:0/1564758754 conn(0x55cb114ea7a0 msgr2=0x55cb114e99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc8e4ff9640 0 client.27602 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.730+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 shutdown_connections 2024-05-09T09:19:46.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.731+0000 7fc905fbe740 1 -- 172.21.15.64:0/1564758754 wait complete. 2024-05-09T09:19:46.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.732+0000 7fc905fbe740 20 client.27602 trim_cache size 0 max 16384 2024-05-09T09:19:46.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 Processor -- start 2024-05-09T09:19:46.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 -- start start 2024-05-09T09:19:46.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb11509920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb11511b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b3250 con 0x55cb114f4c90 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b0b20 con 0x55cb114efb20 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.760+0000 7fc905fbe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc118120 con 0x55cb114ef130 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb11509920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb11509920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43584/0 (socket says 172.21.15.64:43584) 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 -- 172.21.15.64:0/4118047976 learned_addr learned my addr 172.21.15.64:0/4118047976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc901850640 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90084e640 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb11511b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 -- 172.21.15.64:0/4118047976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 msgr2=0x55cb11511b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb11511b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 -- 172.21.15.64:0/4118047976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 msgr2=0x55cb114ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 -- 172.21.15.64:0/4118047976 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb114eb510 con 0x55cb114ef130 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc901850640 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.761+0000 7fc90104f640 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb11509920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80099a0 tx=0x7fc8f800e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acadfea7739e113c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.762+0000 7fc8e6ffd640 1 -- 172.21.15.64:0/4118047976 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8099180 con 0x55cb114ef130 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.763+0000 7fc8e6ffd640 1 -- 172.21.15.64:0/4118047976 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f80993e0 con 0x55cb114ef130 2024-05-09T09:19:46.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.763+0000 7fc8e6ffd640 1 -- 172.21.15.64:0/4118047976 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a3a70 con 0x55cb114ef130 2024-05-09T09:19:46.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.764+0000 7fc905fbe740 1 -- 172.21.15.64:0/4118047976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 msgr2=0x55cb11509920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.764+0000 7fc905fbe740 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb11509920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80099a0 tx=0x7fc8f800e040 comp rx=0 tx=0).stop 2024-05-09T09:19:46.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.765+0000 7fc905fbe740 1 -- 172.21.15.64:0/4118047976 shutdown_connections 2024-05-09T09:19:46.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.765+0000 7fc905fbe740 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114f4c90 0x55cb114ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.765+0000 7fc905fbe740 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb11511b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.765+0000 7fc905fbe740 1 --2- 172.21.15.64:0/4118047976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114ef130 0x55cb11509920 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.765+0000 7fc905fbe740 1 -- 172.21.15.64:0/4118047976 >> 172.21.15.64:0/4118047976 conn(0x55cb114ea7a0 msgr2=0x55cb114ed750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.765+0000 7fc905fbe740 1 -- 172.21.15.64:0/4118047976 shutdown_connections 2024-05-09T09:19:46.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.766+0000 7fc905fbe740 1 -- 172.21.15.64:0/4118047976 wait complete. 2024-05-09T09:19:46.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.767+0000 7fc905fbe740 1 -- start start 2024-05-09T09:19:46.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.767+0000 7fc905fbe740 1 Processor -- start 2024-05-09T09:19:46.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb1150f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114f4c90 0x55cb11510990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1af010 con 0x55cb114ef130 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1b0450 con 0x55cb114efb20 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1af6b0 con 0x55cb114f4c90 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.768+0000 7fc905fbe740 10 client.0 _ll_register_callbacks cb 0x7fc8f4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51994/0 (socket says 172.21.15.64:51994) 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 -- 172.21.15.64:0/2976422271 learned_addr learned my addr 172.21.15.64:0/2976422271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc901850640 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114f4c90 0x55cb11510990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 -- 172.21.15.64:0/2976422271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114f4c90 msgr2=0x55cb11510990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114f4c90 0x55cb11510990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 -- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 msgr2=0x55cb1150f980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb1150f980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.769+0000 7fc90104f640 1 -- 172.21.15.64:0/2976422271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb11510f80 con 0x55cb114ef130 2024-05-09T09:19:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.770+0000 7fc90104f640 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150f440 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc8f809b070 tx=0x7fc8f80bb250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8154f0e7da105e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc8e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/2976422271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a7030 con 0x55cb114ef130 2024-05-09T09:19:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/2976422271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f80a3810 con 0x55cb114ef130 2024-05-09T09:19:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f80993e0 con 0x55cb114ef130 2024-05-09T09:19:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/2976422271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc8f807e050 con 0x55cb114ef130 2024-05-09T09:19:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc905fbe740 10 client.18111 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.771+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cb1157cdb0 con 0x55cb114ef130 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/2976422271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f80a3a10 con 0x55cb114ef130 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/2976422271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f8096e30 con 0x55cb114ef130 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc905fbe740 10 client.18111 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc905fbe740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc905fbe740 20 client.18111 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc905fbe740 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb1157d590 0x55cb1157f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.772+0000 7fc905fbe740 4 client.18111 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:19:46.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.773+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc8f80993e0 con 0x55cb1157d590 2024-05-09T09:19:46.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.773+0000 7fc90084e640 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb1157d590 0x55cb1157f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.775+0000 7fc90084e640 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb1157d590 0x55cb1157f9b0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd88612ab3283919 server_cookie=67756a42fd52b2bc in_seq=0 out_seq=0 2024-05-09T09:19:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.775+0000 7fc8e4ff9640 10 client.18111 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.776+0000 7fc8e4ff9640 1 -- 172.21.15.64:0/2976422271 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc8f80993e0 con 0x55cb1157d590 2024-05-09T09:19:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.776+0000 7fc8e4ff9640 10 client.18111 handle_command_reply: tid=0 2024-05-09T09:19:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.776+0000 7fc905fbe740 1 client.18111 shutdown 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.776+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 msgr2=0x55cb1150f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.776+0000 7fc905fbe740 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150f440 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc8f809b070 tx=0x7fc8f80bb250 comp rx=0 tx=0).stop 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 shutdown_connections 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cb1157d590 0x55cb1157f9b0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb114f4c90 0x55cb11510990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cb114efb20 0x55cb1150f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 --2- 172.21.15.64:0/2976422271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb114ef130 0x55cb1150f440 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 >> 172.21.15.64:0/2976422271 conn(0x55cb114ea7a0 msgr2=0x55cb114edfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc8e4ff9640 0 client.18111 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 shutdown_connections 2024-05-09T09:19:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 1 -- 172.21.15.64:0/2976422271 wait complete. 2024-05-09T09:19:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.777+0000 7fc905fbe740 20 client.18111 trim_cache size 0 max 16384 2024-05-09T09:19:46.777 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8e80a2d90 msgr2=0x7fc8e80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8e80a2d90 0x7fc8e80a5250 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55cb114f0380 tx=0x7fc8f40314a0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc072c70 msgr2=0x7fc8fc1a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc072c70 0x7fc8fc1a68a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800be20 tx=0x7fc8f8009f70 comp rx=0 tx=0).stop 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 shutdown_connections 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8e80a2d90 0x7fc8e80a5250 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8fc1098b0 0x7fc8fc1ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc108f10 0x7fc8fc1a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 --2- 172.21.15.64:0/3775793498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc072c70 0x7fc8fc1a68a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 >> 172.21.15.64:0/3775793498 conn(0x7fc8fc06e740 msgr2=0x7fc8fc070410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.779+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 shutdown_connections 2024-05-09T09:19:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:46.780+0000 7fc902051640 1 -- 172.21.15.64:0/3775793498 wait complete. 2024-05-09T09:19:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:46 smithi064 ceph-mon[30149]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 924 op/s 2024-05-09T09:19:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:46 smithi179 ceph-mon[34086]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 924 op/s 2024-05-09T09:19:47.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:46 smithi139 ceph-mon[33599]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 924 op/s 2024-05-09T09:19:47.808 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:19:47.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:19:47.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:19:48.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 -- 172.21.15.64:0/2988318754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c108f10 msgr2=0x7f318c109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 --2- 172.21.15.64:0/2988318754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c108f10 0x7f318c109370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f317c009be0 tx=0x7f317c02a6e0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 -- 172.21.15.64:0/2988318754 shutdown_connections 2024-05-09T09:19:48.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 --2- 172.21.15.64:0/2988318754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c1098b0 0x7f318c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 --2- 172.21.15.64:0/2988318754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c108f10 0x7f318c109370 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 --2- 172.21.15.64:0/2988318754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c072c70 0x7f318c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.012+0000 7f31920bf640 1 -- 172.21.15.64:0/2988318754 >> 172.21.15.64:0/2988318754 conn(0x7f318c06e740 msgr2=0x7f318c070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.014+0000 7f31920bf640 1 -- 172.21.15.64:0/2988318754 shutdown_connections 2024-05-09T09:19:48.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.014+0000 7f31920bf640 1 -- 172.21.15.64:0/2988318754 wait complete. 2024-05-09T09:19:48.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.015+0000 7f31920bf640 1 Processor -- start 2024-05-09T09:19:48.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.015+0000 7f31920bf640 1 -- start start 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.015+0000 7f31920bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c072c70 0x7f318c1199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.015+0000 7f31920bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c108f10 0x7f318c119ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.015+0000 7f31920bf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c1098b0 0x7f318c117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31920bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f318c110820 con 0x7f318c108f10 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31920bf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1106a0 con 0x7f318c072c70 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31920bf640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1109a0 con 0x7f318c1098b0 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31908bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c108f10 0x7f318c119ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31910bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c072c70 0x7f318c1199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31908bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c108f10 0x7f318c119ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52004/0 (socket says 172.21.15.64:52004) 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31908bc640 1 -- 172.21.15.64:0/3443598288 learned_addr learned my addr 172.21.15.64:0/3443598288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31918be640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c1098b0 0x7f318c117ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31910bd640 1 -- 172.21.15.64:0/3443598288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c1098b0 msgr2=0x7f318c117ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31910bd640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c1098b0 0x7f318c117ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31910bd640 1 -- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c108f10 msgr2=0x7f318c119ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31910bd640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c108f10 0x7f318c119ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.016+0000 7f31910bd640 1 -- 172.21.15.64:0/3443598288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c118690 con 0x7f318c072c70 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.017+0000 7f31910bd640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c072c70 0x7f318c1199b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f318400e790 tx=0x7f318400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.017+0000 7f318a7fc640 1 -- 172.21.15.64:0/3443598288 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400eea0 con 0x7f318c072c70 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.017+0000 7f318a7fc640 1 -- 172.21.15.64:0/3443598288 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f318400ccc0 con 0x7f318c072c70 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.017+0000 7f318a7fc640 1 -- 172.21.15.64:0/3443598288 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184018710 con 0x7f318c072c70 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.017+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c118980 con 0x7f318c072c70 2024-05-09T09:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.018+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c118dc0 con 0x7f318c072c70 2024-05-09T09:19:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.022+0000 7f318a7fc640 1 -- 172.21.15.64:0/3443598288 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3184010430 con 0x7f318c072c70 2024-05-09T09:19:48.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.023+0000 7f318a7fc640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f31740a2c10 0x7f31740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.023+0000 7f319602a740 1 Processor -- start 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.023+0000 7f318a7fc640 1 -- 172.21.15.64:0/3443598288 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f31840cf640 con 0x7f318c072c70 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.023+0000 7f319602a740 1 -- start start 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f319602a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a46ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f319602a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 0x55d450a4bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f319602a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a4c210 0x55d450a4e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f319602a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4509aa990 con 0x55d450a4c210 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f319602a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d4509525b0 con 0x55d450a470a0 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f319602a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d4509a6950 con 0x55d450a466b0 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f31908bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 0x55d450a4bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f31908bc640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f31740a2c10 0x7f31740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f31908bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 0x55d450a4bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43748/0 (socket says 172.21.15.64:43748) 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f31908bc640 1 -- 172.21.15.64:0/1185210162 learned_addr learned my addr 172.21.15.64:0/1185210162 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f31918be640 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a4c210 0x55d450a4e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.024+0000 7f31910bd640 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a46ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.025+0000 7f31908bc640 1 -- 172.21.15.64:0/1185210162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 msgr2=0x55d450a46ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.025+0000 7f31908bc640 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a46ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.025+0000 7f31908bc640 1 -- 172.21.15.64:0/1185210162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a4c210 msgr2=0x55d450a4e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.025+0000 7f31908bc640 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a4c210 0x55d450a4e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.025+0000 7f31908bc640 1 -- 172.21.15.64:0/1185210162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4507867e0 con 0x55d450a470a0 2024-05-09T09:19:48.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.026+0000 7f31908bc640 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 0x55d450a4bba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f317c02acd0 tx=0x7f317c003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674ecd582fdd15a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.026+0000 7f31908bc640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f31740a2c10 0x7f31740a50d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f317c004af0 tx=0x7f317c044000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.026+0000 7f31737fe640 1 -- 172.21.15.64:0/1185210162 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c036070 con 0x55d450a470a0 2024-05-09T09:19:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.026+0000 7f31737fe640 1 -- 172.21.15.64:0/1185210162 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f317c04f070 con 0x55d450a470a0 2024-05-09T09:19:48.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.027+0000 7f31737fe640 1 -- 172.21.15.64:0/1185210162 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c0042a0 con 0x55d450a470a0 2024-05-09T09:19:48.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.027+0000 7f319602a740 1 -- 172.21.15.64:0/1185210162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 msgr2=0x55d450a4bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.027+0000 7f319602a740 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 0x55d450a4bba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f317c02acd0 tx=0x7f317c003970 comp rx=0 tx=0).stop 2024-05-09T09:19:48.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 -- 172.21.15.64:0/1185210162 shutdown_connections 2024-05-09T09:19:48.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a4c210 0x55d450a4e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a470a0 0x55d450a4bba0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 --2- 172.21.15.64:0/1185210162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a46ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 -- 172.21.15.64:0/1185210162 >> 172.21.15.64:0/1185210162 conn(0x55d450a0e0a0 msgr2=0x55d450a42130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 -- 172.21.15.64:0/1185210162 shutdown_connections 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 -- 172.21.15.64:0/1185210162 wait complete. 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 -- start start 2024-05-09T09:19:48.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.028+0000 7f319602a740 1 Processor -- start 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a50be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 0x55d450a51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a4c210 0x55d450a58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d450a4f1b0 con 0x55d450a470a0 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d450965b40 con 0x55d450a4c210 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d450a4f330 con 0x55d450a466b0 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f319602a740 10 client.0 _ll_register_callbacks cb 0x55d45095a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f31908bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 0x55d450a51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f31908bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 0x55d450a51120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52014/0 (socket says 172.21.15.64:52014) 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.029+0000 7f31908bc640 1 -- 172.21.15.64:0/2216630223 learned_addr learned my addr 172.21.15.64:0/2216630223 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f31908bc640 1 -- 172.21.15.64:0/2216630223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 msgr2=0x55d450a50be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f31908bc640 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a50be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f31908bc640 1 -- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a4c210 msgr2=0x55d450a58160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f31908bc640 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a4c210 0x55d450a58160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f31908bc640 1 -- 172.21.15.64:0/2216630223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4507e6f70 con 0x55d450a470a0 2024-05-09T09:19:48.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f31908bc640 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 0x55d450a51120 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f317c031a50 tx=0x7f317c02a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247e3b4f24d73a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.030+0000 7f3170ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:48.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.032+0000 7f3170ff9640 1 -- 172.21.15.64:0/2216630223 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c057070 con 0x55d450a470a0 2024-05-09T09:19:48.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.032+0000 7f3170ff9640 1 -- 172.21.15.64:0/2216630223 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f317c004b40 con 0x55d450a470a0 2024-05-09T09:19:48.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.033+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317c04f070 con 0x55d450a470a0 2024-05-09T09:19:48.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.033+0000 7f3170ff9640 1 -- 172.21.15.64:0/2216630223 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f317c036040 con 0x55d450a470a0 2024-05-09T09:19:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.035+0000 7f319602a740 10 client.18123 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.035+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d45067e170 con 0x55d450a470a0 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f3170ff9640 1 -- 172.21.15.64:0/2216630223 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f317c04f070 con 0x55d450a470a0 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f3170ff9640 1 -- 172.21.15.64:0/2216630223 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f317c049070 con 0x55d450a470a0 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f319602a740 10 client.18123 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f319602a740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f319602a740 20 client.18123 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f319602a740 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a5c2c0 0x55d450a5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f319602a740 4 client.18123 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.036+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4509b80c0 con 0x55d450a5c2c0 2024-05-09T09:19:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.037+0000 7f31910bd640 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a5c2c0 0x55d450a5e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.038+0000 7f31910bd640 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a5c2c0 0x55d450a5e6b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35867073449b5e3 server_cookie=3d03b58488f607af in_seq=0 out_seq=0 2024-05-09T09:19:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.038+0000 7f3170ff9640 10 client.18123 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.038+0000 7f3170ff9640 1 -- 172.21.15.64:0/2216630223 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4509b80c0 con 0x55d450a5c2c0 2024-05-09T09:19:48.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.038+0000 7f3170ff9640 10 client.18123 handle_command_reply: tid=0 2024-05-09T09:19:48.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.038+0000 7f319602a740 1 client.18123 shutdown 2024-05-09T09:19:48.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 msgr2=0x55d450a51120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 0x55d450a51120 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f317c031a50 tx=0x7f317c02a6e0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 shutdown_connections 2024-05-09T09:19:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a5c2c0 0x55d450a5e6b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a4c210 0x55d450a58160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a470a0 0x55d450a51120 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 --2- 172.21.15.64:0/2216630223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a466b0 0x55d450a50be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.039+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 >> 172.21.15.64:0/2216630223 conn(0x55d450a0e0a0 msgr2=0x55d450a41270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.042+0000 7f3170ff9640 0 client.18123 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.042+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 shutdown_connections 2024-05-09T09:19:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.043+0000 7f319602a740 1 -- 172.21.15.64:0/2216630223 wait complete. 2024-05-09T09:19:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.044+0000 7f319602a740 20 client.18123 trim_cache size 0 max 16384 2024-05-09T09:19:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.090+0000 7f319602a740 1 Processor -- start 2024-05-09T09:19:48.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.090+0000 7f319602a740 1 -- start start 2024-05-09T09:19:48.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.091+0000 7f319602a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a0e0a0 0x55d450a45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.091+0000 7f319602a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a466b0 0x55d450a58120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.091+0000 7f319602a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a45560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.091+0000 7f319602a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1c4180 con 0x55d450a0e0a0 2024-05-09T09:19:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.091+0000 7f319602a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1c1a50 con 0x55d450a466b0 2024-05-09T09:19:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.091+0000 7f319602a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1bf8a0 con 0x55d450a470a0 2024-05-09T09:19:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a45560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a45560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43636/0 (socket says 172.21.15.64:43636) 2024-05-09T09:19:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 -- 172.21.15.64:0/161978641 learned_addr learned my addr 172.21.15.64:0/161978641 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31908bc640 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a466b0 0x55d450a58120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31910bd640 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a0e0a0 0x55d450a45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 -- 172.21.15.64:0/161978641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a466b0 msgr2=0x55d450a58120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a466b0 0x55d450a58120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 -- 172.21.15.64:0/161978641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a0e0a0 msgr2=0x55d450a45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a0e0a0 0x55d450a45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.092+0000 7f31918be640 1 -- 172.21.15.64:0/161978641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d450a42de0 con 0x55d450a470a0 2024-05-09T09:19:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.093+0000 7f31918be640 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a45560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3180009b50 tx=0x7f3180013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29cb3e30959379b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.093+0000 7f3172ffd640 1 -- 172.21.15.64:0/161978641 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180009e40 con 0x55d450a470a0 2024-05-09T09:19:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.093+0000 7f3172ffd640 1 -- 172.21.15.64:0/161978641 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31800035f0 con 0x55d450a470a0 2024-05-09T09:19:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.093+0000 7f3172ffd640 1 -- 172.21.15.64:0/161978641 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318001c6e0 con 0x55d450a470a0 2024-05-09T09:19:48.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.095+0000 7f319602a740 1 -- 172.21.15.64:0/161978641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 msgr2=0x55d450a45560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.095+0000 7f319602a740 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a45560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3180009b50 tx=0x7f3180013830 comp rx=0 tx=0).stop 2024-05-09T09:19:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.096+0000 7f319602a740 1 -- 172.21.15.64:0/161978641 shutdown_connections 2024-05-09T09:19:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.096+0000 7f319602a740 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a45560 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.096+0000 7f319602a740 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a466b0 0x55d450a58120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.096+0000 7f319602a740 1 --2- 172.21.15.64:0/161978641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a0e0a0 0x55d450a45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.097+0000 7f319602a740 1 -- 172.21.15.64:0/161978641 >> 172.21.15.64:0/161978641 conn(0x55d450a47e80 msgr2=0x55d450a485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.098+0000 7f319602a740 1 -- 172.21.15.64:0/161978641 shutdown_connections 2024-05-09T09:19:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.098+0000 7f319602a740 1 -- 172.21.15.64:0/161978641 wait complete. 2024-05-09T09:19:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.099+0000 7f319602a740 1 -- start start 2024-05-09T09:19:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.099+0000 7f319602a740 1 Processor -- start 2024-05-09T09:19:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.100+0000 7f319602a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a0e0a0 0x55d450a55ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.100+0000 7f319602a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 0x55d450a56220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.100+0000 7f319602a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.100+0000 7f319602a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1bff10 con 0x55d450a466b0 2024-05-09T09:19:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.100+0000 7f319602a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1c1380 con 0x55d450a0e0a0 2024-05-09T09:19:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.100+0000 7f319602a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1c05e0 con 0x55d450a470a0 2024-05-09T09:19:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.101+0000 7f319602a740 10 client.0 _ll_register_callbacks cb 0x7f3180005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.102+0000 7f31908bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 0x55d450a56220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.102+0000 7f31908bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 0x55d450a56220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52042/0 (socket says 172.21.15.64:52042) 2024-05-09T09:19:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.102+0000 7f31908bc640 1 -- 172.21.15.64:0/3576559707 learned_addr learned my addr 172.21.15.64:0/3576559707 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.102+0000 7f31918be640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.102+0000 7f31910bd640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a0e0a0 0x55d450a55ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.104+0000 7f31908bc640 1 -- 172.21.15.64:0/3576559707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 msgr2=0x55d450a57230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.104+0000 7f31908bc640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a57230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.104+0000 7f31908bc640 1 -- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a0e0a0 msgr2=0x55d450a55ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.104+0000 7f31908bc640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a0e0a0 0x55d450a55ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.104+0000 7f31908bc640 1 -- 172.21.15.64:0/3576559707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d450a57820 con 0x55d450a466b0 2024-05-09T09:19:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.105+0000 7f31908bc640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 0x55d450a56220 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f317c0052c0 tx=0x7f317c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa75ae009e4fec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.105+0000 7f3170ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:19:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.105+0000 7f3170ff9640 1 -- 172.21.15.64:0/3576559707 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c057040 con 0x55d450a466b0 2024-05-09T09:19:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.105+0000 7f3170ff9640 1 -- 172.21.15.64:0/3576559707 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f317c003e00 con 0x55d450a466b0 2024-05-09T09:19:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.106+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31800035f0 con 0x55d450a466b0 2024-05-09T09:19:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.106+0000 7f3170ff9640 1 -- 172.21.15.64:0/3576559707 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f317c031dc0 con 0x55d450a466b0 2024-05-09T09:19:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.106+0000 7f319602a740 10 client.18129 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.106+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d450ac2c80 con 0x55d450a466b0 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f3170ff9640 1 -- 172.21.15.64:0/3576559707 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31800035f0 con 0x55d450a466b0 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f3170ff9640 1 -- 172.21.15.64:0/3576559707 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f317c049050 con 0x55d450a466b0 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f319602a740 10 client.18129 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f319602a740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f319602a740 20 client.18129 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f319602a740 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a4c210 0x55d450ac5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f319602a740 4 client.18129 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d450a509d0 con 0x55d450a4c210 2024-05-09T09:19:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.107+0000 7f31910bd640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a4c210 0x55d450ac5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.108+0000 7f31910bd640 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a4c210 0x55d450ac5690 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ae9a86518efc84 server_cookie=b6a365fa686d7271 in_seq=0 out_seq=0 2024-05-09T09:19:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.108+0000 7f3170ff9640 10 client.18129 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.109+0000 7f3170ff9640 1 -- 172.21.15.64:0/3576559707 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d450a509d0 con 0x55d450a4c210 2024-05-09T09:19:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.109+0000 7f3170ff9640 10 client.18129 handle_command_reply: tid=0 2024-05-09T09:19:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.109+0000 7f319602a740 1 client.18129 shutdown 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.109+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 msgr2=0x55d450a56220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.109+0000 7f319602a740 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 0x55d450a56220 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f317c0052c0 tx=0x7f317c005a40 comp rx=0 tx=0).stop 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 shutdown_connections 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d450a4c210 0x55d450ac5690 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d450a470a0 0x55d450a57230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d450a466b0 0x55d450a56220 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 --2- 172.21.15.64:0/3576559707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d450a0e0a0 0x55d450a55ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 >> 172.21.15.64:0/3576559707 conn(0x55d450a47e80 msgr2=0x55d450a409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f3170ff9640 0 client.18129 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 shutdown_connections 2024-05-09T09:19:48.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 1 -- 172.21.15.64:0/3576559707 wait complete. 2024-05-09T09:19:48.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.110+0000 7f319602a740 20 client.18129 trim_cache size 0 max 16384 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "9ef5d04e-008c-43cb-ae88-e1701f132307", 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f31740a2c10 msgr2=0x7f31740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f31740a2c10 0x7f31740a50d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f317c004af0 tx=0x7f317c044000 comp rx=0 tx=0).stop 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c072c70 msgr2=0x7f318c1199b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c072c70 0x7f318c1199b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f318400e790 tx=0x7f318400ec50 comp rx=0 tx=0).stop 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 shutdown_connections 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f31740a2c10 0x7f31740a50d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c1098b0 0x7f318c117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f318c108f10 0x7f318c119ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 --2- 172.21.15.64:0/3443598288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f318c072c70 0x7f318c1199b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 >> 172.21.15.64:0/3443598288 conn(0x7f318c06e740 msgr2=0x7f318c10c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 shutdown_connections 2024-05-09T09:19:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.112+0000 7f31920bf640 1 -- 172.21.15.64:0/3443598288 wait complete. 2024-05-09T09:19:48.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:19:48.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:19:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 -- 172.21.15.64:0/278723944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 msgr2=0x7fe9e810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/278723944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e810e170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009ca0 tx=0x7fe9e002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 -- 172.21.15.64:0/278723944 shutdown_connections 2024-05-09T09:19:48.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/278723944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e810e170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/278723944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9e8108f10 0x7fe9e8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/278723944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8072c70 0x7fe9e8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 -- 172.21.15.64:0/278723944 >> 172.21.15.64:0/278723944 conn(0x7fe9e806e740 msgr2=0x7fe9e8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 -- 172.21.15.64:0/278723944 shutdown_connections 2024-05-09T09:19:48.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.333+0000 7fe9e7fff640 1 -- 172.21.15.64:0/278723944 wait complete. 2024-05-09T09:19:48.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 Processor -- start 2024-05-09T09:19:48.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 -- start start 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9e8072c70 0x7fe9e81a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8108f10 0x7fe9e81a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e81adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e811c070 con 0x7fe9e8108f10 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e811bef0 con 0x7fe9e81098b0 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.334+0000 7fe9e7fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e811c1f0 con 0x7fe9e8072c70 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e81adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e81adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43792/0 (socket says 172.21.15.64:43792) 2024-05-09T09:19:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 -- 172.21.15.64:0/1571021273 learned_addr learned my addr 172.21.15.64:0/1571021273 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9e8072c70 0x7fe9e81a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9dffff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8108f10 0x7fe9e81a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 -- 172.21.15.64:0/1571021273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9e8072c70 msgr2=0x7fe9e81a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9e8072c70 0x7fe9e81a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 -- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8108f10 msgr2=0x7fe9e81a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8108f10 0x7fe9e81a6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9e77fe640 1 -- 172.21.15.64:0/1571021273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e81ae310 con 0x7fe9e81098b0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.335+0000 7fe9dffff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8108f10 0x7fe9e81a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.336+0000 7fe9e77fe640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e81adca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0031040 tx=0x7fe9e0003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.336+0000 7fe9e4ff9640 1 -- 172.21.15.64:0/1571021273 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0004040 con 0x7fe9e81098b0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.336+0000 7fe9e4ff9640 1 -- 172.21.15.64:0/1571021273 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9e0004690 con 0x7fe9e81098b0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.336+0000 7fe9e4ff9640 1 -- 172.21.15.64:0/1571021273 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0030cb0 con 0x7fe9e81098b0 2024-05-09T09:19:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.336+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e81ae5a0 con 0x7fe9e81098b0 2024-05-09T09:19:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.336+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e81aeaa0 con 0x7fe9e81098b0 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.340+0000 7fe9e4ff9640 1 -- 172.21.15.64:0/1571021273 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9e00041e0 con 0x7fe9e81098b0 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.340+0000 7fe9f0186740 1 Processor -- start 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.340+0000 7fe9f0186740 1 -- start start 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b35540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35b10 0x5615f6b3a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9f0186740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5615f6a99d00 con 0x5615f6b35b10 2024-05-09T09:19:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9f0186740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5615f6a414c0 con 0x5615f6b3ac80 2024-05-09T09:19:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9f0186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5615f6a95cc0 con 0x5615f6b35120 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b35540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b35540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43652/0 (socket says 172.21.15.64:43652) 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.341+0000 7fe9dffff640 1 -- 172.21.15.64:0/2071081539 learned_addr learned my addr 172.21.15.64:0/2071081539 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dffff640 1 -- 172.21.15.64:0/2071081539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 msgr2=0x5615f6b3d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dffff640 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b3d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dffff640 1 -- 172.21.15.64:0/2071081539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35b10 msgr2=0x5615f6b3a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dffff640 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35b10 0x5615f6b3a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dffff640 1 -- 172.21.15.64:0/2071081539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f68777d0 con 0x5615f6b35120 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9e4ff9640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9d40a2d10 0x7fe9d40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dffff640 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b35540 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8014a00 tx=0x7fe9d800b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faee76a88faa59bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dd7fa640 1 -- 172.21.15.64:0/2071081539 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8004020 con 0x5615f6b35120 2024-05-09T09:19:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dd7fa640 1 -- 172.21.15.64:0/2071081539 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9d8011cf0 con 0x5615f6b35120 2024-05-09T09:19:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9dd7fa640 1 -- 172.21.15.64:0/2071081539 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8003850 con 0x5615f6b35120 2024-05-09T09:19:48.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.342+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9d40a2d10 0x7fe9d40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.343+0000 7fe9e4ff9640 1 -- 172.21.15.64:0/1571021273 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe9e00f0a80 con 0x7fe9e81098b0 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 -- 172.21.15.64:0/2071081539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 msgr2=0x5615f6b35540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b35540 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8014a00 tx=0x7fe9d800b2c0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 -- 172.21.15.64:0/2071081539 shutdown_connections 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b3d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35b10 0x5615f6b3a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2071081539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b35540 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 -- 172.21.15.64:0/2071081539 >> 172.21.15.64:0/2071081539 conn(0x5615f6b30790 msgr2=0x5615f6b30b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9d40a2d10 0x7fe9d40a51d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b620 tx=0x7fe9d000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 -- 172.21.15.64:0/2071081539 shutdown_connections 2024-05-09T09:19:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.344+0000 7fe9f0186740 1 -- 172.21.15.64:0/2071081539 wait complete. 2024-05-09T09:19:48.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.345+0000 7fe9f0186740 1 -- start start 2024-05-09T09:19:48.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.345+0000 7fe9f0186740 1 Processor -- start 2024-05-09T09:19:48.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.345+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b3f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b3fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b3ac80 0x5615f6b46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9f0186740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5615f6b3dc30 con 0x5615f6b3ac80 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9f0186740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5615f6a54b90 con 0x5615f6b35b10 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9f0186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5615f6b3ddb0 con 0x5615f6b35120 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9f0186740 10 client.0 _ll_register_callbacks cb 0x5615f6a90da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b3f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b3f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43662/0 (socket says 172.21.15.64:43662) 2024-05-09T09:19:48.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.346+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/2698574623 learned_addr learned my addr 172.21.15.64:0/2698574623 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 msgr2=0x5615f6b3fb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b3fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/2698574623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b3ac80 msgr2=0x5615f6b46bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:19:48.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b3ac80 0x5615f6b46bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/2698574623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f67ad3f0 con 0x5615f6b35120 2024-05-09T09:19:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b3f630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004700 tx=0x7fe9d000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb16f0d4b1fbc31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.347+0000 7fe9ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.348+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/2698574623 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d001e620 con 0x5615f6b35120 2024-05-09T09:19:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.348+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/2698574623 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe9d001e7c0 con 0x5615f6b35120 2024-05-09T09:19:48.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.348+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d8011cf0 con 0x5615f6b35120 2024-05-09T09:19:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/2698574623 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe9d001e9c0 con 0x5615f6b35120 2024-05-09T09:19:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9f0186740 10 client.27763 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5615f676d0f0 con 0x5615f6b35120 2024-05-09T09:19:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/2698574623 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9d8011cf0 con 0x5615f6b35120 2024-05-09T09:19:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/2698574623 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fe9d0026be0 con 0x5615f6b35120 2024-05-09T09:19:48.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9f0186740 10 client.27763 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.349+0000 7fe9f0186740 10 client.27763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.350+0000 7fe9f0186740 20 client.27763 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.350+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6b4b520 0x5615f6b4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.350+0000 7fe9f0186740 4 client.27763 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.350+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f6aa7430 con 0x5615f6b4b520 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.350+0000 7fe9dffff640 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6b4b520 0x5615f6b4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.351+0000 7fe9dffff640 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6b4b520 0x5615f6b4d9e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98eccc6de111ca34 server_cookie=d821b13fb7746ab0 in_seq=0 out_seq=0 2024-05-09T09:19:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.351+0000 7fe9ce7fc640 10 client.27763 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.352+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/2698574623 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615f6aa7430 con 0x5615f6b4b520 2024-05-09T09:19:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.352+0000 7fe9ce7fc640 10 client.27763 handle_command_reply: tid=0 2024-05-09T09:19:48.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.352+0000 7fe9f0186740 1 client.27763 shutdown 2024-05-09T09:19:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.352+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 msgr2=0x5615f6b3f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.352+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b3f630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004700 tx=0x7fe9d000b390 comp rx=0 tx=0).stop 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 shutdown_connections 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6b4b520 0x5615f6b4d9e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b3ac80 0x5615f6b46bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 --2- 172.21.15.64:0/2698574623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35120 0x5615f6b3f630 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 >> 172.21.15.64:0/2698574623 conn(0x5615f6b30790 msgr2=0x5615f6b2fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9ce7fc640 0 client.27763 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 shutdown_connections 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 1 -- 172.21.15.64:0/2698574623 wait complete. 2024-05-09T09:19:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.353+0000 7fe9f0186740 20 client.27763 trim_cache size 0 max 16384 2024-05-09T09:19:48.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.392+0000 7fe9f0186740 1 Processor -- start 2024-05-09T09:19:48.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.392+0000 7fe9f0186740 1 -- start start 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.392+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b3f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.392+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b46b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.392+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b3ac80 0x5615f6b2f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9f0186740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81bb920 con 0x5615f6b35120 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9f0186740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b91f0 con 0x5615f6b35b10 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9f0186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b7040 con 0x5615f6b3ac80 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b46b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b46b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43812/0 (socket says 172.21.15.64:43812) 2024-05-09T09:19:48.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9dffff640 1 -- 172.21.15.64:0/408060700 learned_addr learned my addr 172.21.15.64:0/408060700 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9e77fe640 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b3ac80 0x5615f6b2f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.393+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b3f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dffff640 1 -- 172.21.15.64:0/408060700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b3ac80 msgr2=0x5615f6b2f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dffff640 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b3ac80 0x5615f6b2f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dffff640 1 -- 172.21.15.64:0/408060700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 msgr2=0x5615f6b3f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dffff640 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b3f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dffff640 1 -- 172.21.15.64:0/408060700 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f6b21930 con 0x5615f6b35b10 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dffff640 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b46b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8014a00 tx=0x7fe9d8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353d5cdf4a43f104 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dcff9640 1 -- 172.21.15.64:0/408060700 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80140d0 con 0x5615f6b35b10 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dcff9640 1 -- 172.21.15.64:0/408060700 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9d80278a0 con 0x5615f6b35b10 2024-05-09T09:19:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.394+0000 7fe9dcff9640 1 -- 172.21.15.64:0/408060700 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8011aa0 con 0x5615f6b35b10 2024-05-09T09:19:48.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 -- 172.21.15.64:0/408060700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 msgr2=0x5615f6b46b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b46b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8014a00 tx=0x7fe9d8005fb0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 -- 172.21.15.64:0/408060700 shutdown_connections 2024-05-09T09:19:48.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b3ac80 0x5615f6b2f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b35b10 0x5615f6b46b70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 --2- 172.21.15.64:0/408060700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b3f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.396+0000 7fe9f0186740 1 -- 172.21.15.64:0/408060700 >> 172.21.15.64:0/408060700 conn(0x5615f6b30790 msgr2=0x5615f6b3f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.398+0000 7fe9f0186740 1 -- 172.21.15.64:0/408060700 shutdown_connections 2024-05-09T09:19:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.398+0000 7fe9f0186740 1 -- 172.21.15.64:0/408060700 wait complete. 2024-05-09T09:19:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.398+0000 7fe9f0186740 1 -- start start 2024-05-09T09:19:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 Processor -- start 2024-05-09T09:19:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b43f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 0x5615f6b444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b45440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b76b0 con 0x5615f6b35120 2024-05-09T09:19:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b8b20 con 0x5615f6b3ac80 2024-05-09T09:19:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b7d80 con 0x5615f6b35b10 2024-05-09T09:19:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.399+0000 7fe9f0186740 10 client.0 _ll_register_callbacks cb 0x7fe9d00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.400+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 0x5615f6b444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.400+0000 7fe9dffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 0x5615f6b444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43666/0 (socket says 172.21.15.64:43666) 2024-05-09T09:19:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.400+0000 7fe9dffff640 1 -- 172.21.15.64:0/354332372 learned_addr learned my addr 172.21.15.64:0/354332372 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.401+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b43f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9dffff640 1 -- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 msgr2=0x5615f6b45440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9dffff640 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b45440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9dffff640 1 -- 172.21.15.64:0/354332372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 msgr2=0x5615f6b43f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9dffff640 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b43f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9dffff640 1 -- 172.21.15.64:0/354332372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f6b45a30 con 0x5615f6b35b10 2024-05-09T09:19:48.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9dffff640 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 0x5615f6b444a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800bd30 tx=0x7fe9d800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5fdb6420f9dd0a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:48.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.402+0000 7fe9ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:48.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.404+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/354332372 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8034030 con 0x5615f6b35b10 2024-05-09T09:19:48.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.404+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d80278a0 con 0x5615f6b35b10 2024-05-09T09:19:48.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.404+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/354332372 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe9d8027a50 con 0x5615f6b35b10 2024-05-09T09:19:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.404+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/354332372 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe9d803e050 con 0x5615f6b35b10 2024-05-09T09:19:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.405+0000 7fe9f0186740 10 client.27769 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.405+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5615f6b46d40 con 0x5615f6b35b10 2024-05-09T09:19:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.405+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/354332372 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9d80278a0 con 0x5615f6b35b10 2024-05-09T09:19:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.406+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/354332372 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fe9d800f040 con 0x5615f6b35b10 2024-05-09T09:19:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9f0186740 10 client.27769 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9f0186740 10 client.27769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9f0186740 20 client.27769 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9f0186740 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6bb2060 0x5615f6bb4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9f0186740 4 client.27769 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:19:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615f6aa1780 con 0x5615f6bb2060 2024-05-09T09:19:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.407+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6bb2060 0x5615f6bb4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.409+0000 7fe9e6ffd640 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6bb2060 0x5615f6bb4480 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf224fac6ee117 server_cookie=8c2f30f4abb39f83 in_seq=0 out_seq=0 2024-05-09T09:19:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.409+0000 7fe9ce7fc640 10 client.27769 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.409+0000 7fe9ce7fc640 1 -- 172.21.15.64:0/354332372 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5615f6aa1780 con 0x5615f6bb2060 2024-05-09T09:19:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.410+0000 7fe9ce7fc640 10 client.27769 handle_command_reply: tid=0 2024-05-09T09:19:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.410+0000 7fe9f0186740 1 client.27769 shutdown 2024-05-09T09:19:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.410+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 msgr2=0x5615f6b444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.410+0000 7fe9f0186740 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 0x5615f6b444a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800bd30 tx=0x7fe9d800bd60 comp rx=0 tx=0).stop 2024-05-09T09:19:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.411+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 shutdown_connections 2024-05-09T09:19:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.411+0000 7fe9f0186740 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5615f6bb2060 0x5615f6bb4480 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.411+0000 7fe9f0186740 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5615f6b3ac80 0x5615f6b45440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.411+0000 7fe9f0186740 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615f6b35b10 0x5615f6b444a0 secure :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800bd30 tx=0x7fe9d800bd60 comp rx=0 tx=0).stop 2024-05-09T09:19:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.411+0000 7fe9f0186740 1 --2- 172.21.15.64:0/354332372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615f6b35120 0x5615f6b43f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.411+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 >> 172.21.15.64:0/354332372 conn(0x5615f6b30790 msgr2=0x5615f6b2f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.412+0000 7fe9ce7fc640 0 client.27769 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.412+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 shutdown_connections 2024-05-09T09:19:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.412+0000 7fe9f0186740 1 -- 172.21.15.64:0/354332372 wait complete. 2024-05-09T09:19:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.412+0000 7fe9f0186740 20 client.27769 trim_cache size 0 max 16384 2024-05-09T09:19:48.415 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:19:48.415 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: "9ef5d04e-008c-43cb-ae88-e1701f132307": { 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "9ef5d04e-008c-43cb-ae88-e1701f132307", 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:19:48.416 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:19:48.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.417+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9d40a2d10 msgr2=0x7fe9d40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.417+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9d40a2d10 0x7fe9d40a51d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b620 tx=0x7fe9d000b5b0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.417+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 msgr2=0x7fe9e81adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:48.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.417+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e81adca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0031040 tx=0x7fe9e0003ba0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 shutdown_connections 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe9d40a2d10 0x7fe9d40a51d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e81098b0 0x7fe9e81adca0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8108f10 0x7fe9e81a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1571021273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9e8072c70 0x7fe9e81a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 >> 172.21.15.64:0/1571021273 conn(0x7fe9e806e740 msgr2=0x7fe9e80703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 shutdown_connections 2024-05-09T09:19:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:19:48.418+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1571021273 wait complete. 2024-05-09T09:19:48.439 INFO:tasks.cephfs.filesystem:scrub status for tag:9ef5d04e-008c-43cb-ae88-e1701f132307 - {'path': '/', 'tag': '9ef5d04e-008c-43cb-ae88-e1701f132307', 'options': 'recursive,force'} 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:48 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:48 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:48 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:50.223 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:49 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:49 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:50.331 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:49 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:19:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.157+0000 7fa851672640 1 -- 172.21.15.64:0/4122466600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c109890 msgr2=0x7fa84c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.157+0000 7fa851672640 1 --2- 172.21.15.64:0/4122466600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c109890 0x7fa84c110120 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009900 tx=0x7fa83c02a410 comp rx=0 tx=0).stop 2024-05-09T09:19:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.157+0000 7fa84affd640 1 -- 172.21.15.64:0/4122466600 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c031880 con 0x7fa84c109890 2024-05-09T09:19:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 -- 172.21.15.64:0/4122466600 shutdown_connections 2024-05-09T09:19:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 --2- 172.21.15.64:0/4122466600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c109890 0x7fa84c110120 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 --2- 172.21.15.64:0/4122466600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 0x7fa84c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 --2- 172.21.15.64:0/4122466600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c107ce0 0x7fa84c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 -- 172.21.15.64:0/4122466600 >> 172.21.15.64:0/4122466600 conn(0x7fa84c076cc0 msgr2=0x7fa84c0790e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 -- 172.21.15.64:0/4122466600 shutdown_connections 2024-05-09T09:19:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.158+0000 7fa851672640 1 -- 172.21.15.64:0/4122466600 wait complete. 2024-05-09T09:19:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.159+0000 7fa851672640 1 Processor -- start 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.159+0000 7fa851672640 1 -- start start 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa851672640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c107ce0 0x7fa84c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa851672640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 0x7fa84c0707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa851672640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 0x7fa84c07df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa851672640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c113840 con 0x7fa84c109890 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa851672640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1136c0 con 0x7fa84c107ce0 2024-05-09T09:19:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa851672640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1139c0 con 0x7fa84c108ed0 2024-05-09T09:19:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa850e71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 0x7fa84c07df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa850e71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 0x7fa84c07df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52114/0 (socket says 172.21.15.64:52114) 2024-05-09T09:19:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa850e71640 1 -- 172.21.15.64:0/1092729614 learned_addr learned my addr 172.21.15.64:0/1092729614 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa84bfff640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c107ce0 0x7fa84c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.160+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 0x7fa84c0707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa850e71640 1 -- 172.21.15.64:0/1092729614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 msgr2=0x7fa84c0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa850e71640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 0x7fa84c0707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa850e71640 1 -- 172.21.15.64:0/1092729614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c107ce0 msgr2=0x7fa84c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa850e71640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c107ce0 0x7fa84c070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa850e71640 1 -- 172.21.15.64:0/1092729614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c07e4c0 con 0x7fa84c109890 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 0x7fa84c0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa84bfff640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c107ce0 0x7fa84c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.161+0000 7fa850e71640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 0x7fa84c07df80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009a20 tx=0x7fa83c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.162+0000 7fa8497fa640 1 -- 172.21.15.64:0/1092729614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c03f070 con 0x7fa84c109890 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.162+0000 7fa8497fa640 1 -- 172.21.15.64:0/1092729614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83c031c30 con 0x7fa84c109890 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.162+0000 7fa8497fa640 1 -- 172.21.15.64:0/1092729614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c035030 con 0x7fa84c109890 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.162+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c07e750 con 0x7fa84c109890 2024-05-09T09:19:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.163+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c07ebf0 con 0x7fa84c109890 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.167+0000 7fa8555e5740 1 Processor -- start 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 -- start start 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c9f1839340 con 0x55c9f18d53d0 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c9f17e1290 con 0x55c9f18daf30 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8555e5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c9f1835300 con 0x55c9f18d5dc0 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.168+0000 7fa8497fa640 1 -- 172.21.15.64:0/1092729614 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa83c042070 con 0x7fa84c109890 2024-05-09T09:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.170+0000 7fa8497fa640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8340b3db0 0x7fa8340b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.170+0000 7fa8497fa640 1 -- 172.21.15.64:0/1092729614 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa83c0efad0 con 0x7fa84c109890 2024-05-09T09:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.170+0000 7fa850e71640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8340b3db0 0x7fa8340b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.171+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.171+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18da8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43694/0 (socket says 172.21.15.64:43694) 2024-05-09T09:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.171+0000 7fa84bfff640 1 -- 172.21.15.64:0/594556594 learned_addr learned my addr 172.21.15.64:0/594556594 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.172+0000 7fa850e71640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8340b3db0 0x7fa8340b6270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0bbf80 tx=0x7fa83c003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.172+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.173+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.173+0000 7fa84b7fe640 1 -- 172.21.15.64:0/594556594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 msgr2=0x55c9f18da8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.173+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18da8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.173+0000 7fa84b7fe640 1 -- 172.21.15.64:0/594556594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 msgr2=0x55c9f18dd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.173+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18dd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.173+0000 7fa84b7fe640 1 -- 172.21.15.64:0/594556594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f18058e0 con 0x55c9f18d53d0 2024-05-09T09:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.174+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18d57f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa84c071780 tx=0x7fa840005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27637a954ab3bde server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.175+0000 7fa8327fc640 1 -- 172.21.15.64:0/594556594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840012070 con 0x55c9f18d53d0 2024-05-09T09:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.175+0000 7fa8327fc640 1 -- 172.21.15.64:0/594556594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa840009640 con 0x55c9f18d53d0 2024-05-09T09:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.175+0000 7fa8327fc640 1 -- 172.21.15.64:0/594556594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840010980 con 0x55c9f18d53d0 2024-05-09T09:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.176+0000 7fa8555e5740 1 -- 172.21.15.64:0/594556594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 msgr2=0x55c9f18d57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.176+0000 7fa8555e5740 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18d57f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa84c071780 tx=0x7fa840005f90 comp rx=0 tx=0).stop 2024-05-09T09:19:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.178+0000 7fa8555e5740 1 -- 172.21.15.64:0/594556594 shutdown_connections 2024-05-09T09:19:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.178+0000 7fa8555e5740 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.178+0000 7fa8555e5740 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18da8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.178+0000 7fa8555e5740 1 --2- 172.21.15.64:0/594556594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18d57f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.178+0000 7fa8555e5740 1 -- 172.21.15.64:0/594556594 >> 172.21.15.64:0/594556594 conn(0x55c9f18d0a40 msgr2=0x55c9f18d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.182+0000 7fa8555e5740 1 -- 172.21.15.64:0/594556594 shutdown_connections 2024-05-09T09:19:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.183+0000 7fa8555e5740 1 -- 172.21.15.64:0/594556594 wait complete. 2024-05-09T09:19:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.183+0000 7fa8555e5740 1 -- start start 2024-05-09T09:19:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.183+0000 7fa8555e5740 1 Processor -- start 2024-05-09T09:19:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.184+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18df5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.184+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.184+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.184+0000 7fa8555e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c9f18dde80 con 0x55c9f18d53d0 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa8555e5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c9f17f4790 con 0x55c9f18daf30 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa8555e5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c9f18de000 con 0x55c9f18d5dc0 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:43858/0 (socket says 172.21.15.64:43858) 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa84b7fe640 1 -- 172.21.15.64:0/2039427372 learned_addr learned my addr 172.21.15.64:0/2039427372 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa8555e5740 10 client.0 _ll_register_callbacks cb 0x55c9f182fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18df5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.185+0000 7fa84bfff640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa84bfff640 1 -- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 msgr2=0x55c9f18e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa84bfff640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa84bfff640 1 -- 172.21.15.64:0/2039427372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 msgr2=0x55c9f18df5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa84bfff640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18df5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa84bfff640 1 -- 172.21.15.64:0/2039427372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f154d210 con 0x55c9f18d5dc0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa84bfff640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18dfb20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55c9f18d19b0 tx=0x7fa844010f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6908ceb3de90da0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa82b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa82b7fe640 1 -- 172.21.15.64:0/2039427372 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844003970 con 0x55c9f18d5dc0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa840009640 con 0x55c9f18d5dc0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.186+0000 7fa82b7fe640 1 -- 172.21.15.64:0/2039427372 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa844003b10 con 0x55c9f18d5dc0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa82b7fe640 1 -- 172.21.15.64:0/2039427372 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa844003d10 con 0x55c9f18d5dc0 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 10 client.27781 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c9f1619620 con 0x55c9f18d5dc0 2024-05-09T09:19:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa82b7fe640 1 -- 172.21.15.64:0/2039427372 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa840009640 con 0x55c9f18d5dc0 2024-05-09T09:19:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa82b7fe640 1 -- 172.21.15.64:0/2039427372 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa844004160 con 0x55c9f18d5dc0 2024-05-09T09:19:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 10 client.27781 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 10 client.27781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 20 client.27781 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f18eb0f0 0x55c9f18ed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 4 client.27781 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:19:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.187+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f1846a70 con 0x55c9f18eb0f0 2024-05-09T09:19:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.188+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f18eb0f0 0x55c9f18ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.189+0000 7fa82b7fe640 10 client.27781 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.189+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f18eb0f0 0x55c9f18ed520 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4935be41aa208ea server_cookie=f41bafb03aebe70d in_seq=0 out_seq=0 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa82b7fe640 1 -- 172.21.15.64:0/2039427372 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9f1846a70 con 0x55c9f18eb0f0 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa82b7fe640 10 client.27781 handle_command_reply: tid=0 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 client.27781 shutdown 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 msgr2=0x55c9f18dfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18dfb20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55c9f18d19b0 tx=0x7fa844010f90 comp rx=0 tx=0).stop 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 shutdown_connections 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f18eb0f0 0x55c9f18ed520 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18dfb20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.190+0000 7fa8555e5740 1 --2- 172.21.15.64:0/2039427372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18d53d0 0x55c9f18df5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.191+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 >> 172.21.15.64:0/2039427372 conn(0x55c9f18d0a40 msgr2=0x55c9f18cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.191+0000 7fa82b7fe640 0 client.27781 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.191+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 shutdown_connections 2024-05-09T09:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.191+0000 7fa8555e5740 1 -- 172.21.15.64:0/2039427372 wait complete. 2024-05-09T09:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.191+0000 7fa8555e5740 20 client.27781 trim_cache size 0 max 16384 2024-05-09T09:19:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.224+0000 7fa8555e5740 1 Processor -- start 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.224+0000 7fa8555e5740 1 -- start start 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.224+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.224+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.224+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa8555e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b3090 con 0x55c9f18daf30 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa8555e5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b0960 con 0x55c9f18d53d0 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43712/0 (socket says 172.21.15.64:43712) 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84bfff640 1 -- 172.21.15.64:0/433300658 learned_addr learned my addr 172.21.15.64:0/433300658 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa8555e5740 1 -- 172.21.15.64:0/433300658 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1ae7d0 con 0x55c9f18d5dc0 2024-05-09T09:19:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84bfff640 1 -- 172.21.15.64:0/433300658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 msgr2=0x55c9f18d7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.225+0000 7fa84bfff640 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18d7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa84bfff640 1 -- 172.21.15.64:0/433300658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 msgr2=0x55c9f18e6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa84bfff640 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa84bfff640 1 -- 172.21.15.64:0/433300658 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f18c2330 con 0x55c9f18d5dc0 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa84bfff640 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18d7580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa8440078b0 tx=0x7fa84400fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18962952728d8438 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa831ffb640 1 -- 172.21.15.64:0/433300658 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84401a470 con 0x55c9f18d5dc0 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa831ffb640 1 -- 172.21.15.64:0/433300658 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa844004360 con 0x55c9f18d5dc0 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.226+0000 7fa831ffb640 1 -- 172.21.15.64:0/433300658 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844004a40 con 0x55c9f18d5dc0 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.227+0000 7fa8555e5740 1 -- 172.21.15.64:0/433300658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 msgr2=0x55c9f18d7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.227+0000 7fa8555e5740 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18d7580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa8440078b0 tx=0x7fa84400fd40 comp rx=0 tx=0).stop 2024-05-09T09:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.227+0000 7fa8555e5740 1 -- 172.21.15.64:0/433300658 shutdown_connections 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.227+0000 7fa8555e5740 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.227+0000 7fa8555e5740 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18d7580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.227+0000 7fa8555e5740 1 --2- 172.21.15.64:0/433300658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18d7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.228+0000 7fa8555e5740 1 -- 172.21.15.64:0/433300658 >> 172.21.15.64:0/433300658 conn(0x55c9f18d0a40 msgr2=0x55c9f18da640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.228+0000 7fa8555e5740 1 -- 172.21.15.64:0/433300658 shutdown_connections 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.228+0000 7fa8555e5740 1 -- 172.21.15.64:0/433300658 wait complete. 2024-05-09T09:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.228+0000 7fa8555e5740 1 -- start start 2024-05-09T09:19:51.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:50 smithi179 ceph-mon[34086]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-05-09T09:19:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 Processor -- start 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1aee20 con 0x55c9f18daf30 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b0290 con 0x55c9f18d53d0 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1af4f0 con 0x55c9f18d5dc0 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa8555e5740 10 client.0 _ll_register_callbacks cb 0x7fa844010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa84bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18e4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:43726/0 (socket says 172.21.15.64:43726) 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.229+0000 7fa84bfff640 1 -- 172.21.15.64:0/3306705164 learned_addr learned my addr 172.21.15.64:0/3306705164 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84bfff640 1 -- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 msgr2=0x55c9f18e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84bfff640 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84bfff640 1 -- 172.21.15.64:0/3306705164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 msgr2=0x55c9f18e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84bfff640 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84bfff640 1 -- 172.21.15.64:0/3306705164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f18e5b00 con 0x55c9f18d5dc0 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa84bfff640 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18e4500 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa844004170 tx=0x7fa8440041a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d520eeff7782c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa82b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.230+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3306705164 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84400e7d0 con 0x55c9f18d5dc0 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3306705164 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa84402e070 con 0x55c9f18d5dc0 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844004360 con 0x55c9f18d5dc0 2024-05-09T09:19:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3306705164 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa84403e050 con 0x55c9f18d5dc0 2024-05-09T09:19:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa8555e5740 10 client.27787 fetch_fsmap learned FSMap version 45 2024-05-09T09:19:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c9f18e6df0 con 0x55c9f18d5dc0 2024-05-09T09:19:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3306705164 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa84400ee20 con 0x55c9f18d5dc0 2024-05-09T09:19:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.231+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3306705164 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa84401ab20 con 0x55c9f18d5dc0 2024-05-09T09:19:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa8555e5740 10 client.27787 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:19:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa8555e5740 10 client.27787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:19:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa8555e5740 20 client.27787 populate_metadata read hostname 'smithi064' 2024-05-09T09:19:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa8555e5740 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f1951d40 0x55c9f1954160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:19:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa8555e5740 4 client.27787 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:19:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa844004360 con 0x55c9f1951d40 2024-05-09T09:19:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.232+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f1951d40 0x55c9f1954160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:19:51.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:50 smithi139 ceph-mon[33599]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-05-09T09:19:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.234+0000 7fa84b7fe640 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f1951d40 0x55c9f1954160 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63918664400fdc5 server_cookie=3c8dda5497bb5cfc in_seq=0 out_seq=0 2024-05-09T09:19:51.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.234+0000 7fa82b7fe640 10 client.27787 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.261+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3306705164 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa844004360 con 0x55c9f1951d40 2024-05-09T09:19:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.261+0000 7fa82b7fe640 10 client.27787 handle_command_reply: tid=0 2024-05-09T09:19:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.261+0000 7fa8555e5740 1 client.27787 shutdown 2024-05-09T09:19:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.262+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 msgr2=0x55c9f18e4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.262+0000 7fa8555e5740 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18e4500 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa844004170 tx=0x7fa8440041a0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.262+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 shutdown_connections 2024-05-09T09:19:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c9f1951d40 0x55c9f1954160 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa82b7fe640 0 client.27787 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:19:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9f18daf30 0x55c9f18e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9f18d5dc0 0x55c9f18e4500 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 --2- 172.21.15.64:0/3306705164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c9f18d53d0 0x55c9f18e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 >> 172.21.15.64:0/3306705164 conn(0x55c9f18d0a40 msgr2=0x55c9f18da210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 shutdown_connections 2024-05-09T09:19:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 1 -- 172.21.15.64:0/3306705164 wait complete. 2024-05-09T09:19:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.263+0000 7fa8555e5740 20 client.27787 trim_cache size 0 max 16384 2024-05-09T09:19:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:19:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:19:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:19:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:19:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8340b3db0 msgr2=0x7fa8340b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8340b3db0 0x7fa8340b6270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0bbf80 tx=0x7fa83c003f90 comp rx=0 tx=0).stop 2024-05-09T09:19:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 msgr2=0x7fa84c07df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:19:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 0x7fa84c07df80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009a20 tx=0x7fa83c003b90 comp rx=0 tx=0).stop 2024-05-09T09:19:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 shutdown_connections 2024-05-09T09:19:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa8340b3db0 0x7fa8340b6270 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c109890 0x7fa84c07df80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa84c108ed0 0x7fa84c0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 --2- 172.21.15.64:0/1092729614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c107ce0 0x7fa84c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:19:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.264+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 >> 172.21.15.64:0/1092729614 conn(0x7fa84c076cc0 msgr2=0x7fa84c10c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:19:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.265+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 shutdown_connections 2024-05-09T09:19:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:19:51.265+0000 7fa851672640 1 -- 172.21.15.64:0/1092729614 wait complete. 2024-05-09T09:19:51.349 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:50 smithi064 ceph-mon[30149]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-05-09T09:19:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 570.0 s, 889.7 tps, lat 112.949 ms stddev 76.920 2024-05-09T09:19:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:52 smithi179 ceph-mon[34086]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-05-09T09:19:53.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:52 smithi139 ceph-mon[33599]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-05-09T09:19:53.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:52 smithi064 ceph-mon[30149]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-05-09T09:19:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:54 smithi179 ceph-mon[34086]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-05-09T09:19:55.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:54 smithi139 ceph-mon[33599]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-05-09T09:19:55.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:54 smithi064 ceph-mon[30149]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-05-09T09:19:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 575.0 s, 841.9 tps, lat 118.020 ms stddev 81.888 2024-05-09T09:19:57.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:56 smithi179 ceph-mon[34086]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 13 MiB/s wr, 967 op/s 2024-05-09T09:19:57.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:56 smithi139 ceph-mon[33599]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 13 MiB/s wr, 967 op/s 2024-05-09T09:19:57.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:56 smithi064 ceph-mon[30149]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 13 MiB/s wr, 967 op/s 2024-05-09T09:19:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:19:58 smithi179 ceph-mon[34086]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 12 MiB/s wr, 890 op/s 2024-05-09T09:19:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:19:58 smithi139 ceph-mon[33599]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 12 MiB/s wr, 890 op/s 2024-05-09T09:19:59.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:19:58 smithi064 ceph-mon[30149]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 12 MiB/s wr, 890 op/s 2024-05-09T09:20:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:00 smithi139 ceph-mon[33599]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-05-09T09:20:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:00 smithi139 ceph-mon[33599]: overall HEALTH_OK 2024-05-09T09:20:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:20:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:20:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:00 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:20:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:00 smithi179 ceph-mon[34086]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-05-09T09:20:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:00 smithi179 ceph-mon[34086]: overall HEALTH_OK 2024-05-09T09:20:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:20:01.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:20:01.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:00 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:20:01.354 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 580.0 s, 876.4 tps, lat 113.793 ms stddev 87.575 2024-05-09T09:20:01.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:00 smithi064 ceph-mon[30149]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-05-09T09:20:01.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:00 smithi064 ceph-mon[30149]: overall HEALTH_OK 2024-05-09T09:20:01.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:20:01.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:20:01.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:00 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:20:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:02 smithi139 ceph-mon[33599]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 848 op/s 2024-05-09T09:20:03.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:02 smithi179 ceph-mon[34086]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 848 op/s 2024-05-09T09:20:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:02 smithi064 ceph-mon[30149]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 848 op/s 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/3195910101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0100790 msgr2=0x7f2ad0100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/3195910101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0100790 0x7f2ad0100bb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009900 tx=0x7f2acc02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/3195910101 shutdown_connections 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/3195910101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0102340 0x7f2ad0108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/3195910101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 0x7f2ad0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/3195910101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0100790 0x7f2ad0100bb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/3195910101 >> 172.21.15.64:0/3195910101 conn(0x7f2ad00fbd00 msgr2=0x7f2ad00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/3195910101 shutdown_connections 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.415+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/3195910101 wait complete. 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.416+0000 7f2ad7e3f640 1 Processor -- start 2024-05-09T09:20:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 -- start start 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0100790 0x7f2ad010b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 0x7f2ad0109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 0x7f2ad0109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010d400 con 0x7f2ad0102340 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010d280 con 0x7f2ad0100790 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad7e3f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010d580 con 0x7f2ad0101980 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 0x7f2ad0109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 0x7f2ad0109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40038/0 (socket says 172.21.15.64:40038) 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 -- 172.21.15.64:0/511930791 learned_addr learned my addr 172.21.15.64:0/511930791 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad53da640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 0x7f2ad0109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0100790 0x7f2ad010b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 -- 172.21.15.64:0/511930791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 msgr2=0x7f2ad0109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 0x7f2ad0109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 -- 172.21.15.64:0/511930791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0100790 msgr2=0x7f2ad010b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.417+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0100790 0x7f2ad010b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ad63dc640 1 -- 172.21.15.64:0/511930791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad010a520 con 0x7f2ad0102340 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0100790 0x7f2ad010b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ad53da640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 0x7f2ad0109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 0x7f2ad0109c90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800eb10 tx=0x7f2ac800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ac6ffd640 1 -- 172.21.15.64:0/511930791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8004b20 con 0x7f2ad0102340 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ac6ffd640 1 -- 172.21.15.64:0/511930791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac8004cc0 con 0x7f2ad0102340 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.418+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad01a1b70 con 0x7f2ad0102340 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.419+0000 7f2ac6ffd640 1 -- 172.21.15.64:0/511930791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8010690 con 0x7f2ad0102340 2024-05-09T09:20:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.419+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad01a20c0 con 0x7f2ad0102340 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.423+0000 7f2ac6ffd640 1 -- 172.21.15.64:0/511930791 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ac8005020 con 0x7f2ad0102340 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.424+0000 7f2adac2f740 1 Processor -- start 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.424+0000 7f2adac2f740 1 -- start start 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 0x559096771670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x559096776740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x559096779240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2adac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5590966d5340 con 0x559096771c40 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2adac2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55909667d290 con 0x559096776db0 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2adac2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5590966d1300 con 0x559096771250 2024-05-09T09:20:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad5bdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x559096776740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 0x559096771670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 0x559096771670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44382/0 (socket says 172.21.15.64:44382) 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad5bdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x559096776740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40054/0 (socket says 172.21.15.64:40054) 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad53da640 1 -- 172.21.15.64:0/2109273951 learned_addr learned my addr 172.21.15.64:0/2109273951 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad53da640 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x559096779240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad53da640 1 -- 172.21.15.64:0/2109273951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 msgr2=0x559096779240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.425+0000 7f2ad53da640 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x559096779240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ad53da640 1 -- 172.21.15.64:0/2109273951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 msgr2=0x559096776740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ad53da640 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x559096776740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ad53da640 1 -- 172.21.15.64:0/2109273951 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590966a18e0 con 0x559096771250 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ac6ffd640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2abc0a2cb0 0x7f2abc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ad53da640 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 0x559096771670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0005ca0 tx=0x7f2ac000bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37c35961156985e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ac6ffd640 1 -- 172.21.15.64:0/511930791 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ac80cf960 con 0x7f2ad0102340 2024-05-09T09:20:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.426+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2abc0a2cb0 0x7f2abc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.427+0000 7f2abbfff640 1 -- 172.21.15.64:0/2109273951 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0012cd0 con 0x559096771250 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.427+0000 7f2abbfff640 1 -- 172.21.15.64:0/2109273951 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac0010c00 con 0x559096771250 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.427+0000 7f2abbfff640 1 -- 172.21.15.64:0/2109273951 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003b20 con 0x559096771250 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 -- 172.21.15.64:0/2109273951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 msgr2=0x559096771670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 0x559096771670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0005ca0 tx=0x7f2ac000bbf0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2abc0a2cb0 0x7f2abc0a5170 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5590967726b0 tx=0x7f2acc005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 -- 172.21.15.64:0/2109273951 shutdown_connections 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x559096779240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x559096776740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2109273951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771250 0x559096771670 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 -- 172.21.15.64:0/2109273951 >> 172.21.15.64:0/2109273951 conn(0x55909676c8c0 msgr2=0x55909676ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.428+0000 7f2adac2f740 1 -- 172.21.15.64:0/2109273951 shutdown_connections 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.429+0000 7f2adac2f740 1 -- 172.21.15.64:0/2109273951 wait complete. 2024-05-09T09:20:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.429+0000 7f2adac2f740 1 -- start start 2024-05-09T09:20:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.429+0000 7f2adac2f740 1 Processor -- start 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x559096783d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096784250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x55909678b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559096779d60 con 0x559096771250 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559096690790 con 0x559096776db0 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559096779ee0 con 0x559096771c40 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2adac2f740 10 client.0 _ll_register_callbacks cb 0x5590966cba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2ad5bdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x559096783d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2ad5bdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x559096783d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40068/0 (socket says 172.21.15.64:40068) 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096784250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2ad5bdb640 1 -- 172.21.15.64:0/2214672376 learned_addr learned my addr 172.21.15.64:0/2214672376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.430+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x55909678b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ad63dc640 1 -- 172.21.15.64:0/2214672376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 msgr2=0x559096784250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096784250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ad63dc640 1 -- 172.21.15.64:0/2214672376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 msgr2=0x559096783d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x559096783d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ad63dc640 1 -- 172.21.15.64:0/2214672376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590963e9210 con 0x559096776db0 2024-05-09T09:20:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x55909678b290 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80cd120 tx=0x7f2ac809b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e344e66073d1b4a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.431+0000 7f2ab97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.432+0000 7f2ab97fa640 1 -- 172.21.15.64:0/2214672376 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80a4810 con 0x559096776db0 2024-05-09T09:20:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.432+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac0010c00 con 0x559096776db0 2024-05-09T09:20:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.432+0000 7f2ab97fa640 1 -- 172.21.15.64:0/2214672376 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2ac8005310 con 0x559096776db0 2024-05-09T09:20:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.432+0000 7f2ab97fa640 1 -- 172.21.15.64:0/2214672376 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2ac80a01e0 con 0x559096776db0 2024-05-09T09:20:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.432+0000 7f2adac2f740 10 client.27659 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.432+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5590964b5620 con 0x559096776db0 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2ab97fa640 1 -- 172.21.15.64:0/2214672376 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac0010c00 con 0x559096776db0 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2ab97fa640 1 -- 172.21.15.64:0/2214672376 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f2ac809acf0 con 0x559096776db0 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2adac2f740 10 client.27659 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2adac2f740 10 client.27659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2adac2f740 20 client.27659 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55909678fb80 0x559096791fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2adac2f740 4 client.27659 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590966e2a70 con 0x55909678fb80 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.433+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55909678fb80 0x559096791fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.434+0000 7f2ab97fa640 10 client.27659 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.434+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55909678fb80 0x559096791fb0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91eb8b97b6c865 server_cookie=642b2337077f01e4 in_seq=0 out_seq=0 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2ab97fa640 1 -- 172.21.15.64:0/2214672376 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590966e2a70 con 0x55909678fb80 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2ab97fa640 10 client.27659 handle_command_reply: tid=0 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 client.27659 shutdown 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 msgr2=0x55909678b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x55909678b290 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80cd120 tx=0x7f2ac809b790 comp rx=0 tx=0).stop 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 shutdown_connections 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55909678fb80 0x559096791fb0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x55909678b290 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096784250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 --2- 172.21.15.64:0/2214672376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x559096783d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.435+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 >> 172.21.15.64:0/2214672376 conn(0x55909676c8c0 msgr2=0x55909676ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.436+0000 7f2ab97fa640 0 client.27659 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.436+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 shutdown_connections 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.436+0000 7f2adac2f740 1 -- 172.21.15.64:0/2214672376 wait complete. 2024-05-09T09:20:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.436+0000 7f2adac2f740 20 client.27659 trim_cache size 0 max 16384 2024-05-09T09:20:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.464+0000 7f2adac2f740 1 Processor -- start 2024-05-09T09:20:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.464+0000 7f2adac2f740 1 -- start start 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.464+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096771250 0x559096783710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.464+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x55909678b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 0x5590967978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x55909678b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x55909678b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40076/0 (socket says 172.21.15.64:40076) 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2ad63dc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 0x5590967978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2ad63dc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 0x5590967978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44394/0 (socket says 172.21.15.64:44394) 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2ad53da640 1 -- 172.21.15.64:0/162829889 learned_addr learned my addr 172.21.15.64:0/162829889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01aab50 con 0x559096771c40 2024-05-09T09:20:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01a8420 con 0x559096771250 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.465+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01a6270 con 0x559096776db0 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096771250 0x559096783710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad53da640 1 -- 172.21.15.64:0/162829889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 msgr2=0x5590967978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad53da640 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 0x5590967978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad53da640 1 -- 172.21.15.64:0/162829889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096771250 msgr2=0x559096783710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad53da640 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096771250 0x559096783710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad53da640 1 -- 172.21.15.64:0/162829889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909676af00 con 0x559096771c40 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 0x5590967978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096771250 0x559096783710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.466+0000 7f2ad53da640 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x55909678b250 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5590967724a0 tx=0x7f2ac000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b045568c6f57ac1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.467+0000 7f2abb7fe640 1 -- 172.21.15.64:0/162829889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0010dd0 con 0x559096771c40 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.467+0000 7f2abb7fe640 1 -- 172.21.15.64:0/162829889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac0016040 con 0x559096771c40 2024-05-09T09:20:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.467+0000 7f2abb7fe640 1 -- 172.21.15.64:0/162829889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00049e0 con 0x559096771c40 2024-05-09T09:20:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.468+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 msgr2=0x55909678b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.468+0000 7f2adac2f740 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x55909678b250 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5590967724a0 tx=0x7f2ac000bfd0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.473+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 shutdown_connections 2024-05-09T09:20:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.473+0000 7f2adac2f740 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096776db0 0x5590967978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.473+0000 7f2adac2f740 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771c40 0x55909678b250 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.473+0000 7f2adac2f740 1 --2- 172.21.15.64:0/162829889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096771250 0x559096783710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.473+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 >> 172.21.15.64:0/162829889 conn(0x55909676c8c0 msgr2=0x559096783000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.475+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 shutdown_connections 2024-05-09T09:20:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.476+0000 7f2adac2f740 1 -- 172.21.15.64:0/162829889 wait complete. 2024-05-09T09:20:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.476+0000 7f2adac2f740 1 -- start start 2024-05-09T09:20:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.477+0000 7f2adac2f740 1 Processor -- start 2024-05-09T09:20:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.477+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x5590967884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.477+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096788a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.478+0000 7f2adac2f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x5590967899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.478+0000 7f2adac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01a68e0 con 0x559096771250 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.478+0000 7f2adac2f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01a7d50 con 0x559096776db0 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.478+0000 7f2adac2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01a6fb0 con 0x559096771c40 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.478+0000 7f2adac2f740 10 client.0 _ll_register_callbacks cb 0x7f2ac8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.479+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096788a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.479+0000 7f2ad53da640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096788a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44404/0 (socket says 172.21.15.64:44404) 2024-05-09T09:20:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.479+0000 7f2ad53da640 1 -- 172.21.15.64:0/3431448930 learned_addr learned my addr 172.21.15.64:0/3431448930 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.481+0000 7f2ad63dc640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x5590967899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.481+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x5590967884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ad5bdb640 1 -- 172.21.15.64:0/3431448930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 msgr2=0x559096788a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096788a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ad5bdb640 1 -- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 msgr2=0x5590967899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x5590967899d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ad5bdb640 1 -- 172.21.15.64:0/3431448930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559096789fc0 con 0x559096771250 2024-05-09T09:20:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ad5bdb640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x5590967884f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2acc006010 tx=0x7f2acc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bb3c89501beb37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.483+0000 7f2ab97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.484+0000 7f2ab97fa640 1 -- 172.21.15.64:0/3431448930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc035040 con 0x559096771250 2024-05-09T09:20:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.484+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac0016040 con 0x559096771250 2024-05-09T09:20:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.484+0000 7f2ab97fa640 1 -- 172.21.15.64:0/3431448930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2acc051c60 con 0x559096771250 2024-05-09T09:20:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.485+0000 7f2ab97fa640 1 -- 172.21.15.64:0/3431448930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2acc063050 con 0x559096771250 2024-05-09T09:20:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.485+0000 7f2adac2f740 10 client.18174 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.485+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55909678b340 con 0x559096771250 2024-05-09T09:20:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.485+0000 7f2ab97fa640 1 -- 172.21.15.64:0/3431448930 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac0016040 con 0x559096771250 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2ab97fa640 1 -- 172.21.15.64:0/3431448930 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f2acc004950 con 0x559096771250 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2adac2f740 10 client.18174 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2adac2f740 10 client.18174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2adac2f740 20 client.18174 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2adac2f740 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5590967f6660 0x5590967f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2adac2f740 4 client.18174 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.486+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590966dcdc0 con 0x5590967f6660 2024-05-09T09:20:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.487+0000 7f2ad53da640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5590967f6660 0x5590967f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.488+0000 7f2ad53da640 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5590967f6660 0x5590967f8a80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976be88efd34cd33 server_cookie=ee061511ea8e137e in_seq=0 out_seq=0 2024-05-09T09:20:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.488+0000 7f2ab97fa640 10 client.18174 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.514+0000 7f2ab97fa640 1 -- 172.21.15.64:0/3431448930 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590966dcdc0 con 0x5590967f6660 2024-05-09T09:20:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.514+0000 7f2ab97fa640 10 client.18174 handle_command_reply: tid=0 2024-05-09T09:20:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.514+0000 7f2adac2f740 1 client.18174 shutdown 2024-05-09T09:20:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 msgr2=0x5590967884f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x5590967884f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2acc006010 tx=0x7f2acc009260 comp rx=0 tx=0).stop 2024-05-09T09:20:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 shutdown_connections 2024-05-09T09:20:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5590967f6660 0x5590967f8a80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559096776db0 0x5590967899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559096771c40 0x559096788a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 --2- 172.21.15.64:0/3431448930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559096771250 0x5590967884f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 >> 172.21.15.64:0/3431448930 conn(0x55909676c8c0 msgr2=0x55909676f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.517+0000 7f2ab97fa640 0 client.18174 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.519+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 shutdown_connections 2024-05-09T09:20:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.519+0000 7f2adac2f740 1 -- 172.21.15.64:0/3431448930 wait complete. 2024-05-09T09:20:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.519+0000 7f2adac2f740 20 client.18174 trim_cache size 0 max 16384 2024-05-09T09:20:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:20:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:20:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:20:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:20:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.521+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2abc0a2cb0 msgr2=0x7f2abc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.521+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2abc0a2cb0 0x7f2abc0a5170 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5590967726b0 tx=0x7f2acc005b50 comp rx=0 tx=0).stop 2024-05-09T09:20:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.521+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 msgr2=0x7f2ad0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.521+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 0x7f2ad0109c90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800eb10 tx=0x7f2ac800c2d0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.522+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 shutdown_connections 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.522+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2abc0a2cb0 0x7f2abc0a5170 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.522+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad0102340 0x7f2ad0109c90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.522+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ad0101980 0x7f2ad0109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.522+0000 7f2ad7e3f640 1 --2- 172.21.15.64:0/511930791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ad0100790 0x7f2ad010b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.522+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 >> 172.21.15.64:0/511930791 conn(0x7f2ad00fbd00 msgr2=0x7f2ad0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.523+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 shutdown_connections 2024-05-09T09:20:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:04.523+0000 7f2ad7e3f640 1 -- 172.21.15.64:0/511930791 wait complete. 2024-05-09T09:20:05.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:04 smithi179 ceph-mon[34086]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 959 op/s 2024-05-09T09:20:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:04 smithi139 ceph-mon[33599]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 959 op/s 2024-05-09T09:20:05.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:04 smithi064 ceph-mon[30149]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 959 op/s 2024-05-09T09:20:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 585.0 s, 899.6 tps, lat 110.780 ms stddev 65.451 2024-05-09T09:20:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:06 smithi179 ceph-mon[34086]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-05-09T09:20:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:06 smithi139 ceph-mon[33599]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-05-09T09:20:07.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:06 smithi064 ceph-mon[30149]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-05-09T09:20:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:08 smithi179 ceph-mon[34086]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-05-09T09:20:09.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:08 smithi139 ceph-mon[33599]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-05-09T09:20:09.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:08 smithi064 ceph-mon[30149]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-05-09T09:20:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:10 smithi179 ceph-mon[34086]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-05-09T09:20:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:10 smithi139 ceph-mon[33599]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-05-09T09:20:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 590.0 s, 866.2 tps, lat 114.748 ms stddev 77.847 2024-05-09T09:20:11.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:10 smithi064 ceph-mon[30149]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-05-09T09:20:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:12 smithi139 ceph-mon[33599]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 828 op/s 2024-05-09T09:20:13.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:12 smithi064 ceph-mon[30149]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 828 op/s 2024-05-09T09:20:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:12 smithi179 ceph-mon[34086]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 828 op/s 2024-05-09T09:20:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:14 smithi139 ceph-mon[33599]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 947 op/s 2024-05-09T09:20:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:14 smithi064 ceph-mon[30149]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 947 op/s 2024-05-09T09:20:15.483 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:14 smithi179 ceph-mon[34086]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 947 op/s 2024-05-09T09:20:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 595.0 s, 819.4 tps, lat 121.066 ms stddev 113.844 2024-05-09T09:20:17.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:16 smithi139 ceph-mon[33599]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 914 op/s 2024-05-09T09:20:17.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:16 smithi064 ceph-mon[30149]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 914 op/s 2024-05-09T09:20:17.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:16 smithi179 ceph-mon[34086]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 914 op/s 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.680+0000 7ff921414640 1 -- 172.21.15.64:0/2151463605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c1088c0 msgr2=0x7ff91c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.680+0000 7ff921414640 1 --2- 172.21.15.64:0/2151463605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c1088c0 0x7ff91c10acb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff914009ca0 tx=0x7ff91402b9d0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.681+0000 7ff921414640 1 -- 172.21.15.64:0/2151463605 shutdown_connections 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.681+0000 7ff921414640 1 --2- 172.21.15.64:0/2151463605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c10dd60 0x7ff91c110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.681+0000 7ff921414640 1 --2- 172.21.15.64:0/2151463605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10b280 0x7ff91c10d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.681+0000 7ff921414640 1 --2- 172.21.15.64:0/2151463605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c1088c0 0x7ff91c10acb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.681+0000 7ff921414640 1 -- 172.21.15.64:0/2151463605 >> 172.21.15.64:0/2151463605 conn(0x7ff91c06e890 msgr2=0x7ff91c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.682+0000 7ff921414640 1 -- 172.21.15.64:0/2151463605 shutdown_connections 2024-05-09T09:20:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.682+0000 7ff921414640 1 -- 172.21.15.64:0/2151463605 wait complete. 2024-05-09T09:20:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.683+0000 7ff921414640 1 Processor -- start 2024-05-09T09:20:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.683+0000 7ff921414640 1 -- start start 2024-05-09T09:20:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff921414640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c1088c0 0x7ff91c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff921414640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c10b280 0x7ff91c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c1088c0 0x7ff91c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c1088c0 0x7ff91c19dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55984/0 (socket says 172.21.15.64:55984) 2024-05-09T09:20:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff921414640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10dd60 0x7ff91c1a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff91bfff640 1 -- 172.21.15.64:0/4156888517 learned_addr learned my addr 172.21.15.64:0/4156888517 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.684+0000 7ff91b7fe640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c10b280 0x7ff91c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.685+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1137f0 con 0x7ff91c1088c0 2024-05-09T09:20:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.685+0000 7ff920c13640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10dd60 0x7ff91c1a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.685+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c113670 con 0x7ff91c10dd60 2024-05-09T09:20:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.685+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c113970 con 0x7ff91c10b280 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff91b7fe640 1 -- 172.21.15.64:0/4156888517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10dd60 msgr2=0x7ff91c1a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff91b7fe640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10dd60 0x7ff91c1a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff91b7fe640 1 -- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c1088c0 msgr2=0x7ff91c19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff91b7fe640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c1088c0 0x7ff91c19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff91b7fe640 1 -- 172.21.15.64:0/4156888517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c1a5900 con 0x7ff91c10b280 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff920c13640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10dd60 0x7ff91c1a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.686+0000 7ff91b7fe640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c10b280 0x7ff91c19e230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009e60 tx=0x7ff90c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.687+0000 7ff9197fa640 1 -- 172.21.15.64:0/4156888517 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c01e070 con 0x7ff91c10b280 2024-05-09T09:20:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.687+0000 7ff9197fa640 1 -- 172.21.15.64:0/4156888517 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff90c00ccc0 con 0x7ff91c10b280 2024-05-09T09:20:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.687+0000 7ff9197fa640 1 -- 172.21.15.64:0/4156888517 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c019900 con 0x7ff91c10b280 2024-05-09T09:20:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.687+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c1a5bf0 con 0x7ff91c10b280 2024-05-09T09:20:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.687+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1a6090 con 0x7ff91c10b280 2024-05-09T09:20:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.691+0000 7ff925388740 1 Processor -- start 2024-05-09T09:20:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.691+0000 7ff925388740 1 -- start start 2024-05-09T09:20:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff925388740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bde37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff925388740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bde6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff925388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde6ed0 0x55b64bdf3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff925388740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b64bd47340 con 0x55b64bde3380 2024-05-09T09:20:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff925388740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b64bcef290 con 0x55b64bde3d70 2024-05-09T09:20:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff925388740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b64bd43300 con 0x55b64bde6ed0 2024-05-09T09:20:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff920c13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bde37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff920c13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bde37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55996/0 (socket says 172.21.15.64:55996) 2024-05-09T09:20:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff920c13640 1 -- 172.21.15.64:0/4121735242 learned_addr learned my addr 172.21.15.64:0/4121735242 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff9197fa640 1 -- 172.21.15.64:0/4156888517 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff90c021070 con 0x7ff91c10b280 2024-05-09T09:20:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.692+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bde6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.693+0000 7ff91b7fe640 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde6ed0 0x55b64bdf3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff91bfff640 1 -- 172.21.15.64:0/4121735242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde6ed0 msgr2=0x55b64bdf3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde6ed0 0x55b64bdf3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff91bfff640 1 -- 172.21.15.64:0/4121735242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 msgr2=0x55b64bde37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bde37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff91bfff640 1 -- 172.21.15.64:0/4121735242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b64bd138e0 con 0x55b64bde3d70 2024-05-09T09:20:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bde6860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff914005e30 tx=0x7ff914000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4832da67e477aab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff9197fa640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2cd0 0x7ff9080a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff920c13640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2cd0 0x7ff9080a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.694+0000 7ff9197fa640 1 -- 172.21.15.64:0/4156888517 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff90c0cf040 con 0x7ff91c10b280 2024-05-09T09:20:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.695+0000 7ff920c13640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2cd0 0x7ff9080a5190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55b64bde45d0 tx=0x7ff904005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.696+0000 7ff9127fc640 1 -- 172.21.15.64:0/4121735242 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914034070 con 0x55b64bde3d70 2024-05-09T09:20:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.696+0000 7ff9127fc640 1 -- 172.21.15.64:0/4121735242 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff914003970 con 0x55b64bde3d70 2024-05-09T09:20:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.696+0000 7ff9127fc640 1 -- 172.21.15.64:0/4121735242 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914030a90 con 0x55b64bde3d70 2024-05-09T09:20:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.697+0000 7ff925388740 1 -- 172.21.15.64:0/4121735242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 msgr2=0x55b64bde6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.697+0000 7ff925388740 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bde6860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff914005e30 tx=0x7ff914000f40 comp rx=0 tx=0).stop 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 -- 172.21.15.64:0/4121735242 shutdown_connections 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde6ed0 0x55b64bdf3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bde6860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 --2- 172.21.15.64:0/4121735242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bde37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 -- 172.21.15.64:0/4121735242 >> 172.21.15.64:0/4121735242 conn(0x55b64bddea70 msgr2=0x55b64bddee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 -- 172.21.15.64:0/4121735242 shutdown_connections 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 -- 172.21.15.64:0/4121735242 wait complete. 2024-05-09T09:20:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.699+0000 7ff925388740 1 -- start start 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 Processor -- start 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3380 0x55b64bdf5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bdf63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde6ed0 0x55b64bdfd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b64bdf46e0 con 0x55b64bde6ed0 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b64bd02790 con 0x55b64bde3d70 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b64bdf4860 con 0x55b64bde3380 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff925388740 10 client.0 _ll_register_callbacks cb 0x55b64bd3da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3380 0x55b64bdf5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3380 0x55b64bdf5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34740/0 (socket says 172.21.15.64:34740) 2024-05-09T09:20:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff91bfff640 1 -- 172.21.15.64:0/861823 learned_addr learned my addr 172.21.15.64:0/861823 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.700+0000 7ff920c13640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde6ed0 0x55b64bdfd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.701+0000 7ff91b7fe640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bdf63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.701+0000 7ff920c13640 1 -- 172.21.15.64:0/861823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3380 msgr2=0x55b64bdf5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.701+0000 7ff920c13640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3380 0x55b64bdf5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.701+0000 7ff920c13640 1 -- 172.21.15.64:0/861823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 msgr2=0x55b64bdf63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.701+0000 7ff920c13640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bdf63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.701+0000 7ff920c13640 1 -- 172.21.15.64:0/861823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64ba5b210 con 0x55b64bde6ed0 2024-05-09T09:20:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.702+0000 7ff920c13640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde6ed0 0x55b64bdfd420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff90400f740 tx=0x7ff904004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fc044e110de338 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.702+0000 7ff8fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.702+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/861823 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904011850 con 0x55b64bde6ed0 2024-05-09T09:20:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.702+0000 7ff925388740 1 -- 172.21.15.64:0/861823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff914003970 con 0x55b64bde6ed0 2024-05-09T09:20:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.702+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/861823 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9040119f0 con 0x55b64bde6ed0 2024-05-09T09:20:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.703+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/861823 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff904011bf0 con 0x55b64bde6ed0 2024-05-09T09:20:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.704+0000 7ff925388740 10 client.18186 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.704+0000 7ff925388740 1 -- 172.21.15.64:0/861823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b64bb27620 con 0x55b64bde6ed0 2024-05-09T09:20:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.704+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/861823 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff904011d90 con 0x55b64bde6ed0 2024-05-09T09:20:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/861823 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff904027070 con 0x55b64bde6ed0 2024-05-09T09:20:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff925388740 10 client.18186 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff925388740 10 client.18186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff925388740 20 client.18186 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff925388740 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64be01e80 0x55b64be04320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff925388740 4 client.18186 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.705+0000 7ff925388740 1 -- 172.21.15.64:0/861823 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff914003970 con 0x55b64be01e80 2024-05-09T09:20:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.707+0000 7ff91bfff640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64be01e80 0x55b64be04320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.708+0000 7ff91bfff640 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64be01e80 0x55b64be04320 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f7ca844e44c147 server_cookie=f0fa48d53ba1d5e3 in_seq=0 out_seq=0 2024-05-09T09:20:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.710+0000 7ff8fb7fe640 10 client.18186 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.710+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/861823 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff914003970 con 0x55b64be01e80 2024-05-09T09:20:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.711+0000 7ff8fb7fe640 10 client.18186 handle_command_reply: tid=0 2024-05-09T09:20:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.711+0000 7ff925388740 1 client.18186 shutdown 2024-05-09T09:20:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.713+0000 7ff925388740 1 -- 172.21.15.64:0/861823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde6ed0 msgr2=0x55b64bdfd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.713+0000 7ff925388740 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde6ed0 0x55b64bdfd420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff90400f740 tx=0x7ff904004730 comp rx=0 tx=0).stop 2024-05-09T09:20:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.713+0000 7ff925388740 1 -- 172.21.15.64:0/861823 shutdown_connections 2024-05-09T09:20:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.713+0000 7ff925388740 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64be01e80 0x55b64be04320 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.713+0000 7ff925388740 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde6ed0 0x55b64bdfd420 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.714+0000 7ff925388740 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3d70 0x55b64bdf63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.714+0000 7ff925388740 1 --2- 172.21.15.64:0/861823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3380 0x55b64bdf5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.714+0000 7ff925388740 1 -- 172.21.15.64:0/861823 >> 172.21.15.64:0/861823 conn(0x55b64bddea70 msgr2=0x55b64bdddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.714+0000 7ff8fb7fe640 0 client.18186 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.715+0000 7ff925388740 1 -- 172.21.15.64:0/861823 shutdown_connections 2024-05-09T09:20:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.715+0000 7ff925388740 1 -- 172.21.15.64:0/861823 wait complete. 2024-05-09T09:20:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.716+0000 7ff925388740 20 client.18186 trim_cache size 0 max 16384 2024-05-09T09:20:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.746+0000 7ff925388740 1 Processor -- start 2024-05-09T09:20:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.746+0000 7ff925388740 1 -- start start 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff925388740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bddea70 0x55b64be09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff925388740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3380 0x55b64be0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff925388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff925388740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0806e0 con 0x55b64bddea70 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff925388740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c07dfb0 con 0x55b64bde3380 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff920c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff920c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34746/0 (socket says 172.21.15.64:34746) 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff920c13640 1 -- 172.21.15.64:0/4280608785 learned_addr learned my addr 172.21.15.64:0/4280608785 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.747+0000 7ff925388740 1 -- 172.21.15.64:0/4280608785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c07be00 con 0x55b64bde3d70 2024-05-09T09:20:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.748+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bddea70 0x55b64be09c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.748+0000 7ff91bfff640 1 -- 172.21.15.64:0/4280608785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 msgr2=0x55b64bdfd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.748+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.748+0000 7ff91bfff640 1 -- 172.21.15.64:0/4280608785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3380 msgr2=0x55b64be0a170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.749+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3380 0x55b64be0a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.749+0000 7ff91bfff640 1 -- 172.21.15.64:0/4280608785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b64bde29e0 con 0x55b64bddea70 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.749+0000 7ff920c13640 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.749+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bddea70 0x55b64be09c30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x55b64be04bd0 tx=0x7ff91403c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e27b857cad1147e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.750+0000 7ff911ffb640 1 -- 172.21.15.64:0/4280608785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914034040 con 0x55b64bddea70 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.750+0000 7ff911ffb640 1 -- 172.21.15.64:0/4280608785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff91402f780 con 0x55b64bddea70 2024-05-09T09:20:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.750+0000 7ff911ffb640 1 -- 172.21.15.64:0/4280608785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140044a0 con 0x55b64bddea70 2024-05-09T09:20:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.751+0000 7ff925388740 1 -- 172.21.15.64:0/4280608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bddea70 msgr2=0x55b64be09c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.751+0000 7ff925388740 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bddea70 0x55b64be09c30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x55b64be04bd0 tx=0x7ff91403c040 comp rx=0 tx=0).stop 2024-05-09T09:20:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.752+0000 7ff925388740 1 -- 172.21.15.64:0/4280608785 shutdown_connections 2024-05-09T09:20:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.752+0000 7ff925388740 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.752+0000 7ff925388740 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bde3380 0x55b64be0a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.752+0000 7ff925388740 1 --2- 172.21.15.64:0/4280608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bddea70 0x55b64be09c30 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.752+0000 7ff925388740 1 -- 172.21.15.64:0/4280608785 >> 172.21.15.64:0/4280608785 conn(0x55b64bdf3e60 msgr2=0x55b64bddee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.752+0000 7ff925388740 1 -- 172.21.15.64:0/4280608785 shutdown_connections 2024-05-09T09:20:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.753+0000 7ff925388740 1 -- 172.21.15.64:0/4280608785 wait complete. 2024-05-09T09:20:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.753+0000 7ff925388740 1 -- start start 2024-05-09T09:20:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.754+0000 7ff925388740 1 Processor -- start 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.754+0000 7ff925388740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bddea70 0x55b64bdfd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.754+0000 7ff925388740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bdf9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.754+0000 7ff925388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bddea70 0x55b64bdfd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bddea70 0x55b64bdfd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52780/0 (socket says 172.21.15.64:52780) 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff91b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bdf9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff91b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bdf9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56030/0 (socket says 172.21.15.64:56030) 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff91bfff640 1 -- 172.21.15.64:0/4013982324 learned_addr learned my addr 172.21.15.64:0/4013982324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c07c470 con 0x55b64bde3380 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c07d8e0 con 0x55b64bddea70 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c07cb40 con 0x55b64bde3d70 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff920c13640 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff925388740 10 client.0 _ll_register_callbacks cb 0x7ff904012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff920c13640 1 -- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bddea70 msgr2=0x55b64bdfd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff920c13640 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bddea70 0x55b64bdfd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff920c13640 1 -- 172.21.15.64:0/4013982324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 msgr2=0x55b64bdf9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff920c13640 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bdf9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.755+0000 7ff920c13640 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64bdfa7e0 con 0x55b64bde3d70 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.756+0000 7ff920c13640 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfa1d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff904004dc0 tx=0x7ff904011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71aeddafd7fee1bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.756+0000 7ff8fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.756+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/4013982324 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90403b060 con 0x55b64bde3d70 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.756+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff91402f780 con 0x55b64bde3d70 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.756+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/4013982324 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff904004040 con 0x55b64bde3d70 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.756+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/4013982324 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff904042050 con 0x55b64bde3d70 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.758+0000 7ff925388740 10 client.27823 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.758+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55b64be67420 con 0x55b64bde3d70 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/4013982324 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff904027040 con 0x55b64bde3d70 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff925388740 10 client.27823 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff925388740 10 client.27823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff925388740 20 client.27823 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff925388740 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64bde6ed0 0x55b64be69cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff925388740 4 client.27823 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff91402f780 con 0x55b64bde6ed0 2024-05-09T09:20:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.762+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64bde6ed0 0x55b64be69cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.765+0000 7ff91bfff640 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64bde6ed0 0x55b64be69cd0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b2f10d9516f4c2 server_cookie=b6faabbe2a5ae973 in_seq=0 out_seq=0 2024-05-09T09:20:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.765+0000 7ff8fb7fe640 10 client.27823 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.784+0000 7ff8fb7fe640 1 -- 172.21.15.64:0/4013982324 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff91402f780 con 0x55b64bde6ed0 2024-05-09T09:20:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.784+0000 7ff8fb7fe640 10 client.27823 handle_command_reply: tid=0 2024-05-09T09:20:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.784+0000 7ff925388740 1 client.27823 shutdown 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.784+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 msgr2=0x55b64bdfa1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.784+0000 7ff925388740 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfa1d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff904004dc0 tx=0x7ff904011a80 comp rx=0 tx=0).stop 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 shutdown_connections 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b64bde6ed0 0x55b64be69cd0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b64bde3d70 0x55b64bdfa1d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b64bde3380 0x55b64bdf9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff8fb7fe640 0 client.27823 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 --2- 172.21.15.64:0/4013982324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b64bddea70 0x55b64bdfd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 >> 172.21.15.64:0/4013982324 conn(0x55b64bdf3e60 msgr2=0x55b64bdf0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 shutdown_connections 2024-05-09T09:20:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 1 -- 172.21.15.64:0/4013982324 wait complete. 2024-05-09T09:20:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.785+0000 7ff925388740 20 client.27823 trim_cache size 0 max 16384 2024-05-09T09:20:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:20:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:20:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:20:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2cd0 msgr2=0x7ff9080a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2cd0 0x7ff9080a5190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55b64bde45d0 tx=0x7ff904005f90 comp rx=0 tx=0).stop 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c10b280 msgr2=0x7ff91c19e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c10b280 0x7ff91c19e230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009e60 tx=0x7ff90c00ed90 comp rx=0 tx=0).stop 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 shutdown_connections 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2cd0 0x7ff9080a5190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c10dd60 0x7ff91c1a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c10b280 0x7ff91c19e230 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 --2- 172.21.15.64:0/4156888517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff91c1088c0 0x7ff91c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.786+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 >> 172.21.15.64:0/4156888517 conn(0x7ff91c06e890 msgr2=0x7ff91c10bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.787+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 shutdown_connections 2024-05-09T09:20:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:17.787+0000 7ff921414640 1 -- 172.21.15.64:0/4156888517 wait complete. 2024-05-09T09:20:18.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:20:18.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:20:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 -- 172.21.15.64:0/2083101401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4102320 msgr2=0x7fceb4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 --2- 172.21.15.64:0/2083101401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4102320 0x7fceb4108bb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fceac009ca0 tx=0x7fceac02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 -- 172.21.15.64:0/2083101401 shutdown_connections 2024-05-09T09:20:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 --2- 172.21.15.64:0/2083101401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4102320 0x7fceb4108bb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 --2- 172.21.15.64:0/2083101401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4101960 0x7fceb4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 --2- 172.21.15.64:0/2083101401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb4100770 0x7fceb4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.634+0000 7fcebb06b640 1 -- 172.21.15.64:0/2083101401 >> 172.21.15.64:0/2083101401 conn(0x7fceb40fbce0 msgr2=0x7fceb40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.635+0000 7fcebb06b640 1 -- 172.21.15.64:0/2083101401 shutdown_connections 2024-05-09T09:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.635+0000 7fcebb06b640 1 -- 172.21.15.64:0/2083101401 wait complete. 2024-05-09T09:20:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.636+0000 7fcebb06b640 1 Processor -- start 2024-05-09T09:20:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.636+0000 7fcebb06b640 1 -- start start 2024-05-09T09:20:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.637+0000 7fcebb06b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb4100770 0x7fceb4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.637+0000 7fcebb06b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4101960 0x7fceb4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.638+0000 7fcebb06b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 0x7fceb41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.638+0000 7fcebb06b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410f570 con 0x7fceb4102320 2024-05-09T09:20:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.638+0000 7fcebb06b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410f3f0 con 0x7fceb4100770 2024-05-09T09:20:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.638+0000 7fcebb06b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410f6f0 con 0x7fceb4101960 2024-05-09T09:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 0x7fceb41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 0x7fceb41a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56042/0 (socket says 172.21.15.64:56042) 2024-05-09T09:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 -- 172.21.15.64:0/4200202221 learned_addr learned my addr 172.21.15.64:0/4200202221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb3fff640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4101960 0x7fceb4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb8e07640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb4100770 0x7fceb4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 -- 172.21.15.64:0/4200202221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4101960 msgr2=0x7fceb4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4101960 0x7fceb4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 -- 172.21.15.64:0/4200202221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb4100770 msgr2=0x7fceb4199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb4100770 0x7fceb4199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.639+0000 7fceb9608640 1 -- 172.21.15.64:0/4200202221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb41a16c0 con 0x7fceb4102320 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.640+0000 7fceb9608640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 0x7fceb41a1050 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fceac003ba0 tx=0x7fceac003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.640+0000 7fceb1ffb640 1 -- 172.21.15.64:0/4200202221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004040 con 0x7fceb4102320 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.641+0000 7fceb1ffb640 1 -- 172.21.15.64:0/4200202221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac004690 con 0x7fceb4102320 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.641+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb41a1950 con 0x7fceb4102320 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.641+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41a1ea0 con 0x7fceb4102320 2024-05-09T09:20:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.641+0000 7fceb1ffb640 1 -- 172.21.15.64:0/4200202221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac030d40 con 0x7fceb4102320 2024-05-09T09:20:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.644+0000 7fcebdd7a740 1 Processor -- start 2024-05-09T09:20:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.646+0000 7fceb1ffb640 1 -- 172.21.15.64:0/4200202221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fceac0041e0 con 0x7fceb4102320 2024-05-09T09:20:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.646+0000 7fcebdd7a740 1 -- start start 2024-05-09T09:20:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 0x556c76ff3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76ff8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c76f57d00 con 0x556c76ff8a90 2024-05-09T09:20:18.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556c76eff4c0 con 0x556c76ff3920 2024-05-09T09:20:18.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556c76f53cc0 con 0x556c76ff2f30 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.647+0000 7fceb1ffb640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcea40a2d90 0x7fcea40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb1ffb640 1 -- 172.21.15.64:0/4200202221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fceac0f0a60 con 0x7fceb4102320 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 0x556c76ff3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 0x556c76ff3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34786/0 (socket says 172.21.15.64:34786) 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 -- 172.21.15.64:0/3257704380 learned_addr learned my addr 172.21.15.64:0/3257704380 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb9608640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcea40a2d90 0x7fcea40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 -- 172.21.15.64:0/3257704380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 msgr2=0x556c76ff8420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76ff8420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 -- 172.21.15.64:0/3257704380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 msgr2=0x556c76ffaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.648+0000 7fceb3fff640 1 -- 172.21.15.64:0/3257704380 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c76d357d0 con 0x556c76ff2f30 2024-05-09T09:20:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.649+0000 7fceb9608640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcea40a2d90 0x7fcea40a5250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fceac003c40 tx=0x7fceac02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.652+0000 7fceb3fff640 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 0x556c76ff3350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcea0012780 tx=0x7fcea00096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08031c51e89fac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.652+0000 7fce9effd640 1 -- 172.21.15.64:0/3257704380 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0012d00 con 0x556c76ff2f30 2024-05-09T09:20:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.652+0000 7fce9effd640 1 -- 172.21.15.64:0/3257704380 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea00107d0 con 0x556c76ff2f30 2024-05-09T09:20:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.652+0000 7fce9effd640 1 -- 172.21.15.64:0/3257704380 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0003850 con 0x556c76ff2f30 2024-05-09T09:20:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 -- 172.21.15.64:0/3257704380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 msgr2=0x556c76ff3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 0x556c76ff3350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcea0012780 tx=0x7fcea00096e0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 -- 172.21.15.64:0/3257704380 shutdown_connections 2024-05-09T09:20:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76ff8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/3257704380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff2f30 0x556c76ff3350 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcea0012780 tx=0x7fcea00096e0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.654+0000 7fcebdd7a740 1 -- 172.21.15.64:0/3257704380 >> 172.21.15.64:0/3257704380 conn(0x556c76fee5a0 msgr2=0x556c76fee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.655+0000 7fcebdd7a740 1 -- 172.21.15.64:0/3257704380 shutdown_connections 2024-05-09T09:20:18.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.655+0000 7fcebdd7a740 1 -- 172.21.15.64:0/3257704380 wait complete. 2024-05-09T09:20:18.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.656+0000 7fcebdd7a740 1 -- start start 2024-05-09T09:20:18.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.656+0000 7fcebdd7a740 1 Processor -- start 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 0x556c76ffd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c77002bc0 0x556c77004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 0x556c76ffd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c76ffba40 con 0x556c76ff8a90 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 0x556c76ffd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34790/0 (socket says 172.21.15.64:34790) 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fceb3fff640 1 -- 172.21.15.64:0/712945516 learned_addr learned my addr 172.21.15.64:0/712945516 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556c76f12b90 con 0x556c77002bc0 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556c76ffbbc0 con 0x556c76ff3920 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fcebdd7a740 10 client.0 _ll_register_callbacks cb 0x556c76f4eda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:18.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.657+0000 7fceb8e07640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c77002bc0 0x556c77004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 -- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c77002bc0 msgr2=0x556c77004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:18.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c77002bc0 0x556c77004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 -- 172.21.15.64:0/712945516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 msgr2=0x556c76ffdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 -- 172.21.15.64:0/712945516 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c76c6b3f0 con 0x556c76ff3920 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.658+0000 7fceb3fff640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 0x556c76ffd650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcea0012750 tx=0x7fcea0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f5432a46458681 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.659+0000 7fce93fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.659+0000 7fce93fff640 1 -- 172.21.15.64:0/712945516 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0012030 con 0x556c76ff3920 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.659+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea00107d0 con 0x556c76ff3920 2024-05-09T09:20:18.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.659+0000 7fce93fff640 1 -- 172.21.15.64:0/712945516 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcea00121d0 con 0x556c76ff3920 2024-05-09T09:20:18.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.660+0000 7fce93fff640 1 -- 172.21.15.64:0/712945516 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcea0032050 con 0x556c76ff3920 2024-05-09T09:20:18.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.660+0000 7fcebdd7a740 10 client.27835 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:18.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.660+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556c76c2b0f0 con 0x556c76ff3920 2024-05-09T09:20:18.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fce93fff640 1 -- 172.21.15.64:0/712945516 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea00107d0 con 0x556c76ff3920 2024-05-09T09:20:18.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fce93fff640 1 -- 172.21.15.64:0/712945516 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fcea00037a0 con 0x556c76ff3920 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fcebdd7a740 10 client.27835 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fcebdd7a740 10 client.27835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fcebdd7a740 20 client.27835 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c77009530 0x556c7700b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.661+0000 7fcebdd7a740 4 client.27835 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.662+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c76f65430 con 0x556c77009530 2024-05-09T09:20:18.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.662+0000 7fceb8e07640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c77009530 0x556c7700b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.666+0000 7fceb8e07640 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c77009530 0x556c7700b940 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f482c88f9a75c57 server_cookie=882e582594830292 in_seq=0 out_seq=0 2024-05-09T09:20:18.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.667+0000 7fce93fff640 10 client.27835 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:18.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.667+0000 7fce93fff640 1 -- 172.21.15.64:0/712945516 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c76f65430 con 0x556c77009530 2024-05-09T09:20:18.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.667+0000 7fce93fff640 10 client.27835 handle_command_reply: tid=0 2024-05-09T09:20:18.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.668+0000 7fcebdd7a740 1 client.27835 shutdown 2024-05-09T09:20:18.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.668+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 msgr2=0x556c76ffd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.668+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 0x556c76ffd650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcea0012750 tx=0x7fcea0000bc0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.669+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 shutdown_connections 2024-05-09T09:20:18.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.669+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c77009530 0x556c7700b940 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.669+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c77002bc0 0x556c77004fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.669+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ffdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.669+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/712945516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c76ff3920 0x556c76ffd650 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.669+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 >> 172.21.15.64:0/712945516 conn(0x556c76fee5a0 msgr2=0x556c76fed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.672+0000 7fce93fff640 0 client.27835 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:18.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.672+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 shutdown_connections 2024-05-09T09:20:18.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.672+0000 7fcebdd7a740 1 -- 172.21.15.64:0/712945516 wait complete. 2024-05-09T09:20:18.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.672+0000 7fcebdd7a740 20 client.27835 trim_cache size 0 max 16384 2024-05-09T09:20:18.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.716+0000 7fcebdd7a740 1 Processor -- start 2024-05-09T09:20:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.716+0000 7fcebdd7a740 1 -- start start 2024-05-09T09:20:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.716+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76fedf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.716+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ff4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.716+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c76ff54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce980125b0 con 0x556c76ff8a90 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76fedf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76fedf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52832/0 (socket says 172.21.15.64:52832) 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 -- 172.21.15.64:0/2271754772 learned_addr learned my addr 172.21.15.64:0/2271754772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb8e07640 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ff4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce9800fe80 con 0x556c76ff3920 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fce9800dcd0 con 0x556c77002bc0 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c76ff54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 -- 172.21.15.64:0/2271754772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 msgr2=0x556c76ff54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c76ff54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 -- 172.21.15.64:0/2271754772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 msgr2=0x556c76ff4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ff4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.717+0000 7fceb3fff640 1 -- 172.21.15.64:0/2271754772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c76ff2b70 con 0x556c76ff3920 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.718+0000 7fceb8e07640 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ff4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.718+0000 7fceb3fff640 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76fedf60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcea0004000 tx=0x7fcea0001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763d85722913ec21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.718+0000 7fce9e7fc640 1 -- 172.21.15.64:0/2271754772 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0009a10 con 0x556c76ff3920 2024-05-09T09:20:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.718+0000 7fce9e7fc640 1 -- 172.21.15.64:0/2271754772 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea00107a0 con 0x556c76ff3920 2024-05-09T09:20:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.718+0000 7fce9e7fc640 1 -- 172.21.15.64:0/2271754772 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0012030 con 0x556c76ff3920 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.719+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 msgr2=0x556c76fedf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76fedf60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcea0004000 tx=0x7fcea0001e70 comp rx=0 tx=0).stop 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 shutdown_connections 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c76ff54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff8a90 0x556c76ff4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/2271754772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff3920 0x556c76fedf60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 >> 172.21.15.64:0/2271754772 conn(0x556c76fee5a0 msgr2=0x556c7700b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 shutdown_connections 2024-05-09T09:20:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.720+0000 7fcebdd7a740 1 -- 172.21.15.64:0/2271754772 wait complete. 2024-05-09T09:20:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.721+0000 7fcebdd7a740 1 -- start start 2024-05-09T09:20:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.722+0000 7fcebdd7a740 1 Processor -- start 2024-05-09T09:20:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.722+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff3920 0x556c77019a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.722+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 0x556c77019fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fcebdd7a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c7706d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce9800e340 con 0x556c76ff3920 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fcebdd7a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce9800f7b0 con 0x556c76ff8a90 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 0x556c77019fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 0x556c77019fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52840/0 (socket says 172.21.15.64:52840) 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 -- 172.21.15.64:0/907833603 learned_addr learned my addr 172.21.15.64:0/907833603 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fce9800ea10 con 0x556c77002bc0 2024-05-09T09:20:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fcebdd7a740 10 client.0 _ll_register_callbacks cb 0x7fcea0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 -- 172.21.15.64:0/907833603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 msgr2=0x556c7706d590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c7706d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 -- 172.21.15.64:0/907833603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff3920 msgr2=0x556c77019a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff3920 0x556c77019a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.723+0000 7fceb8e07640 1 -- 172.21.15.64:0/907833603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7701d040 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fceb8e07640 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 0x556c77019fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x556c76ff3d20 tx=0x7fcea8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd552f2bf2028b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fce93fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fce93fff640 1 -- 172.21.15.64:0/907833603 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8029060 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fce93fff640 1 -- 172.21.15.64:0/907833603 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcea8004230 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea00107a0 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fce93fff640 1 -- 172.21.15.64:0/907833603 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcea8030050 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fcebdd7a740 10 client.27695 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.724+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556c7706ebb0 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fce93fff640 1 -- 172.21.15.64:0/907833603 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea00107a0 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fce93fff640 1 -- 172.21.15.64:0/907833603 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fcea80177d0 con 0x556c76ff8a90 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fcebdd7a740 10 client.27695 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fcebdd7a740 10 client.27695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fcebdd7a740 20 client.27695 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c7706f5e0 0x556c77071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fcebdd7a740 4 client.27695 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:20:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.725+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c76f5f780 con 0x556c7706f5e0 2024-05-09T09:20:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.726+0000 7fceb3fff640 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c7706f5e0 0x556c77071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.727+0000 7fce93fff640 10 client.27695 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.727+0000 7fceb3fff640 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c7706f5e0 0x556c77071a00 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efcb8bddf96cd047 server_cookie=1eb97306c7783631 in_seq=0 out_seq=0 2024-05-09T09:20:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.727+0000 7fce93fff640 1 -- 172.21.15.64:0/907833603 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c76f5f780 con 0x556c7706f5e0 2024-05-09T09:20:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.727+0000 7fce93fff640 10 client.27695 handle_command_reply: tid=0 2024-05-09T09:20:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.727+0000 7fcebdd7a740 1 client.27695 shutdown 2024-05-09T09:20:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.729+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 msgr2=0x556c77019fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.729+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 0x556c77019fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x556c76ff3d20 tx=0x7fcea8005f90 comp rx=0 tx=0).stop 2024-05-09T09:20:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 shutdown_connections 2024-05-09T09:20:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556c7706f5e0 0x556c77071a00 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c77002bc0 0x556c7706d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556c76ff8a90 0x556c77019fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fcebdd7a740 1 --2- 172.21.15.64:0/907833603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c76ff3920 0x556c77019a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 >> 172.21.15.64:0/907833603 conn(0x556c76fee5a0 msgr2=0x556c76ff14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.730+0000 7fce93fff640 0 client.27695 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.731+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 shutdown_connections 2024-05-09T09:20:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.731+0000 7fcebdd7a740 1 -- 172.21.15.64:0/907833603 wait complete. 2024-05-09T09:20:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.731+0000 7fcebdd7a740 20 client.27695 trim_cache size 0 max 16384 2024-05-09T09:20:18.731 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:20:18.731 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:20:18.731 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:20:18.731 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:20:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.735+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcea40a2d90 msgr2=0x7fcea40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.735+0000 7fcebb06b640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcea40a2d90 0x7fcea40a5250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fceac003c40 tx=0x7fceac02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.735+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 msgr2=0x7fceb41a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.735+0000 7fcebb06b640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 0x7fceb41a1050 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fceac003ba0 tx=0x7fceac003bd0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.736+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 shutdown_connections 2024-05-09T09:20:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.736+0000 7fcebb06b640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcea40a2d90 0x7fcea40a5250 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.736+0000 7fcebb06b640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceb4102320 0x7fceb41a1050 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.736+0000 7fcebb06b640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb4101960 0x7fceb4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.736+0000 7fcebb06b640 1 --2- 172.21.15.64:0/4200202221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb4100770 0x7fceb4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.736+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 >> 172.21.15.64:0/4200202221 conn(0x7fceb40fbce0 msgr2=0x7fceb4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.739+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 shutdown_connections 2024-05-09T09:20:18.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.739+0000 7fcebb06b640 1 -- 172.21.15.64:0/4200202221 wait complete. 2024-05-09T09:20:18.763 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:20:18.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:20:18.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.956+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1012418549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80100770 msgr2=0x7f4d80100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.956+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1012418549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80100770 0x7f4d80100b90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009860 tx=0x7f4d7002a340 comp rx=0 tx=0).stop 2024-05-09T09:20:18.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.957+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1012418549 shutdown_connections 2024-05-09T09:20:18.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.957+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1012418549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 0x7f4d80108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.957+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1012418549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d80101960 0x7f4d80101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.957+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1012418549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80100770 0x7f4d80100b90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.957+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1012418549 >> 172.21.15.64:0/1012418549 conn(0x7f4d800fbce0 msgr2=0x7f4d800fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:18.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.958+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1012418549 shutdown_connections 2024-05-09T09:20:18.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.958+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1012418549 wait complete. 2024-05-09T09:20:18.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 Processor -- start 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 -- start start 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d80100770 0x7f4d8006da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80101960 0x7f4d8006df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 0x7f4d80072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010a970 con 0x7f4d80101960 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010a7f0 con 0x7f4d80100770 2024-05-09T09:20:18.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.959+0000 7f4d87b3b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010aaf0 con 0x7f4d80102320 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.960+0000 7f4d850d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80101960 0x7f4d8006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.960+0000 7f4d850d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80101960 0x7f4d8006df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56094/0 (socket says 172.21.15.64:56094) 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.960+0000 7f4d850d6640 1 -- 172.21.15.64:0/1340321393 learned_addr learned my addr 172.21.15.64:0/1340321393 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.960+0000 7f4d860d8640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 0x7f4d80072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d860d8640 1 -- 172.21.15.64:0/1340321393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d80100770 msgr2=0x7f4d8006da20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d860d8640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d80100770 0x7f4d8006da20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d860d8640 1 -- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80101960 msgr2=0x7f4d8006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:18.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d860d8640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80101960 0x7f4d8006df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:18.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d860d8640 1 -- 172.21.15.64:0/1340321393 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d80073630 con 0x7f4d80102320 2024-05-09T09:20:18.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d860d8640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 0x7f4d80072fc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800ed70 tx=0x7f4d7800c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800ee60 con 0x7f4d80102320 2024-05-09T09:20:18.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.961+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d78003b70 con 0x7f4d80102320 2024-05-09T09:20:18.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.962+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78017770 con 0x7f4d80102320 2024-05-09T09:20:18.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.962+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801ae990 con 0x7f4d80102320 2024-05-09T09:20:18.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.962+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801aeda0 con 0x7f4d80102320 2024-05-09T09:20:18.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.964+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d80068650 con 0x7f4d80102320 2024-05-09T09:20:18.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.968+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d78017910 con 0x7f4d80102320 2024-05-09T09:20:18.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.969+0000 7f4d76ffd640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d6c0a2ce0 0x7f4d6c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:18.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.969+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4d780cf5a0 con 0x7f4d80102320 2024-05-09T09:20:18.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.969+0000 7f4d858d7640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d6c0a2ce0 0x7f4d6c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:18.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.970+0000 7f4d858d7640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d6c0a2ce0 0x7f4d6c0a51a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4d700095f0 tx=0x7f4d700056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:18.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:18.973+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d7809c110 con 0x7f4d80102320 2024-05-09T09:20:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:18 smithi064 ceph-mon[30149]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-05-09T09:20:19.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:18 smithi139 ceph-mon[33599]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-05-09T09:20:19.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.436+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d80062430 con 0x7f4d80102320 2024-05-09T09:20:19.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.437+0000 7f4d76ffd640 1 -- 172.21.15.64:0/1340321393 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11879 (secure 0 0 0) 0x7f4d7809ba10 con 0x7f4d80102320 2024-05-09T09:20:19.437 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:20:19.437 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:17:16.247105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:20:19.437 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 45 2024-05-09T09:20:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d6c0a2ce0 msgr2=0x7f4d6c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d6c0a2ce0 0x7f4d6c0a51a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4d700095f0 tx=0x7f4d700056d0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 msgr2=0x7f4d80072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 0x7f4d80072fc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800ed70 tx=0x7f4d7800c6e0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 shutdown_connections 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d6c0a2ce0 0x7f4d6c0a51a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d80102320 0x7f4d80072fc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.440+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d80101960 0x7f4d8006df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.441+0000 7f4d87b3b640 1 --2- 172.21.15.64:0/1340321393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d80100770 0x7f4d8006da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.441+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 >> 172.21.15.64:0/1340321393 conn(0x7f4d800fbce0 msgr2=0x7f4d8010d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.441+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 shutdown_connections 2024-05-09T09:20:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.441+0000 7f4d87b3b640 1 -- 172.21.15.64:0/1340321393 wait complete. 2024-05-09T09:20:19.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:20:19.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:18 smithi179 ceph-mon[34086]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.637+0000 7fdeb539e640 1 -- 172.21.15.64:0/1973410739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809c370 msgr2=0x7fdea809c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.637+0000 7fdeb539e640 1 --2- 172.21.15.64:0/1973410739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809c370 0x7fdea809c790 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009900 tx=0x7fdea402a410 comp rx=0 tx=0).stop 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 -- 172.21.15.64:0/1973410739 shutdown_connections 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 --2- 172.21.15.64:0/1973410739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 0x7fdea80a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 --2- 172.21.15.64:0/1973410739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea809d560 0x7fdea809d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 --2- 172.21.15.64:0/1973410739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809c370 0x7fdea809c790 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 -- 172.21.15.64:0/1973410739 >> 172.21.15.64:0/1973410739 conn(0x7fdea800b4d0 msgr2=0x7fdea809a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 -- 172.21.15.64:0/1973410739 shutdown_connections 2024-05-09T09:20:19.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.638+0000 7fdeb539e640 1 -- 172.21.15.64:0/1973410739 wait complete. 2024-05-09T09:20:19.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.639+0000 7fdeb539e640 1 Processor -- start 2024-05-09T09:20:19.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.639+0000 7fdeb539e640 1 -- start start 2024-05-09T09:20:19.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.639+0000 7fdeb539e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea809c370 0x7fdea81325c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.639+0000 7fdeb539e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 0x7fdea8132b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.639+0000 7fdeb539e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 0x7fdea8139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.639+0000 7fdeb539e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80a7eb0 con 0x7fdea809d560 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeb539e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80a7d30 con 0x7fdea809df20 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeb539e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80a8030 con 0x7fdea809c370 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeae7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 0x7fdea8132b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeae7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 0x7fdea8132b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56108/0 (socket says 172.21.15.64:56108) 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 0x7fdea8139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeae7fc640 1 -- 172.21.15.64:0/2144085600 learned_addr learned my addr 172.21.15.64:0/2144085600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea809c370 0x7fdea81325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/2144085600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea809c370 msgr2=0x7fdea81325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea809c370 0x7fdea81325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 msgr2=0x7fdea8132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 0x7fdea8132b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/2144085600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea813a1e0 con 0x7fdea809df20 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 0x7fdea8132b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.640+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 0x7fdea8139b70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdeb004f8c0 tx=0x7fdeb0079ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.641+0000 7fde97fff640 1 -- 172.21.15.64:0/2144085600 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0084070 con 0x7fdea809df20 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.641+0000 7fde97fff640 1 -- 172.21.15.64:0/2144085600 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb007f3f0 con 0x7fdea809df20 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.641+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea813a4d0 con 0x7fdea809df20 2024-05-09T09:20:19.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.641+0000 7fde97fff640 1 -- 172.21.15.64:0/2144085600 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00690d0 con 0x7fdea809df20 2024-05-09T09:20:19.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.641+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea813a9a0 con 0x7fdea809df20 2024-05-09T09:20:19.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.646+0000 7fdeb8185740 1 Processor -- start 2024-05-09T09:20:19.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.646+0000 7fde97fff640 1 -- 172.21.15.64:0/2144085600 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdeb0087070 con 0x7fdea809df20 2024-05-09T09:20:19.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 -- start start 2024-05-09T09:20:19.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8100 0x562538bd8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 0x562538bdd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bddc60 0x562538be00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562538b3cc40 con 0x562538bd8100 2024-05-09T09:20:19.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562538ae4580 con 0x562538bddc60 2024-05-09T09:20:19.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.647+0000 7fdeb8185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562538b38c00 con 0x562538bd8af0 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.648+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 0x562538bdd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.648+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 0x562538bdd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34864/0 (socket says 172.21.15.64:34864) 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.648+0000 7fdeaeffd640 1 -- 172.21.15.64:0/396244969 learned_addr learned my addr 172.21.15.64:0/396244969 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.648+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8100 0x562538bd8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.648+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bddc60 0x562538be00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.649+0000 7fdeaeffd640 1 -- 172.21.15.64:0/396244969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bddc60 msgr2=0x562538be00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.649+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bddc60 0x562538be00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.649+0000 7fdeaeffd640 1 -- 172.21.15.64:0/396244969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8100 msgr2=0x562538bd8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.649+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8100 0x562538bd8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.649+0000 7fdeaeffd640 1 -- 172.21.15.64:0/396244969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625388503f0 con 0x562538bd8af0 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.649+0000 7fde97fff640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde980a2c10 0x7fde980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.650+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 0x562538bdd5f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009900 tx=0x7fdea4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de010eae36d4fe27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.650+0000 7fde97fff640 1 -- 172.21.15.64:0/2144085600 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdeb013a480 con 0x7fdea809df20 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.651+0000 7fde957fa640 1 -- 172.21.15.64:0/396244969 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4035030 con 0x562538bd8af0 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.651+0000 7fde957fa640 1 -- 172.21.15.64:0/396244969 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdea4040070 con 0x562538bd8af0 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.651+0000 7fde957fa640 1 -- 172.21.15.64:0/396244969 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4003a30 con 0x562538bd8af0 2024-05-09T09:20:19.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.651+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde980a2c10 0x7fde980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 -- 172.21.15.64:0/396244969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 msgr2=0x562538bdd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 0x562538bdd5f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009900 tx=0x7fdea4005c00 comp rx=0 tx=0).stop 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 -- 172.21.15.64:0/396244969 shutdown_connections 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde980a2c10 0x7fde980a50d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x562538bd9350 tx=0x7fdea0009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bddc60 0x562538be00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8af0 0x562538bdd5f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 --2- 172.21.15.64:0/396244969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8100 0x562538bd8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 -- 172.21.15.64:0/396244969 >> 172.21.15.64:0/396244969 conn(0x562538bd3770 msgr2=0x562538bd3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 -- 172.21.15.64:0/396244969 shutdown_connections 2024-05-09T09:20:19.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.652+0000 7fdeb8185740 1 -- 172.21.15.64:0/396244969 wait complete. 2024-05-09T09:20:19.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.653+0000 7fdeb8185740 1 -- start start 2024-05-09T09:20:19.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.653+0000 7fdeb8185740 1 Processor -- start 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.653+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 0x562538be2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeb8185740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562538be0bb0 con 0x562538bddc60 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeb8185740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562538af7c40 con 0x562538bd8af0 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeb8185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562538be0d30 con 0x562538bd8100 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56126/0 (socket says 172.21.15.64:56126) 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1388427143 learned_addr learned my addr 172.21.15.64:0/1388427143 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeb8185740 10 client.0 _ll_register_callbacks cb 0x56253884fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 msgr2=0x562538be2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1388427143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 msgr2=0x562538be2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 0x562538be2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.654+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1388427143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253891a7d0 con 0x562538bddc60 2024-05-09T09:20:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be9aa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdeb01066a0 tx=0x7fdeb0106a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5108496f29fa60cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fde92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fde92ff5640 1 -- 172.21.15.64:0/1388427143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb013a0d0 con 0x562538bddc60 2024-05-09T09:20:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea4040070 con 0x562538bddc60 2024-05-09T09:20:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fde92ff5640 1 -- 172.21.15.64:0/1388427143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdeb007f850 con 0x562538bddc60 2024-05-09T09:20:19.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fde92ff5640 1 -- 172.21.15.64:0/1388427143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdeb013a270 con 0x562538bddc60 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fdeb8185740 10 client.18225 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.655+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562538beced0 con 0x562538bddc60 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.656+0000 7fde92ff5640 1 -- 172.21.15.64:0/1388427143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdea4040070 con 0x562538bddc60 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.656+0000 7fde92ff5640 1 -- 172.21.15.64:0/1388427143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdeb007e070 con 0x562538bddc60 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.656+0000 7fdeb8185740 10 client.18225 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.656+0000 7fdeb8185740 10 client.18225 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.656+0000 7fdeb8185740 20 client.18225 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.656+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538bf0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.657+0000 7fdeb8185740 4 client.18225 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.657+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562538b4a370 con 0x562538bee5f0 2024-05-09T09:20:19.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.657+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538bf0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.657+0000 7fde92ff5640 10 client.18225 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.657+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538bf0a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c22975eed7587a17 server_cookie=ca1785b8743d709e in_seq=0 out_seq=0 2024-05-09T09:20:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.658+0000 7fde92ff5640 1 -- 172.21.15.64:0/1388427143 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562538b4a370 con 0x562538bee5f0 2024-05-09T09:20:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.658+0000 7fde92ff5640 10 client.18225 handle_command_reply: tid=0 2024-05-09T09:20:19.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.658+0000 7fdeb8185740 1 client.18225 shutdown 2024-05-09T09:20:19.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 msgr2=0x562538be9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be9aa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdeb01066a0 tx=0x7fdeb0106a60 comp rx=0 tx=0).stop 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 shutdown_connections 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538bf0a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be9aa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 0x562538be2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fde92ff5640 0 client.18225 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1388427143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.659+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 >> 172.21.15.64:0/1388427143 conn(0x562538bd3770 msgr2=0x562538bd2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.660+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 shutdown_connections 2024-05-09T09:20:19.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.660+0000 7fdeb8185740 1 -- 172.21.15.64:0/1388427143 wait complete. 2024-05-09T09:20:19.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.660+0000 7fdeb8185740 20 client.18225 trim_cache size 0 max 16384 2024-05-09T09:20:19.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.684+0000 7fdeb8185740 1 Processor -- start 2024-05-09T09:20:19.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.684+0000 7fdeb8185740 1 -- start start 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.685+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.685+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 0x562538be1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.685+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.685+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeb8185740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8147690 con 0x562538bddc60 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeb8185740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8144f60 con 0x562538bd8af0 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeb8185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8142db0 con 0x562538bd8100 2024-05-09T09:20:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56132/0 (socket says 172.21.15.64:56132) 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1858517181 learned_addr learned my addr 172.21.15.64:0/1858517181 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1858517181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 msgr2=0x562538be1130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1858517181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 msgr2=0x562538be1670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 0x562538be1670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.686+0000 7fdeaf7fe640 1 -- 172.21.15.64:0/1858517181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562538bd32b0 con 0x562538bddc60 2024-05-09T09:20:19.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.687+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be1bb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0067530 tx=0x7fdeb010f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4367c13a93e01626 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.688+0000 7fde94ff9640 1 -- 172.21.15.64:0/1858517181 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb013a270 con 0x562538bddc60 2024-05-09T09:20:19.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.688+0000 7fde94ff9640 1 -- 172.21.15.64:0/1858517181 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb00f10f0 con 0x562538bddc60 2024-05-09T09:20:19.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.688+0000 7fde94ff9640 1 -- 172.21.15.64:0/1858517181 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb01026f0 con 0x562538bddc60 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.689+0000 7fdeb8185740 1 -- 172.21.15.64:0/1858517181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 msgr2=0x562538be1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.689+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be1bb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0067530 tx=0x7fdeb010f6e0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.689+0000 7fdeb8185740 1 -- 172.21.15.64:0/1858517181 shutdown_connections 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.689+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bddc60 0x562538be1bb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.689+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8af0 0x562538be1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.689+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1858517181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bd8100 0x562538be1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.690+0000 7fdeb8185740 1 -- 172.21.15.64:0/1858517181 >> 172.21.15.64:0/1858517181 conn(0x562538bd3770 msgr2=0x562538bd9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.691+0000 7fdeb8185740 1 -- 172.21.15.64:0/1858517181 shutdown_connections 2024-05-09T09:20:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.691+0000 7fdeb8185740 1 -- 172.21.15.64:0/1858517181 wait complete. 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.691+0000 7fdeb8185740 1 -- start start 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 Processor -- start 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8100 0x562538be6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 0x562538be66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bddc60 0x562538be76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8143420 con 0x562538bd8af0 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8144890 con 0x562538bd8100 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeae7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 0x562538be66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeb8185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8143af0 con 0x562538bddc60 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeae7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 0x562538be66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56136/0 (socket says 172.21.15.64:56136) 2024-05-09T09:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.692+0000 7fdeae7fc640 1 -- 172.21.15.64:0/1762821989 learned_addr learned my addr 172.21.15.64:0/1762821989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeb8185740 10 client.0 _ll_register_callbacks cb 0x7fdeb0102c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8100 0x562538be6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bddc60 0x562538be76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeae7fc640 1 -- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bddc60 msgr2=0x562538be76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bddc60 0x562538be76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeae7fc640 1 -- 172.21.15.64:0/1762821989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8100 msgr2=0x562538be6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8100 0x562538be6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeae7fc640 1 -- 172.21.15.64:0/1762821989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562538be7cb0 con 0x562538bd8af0 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeaf7fe640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bddc60 0x562538be76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.693+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8100 0x562538be6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.694+0000 7fdeae7fc640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 0x562538be66b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdea0005b80 tx=0x7fdea00134e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ba24fe74bf5024 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.694+0000 7fde92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.694+0000 7fde92ff5640 1 -- 172.21.15.64:0/1762821989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea002f060 con 0x562538bd8af0 2024-05-09T09:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.694+0000 7fde92ff5640 1 -- 172.21.15.64:0/1762821989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdea001c030 con 0x562538bd8af0 2024-05-09T09:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.694+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562538b446c0 con 0x562538bd8af0 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.695+0000 7fde92ff5640 1 -- 172.21.15.64:0/1762821989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdea0020440 con 0x562538bd8af0 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.695+0000 7fdeb8185740 10 client.18231 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.695+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562538be9030 con 0x562538bd8af0 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fde92ff5640 1 -- 172.21.15.64:0/1762821989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562538b446c0 con 0x562538bd8af0 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fde92ff5640 1 -- 172.21.15.64:0/1762821989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdea001f470 con 0x562538bd8af0 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fdeb8185740 10 client.18231 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fdeb8185740 10 client.18231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fdeb8185740 20 client.18231 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538c571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fdeb8185740 4 client.18231 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.696+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562538ac1820 con 0x562538bee5f0 2024-05-09T09:20:19.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.697+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538c571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeaeffd640 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538c571b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f0df8d480610935 server_cookie=5ef5d2753dc25e78 in_seq=0 out_seq=0 2024-05-09T09:20:19.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fde92ff5640 10 client.18231 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:19.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fde92ff5640 1 -- 172.21.15.64:0/1762821989 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562538ac1820 con 0x562538bee5f0 2024-05-09T09:20:19.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fde92ff5640 10 client.18231 handle_command_reply: tid=0 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 client.18231 shutdown 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 msgr2=0x562538be66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 0x562538be66b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdea0005b80 tx=0x7fdea00134e0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 shutdown_connections 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x562538bee5f0 0x562538c571b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562538bddc60 0x562538be76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562538bd8af0 0x562538be66b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 --2- 172.21.15.64:0/1762821989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562538bd8100 0x562538be6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 >> 172.21.15.64:0/1762821989 conn(0x562538bd3770 msgr2=0x562538be1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.698+0000 7fde92ff5640 0 client.18231 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 shutdown_connections 2024-05-09T09:20:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb8185740 1 -- 172.21.15.64:0/1762821989 wait complete. 2024-05-09T09:20:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb8185740 20 client.18231 trim_cache size 0 max 16384 2024-05-09T09:20:19.700 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde980a2c10 msgr2=0x7fde980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde980a2c10 0x7fde980a50d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x562538bd9350 tx=0x7fdea0009580 comp rx=0 tx=0).stop 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 msgr2=0x7fdea8139b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 0x7fdea8139b70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdeb004f8c0 tx=0x7fdeb0079ef0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 shutdown_connections 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fde980a2c10 0x7fde980a50d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdea809df20 0x7fdea8139b70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea809d560 0x7fdea8132b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 --2- 172.21.15.64:0/2144085600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea809c370 0x7fdea81325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.699+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 >> 172.21.15.64:0/2144085600 conn(0x7fdea800b4d0 msgr2=0x7fdea80a0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.700+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 shutdown_connections 2024-05-09T09:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.700+0000 7fdeb539e640 1 -- 172.21.15.64:0/2144085600 wait complete. 2024-05-09T09:20:19.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:20:19.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 -- 172.21.15.64:0/2984502288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 msgr2=0x7f7a28108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 --2- 172.21.15.64:0/2984502288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a28108190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009900 tx=0x7f7a1802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 -- 172.21.15.64:0/2984502288 shutdown_connections 2024-05-09T09:20:19.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 --2- 172.21.15.64:0/2984502288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 --2- 172.21.15.64:0/2984502288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a281093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 --2- 172.21.15.64:0/2984502288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a28108190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.907+0000 7f7a2d187640 1 -- 172.21.15.64:0/2984502288 >> 172.21.15.64:0/2984502288 conn(0x7f7a28076d00 msgr2=0x7f7a28079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.908+0000 7f7a2d187640 1 -- 172.21.15.64:0/2984502288 shutdown_connections 2024-05-09T09:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.908+0000 7f7a2d187640 1 -- 172.21.15.64:0/2984502288 wait complete. 2024-05-09T09:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.909+0000 7f7a2d187640 1 Processor -- start 2024-05-09T09:20:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.909+0000 7f7a2d187640 1 -- start start 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2d187640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a2819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2d187640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a2819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2d187640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2d187640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281138c0 con 0x7f7a28109920 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2d187640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28113740 con 0x7f7a28108f60 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2d187640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28113a40 con 0x7f7a28107d70 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a2819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a2819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a2819dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34888/0 (socket says 172.21.15.64:34888) 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a2819e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52932/0 (socket says 172.21.15.64:52932) 2024-05-09T09:20:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a27fff640 1 -- 172.21.15.64:0/1222598776 learned_addr learned my addr 172.21.15.64:0/1222598776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56156/0 (socket says 172.21.15.64:56156) 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 -- 172.21.15.64:0/1222598776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 msgr2=0x7f7a2819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a2819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 -- 172.21.15.64:0/1222598776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 msgr2=0x7f7a2819e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a2819e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 -- 172.21.15.64:0/1222598776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a281a5be0 con 0x7f7a28109920 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a277fe640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a2819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.910+0000 7f7a2c986640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281a5570 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009fd0 tx=0x7f7a2000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.911+0000 7f7a257fa640 1 -- 172.21.15.64:0/1222598776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000ce20 con 0x7f7a28109920 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.911+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a281a5ed0 con 0x7f7a28109920 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.911+0000 7f7a257fa640 1 -- 172.21.15.64:0/1222598776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a20010430 con 0x7f7a28109920 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.911+0000 7f7a257fa640 1 -- 172.21.15.64:0/1222598776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000f620 con 0x7f7a28109920 2024-05-09T09:20:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.911+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281a6330 con 0x7f7a28109920 2024-05-09T09:20:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.916+0000 7f7a311cb740 1 Processor -- start 2024-05-09T09:20:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 -- start start 2024-05-09T09:20:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bcf0907c40 con 0x55bcf09a8c90 2024-05-09T09:20:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcf08af580 con 0x55bcf09a3b20 2024-05-09T09:20:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a311cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bcf0903c00 con 0x55bcf09a3130 2024-05-09T09:20:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.917+0000 7f7a257fa640 1 -- 172.21.15.64:0/1222598776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a200105e0 con 0x7f7a28109920 2024-05-09T09:20:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.919+0000 7f7a257fa640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a100a2c60 0x7f7a100a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.919+0000 7f7a257fa640 1 -- 172.21.15.64:0/1222598776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7a200cf760 con 0x7f7a28109920 2024-05-09T09:20:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.920+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.920+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09a8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52936/0 (socket says 172.21.15.64:52936) 2024-05-09T09:20:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.920+0000 7f7a27fff640 1 -- 172.21.15.64:0/3760233188 learned_addr learned my addr 172.21.15.64:0/3760233188 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.920+0000 7f7a277fe640 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.920+0000 7f7a277fe640 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.921+0000 7f7a2c986640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a100a2c60 0x7f7a100a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.921+0000 7f7a27fff640 1 -- 172.21.15.64:0/3760233188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 msgr2=0x55bcf09a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.921+0000 7f7a27fff640 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.921+0000 7f7a27fff640 1 -- 172.21.15.64:0/3760233188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 msgr2=0x55bcf09ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.921+0000 7f7a27fff640 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.921+0000 7f7a27fff640 1 -- 172.21.15.64:0/3760233188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf061b3f0 con 0x55bcf09a3b20 2024-05-09T09:20:19.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.922+0000 7f7a27fff640 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09a8620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009230 tx=0x7f7a18004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940989eaac5740ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.922+0000 7f7a0e7fc640 1 -- 172.21.15.64:0/3760233188 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18035030 con 0x55bcf09a3b20 2024-05-09T09:20:19.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.922+0000 7f7a0e7fc640 1 -- 172.21.15.64:0/3760233188 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a18040070 con 0x55bcf09a3b20 2024-05-09T09:20:19.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.922+0000 7f7a0e7fc640 1 -- 172.21.15.64:0/3760233188 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18031b40 con 0x55bcf09a3b20 2024-05-09T09:20:19.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.922+0000 7f7a2c986640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a100a2c60 0x7f7a100a5120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a20098030 tx=0x7f7a2000c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 -- 172.21.15.64:0/3760233188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 msgr2=0x55bcf09a8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09a8620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009230 tx=0x7f7a18004180 comp rx=0 tx=0).stop 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 -- 172.21.15.64:0/3760233188 shutdown_connections 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09a8620 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 --2- 172.21.15.64:0/3760233188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 -- 172.21.15.64:0/3760233188 >> 172.21.15.64:0/3760233188 conn(0x55bcf099e7a0 msgr2=0x55bcf099eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 -- 172.21.15.64:0/3760233188 shutdown_connections 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 -- 172.21.15.64:0/3760233188 wait complete. 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.924+0000 7f7a311cb740 1 -- start start 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 Processor -- start 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a3b20 0x55bcf09beae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a8c90 0x55bcf09c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bcf09bcbf0 con 0x55bcf09a3b20 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcf08c2c40 con 0x55bcf09a3130 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bcf09bcd70 con 0x55bcf09a8c90 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a311cb740 10 client.0 _ll_register_callbacks cb 0x55bcf061abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52938/0 (socket says 172.21.15.64:52938) 2024-05-09T09:20:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a277fe640 1 -- 172.21.15.64:0/2653298523 learned_addr learned my addr 172.21.15.64:0/2653298523 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.925+0000 7f7a27fff640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a3b20 0x55bcf09beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a8c90 0x55bcf09c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 -- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a8c90 msgr2=0x55bcf09c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a8c90 0x55bcf09c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 -- 172.21.15.64:0/2653298523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a3b20 msgr2=0x55bcf09beae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a3b20 0x55bcf09beae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 -- 172.21.15.64:0/2653298523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf06e57d0 con 0x55bcf09a3130 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a277fe640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c013e10 tx=0x7f7a1c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3a7dacf53a49e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a077fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a077fe640 1 -- 172.21.15.64:0/2653298523 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c010e70 con 0x55bcf09a3130 2024-05-09T09:20:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.926+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a18040070 con 0x55bcf09a3130 2024-05-09T09:20:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.927+0000 7f7a077fe640 1 -- 172.21.15.64:0/2653298523 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7a1c013020 con 0x55bcf09a3130 2024-05-09T09:20:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.927+0000 7f7a077fe640 1 -- 172.21.15.64:0/2653298523 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c02b050 con 0x55bcf09a3130 2024-05-09T09:20:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 10 client.27722 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bcf09c8ee0 con 0x55bcf09a3130 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a077fe640 1 -- 172.21.15.64:0/2653298523 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a18040070 con 0x55bcf09a3130 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a077fe640 1 -- 172.21.15.64:0/2653298523 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7a1c003710 con 0x55bcf09a3130 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 10 client.27722 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 10 client.27722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 20 client.27722 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf09ca550 0x55bcf09cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 4 client.27722 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.928+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcf0915370 con 0x55bcf09ca550 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.929+0000 7f7a27fff640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf09ca550 0x55bcf09cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.930+0000 7f7a27fff640 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf09ca550 0x55bcf09cc9f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eae6718f52bcc36b server_cookie=59d36923a5eb3325 in_seq=0 out_seq=0 2024-05-09T09:20:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.930+0000 7f7a077fe640 10 client.27722 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.932+0000 7f7a077fe640 1 -- 172.21.15.64:0/2653298523 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcf0915370 con 0x55bcf09ca550 2024-05-09T09:20:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.933+0000 7f7a077fe640 10 client.27722 handle_command_reply: tid=0 2024-05-09T09:20:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.934+0000 7f7a311cb740 1 client.27722 shutdown 2024-05-09T09:20:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.935+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 msgr2=0x55bcf09be5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.935+0000 7f7a311cb740 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c013e10 tx=0x7f7a1c004730 comp rx=0 tx=0).stop 2024-05-09T09:20:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 shutdown_connections 2024-05-09T09:20:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a8c90 0x55bcf09c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf09ca550 0x55bcf09cc9f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a3b20 0x55bcf09beae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 --2- 172.21.15.64:0/2653298523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be5a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 >> 172.21.15.64:0/2653298523 conn(0x55bcf099e7a0 msgr2=0x55bcf099d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a077fe640 0 client.27722 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 shutdown_connections 2024-05-09T09:20:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.936+0000 7f7a311cb740 1 -- 172.21.15.64:0/2653298523 wait complete. 2024-05-09T09:20:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.937+0000 7f7a311cb740 20 client.27722 trim_cache size 0 max 16384 2024-05-09T09:20:19.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.965+0000 7f7a311cb740 1 Processor -- start 2024-05-09T09:20:19.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.965+0000 7f7a311cb740 1 -- start start 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.965+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.965+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 0x55bcf09c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.965+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09a1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a311cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281aa8e0 con 0x55bcf09a8c90 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a311cb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a81b0 con 0x55bcf09a3130 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a311cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28073400 con 0x55bcf09a3b20 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 0x55bcf09c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 0x55bcf09c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34914/0 (socket says 172.21.15.64:34914) 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 -- 172.21.15.64:0/1803478545 learned_addr learned my addr 172.21.15.64:0/1803478545 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a277fe640 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 -- 172.21.15.64:0/1803478545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 msgr2=0x55bcf09be8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a277fe640 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 -- 172.21.15.64:0/1803478545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 msgr2=0x55bcf09a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a27fff640 1 -- 172.21.15.64:0/1803478545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf099f510 con 0x55bcf09a3b20 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.966+0000 7f7a277fe640 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.967+0000 7f7a27fff640 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 0x55bcf09c5a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005ab0 tx=0x7f7a18004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbab2e22981ab0ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.967+0000 7f7a0dffb640 1 -- 172.21.15.64:0/1803478545 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18035040 con 0x55bcf09a3b20 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.967+0000 7f7a0dffb640 1 -- 172.21.15.64:0/1803478545 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a18040070 con 0x55bcf09a3b20 2024-05-09T09:20:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.967+0000 7f7a0dffb640 1 -- 172.21.15.64:0/1803478545 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a180544c0 con 0x55bcf09a3b20 2024-05-09T09:20:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 -- 172.21.15.64:0/1803478545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 msgr2=0x55bcf09c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 0x55bcf09c5a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005ab0 tx=0x7f7a18004540 comp rx=0 tx=0).stop 2024-05-09T09:20:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 -- 172.21.15.64:0/1803478545 shutdown_connections 2024-05-09T09:20:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3b20 0x55bcf09c5a70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 --2- 172.21.15.64:0/1803478545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3130 0x55bcf09be8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 -- 172.21.15.64:0/1803478545 >> 172.21.15.64:0/1803478545 conn(0x55bcf099e7a0 msgr2=0x55bcf09be1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 -- 172.21.15.64:0/1803478545 shutdown_connections 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 -- 172.21.15.64:0/1803478545 wait complete. 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.968+0000 7f7a311cb740 1 -- start start 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 Processor -- start 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a66a0 con 0x55bcf09a8c90 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a7ae0 con 0x55bcf09a3b20 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a6d40 con 0x55bcf09a3130 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.969+0000 7f7a311cb740 10 client.0 _ll_register_callbacks cb 0x7f7a1c013c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52962/0 (socket says 172.21.15.64:52962) 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 -- 172.21.15.64:0/523310731 learned_addr learned my addr 172.21.15.64:0/523310731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 -- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 msgr2=0x55bcf09c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 -- 172.21.15.64:0/523310731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 msgr2=0x55bcf09c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 -- 172.21.15.64:0/523310731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf09c4c90 con 0x55bcf09a3b20 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.970+0000 7f7a27fff640 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09c36e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a18004010 tx=0x7f7a18005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481354d3ba00102a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.971+0000 7f7a077fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.971+0000 7f7a077fe640 1 -- 172.21.15.64:0/523310731 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18062060 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.971+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a18040070 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.971+0000 7f7a077fe640 1 -- 172.21.15.64:0/523310731 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7a18054690 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a077fe640 1 -- 172.21.15.64:0/523310731 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7a18069050 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 10 client.27725 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bcf0a30b30 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a077fe640 1 -- 172.21.15.64:0/523310731 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a18040070 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a077fe640 1 -- 172.21.15.64:0/523310731 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7a18031690 con 0x55bcf09a3b20 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 10 client.27725 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:19.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 10 client.27725 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 20 client.27725 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf0a31350 0x55bcf0a33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 4 client.27725 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.972+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcf090f6c0 con 0x55bcf0a31350 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.973+0000 7f7a277fe640 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf0a31350 0x55bcf0a33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.975+0000 7f7a077fe640 10 client.27725 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.975+0000 7f7a277fe640 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf0a31350 0x55bcf0a33770 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7bad1427518f27f server_cookie=1ce31d9e91cf4c3f in_seq=0 out_seq=0 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.975+0000 7f7a077fe640 1 -- 172.21.15.64:0/523310731 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcf090f6c0 con 0x55bcf0a31350 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.975+0000 7f7a077fe640 10 client.27725 handle_command_reply: tid=0 2024-05-09T09:20:19.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.975+0000 7f7a311cb740 1 client.27725 shutdown 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 msgr2=0x55bcf09c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09c36e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a18004010 tx=0x7f7a18005e30 comp rx=0 tx=0).stop 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 shutdown_connections 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bcf0a31350 0x55bcf0a33770 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcf09a8c90 0x55bcf09c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcf09a3b20 0x55bcf09c36e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a077fe640 0 client.27725 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 --2- 172.21.15.64:0/523310731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcf09a3130 0x55bcf09c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 >> 172.21.15.64:0/523310731 conn(0x55bcf099e7a0 msgr2=0x55bcf099d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 shutdown_connections 2024-05-09T09:20:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 1 -- 172.21.15.64:0/523310731 wait complete. 2024-05-09T09:20:19.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.976+0000 7f7a311cb740 20 client.27725 trim_cache size 0 max 16384 2024-05-09T09:20:19.979 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.977+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a100a2c60 msgr2=0x7f7a100a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.977+0000 7f7a2d187640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a100a2c60 0x7f7a100a5120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a20098030 tx=0x7f7a2000c890 comp rx=0 tx=0).stop 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.977+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 msgr2=0x7f7a281a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.977+0000 7f7a2d187640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281a5570 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009fd0 tx=0x7f7a2000ef10 comp rx=0 tx=0).stop 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.977+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 shutdown_connections 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7a100a2c60 0x7f7a100a5120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a28109920 0x7f7a281a5570 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a28108f60 0x7f7a2819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 --2- 172.21.15.64:0/1222598776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a28107d70 0x7f7a2819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 >> 172.21.15.64:0/1222598776 conn(0x7f7a28076d00 msgr2=0x7f7a2810c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 shutdown_connections 2024-05-09T09:20:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:19.978+0000 7f7a2d187640 1 -- 172.21.15.64:0/1222598776 wait complete. 2024-05-09T09:20:19.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:20:20.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.179+0000 7f728696a640 1 -- 172.21.15.64:0/1658724117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7280108830 msgr2=0x7f728010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.179+0000 7f728696a640 1 --2- 172.21.15.64:0/1658724117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7280108830 0x7f728010ac60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7270009ca0 tx=0x7f727002a8b0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 -- 172.21.15.64:0/1658724117 shutdown_connections 2024-05-09T09:20:20.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 --2- 172.21.15.64:0/1658724117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f728010db70 0x7f728010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 --2- 172.21.15.64:0/1658724117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728010b1a0 0x7f728010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 --2- 172.21.15.64:0/1658724117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7280108830 0x7f728010ac60 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 -- 172.21.15.64:0/1658724117 >> 172.21.15.64:0/1658724117 conn(0x7f728006e6c0 msgr2=0x7f7280070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:20.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 -- 172.21.15.64:0/1658724117 shutdown_connections 2024-05-09T09:20:20.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.180+0000 7f728696a640 1 -- 172.21.15.64:0/1658724117 wait complete. 2024-05-09T09:20:20.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.181+0000 7f728696a640 1 Processor -- start 2024-05-09T09:20:20.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.181+0000 7f728696a640 1 -- start start 2024-05-09T09:20:20.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f728696a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7280108830 0x7f72801a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f728696a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f728010b1a0 0x7f72801a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f728696a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 0x7f72801ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f728696a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f728011c2f0 con 0x7f728010db70 2024-05-09T09:20:20.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f728696a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f728011c170 con 0x7f728010b1a0 2024-05-09T09:20:20.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f728696a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f728011c470 con 0x7f7280108830 2024-05-09T09:20:20.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f7284f07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 0x7f72801ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f727ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7280108830 0x7f72801a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f7284f07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 0x7f72801ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56202/0 (socket says 172.21.15.64:56202) 2024-05-09T09:20:20.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f7284f07640 1 -- 172.21.15.64:0/3961523321 learned_addr learned my addr 172.21.15.64:0/3961523321 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:20.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f727f7fe640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f728010b1a0 0x7f72801a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f7284f07640 1 -- 172.21.15.64:0/3961523321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7280108830 msgr2=0x7f72801a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f7284f07640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7280108830 0x7f72801a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.182+0000 7f7284f07640 1 -- 172.21.15.64:0/3961523321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f728010b1a0 msgr2=0x7f72801a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f7284f07640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f728010b1a0 0x7f72801a7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f7284f07640 1 -- 172.21.15.64:0/3961523321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72801ae6a0 con 0x7f728010db70 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f7284f07640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 0x7f72801ae030 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f726800b3f0 tx=0x7f726800b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:20.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f727d7fa640 1 -- 172.21.15.64:0/3961523321 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268015020 con 0x7f728010db70 2024-05-09T09:20:20.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f727d7fa640 1 -- 172.21.15.64:0/3961523321 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7268011ce0 con 0x7f728010db70 2024-05-09T09:20:20.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f727d7fa640 1 -- 172.21.15.64:0/3961523321 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268010d30 con 0x7f728010db70 2024-05-09T09:20:20.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72801ae990 con 0x7f728010db70 2024-05-09T09:20:20.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.183+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801aef10 con 0x7f728010db70 2024-05-09T09:20:20.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.187+0000 7f728967e740 1 Processor -- start 2024-05-09T09:20:20.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.188+0000 7f727d7fa640 1 -- 172.21.15.64:0/3961523321 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7268018020 con 0x7f728010db70 2024-05-09T09:20:20.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.189+0000 7f728967e740 1 -- start start 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.189+0000 7f728967e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.189+0000 7f728967e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.189+0000 7f728967e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.189+0000 7f728967e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557589920c40 con 0x5575899bbf40 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.189+0000 7f728967e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5575898c8580 con 0x5575899bc930 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f728967e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55758991cc00 con 0x5575899c1aa0 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727d7fa640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f726c0a2c10 0x7f726c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899bc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56214/0 (socket says 172.21.15.64:56214) 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 -- 172.21.15.64:0/2709131222 learned_addr learned my addr 172.21.15.64:0/2709131222 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727d7fa640 1 -- 172.21.15.64:0/3961523321 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72680ceae0 con 0x7f728010db70 2024-05-09T09:20:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 -- 172.21.15.64:0/2709131222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 msgr2=0x5575899c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 -- 172.21.15.64:0/2709131222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 msgr2=0x5575899c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.190+0000 7f727f7fe640 1 -- 172.21.15.64:0/2709131222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575896343f0 con 0x5575899bbf40 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.191+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899bc360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7278009f30 tx=0x7f7278009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad2e1688a4ef50e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.191+0000 7f7284f07640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f726c0a2c10 0x7f726c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.191+0000 7f7284f07640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f726c0a2c10 0x7f726c0a50d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f726809af40 tx=0x7f72680105e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.191+0000 7f72767fc640 1 -- 172.21.15.64:0/2709131222 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278017610 con 0x5575899bbf40 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.191+0000 7f72767fc640 1 -- 172.21.15.64:0/2709131222 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727801c070 con 0x5575899bbf40 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.191+0000 7f72767fc640 1 -- 172.21.15.64:0/2709131222 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72780159a0 con 0x5575899bbf40 2024-05-09T09:20:20.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 -- 172.21.15.64:0/2709131222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 msgr2=0x5575899bc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899bc360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7278009f30 tx=0x7f7278009570 comp rx=0 tx=0).stop 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 -- 172.21.15.64:0/2709131222 shutdown_connections 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 --2- 172.21.15.64:0/2709131222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899bc360 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.193+0000 7f728967e740 1 -- 172.21.15.64:0/2709131222 >> 172.21.15.64:0/2709131222 conn(0x5575899b75b0 msgr2=0x5575899b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.194+0000 7f728967e740 1 -- 172.21.15.64:0/2709131222 shutdown_connections 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.194+0000 7f728967e740 1 -- 172.21.15.64:0/2709131222 wait complete. 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.194+0000 7f728967e740 1 -- start start 2024-05-09T09:20:20.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 Processor -- start 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 0x5575899c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899c1aa0 0x5575899cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5575899c4a50 con 0x5575899bbf40 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5575898dbc40 con 0x5575899c1aa0 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5575899c4bd0 con 0x5575899bc930 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f728967e740 10 client.0 _ll_register_callbacks cb 0x557589633be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:20.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.195+0000 7f727ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 0x5575899c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 0x5575899c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34964/0 (socket says 172.21.15.64:34964) 2024-05-09T09:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 -- 172.21.15.64:0/2582614220 learned_addr learned my addr 172.21.15.64:0/2582614220 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:20.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899c1aa0 0x5575899cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 -- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899c1aa0 msgr2=0x5575899cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899c1aa0 0x5575899cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 -- 172.21.15.64:0/2582614220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 msgr2=0x5575899c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.196+0000 7f727ffff640 1 -- 172.21.15.64:0/2582614220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575896fe7d0 con 0x5575899bc930 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.197+0000 7f727ffff640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 0x5575899c6960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7270009ca0 tx=0x7f7270030df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e2610449161673 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.197+0000 7f725f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:20.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.197+0000 7f725f7fe640 1 -- 172.21.15.64:0/2582614220 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270003c40 con 0x5575899bc930 2024-05-09T09:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.197+0000 7f725f7fe640 1 -- 172.21.15.64:0/2582614220 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7270003de0 con 0x5575899bc930 2024-05-09T09:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.197+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727801c070 con 0x5575899bc930 2024-05-09T09:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.198+0000 7f725f7fe640 1 -- 172.21.15.64:0/2582614220 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7270050050 con 0x5575899bc930 2024-05-09T09:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.198+0000 7f728967e740 10 client.27883 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:20.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.198+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5575899d0d80 con 0x5575899bc930 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f725f7fe640 1 -- 172.21.15.64:0/2582614220 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727801c070 con 0x5575899bc930 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f725f7fe640 1 -- 172.21.15.64:0/2582614220 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7270031940 con 0x5575899bc930 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f728967e740 10 client.27883 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f728967e740 10 client.27883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f728967e740 20 client.27883 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f728967e740 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5575899d2440 0x5575899d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f728967e740 4 client.27883 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:20.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.203+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55758992e370 con 0x5575899d2440 2024-05-09T09:20:20.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.204+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5575899d2440 0x5575899d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.206+0000 7f725f7fe640 10 client.27883 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:20.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.206+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5575899d2440 0x5575899d4870 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4795ef476aca1132 server_cookie=460f383e8a2f8c07 in_seq=0 out_seq=0 2024-05-09T09:20:20.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.207+0000 7f725f7fe640 1 -- 172.21.15.64:0/2582614220 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55758992e370 con 0x5575899d2440 2024-05-09T09:20:20.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.207+0000 7f725f7fe640 10 client.27883 handle_command_reply: tid=0 2024-05-09T09:20:20.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.207+0000 7f728967e740 1 client.27883 shutdown 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 msgr2=0x5575899c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 0x5575899c6960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7270009ca0 tx=0x7f7270030df0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 shutdown_connections 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5575899d2440 0x5575899d4870 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899c1aa0 0x5575899cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bc930 0x5575899c6960 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 --2- 172.21.15.64:0/2582614220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bbf40 0x5575899c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.208+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 >> 172.21.15.64:0/2582614220 conn(0x5575899b75b0 msgr2=0x5575899b67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.209+0000 7f725f7fe640 0 client.27883 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:20.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.209+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 shutdown_connections 2024-05-09T09:20:20.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.209+0000 7f728967e740 1 -- 172.21.15.64:0/2582614220 wait complete. 2024-05-09T09:20:20.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.209+0000 7f728967e740 20 client.27883 trim_cache size 0 max 16384 2024-05-09T09:20:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.237+0000 7f728967e740 1 Processor -- start 2024-05-09T09:20:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.237+0000 7f728967e740 1 -- start start 2024-05-09T09:20:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.237+0000 7f728967e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 0x5575899c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.237+0000 7f728967e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.238+0000 7f728967e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899c1aa0 0x5575899b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.238+0000 7f728967e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72801bbcf0 con 0x5575899c1aa0 2024-05-09T09:20:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.238+0000 7f728967e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b95c0 con 0x5575899bc930 2024-05-09T09:20:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.238+0000 7f728967e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b7410 con 0x5575899bbf40 2024-05-09T09:20:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 0x5575899c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 0x5575899c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:34976/0 (socket says 172.21.15.64:34976) 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 -- 172.21.15.64:0/494135814 learned_addr learned my addr 172.21.15.64:0/494135814 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899c1aa0 0x5575899b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 -- 172.21.15.64:0/494135814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 msgr2=0x5575899cd960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899cd960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 -- 172.21.15.64:0/494135814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899c1aa0 msgr2=0x5575899b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899c1aa0 0x5575899b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 -- 172.21.15.64:0/494135814 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575899a8d20 con 0x5575899bbf40 2024-05-09T09:20:20.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.239+0000 7f727f7fe640 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 0x5575899c56f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7278009ed0 tx=0x7f7278009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445ec0583fd1ee2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:20.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.240+0000 7f7275ffb640 1 -- 172.21.15.64:0/494135814 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727802e510 con 0x5575899bbf40 2024-05-09T09:20:20.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.240+0000 7f7275ffb640 1 -- 172.21.15.64:0/494135814 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727801c070 con 0x5575899bbf40 2024-05-09T09:20:20.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.240+0000 7f7275ffb640 1 -- 172.21.15.64:0/494135814 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278006300 con 0x5575899bbf40 2024-05-09T09:20:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.241+0000 7f728967e740 1 -- 172.21.15.64:0/494135814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 msgr2=0x5575899c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.241+0000 7f728967e740 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 0x5575899c56f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7278009ed0 tx=0x7f7278009110 comp rx=0 tx=0).stop 2024-05-09T09:20:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 -- 172.21.15.64:0/494135814 shutdown_connections 2024-05-09T09:20:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899c1aa0 0x5575899b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bc930 0x5575899cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 --2- 172.21.15.64:0/494135814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899bbf40 0x5575899c56f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 -- 172.21.15.64:0/494135814 >> 172.21.15.64:0/494135814 conn(0x5575899b75b0 msgr2=0x5575899bde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 -- 172.21.15.64:0/494135814 shutdown_connections 2024-05-09T09:20:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 -- 172.21.15.64:0/494135814 wait complete. 2024-05-09T09:20:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.242+0000 7f728967e740 1 -- start start 2024-05-09T09:20:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.243+0000 7f728967e740 1 Processor -- start 2024-05-09T09:20:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.243+0000 7f728967e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 0x5575899cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.243+0000 7f728967e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bc930 0x5575899cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.243+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 0x5575899cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 0x5575899cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53002/0 (socket says 172.21.15.64:53002) 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f728967e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f728967e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b7a80 con 0x5575899bc930 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f728967e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b8ef0 con 0x5575899bbf40 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f728967e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b8150 con 0x5575899c1aa0 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 -- 172.21.15.64:0/2459141750 learned_addr learned my addr 172.21.15.64:0/2459141750 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f728967e740 10 client.0 _ll_register_callbacks cb 0x7f7270030a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727ffff640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bc930 0x5575899cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 -- 172.21.15.64:0/2459141750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 msgr2=0x5575899cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 -- 172.21.15.64:0/2459141750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bc930 msgr2=0x5575899cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bc930 0x5575899cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727f7fe640 1 -- 172.21.15.64:0/2459141750 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575899cc8e0 con 0x5575899bbf40 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.244+0000 7f727ffff640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bc930 0x5575899cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.245+0000 7f727f7fe640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 0x5575899cada0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f727802e430 tx=0x7f7278009180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e74daeb39e0b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.245+0000 7f725f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.245+0000 7f725f7fe640 1 -- 172.21.15.64:0/2459141750 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278005bc0 con 0x5575899bbf40 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.245+0000 7f725f7fe640 1 -- 172.21.15.64:0/2459141750 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7278005d60 con 0x5575899bbf40 2024-05-09T09:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.245+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727801c070 con 0x5575899bbf40 2024-05-09T09:20:20.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.246+0000 7f725f7fe640 1 -- 172.21.15.64:0/2459141750 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7278044050 con 0x5575899bbf40 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.246+0000 7f728967e740 10 client.27737 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.246+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5575899cdbc0 con 0x5575899bbf40 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.246+0000 7f725f7fe640 1 -- 172.21.15.64:0/2459141750 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727801c070 con 0x5575899bbf40 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f725f7fe640 1 -- 172.21.15.64:0/2459141750 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7278039020 con 0x5575899bbf40 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f728967e740 10 client.27737 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f728967e740 10 client.27737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f728967e740 20 client.27737 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f728967e740 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557589a38c30 0x557589a3b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f728967e740 4 client.27737 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:20:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575899286c0 con 0x557589a38c30 2024-05-09T09:20:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.247+0000 7f727ffff640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557589a38c30 0x557589a3b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.248+0000 7f727ffff640 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557589a38c30 0x557589a3b050 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fdd8e22b5e0937 server_cookie=ad997ed13a465aac in_seq=0 out_seq=0 2024-05-09T09:20:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.248+0000 7f725f7fe640 10 client.27737 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.249+0000 7f725f7fe640 1 -- 172.21.15.64:0/2459141750 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575899286c0 con 0x557589a38c30 2024-05-09T09:20:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.249+0000 7f725f7fe640 10 client.27737 handle_command_reply: tid=0 2024-05-09T09:20:20.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.249+0000 7f728967e740 1 client.27737 shutdown 2024-05-09T09:20:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.252+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 msgr2=0x5575899cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.252+0000 7f728967e740 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 0x5575899cada0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f727802e430 tx=0x7f7278009180 comp rx=0 tx=0).stop 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.253+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 shutdown_connections 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.253+0000 7f728967e740 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557589a38c30 0x557589a3b050 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f725f7fe640 0 client.27737 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575899c1aa0 0x5575899cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575899bc930 0x5575899cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 1 --2- 172.21.15.64:0/2459141750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575899bbf40 0x5575899cada0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 >> 172.21.15.64:0/2459141750 conn(0x5575899b75b0 msgr2=0x5575899ba330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 shutdown_connections 2024-05-09T09:20:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 1 -- 172.21.15.64:0/2459141750 wait complete. 2024-05-09T09:20:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.254+0000 7f728967e740 20 client.27737 trim_cache size 0 max 16384 2024-05-09T09:20:20.255 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:20:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f726c0a2c10 msgr2=0x7f726c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f726c0a2c10 0x7f726c0a50d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f726809af40 tx=0x7f72680105e0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 msgr2=0x7f72801ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 0x7f72801ae030 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f726800b3f0 tx=0x7f726800b7f0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 shutdown_connections 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f726c0a2c10 0x7f726c0a50d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f728010db70 0x7f72801ae030 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f728010b1a0 0x7f72801a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 --2- 172.21.15.64:0/3961523321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7280108830 0x7f72801a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.255+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 >> 172.21.15.64:0/3961523321 conn(0x7f728006e6c0 msgr2=0x7f728010b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.261+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 shutdown_connections 2024-05-09T09:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:20.261+0000 7f728696a640 1 -- 172.21.15.64:0/3961523321 wait complete. 2024-05-09T09:20:20.281 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:19 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1340321393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:20:20.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:19 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1340321393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:20:20.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:19 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1340321393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:20:21.284 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:20:21.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:20:21.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:20:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:21 smithi139 ceph-mon[33599]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1k op/s 2024-05-09T09:20:21.325 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:21 smithi064 ceph-mon[30149]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1k op/s 2024-05-09T09:20:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 600.0 s, 880.2 tps, lat 116.075 ms stddev 101.238 2024-05-09T09:20:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 -- 172.21.15.64:0/988402437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0101960 msgr2=0x7f1bb0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/988402437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0101960 0x7f1bb0101de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009900 tx=0x7f1ba402a420 comp rx=0 tx=0).stop 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 -- 172.21.15.64:0/988402437 shutdown_connections 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/988402437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0102320 0x7f1bb0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/988402437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0101960 0x7f1bb0101de0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/988402437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb0100770 0x7f1bb0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 -- 172.21.15.64:0/988402437 >> 172.21.15.64:0/988402437 conn(0x7f1bb00fbce0 msgr2=0x7f1bb00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 -- 172.21.15.64:0/988402437 shutdown_connections 2024-05-09T09:20:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.475+0000 7f1bb53ee640 1 -- 172.21.15.64:0/988402437 wait complete. 2024-05-09T09:20:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.476+0000 7f1bb53ee640 1 Processor -- start 2024-05-09T09:20:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 -- start start 2024-05-09T09:20:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0100770 0x7f1bb019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 0x7f1bb019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb0102320 0x7f1bb01a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb010b250 con 0x7f1bb0102320 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb010b0d0 con 0x7f1bb0101960 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bb53ee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb010b3d0 con 0x7f1bb0100770 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bae7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 0x7f1bb019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bae7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 0x7f1bb019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53024/0 (socket says 172.21.15.64:53024) 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1bae7fc640 1 -- 172.21.15.64:0/3437960411 learned_addr learned my addr 172.21.15.64:0/3437960411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.477+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb0102320 0x7f1bb01a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.478+0000 7f1baeffd640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0100770 0x7f1bb019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.478+0000 7f1bae7fc640 1 -- 172.21.15.64:0/3437960411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0100770 msgr2=0x7f1bb019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.478+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0100770 0x7f1bb019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.478+0000 7f1bae7fc640 1 -- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb0102320 msgr2=0x7f1bb01a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.478+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb0102320 0x7f1bb01a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.478+0000 7f1bae7fc640 1 -- 172.21.15.64:0/3437960411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb01a5d90 con 0x7f1bb0101960 2024-05-09T09:20:21.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:21 smithi179 ceph-mon[34086]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1k op/s 2024-05-09T09:20:21.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.479+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 0x7f1bb019e6b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005ab0 tx=0x7f1ba4003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.480+0000 7f1b97fff640 1 -- 172.21.15.64:0/3437960411 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba403f070 con 0x7f1bb0101960 2024-05-09T09:20:21.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.480+0000 7f1b97fff640 1 -- 172.21.15.64:0/3437960411 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba4031c30 con 0x7f1bb0101960 2024-05-09T09:20:21.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.480+0000 7f1b97fff640 1 -- 172.21.15.64:0/3437960411 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4035030 con 0x7f1bb0101960 2024-05-09T09:20:21.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.480+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb01a6020 con 0x7f1bb0101960 2024-05-09T09:20:21.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.480+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01a6590 con 0x7f1bb0101960 2024-05-09T09:20:21.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.484+0000 7f1bb81dc740 1 Processor -- start 2024-05-09T09:20:21.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1b97fff640 1 -- 172.21.15.64:0/3437960411 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ba4042070 con 0x7f1bb0101960 2024-05-09T09:20:21.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 -- start start 2024-05-09T09:20:21.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acb938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acb989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acb9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1acaf7990 con 0x55d1acb93eb0 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d1aca9f5b0 con 0x55d1acb934c0 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.485+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d1acaf3950 con 0x55d1acb99020 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acb938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acb938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53034/0 (socket says 172.21.15.64:53034) 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acb989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 -- 172.21.15.64:0/3631677761 learned_addr learned my addr 172.21.15.64:0/3631677761 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acb9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 -- 172.21.15.64:0/3631677761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 msgr2=0x55d1acb9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acb9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 -- 172.21.15.64:0/3631677761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 msgr2=0x55d1acb989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acb989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.486+0000 7f1baf7fe640 1 -- 172.21.15.64:0/3631677761 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ac8d37e0 con 0x55d1acb934c0 2024-05-09T09:20:21.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.487+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acb938e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800a4d0 tx=0x7f1ba8009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b5440828398012 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.487+0000 7f1b957fa640 1 -- 172.21.15.64:0/3631677761 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8018020 con 0x55d1acb934c0 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.487+0000 7f1b957fa640 1 -- 172.21.15.64:0/3631677761 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba8014cc0 con 0x55d1acb934c0 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.487+0000 7f1b957fa640 1 -- 172.21.15.64:0/3631677761 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8013d40 con 0x55d1acb934c0 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.488+0000 7f1b97fff640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b980a2d90 0x7f1b980a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.488+0000 7f1baeffd640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b980a2d90 0x7f1b980a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.488+0000 7f1bb81dc740 1 -- 172.21.15.64:0/3631677761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 msgr2=0x55d1acb938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.488+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acb938e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800a4d0 tx=0x7f1ba8009650 comp rx=0 tx=0).stop 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.488+0000 7f1b97fff640 1 -- 172.21.15.64:0/3437960411 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ba40efbe0 con 0x7f1bb0101960 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 -- 172.21.15.64:0/3631677761 shutdown_connections 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acb9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acb989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/3631677761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acb938e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 -- 172.21.15.64:0/3631677761 >> 172.21.15.64:0/3631677761 conn(0x55d1acb5b0a0 msgr2=0x55d1acb8ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1baeffd640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b980a2d90 0x7f1b980a5250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55d1acb942b0 tx=0x7f1ba0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 -- 172.21.15.64:0/3631677761 shutdown_connections 2024-05-09T09:20:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.489+0000 7f1bb81dc740 1 -- 172.21.15.64:0/3631677761 wait complete. 2024-05-09T09:20:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.490+0000 7f1bb81dc740 1 -- start start 2024-05-09T09:20:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.490+0000 7f1bb81dc740 1 Processor -- start 2024-05-09T09:20:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.490+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.490+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acb9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acba4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1acb9bfd0 con 0x55d1acb934c0 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bae7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acb9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bae7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acb9dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53048/0 (socket says 172.21.15.64:53048) 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bae7fc640 1 -- 172.21.15.64:0/2572108244 learned_addr learned my addr 172.21.15.64:0/2572108244 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d1acab2b40 con 0x55d1acb93eb0 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d1acb9c150 con 0x55d1acb99020 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acba4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.491+0000 7f1bb81dc740 10 client.0 _ll_register_callbacks cb 0x55d1acaa70f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.492+0000 7f1baf7fe640 1 -- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 msgr2=0x55d1acb9dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.492+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acb9dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.492+0000 7f1baf7fe640 1 -- 172.21.15.64:0/2572108244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 msgr2=0x55d1acb9d740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.492+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb9d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.492+0000 7f1baf7fe640 1 -- 172.21.15.64:0/2572108244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ac933f70 con 0x55d1acb99020 2024-05-09T09:20:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.492+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba8014cc0 con 0x55d1acb99020 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acba4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800afd0 tx=0x7f1ba8013ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562d9d8b40cf4bd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1b92ff5640 10 client.27898 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1b92ff5640 1 -- 172.21.15.64:0/2572108244 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8018030 con 0x55d1acb99020 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1b92ff5640 1 -- 172.21.15.64:0/2572108244 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ba8014820 con 0x55d1acb99020 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1b92ff5640 1 -- 172.21.15.64:0/2572108244 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1ba8023070 con 0x55d1acb99020 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1bb81dc740 10 client.27898 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.493+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55d1ac7cb170 con 0x55d1acb99020 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1b92ff5640 1 -- 172.21.15.64:0/2572108244 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f1ba8031470 con 0x55d1acb99020 2024-05-09T09:20:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1bb81dc740 10 client.27898 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1bb81dc740 10 client.27898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1bb81dc740 20 client.27898 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acba9450 0x55d1acbab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1bb81dc740 4 client.27898 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.494+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1acb050c0 con 0x55d1acba9450 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.496+0000 7f1baeffd640 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acba9450 0x55d1acbab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.497+0000 7f1baeffd640 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acba9450 0x55d1acbab8c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4801a7d9a9896b4 server_cookie=95c2d988ffa661c3 in_seq=0 out_seq=0 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.497+0000 7f1b92ff5640 10 client.27898 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.498+0000 7f1b92ff5640 1 -- 172.21.15.64:0/2572108244 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1acb050c0 con 0x55d1acba9450 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.498+0000 7f1b92ff5640 10 client.27898 handle_command_reply: tid=0 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.498+0000 7f1bb81dc740 1 client.27898 shutdown 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 msgr2=0x55d1acba4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acba4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800afd0 tx=0x7f1ba8013ff0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 shutdown_connections 2024-05-09T09:20:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acba9450 0x55d1acbab8c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acba4cc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acb9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2572108244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 >> 172.21.15.64:0/2572108244 conn(0x55d1acb5b0a0 msgr2=0x55d1acb8e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.499+0000 7f1b92ff5640 0 client.27898 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.500+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 shutdown_connections 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.500+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2572108244 wait complete. 2024-05-09T09:20:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.500+0000 7f1bb81dc740 20 client.27898 trim_cache size 0 max 16384 2024-05-09T09:20:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.532+0000 7f1bb81dc740 1 Processor -- start 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.532+0000 7f1bb81dc740 1 -- start start 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.532+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb92710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.532+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acba4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.532+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acbb01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01aaae0 con 0x55d1acb934c0 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01a83b0 con 0x55d1acb93eb0 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0073440 con 0x55d1acb99020 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1baeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb92710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1baeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb92710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56300/0 (socket says 172.21.15.64:56300) 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1baeffd640 1 -- 172.21.15.64:0/2397585180 learned_addr learned my addr 172.21.15.64:0/2397585180 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.533+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acbb01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.534+0000 7f1baeffd640 1 -- 172.21.15.64:0/2397585180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 msgr2=0x55d1acbb01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.534+0000 7f1baeffd640 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acbb01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.534+0000 7f1baeffd640 1 -- 172.21.15.64:0/2397585180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 msgr2=0x55d1acba4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.534+0000 7f1baeffd640 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acba4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.534+0000 7f1baeffd640 1 -- 172.21.15.64:0/2397585180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1acb8fbf0 con 0x55d1acb934c0 2024-05-09T09:20:21.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.534+0000 7f1baeffd640 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb92710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1bb01017c0 tx=0x7f1ba002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8cc5ccddec8907 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.535+0000 7f1b94ff9640 1 -- 172.21.15.64:0/2397585180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0004020 con 0x55d1acb934c0 2024-05-09T09:20:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.535+0000 7f1b94ff9640 1 -- 172.21.15.64:0/2397585180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba002fe20 con 0x55d1acb934c0 2024-05-09T09:20:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.535+0000 7f1b94ff9640 1 -- 172.21.15.64:0/2397585180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0004020 con 0x55d1acb934c0 2024-05-09T09:20:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.536+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2397585180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 msgr2=0x55d1acb92710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.536+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb92710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1bb01017c0 tx=0x7f1ba002a000 comp rx=0 tx=0).stop 2024-05-09T09:20:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2397585180 shutdown_connections 2024-05-09T09:20:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acbb01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb93eb0 0x55d1acba4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/2397585180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb934c0 0x55d1acb92710 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2397585180 >> 172.21.15.64:0/2397585180 conn(0x55d1acb5b0a0 msgr2=0x55d1acb9ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2397585180 shutdown_connections 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 -- 172.21.15.64:0/2397585180 wait complete. 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 -- start start 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.537+0000 7f1bb81dc740 1 Processor -- start 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.538+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acba1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.538+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acba1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.538+0000 7f1bb81dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acc0da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.538+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01a6890 con 0x55d1acb93eb0 2024-05-09T09:20:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.538+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01a7ce0 con 0x55d1acb934c0 2024-05-09T09:20:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.538+0000 7f1bb81dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01a6f40 con 0x55d1acb99020 2024-05-09T09:20:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.539+0000 7f1bae7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acba1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.539+0000 7f1bae7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acba1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56306/0 (socket says 172.21.15.64:56306) 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.539+0000 7f1bae7fc640 1 -- 172.21.15.64:0/1815616311 learned_addr learned my addr 172.21.15.64:0/1815616311 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.539+0000 7f1baf7fe640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acc0da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.539+0000 7f1bb81dc740 10 client.0 _ll_register_callbacks cb 0x7f1ba00095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1baeffd640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acba1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1bae7fc640 1 -- 172.21.15.64:0/1815616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 msgr2=0x55d1acc0da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acc0da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1bae7fc640 1 -- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 msgr2=0x55d1acba1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acba1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1bae7fc640 1 -- 172.21.15.64:0/1815616311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1acc0df70 con 0x55d1acb93eb0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1bae7fc640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acba1ff0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006010 tx=0x7f1ba40bbee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9bb4dc66917b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.540+0000 7f1b92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.541+0000 7f1b92ff5640 1 -- 172.21.15.64:0/1815616311 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4049040 con 0x55d1acb93eb0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.541+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba002fe20 con 0x55d1acb93eb0 2024-05-09T09:20:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.541+0000 7f1b92ff5640 1 -- 172.21.15.64:0/1815616311 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1ba40ba350 con 0x55d1acb93eb0 2024-05-09T09:20:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.541+0000 7f1b92ff5640 1 -- 172.21.15.64:0/1815616311 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1ba40bc050 con 0x55d1acb93eb0 2024-05-09T09:20:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.542+0000 7f1bb81dc740 10 client.18279 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.542+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d1acc0f1d0 con 0x55d1acb93eb0 2024-05-09T09:20:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.543+0000 7f1b92ff5640 1 -- 172.21.15.64:0/1815616311 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ba40c0400 con 0x55d1acb93eb0 2024-05-09T09:20:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1b92ff5640 1 -- 172.21.15.64:0/1815616311 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f1ba40c4830 con 0x55d1acb93eb0 2024-05-09T09:20:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1bb81dc740 10 client.18279 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1bb81dc740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1bb81dc740 20 client.18279 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acc0fb00 0x55d1acc11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1bb81dc740 4 client.18279 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:20:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1ba002fe20 con 0x55d1acc0fb00 2024-05-09T09:20:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.544+0000 7f1baeffd640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acc0fb00 0x55d1acc11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.546+0000 7f1baeffd640 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acc0fb00 0x55d1acc11f20 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83b529b9c6e7f5 server_cookie=40ed580f38ac6a88 in_seq=0 out_seq=0 2024-05-09T09:20:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.546+0000 7f1b92ff5640 10 client.18279 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.548+0000 7f1b92ff5640 1 -- 172.21.15.64:0/1815616311 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1ba002fe20 con 0x55d1acc0fb00 2024-05-09T09:20:21.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.548+0000 7f1b92ff5640 10 client.18279 handle_command_reply: tid=0 2024-05-09T09:20:21.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.550+0000 7f1bb81dc740 1 client.18279 shutdown 2024-05-09T09:20:21.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.551+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 msgr2=0x55d1acba1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.551+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acba1ff0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006010 tx=0x7f1ba40bbee0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 shutdown_connections 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d1acc0fb00 0x55d1acc11f20 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1acb99020 0x55d1acc0da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1acb93eb0 0x55d1acba1ff0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1bb81dc740 1 --2- 172.21.15.64:0/1815616311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d1acb934c0 0x55d1acba1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 >> 172.21.15.64:0/1815616311 conn(0x55d1acb5b0a0 msgr2=0x55d1acb92590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.554+0000 7f1b92ff5640 0 client.18279 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.555+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 shutdown_connections 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.555+0000 7f1bb81dc740 1 -- 172.21.15.64:0/1815616311 wait complete. 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.555+0000 7f1bb81dc740 20 client.18279 trim_cache size 0 max 16384 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "fb16cd52-a846-4fb1-b608-f1c2665338c3", 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:20:21.555 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:20:21.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.556+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b980a2d90 msgr2=0x7f1b980a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.556+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b980a2d90 0x7f1b980a5250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55d1acb942b0 tx=0x7f1ba0005f90 comp rx=0 tx=0).stop 2024-05-09T09:20:21.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.556+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 msgr2=0x7f1bb019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.556+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 0x7f1bb019e6b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005ab0 tx=0x7f1ba4003c20 comp rx=0 tx=0).stop 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 shutdown_connections 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1b980a2d90 0x7f1b980a5250 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb0102320 0x7f1bb01a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb0101960 0x7f1bb019e6b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 --2- 172.21.15.64:0/3437960411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb0100770 0x7f1bb019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 >> 172.21.15.64:0/3437960411 conn(0x7f1bb00fbce0 msgr2=0x7f1bb010de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 shutdown_connections 2024-05-09T09:20:21.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.558+0000 7f1bb53ee640 1 -- 172.21.15.64:0/3437960411 wait complete. 2024-05-09T09:20:21.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:20:21.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:20:21.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 -- 172.21.15.64:0/696676612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c107d70 msgr2=0x7f9d8c108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 --2- 172.21.15.64:0/696676612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c107d70 0x7f9d8c108190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 -- 172.21.15.64:0/696676612 shutdown_connections 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 --2- 172.21.15.64:0/696676612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c109920 0x7f9d8c1101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 --2- 172.21.15.64:0/696676612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108f60 0x7f9d8c1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 --2- 172.21.15.64:0/696676612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c107d70 0x7f9d8c108190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.814+0000 7f9d9232e640 1 -- 172.21.15.64:0/696676612 >> 172.21.15.64:0/696676612 conn(0x7f9d8c076d00 msgr2=0x7f9d8c079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.815+0000 7f9d9232e640 1 -- 172.21.15.64:0/696676612 shutdown_connections 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.815+0000 7f9d9232e640 1 -- 172.21.15.64:0/696676612 wait complete. 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 Processor -- start 2024-05-09T09:20:21.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 -- start start 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c107d70 0x7f9d8c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108f60 0x7f9d8c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c109920 0x7f9d8c1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1138c0 con 0x7f9d8c107d70 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c113740 con 0x7f9d8c109920 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d9232e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c113a40 con 0x7f9d8c108f60 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.816+0000 7f9d91b2d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c109920 0x7f9d8c1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c107d70 0x7f9d8c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c107d70 0x7f9d8c19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56334/0 (socket says 172.21.15.64:56334) 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d9132c640 1 -- 172.21.15.64:0/162255825 learned_addr learned my addr 172.21.15.64:0/162255825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108f60 0x7f9d8c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d91b2d640 1 -- 172.21.15.64:0/162255825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108f60 msgr2=0x7f9d8c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d91b2d640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108f60 0x7f9d8c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d91b2d640 1 -- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c107d70 msgr2=0x7f9d8c19df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d91b2d640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c107d70 0x7f9d8c19df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d91b2d640 1 -- 172.21.15.64:0/162255825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c1a5b50 con 0x7f9d8c109920 2024-05-09T09:20:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.817+0000 7f9d91b2d640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c109920 0x7f9d8c1a54e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400e940 tx=0x7f9d8400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.818+0000 7f9d827fc640 1 -- 172.21.15.64:0/162255825 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84004d20 con 0x7f9d8c109920 2024-05-09T09:20:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.818+0000 7f9d827fc640 1 -- 172.21.15.64:0/162255825 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d84005020 con 0x7f9d8c109920 2024-05-09T09:20:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.818+0000 7f9d827fc640 1 -- 172.21.15.64:0/162255825 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840116b0 con 0x7f9d8c109920 2024-05-09T09:20:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.818+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c1a5e40 con 0x7f9d8c109920 2024-05-09T09:20:21.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.818+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a62b0 con 0x7f9d8c109920 2024-05-09T09:20:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.823+0000 7f9d96379740 1 Processor -- start 2024-05-09T09:20:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.823+0000 7f9d96379740 1 -- start start 2024-05-09T09:20:21.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.823+0000 7f9d827fc640 1 -- 172.21.15.64:0/162255825 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d8400ca60 con 0x7f9d8c109920 2024-05-09T09:20:21.823 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:20:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:20:21.571+0000 7f3d1eb03700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans/ auth{2=1} v3661 ap=2 f(v10 m2024-05-09T09:19:30.063190+0000 3=3+0) n(v28 rc2024-05-09T09:19:30.145315+0000 b679936 4=3+1) (inest sync->mix g=2) caps={15456=pAsLsXsFs/-@43} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55f5a7ce6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v501>,<0x10000000005/data v111>,<0x10000000003/postgres v28>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v39>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v45>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v3661>,<0x10000000005/data v1551>,<0x10000000003/postgres v731>,<0x10000000002/bd90de25-69ce-4c1d-951e-83ff78942c85 v736>,<0x10000000001/sv_0 v732>,<0x10000000000/qa v739>,<0x1/volumes v692>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T09:18:28.080509+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T09:19:30.145315+0000 b679936 4=3+1)","memory_value.dirstat":"f(v10 m2024-05-09T09:19:30.063190+0000 3=3+0)","memory_value.rstat":"n(v28 rc2024-05-09T09:19:30.145315+0000 b679936 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T09:20:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.823+0000 7f9d96379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669df540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.823+0000 7f9d96379740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.824+0000 7f9d90b2b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669df540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.824+0000 7f9d90b2b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669df540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35060/0 (socket says 172.21.15.64:35060) 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.824+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.824+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53070/0 (socket says 172.21.15.64:53070) 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.824+0000 7f9d827fc640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2d90 0x7f9d780a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d96379740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d827fc640 1 -- 172.21.15.64:0/162255825 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d840cf390 con 0x7f9d8c109920 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d91b2d640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2d90 0x7f9d780a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d96379740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564066943d00 con 0x5640669e4c80 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d96379740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5640668eb4c0 con 0x5640669dfb10 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d96379740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56406693fcc0 con 0x5640669df120 2024-05-09T09:20:21.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.825+0000 7f9d90b2b640 1 -- 172.21.15.64:0/4147236200 learned_addr learned my addr 172.21.15.64:0/4147236200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d91b2d640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2d90 0x7f9d780a5250 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9d84097c60 tx=0x7f9d84005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d9132c640 1 -- 172.21.15.64:0/4147236200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 msgr2=0x5640669df540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d9132c640 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669df540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d9132c640 1 -- 172.21.15.64:0/4147236200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 msgr2=0x5640669e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d9132c640 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d9132c640 1 -- 172.21.15.64:0/4147236200 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640667217d0 con 0x5640669dfb10 2024-05-09T09:20:21.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:21.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.826+0000 7f9d9132c640 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e4610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c006010 tx=0x7f9d7c004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46a7d79c500cb95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.827+0000 7f9d777fe640 1 -- 172.21.15.64:0/4147236200 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035030 con 0x5640669dfb10 2024-05-09T09:20:21.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.827+0000 7f9d777fe640 1 -- 172.21.15.64:0/4147236200 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c040070 con 0x5640669dfb10 2024-05-09T09:20:21.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.827+0000 7f9d777fe640 1 -- 172.21.15.64:0/4147236200 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c031cd0 con 0x5640669dfb10 2024-05-09T09:20:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 -- 172.21.15.64:0/4147236200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 msgr2=0x5640669e4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e4610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c006010 tx=0x7f9d7c004310 comp rx=0 tx=0).stop 2024-05-09T09:20:21.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 -- 172.21.15.64:0/4147236200 shutdown_connections 2024-05-09T09:20:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e4610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 --2- 172.21.15.64:0/4147236200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669df540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.830+0000 7f9d96379740 1 -- 172.21.15.64:0/4147236200 >> 172.21.15.64:0/4147236200 conn(0x5640669da790 msgr2=0x5640669dab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.832+0000 7f9d96379740 1 -- 172.21.15.64:0/4147236200 shutdown_connections 2024-05-09T09:20:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.833+0000 7f9d96379740 1 -- 172.21.15.64:0/4147236200 wait complete. 2024-05-09T09:20:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.833+0000 7f9d96379740 1 -- start start 2024-05-09T09:20:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.834+0000 7f9d96379740 1 Processor -- start 2024-05-09T09:20:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5640669e7bd0 con 0x5640669e4c80 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d90b2b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53082/0 (socket says 172.21.15.64:53082) 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d9132c640 1 -- 172.21.15.64:0/3374643427 learned_addr learned my addr 172.21.15.64:0/3374643427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:35068/0 (socket says 172.21.15.64:35068) 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5640668feb90 con 0x5640669dfb10 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640669e7d50 con 0x5640669df120 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.835+0000 7f9d96379740 10 client.0 _ll_register_callbacks cb 0x56406693ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d9132c640 1 -- 172.21.15.64:0/3374643427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 msgr2=0x5640669e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669e95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d9132c640 1 -- 172.21.15.64:0/3374643427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 msgr2=0x5640669f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d9132c640 1 -- 172.21.15.64:0/3374643427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640666573f0 con 0x5640669dfb10 2024-05-09T09:20:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:21.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e9b00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c02a430 tx=0x7f9d7c02a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940893f9ae35d602 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.836+0000 7f9d74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:21.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.837+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3374643427 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035050 con 0x5640669dfb10 2024-05-09T09:20:21.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.837+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3374643427 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d7c0035f0 con 0x5640669dfb10 2024-05-09T09:20:21.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.837+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c040070 con 0x5640669dfb10 2024-05-09T09:20:21.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.837+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3374643427 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c05c050 con 0x5640669dfb10 2024-05-09T09:20:21.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.837+0000 7f9d96379740 10 client.27758 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:21.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.838+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5640666170f0 con 0x5640669dfb10 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.839+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3374643427 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c040070 con 0x5640669dfb10 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.839+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3374643427 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9d7c045880 con 0x5640669dfb10 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.839+0000 7f9d96379740 10 client.27758 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.839+0000 7f9d96379740 10 client.27758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.839+0000 7f9d96379740 20 client.27758 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.839+0000 7f9d96379740 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5640669f4f80 0x5640669f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.840+0000 7f9d96379740 4 client.27758 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:21.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.840+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564066951430 con 0x5640669f4f80 2024-05-09T09:20:21.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.841+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5640669f4f80 0x5640669f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.841+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5640669f4f80 0x5640669f7370 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12822daf770257de server_cookie=85913f726f764326 in_seq=0 out_seq=0 2024-05-09T09:20:21.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.842+0000 7f9d74ff9640 10 client.27758 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.842+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3374643427 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564066951430 con 0x5640669f4f80 2024-05-09T09:20:21.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.842+0000 7f9d74ff9640 10 client.27758 handle_command_reply: tid=0 2024-05-09T09:20:21.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.843+0000 7f9d96379740 1 client.27758 shutdown 2024-05-09T09:20:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.845+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 msgr2=0x5640669e9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.845+0000 7f9d96379740 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e9b00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c02a430 tx=0x7f9d7c02a910 comp rx=0 tx=0).stop 2024-05-09T09:20:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.846+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 shutdown_connections 2024-05-09T09:20:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.846+0000 7f9d96379740 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5640669f4f80 0x5640669f7370 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.846+0000 7f9d96379740 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669e4c80 0x5640669f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.846+0000 7f9d96379740 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669dfb10 0x5640669e9b00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.846+0000 7f9d96379740 1 --2- 172.21.15.64:0/3374643427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.846+0000 7f9d74ff9640 0 client.27758 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.847+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 >> 172.21.15.64:0/3374643427 conn(0x5640669da790 msgr2=0x5640669d9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.847+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 shutdown_connections 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.847+0000 7f9d96379740 1 -- 172.21.15.64:0/3374643427 wait complete. 2024-05-09T09:20:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.847+0000 7f9d96379740 20 client.27758 trim_cache size 0 max 16384 2024-05-09T09:20:21.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.884+0000 7f9d96379740 1 Processor -- start 2024-05-09T09:20:21.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d96379740 1 -- start start 2024-05-09T09:20:21.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d96379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d96379740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d96379740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x5640669dd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56346/0 (socket says 172.21.15.64:56346) 2024-05-09T09:20:21.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.885+0000 7f9d90b2b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d9132c640 1 -- 172.21.15.64:0/3570081694 learned_addr learned my addr 172.21.15.64:0/3570081694 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1b3150 con 0x5640669dfb10 2024-05-09T09:20:21.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1b0a20 con 0x5640669e4c80 2024-05-09T09:20:21.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ae870 con 0x5640669df120 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x5640669dd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d9132c640 1 -- 172.21.15.64:0/3570081694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 msgr2=0x5640669dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d9132c640 1 -- 172.21.15.64:0/3570081694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 msgr2=0x5640669dd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x5640669dd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d9132c640 1 -- 172.21.15.64:0/3570081694 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640669cb930 con 0x5640669dfb10 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.886+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x5640669dd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:21.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.887+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669f0b00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c045e40 tx=0x7f9d7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b910dd6ea0329a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.887+0000 7f9d76ffd640 1 -- 172.21.15.64:0/3570081694 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c065070 con 0x5640669dfb10 2024-05-09T09:20:21.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.887+0000 7f9d76ffd640 1 -- 172.21.15.64:0/3570081694 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c040040 con 0x5640669dfb10 2024-05-09T09:20:21.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.888+0000 7f9d76ffd640 1 -- 172.21.15.64:0/3570081694 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035070 con 0x5640669dfb10 2024-05-09T09:20:21.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.889+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 msgr2=0x5640669f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.889+0000 7f9d96379740 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669f0b00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c045e40 tx=0x7f9d7c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.890+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 shutdown_connections 2024-05-09T09:20:21.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.890+0000 7f9d96379740 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x5640669dd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.890+0000 7f9d96379740 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669f0b00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.890+0000 7f9d96379740 1 --2- 172.21.15.64:0/3570081694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.890+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 >> 172.21.15.64:0/3570081694 conn(0x5640669da790 msgr2=0x5640669e95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.891+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 shutdown_connections 2024-05-09T09:20:21.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.891+0000 7f9d96379740 1 -- 172.21.15.64:0/3570081694 wait complete. 2024-05-09T09:20:21.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.891+0000 7f9d96379740 1 -- start start 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 Processor -- start 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x564066a592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1aeee0 con 0x5640669dfb10 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1b0350 con 0x5640669e4c80 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d96379740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1af5b0 con 0x5640669df120 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d9132c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669edda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56360/0 (socket says 172.21.15.64:56360) 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.892+0000 7f9d9132c640 1 -- 172.21.15.64:0/3972491615 learned_addr learned my addr 172.21.15.64:0/3972491615 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:21.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d96379740 10 client.0 _ll_register_callbacks cb 0x7f9d8800a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d9132c640 1 -- 172.21.15.64:0/3972491615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 msgr2=0x5640669ed860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669ed860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d9132c640 1 -- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 msgr2=0x564066a592f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x564066a592f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d9132c640 1 -- 172.21.15.64:0/3972491615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640669ee410 con 0x5640669dfb10 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d9132c640 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669edda0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c001eb0 tx=0x7f9d7c03c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef88960a226c7a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.893+0000 7f9d74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.894+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3972491615 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c065030 con 0x5640669dfb10 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.894+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3972491615 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d7c031e00 con 0x5640669dfb10 2024-05-09T09:20:21.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.894+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c040040 con 0x5640669dfb10 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.894+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3972491615 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c072050 con 0x5640669dfb10 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.894+0000 7f9d96379740 10 client.18294 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.894+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564066a6b880 con 0x5640669dfb10 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3972491615 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c040040 con 0x5640669dfb10 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3972491615 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9d7c003bb0 con 0x5640669dfb10 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d96379740 10 client.18294 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d96379740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d96379740 20 client.18294 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d96379740 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564066a6c1f0 0x564066a6e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:21.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d96379740 4 client.18294 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:20:21.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.895+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56406694b780 con 0x564066a6c1f0 2024-05-09T09:20:21.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.897+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564066a6c1f0 0x564066a6e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.898+0000 7f9d90b2b640 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564066a6c1f0 0x564066a6e610 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc519a6231b64d5 server_cookie=9108aca0dcdaa9da in_seq=0 out_seq=0 2024-05-09T09:20:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.899+0000 7f9d74ff9640 10 client.18294 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.899+0000 7f9d74ff9640 1 -- 172.21.15.64:0/3972491615 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56406694b780 con 0x564066a6c1f0 2024-05-09T09:20:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.899+0000 7f9d74ff9640 10 client.18294 handle_command_reply: tid=0 2024-05-09T09:20:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.899+0000 7f9d96379740 1 client.18294 shutdown 2024-05-09T09:20:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.899+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 msgr2=0x5640669edda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.899+0000 7f9d96379740 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669edda0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c001eb0 tx=0x7f9d7c03c940 comp rx=0 tx=0).stop 2024-05-09T09:20:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 shutdown_connections 2024-05-09T09:20:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564066a6c1f0 0x564066a6e610 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5640669e4c80 0x564066a592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640669dfb10 0x5640669edda0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 --2- 172.21.15.64:0/3972491615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640669df120 0x5640669ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 >> 172.21.15.64:0/3972491615 conn(0x5640669da790 msgr2=0x5640669d91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d74ff9640 0 client.18294 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 shutdown_connections 2024-05-09T09:20:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.900+0000 7f9d96379740 1 -- 172.21.15.64:0/3972491615 wait complete. 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.901+0000 7f9d96379740 20 client.18294 trim_cache size 0 max 16384 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: "fb16cd52-a846-4fb1-b608-f1c2665338c3": { 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "fb16cd52-a846-4fb1-b608-f1c2665338c3", 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:20:21.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:20:21.902 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:20:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.901+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2d90 msgr2=0x7f9d780a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2d90 0x7f9d780a5250 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9d84097c60 tx=0x7f9d84005f50 comp rx=0 tx=0).stop 2024-05-09T09:20:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c109920 msgr2=0x7f9d8c1a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c109920 0x7f9d8c1a54e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400e940 tx=0x7f9d8400ed40 comp rx=0 tx=0).stop 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 shutdown_connections 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d780a2d90 0x7f9d780a5250 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d8c109920 0x7f9d8c1a54e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d8c108f60 0x7f9d8c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 --2- 172.21.15.64:0/162255825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d8c107d70 0x7f9d8c19df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 >> 172.21.15.64:0/162255825 conn(0x7f9d8c076d00 msgr2=0x7f9d8c10c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 shutdown_connections 2024-05-09T09:20:21.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:21.902+0000 7f9d9232e640 1 -- 172.21.15.64:0/162255825 wait complete. 2024-05-09T09:20:21.921 INFO:tasks.cephfs.filesystem:scrub status for tag:fb16cd52-a846-4fb1-b608-f1c2665338c3 - {'path': '/', 'tag': 'fb16cd52-a846-4fb1-b608-f1c2665338c3', 'options': 'recursive,force'} 2024-05-09T09:20:22.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:22 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:20:22.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:22 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:20:22.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:22 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:20:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-09T09:20:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:20:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:20:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:20:23.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:20:23.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:23 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-09T09:20:23.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:20:23.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:20:23.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:20:23.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:20:23.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:23 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.4 MiB/s wr, 780 op/s 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_0/bd90de25-69ce-4c1d-951e-83ff78942c85/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:23 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:25 smithi139 ceph-mon[33599]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 11 MiB/s wr, 923 op/s 2024-05-09T09:20:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:25 smithi064 ceph-mon[30149]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 11 MiB/s wr, 923 op/s 2024-05-09T09:20:25.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:25 smithi179 ceph-mon[34086]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 11 MiB/s wr, 923 op/s 2024-05-09T09:20:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 605.0 s, 771.4 tps, lat 127.710 ms stddev 128.791 2024-05-09T09:20:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:27 smithi139 ceph-mon[33599]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.9 MiB/s wr, 918 op/s 2024-05-09T09:20:27.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:27 smithi064 ceph-mon[30149]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.9 MiB/s wr, 918 op/s 2024-05-09T09:20:27.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:27 smithi179 ceph-mon[34086]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.9 MiB/s wr, 918 op/s 2024-05-09T09:20:29.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:29 smithi139 ceph-mon[33599]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 9.3 MiB/s wr, 830 op/s 2024-05-09T09:20:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:29 smithi064 ceph-mon[30149]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 9.3 MiB/s wr, 830 op/s 2024-05-09T09:20:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:29 smithi179 ceph-mon[34086]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 9.3 MiB/s wr, 830 op/s 2024-05-09T09:20:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.953+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1558306464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4105670 msgr2=0x7ff4a410bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.953+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1558306464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4105670 0x7ff4a410bf00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff49c01acb0 tx=0x7ff49c03b900 comp rx=0 tx=0).stop 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.954+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1558306464 shutdown_connections 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.954+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1558306464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4105670 0x7ff4a410bf00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.954+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1558306464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4104cb0 0x7ff4a4105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.954+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1558306464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a4103ac0 0x7ff4a4103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.954+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1558306464 >> 172.21.15.64:0/1558306464 conn(0x7ff4a40ff050 msgr2=0x7ff4a4101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.955+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1558306464 shutdown_connections 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.955+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1558306464 wait complete. 2024-05-09T09:20:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.956+0000 7ff4a9c09640 1 Processor -- start 2024-05-09T09:20:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.956+0000 7ff4a9c09640 1 -- start start 2024-05-09T09:20:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.956+0000 7ff4a9c09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4103ac0 0x7ff4a4107c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a9c09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a4104cb0 0x7ff4a41081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a9c09640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 0x7ff4a410b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a9c09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a406ddb0 con 0x7ff4a4103ac0 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 0x7ff4a410b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 0x7ff4a410b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54010/0 (socket says 172.21.15.64:54010) 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1165024874 learned_addr learned my addr 172.21.15.64:0/1165024874 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a9c09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a406dc30 con 0x7ff4a4104cb0 2024-05-09T09:20:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.957+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a406df30 con 0x7ff4a4105670 2024-05-09T09:20:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.958+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1165024874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a4104cb0 msgr2=0x7ff4a41081c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:20:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.958+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a4104cb0 0x7ff4a41081c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.958+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4103ac0 msgr2=0x7ff4a4107c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.958+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4103ac0 0x7ff4a4107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.958+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1165024874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a410b830 con 0x7ff4a4105670 2024-05-09T09:20:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.958+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 0x7ff4a410b1c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff49c03be20 tx=0x7ff49c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.959+0000 7ff4a0ff9640 1 -- 172.21.15.64:0/1165024874 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c045070 con 0x7ff4a4105670 2024-05-09T09:20:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.959+0000 7ff4a0ff9640 1 -- 172.21.15.64:0/1165024874 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff49c042e80 con 0x7ff4a4105670 2024-05-09T09:20:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.959+0000 7ff4a0ff9640 1 -- 172.21.15.64:0/1165024874 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c004680 con 0x7ff4a4105670 2024-05-09T09:20:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.959+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a410bac0 con 0x7ff4a4105670 2024-05-09T09:20:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.960+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a410bf20 con 0x7ff4a4105670 2024-05-09T09:20:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.967+0000 7ff4a0ff9640 1 -- 172.21.15.64:0/1165024874 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff49c048020 con 0x7ff4a4105670 2024-05-09T09:20:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.967+0000 7ff4ac9f0740 1 Processor -- start 2024-05-09T09:20:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.968+0000 7ff4a0ff9640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff48c0a2e20 0x7ff48c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.968+0000 7ff4a0ff9640 1 -- 172.21.15.64:0/1165024874 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff49c1010b0 con 0x7ff4a4105670 2024-05-09T09:20:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 -- start start 2024-05-09T09:20:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c682399670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399c40 0x55c68239e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c6823a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6822fd340 con 0x55c682399c40 2024-05-09T09:20:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c6822a5290 con 0x55c68239edb0 2024-05-09T09:20:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c6822f9300 con 0x55c682399250 2024-05-09T09:20:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c682399670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c682399670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54022/0 (socket says 172.21.15.64:54022) 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.969+0000 7ff4a37fe640 1 -- 172.21.15.64:0/3282169781 learned_addr learned my addr 172.21.15.64:0/3282169781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff4a37fe640 1 -- 172.21.15.64:0/3282169781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 msgr2=0x55c6823a1240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c6823a1240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff4a37fe640 1 -- 172.21.15.64:0/3282169781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399c40 msgr2=0x55c68239e740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399c40 0x55c68239e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff4a37fe640 1 -- 172.21.15.64:0/3282169781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6822c98e0 con 0x55c682399250 2024-05-09T09:20:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c682399670 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4940097b0 tx=0x7ff494009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14bc5c64e1babb1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.970+0000 7ff489ffb640 1 -- 172.21.15.64:0/3282169781 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494015020 con 0x55c682399250 2024-05-09T09:20:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.971+0000 7ff489ffb640 1 -- 172.21.15.64:0/3282169781 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494011420 con 0x55c682399250 2024-05-09T09:20:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.971+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff48c0a2e20 0x7ff48c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.971+0000 7ff489ffb640 1 -- 172.21.15.64:0/3282169781 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940105c0 con 0x55c682399250 2024-05-09T09:20:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.972+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3282169781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 msgr2=0x55c682399670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.972+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c682399670 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4940097b0 tx=0x7ff494009c70 comp rx=0 tx=0).stop 2024-05-09T09:20:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.972+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff48c0a2e20 0x7ff48c0a52e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff498005d00 tx=0x7ff498005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3282169781 shutdown_connections 2024-05-09T09:20:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c6823a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399c40 0x55c68239e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3282169781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c682399670 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3282169781 >> 172.21.15.64:0/3282169781 conn(0x55c6823948c0 msgr2=0x55c682394cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3282169781 shutdown_connections 2024-05-09T09:20:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.973+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3282169781 wait complete. 2024-05-09T09:20:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.974+0000 7ff4ac9f0740 1 -- start start 2024-05-09T09:20:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.974+0000 7ff4ac9f0740 1 Processor -- start 2024-05-09T09:20:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.975+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.975+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.975+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823bbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.975+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6823b2d70 con 0x55c68239edb0 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c6822b8790 con 0x55c682399c40 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c6823b2ef0 con 0x55c682399250 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4ac9f0740 10 client.0 _ll_register_callbacks cb 0x55c6822f3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823bbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823bbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36254/0 (socket says 172.21.15.64:36254) 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.976+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1822967427 learned_addr learned my addr 172.21.15.64:0/1822967427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1822967427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 msgr2=0x55c6823b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 msgr2=0x55c6823b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a3fff640 1 -- 172.21.15.64:0/1822967427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c682011210 con 0x55c68239edb0 2024-05-09T09:20:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.977+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823bbac0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0cd4b0 tx=0x7ff49c0cd120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dbbd18774aebf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.980+0000 7ff482ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.980+0000 7ff482ffd640 1 -- 172.21.15.64:0/1822967427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0cdc20 con 0x55c68239edb0 2024-05-09T09:20:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.980+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff494011420 con 0x55c68239edb0 2024-05-09T09:20:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.980+0000 7ff482ffd640 1 -- 172.21.15.64:0/1822967427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff49c0fe8d0 con 0x55c68239edb0 2024-05-09T09:20:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.981+0000 7ff482ffd640 1 -- 172.21.15.64:0/1822967427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff49c0cd520 con 0x55c68239edb0 2024-05-09T09:20:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.981+0000 7ff4ac9f0740 10 client.18300 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.981+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c6820dd620 con 0x55c68239edb0 2024-05-09T09:20:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.981+0000 7ff482ffd640 1 -- 172.21.15.64:0/1822967427 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494011420 con 0x55c68239edb0 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff482ffd640 1 -- 172.21.15.64:0/1822967427 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff49c004b40 con 0x55c68239edb0 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff4ac9f0740 10 client.18300 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff4ac9f0740 10 client.18300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff4ac9f0740 20 client.18300 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c6823bfe50 0x55c6823c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff4ac9f0740 4 client.18300 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.982+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c68230aa70 con 0x55c6823bfe50 2024-05-09T09:20:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.983+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c6823bfe50 0x55c6823c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.989+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c6823bfe50 0x55c6823c22b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df80aa17724072a server_cookie=cb86fd26689341e3 in_seq=0 out_seq=0 2024-05-09T09:20:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.989+0000 7ff482ffd640 10 client.18300 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.989+0000 7ff482ffd640 1 -- 172.21.15.64:0/1822967427 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c68230aa70 con 0x55c6823bfe50 2024-05-09T09:20:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.989+0000 7ff482ffd640 10 client.18300 handle_command_reply: tid=0 2024-05-09T09:20:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.990+0000 7ff4ac9f0740 1 client.18300 shutdown 2024-05-09T09:20:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.990+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 msgr2=0x55c6823bbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.990+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823bbac0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0cd4b0 tx=0x7ff49c0cd120 comp rx=0 tx=0).stop 2024-05-09T09:20:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 shutdown_connections 2024-05-09T09:20:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c6823bfe50 0x55c6823c22b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823bbac0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1822967427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 >> 172.21.15.64:0/1822967427 conn(0x55c6823948c0 msgr2=0x55c682393a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff482ffd640 0 client.18300 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 shutdown_connections 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1822967427 wait complete. 2024-05-09T09:20:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:30.991+0000 7ff4ac9f0740 20 client.18300 trim_cache size 0 max 16384 2024-05-09T09:20:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.021+0000 7ff4ac9f0740 1 Processor -- start 2024-05-09T09:20:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.021+0000 7ff4ac9f0740 1 -- start start 2024-05-09T09:20:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.022+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399250 0x55c6823b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.022+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 0x55c682397870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.022+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c682397db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.022+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 0x55c682397870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.022+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399250 0x55c6823b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399250 0x55c6823b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36266/0 (socket says 172.21.15.64:36266) 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 0x55c682397870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54032/0 (socket says 172.21.15.64:54032) 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c682397db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c682397db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58204/0 (socket says 172.21.15.64:58204) 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4a37fe640 1 -- 172.21.15.64:0/3785089717 learned_addr learned my addr 172.21.15.64:0/3785089717 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900124b0 con 0x55c682399250 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff49000fd80 con 0x55c68239edb0 2024-05-09T09:20:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.023+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff49000dbd0 con 0x55c682399c40 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.024+0000 7ff4a2ffd640 1 -- 172.21.15.64:0/3785089717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 msgr2=0x55c682397db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.024+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c682397db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.024+0000 7ff4a2ffd640 1 -- 172.21.15.64:0/3785089717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399250 msgr2=0x55c6823b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.024+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399250 0x55c6823b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.024+0000 7ff4a2ffd640 1 -- 172.21.15.64:0/3785089717 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68239a4d0 con 0x55c682399c40 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.024+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 0x55c682397870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55c6823b5a20 tx=0x7ff498010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc85e785045c624 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.025+0000 7ff4897fa640 1 -- 172.21.15.64:0/3785089717 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498022030 con 0x55c682399c40 2024-05-09T09:20:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.025+0000 7ff4897fa640 1 -- 172.21.15.64:0/3785089717 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff498017040 con 0x55c682399c40 2024-05-09T09:20:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.025+0000 7ff4897fa640 1 -- 172.21.15.64:0/3785089717 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49800f3c0 con 0x55c682399c40 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 msgr2=0x55c682397870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 0x55c682397870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55c6823b5a20 tx=0x7ff498010000 comp rx=0 tx=0).stop 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 shutdown_connections 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c68239edb0 0x55c682397db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399c40 0x55c682397870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/3785089717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c682399250 0x55c6823b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.026+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 >> 172.21.15.64:0/3785089717 conn(0x55c6823948c0 msgr2=0x55c6823b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.027+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 shutdown_connections 2024-05-09T09:20:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.027+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/3785089717 wait complete. 2024-05-09T09:20:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.027+0000 7ff4ac9f0740 1 -- start start 2024-05-09T09:20:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.027+0000 7ff4ac9f0740 1 Processor -- start 2024-05-09T09:20:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4ac9f0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:54040/0 (socket says 172.21.15.64:54040) 2024-05-09T09:20:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58206/0 (socket says 172.21.15.64:58206) 2024-05-09T09:20:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff49000e240 con 0x55c68239edb0 2024-05-09T09:20:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff49000f6b0 con 0x55c682399c40 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4ac9f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff49000e910 con 0x55c682399250 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.028+0000 7ff4a37fe640 1 -- 172.21.15.64:0/1850389772 learned_addr learned my addr 172.21.15.64:0/1850389772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4ac9f0740 10 client.0 _ll_register_callbacks cb 0x7ff49c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a2ffd640 1 -- 172.21.15.64:0/1850389772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 msgr2=0x55c6823b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a2ffd640 1 -- 172.21.15.64:0/1850389772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 msgr2=0x55c6823ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a2ffd640 1 -- 172.21.15.64:0/1850389772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6823ba5f0 con 0x55c682399c40 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a3fff640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff4a2ffd640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b90d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55c6823b5a20 tx=0x7ff49801e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9308b945f756c299 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff482ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.029+0000 7ff482ffd640 1 -- 172.21.15.64:0/1850389772 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498022040 con 0x55c682399c40 2024-05-09T09:20:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.030+0000 7ff482ffd640 1 -- 172.21.15.64:0/1850389772 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff498005050 con 0x55c682399c40 2024-05-09T09:20:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.030+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff498017040 con 0x55c682399c40 2024-05-09T09:20:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.030+0000 7ff482ffd640 1 -- 172.21.15.64:0/1850389772 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff498038050 con 0x55c682399c40 2024-05-09T09:20:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.030+0000 7ff4ac9f0740 10 client.27767 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.030+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c682426780 con 0x55c682399c40 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.030+0000 7ff482ffd640 1 -- 172.21.15.64:0/1850389772 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff498017040 con 0x55c682399c40 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff482ffd640 1 -- 172.21.15.64:0/1850389772 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff498004c20 con 0x55c682399c40 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4ac9f0740 10 client.27767 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4ac9f0740 10 client.27767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4ac9f0740 20 client.27767 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c682426f90 0x55c6824293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4ac9f0740 4 client.27767 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c682304dc0 con 0x55c682426f90 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c682426f90 0x55c6824293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.031+0000 7ff482ffd640 10 client.27767 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.032+0000 7ff4a37fe640 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c682426f90 0x55c6824293b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db144634b3d0f5f server_cookie=9b679f90f29dbddd in_seq=0 out_seq=0 2024-05-09T09:20:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.051+0000 7ff482ffd640 1 -- 172.21.15.64:0/1850389772 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c682304dc0 con 0x55c682426f90 2024-05-09T09:20:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.051+0000 7ff482ffd640 10 client.27767 handle_command_reply: tid=0 2024-05-09T09:20:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.052+0000 7ff4ac9f0740 1 client.27767 shutdown 2024-05-09T09:20:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.052+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 msgr2=0x55c6823b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.052+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b90d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55c6823b5a20 tx=0x7ff49801e640 comp rx=0 tx=0).stop 2024-05-09T09:20:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 shutdown_connections 2024-05-09T09:20:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55c682426f90 0x55c6824293b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c68239edb0 0x55c6823ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c682399c40 0x55c6823b90d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff4ac9f0740 1 --2- 172.21.15.64:0/1850389772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c682399250 0x55c6823b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 >> 172.21.15.64:0/1850389772 conn(0x55c6823948c0 msgr2=0x55c6823b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.054+0000 7ff482ffd640 0 client.27767 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.056+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 shutdown_connections 2024-05-09T09:20:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.056+0000 7ff4ac9f0740 1 -- 172.21.15.64:0/1850389772 wait complete. 2024-05-09T09:20:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.060+0000 7ff4ac9f0740 20 client.27767 trim_cache size 0 max 16384 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.064+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff48c0a2e20 msgr2=0x7ff48c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.064+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff48c0a2e20 0x7ff48c0a52e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff498005d00 tx=0x7ff498005c90 comp rx=0 tx=0).stop 2024-05-09T09:20:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.064+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 msgr2=0x7ff4a410b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.064+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 0x7ff4a410b1c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff49c03be20 tx=0x7ff49c003ba0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.065+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 shutdown_connections 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.065+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff48c0a2e20 0x7ff48c0a52e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.065+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a4105670 0x7ff4a410b1c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.065+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a4104cb0 0x7ff4a41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.065+0000 7ff4a9c09640 1 --2- 172.21.15.64:0/1165024874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a4103ac0 0x7ff4a4107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.065+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 >> 172.21.15.64:0/1165024874 conn(0x7ff4a40ff050 msgr2=0x7ff4a4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.067+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 shutdown_connections 2024-05-09T09:20:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:31.067+0000 7ff4a9c09640 1 -- 172.21.15.64:0/1165024874 wait complete. 2024-05-09T09:20:31.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:31 smithi139 ceph-mon[33599]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-05-09T09:20:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 610.0 s, 805.2 tps, lat 121.632 ms stddev 71.268 2024-05-09T09:20:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:31 smithi064 ceph-mon[30149]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-05-09T09:20:31.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:31 smithi179 ceph-mon[34086]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 1.03k op/s 2024-05-09T09:20:33.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:33 smithi139 ceph-mon[33599]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-05-09T09:20:33.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:33 smithi064 ceph-mon[30149]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-05-09T09:20:33.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:33 smithi179 ceph-mon[34086]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-05-09T09:20:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:35 smithi139 ceph-mon[33599]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 958 op/s 2024-05-09T09:20:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:35 smithi064 ceph-mon[30149]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 958 op/s 2024-05-09T09:20:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:35 smithi179 ceph-mon[34086]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 958 op/s 2024-05-09T09:20:36.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:36 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:20:36.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:36 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:20:36.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:36 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:20:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 615.0 s, 810.2 tps, lat 127.473 ms stddev 181.956 2024-05-09T09:20:36.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:36 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:20:36.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:36 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:20:36.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:36 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:20:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:36 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:20:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:36 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:20:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:36 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:20:37.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:37 smithi139 ceph-mon[33599]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 874 op/s 2024-05-09T09:20:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:37 smithi064 ceph-mon[30149]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 874 op/s 2024-05-09T09:20:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:37 smithi179 ceph-mon[34086]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 874 op/s 2024-05-09T09:20:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:39 smithi139 ceph-mon[33599]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.8 MiB/s wr, 733 op/s 2024-05-09T09:20:39.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:39 smithi064 ceph-mon[30149]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.8 MiB/s wr, 733 op/s 2024-05-09T09:20:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:39 smithi179 ceph-mon[34086]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.8 MiB/s wr, 733 op/s 2024-05-09T09:20:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 620.0 s, 714.4 tps, lat 140.363 ms stddev 181.709 2024-05-09T09:20:41.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:41 smithi064 ceph-mon[30149]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 978 op/s 2024-05-09T09:20:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:41 smithi179 ceph-mon[34086]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 978 op/s 2024-05-09T09:20:41.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:41 smithi139 ceph-mon[33599]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 978 op/s 2024-05-09T09:20:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:43 smithi064 ceph-mon[30149]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 753 op/s 2024-05-09T09:20:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:43 smithi179 ceph-mon[34086]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 753 op/s 2024-05-09T09:20:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:43 smithi139 ceph-mon[33599]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 753 op/s 2024-05-09T09:20:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.226+0000 7fc3a1be2640 1 -- 172.21.15.64:0/671103049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 msgr2=0x7fc39c10dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.226+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/671103049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c10dfd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc38402a410 comp rx=0 tx=0).stop 2024-05-09T09:20:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.226+0000 7fc3a1be2640 1 -- 172.21.15.64:0/671103049 shutdown_connections 2024-05-09T09:20:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.227+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/671103049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c10dfd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.227+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/671103049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc39c10b1a0 0x7fc39c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.227+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/671103049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c108830 0x7fc39c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.227+0000 7fc3a1be2640 1 -- 172.21.15.64:0/671103049 >> 172.21.15.64:0/671103049 conn(0x7fc39c06e8a0 msgr2=0x7fc39c070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.230+0000 7fc3a1be2640 1 -- 172.21.15.64:0/671103049 shutdown_connections 2024-05-09T09:20:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.230+0000 7fc3a1be2640 1 -- 172.21.15.64:0/671103049 wait complete. 2024-05-09T09:20:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.231+0000 7fc3a1be2640 1 Processor -- start 2024-05-09T09:20:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 -- start start 2024-05-09T09:20:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc39c108830 0x7fc39c1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c10b1a0 0x7fc39c1a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c1addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c11c210 con 0x7fc39c10b1a0 2024-05-09T09:20:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c11c090 con 0x7fc39c10db50 2024-05-09T09:20:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc3a1be2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c11c390 con 0x7fc39c108830 2024-05-09T09:20:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc39bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c1addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc39bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c1addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36684/0 (socket says 172.21.15.64:36684) 2024-05-09T09:20:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.232+0000 7fc39bfff640 1 -- 172.21.15.64:0/2216379736 learned_addr learned my addr 172.21.15.64:0/2216379736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.233+0000 7fc39affd640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c10b1a0 0x7fc39c1a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc39c108830 0x7fc39c1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39affd640 1 -- 172.21.15.64:0/2216379736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc39c108830 msgr2=0x7fc39c1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39affd640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc39c108830 0x7fc39c1a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39affd640 1 -- 172.21.15.64:0/2216379736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 msgr2=0x7fc39c1addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39affd640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c1addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39affd640 1 -- 172.21.15.64:0/2216379736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c1ae430 con 0x7fc39c10b1a0 2024-05-09T09:20:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.234+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c1addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.235+0000 7fc39affd640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c10b1a0 0x7fc39c1a6da0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc39400e9f0 tx=0x7fc39400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.235+0000 7fc398ff9640 1 -- 172.21.15.64:0/2216379736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39400ccc0 con 0x7fc39c10b1a0 2024-05-09T09:20:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.236+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c1ae720 con 0x7fc39c10b1a0 2024-05-09T09:20:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.236+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1aecf0 con 0x7fc39c10b1a0 2024-05-09T09:20:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.238+0000 7fc398ff9640 1 -- 172.21.15.64:0/2216379736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc394005020 con 0x7fc39c10b1a0 2024-05-09T09:20:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.238+0000 7fc398ff9640 1 -- 172.21.15.64:0/2216379736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394004a00 con 0x7fc39c10b1a0 2024-05-09T09:20:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.240+0000 7fc3a49c5740 1 Processor -- start 2024-05-09T09:20:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 -- start start 2024-05-09T09:20:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f64a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f64f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f652240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56220f5ae340 con 0x56220f64ac40 2024-05-09T09:20:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56220f556290 con 0x56220f64fdb0 2024-05-09T09:20:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.241+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56220f5aa300 con 0x56220f64a250 2024-05-09T09:20:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.242+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f64f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.242+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f64f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50916/0 (socket says 172.21.15.64:50916) 2024-05-09T09:20:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.242+0000 7fc39b7fe640 1 -- 172.21.15.64:0/2270699097 learned_addr learned my addr 172.21.15.64:0/2270699097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.242+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f64a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.243+0000 7fc39bfff640 1 -- 172.21.15.64:0/2270699097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 msgr2=0x56220f652240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.243+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f652240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.243+0000 7fc39bfff640 1 -- 172.21.15.64:0/2270699097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 msgr2=0x56220f64f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.243+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f64f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.243+0000 7fc39bfff640 1 -- 172.21.15.64:0/2270699097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56220f57a8e0 con 0x56220f64a250 2024-05-09T09:20:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.243+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f64f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.244+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f64a670 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc3840314e0 tx=0x7fc384004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36e2283297f231b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.244+0000 7fc398ff9640 1 -- 172.21.15.64:0/2216379736 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc394004ba0 con 0x7fc39c10b1a0 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.245+0000 7fc391ffb640 1 -- 172.21.15.64:0/2270699097 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384035030 con 0x56220f64a250 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.245+0000 7fc391ffb640 1 -- 172.21.15.64:0/2270699097 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3840054f0 con 0x56220f64a250 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.245+0000 7fc391ffb640 1 -- 172.21.15.64:0/2270699097 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384003850 con 0x56220f64a250 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.245+0000 7fc398ff9640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3880a2e70 0x7fc3880a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.245+0000 7fc398ff9640 1 -- 172.21.15.64:0/2216379736 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc3940cf340 con 0x7fc39c10b1a0 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.246+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2270699097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 msgr2=0x56220f64a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.246+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f64a670 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc3840314e0 tx=0x7fc384004260 comp rx=0 tx=0).stop 2024-05-09T09:20:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.246+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3880a2e70 0x7fc3880a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.247+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3880a2e70 0x7fc3880a5330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x56220f64b040 tx=0x7fc38c007650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.247+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2270699097 shutdown_connections 2024-05-09T09:20:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.247+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f652240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.247+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f64f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.247+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2270699097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f64a670 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.247+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2270699097 >> 172.21.15.64:0/2270699097 conn(0x56220f6458c0 msgr2=0x56220f645cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.249+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2270699097 shutdown_connections 2024-05-09T09:20:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.250+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2270699097 wait complete. 2024-05-09T09:20:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.250+0000 7fc3a49c5740 1 -- start start 2024-05-09T09:20:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.250+0000 7fc3a49c5740 1 Processor -- start 2024-05-09T09:20:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56220f652d00 con 0x56220f64fdb0 2024-05-09T09:20:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56220f569790 con 0x56220f64a250 2024-05-09T09:20:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56220f652e80 con 0x56220f64ac40 2024-05-09T09:20:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc3a49c5740 10 client.0 _ll_register_callbacks cb 0x56220f5a4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc39affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc39affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33968/0 (socket says 172.21.15.64:33968) 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc39affd640 1 -- 172.21.15.64:0/3040431749 learned_addr learned my addr 172.21.15.64:0/3040431749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.251+0000 7fc39bfff640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc39affd640 1 -- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 msgr2=0x56220f654530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc39affd640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc39affd640 1 -- 172.21.15.64:0/3040431749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 msgr2=0x56220f65bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc39affd640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc39affd640 1 -- 172.21.15.64:0/3040431749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220f2c2210 con 0x56220f64ac40 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc39affd640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc39409b4b0 tx=0x7fc39409b1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d4010affd3a785 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc37affd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc37affd640 1 -- 172.21.15.64:0/3040431749 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39401c040 con 0x56220f64ac40 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3840054f0 con 0x56220f64ac40 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc37affd640 1 -- 172.21.15.64:0/3040431749 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc3940a3f90 con 0x56220f64ac40 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.252+0000 7fc37affd640 1 -- 172.21.15.64:0/3040431749 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc39409fb60 con 0x56220f64ac40 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc3a49c5740 10 client.27940 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56220f38e620 con 0x56220f64ac40 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc37affd640 1 -- 172.21.15.64:0/3040431749 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3840054f0 con 0x56220f64ac40 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc37affd640 1 -- 172.21.15.64:0/3040431749 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fc394097500 con 0x56220f64ac40 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc3a49c5740 10 client.27940 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc3a49c5740 10 client.27940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.253+0000 7fc3a49c5740 20 client.27940 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.254+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6605f0 0x56220f662a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.254+0000 7fc3a49c5740 4 client.27940 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.254+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56220f5bba70 con 0x56220f6605f0 2024-05-09T09:20:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.254+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6605f0 0x56220f662a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.255+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6605f0 0x56220f662a10 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e84888be447fae server_cookie=12a24290eaf7c5ae in_seq=0 out_seq=0 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.255+0000 7fc37affd640 10 client.27940 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc37affd640 1 -- 172.21.15.64:0/3040431749 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56220f5bba70 con 0x56220f6605f0 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc37affd640 10 client.27940 handle_command_reply: tid=0 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 client.27940 shutdown 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 msgr2=0x56220f654a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc39409b4b0 tx=0x7fc39409b1d0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 shutdown_connections 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6605f0 0x56220f662a10 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/3040431749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 >> 172.21.15.64:0/3040431749 conn(0x56220f6458c0 msgr2=0x56220f644a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc37affd640 0 client.27940 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 shutdown_connections 2024-05-09T09:20:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.256+0000 7fc3a49c5740 1 -- 172.21.15.64:0/3040431749 wait complete. 2024-05-09T09:20:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.257+0000 7fc3a49c5740 20 client.27940 trim_cache size 0 max 16384 2024-05-09T09:20:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.286+0000 7fc3a49c5740 1 Processor -- start 2024-05-09T09:20:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.286+0000 7fc3a49c5740 1 -- start start 2024-05-09T09:20:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36692/0 (socket says 172.21.15.64:36692) 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc39affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.287+0000 7fc39affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33976/0 (socket says 172.21.15.64:33976) 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39b7fe640 1 -- 172.21.15.64:0/2259957189 learned_addr learned my addr 172.21.15.64:0/2259957189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c012510 con 0x56220f64fdb0 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c00fde0 con 0x56220f64a250 2024-05-09T09:20:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c00dc30 con 0x56220f64ac40 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39b7fe640 1 -- 172.21.15.64:0/2259957189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 msgr2=0x56220f654a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39b7fe640 1 -- 172.21.15.64:0/2259957189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 msgr2=0x56220f65ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.288+0000 7fc39b7fe640 1 -- 172.21.15.64:0/2259957189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56220f649e90 con 0x56220f64a250 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.289+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc38c014760 tx=0x7fc38c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1861341cc9c4420b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.289+0000 7fc3917fa640 1 -- 172.21.15.64:0/2259957189 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01a040 con 0x56220f64a250 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.289+0000 7fc3917fa640 1 -- 172.21.15.64:0/2259957189 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc38c02d070 con 0x56220f64a250 2024-05-09T09:20:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.289+0000 7fc3917fa640 1 -- 172.21.15.64:0/2259957189 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0123f0 con 0x56220f64a250 2024-05-09T09:20:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.291+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 msgr2=0x56220f654530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.291+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc38c014760 tx=0x7fc38c011000 comp rx=0 tx=0).stop 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.292+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 shutdown_connections 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.292+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64fdb0 0x56220f65ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.292+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64ac40 0x56220f654a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.292+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2259957189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64a250 0x56220f654530 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.292+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 >> 172.21.15.64:0/2259957189 conn(0x56220f6458c0 msgr2=0x56220f645ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.293+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 shutdown_connections 2024-05-09T09:20:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.294+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2259957189 wait complete. 2024-05-09T09:20:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.295+0000 7fc3a49c5740 1 -- start start 2024-05-09T09:20:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.295+0000 7fc3a49c5740 1 Processor -- start 2024-05-09T09:20:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.297+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f658910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.297+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f658e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.297+0000 7fc3a49c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f6c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.297+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c00e2a0 con 0x56220f64ac40 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.297+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c00f710 con 0x56220f64fdb0 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.297+0000 7fc3a49c5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c00e970 con 0x56220f64a250 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.298+0000 7fc3a49c5740 10 client.0 _ll_register_callbacks cb 0x7fc3940046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.299+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f658910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.299+0000 7fc39b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f658910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33988/0 (socket says 172.21.15.64:33988) 2024-05-09T09:20:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.299+0000 7fc39b7fe640 1 -- 172.21.15.64:0/2926702589 learned_addr learned my addr 172.21.15.64:0/2926702589 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.299+0000 7fc39affd640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f658e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.299+0000 7fc39bfff640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f6c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.300+0000 7fc39affd640 1 -- 172.21.15.64:0/2926702589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 msgr2=0x56220f658910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.300+0000 7fc39affd640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f658910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.300+0000 7fc39affd640 1 -- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 msgr2=0x56220f6c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.300+0000 7fc39affd640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f6c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.300+0000 7fc39affd640 1 -- 172.21.15.64:0/2926702589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220f6c50a0 con 0x56220f64ac40 2024-05-09T09:20:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.300+0000 7fc39affd640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f658e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc394098180 tx=0x7fc39409a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ec530ec4d41b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.301+0000 7fc37affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.301+0000 7fc37affd640 1 -- 172.21.15.64:0/2926702589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39401c040 con 0x56220f64ac40 2024-05-09T09:20:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.301+0000 7fc37affd640 1 -- 172.21.15.64:0/2926702589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc39409a7e0 con 0x56220f64ac40 2024-05-09T09:20:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.301+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc38c02d070 con 0x56220f64ac40 2024-05-09T09:20:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.302+0000 7fc37affd640 1 -- 172.21.15.64:0/2926702589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc39409b580 con 0x56220f64ac40 2024-05-09T09:20:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.302+0000 7fc3a49c5740 10 client.18318 fetch_fsmap learned FSMap version 45 2024-05-09T09:20:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.302+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56220f6c6290 con 0x56220f64ac40 2024-05-09T09:20:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc37affd640 1 -- 172.21.15.64:0/2926702589 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3940a9020 con 0x56220f64ac40 2024-05-09T09:20:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc37affd640 1 -- 172.21.15.64:0/2926702589 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fc3940a3f90 con 0x56220f64ac40 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc3a49c5740 10 client.18318 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc3a49c5740 10 client.18318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc3a49c5740 20 client.18318 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6c6cc0 0x56220f6c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc3a49c5740 4 client.18318 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.303+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56220f5b5dc0 con 0x56220f6c6cc0 2024-05-09T09:20:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.304+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6c6cc0 0x56220f6c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.305+0000 7fc39b7fe640 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6c6cc0 0x56220f6c90e0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ceb61f74f2e648c server_cookie=445b1e6562e5c4e8 in_seq=0 out_seq=0 2024-05-09T09:20:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.305+0000 7fc37affd640 10 client.18318 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.336+0000 7fc37affd640 1 -- 172.21.15.64:0/2926702589 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56220f5b5dc0 con 0x56220f6c6cc0 2024-05-09T09:20:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.336+0000 7fc37affd640 10 client.18318 handle_command_reply: tid=0 2024-05-09T09:20:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.336+0000 7fc3a49c5740 1 client.18318 shutdown 2024-05-09T09:20:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.337+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 msgr2=0x56220f658e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.337+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f658e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc394098180 tx=0x7fc39409a670 comp rx=0 tx=0).stop 2024-05-09T09:20:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.337+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 shutdown_connections 2024-05-09T09:20:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.337+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56220f6c6cc0 0x56220f6c90e0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56220f64fdb0 0x56220f6c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56220f64ac40 0x56220f658e50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc3a49c5740 1 --2- 172.21.15.64:0/2926702589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56220f64a250 0x56220f658910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 >> 172.21.15.64:0/2926702589 conn(0x56220f6458c0 msgr2=0x56220f64bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc37affd640 0 client.18318 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 shutdown_connections 2024-05-09T09:20:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.338+0000 7fc3a49c5740 1 -- 172.21.15.64:0/2926702589 wait complete. 2024-05-09T09:20:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.339+0000 7fc3a49c5740 20 client.18318 trim_cache size 0 max 16384 2024-05-09T09:20:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:20:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:20:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:20:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:20:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.341+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3880a2e70 msgr2=0x7fc3880a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.341+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3880a2e70 0x7fc3880a5330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x56220f64b040 tx=0x7fc38c007650 comp rx=0 tx=0).stop 2024-05-09T09:20:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.342+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c10b1a0 msgr2=0x7fc39c1a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.342+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c10b1a0 0x7fc39c1a6da0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc39400e9f0 tx=0x7fc39400eeb0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.342+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 shutdown_connections 2024-05-09T09:20:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.342+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc3880a2e70 0x7fc3880a5330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.342+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c10db50 0x7fc39c1addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.343+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39c10b1a0 0x7fc39c1a6da0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.343+0000 7fc3a1be2640 1 --2- 172.21.15.64:0/2216379736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc39c108830 0x7fc39c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.343+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 >> 172.21.15.64:0/2216379736 conn(0x7fc39c06e8a0 msgr2=0x7fc39c10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.344+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 shutdown_connections 2024-05-09T09:20:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:44.344+0000 7fc3a1be2640 1 -- 172.21.15.64:0/2216379736 wait complete. 2024-05-09T09:20:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:45 smithi064 ceph-mon[30149]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 819 op/s 2024-05-09T09:20:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:45 smithi179 ceph-mon[34086]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 819 op/s 2024-05-09T09:20:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:45 smithi139 ceph-mon[33599]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 819 op/s 2024-05-09T09:20:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 625.0 s, 812.0 tps, lat 123.105 ms stddev 154.017 2024-05-09T09:20:46.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:46 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:46.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:46 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:46.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:46 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:20:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 789 op/s 2024-05-09T09:20:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:47 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 789 op/s 2024-05-09T09:20:47.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:47 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 789 op/s 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:47 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:48.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:48.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:48.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:48.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:20:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:20:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:20:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:49 smithi064 ceph-mon[30149]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 701 op/s 2024-05-09T09:20:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:49 smithi179 ceph-mon[34086]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 701 op/s 2024-05-09T09:20:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:49 smithi139 ceph-mon[33599]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 701 op/s 2024-05-09T09:20:50.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:50 smithi139 ceph-mon[33599]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 912 op/s 2024-05-09T09:20:50.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:50 smithi179 ceph-mon[34086]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 912 op/s 2024-05-09T09:20:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:50 smithi064 ceph-mon[30149]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 912 op/s 2024-05-09T09:20:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 630.0 s, 859.0 tps, lat 116.316 ms stddev 58.117 2024-05-09T09:20:51.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:20:51.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:20:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.172+0000 7efca4763640 1 -- 172.21.15.64:0/4110110126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 msgr2=0x7efc9c110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.172+0000 7efca4763640 1 --2- 172.21.15.64:0/4110110126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c110830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efc9400adb0 tx=0x7efc9402b7c0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.181+0000 7efca4763640 1 -- 172.21.15.64:0/4110110126 shutdown_connections 2024-05-09T09:20:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.181+0000 7efca4763640 1 --2- 172.21.15.64:0/4110110126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c110830 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.181+0000 7efca4763640 1 --2- 172.21.15.64:0/4110110126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c10b910 0x7efc9c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.181+0000 7efca4763640 1 --2- 172.21.15.64:0/4110110126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc9c06a670 0x7efc9c10b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.181+0000 7efca4763640 1 -- 172.21.15.64:0/4110110126 >> 172.21.15.64:0/4110110126 conn(0x7efc9c06ea60 msgr2=0x7efc9c070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.189+0000 7efca4763640 1 -- 172.21.15.64:0/4110110126 shutdown_connections 2024-05-09T09:20:52.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.193+0000 7efca4763640 1 -- 172.21.15.64:0/4110110126 wait complete. 2024-05-09T09:20:52.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.194+0000 7efca4763640 1 Processor -- start 2024-05-09T09:20:52.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.194+0000 7efca4763640 1 -- start start 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.194+0000 7efca4763640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c06a670 0x7efc9c1a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca4763640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc9c10b910 0x7efc9c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca4763640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c1a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca4763640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c113a30 con 0x7efc9c06a670 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca4763640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1138b0 con 0x7efc9c10b910 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca4763640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c113bb0 con 0x7efc9c10e370 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca2d00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c1a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca2d00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c1a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59480/0 (socket says 172.21.15.64:59480) 2024-05-09T09:20:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.195+0000 7efca2d00640 1 -- 172.21.15.64:0/3606094887 learned_addr learned my addr 172.21.15.64:0/3606094887 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:52.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca24ff640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c06a670 0x7efc9c1a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca2d00640 1 -- 172.21.15.64:0/3606094887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc9c10b910 msgr2=0x7efc9c1a0880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:20:52.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca2d00640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc9c10b910 0x7efc9c1a0880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca2d00640 1 -- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c06a670 msgr2=0x7efc9c1a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca2d00640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c06a670 0x7efc9c1a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca2d00640 1 -- 172.21.15.64:0/3606094887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c1a7f60 con 0x7efc9c10e370 2024-05-09T09:20:52.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.197+0000 7efca24ff640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c06a670 0x7efc9c1a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:52.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.198+0000 7efca2d00640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c1a78f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efc94005e30 tx=0x7efc94030740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.201+0000 7efc937fe640 1 -- 172.21.15.64:0/3606094887 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94040070 con 0x7efc9c10e370 2024-05-09T09:20:52.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.201+0000 7efc937fe640 1 -- 172.21.15.64:0/3606094887 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc9403c8c0 con 0x7efc9c10e370 2024-05-09T09:20:52.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.201+0000 7efc937fe640 1 -- 172.21.15.64:0/3606094887 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9403b8f0 con 0x7efc9c10e370 2024-05-09T09:20:52.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.202+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c109330 con 0x7efc9c10e370 2024-05-09T09:20:52.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.202+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c1097a0 con 0x7efc9c10e370 2024-05-09T09:20:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.210+0000 7efca7473740 1 Processor -- start 2024-05-09T09:20:52.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.214+0000 7efc937fe640 1 -- 172.21.15.64:0/3606094887 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc9403c410 con 0x7efc9c10e370 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.215+0000 7efc937fe640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efc880a71a0 0x7efc880a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.215+0000 7efc937fe640 1 -- 172.21.15.64:0/3606094887 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efc940f1130 con 0x7efc9c10e370 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.216+0000 7efca7473740 1 -- start start 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.217+0000 7efca7473740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.217+0000 7efca7473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d6130 0x55edb43dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.217+0000 7efca7473740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43dd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.217+0000 7efca7473740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55edb4331d00 con 0x55edb43db2a0 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.217+0000 7efca7473740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55edb42d94c0 con 0x55edb43d5740 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.217+0000 7efca7473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55edb432dcc0 con 0x55edb43d6130 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca1cfe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d6130 0x55edb43dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca1cfe640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efc880a71a0 0x7efc880a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43dd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36844/0 (socket says 172.21.15.64:36844) 2024-05-09T09:20:52.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca24ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 -- 172.21.15.64:0/1157712454 learned_addr learned my addr 172.21.15.64:0/1157712454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 -- 172.21.15.64:0/1157712454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d6130 msgr2=0x55edb43dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d6130 0x55edb43dac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 -- 172.21.15.64:0/1157712454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 msgr2=0x55edb43d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43d5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.219+0000 7efca2d00640 1 -- 172.21.15.64:0/1157712454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb410f7d0 con 0x55edb43db2a0 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.220+0000 7efca2d00640 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43dd730 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efc9400ad00 tx=0x7efc9403b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e45a6414437af44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.221+0000 7efc90ff9640 1 -- 172.21.15.64:0/1157712454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc940d5070 con 0x55edb43db2a0 2024-05-09T09:20:52.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.221+0000 7efc90ff9640 1 -- 172.21.15.64:0/1157712454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc940bdca0 con 0x55edb43db2a0 2024-05-09T09:20:52.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.222+0000 7efca7473740 1 -- 172.21.15.64:0/1157712454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 msgr2=0x55edb43dd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.222+0000 7efca7473740 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43dd730 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efc9400ad00 tx=0x7efc9403b3a0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.222+0000 7efca1cfe640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efc880a71a0 0x7efc880a9660 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55edb43d6530 tx=0x7efc8c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.223+0000 7efca7473740 1 -- 172.21.15.64:0/1157712454 shutdown_connections 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.223+0000 7efca7473740 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43dd730 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.223+0000 7efca7473740 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d6130 0x55edb43dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.223+0000 7efca7473740 1 --2- 172.21.15.64:0/1157712454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43d5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.223+0000 7efca7473740 1 -- 172.21.15.64:0/1157712454 >> 172.21.15.64:0/1157712454 conn(0x55edb43c85a0 msgr2=0x55edb43c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.223+0000 7efca7473740 1 -- 172.21.15.64:0/1157712454 shutdown_connections 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.224+0000 7efca7473740 1 -- 172.21.15.64:0/1157712454 wait complete. 2024-05-09T09:20:52.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.224+0000 7efca7473740 1 -- start start 2024-05-09T09:20:52.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 Processor -- start 2024-05-09T09:20:52.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 0x55edb43dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55edb43de010 con 0x55edb43db2a0 2024-05-09T09:20:52.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55edb42ecb90 con 0x55edb43d6130 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55edb43de190 con 0x55edb43d5740 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.225+0000 7efca7473740 10 client.0 _ll_register_callbacks cb 0x55edb4328da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 0x55edb43dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 0x55edb43dfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50022/0 (socket says 172.21.15.64:50022) 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 -- 172.21.15.64:0/3322278802 learned_addr learned my addr 172.21.15.64:0/3322278802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca2d00640 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 -- 172.21.15.64:0/3322278802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 msgr2=0x55edb43df920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43df920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 -- 172.21.15.64:0/3322278802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 msgr2=0x55edb43e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43e6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.226+0000 7efca1cfe640 1 -- 172.21.15.64:0/3322278802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb40453f0 con 0x55edb43d6130 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.227+0000 7efca1cfe640 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 0x55edb43dfe60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0739a0 tx=0x7efc8c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3013faca1ff99e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.227+0000 7efc85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.227+0000 7efc85ffb640 1 -- 172.21.15.64:0/3322278802 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c022030 con 0x55edb43d6130 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.228+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc940bdca0 con 0x55edb43d6130 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.228+0000 7efc85ffb640 1 -- 172.21.15.64:0/3322278802 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efc8c0119f0 con 0x55edb43d6130 2024-05-09T09:20:52.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.228+0000 7efc85ffb640 1 -- 172.21.15.64:0/3322278802 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7efc8c02f050 con 0x55edb43d6130 2024-05-09T09:20:52.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.234+0000 7efca7473740 10 client.27785 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:52.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.234+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55edb40050f0 con 0x55edb43d6130 2024-05-09T09:20:52.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efc85ffb640 1 -- 172.21.15.64:0/3322278802 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc940bdca0 con 0x55edb43d6130 2024-05-09T09:20:52.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efc85ffb640 1 -- 172.21.15.64:0/3322278802 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7efc8c01b050 con 0x55edb43d6130 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca7473740 10 client.27785 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca7473740 10 client.27785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca7473740 20 client.27785 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca7473740 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb43ce6c0 0x55edb43d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca7473740 4 client.27785 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edb433f430 con 0x55edb43ce6c0 2024-05-09T09:20:52.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.235+0000 7efca24ff640 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb43ce6c0 0x55edb43d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.239+0000 7efca24ff640 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb43ce6c0 0x55edb43d0b60 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3121c8687d521d67 server_cookie=b8f166aafd53f953 in_seq=0 out_seq=0 2024-05-09T09:20:52.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.243+0000 7efc85ffb640 10 client.27785 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:52.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.243+0000 7efc85ffb640 1 -- 172.21.15.64:0/3322278802 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edb433f430 con 0x55edb43ce6c0 2024-05-09T09:20:52.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.243+0000 7efc85ffb640 10 client.27785 handle_command_reply: tid=0 2024-05-09T09:20:52.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.243+0000 7efca7473740 1 client.27785 shutdown 2024-05-09T09:20:52.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.247+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 msgr2=0x55edb43dfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.247+0000 7efca7473740 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 0x55edb43dfe60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0739a0 tx=0x7efc8c0046c0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.248+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 shutdown_connections 2024-05-09T09:20:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.248+0000 7efca7473740 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb43ce6c0 0x55edb43d0b60 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.248+0000 7efca7473740 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43db2a0 0x55edb43e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.248+0000 7efca7473740 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d6130 0x55edb43dfe60 secure :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0739a0 tx=0x7efc8c0046c0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.248+0000 7efca7473740 1 --2- 172.21.15.64:0/3322278802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43df920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.248+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 >> 172.21.15.64:0/3322278802 conn(0x55edb43c85a0 msgr2=0x55edb43c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.249+0000 7efc85ffb640 0 client.27785 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.251+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 shutdown_connections 2024-05-09T09:20:52.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.254+0000 7efca7473740 1 -- 172.21.15.64:0/3322278802 wait complete. 2024-05-09T09:20:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.256+0000 7efca7473740 20 client.27785 trim_cache size 0 max 16384 2024-05-09T09:20:52.266 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:52 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:20:52.267 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:52 smithi064 ceph-mon[30149]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 674 op/s 2024-05-09T09:20:52.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 Processor -- start 2024-05-09T09:20:52.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 -- start start 2024-05-09T09:20:52.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 0x55edb43cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b3410 con 0x55edb43c85a0 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b0cb0 con 0x55edb43d5740 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca7473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1aeb00 con 0x55edb43db2a0 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca2d00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 0x55edb43cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca2d00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 0x55edb43cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59496/0 (socket says 172.21.15.64:59496) 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.295+0000 7efca2d00640 1 -- 172.21.15.64:0/585960609 learned_addr learned my addr 172.21.15.64:0/585960609 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca1cfe640 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca2d00640 1 -- 172.21.15.64:0/585960609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 msgr2=0x55edb43dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca2d00640 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43dfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca2d00640 1 -- 172.21.15.64:0/585960609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 msgr2=0x55edb43df920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca2d00640 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca2d00640 1 -- 172.21.15.64:0/585960609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb43c9270 con 0x55edb43db2a0 2024-05-09T09:20:52.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.296+0000 7efca2d00640 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 0x55edb43cb480 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc940b9a00 tx=0x7efc9400a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31062be5543fd66d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.297+0000 7efc87fff640 1 -- 172.21.15.64:0/585960609 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc940d5040 con 0x55edb43db2a0 2024-05-09T09:20:52.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.298+0000 7efc87fff640 1 -- 172.21.15.64:0/585960609 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc940c1950 con 0x55edb43db2a0 2024-05-09T09:20:52.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.298+0000 7efc87fff640 1 -- 172.21.15.64:0/585960609 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc940b9c10 con 0x55edb43db2a0 2024-05-09T09:20:52.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.299+0000 7efca7473740 1 -- 172.21.15.64:0/585960609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 msgr2=0x55edb43cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.299+0000 7efca7473740 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 0x55edb43cb480 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc940b9a00 tx=0x7efc9400a6d0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.302+0000 7efca7473740 1 -- 172.21.15.64:0/585960609 shutdown_connections 2024-05-09T09:20:52.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.302+0000 7efca7473740 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43db2a0 0x55edb43cb480 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.302+0000 7efca7473740 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43d5740 0x55edb43dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.302+0000 7efca7473740 1 --2- 172.21.15.64:0/585960609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43df920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.302+0000 7efca7473740 1 -- 172.21.15.64:0/585960609 >> 172.21.15.64:0/585960609 conn(0x55edb43c39e0 msgr2=0x55edb43d7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.303+0000 7efca7473740 1 -- 172.21.15.64:0/585960609 shutdown_connections 2024-05-09T09:20:52.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.303+0000 7efca7473740 1 -- 172.21.15.64:0/585960609 wait complete. 2024-05-09T09:20:52.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.303+0000 7efca7473740 1 -- start start 2024-05-09T09:20:52.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 Processor -- start 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43db2a0 0x55edb4447de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1af170 con 0x55edb43c85a0 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b05e0 con 0x55edb43db2a0 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1af840 con 0x55edb43d5740 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.304+0000 7efca7473740 10 client.0 _ll_register_callbacks cb 0x7efc8c00cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:52.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.305+0000 7efca24ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.305+0000 7efca24ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43eeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36874/0 (socket says 172.21.15.64:36874) 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.305+0000 7efca24ff640 1 -- 172.21.15.64:0/4201755754 learned_addr learned my addr 172.21.15.64:0/4201755754 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.306+0000 7efca1cfe640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.307+0000 7efca2d00640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43db2a0 0x55edb4447de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.307+0000 7efca24ff640 1 -- 172.21.15.64:0/4201755754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 msgr2=0x55edb43ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.307+0000 7efca24ff640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43ef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.307+0000 7efca24ff640 1 -- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43db2a0 msgr2=0x55edb4447de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.307+0000 7efca24ff640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43db2a0 0x55edb4447de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.307+0000 7efca24ff640 1 -- 172.21.15.64:0/4201755754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb44483d0 con 0x55edb43c85a0 2024-05-09T09:20:52.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.309+0000 7efca24ff640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43eeb70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x55edb43d1410 tx=0x7efc9800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e496ecf74efefda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.310+0000 7efc85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.311+0000 7efc85ffb640 1 -- 172.21.15.64:0/4201755754 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98014c80 con 0x55edb43c85a0 2024-05-09T09:20:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.311+0000 7efc85ffb640 1 -- 172.21.15.64:0/4201755754 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efc98014e20 con 0x55edb43c85a0 2024-05-09T09:20:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.312+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edb4339780 con 0x55edb43c85a0 2024-05-09T09:20:52.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.314+0000 7efc85ffb640 1 -- 172.21.15.64:0/4201755754 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7efc9800be50 con 0x55edb43c85a0 2024-05-09T09:20:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.318+0000 7efca7473740 10 client.18330 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.318+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55edb44497b0 con 0x55edb43c85a0 2024-05-09T09:20:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.321+0000 7efc85ffb640 1 -- 172.21.15.64:0/4201755754 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55edb4339780 con 0x55edb43c85a0 2024-05-09T09:20:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.321+0000 7efc85ffb640 1 -- 172.21.15.64:0/4201755754 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7efc980219d0 con 0x55edb43c85a0 2024-05-09T09:20:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.322+0000 7efca7473740 10 client.18330 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.322+0000 7efca7473740 10 client.18330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.322+0000 7efca7473740 20 client.18330 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.322+0000 7efca7473740 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb444a140 0x55edb444c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.322+0000 7efca7473740 4 client.18330 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:20:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.322+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55edb42b66f0 con 0x55edb444a140 2024-05-09T09:20:52.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.326+0000 7efca1cfe640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb444a140 0x55edb444c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.328+0000 7efca1cfe640 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb444a140 0x55edb444c560 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985d7a3b60942f88 server_cookie=c8750e33d81dd37e in_seq=0 out_seq=0 2024-05-09T09:20:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.328+0000 7efc85ffb640 10 client.18330 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.329+0000 7efc85ffb640 1 -- 172.21.15.64:0/4201755754 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55edb42b66f0 con 0x55edb444a140 2024-05-09T09:20:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.329+0000 7efc85ffb640 10 client.18330 handle_command_reply: tid=0 2024-05-09T09:20:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.330+0000 7efca7473740 1 client.18330 shutdown 2024-05-09T09:20:52.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.339+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 msgr2=0x55edb43eeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.339+0000 7efca7473740 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43eeb70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x55edb43d1410 tx=0x7efc9800bc60 comp rx=0 tx=0).stop 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 shutdown_connections 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efca7473740 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55edb444a140 0x55edb444c560 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efca7473740 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edb43db2a0 0x55edb4447de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efca7473740 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55edb43d5740 0x55edb43ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efca7473740 1 --2- 172.21.15.64:0/4201755754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55edb43c85a0 0x55edb43eeb70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 >> 172.21.15.64:0/4201755754 conn(0x55edb43c39e0 msgr2=0x55edb43df7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.340+0000 7efc85ffb640 0 client.18330 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.343+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 shutdown_connections 2024-05-09T09:20:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.346+0000 7efca7473740 1 -- 172.21.15.64:0/4201755754 wait complete. 2024-05-09T09:20:52.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.347+0000 7efca7473740 20 client.18330 trim_cache size 0 max 16384 2024-05-09T09:20:52.350 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:20:52.350 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:20:52.350 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:20:52.350 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:20:52.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.354+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efc880a71a0 msgr2=0x7efc880a9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.354+0000 7efca4763640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efc880a71a0 0x7efc880a9660 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55edb43d6530 tx=0x7efc8c005c70 comp rx=0 tx=0).stop 2024-05-09T09:20:52.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.354+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 msgr2=0x7efc9c1a78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.354+0000 7efca4763640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c1a78f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efc94005e30 tx=0x7efc94030740 comp rx=0 tx=0).stop 2024-05-09T09:20:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.356+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 shutdown_connections 2024-05-09T09:20:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.356+0000 7efca4763640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7efc880a71a0 0x7efc880a9660 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.356+0000 7efca4763640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc9c10e370 0x7efc9c1a78f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.356+0000 7efca4763640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc9c10b910 0x7efc9c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.356+0000 7efca4763640 1 --2- 172.21.15.64:0/3606094887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c06a670 0x7efc9c1a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.356+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 >> 172.21.15.64:0/3606094887 conn(0x7efc9c06ea60 msgr2=0x7efc9c070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.359+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 shutdown_connections 2024-05-09T09:20:52.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.359+0000 7efca4763640 1 -- 172.21.15.64:0/3606094887 wait complete. 2024-05-09T09:20:52.383 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:20:52.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:20:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:52 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:20:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:52 smithi179 ceph-mon[34086]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 674 op/s 2024-05-09T09:20:52.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.557+0000 7f385b45b640 1 -- 172.21.15.64:0/2977440967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c094450 msgr2=0x7f384c09b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.557+0000 7f385b45b640 1 --2- 172.21.15.64:0/2977440967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c094450 0x7f384c09b450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3850009860 tx=0x7f385002a360 comp rx=0 tx=0).stop 2024-05-09T09:20:52.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.558+0000 7f385b45b640 1 -- 172.21.15.64:0/2977440967 shutdown_connections 2024-05-09T09:20:52.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.558+0000 7f385b45b640 1 --2- 172.21.15.64:0/2977440967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f384c09b9e0 0x7f384c09dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.558+0000 7f385b45b640 1 --2- 172.21.15.64:0/2977440967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c094450 0x7f384c09b450 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.558+0000 7f385b45b640 1 --2- 172.21.15.64:0/2977440967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c093af0 0x7f384c093f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.558+0000 7f385b45b640 1 -- 172.21.15.64:0/2977440967 >> 172.21.15.64:0/2977440967 conn(0x7f384c08f500 msgr2=0x7f384c091960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.559+0000 7f385b45b640 1 -- 172.21.15.64:0/2977440967 shutdown_connections 2024-05-09T09:20:52.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.559+0000 7f385b45b640 1 -- 172.21.15.64:0/2977440967 wait complete. 2024-05-09T09:20:52.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.561+0000 7f385b45b640 1 Processor -- start 2024-05-09T09:20:52.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.561+0000 7f385b45b640 1 -- start start 2024-05-09T09:20:52.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.561+0000 7f385b45b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f384c093af0 0x7f384c12e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.561+0000 7f385b45b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 0x7f384c12e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.561+0000 7f385b45b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c09b9e0 0x7f384c1357a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f385b45b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0a3af0 con 0x7f384c094450 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f385b45b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0a3970 con 0x7f384c093af0 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f385b45b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0a3c70 con 0x7f384c09b9e0 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 0x7f384c12e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 0x7f384c12e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36906/0 (socket says 172.21.15.64:36906) 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 -- 172.21.15.64:0/352842327 learned_addr learned my addr 172.21.15.64:0/352842327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f385ac5a640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c09b9e0 0x7f384c1357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 -- 172.21.15.64:0/352842327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c09b9e0 msgr2=0x7f384c1357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c09b9e0 0x7f384c1357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 -- 172.21.15.64:0/352842327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f384c093af0 msgr2=0x7f384c12e1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:20:52.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f384c093af0 0x7f384c12e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.562+0000 7f3859c58640 1 -- 172.21.15.64:0/352842327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c135e10 con 0x7f384c094450 2024-05-09T09:20:52.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.563+0000 7f3859c58640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 0x7f384c12e730 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f385002a880 tx=0x7f38500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.565+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850040020 con 0x7f384c094450 2024-05-09T09:20:52.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.566+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3850003ca0 con 0x7f384c094450 2024-05-09T09:20:52.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.566+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850037040 con 0x7f384c094450 2024-05-09T09:20:52.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.566+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c1360a0 con 0x7f384c094450 2024-05-09T09:20:52.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.566+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c136480 con 0x7f384c094450 2024-05-09T09:20:52.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.569+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c136740 con 0x7f384c094450 2024-05-09T09:20:52.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.572+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3850043020 con 0x7f384c094450 2024-05-09T09:20:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:52 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:20:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:52 smithi139 ceph-mon[33599]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 674 op/s 2024-05-09T09:20:52.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.573+0000 7f384b7fe640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f38400a2d60 0x7f38400a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:52.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.573+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f38500ef7b0 con 0x7f384c094450 2024-05-09T09:20:52.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.574+0000 7f385a459640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f38400a2d60 0x7f38400a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:52.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.575+0000 7f385a459640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f38400a2d60 0x7f38400a5220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f38440058c0 tx=0x7f3844005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.585+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f385004a020 con 0x7f384c094450 2024-05-09T09:20:52.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.990+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f384c136ad0 con 0x7f384c094450 2024-05-09T09:20:52.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.992+0000 7f384b7fe640 1 -- 172.21.15.64:0/352842327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11880 (secure 0 0 0) 0x7f38500bc320 con 0x7f384c094450 2024-05-09T09:20:52.993 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:20:52.993 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:20:50.564954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:20:52.993 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2024-05-09T09:20:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.995+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f38400a2d60 msgr2=0x7f38400a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.995+0000 7f385b45b640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f38400a2d60 0x7f38400a5220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f38440058c0 tx=0x7f3844005850 comp rx=0 tx=0).stop 2024-05-09T09:20:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.995+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 msgr2=0x7f384c12e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:52.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.995+0000 7f385b45b640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 0x7f384c12e730 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f385002a880 tx=0x7f38500042f0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 shutdown_connections 2024-05-09T09:20:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f38400a2d60 0x7f38400a5220 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c09b9e0 0x7f384c1357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c094450 0x7f384c12e730 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 --2- 172.21.15.64:0/352842327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f384c093af0 0x7f384c12e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:52.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 >> 172.21.15.64:0/352842327 conn(0x7f384c08f500 msgr2=0x7f384c090170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.996+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 shutdown_connections 2024-05-09T09:20:52.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:52.997+0000 7f385b45b640 1 -- 172.21.15.64:0/352842327 wait complete. 2024-05-09T09:20:53.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.207+0000 7ff761058640 1 -- 172.21.15.64:0/2715348491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c100050 msgr2=0x7ff75c100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.207+0000 7ff761058640 1 --2- 172.21.15.64:0/2715348491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c100050 0x7ff75c100470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009900 tx=0x7ff74c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 -- 172.21.15.64:0/2715348491 shutdown_connections 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 --2- 172.21.15.64:0/2715348491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c101350 0x7ff75c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 --2- 172.21.15.64:0/2715348491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c1009b0 0x7ff75c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 --2- 172.21.15.64:0/2715348491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c100050 0x7ff75c100470 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 -- 172.21.15.64:0/2715348491 >> 172.21.15.64:0/2715348491 conn(0x7ff75c0fbbd0 msgr2=0x7ff75c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 -- 172.21.15.64:0/2715348491 shutdown_connections 2024-05-09T09:20:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.208+0000 7ff761058640 1 -- 172.21.15.64:0/2715348491 wait complete. 2024-05-09T09:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.209+0000 7ff761058640 1 Processor -- start 2024-05-09T09:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.209+0000 7ff761058640 1 -- start start 2024-05-09T09:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff761058640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c100050 0x7ff75c1109a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff761058640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 0x7ff75c110ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff761058640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c101350 0x7ff75c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff761058640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c105030 con 0x7ff75c101350 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff761058640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c104eb0 con 0x7ff75c1009b0 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff761058640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1051b0 con 0x7ff75c100050 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 0x7ff75c110ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 0x7ff75c110ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50064/0 (socket says 172.21.15.64:50064) 2024-05-09T09:20:53.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff75b7fe640 1 -- 172.21.15.64:0/1842109454 learned_addr learned my addr 172.21.15.64:0/1842109454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.210+0000 7ff760857640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c101350 0x7ff75c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75bfff640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c100050 0x7ff75c1109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75b7fe640 1 -- 172.21.15.64:0/1842109454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c100050 msgr2=0x7ff75c1109a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c100050 0x7ff75c1109a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75b7fe640 1 -- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c101350 msgr2=0x7ff75c113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c101350 0x7ff75c113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75b7fe640 1 -- 172.21.15.64:0/1842109454 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c1145a0 con 0x7ff75c1009b0 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff760857640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c101350 0x7ff75c113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75bfff640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c100050 0x7ff75c1109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 0x7ff75c110ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff75000e9f0 tx=0x7ff75000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff7597fa640 1 -- 172.21.15.64:0/1842109454 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75000cd80 con 0x7ff75c1009b0 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff7597fa640 1 -- 172.21.15.64:0/1842109454 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7500108e0 con 0x7ff75c1009b0 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.211+0000 7ff7597fa640 1 -- 172.21.15.64:0/1842109454 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750018920 con 0x7ff75c1009b0 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.212+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c114890 con 0x7ff75c1009b0 2024-05-09T09:20:53.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.212+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c06b250 con 0x7ff75c1009b0 2024-05-09T09:20:53.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.216+0000 7ff7597fa640 1 -- 172.21.15.64:0/1842109454 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff750010a90 con 0x7ff75c1009b0 2024-05-09T09:20:53.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.218+0000 7ff764fc8740 1 Processor -- start 2024-05-09T09:20:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.218+0000 7ff7597fa640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7440a2c90 0x7ff7440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff7597fa640 1 -- 172.21.15.64:0/1842109454 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff7500cf7e0 con 0x7ff75c1009b0 2024-05-09T09:20:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff760857640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7440a2c90 0x7ff7440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 -- start start 2024-05-09T09:20:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bb041bc40 con 0x563bb04b7130 2024-05-09T09:20:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563bb03c3580 con 0x563bb04b7b20 2024-05-09T09:20:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.219+0000 7ff764fc8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563bb0417c00 con 0x563bb04bcc90 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04bc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50072/0 (socket says 172.21.15.64:50072) 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 -- 172.21.15.64:0/3768780475 learned_addr learned my addr 172.21.15.64:0/3768780475 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75bfff640 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 -- 172.21.15.64:0/3768780475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 msgr2=0x563bb04bf120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04bf120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 -- 172.21.15.64:0/3768780475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 msgr2=0x563bb04b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.221+0000 7ff75b7fe640 1 -- 172.21.15.64:0/3768780475 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb012f3f0 con 0x563bb04b7b20 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.222+0000 7ff75bfff640 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.222+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04bc620 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff750098620 tx=0x7ff75009c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e01694a658360c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.223+0000 7ff760857640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7440a2c90 0x7ff7440a5150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff754005fd0 tx=0x7ff754009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.223+0000 7ff7427fc640 1 -- 172.21.15.64:0/3768780475 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b1030 con 0x563bb04b7b20 2024-05-09T09:20:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.223+0000 7ff7427fc640 1 -- 172.21.15.64:0/3768780475 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7500a48c0 con 0x563bb04b7b20 2024-05-09T09:20:53.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.224+0000 7ff7427fc640 1 -- 172.21.15.64:0/3768780475 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75009b0a0 con 0x563bb04b7b20 2024-05-09T09:20:53.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.224+0000 7ff764fc8740 1 -- 172.21.15.64:0/3768780475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 msgr2=0x563bb04bc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.224+0000 7ff764fc8740 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04bc620 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff750098620 tx=0x7ff75009c350 comp rx=0 tx=0).stop 2024-05-09T09:20:53.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.224+0000 7ff764fc8740 1 -- 172.21.15.64:0/3768780475 shutdown_connections 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.224+0000 7ff764fc8740 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04bf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.225+0000 7ff764fc8740 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04bc620 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.225+0000 7ff764fc8740 1 --2- 172.21.15.64:0/3768780475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.225+0000 7ff764fc8740 1 -- 172.21.15.64:0/3768780475 >> 172.21.15.64:0/3768780475 conn(0x563bb04b27a0 msgr2=0x563bb04b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.225+0000 7ff764fc8740 1 -- 172.21.15.64:0/3768780475 shutdown_connections 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.225+0000 7ff764fc8740 1 -- 172.21.15.64:0/3768780475 wait complete. 2024-05-09T09:20:53.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.226+0000 7ff764fc8740 1 -- start start 2024-05-09T09:20:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.226+0000 7ff764fc8740 1 Processor -- start 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb04c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bb04bfce0 con 0x563bb04bcc90 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563bb03d6c40 con 0x563bb04b7b20 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563bb04bfe60 con 0x563bb04b7130 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff760857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb04c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff760857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb04c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36934/0 (socket says 172.21.15.64:36934) 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff760857640 1 -- 172.21.15.64:0/1633473855 learned_addr learned my addr 172.21.15.64:0/1633473855 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff764fc8740 10 client.0 _ll_register_callbacks cb 0x563bb012ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.227+0000 7ff75bfff640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff760857640 1 -- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7130 msgr2=0x563bb04c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff760857640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff760857640 1 -- 172.21.15.64:0/1633473855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 msgr2=0x563bb04c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff760857640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff760857640 1 -- 172.21.15.64:0/1633473855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb01f97d0 con 0x563bb04bcc90 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff760857640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb04c8c50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff754004260 tx=0x7ff754004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68057224db24dce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.228+0000 7ff73b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.229+0000 7ff73b7fe640 1 -- 172.21.15.64:0/1633473855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754011c80 con 0x563bb04bcc90 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.229+0000 7ff73b7fe640 1 -- 172.21.15.64:0/1633473855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff75401f430 con 0x563bb04bcc90 2024-05-09T09:20:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.229+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7500a48c0 con 0x563bb04bcc90 2024-05-09T09:20:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.230+0000 7ff73b7fe640 1 -- 172.21.15.64:0/1633473855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff754011e20 con 0x563bb04bcc90 2024-05-09T09:20:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.230+0000 7ff764fc8740 10 client.18345 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.230+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563bb04cc000 con 0x563bb04bcc90 2024-05-09T09:20:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.230+0000 7ff73b7fe640 1 -- 172.21.15.64:0/1633473855 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7500a48c0 con 0x563bb04bcc90 2024-05-09T09:20:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff73b7fe640 1 -- 172.21.15.64:0/1633473855 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7ff75400e6b0 con 0x563bb04bcc90 2024-05-09T09:20:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff764fc8740 10 client.18345 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff764fc8740 10 client.18345 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:20:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff764fc8740 20 client.18345 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff764fc8740 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb04cd7c0 0x563bb04cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff764fc8740 4 client.18345 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb0429370 con 0x563bb04cd7c0 2024-05-09T09:20:53.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.231+0000 7ff75bfff640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb04cd7c0 0x563bb04cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.232+0000 7ff75bfff640 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb04cd7c0 0x563bb04cfc10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c43ddccce1e1d265 server_cookie=32f4d2d282a75fea in_seq=0 out_seq=0 2024-05-09T09:20:53.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.232+0000 7ff73b7fe640 10 client.18345 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:53.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.234+0000 7ff73b7fe640 1 -- 172.21.15.64:0/1633473855 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bb0429370 con 0x563bb04cd7c0 2024-05-09T09:20:53.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.234+0000 7ff73b7fe640 10 client.18345 handle_command_reply: tid=0 2024-05-09T09:20:53.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.234+0000 7ff764fc8740 1 client.18345 shutdown 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.234+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 msgr2=0x563bb04c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.234+0000 7ff764fc8740 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb04c8c50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff754004260 tx=0x7ff754004ab0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.234+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 shutdown_connections 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb04cd7c0 0x563bb04cfc10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb04c8c50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 --2- 172.21.15.64:0/1633473855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff73b7fe640 0 client.18345 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 >> 172.21.15.64:0/1633473855 conn(0x563bb04b27a0 msgr2=0x563bb04b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 shutdown_connections 2024-05-09T09:20:53.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 1 -- 172.21.15.64:0/1633473855 wait complete. 2024-05-09T09:20:53.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.235+0000 7ff764fc8740 20 client.18345 trim_cache size 0 max 16384 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.261+0000 7ff764fc8740 1 Processor -- start 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.261+0000 7ff764fc8740 1 -- start start 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.261+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50082/0 (socket says 172.21.15.64:50082) 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff75b7fe640 1 -- 172.21.15.64:0/4116195425 learned_addr learned my addr 172.21.15.64:0/4116195425 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1c4180 con 0x563bb04b7130 2024-05-09T09:20:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1c1a20 con 0x563bb04b7b20 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff760857640 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff75bfff640 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.262+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c0736d0 con 0x563bb04bcc90 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.263+0000 7ff75b7fe640 1 -- 172.21.15.64:0/4116195425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 msgr2=0x563bb04c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.263+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.263+0000 7ff75b7fe640 1 -- 172.21.15.64:0/4116195425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 msgr2=0x563bb04c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.263+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.263+0000 7ff75b7fe640 1 -- 172.21.15.64:0/4116195425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb04b8fe0 con 0x563bb04b7b20 2024-05-09T09:20:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.263+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff750009380 tx=0x7ff75009ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dfaaf589232b223 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.264+0000 7ff741ffb640 1 -- 172.21.15.64:0/4116195425 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b1070 con 0x563bb04b7b20 2024-05-09T09:20:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.264+0000 7ff741ffb640 1 -- 172.21.15.64:0/4116195425 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7500b4e50 con 0x563bb04b7b20 2024-05-09T09:20:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.264+0000 7ff741ffb640 1 -- 172.21.15.64:0/4116195425 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750018ac0 con 0x563bb04b7b20 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 msgr2=0x563bb04c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff750009380 tx=0x7ff75009ba20 comp rx=0 tx=0).stop 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 shutdown_connections 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04bcc90 0x563bb04c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7b20 0x563bb04c1c10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 --2- 172.21.15.64:0/4116195425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04b7130 0x563bb04c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 >> 172.21.15.64:0/4116195425 conn(0x563bb04b27a0 msgr2=0x563bb04b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 shutdown_connections 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.266+0000 7ff764fc8740 1 -- 172.21.15.64:0/4116195425 wait complete. 2024-05-09T09:20:53.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.267+0000 7ff764fc8740 1 -- start start 2024-05-09T09:20:53.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.267+0000 7ff764fc8740 1 Processor -- start 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.267+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7130 0x563bb04c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.267+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7b20 0x563bb04c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff764fc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb0531d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff764fc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1bfee0 con 0x563bb04bcc90 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff760857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb0531d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7130 0x563bb04c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff760857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb0531d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36946/0 (socket says 172.21.15.64:36946) 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff760857640 1 -- 172.21.15.64:0/784733857 learned_addr learned my addr 172.21.15.64:0/784733857 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff764fc8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1c1350 con 0x563bb04b7130 2024-05-09T09:20:53.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1c05b0 con 0x563bb04b7b20 2024-05-09T09:20:53.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7b20 0x563bb04c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff764fc8740 10 client.0 _ll_register_callbacks cb 0x7ff754009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75bfff640 1 -- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7b20 msgr2=0x563bb04c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75bfff640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7b20 0x563bb04c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75bfff640 1 -- 172.21.15.64:0/784733857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 msgr2=0x563bb0531d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75bfff640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb0531d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff75bfff640 1 -- 172.21.15.64:0/784733857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb0532280 con 0x563bb04b7130 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.268+0000 7ff760857640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb0531d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.269+0000 7ff75bfff640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7130 0x563bb04c5bb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x563bb04b3710 tx=0x7ff74c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e6279538cab5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.269+0000 7ff73b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.269+0000 7ff73b7fe640 1 -- 172.21.15.64:0/784733857 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c035020 con 0x563bb04b7130 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.269+0000 7ff73b7fe640 1 -- 172.21.15.64:0/784733857 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff74c004040 con 0x563bb04b7130 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.269+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7500b4e50 con 0x563bb04b7130 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.270+0000 7ff73b7fe640 1 -- 172.21.15.64:0/784733857 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff74c054050 con 0x563bb04b7130 2024-05-09T09:20:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.270+0000 7ff764fc8740 10 client.27806 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.270+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563bb0533550 con 0x563bb04b7130 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff73b7fe640 1 -- 172.21.15.64:0/784733857 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff74c044e20 con 0x563bb04b7130 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff73b7fe640 1 -- 172.21.15.64:0/784733857 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7ff74c03a050 con 0x563bb04b7130 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff764fc8740 10 client.27806 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff764fc8740 10 client.27806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff764fc8740 20 client.27806 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff764fc8740 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb0533f80 0x563bb05363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff764fc8740 4 client.27806 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:20:53.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.271+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff7500b4e50 con 0x563bb0533f80 2024-05-09T09:20:53.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.272+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb0533f80 0x563bb05363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.272+0000 7ff75b7fe640 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb0533f80 0x563bb05363a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d63372ea3eb0e53 server_cookie=dade0c346471d5ab in_seq=0 out_seq=0 2024-05-09T09:20:53.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.272+0000 7ff73b7fe640 10 client.27806 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:53.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.272+0000 7ff73b7fe640 1 -- 172.21.15.64:0/784733857 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff7500b4e50 con 0x563bb0533f80 2024-05-09T09:20:53.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.272+0000 7ff73b7fe640 10 client.27806 handle_command_reply: tid=0 2024-05-09T09:20:53.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.273+0000 7ff764fc8740 1 client.27806 shutdown 2024-05-09T09:20:53.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7130 msgr2=0x563bb04c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7130 0x563bb04c5bb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x563bb04b3710 tx=0x7ff74c005fd0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 shutdown_connections 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x563bb0533f80 0x563bb05363a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb04bcc90 0x563bb0531d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bb04b7b20 0x563bb04c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.274+0000 7ff764fc8740 1 --2- 172.21.15.64:0/784733857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bb04b7130 0x563bb04c5bb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.275+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 >> 172.21.15.64:0/784733857 conn(0x563bb04b27a0 msgr2=0x563bb04c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.275+0000 7ff73b7fe640 0 client.27806 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.275+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 shutdown_connections 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.275+0000 7ff764fc8740 1 -- 172.21.15.64:0/784733857 wait complete. 2024-05-09T09:20:53.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.275+0000 7ff764fc8740 20 client.27806 trim_cache size 0 max 16384 2024-05-09T09:20:53.278 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.276+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7440a2c90 msgr2=0x7ff7440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.276+0000 7ff761058640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7440a2c90 0x7ff7440a5150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff754005fd0 tx=0x7ff754009500 comp rx=0 tx=0).stop 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.276+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 msgr2=0x7ff75c110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.276+0000 7ff761058640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 0x7ff75c110ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff75000e9f0 tx=0x7ff75000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 shutdown_connections 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff7440a2c90 0x7ff7440a5150 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff75c101350 0x7ff75c113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff75c1009b0 0x7ff75c110ee0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 --2- 172.21.15.64:0/1842109454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff75c100050 0x7ff75c1109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 >> 172.21.15.64:0/1842109454 conn(0x7ff75c0fbbd0 msgr2=0x7ff75c0fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.277+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 shutdown_connections 2024-05-09T09:20:53.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.278+0000 7ff761058640 1 -- 172.21.15.64:0/1842109454 wait complete. 2024-05-09T09:20:53.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:53 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/352842327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:20:53.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:20:53.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:53 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/352842327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:20:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 -- 172.21.15.64:0/850915087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c108830 msgr2=0x7f613c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 --2- 172.21.15.64:0/850915087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c108830 0x7f613c10ac60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6124009860 tx=0x7f612402a340 comp rx=0 tx=0).stop 2024-05-09T09:20:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 -- 172.21.15.64:0/850915087 shutdown_connections 2024-05-09T09:20:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 --2- 172.21.15.64:0/850915087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c10db70 0x7f613c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 --2- 172.21.15.64:0/850915087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c10b1a0 0x7f613c10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 --2- 172.21.15.64:0/850915087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c108830 0x7f613c10ac60 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6124009860 tx=0x7f612402a340 comp rx=0 tx=0).stop 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 -- 172.21.15.64:0/850915087 >> 172.21.15.64:0/850915087 conn(0x7f613c06e6c0 msgr2=0x7f613c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 -- 172.21.15.64:0/850915087 shutdown_connections 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.492+0000 7f6141570640 1 -- 172.21.15.64:0/850915087 wait complete. 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.493+0000 7f6141570640 1 Processor -- start 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.493+0000 7f6141570640 1 -- start start 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.493+0000 7f6141570640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c10b1a0 0x7f613c1151e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.493+0000 7f6141570640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 0x7f613c115720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f6141570640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c1b48f0 0x7f613c1b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f6141570640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f613c11c210 con 0x7f613c1b48f0 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f6141570640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f613c11c090 con 0x7f613c10b1a0 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f6141570640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f613c11c390 con 0x7f613c10db70 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 0x7f613c115720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 0x7f613c115720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59574/0 (socket says 172.21.15.64:59574) 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613a7fc640 1 -- 172.21.15.64:0/4165297506 learned_addr learned my addr 172.21.15.64:0/4165297506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613affd640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c10b1a0 0x7f613c1151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613b7fe640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c1b48f0 0x7f613c1b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613a7fc640 1 -- 172.21.15.64:0/4165297506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c10b1a0 msgr2=0x7f613c1151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613a7fc640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c10b1a0 0x7f613c1151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.494+0000 7f613a7fc640 1 -- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c1b48f0 msgr2=0x7f613c1b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f613a7fc640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c1b48f0 0x7f613c1b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f613a7fc640 1 -- 172.21.15.64:0/4165297506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c1b7320 con 0x7f613c10db70 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f613b7fe640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c1b48f0 0x7f613c1b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f613a7fc640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 0x7f613c115720 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f612c00eb70 tx=0x7f612c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f6133fff640 1 -- 172.21.15.64:0/4165297506 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c003b70 con 0x7f613c10db70 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c1b7610 con 0x7f613c10db70 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.495+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1b7b60 con 0x7f613c10db70 2024-05-09T09:20:53.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.496+0000 7f6133fff640 1 -- 172.21.15.64:0/4165297506 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f612c00ee70 con 0x7f613c10db70 2024-05-09T09:20:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.496+0000 7f6133fff640 1 -- 172.21.15.64:0/4165297506 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c010690 con 0x7f613c10db70 2024-05-09T09:20:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.500+0000 7f6143f22740 1 Processor -- start 2024-05-09T09:20:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.500+0000 7f6143f22740 1 -- start start 2024-05-09T09:20:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6143f22740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6143f22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6143f22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6143f22740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e29d78c40 con 0x555e29e13f40 2024-05-09T09:20:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6143f22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e29d20580 con 0x555e29e14930 2024-05-09T09:20:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6143f22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555e29d74c00 con 0x555e29e19aa0 2024-05-09T09:20:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f6133fff640 1 -- 172.21.15.64:0/4165297506 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f612c010970 con 0x7f613c10db70 2024-05-09T09:20:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59578/0 (socket says 172.21.15.64:59578) 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.501+0000 7f613a7fc640 1 -- 172.21.15.64:0/830933534 learned_addr learned my addr 172.21.15.64:0/830933534 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613affd640 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613b7fe640 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613a7fc640 1 -- 172.21.15.64:0/830933534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 msgr2=0x555e29e19430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613a7fc640 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e19430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613a7fc640 1 -- 172.21.15.64:0/830933534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 msgr2=0x555e29e14360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613a7fc640 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e14360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613a7fc640 1 -- 172.21.15.64:0/830933534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e29a8c3f0 con 0x555e29e19aa0 2024-05-09T09:20:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.502+0000 7f613affd640 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e19430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.503+0000 7f6133fff640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61280a2da0 0x7f61280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.503+0000 7f613b7fe640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61280a2da0 0x7f61280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.504+0000 7f6133fff640 1 -- 172.21.15.64:0/4165297506 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f612c0cfb50 con 0x7f613c10db70 2024-05-09T09:20:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.504+0000 7f613a7fc640 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1bf30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f612c0d3ae0 tx=0x7f612c0d3bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b10018ba3a9d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.504+0000 7f613b7fe640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61280a2da0 0x7f61280a5260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x555e29e15190 tx=0x7f6134009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.505+0000 7f61317fa640 1 -- 172.21.15.64:0/830933534 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c099130 con 0x555e29e19aa0 2024-05-09T09:20:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.505+0000 7f61317fa640 1 -- 172.21.15.64:0/830933534 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f612c09c010 con 0x555e29e19aa0 2024-05-09T09:20:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f61317fa640 1 -- 172.21.15.64:0/830933534 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c09be30 con 0x555e29e19aa0 2024-05-09T09:20:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 -- 172.21.15.64:0/830933534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 msgr2=0x555e29e1bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1bf30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f612c0d3ae0 tx=0x7f612c0d3bd0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 -- 172.21.15.64:0/830933534 shutdown_connections 2024-05-09T09:20:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1bf30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e19430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 --2- 172.21.15.64:0/830933534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 -- 172.21.15.64:0/830933534 >> 172.21.15.64:0/830933534 conn(0x555e29e0f5b0 msgr2=0x555e29e0f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.506+0000 7f6143f22740 1 -- 172.21.15.64:0/830933534 shutdown_connections 2024-05-09T09:20:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.507+0000 7f6143f22740 1 -- 172.21.15.64:0/830933534 wait complete. 2024-05-09T09:20:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.507+0000 7f6143f22740 1 -- start start 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.507+0000 7f6143f22740 1 Processor -- start 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.507+0000 7f6143f22740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f6143f22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f6143f22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f6143f22740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e29e1ca40 con 0x555e29e13f40 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f6143f22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e29d33c40 con 0x555e29e19aa0 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f6143f22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555e29e1cbc0 con 0x555e29e14930 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f6143f22740 10 client.0 _ll_register_callbacks cb 0x555e29a8bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f613affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f613affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36964/0 (socket says 172.21.15.64:36964) 2024-05-09T09:20:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.508+0000 7f613affd640 1 -- 172.21.15.64:0/267736191 learned_addr learned my addr 172.21.15.64:0/267736191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f613affd640 1 -- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 msgr2=0x555e29e1e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f613affd640 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f613affd640 1 -- 172.21.15.64:0/267736191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 msgr2=0x555e29e259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f613affd640 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f613affd640 1 -- 172.21.15.64:0/267736191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e29b567d0 con 0x555e29e13f40 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f613affd640 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6124009860 tx=0x7f6124030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1169b26e1edcf65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.509+0000 7f61227fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.510+0000 7f61227fc640 1 -- 172.21.15.64:0/267736191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124037070 con 0x555e29e13f40 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.510+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612c09c010 con 0x555e29e13f40 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.510+0000 7f61227fc640 1 -- 172.21.15.64:0/267736191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6124004230 con 0x555e29e13f40 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.510+0000 7f61227fc640 1 -- 172.21.15.64:0/267736191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6124049050 con 0x555e29e13f40 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.510+0000 7f6143f22740 10 client.18357 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.510+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555e29e28d30 con 0x555e29e13f40 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f61227fc640 1 -- 172.21.15.64:0/267736191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f612c09c010 con 0x555e29e13f40 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f61227fc640 1 -- 172.21.15.64:0/267736191 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f6124044030 con 0x555e29e13f40 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f6143f22740 10 client.18357 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f6143f22740 10 client.18357 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f6143f22740 20 client.18357 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f6143f22740 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e2a960 0x555e29e2ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f6143f22740 4 client.18357 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.511+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e29d86370 con 0x555e29e2a960 2024-05-09T09:20:53.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.512+0000 7f613a7fc640 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e2a960 0x555e29e2ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.512+0000 7f61227fc640 10 client.18357 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:53.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.513+0000 7f613a7fc640 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e2a960 0x555e29e2ce20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=261aa8bea80eecf3 server_cookie=a743901c6c48f680 in_seq=0 out_seq=0 2024-05-09T09:20:53.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.514+0000 7f61227fc640 1 -- 172.21.15.64:0/267736191 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e29d86370 con 0x555e29e2a960 2024-05-09T09:20:53.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.514+0000 7f61227fc640 10 client.18357 handle_command_reply: tid=0 2024-05-09T09:20:53.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.514+0000 7f6143f22740 1 client.18357 shutdown 2024-05-09T09:20:53.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.514+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 msgr2=0x555e29e1e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.514+0000 7f6143f22740 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6124009860 tx=0x7f6124030ce0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 shutdown_connections 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f6143f22740 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f6143f22740 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e1e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f6143f22740 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e2a960 0x555e29e2ce20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f6143f22740 1 --2- 172.21.15.64:0/267736191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e430 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f61227fc640 0 client.18357 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:53.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.515+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 >> 172.21.15.64:0/267736191 conn(0x555e29e0f5b0 msgr2=0x555e29e0e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.516+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 shutdown_connections 2024-05-09T09:20:53.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.516+0000 7f6143f22740 1 -- 172.21.15.64:0/267736191 wait complete. 2024-05-09T09:20:53.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.516+0000 7f6143f22740 20 client.18357 trim_cache size 0 max 16384 2024-05-09T09:20:53.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.543+0000 7f6143f22740 1 Processor -- start 2024-05-09T09:20:53.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.544+0000 7f6143f22740 1 -- start start 2024-05-09T09:20:53.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.544+0000 7f6143f22740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.544+0000 7f6143f22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.544+0000 7f6143f22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f6143f22740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f611c0125d0 con 0x555e29e13f40 2024-05-09T09:20:53.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f6143f22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f611c00fe70 con 0x555e29e14930 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59600/0 (socket says 172.21.15.64:59600) 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 -- 172.21.15.64:0/660936921 learned_addr learned my addr 172.21.15.64:0/660936921 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f6143f22740 1 -- 172.21.15.64:0/660936921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f611c00dcc0 con 0x555e29e19aa0 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613a7fc640 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 -- 172.21.15.64:0/660936921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 msgr2=0x555e29e25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 -- 172.21.15.64:0/660936921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 msgr2=0x555e29e1e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613b7fe640 1 -- 172.21.15.64:0/660936921 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e29e0f150 con 0x555e29e19aa0 2024-05-09T09:20:53.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.545+0000 7f613a7fc640 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:53.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.546+0000 7f613affd640 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:53.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.546+0000 7f613b7fe640 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1e580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6134005f70 tx=0x7f6134014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ecc9e159c42781 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.546+0000 7f6130ff9640 1 -- 172.21.15.64:0/660936921 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613402a070 con 0x555e29e19aa0 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.546+0000 7f6130ff9640 1 -- 172.21.15.64:0/660936921 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f613401dd10 con 0x555e29e19aa0 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.546+0000 7f6130ff9640 1 -- 172.21.15.64:0/660936921 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134025800 con 0x555e29e19aa0 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.547+0000 7f6143f22740 1 -- 172.21.15.64:0/660936921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 msgr2=0x555e29e1e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.547+0000 7f6143f22740 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1e580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6134005f70 tx=0x7f6134014040 comp rx=0 tx=0).stop 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 -- 172.21.15.64:0/660936921 shutdown_connections 2024-05-09T09:20:53.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e19aa0 0x555e29e1e580 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e14930 0x555e29e25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 --2- 172.21.15.64:0/660936921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 -- 172.21.15.64:0/660936921 >> 172.21.15.64:0/660936921 conn(0x555e29e0f5b0 msgr2=0x555e29e0f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 -- 172.21.15.64:0/660936921 shutdown_connections 2024-05-09T09:20:53.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 -- 172.21.15.64:0/660936921 wait complete. 2024-05-09T09:20:53.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.548+0000 7f6143f22740 1 -- start start 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.549+0000 7f6143f22740 1 Processor -- start 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.549+0000 7f6143f22740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e22700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f6143f22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f613affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e22700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f6143f22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e8ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f613affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e22700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36982/0 (socket says 172.21.15.64:36982) 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e22c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59606/0 (socket says 172.21.15.64:59606) 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f613affd640 1 -- 172.21.15.64:0/1524648969 learned_addr learned my addr 172.21.15.64:0/1524648969 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f611c00e330 con 0x555e29e13f40 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f611c00f7a0 con 0x555e29e19aa0 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f613b7fe640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e8ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f611c00ea00 con 0x555e29e14930 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.550+0000 7f6143f22740 10 client.0 _ll_register_callbacks cb 0x7f6124035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613a7fc640 1 -- 172.21.15.64:0/1524648969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 msgr2=0x555e29e8ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613a7fc640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e8ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613a7fc640 1 -- 172.21.15.64:0/1524648969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 msgr2=0x555e29e22700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613a7fc640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e22700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613a7fc640 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e29e232e0 con 0x555e29e14930 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613b7fe640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e8ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f613a7fc640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e22c40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x555e29e1f910 tx=0x7f612c00ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f3a45b058b7f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f61227fc640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f61227fc640 1 -- 172.21.15.64:0/1524648969 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c09be30 con 0x555e29e14930 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f61227fc640 1 -- 172.21.15.64:0/1524648969 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f612c099500 con 0x555e29e14930 2024-05-09T09:20:53.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.551+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f613401dd10 con 0x555e29e14930 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f61227fc640 1 -- 172.21.15.64:0/1524648969 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f612c099130 con 0x555e29e14930 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f6143f22740 10 client.27994 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555e29e905c0 con 0x555e29e14930 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f61227fc640 1 -- 172.21.15.64:0/1524648969 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f613401dd10 con 0x555e29e14930 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f61227fc640 1 -- 172.21.15.64:0/1524648969 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f612c09c6f0 con 0x555e29e14930 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f6143f22740 10 client.27994 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:53.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f6143f22740 10 client.27994 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.552+0000 7f6143f22740 20 client.27994 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.553+0000 7f6143f22740 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e90b10 0x555e29e92f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.553+0000 7f6143f22740 4 client.27994 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.553+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e29d806c0 con 0x555e29e90b10 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.553+0000 7f613affd640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e90b10 0x555e29e92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.554+0000 7f613affd640 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e90b10 0x555e29e92f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef6d53ca27973f server_cookie=a2b932174e2cf3c0 in_seq=0 out_seq=0 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.554+0000 7f61227fc640 10 client.27994 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.554+0000 7f61227fc640 1 -- 172.21.15.64:0/1524648969 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e29d806c0 con 0x555e29e90b10 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.554+0000 7f61227fc640 10 client.27994 handle_command_reply: tid=0 2024-05-09T09:20:53.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.554+0000 7f6143f22740 1 client.27994 shutdown 2024-05-09T09:20:53.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 msgr2=0x555e29e22c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e22c40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x555e29e1f910 tx=0x7f612c00ced0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 shutdown_connections 2024-05-09T09:20:53.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e29e19aa0 0x555e29e8ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e29e14930 0x555e29e22c40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555e29e90b10 0x555e29e92f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 --2- 172.21.15.64:0/1524648969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e29e13f40 0x555e29e22700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 >> 172.21.15.64:0/1524648969 conn(0x555e29e0f5b0 msgr2=0x555e29e1e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.555+0000 7f61227fc640 0 client.27994 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.556+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 shutdown_connections 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.556+0000 7f6143f22740 1 -- 172.21.15.64:0/1524648969 wait complete. 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.556+0000 7f6143f22740 20 client.27994 trim_cache size 0 max 16384 2024-05-09T09:20:53.558 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:20:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61280a2da0 msgr2=0x7f61280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61280a2da0 0x7f61280a5260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x555e29e15190 tx=0x7f6134009260 comp rx=0 tx=0).stop 2024-05-09T09:20:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 msgr2=0x7f613c115720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 0x7f613c115720 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f612c00eb70 tx=0x7f612c00c6a0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 shutdown_connections 2024-05-09T09:20:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61280a2da0 0x7f61280a5260 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c1b48f0 0x7f613c1b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f613c10db70 0x7f613c115720 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 --2- 172.21.15.64:0/4165297506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f613c10b1a0 0x7f613c1151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 >> 172.21.15.64:0/4165297506 conn(0x7f613c06e6c0 msgr2=0x7f613c10b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.557+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 shutdown_connections 2024-05-09T09:20:53.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.558+0000 7f6141570640 1 -- 172.21.15.64:0/4165297506 wait complete. 2024-05-09T09:20:53.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:53 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/352842327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:20:53.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:20:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.761+0000 7f735759e640 1 -- 172.21.15.64:0/3618638529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 msgr2=0x7f735009c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.761+0000 7f735759e640 1 --2- 172.21.15.64:0/3618638529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f735009c790 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7348009900 tx=0x7f734802a340 comp rx=0 tx=0).stop 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.761+0000 7f735759e640 1 -- 172.21.15.64:0/3618638529 shutdown_connections 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.761+0000 7f735759e640 1 --2- 172.21.15.64:0/3618638529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 0x7f73500a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.761+0000 7f735759e640 1 --2- 172.21.15.64:0/3618638529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 0x7f735009d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.761+0000 7f735759e640 1 --2- 172.21.15.64:0/3618638529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f735009c790 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.762+0000 7f735759e640 1 -- 172.21.15.64:0/3618638529 >> 172.21.15.64:0/3618638529 conn(0x7f735000b390 msgr2=0x7f735000b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.762+0000 7f735759e640 1 -- 172.21.15.64:0/3618638529 shutdown_connections 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.762+0000 7f735759e640 1 -- 172.21.15.64:0/3618638529 wait complete. 2024-05-09T09:20:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.763+0000 7f735759e640 1 Processor -- start 2024-05-09T09:20:53.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 -- start start 2024-05-09T09:20:53.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f73501325e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 0x7f7350132b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 0x7f7350139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73500a7ed0 con 0x7f735009c370 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73500a7d50 con 0x7f735009d560 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735759e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73500a8050 con 0x7f735009df20 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f73501325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f73501325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36992/0 (socket says 172.21.15.64:36992) 2024-05-09T09:20:53.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.764+0000 7f735659c640 1 -- 172.21.15.64:0/1218833360 learned_addr learned my addr 172.21.15.64:0/1218833360 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 0x7f7350132b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 0x7f7350139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 -- 172.21.15.64:0/1218833360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 msgr2=0x7f7350139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 0x7f7350139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 -- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 msgr2=0x7f73501325e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f73501325e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 -- 172.21.15.64:0/1218833360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735013a200 con 0x7f735009d560 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 0x7f7350139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f7355d9b640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 0x7f7350132b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f734c00e9f0 tx=0x7f734c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f73477fe640 1 -- 172.21.15.64:0/1218833360 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c00ccc0 con 0x7f735009d560 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f73477fe640 1 -- 172.21.15.64:0/1218833360 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f734c0108e0 con 0x7f735009d560 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.765+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735013a4f0 con 0x7f735009d560 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.766+0000 7f73477fe640 1 -- 172.21.15.64:0/1218833360 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c018750 con 0x7f735009d560 2024-05-09T09:20:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.766+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735013aa90 con 0x7f735009d560 2024-05-09T09:20:53.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.770+0000 7f735fb8c740 1 Processor -- start 2024-05-09T09:20:53.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.770+0000 7f735fb8c740 1 -- start start 2024-05-09T09:20:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 0x55b94de45550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 0x55b94de4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de4ac90 0x55b94de4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735fb8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b94dda9c40 con 0x55b94de4ac90 2024-05-09T09:20:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735fb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b94dd51580 con 0x55b94de45b20 2024-05-09T09:20:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735fb8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b94dda5c00 con 0x55b94de45130 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 0x55b94de4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 0x55b94de4a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50150/0 (socket says 172.21.15.64:50150) 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f735659c640 1 -- 172.21.15.64:0/90761736 learned_addr learned my addr 172.21.15.64:0/90761736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f7355d9b640 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de4ac90 0x55b94de4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.771+0000 7f73477fe640 1 -- 172.21.15.64:0/1218833360 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f734c010a90 con 0x7f735009d560 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.772+0000 7f735659c640 1 -- 172.21.15.64:0/90761736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 msgr2=0x55b94de45550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.772+0000 7f735659c640 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 0x55b94de45550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.772+0000 7f735659c640 1 -- 172.21.15.64:0/90761736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de4ac90 msgr2=0x55b94de4d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.772+0000 7f735659c640 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de4ac90 0x55b94de4d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.772+0000 7f735659c640 1 -- 172.21.15.64:0/90761736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94dabd3f0 con 0x55b94de45b20 2024-05-09T09:20:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.772+0000 7f735659c640 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 0x55b94de4a620 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7348034040 tx=0x7f7348005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f866d1e06eac1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.773+0000 7f7344ff9640 1 -- 172.21.15.64:0/90761736 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348043070 con 0x55b94de45b20 2024-05-09T09:20:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.773+0000 7f7344ff9640 1 -- 172.21.15.64:0/90761736 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7348032640 con 0x55b94de45b20 2024-05-09T09:20:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.773+0000 7f7344ff9640 1 -- 172.21.15.64:0/90761736 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348003850 con 0x55b94de45b20 2024-05-09T09:20:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.774+0000 7f73477fe640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73400a2c90 0x7f73400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.774+0000 7f735fb8c740 1 -- 172.21.15.64:0/90761736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 msgr2=0x55b94de4a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.774+0000 7f735fb8c740 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 0x55b94de4a620 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7348034040 tx=0x7f7348005a20 comp rx=0 tx=0).stop 2024-05-09T09:20:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.774+0000 7f73477fe640 1 -- 172.21.15.64:0/1218833360 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f734c0cf630 con 0x7f735009d560 2024-05-09T09:20:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73400a2c90 0x7f73400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 -- 172.21.15.64:0/90761736 shutdown_connections 2024-05-09T09:20:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de4ac90 0x55b94de4d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45b20 0x55b94de4a620 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 --2- 172.21.15.64:0/90761736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 0x55b94de45550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 -- 172.21.15.64:0/90761736 >> 172.21.15.64:0/90761736 conn(0x55b94de407a0 msgr2=0x55b94de40ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 -- 172.21.15.64:0/90761736 shutdown_connections 2024-05-09T09:20:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.775+0000 7f735fb8c740 1 -- 172.21.15.64:0/90761736 wait complete. 2024-05-09T09:20:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.776+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73400a2c90 0x7f73400a5150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f735806b220 tx=0x7f7358050560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.776+0000 7f735fb8c740 1 -- start start 2024-05-09T09:20:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.776+0000 7f735fb8c740 1 Processor -- start 2024-05-09T09:20:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b94de4dbe0 con 0x55b94de45b20 2024-05-09T09:20:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b94dd64c40 con 0x55b94de45130 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b94de4dd60 con 0x55b94de4ac90 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f7356d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.777+0000 7f735fb8c740 10 client.0 _ll_register_callbacks cb 0x55b94dabcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f7356d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:59632/0 (socket says 172.21.15.64:59632) 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f7356d9d640 1 -- 172.21.15.64:0/4188433562 learned_addr learned my addr 172.21.15.64:0/4188433562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f7355d9b640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f735659c640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f735659c640 1 -- 172.21.15.64:0/4188433562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 msgr2=0x55b94de568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f735659c640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f735659c640 1 -- 172.21.15.64:0/4188433562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 msgr2=0x55b94de4f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f735659c640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de4f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f735659c640 1 -- 172.21.15.64:0/4188433562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94db877d0 con 0x55b94de45130 2024-05-09T09:20:53.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.778+0000 7f7355d9b640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de4f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:53.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.779+0000 7f735659c640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de4f320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55b94de41710 tx=0x7f7348032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b32c7d998f3130a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.779+0000 7f733dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.779+0000 7f733dffb640 1 -- 172.21.15.64:0/4188433562 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734804e070 con 0x55b94de45130 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.779+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7348032640 con 0x55b94de45130 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.779+0000 7f733dffb640 1 -- 172.21.15.64:0/4188433562 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7348003800 con 0x55b94de45130 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.780+0000 7f733dffb640 1 -- 172.21.15.64:0/4188433562 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7348059050 con 0x55b94de45130 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.780+0000 7f735fb8c740 10 client.27827 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:53.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.780+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b94de59c10 con 0x55b94de45130 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f733dffb640 1 -- 172.21.15.64:0/4188433562 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7348048450 con 0x55b94de45130 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f733dffb640 1 -- 172.21.15.64:0/4188433562 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f734803d440 con 0x55b94de45130 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f735fb8c740 10 client.27827 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f735fb8c740 10 client.27827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f735fb8c740 20 client.27827 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f735fb8c740 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94de5af60 0x55b94de5d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f735fb8c740 4 client.27827 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:53.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7348032640 con 0x55b94de5af60 2024-05-09T09:20:53.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.781+0000 7f7355d9b640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94de5af60 0x55b94de5d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.782+0000 7f7355d9b640 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94de5af60 0x55b94de5d350 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d924d2709c6fae9 server_cookie=2a1f49b22fc4a28 in_seq=0 out_seq=0 2024-05-09T09:20:53.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.782+0000 7f733dffb640 10 client.27827 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:53.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.783+0000 7f733dffb640 1 -- 172.21.15.64:0/4188433562 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7348032640 con 0x55b94de5af60 2024-05-09T09:20:53.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.783+0000 7f733dffb640 10 client.27827 handle_command_reply: tid=0 2024-05-09T09:20:53.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.783+0000 7f735fb8c740 1 client.27827 shutdown 2024-05-09T09:20:53.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.783+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 msgr2=0x55b94de4f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.783+0000 7f735fb8c740 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de4f320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55b94de41710 tx=0x7f7348032b90 comp rx=0 tx=0).stop 2024-05-09T09:20:53.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 shutdown_connections 2024-05-09T09:20:53.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f735fb8c740 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94de5af60 0x55b94de5d350 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f735fb8c740 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f735fb8c740 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de4f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f735fb8c740 1 --2- 172.21.15.64:0/4188433562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de4f320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 >> 172.21.15.64:0/4188433562 conn(0x55b94de407a0 msgr2=0x55b94de3f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.784+0000 7f733dffb640 0 client.27827 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:53.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.785+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 shutdown_connections 2024-05-09T09:20:53.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.785+0000 7f735fb8c740 1 -- 172.21.15.64:0/4188433562 wait complete. 2024-05-09T09:20:53.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.785+0000 7f735fb8c740 20 client.27827 trim_cache size 0 max 16384 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.811+0000 7f735fb8c740 1 Processor -- start 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 -- start start 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de43f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de56860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de61ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7350147790 con 0x55b94de45b20 2024-05-09T09:20:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7350145030 con 0x55b94de45130 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735fb8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7350142e80 con 0x55b94de4ac90 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de43f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f7355d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de56860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de43f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50166/0 (socket says 172.21.15.64:50166) 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f735659c640 1 -- 172.21.15.64:0/3675309225 learned_addr learned my addr 172.21.15.64:0/3675309225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.812+0000 7f7356d9d640 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de61ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f735659c640 1 -- 172.21.15.64:0/3675309225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 msgr2=0x55b94de61ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f735659c640 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de61ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f735659c640 1 -- 172.21.15.64:0/3675309225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 msgr2=0x55b94de56860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f735659c640 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de56860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f735659c640 1 -- 172.21.15.64:0/3675309225 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94de4d4e0 con 0x55b94de45130 2024-05-09T09:20:53.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f735659c640 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de43f40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7348009230 tx=0x7f7348005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf4e8591bb47af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f733ffff640 1 -- 172.21.15.64:0/3675309225 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348056070 con 0x55b94de45130 2024-05-09T09:20:53.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.813+0000 7f733ffff640 1 -- 172.21.15.64:0/3675309225 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7348005850 con 0x55b94de45130 2024-05-09T09:20:53.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.814+0000 7f733ffff640 1 -- 172.21.15.64:0/3675309225 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734804e040 con 0x55b94de45130 2024-05-09T09:20:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.815+0000 7f735fb8c740 1 -- 172.21.15.64:0/3675309225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 msgr2=0x55b94de43f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.815+0000 7f735fb8c740 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de43f40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7348009230 tx=0x7f7348005c90 comp rx=0 tx=0).stop 2024-05-09T09:20:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 -- 172.21.15.64:0/3675309225 shutdown_connections 2024-05-09T09:20:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de4ac90 0x55b94de61ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de56860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 --2- 172.21.15.64:0/3675309225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de45130 0x55b94de43f40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 -- 172.21.15.64:0/3675309225 >> 172.21.15.64:0/3675309225 conn(0x55b94de407a0 msgr2=0x55b94de43830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 -- 172.21.15.64:0/3675309225 shutdown_connections 2024-05-09T09:20:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 -- 172.21.15.64:0/3675309225 wait complete. 2024-05-09T09:20:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.816+0000 7f735fb8c740 1 -- start start 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 Processor -- start 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 0x55b94de566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 0x55b94de536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73501434f0 con 0x55b94de45b20 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7350144960 con 0x55b94de4ac90 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7350143bc0 con 0x55b94de45130 2024-05-09T09:20:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f735fb8c740 10 client.0 _ll_register_callbacks cb 0x7f734c0d5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f7356d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 0x55b94de536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f7356d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 0x55b94de536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:50182/0 (socket says 172.21.15.64:50182) 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.817+0000 7f7356d9d640 1 -- 172.21.15.64:0/1216441356 learned_addr learned my addr 172.21.15.64:0/1216441356 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7355d9b640 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7356d9d640 1 -- 172.21.15.64:0/1216441356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 msgr2=0x55b94de566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 0x55b94de566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7356d9d640 1 -- 172.21.15.64:0/1216441356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 msgr2=0x55b94de53170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de53170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7356d9d640 1 -- 172.21.15.64:0/1216441356 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94debf410 con 0x55b94de4ac90 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f7356d9d640 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 0x55b94de536b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7358067530 tx=0x7f7358067600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d06d43ec2a1b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.818+0000 7f733dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.819+0000 7f733dffb640 1 -- 172.21.15.64:0/1216441356 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7358095060 con 0x55b94de4ac90 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.819+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7348005850 con 0x55b94de4ac90 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.819+0000 7f733dffb640 1 -- 172.21.15.64:0/1216441356 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7358080b30 con 0x55b94de4ac90 2024-05-09T09:20:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.819+0000 7f733dffb640 1 -- 172.21.15.64:0/1216441356 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f735809c050 con 0x55b94de4ac90 2024-05-09T09:20:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.820+0000 7f735fb8c740 10 client.27833 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.820+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b94dec0740 con 0x55b94de4ac90 2024-05-09T09:20:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f733dffb640 1 -- 172.21.15.64:0/1216441356 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7348005850 con 0x55b94de4ac90 2024-05-09T09:20:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f733dffb640 1 -- 172.21.15.64:0/1216441356 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f735808f070 con 0x55b94de4ac90 2024-05-09T09:20:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735fb8c740 10 client.27833 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735fb8c740 10 client.27833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735fb8c740 20 client.27833 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735fb8c740 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94dec10a0 0x55b94dec34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735fb8c740 4 client.27833 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b94ddb7370 con 0x55b94dec10a0 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.821+0000 7f735659c640 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94dec10a0 0x55b94dec34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.822+0000 7f735659c640 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94dec10a0 0x55b94dec34c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1319db4bc0cc08 server_cookie=b8c690588545cdce in_seq=0 out_seq=0 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.822+0000 7f733dffb640 10 client.27833 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.822+0000 7f733dffb640 1 -- 172.21.15.64:0/1216441356 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b94ddb7370 con 0x55b94dec10a0 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.822+0000 7f733dffb640 10 client.27833 handle_command_reply: tid=0 2024-05-09T09:20:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.822+0000 7f735fb8c740 1 client.27833 shutdown 2024-05-09T09:20:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.824+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 msgr2=0x55b94de536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.824+0000 7f735fb8c740 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 0x55b94de536b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7358067530 tx=0x7f7358067600 comp rx=0 tx=0).stop 2024-05-09T09:20:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 shutdown_connections 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f735fb8c740 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b94dec10a0 0x55b94dec34c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f735fb8c740 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94de4ac90 0x55b94de536b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f733dffb640 0 client.27833 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f735fb8c740 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b94de45b20 0x55b94de53170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f735fb8c740 1 --2- 172.21.15.64:0/1216441356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b94de45130 0x55b94de566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.825+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 >> 172.21.15.64:0/1216441356 conn(0x55b94de407a0 msgr2=0x55b94de3f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.826+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 shutdown_connections 2024-05-09T09:20:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.827+0000 7f735fb8c740 1 -- 172.21.15.64:0/1216441356 wait complete. 2024-05-09T09:20:53.828 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:20:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.828+0000 7f735fb8c740 20 client.27833 trim_cache size 0 max 16384 2024-05-09T09:20:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73400a2c90 msgr2=0x7f73400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73400a2c90 0x7f73400a5150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f735806b220 tx=0x7f7358050560 comp rx=0 tx=0).stop 2024-05-09T09:20:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 msgr2=0x7f7350132b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 0x7f7350132b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f734c00e9f0 tx=0x7f734c00edf0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 shutdown_connections 2024-05-09T09:20:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f73400a2c90 0x7f73400a5150 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735009df20 0x7f7350139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f735009d560 0x7f7350132b20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 --2- 172.21.15.64:0/1218833360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735009c370 0x7f73501325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.829+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 >> 172.21.15.64:0/1218833360 conn(0x7f735000b390 msgr2=0x7f73500a08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.833+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 shutdown_connections 2024-05-09T09:20:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:53.833+0000 7f735759e640 1 -- 172.21.15.64:0/1218833360 wait complete. 2024-05-09T09:20:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:54 smithi179 ceph-mon[34086]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 762 op/s 2024-05-09T09:20:54.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:54 smithi139 ceph-mon[33599]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 762 op/s 2024-05-09T09:20:54.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:54 smithi064 ceph-mon[30149]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 762 op/s 2024-05-09T09:20:54.856 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:20:54.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:20:54.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:20:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.058+0000 7f0d006a5640 1 -- 172.21.15.64:0/3376252856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf80761f0 msgr2=0x7f0cf8076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.059+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3376252856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf80761f0 0x7f0cf8076610 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009900 tx=0x7f0ce802a420 comp rx=0 tx=0).stop 2024-05-09T09:20:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.060+0000 7f0d006a5640 1 -- 172.21.15.64:0/3376252856 shutdown_connections 2024-05-09T09:20:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.060+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3376252856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cf8077dd0 0x7f0cf8078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.060+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3376252856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf8077430 0x7f0cf8077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.060+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3376252856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf80761f0 0x7f0cf8076610 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.060+0000 7f0d006a5640 1 -- 172.21.15.64:0/3376252856 >> 172.21.15.64:0/3376252856 conn(0x7f0cf80ff020 msgr2=0x7f0cf8101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.062+0000 7f0d006a5640 1 -- 172.21.15.64:0/3376252856 shutdown_connections 2024-05-09T09:20:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.062+0000 7f0d006a5640 1 -- 172.21.15.64:0/3376252856 wait complete. 2024-05-09T09:20:55.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.063+0000 7f0d006a5640 1 Processor -- start 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.063+0000 7f0d006a5640 1 -- start start 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0d006a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cf80761f0 0x7f0cf819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0d006a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 0x7f0cf819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0d006a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf8077dd0 0x7f0cf81a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0d006a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81139d0 con 0x7f0cf8077430 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0d006a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8113850 con 0x7f0cf80761f0 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0d006a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8113b50 con 0x7f0cf8077dd0 2024-05-09T09:20:55.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0cfdc40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 0x7f0cf819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0cfdc40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 0x7f0cf819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46068/0 (socket says 172.21.15.64:46068) 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0cfdc40640 1 -- 172.21.15.64:0/3871838200 learned_addr learned my addr 172.21.15.64:0/3871838200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.064+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf8077dd0 0x7f0cf81a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.065+0000 7f0cfdc40640 1 -- 172.21.15.64:0/3871838200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf8077dd0 msgr2=0x7f0cf81a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.065+0000 7f0cfdc40640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf8077dd0 0x7f0cf81a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.065+0000 7f0cfdc40640 1 -- 172.21.15.64:0/3871838200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cf80761f0 msgr2=0x7f0cf819e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.065+0000 7f0cfdc40640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cf80761f0 0x7f0cf819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.065+0000 7f0cfdc40640 1 -- 172.21.15.64:0/3871838200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a5d30 con 0x7f0cf8077430 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.066+0000 7f0cfdc40640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 0x7f0cf819e650 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400c890 tx=0x7f0cf400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.066+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3871838200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004220 con 0x7f0cf8077430 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.067+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3871838200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf40043c0 con 0x7f0cf8077430 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.067+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a6020 con 0x7f0cf8077430 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.067+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3871838200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004a00 con 0x7f0cf8077430 2024-05-09T09:20:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.067+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81a6590 con 0x7f0cf8077430 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 Processor -- start 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 -- start start 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce57b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57bec0 0x55e0ce5809c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce5834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0ce4ce990 con 0x55e0ce57bec0 2024-05-09T09:20:55.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e0ce4765b0 con 0x55e0ce581030 2024-05-09T09:20:55.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.072+0000 7f0d033b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e0ce4ca950 con 0x55e0ce57b4d0 2024-05-09T09:20:55.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.073+0000 7f0cfec42640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce57b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.073+0000 7f0cfec42640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce57b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48490/0 (socket says 172.21.15.64:48490) 2024-05-09T09:20:55.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.073+0000 7f0cfec42640 1 -- 172.21.15.64:0/449959988 learned_addr learned my addr 172.21.15.64:0/449959988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cfec42640 1 -- 172.21.15.64:0/449959988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 msgr2=0x55e0ce5834c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cfec42640 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce5834c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cfec42640 1 -- 172.21.15.64:0/449959988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57bec0 msgr2=0x55e0ce5809c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cfec42640 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57bec0 0x55e0ce5809c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cfec42640 1 -- 172.21.15.64:0/449959988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0ce2aa7e0 con 0x55e0ce57b4d0 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3871838200 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0cf40079b0 con 0x7f0cf8077430 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.074+0000 7f0cfec42640 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce57b8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009fa0 tx=0x7f0cf000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb575069d5a3d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.075+0000 7f0cef7fe640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ce40a2c60 0x7f0ce40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.075+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3871838200 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0cf40cf360 con 0x7f0cf8077430 2024-05-09T09:20:55.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.076+0000 7f0cfe441640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ce40a2c60 0x7f0ce40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.077+0000 7f0cecff9640 1 -- 172.21.15.64:0/449959988 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0015070 con 0x55e0ce57b4d0 2024-05-09T09:20:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.077+0000 7f0cecff9640 1 -- 172.21.15.64:0/449959988 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf0011bb0 con 0x55e0ce57b4d0 2024-05-09T09:20:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.077+0000 7f0cecff9640 1 -- 172.21.15.64:0/449959988 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0010c20 con 0x55e0ce57b4d0 2024-05-09T09:20:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 -- 172.21.15.64:0/449959988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 msgr2=0x55e0ce57b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce57b8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009fa0 tx=0x7f0cf000f7f0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0cfe441640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ce40a2c60 0x7f0ce40a5120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005ab0 tx=0x7f0ce8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 -- 172.21.15.64:0/449959988 shutdown_connections 2024-05-09T09:20:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce5834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57bec0 0x55e0ce5809c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 --2- 172.21.15.64:0/449959988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce57b8f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 -- 172.21.15.64:0/449959988 >> 172.21.15.64:0/449959988 conn(0x55e0ce5320a0 msgr2=0x55e0ce565f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 -- 172.21.15.64:0/449959988 shutdown_connections 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.079+0000 7f0d033b1740 1 -- 172.21.15.64:0/449959988 wait complete. 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.080+0000 7f0d033b1740 1 -- start start 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.080+0000 7f0d033b1740 1 Processor -- start 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.080+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce585860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.080+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce585da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.080+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce58cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0d033b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0ce584030 con 0x55e0ce581030 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0d033b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e0ce489b40 con 0x55e0ce57bec0 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0d033b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e0ce5841b0 con 0x55e0ce57b4d0 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0d033b1740 10 client.0 _ll_register_callbacks cb 0x55e0ce47e0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfdc40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce585da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfdc40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce585da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34168/0 (socket says 172.21.15.64:34168) 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfdc40640 1 -- 172.21.15.64:0/373622112 learned_addr learned my addr 172.21.15.64:0/373622112 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfec42640 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce58cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfec42640 1 -- 172.21.15.64:0/373622112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 msgr2=0x55e0ce585860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfec42640 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce585860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfec42640 1 -- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 msgr2=0x55e0ce585da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.081+0000 7f0cfec42640 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce585da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.082+0000 7f0cfec42640 1 -- 172.21.15.64:0/373622112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0ce30af70 con 0x55e0ce581030 2024-05-09T09:20:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.082+0000 7f0cfec42640 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce58cde0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009320 tx=0x7f0cf0004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed8431954da35a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.082+0000 7f0ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.083+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/373622112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0015070 con 0x55e0ce581030 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.083+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0011bb0 con 0x55e0ce581030 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.083+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/373622112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0cf00109a0 con 0x55e0ce581030 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.083+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/373622112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0cf0010ba0 con 0x55e0ce581030 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.085+0000 7f0d033b1740 10 client.18390 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.085+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e0ce1a2170 con 0x55e0ce581030 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.085+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/373622112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf0003e20 con 0x55e0ce581030 2024-05-09T09:20:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.085+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/373622112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f0cf0004030 con 0x55e0ce581030 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.087+0000 7f0d033b1740 10 client.18390 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.087+0000 7f0d033b1740 10 client.18390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.088+0000 7f0d033b1740 20 client.18390 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.088+0000 7f0d033b1740 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce590f90 0x55e0ce5933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.088+0000 7f0d033b1740 4 client.18390 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.088+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0cf0011bb0 con 0x55e0ce590f90 2024-05-09T09:20:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.089+0000 7f0cfe441640 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce590f90 0x55e0ce5933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.090+0000 7f0cfe441640 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce590f90 0x55e0ce5933a0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fc1252b641340e server_cookie=911cf1c1421646d4 in_seq=0 out_seq=0 2024-05-09T09:20:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.090+0000 7f0ce1ffb640 10 client.18390 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.090+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/373622112 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0cf0011bb0 con 0x55e0ce590f90 2024-05-09T09:20:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.091+0000 7f0ce1ffb640 10 client.18390 handle_command_reply: tid=0 2024-05-09T09:20:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.091+0000 7f0d033b1740 1 client.18390 shutdown 2024-05-09T09:20:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.091+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 msgr2=0x55e0ce58cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.091+0000 7f0d033b1740 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce58cde0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009320 tx=0x7f0cf0004c50 comp rx=0 tx=0).stop 2024-05-09T09:20:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.092+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 shutdown_connections 2024-05-09T09:20:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.092+0000 7f0d033b1740 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce590f90 0x55e0ce5933a0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.092+0000 7f0d033b1740 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce58cde0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.092+0000 7f0d033b1740 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce585da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.092+0000 7f0d033b1740 1 --2- 172.21.15.64:0/373622112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce585860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.092+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 >> 172.21.15.64:0/373622112 conn(0x55e0ce5320a0 msgr2=0x55e0ce565080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.093+0000 7f0ce1ffb640 0 client.18390 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.093+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 shutdown_connections 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.094+0000 7f0d033b1740 1 -- 172.21.15.64:0/373622112 wait complete. 2024-05-09T09:20:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.094+0000 7f0d033b1740 20 client.18390 trim_cache size 0 max 16384 2024-05-09T09:20:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.130+0000 7f0d033b1740 1 Processor -- start 2024-05-09T09:20:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.130+0000 7f0d033b1740 1 -- start start 2024-05-09T09:20:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.130+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce569600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.130+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce58cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.130+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce5845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.131+0000 7f0d033b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc012590 con 0x55e0ce581030 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.131+0000 7f0cfec42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce5845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.131+0000 7f0cfec42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce5845e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46088/0 (socket says 172.21.15.64:46088) 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.131+0000 7f0cfec42640 1 -- 172.21.15.64:0/3304810218 learned_addr learned my addr 172.21.15.64:0/3304810218 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.131+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc00fe30 con 0x55e0ce57bec0 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.131+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc00dc80 con 0x55e0ce57b4d0 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.132+0000 7f0cfec42640 1 -- 172.21.15.64:0/3304810218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 msgr2=0x55e0ce569600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.132+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce569600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.132+0000 7f0cfec42640 1 -- 172.21.15.64:0/3304810218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 msgr2=0x55e0ce58cda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.132+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce58cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.132+0000 7f0cfec42640 1 -- 172.21.15.64:0/3304810218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0ce566bf0 con 0x55e0ce581030 2024-05-09T09:20:55.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.133+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce5845e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009900 tx=0x7f0cf0005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aa9ef386ab66f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.135+0000 7f0ce3fff640 1 -- 172.21.15.64:0/3304810218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0059070 con 0x55e0ce581030 2024-05-09T09:20:55.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.135+0000 7f0ce3fff640 1 -- 172.21.15.64:0/3304810218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf0010490 con 0x55e0ce581030 2024-05-09T09:20:55.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.136+0000 7f0ce3fff640 1 -- 172.21.15.64:0/3304810218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0015040 con 0x55e0ce581030 2024-05-09T09:20:55.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.137+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 msgr2=0x55e0ce5845e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.137+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce5845e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009900 tx=0x7f0cf0005cf0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.138+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 shutdown_connections 2024-05-09T09:20:55.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.139+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce581030 0x55e0ce5845e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.139+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce57bec0 0x55e0ce58cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.139+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3304810218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57b4d0 0x55e0ce569600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.139+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 >> 172.21.15.64:0/3304810218 conn(0x55e0ce5320a0 msgr2=0x55e0ce568ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.140+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 shutdown_connections 2024-05-09T09:20:55.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.140+0000 7f0d033b1740 1 -- 172.21.15.64:0/3304810218 wait complete. 2024-05-09T09:20:55.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.140+0000 7f0d033b1740 1 -- start start 2024-05-09T09:20:55.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.140+0000 7f0d033b1740 1 Processor -- start 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57b4d0 0x55e0ce58a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57bec0 0x55e0ce58a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce58b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc00e2f0 con 0x55e0ce57b4d0 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc00f760 con 0x55e0ce581030 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc00e9c0 con 0x55e0ce57bec0 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0d033b1740 10 client.0 _ll_register_callbacks cb 0x7f0cf0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0cfec42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce58b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0cfec42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce58b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34180/0 (socket says 172.21.15.64:34180) 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0cfec42640 1 -- 172.21.15.64:0/3313366939 learned_addr learned my addr 172.21.15.64:0/3313366939 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.141+0000 7f0cfe441640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57b4d0 0x55e0ce58a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfdc40640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57bec0 0x55e0ce58a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfec42640 1 -- 172.21.15.64:0/3313366939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57bec0 msgr2=0x55e0ce58a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57bec0 0x55e0ce58a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfec42640 1 -- 172.21.15.64:0/3313366939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57b4d0 msgr2=0x55e0ce58a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57b4d0 0x55e0ce58a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfec42640 1 -- 172.21.15.64:0/3313366939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0ce58be30 con 0x55e0ce581030 2024-05-09T09:20:55.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0cfec42640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce58b840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0000f80 tx=0x7f0cf00094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c402ff80319fb48c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.142+0000 7f0ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:20:55.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.143+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/3313366939 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0059070 con 0x55e0ce581030 2024-05-09T09:20:55.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.143+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0010490 con 0x55e0ce581030 2024-05-09T09:20:55.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.143+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/3313366939 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0cf0002030 con 0x55e0ce581030 2024-05-09T09:20:55.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.144+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/3313366939 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0cf0015070 con 0x55e0ce581030 2024-05-09T09:20:55.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.144+0000 7f0d033b1740 10 client.27842 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:55.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.144+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e0ce58d170 con 0x55e0ce581030 2024-05-09T09:20:55.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/3313366939 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf0010490 con 0x55e0ce581030 2024-05-09T09:20:55.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/3313366939 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f0cf0004030 con 0x55e0ce581030 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0d033b1740 10 client.27842 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0d033b1740 10 client.27842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0d033b1740 20 client.27842 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce5f79a0 0x55e0ce5f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0d033b1740 4 client.27842 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e0ce4dc0c0 con 0x55e0ce5f79a0 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.145+0000 7f0cfe441640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce5f79a0 0x55e0ce5f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.146+0000 7f0cfe441640 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce5f79a0 0x55e0ce5f9dc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c4eb87f17571e0 server_cookie=d5723ca1a80f969d in_seq=0 out_seq=0 2024-05-09T09:20:55.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.146+0000 7f0ce1ffb640 10 client.27842 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.147+0000 7f0ce1ffb640 1 -- 172.21.15.64:0/3313366939 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e0ce4dc0c0 con 0x55e0ce5f79a0 2024-05-09T09:20:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.147+0000 7f0ce1ffb640 10 client.27842 handle_command_reply: tid=0 2024-05-09T09:20:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.147+0000 7f0d033b1740 1 client.27842 shutdown 2024-05-09T09:20:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 msgr2=0x55e0ce58b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce58b840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0000f80 tx=0x7f0cf00094b0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 shutdown_connections 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e0ce5f79a0 0x55e0ce5f9dc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0ce581030 0x55e0ce58b840 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e0ce57bec0 0x55e0ce58a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 --2- 172.21.15.64:0/3313366939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0ce57b4d0 0x55e0ce58a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 >> 172.21.15.64:0/3313366939 conn(0x55e0ce5320a0 msgr2=0x55e0ce569600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.148+0000 7f0ce1ffb640 0 client.27842 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.149+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 shutdown_connections 2024-05-09T09:20:55.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.149+0000 7f0d033b1740 1 -- 172.21.15.64:0/3313366939 wait complete. 2024-05-09T09:20:55.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.149+0000 7f0d033b1740 20 client.27842 trim_cache size 0 max 16384 2024-05-09T09:20:55.150 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:20:55.150 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:20:55.150 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "1fca5bbc-5a03-4aae-ab19-53ac87a9ce35", 2024-05-09T09:20:55.151 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:20:55.151 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:20:55.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ce40a2c60 msgr2=0x7f0ce40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ce40a2c60 0x7f0ce40a5120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005ab0 tx=0x7f0ce8000bc0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 msgr2=0x7f0cf819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 0x7f0cf819e650 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400c890 tx=0x7f0cf400cd50 comp rx=0 tx=0).stop 2024-05-09T09:20:55.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 shutdown_connections 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ce40a2c60 0x7f0ce40a5120 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf8077dd0 0x7f0cf81a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8077430 0x7f0cf819e650 secure :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400c890 tx=0x7f0cf400cd50 comp rx=0 tx=0).stop 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 --2- 172.21.15.64:0/3871838200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cf80761f0 0x7f0cf819e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 >> 172.21.15.64:0/3871838200 conn(0x7f0cf80ff020 msgr2=0x7f0cf8100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.153+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 shutdown_connections 2024-05-09T09:20:55.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.154+0000 7f0d006a5640 1 -- 172.21.15.64:0/3871838200 wait complete. 2024-05-09T09:20:55.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:20:55.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:20:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 -- 172.21.15.64:0/1272641127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c580731e0 msgr2=0x7f1c5810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 --2- 172.21.15.64:0/1272641127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c580731e0 0x7f1c5810a0f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00fb10 tx=0x7f1c4c02e590 comp rx=0 tx=0).stop 2024-05-09T09:20:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 -- 172.21.15.64:0/1272641127 shutdown_connections 2024-05-09T09:20:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 --2- 172.21.15.64:0/1272641127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c58108830 0x7f1c58108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 --2- 172.21.15.64:0/1272641127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c580731e0 0x7f1c5810a0f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 --2- 172.21.15.64:0/1272641127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c58072d30 0x7f1c58109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 -- 172.21.15.64:0/1272641127 >> 172.21.15.64:0/1272641127 conn(0x7f1c5806e930 msgr2=0x7f1c58070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 -- 172.21.15.64:0/1272641127 shutdown_connections 2024-05-09T09:20:55.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.383+0000 7f1c57fff640 1 -- 172.21.15.64:0/1272641127 wait complete. 2024-05-09T09:20:55.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.385+0000 7f1c57fff640 1 Processor -- start 2024-05-09T09:20:55.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.385+0000 7f1c57fff640 1 -- start start 2024-05-09T09:20:55.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.385+0000 7f1c57fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 0x7f1c5819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.385+0000 7f1c57fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c580731e0 0x7f1c5819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.386+0000 7f1c56ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 0x7f1c5819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.387+0000 7f1c56ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 0x7f1c5819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34196/0 (socket says 172.21.15.64:34196) 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.387+0000 7f1c57fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c58108830 0x7f1c581a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.387+0000 7f1c57fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581138d0 con 0x7f1c580731e0 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.387+0000 7f1c57fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58113750 con 0x7f1c58072d30 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.387+0000 7f1c57fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58113a50 con 0x7f1c58108830 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.387+0000 7f1c56ffd640 1 -- 172.21.15.64:0/2731284102 learned_addr learned my addr 172.21.15.64:0/2731284102 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.388+0000 7f1c56ffd640 1 -- 172.21.15.64:0/2731284102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c58108830 msgr2=0x7f1c581a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.388+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c58108830 0x7f1c581a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.388+0000 7f1c56ffd640 1 -- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c580731e0 msgr2=0x7f1c5819e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.388+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c580731e0 0x7f1c5819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.388+0000 7f1c56ffd640 1 -- 172.21.15.64:0/2731284102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c581a59c0 con 0x7f1c58072d30 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.388+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 0x7f1c5819ddb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1c48007830 tx=0x7f1c48007cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.389+0000 7f1c5c8fa640 1 -- 172.21.15.64:0/2731284102 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48019030 con 0x7f1c58072d30 2024-05-09T09:20:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.389+0000 7f1c5c8fa640 1 -- 172.21.15.64:0/2731284102 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c4800f920 con 0x7f1c58072d30 2024-05-09T09:20:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.389+0000 7f1c5c8fa640 1 -- 172.21.15.64:0/2731284102 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48003a20 con 0x7f1c58072d30 2024-05-09T09:20:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.390+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c581a5cb0 con 0x7f1c58072d30 2024-05-09T09:20:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.390+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c581a61b0 con 0x7f1c58072d30 2024-05-09T09:20:55.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.395+0000 7f1c60607740 1 Processor -- start 2024-05-09T09:20:55.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.395+0000 7f1c60607740 1 -- start start 2024-05-09T09:20:55.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c60607740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf1320e0 0x557bbf132500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c60607740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf1375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c60607740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf13a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c60607740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557bbf096d00 con 0x557bbf1320e0 2024-05-09T09:20:55.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c60607740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557bbf03e4c0 con 0x557bbf137c40 2024-05-09T09:20:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c60607740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557bbf092cc0 con 0x557bbf132ad0 2024-05-09T09:20:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf13a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf13a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34210/0 (socket says 172.21.15.64:34210) 2024-05-09T09:20:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 -- 172.21.15.64:0/1213501756 learned_addr learned my addr 172.21.15.64:0/1213501756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 -- 172.21.15.64:0/1213501756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 msgr2=0x557bbf1375d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf1375d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 -- 172.21.15.64:0/1213501756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf1320e0 msgr2=0x557bbf132500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf1320e0 0x557bbf132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.396+0000 7f1c567fc640 1 -- 172.21.15.64:0/1213501756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bbee747d0 con 0x557bbf137c40 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.398+0000 7f1c5c8fa640 1 -- 172.21.15.64:0/2731284102 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c4801c070 con 0x7f1c58072d30 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.398+0000 7f1c567fc640 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf13a0d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c02eb00 tx=0x7f1c4c00e6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a520d66b8989f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.399+0000 7f1c3dffb640 1 -- 172.21.15.64:0/1213501756 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c066070 con 0x557bbf137c40 2024-05-09T09:20:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.400+0000 7f1c3dffb640 1 -- 172.21.15.64:0/1213501756 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c4c0048c0 con 0x557bbf137c40 2024-05-09T09:20:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.400+0000 7f1c5c8fa640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c400a2d90 0x7f1c400a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.400+0000 7f1c5c8fa640 1 -- 172.21.15.64:0/2731284102 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c480cf200 con 0x7f1c58072d30 2024-05-09T09:20:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.400+0000 7f1c577fe640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c400a2d90 0x7f1c400a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.400+0000 7f1c3dffb640 1 -- 172.21.15.64:0/1213501756 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00f410 con 0x557bbf137c40 2024-05-09T09:20:55.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.401+0000 7f1c60607740 1 -- 172.21.15.64:0/1213501756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 msgr2=0x557bbf13a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.401+0000 7f1c60607740 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf13a0d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c02eb00 tx=0x7f1c4c00e6f0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.402+0000 7f1c60607740 1 -- 172.21.15.64:0/1213501756 shutdown_connections 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.402+0000 7f1c60607740 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf13a0d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.402+0000 7f1c60607740 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf1375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.402+0000 7f1c60607740 1 --2- 172.21.15.64:0/1213501756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf1320e0 0x557bbf132500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.402+0000 7f1c60607740 1 -- 172.21.15.64:0/1213501756 >> 172.21.15.64:0/1213501756 conn(0x557bbf12d750 msgr2=0x557bbf12db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.403+0000 7f1c60607740 1 -- 172.21.15.64:0/1213501756 shutdown_connections 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.403+0000 7f1c577fe640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c400a2d90 0x7f1c400a5250 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x557bbf133330 tx=0x7f1c500093c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.403+0000 7f1c60607740 1 -- 172.21.15.64:0/1213501756 wait complete. 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 -- start start 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 Processor -- start 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf1320e0 0x557bbf13c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf132ad0 0x557bbf13cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557bbf13abe0 con 0x557bbf137c40 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557bbf051b90 con 0x557bbf132ad0 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557bbf13ad60 con 0x557bbf1320e0 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.404+0000 7f1c60607740 10 client.0 _ll_register_callbacks cb 0x557bbf08dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46128/0 (socket says 172.21.15.64:46128) 2024-05-09T09:20:55.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 -- 172.21.15.64:0/3850409549 learned_addr learned my addr 172.21.15.64:0/3850409549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c567fc640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf132ad0 0x557bbf13cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 -- 172.21.15.64:0/3850409549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf1320e0 msgr2=0x557bbf13c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf1320e0 0x557bbf13c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf1320e0 0x557bbf13c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 -- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf132ad0 msgr2=0x557bbf13cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf132ad0 0x557bbf13cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.405+0000 7f1c577fe640 1 -- 172.21.15.64:0/3850409549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bbedaa3f0 con 0x557bbf137c40 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.406+0000 7f1c567fc640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf132ad0 0x557bbf13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.406+0000 7f1c577fe640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf143af0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1c50011890 tx=0x7f1c50009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f171cc68068217dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.406+0000 7f1c36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.406+0000 7f1c36ffd640 1 -- 172.21.15.64:0/3850409549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50022020 con 0x557bbf137c40 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.406+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4c0048c0 con 0x557bbf137c40 2024-05-09T09:20:55.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.407+0000 7f1c36ffd640 1 -- 172.21.15.64:0/3850409549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c50010b50 con 0x557bbf137c40 2024-05-09T09:20:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.407+0000 7f1c36ffd640 1 -- 172.21.15.64:0/3850409549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1c5002d050 con 0x557bbf137c40 2024-05-09T09:20:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.407+0000 7f1c60607740 10 client.18402 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.407+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557bbed6a0f0 con 0x557bbf137c40 2024-05-09T09:20:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c36ffd640 1 -- 172.21.15.64:0/3850409549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c4c0048c0 con 0x557bbf137c40 2024-05-09T09:20:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c36ffd640 1 -- 172.21.15.64:0/3850409549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f1c5000f040 con 0x557bbf137c40 2024-05-09T09:20:55.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c60607740 10 client.18402 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c60607740 10 client.18402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c60607740 20 client.18402 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c60607740 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf159910 0x557bbf15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c60607740 4 client.18402 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.408+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bbf0a4430 con 0x557bbf159910 2024-05-09T09:20:55.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.409+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf159910 0x557bbf15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.410+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf159910 0x557bbf15bdb0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df0b8c1f6226718 server_cookie=1ae2be2dccdce087 in_seq=0 out_seq=0 2024-05-09T09:20:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.410+0000 7f1c36ffd640 10 client.18402 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.411+0000 7f1c36ffd640 1 -- 172.21.15.64:0/3850409549 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bbf0a4430 con 0x557bbf159910 2024-05-09T09:20:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.411+0000 7f1c36ffd640 10 client.18402 handle_command_reply: tid=0 2024-05-09T09:20:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.411+0000 7f1c60607740 1 client.18402 shutdown 2024-05-09T09:20:55.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.411+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 msgr2=0x557bbf143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.411+0000 7f1c60607740 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf143af0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1c50011890 tx=0x7f1c50009200 comp rx=0 tx=0).stop 2024-05-09T09:20:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 shutdown_connections 2024-05-09T09:20:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf159910 0x557bbf15bdb0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf143af0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf132ad0 0x557bbf13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c36ffd640 0 client.18402 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 --2- 172.21.15.64:0/3850409549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf1320e0 0x557bbf13c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 >> 172.21.15.64:0/3850409549 conn(0x557bbf12d750 msgr2=0x557bbf12ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.412+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 shutdown_connections 2024-05-09T09:20:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.413+0000 7f1c60607740 1 -- 172.21.15.64:0/3850409549 wait complete. 2024-05-09T09:20:55.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.413+0000 7f1c60607740 20 client.18402 trim_cache size 0 max 16384 2024-05-09T09:20:55.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.447+0000 7f1c60607740 1 Processor -- start 2024-05-09T09:20:55.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.447+0000 7f1c60607740 1 -- start start 2024-05-09T09:20:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.447+0000 7f1c60607740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 0x557bbf137500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.447+0000 7f1c60607740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c60607740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf1306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c56ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 0x557bbf137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c56ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 0x557bbf137500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34234/0 (socket says 172.21.15.64:34234) 2024-05-09T09:20:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c56ffd640 1 -- 172.21.15.64:0/1691602316 learned_addr learned my addr 172.21.15.64:0/1691602316 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c577fe640 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf1306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58080460 con 0x557bbf137c40 2024-05-09T09:20:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5807dd00 con 0x557bbf1320e0 2024-05-09T09:20:55.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.448+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5807bb50 con 0x557bbf132ad0 2024-05-09T09:20:55.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.451+0000 7f1c567fc640 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.452+0000 7f1c56ffd640 1 -- 172.21.15.64:0/1691602316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 msgr2=0x557bbf143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.452+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.452+0000 7f1c56ffd640 1 -- 172.21.15.64:0/1691602316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 msgr2=0x557bbf1306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.452+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf1306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.452+0000 7f1c56ffd640 1 -- 172.21.15.64:0/1691602316 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bbf157db0 con 0x557bbf1320e0 2024-05-09T09:20:55.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.453+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 0x557bbf137500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c48097a40 tx=0x7f1c4800b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc153a1adae1fea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.454+0000 7f1c3d7fa640 1 -- 172.21.15.64:0/1691602316 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800fe40 con 0x557bbf1320e0 2024-05-09T09:20:55.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.454+0000 7f1c3d7fa640 1 -- 172.21.15.64:0/1691602316 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c480b3030 con 0x557bbf1320e0 2024-05-09T09:20:55.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.454+0000 7f1c3d7fa640 1 -- 172.21.15.64:0/1691602316 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480a42e0 con 0x557bbf1320e0 2024-05-09T09:20:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.455+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 msgr2=0x557bbf137500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.455+0000 7f1c60607740 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 0x557bbf137500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c48097a40 tx=0x7f1c4800b6a0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.456+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 shutdown_connections 2024-05-09T09:20:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.456+0000 7f1c60607740 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf137c40 0x557bbf1306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.456+0000 7f1c60607740 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf132ad0 0x557bbf143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.456+0000 7f1c60607740 1 --2- 172.21.15.64:0/1691602316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf1320e0 0x557bbf137500 secure :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c48097a40 tx=0x7f1c4800b6a0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.457+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 >> 172.21.15.64:0/1691602316 conn(0x557bbf12d750 msgr2=0x557bbf133ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.458+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 shutdown_connections 2024-05-09T09:20:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.459+0000 7f1c60607740 1 -- 172.21.15.64:0/1691602316 wait complete. 2024-05-09T09:20:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.459+0000 7f1c60607740 1 -- start start 2024-05-09T09:20:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.460+0000 7f1c60607740 1 Processor -- start 2024-05-09T09:20:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.460+0000 7f1c60607740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf132ad0 0x557bbf141280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.460+0000 7f1c60607740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf1417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c60607740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 0x557bbf142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c60607740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5807c1c0 con 0x557bbf132ad0 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c60607740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5807d630 con 0x557bbf137c40 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c577fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 0x557bbf142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c577fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 0x557bbf142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48562/0 (socket says 172.21.15.64:48562) 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c577fe640 1 -- 172.21.15.64:0/2687444993 learned_addr learned my addr 172.21.15.64:0/2687444993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c567fc640 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf1417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5807c890 con 0x557bbf142780 2024-05-09T09:20:55.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.461+0000 7f1c60607740 10 client.0 _ll_register_callbacks cb 0x7f1c50011540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.464+0000 7f1c577fe640 1 -- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 msgr2=0x557bbf1417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.464+0000 7f1c577fe640 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf1417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.464+0000 7f1c577fe640 1 -- 172.21.15.64:0/2687444993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf132ad0 msgr2=0x557bbf141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.464+0000 7f1c577fe640 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf132ad0 0x557bbf141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.464+0000 7f1c577fe640 1 -- 172.21.15.64:0/2687444993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bbf143140 con 0x557bbf142780 2024-05-09T09:20:55.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.468+0000 7f1c577fe640 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 0x557bbf142c00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1c50013040 tx=0x7f1c50010b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e4447af106da30f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:55.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.468+0000 7f1c36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:55.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.468+0000 7f1c36ffd640 1 -- 172.21.15.64:0/2687444993 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50022040 con 0x557bbf142780 2024-05-09T09:20:55.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.468+0000 7f1c36ffd640 1 -- 172.21.15.64:0/2687444993 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c5001e640 con 0x557bbf142780 2024-05-09T09:20:55.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.468+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c480b3030 con 0x557bbf142780 2024-05-09T09:20:55.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.469+0000 7f1c36ffd640 1 -- 172.21.15.64:0/2687444993 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1c5001e840 con 0x557bbf142780 2024-05-09T09:20:55.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.470+0000 7f1c60607740 10 client.28018 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:55.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.470+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557bbf1c0450 con 0x557bbf142780 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.470+0000 7f1c36ffd640 1 -- 172.21.15.64:0/2687444993 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c480b3030 con 0x557bbf142780 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.470+0000 7f1c36ffd640 1 -- 172.21.15.64:0/2687444993 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f1c50032bc0 con 0x557bbf142780 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c60607740 10 client.28018 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c60607740 10 client.28018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c60607740 20 client.28018 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c60607740 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf1c0db0 0x557bbf1c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:55.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c60607740 4 client.28018 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:20:55.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557bbf09e780 con 0x557bbf1c0db0 2024-05-09T09:20:55.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.471+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf1c0db0 0x557bbf1c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:55.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.472+0000 7f1c56ffd640 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf1c0db0 0x557bbf1c31d0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8208aa3c05c7768 server_cookie=59889684c682f3e2 in_seq=0 out_seq=0 2024-05-09T09:20:55.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.472+0000 7f1c36ffd640 10 client.28018 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.472+0000 7f1c36ffd640 1 -- 172.21.15.64:0/2687444993 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557bbf09e780 con 0x557bbf1c0db0 2024-05-09T09:20:55.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.472+0000 7f1c36ffd640 10 client.28018 handle_command_reply: tid=0 2024-05-09T09:20:55.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.472+0000 7f1c60607740 1 client.28018 shutdown 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 msgr2=0x557bbf142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 0x557bbf142c00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1c50013040 tx=0x7f1c50010b00 comp rx=0 tx=0).stop 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 shutdown_connections 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x557bbf1c0db0 0x557bbf1c31d0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557bbf142780 0x557bbf142c00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557bbf137c40 0x557bbf1417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 --2- 172.21.15.64:0/2687444993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bbf132ad0 0x557bbf141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 >> 172.21.15.64:0/2687444993 conn(0x557bbf12d750 msgr2=0x557bbf12c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.473+0000 7f1c36ffd640 0 client.28018 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:55.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.475+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 shutdown_connections 2024-05-09T09:20:55.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.475+0000 7f1c60607740 1 -- 172.21.15.64:0/2687444993 wait complete. 2024-05-09T09:20:55.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.475+0000 7f1c60607740 20 client.28018 trim_cache size 0 max 16384 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: "1fca5bbc-5a03-4aae-ab19-53ac87a9ce35": { 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "1fca5bbc-5a03-4aae-ab19-53ac87a9ce35", 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:20:55.477 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:20:55.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:55 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:20:55.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:55 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:55.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:55 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:20:55.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.481+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c400a2d90 msgr2=0x7f1c400a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.482+0000 7f1c57fff640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c400a2d90 0x7f1c400a5250 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x557bbf133330 tx=0x7f1c500093c0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.482+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 msgr2=0x7f1c5819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:55.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.482+0000 7f1c57fff640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 0x7f1c5819ddb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1c48007830 tx=0x7f1c48007cf0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.482+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 shutdown_connections 2024-05-09T09:20:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.483+0000 7f1c57fff640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1c400a2d90 0x7f1c400a5250 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.483+0000 7f1c57fff640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c58108830 0x7f1c581a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.483+0000 7f1c57fff640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c580731e0 0x7f1c5819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.483+0000 7f1c57fff640 1 --2- 172.21.15.64:0/2731284102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c58072d30 0x7f1c5819ddb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:55.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.483+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 >> 172.21.15.64:0/2731284102 conn(0x7f1c5806e930 msgr2=0x7f1c5806ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:55.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.486+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 shutdown_connections 2024-05-09T09:20:55.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:20:55.487+0000 7f1c57fff640 1 -- 172.21.15.64:0/2731284102 wait complete. 2024-05-09T09:20:55.510 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:55 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:20:55.510 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:55 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:55.510 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:55 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:20:55.511 INFO:tasks.cephfs.filesystem:scrub status for tag:1fca5bbc-5a03-4aae-ab19-53ac87a9ce35 - {'path': '/', 'tag': '1fca5bbc-5a03-4aae-ab19-53ac87a9ce35', 'options': 'recursive,force'} 2024-05-09T09:20:55.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:55 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:20:55.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:55 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:55.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:55 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:20:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 635.0 s, 647.1 tps, lat 154.611 ms stddev 205.439 2024-05-09T09:20:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:56 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:20:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:56 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:20:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:56 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:56 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:56 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:56 smithi179 ceph-mon[34086]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-05-09T09:20:56.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:56 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:20:56.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:56 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:20:56.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:56 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:56 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:56 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:56 smithi139 ceph-mon[33599]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-05-09T09:20:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:56 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:20:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:56 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:20:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:56 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:56 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:56 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:20:56.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:56 smithi064 ceph-mon[30149]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-05-09T09:20:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.514+0000 7f00e4314640 1 -- 172.21.15.64:0/1538711000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc107e70 msgr2=0x7f00dc108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.514+0000 7f00e4314640 1 --2- 172.21.15.64:0/1538711000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc107e70 0x7f00dc108290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009860 tx=0x7f00d802a340 comp rx=0 tx=0).stop 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.517+0000 7f00e4314640 1 -- 172.21.15.64:0/1538711000 shutdown_connections 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.517+0000 7f00e4314640 1 --2- 172.21.15.64:0/1538711000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00dc109a10 0x7f00dc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.517+0000 7f00e4314640 1 --2- 172.21.15.64:0/1538711000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 0x7f00dc1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.517+0000 7f00e4314640 1 --2- 172.21.15.64:0/1538711000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc107e70 0x7f00dc108290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.517+0000 7f00e4314640 1 -- 172.21.15.64:0/1538711000 >> 172.21.15.64:0/1538711000 conn(0x7f00dc076e80 msgr2=0x7f00dc0792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.518+0000 7f00e4314640 1 -- 172.21.15.64:0/1538711000 shutdown_connections 2024-05-09T09:20:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.519+0000 7f00e4314640 1 -- 172.21.15.64:0/1538711000 wait complete. 2024-05-09T09:20:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.520+0000 7f00e4314640 1 Processor -- start 2024-05-09T09:20:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.522+0000 7f00e4314640 1 -- start start 2024-05-09T09:20:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e4314640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00dc107e70 0x7f00dc071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e4314640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 0x7f00dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e4314640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc109a10 0x7f00dc072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e18af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 0x7f00dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e18af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 0x7f00dc071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46174/0 (socket says 172.21.15.64:46174) 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e18af640 1 -- 172.21.15.64:0/2977292471 learned_addr learned my addr 172.21.15.64:0/2977292471 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc06bb50 con 0x7f00dc109050 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc06b9d0 con 0x7f00dc109a10 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc06bcd0 con 0x7f00dc107e70 2024-05-09T09:20:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.523+0000 7f00e28b1640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc109a10 0x7f00dc072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00e28b1640 1 -- 172.21.15.64:0/2977292471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00dc107e70 msgr2=0x7f00dc071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00e28b1640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00dc107e70 0x7f00dc071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00e28b1640 1 -- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 msgr2=0x7f00dc071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00e28b1640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 0x7f00dc071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00e28b1640 1 -- 172.21.15.64:0/2977292471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc072ff0 con 0x7f00dc109a10 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00e28b1640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc109a10 0x7f00dc072980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f00d400ef00 tx=0x7f00d400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.524+0000 7f00d37fe640 1 -- 172.21.15.64:0/2977292471 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004b20 con 0x7f00dc109a10 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.525+0000 7f00d37fe640 1 -- 172.21.15.64:0/2977292471 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00d4003e20 con 0x7f00dc109a10 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.525+0000 7f00d37fe640 1 -- 172.21.15.64:0/2977292471 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400b570 con 0x7f00dc109a10 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.525+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc0732e0 con 0x7f00dc109a10 2024-05-09T09:20:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.526+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc0737e0 con 0x7f00dc109a10 2024-05-09T09:20:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.529+0000 7f00d37fe640 1 -- 172.21.15.64:0/2977292471 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00d401d020 con 0x7f00dc109a10 2024-05-09T09:20:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.531+0000 7f00d37fe640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f00c80a2d80 0x7f00c80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.531+0000 7f00d37fe640 1 -- 172.21.15.64:0/2977292471 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00d40cf260 con 0x7f00dc109a10 2024-05-09T09:20:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.531+0000 7f00e20b0640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f00c80a2d80 0x7f00c80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.532+0000 7f00e20b0640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f00c80a2d80 0x7f00c80a5240 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f00d80095f0 tx=0x7f00d80056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.532+0000 7f00e7023740 1 Processor -- start 2024-05-09T09:20:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 -- start start 2024-05-09T09:20:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcb1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcb4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55937dc10340 con 0x55937dcb1db0 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55937dbb8290 con 0x55937dcacc40 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e7023740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55937dc0c300 con 0x55937dcac250 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e20b0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e20b0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48588/0 (socket says 172.21.15.64:48588) 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e20b0640 1 -- 172.21.15.64:0/3606770859 learned_addr learned my addr 172.21.15.64:0/3606770859 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.533+0000 7f00e28b1640 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcb4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00e20b0640 1 -- 172.21.15.64:0/3606770859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 msgr2=0x55937dcb1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00e20b0640 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcb1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00e20b0640 1 -- 172.21.15.64:0/3606770859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 msgr2=0x55937dcb4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00e20b0640 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcb4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00e20b0640 1 -- 172.21.15.64:0/3606770859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937dbdc8e0 con 0x55937dcac250 2024-05-09T09:20:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00e20b0640 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcac670 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f00d8034000 tx=0x7f00d8039000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f3a2ed78cffebe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00d0bfe640 1 -- 172.21.15.64:0/3606770859 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d804d070 con 0x55937dcac250 2024-05-09T09:20:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.534+0000 7f00d0bfe640 1 -- 172.21.15.64:0/3606770859 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00d8003ca0 con 0x55937dcac250 2024-05-09T09:20:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.535+0000 7f00d0bfe640 1 -- 172.21.15.64:0/3606770859 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037040 con 0x55937dcac250 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 -- 172.21.15.64:0/3606770859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 msgr2=0x55937dcac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcac670 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f00d8034000 tx=0x7f00d8039000 comp rx=0 tx=0).stop 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 -- 172.21.15.64:0/3606770859 shutdown_connections 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcb4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcb1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 --2- 172.21.15.64:0/3606770859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcac670 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.536+0000 7f00e7023740 1 -- 172.21.15.64:0/3606770859 >> 172.21.15.64:0/3606770859 conn(0x55937dca78c0 msgr2=0x55937dca7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.537+0000 7f00e7023740 1 -- 172.21.15.64:0/3606770859 shutdown_connections 2024-05-09T09:20:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.538+0000 7f00e7023740 1 -- 172.21.15.64:0/3606770859 wait complete. 2024-05-09T09:20:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.538+0000 7f00e7023740 1 -- start start 2024-05-09T09:20:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.538+0000 7f00e7023740 1 Processor -- start 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcac250 0x55937dcb6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcacc40 0x55937dcb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcbdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55937dcb4d00 con 0x55937dcb1db0 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55937dbcb790 con 0x55937dcac250 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55937dcb4e80 con 0x55937dcacc40 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e7023740 10 client.0 _ll_register_callbacks cb 0x55937dc06a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e28b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcbdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e28b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcbdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46186/0 (socket says 172.21.15.64:46186) 2024-05-09T09:20:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.539+0000 7f00e28b1640 1 -- 172.21.15.64:0/8312305 learned_addr learned my addr 172.21.15.64:0/8312305 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.540+0000 7f00e28b1640 1 -- 172.21.15.64:0/8312305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcacc40 msgr2=0x55937dcb6aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.540+0000 7f00e28b1640 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcacc40 0x55937dcb6aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.540+0000 7f00e28b1640 1 -- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcac250 msgr2=0x55937dcb6560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:20:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.540+0000 7f00e28b1640 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcac250 0x55937dcb6560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.540+0000 7f00e28b1640 1 -- 172.21.15.64:0/8312305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937d924210 con 0x55937dcb1db0 2024-05-09T09:20:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.541+0000 7f00e28b1640 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcbdae0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f00d400cc40 tx=0x7f00d409be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5929ce44aaa45101 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.541+0000 7f00c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:20:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.541+0000 7f00c5ffb640 1 -- 172.21.15.64:0/8312305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d40b4060 con 0x55937dcb1db0 2024-05-09T09:20:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.542+0000 7f00c5ffb640 1 -- 172.21.15.64:0/8312305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00d40cca80 con 0x55937dcb1db0 2024-05-09T09:20:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.544+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d8003ca0 con 0x55937dcb1db0 2024-05-09T09:20:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.545+0000 7f00c5ffb640 1 -- 172.21.15.64:0/8312305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f00d40b9050 con 0x55937dcb1db0 2024-05-09T09:20:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.546+0000 7f00e7023740 10 client.18414 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.546+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55937d9f0620 con 0x55937dcb1db0 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00c5ffb640 1 -- 172.21.15.64:0/8312305 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00d8003ca0 con 0x55937dcb1db0 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00c5ffb640 1 -- 172.21.15.64:0/8312305 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f00d400b040 con 0x55937dcb1db0 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00e7023740 10 client.18414 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00e7023740 10 client.18414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00e7023740 20 client.18414 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00e7023740 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dcc2090 0x55937dcc44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00e7023740 4 client.18414 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.547+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937dc1da70 con 0x55937dcc2090 2024-05-09T09:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.552+0000 7f00e20b0640 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dcc2090 0x55937dcc44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.553+0000 7f00e20b0640 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dcc2090 0x55937dcc44e0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16f3413b1ced1ec server_cookie=28123781e7ac3dfb in_seq=0 out_seq=0 2024-05-09T09:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.553+0000 7f00c5ffb640 10 client.18414 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.554+0000 7f00c5ffb640 1 -- 172.21.15.64:0/8312305 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55937dc1da70 con 0x55937dcc2090 2024-05-09T09:20:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.555+0000 7f00c5ffb640 10 client.18414 handle_command_reply: tid=0 2024-05-09T09:20:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.555+0000 7f00e7023740 1 client.18414 shutdown 2024-05-09T09:20:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.556+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 msgr2=0x55937dcbdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.556+0000 7f00e7023740 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcbdae0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f00d400cc40 tx=0x7f00d409be90 comp rx=0 tx=0).stop 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 shutdown_connections 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dcc2090 0x55937dcc44e0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcbdae0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcacc40 0x55937dcb6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 --2- 172.21.15.64:0/8312305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcac250 0x55937dcb6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 >> 172.21.15.64:0/8312305 conn(0x55937dca78c0 msgr2=0x55937dca6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00c5ffb640 0 client.18414 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 shutdown_connections 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 1 -- 172.21.15.64:0/8312305 wait complete. 2024-05-09T09:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.560+0000 7f00e7023740 20 client.18414 trim_cache size 0 max 16384 2024-05-09T09:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.591+0000 7f00e7023740 1 Processor -- start 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 -- start start 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcb5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcbdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcaa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1bbc30 con 0x55937dcb1db0 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1b94d0 con 0x55937dcacc40 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e7023740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1b7320 con 0x55937dcac250 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e18af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcbdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e28b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcaa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e18af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcbdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34282/0 (socket says 172.21.15.64:34282) 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e18af640 1 -- 172.21.15.64:0/305164011 learned_addr learned my addr 172.21.15.64:0/305164011 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.592+0000 7f00e20b0640 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcb5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e18af640 1 -- 172.21.15.64:0/305164011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 msgr2=0x55937dcb5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e18af640 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcb5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e18af640 1 -- 172.21.15.64:0/305164011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 msgr2=0x55937dcaa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e18af640 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcaa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e18af640 1 -- 172.21.15.64:0/305164011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937dca8330 con 0x55937dcacc40 2024-05-09T09:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e28b1640 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcaa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:20:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.593+0000 7f00e18af640 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcbdaa0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f00dc072910 tx=0x7f00cc00af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6d72eb8f737103 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.594+0000 7f00c7fff640 1 -- 172.21.15.64:0/305164011 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc005020 con 0x55937dcacc40 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.594+0000 7f00c7fff640 1 -- 172.21.15.64:0/305164011 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00cc005670 con 0x55937dcacc40 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 -- 172.21.15.64:0/305164011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 msgr2=0x55937dcbdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00c7fff640 1 -- 172.21.15.64:0/305164011 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc03a6a0 con 0x55937dcacc40 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcbdaa0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f00dc072910 tx=0x7f00cc00af90 comp rx=0 tx=0).stop 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 -- 172.21.15.64:0/305164011 shutdown_connections 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcb1db0 0x55937dcaa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcacc40 0x55937dcbdaa0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 --2- 172.21.15.64:0/305164011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcb5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.595+0000 7f00e7023740 1 -- 172.21.15.64:0/305164011 >> 172.21.15.64:0/305164011 conn(0x55937dca78c0 msgr2=0x55937dcae000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.596+0000 7f00e7023740 1 -- 172.21.15.64:0/305164011 shutdown_connections 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.596+0000 7f00e7023740 1 -- 172.21.15.64:0/305164011 wait complete. 2024-05-09T09:20:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.596+0000 7f00e7023740 1 -- start start 2024-05-09T09:20:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.597+0000 7f00e7023740 1 Processor -- start 2024-05-09T09:20:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.597+0000 7f00e7023740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcbb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e7023740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcacc40 0x55937dcbb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e7023740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcb1db0 0x55937dcbc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcbb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcbb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:48608/0 (socket says 172.21.15.64:48608) 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 -- 172.21.15.64:0/687950117 learned_addr learned my addr 172.21.15.64:0/687950117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1b7990 con 0x55937dcacc40 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1b8e00 con 0x55937dcb1db0 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1b8060 con 0x55937dcac250 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e7023740 10 client.0 _ll_register_callbacks cb 0x7f00d40979d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:20:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e18af640 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcacc40 0x55937dcbb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 -- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcb1db0 msgr2=0x55937dcbc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcb1db0 0x55937dcbc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 -- 172.21.15.64:0/687950117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcacc40 msgr2=0x55937dcbb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:20:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcacc40 0x55937dcbb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.598+0000 7f00e20b0640 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937dcbcb90 con 0x55937dcac250 2024-05-09T09:20:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.599+0000 7f00e20b0640 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcbb050 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f00d802a930 tx=0x7f00d8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2ee494b088cc2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.599+0000 7f00c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.599+0000 7f00c5ffb640 1 -- 172.21.15.64:0/687950117 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d804d040 con 0x55937dcac250 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.599+0000 7f00c5ffb640 1 -- 172.21.15.64:0/687950117 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f00d80526b0 con 0x55937dcac250 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00cc005670 con 0x55937dcac250 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00c5ffb640 1 -- 172.21.15.64:0/687950117 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f00d8060050 con 0x55937dcac250 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00e7023740 10 client.28030 fetch_fsmap learned FSMap version 46 2024-05-09T09:20:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55937dd28660 con 0x55937dcac250 2024-05-09T09:20:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00c5ffb640 1 -- 172.21.15.64:0/687950117 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00cc005670 con 0x55937dcac250 2024-05-09T09:20:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00c5ffb640 1 -- 172.21.15.64:0/687950117 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f00d803a030 con 0x55937dcac250 2024-05-09T09:20:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.600+0000 7f00e7023740 10 client.28030 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:20:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.601+0000 7f00e7023740 10 client.28030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:20:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.601+0000 7f00e7023740 20 client.28030 populate_metadata read hostname 'smithi064' 2024-05-09T09:20:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.601+0000 7f00e7023740 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dd28b50 0x55937dd2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:20:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.601+0000 7f00e7023740 4 client.28030 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:20:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.601+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55937dcb6350 con 0x55937dd28b50 2024-05-09T09:20:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.602+0000 7f00e18af640 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dd28b50 0x55937dd2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:20:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.602+0000 7f00c5ffb640 10 client.28030 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.602+0000 7f00e18af640 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dd28b50 0x55937dd2af40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24c9175f894d650 server_cookie=e05b1368f545cdf9 in_seq=0 out_seq=0 2024-05-09T09:20:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.642+0000 7f00c5ffb640 1 -- 172.21.15.64:0/687950117 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55937dcb6350 con 0x55937dd28b50 2024-05-09T09:20:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.642+0000 7f00c5ffb640 10 client.28030 handle_command_reply: tid=0 2024-05-09T09:20:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.643+0000 7f00e7023740 1 client.28030 shutdown 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 msgr2=0x55937dcbb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcbb050 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f00d802a930 tx=0x7f00d8005c90 comp rx=0 tx=0).stop 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 shutdown_connections 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55937dd28b50 0x55937dd2af40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55937dcb1db0 0x55937dcbc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55937dcacc40 0x55937dcbb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 --2- 172.21.15.64:0/687950117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937dcac250 0x55937dcbb050 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00c5ffb640 0 client.28030 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:20:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.644+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 >> 172.21.15.64:0/687950117 conn(0x55937dca78c0 msgr2=0x55937dcaa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.645+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 shutdown_connections 2024-05-09T09:20:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.645+0000 7f00e7023740 1 -- 172.21.15.64:0/687950117 wait complete. 2024-05-09T09:20:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.645+0000 7f00e7023740 20 client.28030 trim_cache size 0 max 16384 2024-05-09T09:20:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:20:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:20:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:20:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:20:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.647+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f00c80a2d80 msgr2=0x7f00c80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.647+0000 7f00e4314640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f00c80a2d80 0x7f00c80a5240 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f00d80095f0 tx=0x7f00d80056d0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.647+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc109a10 msgr2=0x7f00dc072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:20:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.647+0000 7f00e4314640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc109a10 0x7f00dc072980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f00d400ef00 tx=0x7f00d400c740 comp rx=0 tx=0).stop 2024-05-09T09:20:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.649+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 shutdown_connections 2024-05-09T09:20:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.649+0000 7f00e4314640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f00c80a2d80 0x7f00c80a5240 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.649+0000 7f00e4314640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00dc109a10 0x7f00dc072980 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.649+0000 7f00e4314640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc109050 0x7f00dc071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.649+0000 7f00e4314640 1 --2- 172.21.15.64:0/2977292471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00dc107e70 0x7f00dc071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:20:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.649+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 >> 172.21.15.64:0/2977292471 conn(0x7f00dc076e80 msgr2=0x7f00dc10c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:20:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.651+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 shutdown_connections 2024-05-09T09:20:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:20:57.651+0000 7f00e4314640 1 -- 172.21.15.64:0/2977292471 wait complete. 2024-05-09T09:20:58.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:20:58 smithi139 ceph-mon[33599]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 700 op/s 2024-05-09T09:20:58.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:20:58 smithi064 ceph-mon[30149]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 700 op/s 2024-05-09T09:20:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:20:58 smithi179 ceph-mon[34086]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 700 op/s 2024-05-09T09:21:00.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:00 smithi139 ceph-mon[33599]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 977 op/s 2024-05-09T09:21:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:00 smithi064 ceph-mon[30149]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 977 op/s 2024-05-09T09:21:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:00 smithi179 ceph-mon[34086]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 12 MiB/s wr, 977 op/s 2024-05-09T09:21:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 640.0 s, 757.6 tps, lat 131.908 ms stddev 94.479 2024-05-09T09:21:02.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:02 smithi139 ceph-mon[33599]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 8.8 MiB/s wr, 780 op/s 2024-05-09T09:21:02.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:02 smithi064 ceph-mon[30149]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 8.8 MiB/s wr, 780 op/s 2024-05-09T09:21:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:02 smithi179 ceph-mon[34086]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 8.8 MiB/s wr, 780 op/s 2024-05-09T09:21:04.827 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:04 smithi139 ceph-mon[33599]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 9.9 MiB/s wr, 891 op/s 2024-05-09T09:21:04.910 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:04 smithi064 ceph-mon[30149]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 9.9 MiB/s wr, 891 op/s 2024-05-09T09:21:04.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:04 smithi179 ceph-mon[34086]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 9.9 MiB/s wr, 891 op/s 2024-05-09T09:21:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 645.0 s, 797.4 tps, lat 125.156 ms stddev 54.114 2024-05-09T09:21:06.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:06 smithi139 ceph-mon[33599]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 9.0 MiB/s wr, 884 op/s 2024-05-09T09:21:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:06 smithi064 ceph-mon[30149]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 9.0 MiB/s wr, 884 op/s 2024-05-09T09:21:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:06 smithi179 ceph-mon[34086]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 9.0 MiB/s wr, 884 op/s 2024-05-09T09:21:08.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:08 smithi139 ceph-mon[33599]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 7.6 MiB/s wr, 776 op/s 2024-05-09T09:21:08.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:08 smithi064 ceph-mon[30149]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 7.6 MiB/s wr, 776 op/s 2024-05-09T09:21:08.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:08 smithi179 ceph-mon[34086]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 7.6 MiB/s wr, 776 op/s 2024-05-09T09:21:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.802+0000 7f4766ffd640 1 -- 172.21.15.64:0/3484226006 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c031880 con 0x7f4768102a70 2024-05-09T09:21:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.802+0000 7f476e5a5640 1 -- 172.21.15.64:0/3484226006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4768102a70 msgr2=0x7f4768104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.802+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3484226006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4768102a70 0x7f4768104ee0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f475c009900 tx=0x7f475c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 -- 172.21.15.64:0/3484226006 shutdown_connections 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3484226006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4768105420 0x7f47681058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3484226006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4768102a70 0x7f4768104ee0 secure :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f475c009900 tx=0x7f475c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3484226006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768100100 0x7f4768102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 -- 172.21.15.64:0/3484226006 >> 172.21.15.64:0/3484226006 conn(0x7f47680fbd00 msgr2=0x7f47680fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 -- 172.21.15.64:0/3484226006 shutdown_connections 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.803+0000 7f476e5a5640 1 -- 172.21.15.64:0/3484226006 wait complete. 2024-05-09T09:21:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 Processor -- start 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 -- start start 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4768100100 0x7f47680703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 0x7f47680708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47681ac0f0 0x7f47681ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47681139f0 con 0x7f4768100100 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4768113870 con 0x7f4768105420 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.804+0000 7f476e5a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4768113b70 con 0x7f47681ac0f0 2024-05-09T09:21:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 0x7f47680708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 0x7f47680708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:46560/0 (socket says 172.21.15.64:46560) 2024-05-09T09:21:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 -- 172.21.15.64:0/3429805863 learned_addr learned my addr 172.21.15.64:0/3429805863 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f476cb42640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47681ac0f0 0x7f47681ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f4767fff640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4768100100 0x7f47680703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 -- 172.21.15.64:0/3429805863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47681ac0f0 msgr2=0x7f47681ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47681ac0f0 0x7f47681ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 -- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4768100100 msgr2=0x7f47680703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4768100100 0x7f47680703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f47677fe640 1 -- 172.21.15.64:0/3429805863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4768106750 con 0x7f4768105420 2024-05-09T09:21:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.805+0000 7f476cb42640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47681ac0f0 0x7f47681ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:21:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.807+0000 7f47677fe640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 0x7f47680708e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f475c02a960 tx=0x7f475c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.807+0000 7f47657fa640 1 -- 172.21.15.64:0/3429805863 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c03f020 con 0x7f4768105420 2024-05-09T09:21:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.808+0000 7f47657fa640 1 -- 172.21.15.64:0/3429805863 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475c003e40 con 0x7f4768105420 2024-05-09T09:21:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.808+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47681069e0 con 0x7f4768105420 2024-05-09T09:21:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.808+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4768106ef0 con 0x7f4768105420 2024-05-09T09:21:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.808+0000 7f47657fa640 1 -- 172.21.15.64:0/3429805863 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c035070 con 0x7f4768105420 2024-05-09T09:21:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.811+0000 7f477138c740 1 Processor -- start 2024-05-09T09:21:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.811+0000 7f477138c740 1 -- start start 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f477138c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e81250 0x563086e81670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f477138c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 0x563086e86740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f477138c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f477138c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563086de5340 con 0x563086e86db0 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f477138c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563086d8d290 con 0x563086e81c40 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f477138c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563086de1300 con 0x563086e81250 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 0x563086e86740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 0x563086e86740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:46568/0 (socket says 172.21.15.64:46568) 2024-05-09T09:21:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 -- 172.21.15.64:0/3610418735 learned_addr learned my addr 172.21.15.64:0/3610418735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f476cb42640 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e81250 0x563086e81670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 -- 172.21.15.64:0/3610418735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e81250 msgr2=0x563086e81670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e81250 0x563086e81670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 -- 172.21.15.64:0/3610418735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 msgr2=0x563086e89240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e89240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.812+0000 7f4767fff640 1 -- 172.21.15.64:0/3610418735 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563086db18e0 con 0x563086e81c40 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.813+0000 7f47657fa640 1 -- 172.21.15.64:0/3429805863 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f475c042020 con 0x7f4768105420 2024-05-09T09:21:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.813+0000 7f4767fff640 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 0x563086e86740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4760009f60 tx=0x7f4760009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24de5c3e73776 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.813+0000 7f474e7fc640 1 -- 172.21.15.64:0/3610418735 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760016070 con 0x563086e81c40 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.813+0000 7f474e7fc640 1 -- 172.21.15.64:0/3610418735 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f476001bb10 con 0x563086e81c40 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.813+0000 7f474e7fc640 1 -- 172.21.15.64:0/3610418735 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760012c20 con 0x563086e81c40 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.814+0000 7f47657fa640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47500a2d00 0x7f47500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.814+0000 7f476cb42640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47500a2d00 0x7f47500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.814+0000 7f47657fa640 1 -- 172.21.15.64:0/3429805863 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f475c0efe30 con 0x7f4768105420 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.814+0000 7f477138c740 1 -- 172.21.15.64:0/3610418735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 msgr2=0x563086e86740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.814+0000 7f477138c740 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 0x563086e86740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4760009f60 tx=0x7f4760009f90 comp rx=0 tx=0).stop 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.814+0000 7f476cb42640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47500a2d00 0x7f47500a51c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563086e824a0 tx=0x7f4758009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.815+0000 7f477138c740 1 -- 172.21.15.64:0/3610418735 shutdown_connections 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.815+0000 7f477138c740 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.815+0000 7f477138c740 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81c40 0x563086e86740 secure :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4760009f60 tx=0x7f4760009f90 comp rx=0 tx=0).stop 2024-05-09T09:21:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.815+0000 7f477138c740 1 --2- 172.21.15.64:0/3610418735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e81250 0x563086e81670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.815+0000 7f477138c740 1 -- 172.21.15.64:0/3610418735 >> 172.21.15.64:0/3610418735 conn(0x563086e7c8c0 msgr2=0x563086e7ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.816+0000 7f477138c740 1 -- 172.21.15.64:0/3610418735 shutdown_connections 2024-05-09T09:21:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.818+0000 7f477138c740 1 -- 172.21.15.64:0/3610418735 wait complete. 2024-05-09T09:21:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.818+0000 7f477138c740 1 -- start start 2024-05-09T09:21:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.819+0000 7f477138c740 1 Processor -- start 2024-05-09T09:21:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.819+0000 7f477138c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e8b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.819+0000 7f477138c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086e8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.819+0000 7f477138c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086e930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f477138c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563086e89d60 con 0x563086e90d00 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086e8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f477138c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563086da0790 con 0x563086e81250 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086e8bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:41564/0 (socket says 172.21.15.64:41564) 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 -- 172.21.15.64:0/1065663001 learned_addr learned my addr 172.21.15.64:0/1065663001 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f476cb42640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086e930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f4767fff640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e8b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563086e89ee0 con 0x563086e86db0 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f477138c740 10 client.0 _ll_register_callbacks cb 0x563086ddba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 -- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 msgr2=0x563086e8b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e8b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 -- 172.21.15.64:0/1065663001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 msgr2=0x563086e930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086e930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f47677fe640 1 -- 172.21.15.64:0/1065663001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563086af9210 con 0x563086e86db0 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.820+0000 7f4767fff640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e8b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f47677fe640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086e8bcd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f475c0bc290 tx=0x7f475c0bbe40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526052861a13c7d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f47477fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f47477fe640 1 -- 172.21.15.64:0/1065663001 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bc300 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476001bb10 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f47477fe640 1 -- 172.21.15.64:0/1065663001 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f475c03b460 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f47477fe640 1 -- 172.21.15.64:0/1065663001 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f475c0c06b0 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f477138c740 10 client.28039 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563086bc5620 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.821+0000 7f47477fe640 1 -- 172.21.15.64:0/1065663001 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f476001bb10 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f47477fe640 1 -- 172.21.15.64:0/1065663001 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f475c0c4b70 con 0x563086e86db0 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f477138c740 10 client.28039 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f477138c740 10 client.28039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f477138c740 20 client.28039 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f477138c740 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086e99a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f477138c740 4 client.28039 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563086df2a70 con 0x563086e97660 2024-05-09T09:21:10.825 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:10 smithi139 ceph-mon[33599]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.4 MiB/s wr, 976 op/s 2024-05-09T09:21:10.825 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:21:10.825 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:10 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.822+0000 7f4767fff640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086e99a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.823+0000 7f4767fff640 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086e99a70 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3bc5ad7d7251b6 server_cookie=6b699ffb196765fa in_seq=0 out_seq=0 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.823+0000 7f47477fe640 10 client.28039 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f47477fe640 1 -- 172.21.15.64:0/1065663001 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563086df2a70 con 0x563086e97660 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f47477fe640 10 client.28039 handle_command_reply: tid=0 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 client.28039 shutdown 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 msgr2=0x563086e8bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086e8bcd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f475c0bc290 tx=0x7f475c0bbe40 comp rx=0 tx=0).stop 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 shutdown_connections 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086e99a70 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086e930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f47477fe640 0 client.28039 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086e8bcd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 --2- 172.21.15.64:0/1065663001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e8b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 >> 172.21.15.64:0/1065663001 conn(0x563086e7c8c0 msgr2=0x563086e7ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 shutdown_connections 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.824+0000 7f477138c740 1 -- 172.21.15.64:0/1065663001 wait complete. 2024-05-09T09:21:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.825+0000 7f477138c740 20 client.28039 trim_cache size 0 max 16384 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.854+0000 7f477138c740 1 Processor -- start 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.854+0000 7f477138c740 1 -- start start 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.854+0000 7f477138c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e83140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.854+0000 7f477138c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e83680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.854+0000 7f477138c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e90d00 0x563086e7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e83680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e83680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59708/0 (socket says 172.21.15.64:59708) 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f4767fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e83140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f4767fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e83140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:46584/0 (socket says 172.21.15.64:46584) 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 -- 172.21.15.64:0/1381812333 learned_addr learned my addr 172.21.15.64:0/1381812333 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47681c4710 con 0x563086e86db0 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47681c1fb0 con 0x563086e81250 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47681bfe20 con 0x563086e90d00 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f476cb42640 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e90d00 0x563086e7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 -- 172.21.15.64:0/1381812333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e90d00 msgr2=0x563086e7b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e90d00 0x563086e7b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 -- 172.21.15.64:0/1381812333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 msgr2=0x563086e83140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e83140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.855+0000 7f47677fe640 1 -- 172.21.15.64:0/1381812333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563086d880d0 con 0x563086e86db0 2024-05-09T09:21:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.856+0000 7f47677fe640 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e83680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b8760 tx=0x7f475c0bbf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1c59e74233e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.857+0000 7f474dffb640 1 -- 172.21.15.64:0/1381812333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c09c030 con 0x563086e86db0 2024-05-09T09:21:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.857+0000 7f474dffb640 1 -- 172.21.15.64:0/1381812333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475c0bc300 con 0x563086e86db0 2024-05-09T09:21:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.857+0000 7f474dffb640 1 -- 172.21.15.64:0/1381812333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c0b8210 con 0x563086e86db0 2024-05-09T09:21:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.858+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 msgr2=0x563086e83680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.858+0000 7f477138c740 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e83680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b8760 tx=0x7f475c0bbf70 comp rx=0 tx=0).stop 2024-05-09T09:21:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.859+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 shutdown_connections 2024-05-09T09:21:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.859+0000 7f477138c740 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e90d00 0x563086e7b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.859+0000 7f477138c740 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e86db0 0x563086e83680 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.859+0000 7f477138c740 1 --2- 172.21.15.64:0/1381812333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086e83140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.859+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 >> 172.21.15.64:0/1381812333 conn(0x563086e7c8c0 msgr2=0x563086e8b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.864+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 shutdown_connections 2024-05-09T09:21:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.864+0000 7f477138c740 1 -- 172.21.15.64:0/1381812333 wait complete. 2024-05-09T09:21:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.864+0000 7f477138c740 1 -- start start 2024-05-09T09:21:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 Processor -- start 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086ea81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086ea8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086ea96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47681c0470 con 0x563086e90d00 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47681c18e0 con 0x563086e81250 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47681c0b40 con 0x563086e86db0 2024-05-09T09:21:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f477138c740 10 client.0 _ll_register_callbacks cb 0x7f475c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f4767fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086ea81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f4767fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086ea81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:46590/0 (socket says 172.21.15.64:46590) 2024-05-09T09:21:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.865+0000 7f4767fff640 1 -- 172.21.15.64:0/1145644567 learned_addr learned my addr 172.21.15.64:0/1145644567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f47677fe640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086ea8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f476cb42640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086ea96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f4767fff640 1 -- 172.21.15.64:0/1145644567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 msgr2=0x563086ea8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f4767fff640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086ea8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f4767fff640 1 -- 172.21.15.64:0/1145644567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 msgr2=0x563086ea96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f4767fff640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086ea96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f4767fff640 1 -- 172.21.15.64:0/1145644567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563086ea9cb0 con 0x563086e81250 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f4767fff640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086ea81e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4760010c90 tx=0x7f4760005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6225e40742a52bf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.866+0000 7f47477fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.867+0000 7f47477fe640 1 -- 172.21.15.64:0/1145644567 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760016040 con 0x563086e81250 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.867+0000 7f47477fe640 1 -- 172.21.15.64:0/1145644567 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4760003d80 con 0x563086e81250 2024-05-09T09:21:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.867+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c0bc300 con 0x563086e81250 2024-05-09T09:21:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.867+0000 7f47477fe640 1 -- 172.21.15.64:0/1145644567 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4760035050 con 0x563086e81250 2024-05-09T09:21:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.867+0000 7f477138c740 10 client.27884 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.867+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563086eab030 con 0x563086e81250 2024-05-09T09:21:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f47477fe640 1 -- 172.21.15.64:0/1145644567 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47600278b0 con 0x563086e81250 2024-05-09T09:21:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f47477fe640 1 -- 172.21.15.64:0/1145644567 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f476000f040 con 0x563086e81250 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f477138c740 10 client.27884 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f477138c740 10 client.27884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f477138c740 20 client.27884 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f477138c740 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086eff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f477138c740 4 client.27884 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.868+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f475c0bc300 con 0x563086e97660 2024-05-09T09:21:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.870+0000 7f47677fe640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086eff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.871+0000 7f47477fe640 10 client.27884 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.871+0000 7f47677fe640 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086eff960 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd69ca4d631cf262 server_cookie=c366c9e8950bafa1 in_seq=0 out_seq=0 2024-05-09T09:21:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.899+0000 7f47477fe640 1 -- 172.21.15.64:0/1145644567 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f475c0bc300 con 0x563086e97660 2024-05-09T09:21:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.899+0000 7f47477fe640 10 client.27884 handle_command_reply: tid=0 2024-05-09T09:21:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.899+0000 7f477138c740 1 client.27884 shutdown 2024-05-09T09:21:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 msgr2=0x563086ea81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086ea81e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4760010c90 tx=0x7f4760005fd0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 shutdown_connections 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563086e97660 0x563086eff960 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563086e90d00 0x563086ea96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f47477fe640 0 client.27884 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563086e86db0 0x563086ea8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 --2- 172.21.15.64:0/1145644567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563086e81250 0x563086ea81e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 >> 172.21.15.64:0/1145644567 conn(0x563086e7c8c0 msgr2=0x563086e8b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:10.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:10 smithi064 ceph-mon[30149]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.4 MiB/s wr, 976 op/s 2024-05-09T09:21:10.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:21:10.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:10 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:21:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 shutdown_connections 2024-05-09T09:21:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.903+0000 7f477138c740 1 -- 172.21.15.64:0/1145644567 wait complete. 2024-05-09T09:21:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.904+0000 7f477138c740 20 client.27884 trim_cache size 0 max 16384 2024-05-09T09:21:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.905+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47500a2d00 msgr2=0x7f47500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.905+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47500a2d00 0x7f47500a51c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563086e824a0 tx=0x7f4758009400 comp rx=0 tx=0).stop 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.905+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 msgr2=0x7f47680708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.905+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 0x7f47680708e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f475c02a960 tx=0x7f475c003cd0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.907+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 shutdown_connections 2024-05-09T09:21:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.907+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f47500a2d00 0x7f47500a51c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.907+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47681ac0f0 0x7f47681ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.907+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4768105420 0x7f47680708e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.907+0000 7f476e5a5640 1 --2- 172.21.15.64:0/3429805863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4768100100 0x7f47680703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.907+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 >> 172.21.15.64:0/3429805863 conn(0x7f47680fbd00 msgr2=0x7f4768100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.910+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 shutdown_connections 2024-05-09T09:21:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:10.910+0000 7f476e5a5640 1 -- 172.21.15.64:0/3429805863 wait complete. 2024-05-09T09:21:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:10 smithi179 ceph-mon[34086]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 9.4 MiB/s wr, 976 op/s 2024-05-09T09:21:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:21:10.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:10 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:21:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 650.0 s, 911.1 tps, lat 109.648 ms stddev 54.943 2024-05-09T09:21:11.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:11 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:21:11.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:11 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:21:11.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:11 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:21:12.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:12 smithi139 ceph-mon[33599]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 714 op/s 2024-05-09T09:21:12.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:12 smithi064 ceph-mon[30149]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 714 op/s 2024-05-09T09:21:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:12 smithi179 ceph-mon[34086]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 714 op/s 2024-05-09T09:21:14.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:14 smithi139 ceph-mon[33599]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.9 MiB/s wr, 834 op/s 2024-05-09T09:21:14.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:14 smithi064 ceph-mon[30149]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.9 MiB/s wr, 834 op/s 2024-05-09T09:21:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:14 smithi179 ceph-mon[34086]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.9 MiB/s wr, 834 op/s 2024-05-09T09:21:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 655.0 s, 762.6 tps, lat 131.569 ms stddev 47.094 2024-05-09T09:21:16.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:16 smithi139 ceph-mon[33599]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 793 op/s 2024-05-09T09:21:16.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:16 smithi064 ceph-mon[30149]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 793 op/s 2024-05-09T09:21:16.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:16 smithi179 ceph-mon[34086]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 793 op/s 2024-05-09T09:21:18.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:18 smithi139 ceph-mon[33599]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 709 op/s 2024-05-09T09:21:18.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:18 smithi064 ceph-mon[30149]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 709 op/s 2024-05-09T09:21:18.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:18 smithi179 ceph-mon[34086]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 709 op/s 2024-05-09T09:21:20.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:20 smithi139 ceph-mon[33599]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.3 MiB/s wr, 932 op/s 2024-05-09T09:21:20.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:20 smithi064 ceph-mon[30149]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.3 MiB/s wr, 932 op/s 2024-05-09T09:21:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:20 smithi179 ceph-mon[34086]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.3 MiB/s wr, 932 op/s 2024-05-09T09:21:21.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 660.0 s, 754.6 tps, lat 132.263 ms stddev 71.419 2024-05-09T09:21:22.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:22 smithi139 ceph-mon[33599]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 731 op/s 2024-05-09T09:21:22.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:22 smithi064 ceph-mon[30149]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 731 op/s 2024-05-09T09:21:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:22 smithi179 ceph-mon[34086]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 731 op/s 2024-05-09T09:21:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.073+0000 7fd171b16640 1 -- 172.21.15.64:0/3327822832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c109940 msgr2=0x7fd16c10bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.073+0000 7fd171b16640 1 --2- 172.21.15.64:0/3327822832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c109940 0x7fd16c10bd30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009900 tx=0x7fd15c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.074+0000 7fd171b16640 1 -- 172.21.15.64:0/3327822832 shutdown_connections 2024-05-09T09:21:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.074+0000 7fd171b16640 1 --2- 172.21.15.64:0/3327822832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c10ebf0 0x7fd16c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.074+0000 7fd171b16640 1 --2- 172.21.15.64:0/3327822832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c10e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.074+0000 7fd171b16640 1 --2- 172.21.15.64:0/3327822832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c109940 0x7fd16c10bd30 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009900 tx=0x7fd15c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.074+0000 7fd171b16640 1 -- 172.21.15.64:0/3327822832 >> 172.21.15.64:0/3327822832 conn(0x7fd16c06eea0 msgr2=0x7fd16c0712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.075+0000 7fd171b16640 1 -- 172.21.15.64:0/3327822832 shutdown_connections 2024-05-09T09:21:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.075+0000 7fd171b16640 1 -- 172.21.15.64:0/3327822832 wait complete. 2024-05-09T09:21:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 Processor -- start 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 -- start start 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c1a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c10ebf0 0x7fd16c1a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c1abfa0 0x7fd16c1ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1139f0 con 0x7fd16c1abfa0 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c113870 con 0x7fd16c10ebf0 2024-05-09T09:21:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.076+0000 7fd171b16640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c113b70 con 0x7fd16c10c270 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c1a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c1a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32974/0 (socket says 172.21.15.64:32974) 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 -- 172.21.15.64:0/341292373 learned_addr learned my addr 172.21.15.64:0/341292373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16bfff640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c1abfa0 0x7fd16c1ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 -- 172.21.15.64:0/341292373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c10ebf0 msgr2=0x7fd16c1a6f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c10ebf0 0x7fd16c1a6f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 -- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c1abfa0 msgr2=0x7fd16c1ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.077+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c1abfa0 0x7fd16c1ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd16b7fe640 1 -- 172.21.15.64:0/341292373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c1aea00 con 0x7fd16c10c270 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c1a6a00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd15c02a960 tx=0x7fd15c003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd16bfff640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c1abfa0 0x7fd16c1ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd168ff9640 1 -- 172.21.15.64:0/341292373 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c03f020 con 0x7fd16c10c270 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd168ff9640 1 -- 172.21.15.64:0/341292373 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd15c031c30 con 0x7fd16c10c270 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd168ff9640 1 -- 172.21.15.64:0/341292373 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c035070 con 0x7fd16c10c270 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.078+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c1aec90 con 0x7fd16c10c270 2024-05-09T09:21:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.079+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c1af100 con 0x7fd16c10c270 2024-05-09T09:21:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.083+0000 7fd168ff9640 1 -- 172.21.15.64:0/341292373 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd15c042020 con 0x7fd16c10c270 2024-05-09T09:21:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 Processor -- start 2024-05-09T09:21:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 -- start start 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfed6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6c40 0x55e5dfedb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfede240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e5dfe3a340 con 0x55e5dfed6c40 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e5dfde2290 con 0x55e5dfedbdb0 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.084+0000 7fd17481b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e5dfe36300 con 0x55e5dfed6250 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.085+0000 7fd16bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfed6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.085+0000 7fd16bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfed6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32980/0 (socket says 172.21.15.64:32980) 2024-05-09T09:21:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.085+0000 7fd16bfff640 1 -- 172.21.15.64:0/3379781116 learned_addr learned my addr 172.21.15.64:0/3379781116 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.085+0000 7fd168ff9640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1540a2df0 0x7fd1540a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.085+0000 7fd16affd640 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfede240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.085+0000 7fd168ff9640 1 -- 172.21.15.64:0/341292373 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd15c0efc30 con 0x7fd16c10c270 2024-05-09T09:21:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 -- 172.21.15.64:0/3379781116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 msgr2=0x55e5dfede240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfede240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 -- 172.21.15.64:0/3379781116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6c40 msgr2=0x55e5dfedb740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6c40 0x55e5dfedb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6c40 0x55e5dfedb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 -- 172.21.15.64:0/3379781116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dfe068e0 con 0x55e5dfed6250 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6c40 0x55e5dfedb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfed6670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd1640117e0 tx=0x7fd164011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0a85875b615e59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.086+0000 7fd16bfff640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1540a2df0 0x7fd1540a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.087+0000 7fd151ffb640 1 -- 172.21.15.64:0/3379781116 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164011290 con 0x55e5dfed6250 2024-05-09T09:21:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.087+0000 7fd151ffb640 1 -- 172.21.15.64:0/3379781116 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd164003e20 con 0x55e5dfed6250 2024-05-09T09:21:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.087+0000 7fd151ffb640 1 -- 172.21.15.64:0/3379781116 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164004610 con 0x55e5dfed6250 2024-05-09T09:21:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.088+0000 7fd16bfff640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1540a2df0 0x7fd1540a52b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd16401b000 tx=0x7fd164004890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.089+0000 7fd17481b740 1 -- 172.21.15.64:0/3379781116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 msgr2=0x55e5dfed6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.089+0000 7fd17481b740 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfed6670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd1640117e0 tx=0x7fd164011ca0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.090+0000 7fd17481b740 1 -- 172.21.15.64:0/3379781116 shutdown_connections 2024-05-09T09:21:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.090+0000 7fd17481b740 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfede240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.090+0000 7fd17481b740 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6c40 0x55e5dfedb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.090+0000 7fd17481b740 1 --2- 172.21.15.64:0/3379781116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfed6670 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.091+0000 7fd17481b740 1 -- 172.21.15.64:0/3379781116 >> 172.21.15.64:0/3379781116 conn(0x55e5dfed18c0 msgr2=0x55e5dfed1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.092+0000 7fd17481b740 1 -- 172.21.15.64:0/3379781116 shutdown_connections 2024-05-09T09:21:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.092+0000 7fd17481b740 1 -- 172.21.15.64:0/3379781116 wait complete. 2024-05-09T09:21:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.092+0000 7fd17481b740 1 -- start start 2024-05-09T09:21:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 Processor -- start 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6250 0x55e5dfee0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 0x55e5dfee09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e5dfeded60 con 0x55e5dfed6250 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e5dfdf5790 con 0x55e5dfedbdb0 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e5dfedeee0 con 0x55e5dfed6c40 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd17481b740 10 client.0 _ll_register_callbacks cb 0x55e5dfe30a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd16affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 0x55e5dfee09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd16affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 0x55e5dfee09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32990/0 (socket says 172.21.15.64:32990) 2024-05-09T09:21:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.093+0000 7fd16affd640 1 -- 172.21.15.64:0/3800593247 learned_addr learned my addr 172.21.15.64:0/3800593247 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6250 0x55e5dfee0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16affd640 1 -- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 msgr2=0x55e5dfee7a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16affd640 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee7a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16affd640 1 -- 172.21.15.64:0/3800593247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6250 msgr2=0x55e5dfee0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16affd640 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6250 0x55e5dfee0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16affd640 1 -- 172.21.15.64:0/3800593247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5dfb4e210 con 0x55e5dfed6c40 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd16affd640 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 0x55e5dfee09d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd160003850 tx=0x7fd160003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541e46334eae0245 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.094+0000 7fd14affd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.095+0000 7fd14affd640 1 -- 172.21.15.64:0/3800593247 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16001a470 con 0x55e5dfed6c40 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.095+0000 7fd14affd640 1 -- 172.21.15.64:0/3800593247 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd16001a610 con 0x55e5dfed6c40 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.095+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd164003e20 con 0x55e5dfed6c40 2024-05-09T09:21:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.095+0000 7fd14affd640 1 -- 172.21.15.64:0/3800593247 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd160023050 con 0x55e5dfed6c40 2024-05-09T09:21:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.095+0000 7fd17481b740 10 client.28057 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.095+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e5dfc1a620 con 0x55e5dfed6c40 2024-05-09T09:21:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd14affd640 1 -- 172.21.15.64:0/3800593247 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd164003e20 con 0x55e5dfed6c40 2024-05-09T09:21:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd14affd640 1 -- 172.21.15.64:0/3800593247 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7fd160011910 con 0x55e5dfed6c40 2024-05-09T09:21:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd17481b740 10 client.28057 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd17481b740 10 client.28057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd17481b740 20 client.28057 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd17481b740 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dfeec340 0x55e5dfeee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd17481b740 4 client.28057 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.096+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5dfe47a70 con 0x55e5dfeec340 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.097+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dfeec340 0x55e5dfeee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.097+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dfeec340 0x55e5dfeee800 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b861e4f2e4775190 server_cookie=591b51dbbcca9f98 in_seq=0 out_seq=0 2024-05-09T09:21:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.097+0000 7fd14affd640 10 client.28057 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.098+0000 7fd14affd640 1 -- 172.21.15.64:0/3800593247 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5dfe47a70 con 0x55e5dfeec340 2024-05-09T09:21:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.098+0000 7fd14affd640 10 client.28057 handle_command_reply: tid=0 2024-05-09T09:21:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.098+0000 7fd17481b740 1 client.28057 shutdown 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.098+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 msgr2=0x55e5dfee09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.098+0000 7fd17481b740 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 0x55e5dfee09d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd160003850 tx=0x7fd160003d10 comp rx=0 tx=0).stop 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 shutdown_connections 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dfeec340 0x55e5dfeee800 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6c40 0x55e5dfee09d0 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd160003850 tx=0x7fd160003d10 comp rx=0 tx=0).stop 2024-05-09T09:21:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 --2- 172.21.15.64:0/3800593247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfed6250 0x55e5dfee0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 >> 172.21.15.64:0/3800593247 conn(0x55e5dfed18c0 msgr2=0x55e5dfed0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd14affd640 0 client.28057 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 shutdown_connections 2024-05-09T09:21:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 1 -- 172.21.15.64:0/3800593247 wait complete. 2024-05-09T09:21:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.099+0000 7fd17481b740 20 client.28057 trim_cache size 0 max 16384 2024-05-09T09:21:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.141+0000 7fd17481b740 1 Processor -- start 2024-05-09T09:21:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.141+0000 7fd17481b740 1 -- start start 2024-05-09T09:21:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.141+0000 7fd17481b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 0x55e5dfef4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.141+0000 7fd17481b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfedbdb0 0x55e5dfef4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.141+0000 7fd17481b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfed7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd16b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 0x55e5dfef4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd16b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 0x55e5dfef4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:52124/0 (socket says 172.21.15.64:52124) 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd16b7fe640 1 -- 172.21.15.64:0/3955702134 learned_addr learned my addr 172.21.15.64:0/3955702134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1bc0d0 con 0x55e5dfee79d0 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1b9970 con 0x55e5dfed6250 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1b77c0 con 0x55e5dfedbdb0 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.142+0000 7fd16bfff640 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfed7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd16b7fe640 1 -- 172.21.15.64:0/3955702134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfedbdb0 msgr2=0x55e5dfef4650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfedbdb0 0x55e5dfef4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd16b7fe640 1 -- 172.21.15.64:0/3955702134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 msgr2=0x55e5dfed7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfed7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd16b7fe640 1 -- 172.21.15.64:0/3955702134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dfecff00 con 0x55e5dfed6250 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 0x55e5dfef4110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd15c02a9b0 tx=0x7fd15c0ed270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39e80c487b99254 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.143+0000 7fd1517fa640 1 -- 172.21.15.64:0/3955702134 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0c04b0 con 0x55e5dfed6250 2024-05-09T09:21:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.144+0000 7fd1517fa640 1 -- 172.21.15.64:0/3955702134 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd15c0c4d70 con 0x55e5dfed6250 2024-05-09T09:21:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 msgr2=0x55e5dfef4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 0x55e5dfef4110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd15c02a9b0 tx=0x7fd15c0ed270 comp rx=0 tx=0).stop 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 shutdown_connections 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfed7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfedbdb0 0x55e5dfef4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 --2- 172.21.15.64:0/3955702134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfed6250 0x55e5dfef4110 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.145+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 >> 172.21.15.64:0/3955702134 conn(0x55e5dfed18c0 msgr2=0x55e5dfee0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.146+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 shutdown_connections 2024-05-09T09:21:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.146+0000 7fd17481b740 1 -- 172.21.15.64:0/3955702134 wait complete. 2024-05-09T09:21:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.147+0000 7fd17481b740 1 -- start start 2024-05-09T09:21:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.147+0000 7fd17481b740 1 Processor -- start 2024-05-09T09:21:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd17481b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfee4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd17481b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd16b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfee4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd16b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfee4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:32996/0 (socket says 172.21.15.64:32996) 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd16b7fe640 1 -- 172.21.15.64:0/910120591 learned_addr learned my addr 172.21.15.64:0/910120591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfee6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1b7e30 con 0x55e5dfee79d0 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1b92a0 con 0x55e5dfedbdb0 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1b8500 con 0x55e5dfed6250 2024-05-09T09:21:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.148+0000 7fd16affd640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd17481b740 10 client.0 _ll_register_callbacks cb 0x7fd160004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16bfff640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfee6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16b7fe640 1 -- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 msgr2=0x55e5dfee5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16b7fe640 1 -- 172.21.15.64:0/910120591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 msgr2=0x55e5dfee6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfee6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16b7fe640 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5dfee6680 con 0x55e5dfed6250 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd16b7fe640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfee4c50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0bc090 tx=0x7fd15c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9459ce0ad809d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.149+0000 7fd14affd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.150+0000 7fd14affd640 1 -- 172.21.15.64:0/910120591 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0d4070 con 0x55e5dfed6250 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.150+0000 7fd14affd640 1 -- 172.21.15.64:0/910120591 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd15c0b8770 con 0x55e5dfed6250 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.150+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5dfe41dc0 con 0x55e5dfed6250 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.150+0000 7fd14affd640 1 -- 172.21.15.64:0/910120591 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd15c0c04b0 con 0x55e5dfed6250 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.150+0000 7fd17481b740 10 client.28060 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.150+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e5dff529f0 con 0x55e5dfed6250 2024-05-09T09:21:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd14affd640 1 -- 172.21.15.64:0/910120591 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd15c0c4d70 con 0x55e5dfed6250 2024-05-09T09:21:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd14affd640 1 -- 172.21.15.64:0/910120591 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7fd15c0bb550 con 0x55e5dfed6250 2024-05-09T09:21:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd17481b740 10 client.28060 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd17481b740 10 client.28060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd17481b740 20 client.28060 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dff531b0 0x55e5dff555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd17481b740 4 client.28060 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:21:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.151+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5dfe41dc0 con 0x55e5dff531b0 2024-05-09T09:21:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.152+0000 7fd16affd640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dff531b0 0x55e5dff555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.153+0000 7fd16affd640 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dff531b0 0x55e5dff555d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11c89538c88d39f server_cookie=ec2fe7f0c98ce16c in_seq=0 out_seq=0 2024-05-09T09:21:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.153+0000 7fd14affd640 10 client.28060 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.181+0000 7fd14affd640 1 -- 172.21.15.64:0/910120591 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5dfe41dc0 con 0x55e5dff531b0 2024-05-09T09:21:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.182+0000 7fd14affd640 10 client.28060 handle_command_reply: tid=0 2024-05-09T09:21:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.182+0000 7fd17481b740 1 client.28060 shutdown 2024-05-09T09:21:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.182+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 msgr2=0x55e5dfee4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.182+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfee4c50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0bc090 tx=0x7fd15c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 shutdown_connections 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e5dff531b0 0x55e5dff555d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd14affd640 0 client.28060 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5dfee79d0 0x55e5dfee6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5dfedbdb0 0x55e5dfee5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 --2- 172.21.15.64:0/910120591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5dfed6250 0x55e5dfee4c50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 >> 172.21.15.64:0/910120591 conn(0x55e5dfed18c0 msgr2=0x55e5dfed7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.183+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 shutdown_connections 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.184+0000 7fd17481b740 1 -- 172.21.15.64:0/910120591 wait complete. 2024-05-09T09:21:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.184+0000 7fd17481b740 20 client.28060 trim_cache size 0 max 16384 2024-05-09T09:21:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:21:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:21:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:21:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:21:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.191+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1540a2df0 msgr2=0x7fd1540a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.191+0000 7fd171b16640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1540a2df0 0x7fd1540a52b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd16401b000 tx=0x7fd164004890 comp rx=0 tx=0).stop 2024-05-09T09:21:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.191+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 msgr2=0x7fd16c1a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.191+0000 7fd171b16640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c1a6a00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd15c02a960 tx=0x7fd15c003c00 comp rx=0 tx=0).stop 2024-05-09T09:21:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.193+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 shutdown_connections 2024-05-09T09:21:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.193+0000 7fd171b16640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1540a2df0 0x7fd1540a52b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.193+0000 7fd171b16640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd16c1abfa0 0x7fd16c1ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.193+0000 7fd171b16640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16c10ebf0 0x7fd16c1a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.193+0000 7fd171b16640 1 --2- 172.21.15.64:0/341292373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c10c270 0x7fd16c1a6a00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:24.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.193+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 >> 172.21.15.64:0/341292373 conn(0x7fd16c06eea0 msgr2=0x7fd16c10a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.194+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 shutdown_connections 2024-05-09T09:21:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:24.194+0000 7fd171b16640 1 -- 172.21.15.64:0/341292373 wait complete. 2024-05-09T09:21:24.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:24 smithi139 ceph-mon[33599]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.6 MiB/s wr, 832 op/s 2024-05-09T09:21:24.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:24 smithi064 ceph-mon[30149]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.6 MiB/s wr, 832 op/s 2024-05-09T09:21:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:24 smithi179 ceph-mon[34086]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.6 MiB/s wr, 832 op/s 2024-05-09T09:21:25.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:21:25.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:21:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.720+0000 7f4dae06d640 1 -- 172.21.15.64:0/3920682860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da81011e0 msgr2=0x7f4da8101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.720+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3920682860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da81011e0 0x7f4da8101a80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009900 tx=0x7f4d9802a440 comp rx=0 tx=0).stop 2024-05-09T09:21:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.722+0000 7f4dae06d640 1 -- 172.21.15.64:0/3920682860 shutdown_connections 2024-05-09T09:21:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.722+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3920682860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da81011e0 0x7f4da8101a80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.722+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3920682860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8100840 0x7f4da8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.722+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3920682860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da80ff710 0x7f4da80ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.722+0000 7f4dae06d640 1 -- 172.21.15.64:0/3920682860 >> 172.21.15.64:0/3920682860 conn(0x7f4da80fb030 msgr2=0x7f4da80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.724+0000 7f4dae06d640 1 -- 172.21.15.64:0/3920682860 shutdown_connections 2024-05-09T09:21:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.725+0000 7f4dae06d640 1 -- 172.21.15.64:0/3920682860 wait complete. 2024-05-09T09:21:25.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 Processor -- start 2024-05-09T09:21:25.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 -- start start 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da80ff710 0x7f4da8199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da8100840 0x7f4da819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 0x7f4da81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4da810f6c0 con 0x7f4da81011e0 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4da810f540 con 0x7f4da8100840 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.726+0000 7f4dae06d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4da810f840 con 0x7f4da80ff710 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.727+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 0x7f4da81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.727+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da8100840 0x7f4da819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.727+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 0x7f4da81a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38738/0 (socket says 172.21.15.64:38738) 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.727+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da8100840 0x7f4da819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51504/0 (socket says 172.21.15.64:51504) 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.727+0000 7f4da7fff640 1 -- 172.21.15.64:0/3457452488 learned_addr learned my addr 172.21.15.64:0/3457452488 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.727+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da80ff710 0x7f4da8199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.728+0000 7f4da7fff640 1 -- 172.21.15.64:0/3457452488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da80ff710 msgr2=0x7f4da8199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.728+0000 7f4da7fff640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da80ff710 0x7f4da8199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.728+0000 7f4da7fff640 1 -- 172.21.15.64:0/3457452488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da8100840 msgr2=0x7f4da819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.728+0000 7f4da7fff640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da8100840 0x7f4da819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.728+0000 7f4da7fff640 1 -- 172.21.15.64:0/3457452488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da81a1a20 con 0x7f4da81011e0 2024-05-09T09:21:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.732+0000 7f4da7fff640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 0x7f4da81a13b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4d9802a960 tx=0x7f4d98003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.733+0000 7f4da4ff9640 1 -- 172.21.15.64:0/3457452488 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9803f070 con 0x7f4da81011e0 2024-05-09T09:21:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.733+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da81a1cb0 con 0x7f4da81011e0 2024-05-09T09:21:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.733+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81a2220 con 0x7f4da81011e0 2024-05-09T09:21:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.733+0000 7f4da4ff9640 1 -- 172.21.15.64:0/3457452488 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d98031e40 con 0x7f4da81011e0 2024-05-09T09:21:25.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.734+0000 7f4da4ff9640 1 -- 172.21.15.64:0/3457452488 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98035030 con 0x7f4da81011e0 2024-05-09T09:21:25.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.739+0000 7f4db0d76740 1 Processor -- start 2024-05-09T09:21:25.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.739+0000 7f4da4ff9640 1 -- 172.21.15.64:0/3457452488 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d98042070 con 0x7f4da81011e0 2024-05-09T09:21:25.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.739+0000 7f4db0d76740 1 -- start start 2024-05-09T09:21:25.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f914b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f919c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f91c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4db0d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56320f870d00 con 0x56320f915130 2024-05-09T09:21:25.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4db0d76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56320f8184c0 con 0x56320f91a2a0 2024-05-09T09:21:25.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4db0d76740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56320f86ccc0 con 0x56320f914740 2024-05-09T09:21:25.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f914b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f914b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56984/0 (socket says 172.21.15.64:56984) 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 -- 172.21.15.64:0/1101606697 learned_addr learned my addr 172.21.15.64:0/1101606697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f91c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da77fe640 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f919c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 -- 172.21.15.64:0/1101606697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 msgr2=0x56320f91c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f91c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.740+0000 7f4da6ffd640 1 -- 172.21.15.64:0/1101606697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 msgr2=0x56320f919c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.741+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f919c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.741+0000 7f4da6ffd640 1 -- 172.21.15.64:0/1101606697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320f64e7d0 con 0x56320f914740 2024-05-09T09:21:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.741+0000 7f4da4ff9640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d900a2b70 0x7f4d900a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.741+0000 7f4da77fe640 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f919c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.741+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f914b60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c006aa0 tx=0x7f4d9c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b54f9e02282dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.741+0000 7f4da4ff9640 1 -- 172.21.15.64:0/3457452488 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4d980efb80 con 0x7f4da81011e0 2024-05-09T09:21:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.742+0000 7f4d8dffb640 1 -- 172.21.15.64:0/1101606697 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c015760 con 0x56320f914740 2024-05-09T09:21:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.742+0000 7f4d8dffb640 1 -- 172.21.15.64:0/1101606697 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d9c01d070 con 0x56320f914740 2024-05-09T09:21:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.742+0000 7f4d8dffb640 1 -- 172.21.15.64:0/1101606697 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c018ab0 con 0x56320f914740 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.742+0000 7f4da77fe640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d900a2b70 0x7f4d900a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 -- 172.21.15.64:0/1101606697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 msgr2=0x56320f914b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f914b60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c006aa0 tx=0x7f4d9c009510 comp rx=0 tx=0).stop 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 -- 172.21.15.64:0/1101606697 shutdown_connections 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f91c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f919c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 --2- 172.21.15.64:0/1101606697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f914b60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 -- 172.21.15.64:0/1101606697 >> 172.21.15.64:0/1101606697 conn(0x56320f90fdb0 msgr2=0x56320f9101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 -- 172.21.15.64:0/1101606697 shutdown_connections 2024-05-09T09:21:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.743+0000 7f4db0d76740 1 -- 172.21.15.64:0/1101606697 wait complete. 2024-05-09T09:21:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.744+0000 7f4db0d76740 1 -- start start 2024-05-09T09:21:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.744+0000 7f4db0d76740 1 Processor -- start 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.744+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f91ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f91f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f9261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d900a2b70 0x7f4d900a5030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x56320f910c80 tx=0x7f4da000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4db0d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56320f91d1f0 con 0x56320f915130 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f91ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4db0d76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56320f82bb90 con 0x56320f91a2a0 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4db0d76740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56320f91d370 con 0x56320f914740 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f91ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56988/0 (socket says 172.21.15.64:56988) 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 -- 172.21.15.64:0/24907324 learned_addr learned my addr 172.21.15.64:0/24907324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da7fff640 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f9261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4db0d76740 10 client.0 _ll_register_callbacks cb 0x56320f867da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 -- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 msgr2=0x56320f9261a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f9261a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 -- 172.21.15.64:0/24907324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 msgr2=0x56320f91f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f91f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 -- 172.21.15.64:0/24907324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320f5843f0 con 0x56320f914740 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.745+0000 7f4da77fe640 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f91ec20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4da0007e00 tx=0x7f4da0004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e36dfb8107d007 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.746+0000 7f4d86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.746+0000 7f4d86ffd640 1 -- 172.21.15.64:0/24907324 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0027060 con 0x56320f914740 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.746+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9c01d070 con 0x56320f914740 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.746+0000 7f4d86ffd640 1 -- 172.21.15.64:0/24907324 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4da0022470 con 0x56320f914740 2024-05-09T09:21:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.747+0000 7f4d86ffd640 1 -- 172.21.15.64:0/24907324 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4da002e050 con 0x56320f914740 2024-05-09T09:21:25.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.747+0000 7f4db0d76740 10 client.28072 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:25.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.747+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56320f5440f0 con 0x56320f914740 2024-05-09T09:21:25.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4d86ffd640 1 -- 172.21.15.64:0/24907324 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da0022b20 con 0x56320f914740 2024-05-09T09:21:25.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4d86ffd640 1 -- 172.21.15.64:0/24907324 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f4da0016070 con 0x56320f914740 2024-05-09T09:21:25.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4db0d76740 10 client.28072 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:25.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4db0d76740 10 client.28072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4db0d76740 20 client.28072 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4db0d76740 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f92ab60 0x56320f92cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4db0d76740 4 client.28072 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.748+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d9c01d070 con 0x56320f92ab60 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.749+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f92ab60 0x56320f92cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.749+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f92ab60 0x56320f92cfe0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbee05df200c989 server_cookie=a7c5c0e930e3b7a2 in_seq=0 out_seq=0 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.749+0000 7f4d86ffd640 10 client.28072 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.750+0000 7f4d86ffd640 1 -- 172.21.15.64:0/24907324 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4d9c01d070 con 0x56320f92ab60 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.750+0000 7f4d86ffd640 10 client.28072 handle_command_reply: tid=0 2024-05-09T09:21:25.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.750+0000 7f4db0d76740 1 client.28072 shutdown 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.750+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 msgr2=0x56320f91ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.750+0000 7f4db0d76740 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f91ec20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4da0007e00 tx=0x7f4da0004a60 comp rx=0 tx=0).stop 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 shutdown_connections 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f92ab60 0x56320f92cfe0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f9261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f91f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 --2- 172.21.15.64:0/24907324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f91ec20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 >> 172.21.15.64:0/24907324 conn(0x56320f90fdb0 msgr2=0x56320f9067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4d86ffd640 0 client.28072 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 shutdown_connections 2024-05-09T09:21:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 1 -- 172.21.15.64:0/24907324 wait complete. 2024-05-09T09:21:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.751+0000 7f4db0d76740 20 client.28072 trim_cache size 0 max 16384 2024-05-09T09:21:25.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.784+0000 7f4db0d76740 1 Processor -- start 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.785+0000 7f4db0d76740 1 -- start start 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.785+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f9165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.785+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f916b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.785+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f926160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4db0d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94012590 con 0x56320f915130 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4db0d76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400fe30 con 0x56320f91a2a0 2024-05-09T09:21:25.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f926160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f926160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51528/0 (socket says 172.21.15.64:51528) 2024-05-09T09:21:25.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4da7fff640 1 -- 172.21.15.64:0/2286695233 learned_addr learned my addr 172.21.15.64:0/2286695233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:25.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4db0d76740 1 -- 172.21.15.64:0/2286695233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400dc80 con 0x56320f914740 2024-05-09T09:21:25.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.786+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f916b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.787+0000 7f4da77fe640 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f9165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.787+0000 7f4da7fff640 1 -- 172.21.15.64:0/2286695233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 msgr2=0x56320f9165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.787+0000 7f4da7fff640 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f9165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.787+0000 7f4da7fff640 1 -- 172.21.15.64:0/2286695233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 msgr2=0x56320f916b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.787+0000 7f4da7fff640 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f916b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.787+0000 7f4da7fff640 1 -- 172.21.15.64:0/2286695233 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320f90fbe0 con 0x56320f91a2a0 2024-05-09T09:21:25.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.789+0000 7f4da7fff640 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f926160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d980b8450 tx=0x7f4d98031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a7c91bec6bb54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:25.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.791+0000 7f4d8d7fa640 1 -- 172.21.15.64:0/2286695233 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98049040 con 0x56320f91a2a0 2024-05-09T09:21:25.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.791+0000 7f4d8d7fa640 1 -- 172.21.15.64:0/2286695233 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d980bbff0 con 0x56320f91a2a0 2024-05-09T09:21:25.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.791+0000 7f4d8d7fa640 1 -- 172.21.15.64:0/2286695233 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9803b6c0 con 0x56320f91a2a0 2024-05-09T09:21:25.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 -- 172.21.15.64:0/2286695233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 msgr2=0x56320f926160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f926160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d980b8450 tx=0x7f4d98031400 comp rx=0 tx=0).stop 2024-05-09T09:21:25.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 -- 172.21.15.64:0/2286695233 shutdown_connections 2024-05-09T09:21:25.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f926160 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f916b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 --2- 172.21.15.64:0/2286695233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f9165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.792+0000 7f4db0d76740 1 -- 172.21.15.64:0/2286695233 >> 172.21.15.64:0/2286695233 conn(0x56320f90fdb0 msgr2=0x56320f91ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.793+0000 7f4db0d76740 1 -- 172.21.15.64:0/2286695233 shutdown_connections 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.793+0000 7f4db0d76740 1 -- 172.21.15.64:0/2286695233 wait complete. 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.793+0000 7f4db0d76740 1 -- start start 2024-05-09T09:21:25.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.793+0000 7f4db0d76740 1 Processor -- start 2024-05-09T09:21:25.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f923920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f923e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f924e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400e2f0 con 0x56320f915130 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400f760 con 0x56320f91a2a0 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400e9c0 con 0x56320f914740 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4db0d76740 10 client.0 _ll_register_callbacks cb 0x7f4da0009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f923920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f923920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57008/0 (socket says 172.21.15.64:57008) 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4da77fe640 1 -- 172.21.15.64:0/209332536 learned_addr learned my addr 172.21.15.64:0/209332536 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f923e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.794+0000 7f4da7fff640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f924e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da77fe640 1 -- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 msgr2=0x56320f924e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da77fe640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f924e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da77fe640 1 -- 172.21.15.64:0/209332536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 msgr2=0x56320f923e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da77fe640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f923e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da77fe640 1 -- 172.21.15.64:0/209332536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320f925460 con 0x56320f914740 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da7fff640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f924e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4da77fe640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f923920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4da0004b10 tx=0x7f4da000a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a33e55a015fb352 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4d86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:25.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4d86ffd640 1 -- 172.21.15.64:0/209332536 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0036030 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d980bbff0 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.795+0000 7f4d86ffd640 1 -- 172.21.15.64:0/209332536 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4da001e040 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4d86ffd640 1 -- 172.21.15.64:0/209332536 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4da0040050 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 10 client.28075 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56320f9911c0 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4d86ffd640 1 -- 172.21.15.64:0/209332536 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d980bbff0 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4d86ffd640 1 -- 172.21.15.64:0/209332536 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f4da0016040 con 0x56320f914740 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 10 client.28075 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 10 client.28075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 20 client.28075 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f991ba0 0x56320f993fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:25.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 4 client.28075 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:21:25.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.796+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56320f87e430 con 0x56320f991ba0 2024-05-09T09:21:25.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.797+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f991ba0 0x56320f993fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:25.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4da6ffd640 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f991ba0 0x56320f993fc0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449c069e2c7e10a5 server_cookie=dc6fd4d8f2420454 in_seq=0 out_seq=0 2024-05-09T09:21:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4d86ffd640 10 client.28075 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4d86ffd640 1 -- 172.21.15.64:0/209332536 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56320f87e430 con 0x56320f991ba0 2024-05-09T09:21:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4d86ffd640 10 client.28075 handle_command_reply: tid=0 2024-05-09T09:21:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4db0d76740 1 client.28075 shutdown 2024-05-09T09:21:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 msgr2=0x56320f923920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.799+0000 7f4db0d76740 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f923920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4da0004b10 tx=0x7f4da000a510 comp rx=0 tx=0).stop 2024-05-09T09:21:25.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 shutdown_connections 2024-05-09T09:21:25.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56320f991ba0 0x56320f993fc0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56320f91a2a0 0x56320f924e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56320f915130 0x56320f923e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 --2- 172.21.15.64:0/209332536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56320f914740 0x56320f923920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 >> 172.21.15.64:0/209332536 conn(0x56320f90fdb0 msgr2=0x56320f91ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4d86ffd640 0 client.28075 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.805+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 shutdown_connections 2024-05-09T09:21:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.806+0000 7f4db0d76740 1 -- 172.21.15.64:0/209332536 wait complete. 2024-05-09T09:21:25.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.806+0000 7f4db0d76740 20 client.28075 trim_cache size 0 max 16384 2024-05-09T09:21:25.807 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:21:25.807 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:21:25.807 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:21:25.807 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.807+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d900a2b70 msgr2=0x7f4d900a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.807+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d900a2b70 0x7f4d900a5030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x56320f910c80 tx=0x7f4da000e040 comp rx=0 tx=0).stop 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 msgr2=0x7f4da81a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 0x7f4da81a13b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4d9802a960 tx=0x7f4d98003cd0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 shutdown_connections 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4d900a2b70 0x7f4d900a5030 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81011e0 0x7f4da81a13b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da8100840 0x7f4da819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 --2- 172.21.15.64:0/3457452488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4da80ff710 0x7f4da8199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 >> 172.21.15.64:0/3457452488 conn(0x7f4da80fb030 msgr2=0x7f4da80fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.808+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 shutdown_connections 2024-05-09T09:21:25.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:25.809+0000 7f4dae06d640 1 -- 172.21.15.64:0/3457452488 wait complete. 2024-05-09T09:21:25.829 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:21:25.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:21:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.031+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/1451304022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24102320 msgr2=0x7f5d24108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.031+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/1451304022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24102320 0x7f5d24108bb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00d910 tx=0x7f5d1c02e5c0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.032+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/1451304022 shutdown_connections 2024-05-09T09:21:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.032+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/1451304022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24102320 0x7f5d24108bb0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.032+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/1451304022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d24101960 0x7f5d24101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.032+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/1451304022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24100770 0x7f5d24100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.032+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/1451304022 >> 172.21.15.64:0/1451304022 conn(0x7f5d240fbce0 msgr2=0x7f5d240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.032+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/1451304022 shutdown_connections 2024-05-09T09:21:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.033+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/1451304022 wait complete. 2024-05-09T09:21:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.034+0000 7f5d2a1b6640 1 Processor -- start 2024-05-09T09:21:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.034+0000 7f5d2a1b6640 1 -- start start 2024-05-09T09:21:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.034+0000 7f5d2a1b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24100770 0x7f5d240702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.034+0000 7f5d2a1b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d24101960 0x7f5d24070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d2a1b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 0x7f5d241ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d2a1b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24113ab0 con 0x7f5d24100770 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d2a1b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24113930 con 0x7f5d24101960 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d2a1b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24113c30 con 0x7f5d24102320 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d23fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 0x7f5d241ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d23fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 0x7f5d241ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57012/0 (socket says 172.21.15.64:57012) 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.035+0000 7f5d23fff640 1 -- 172.21.15.64:0/2964763498 learned_addr learned my addr 172.21.15.64:0/2964763498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d23fff640 1 -- 172.21.15.64:0/2964763498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d24101960 msgr2=0x7f5d24070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d23fff640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d24101960 0x7f5d24070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d23fff640 1 -- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24100770 msgr2=0x7f5d240702c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:21:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d23fff640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24100770 0x7f5d240702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d23fff640 1 -- 172.21.15.64:0/2964763498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d241ae700 con 0x7f5d24102320 2024-05-09T09:21:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d23fff640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 0x7f5d241ae1c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006010 tx=0x7f5d1c00c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.036+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c043070 con 0x7f5d24102320 2024-05-09T09:21:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.037+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d1c0048c0 con 0x7f5d24102320 2024-05-09T09:21:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.037+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c004030 con 0x7f5d24102320 2024-05-09T09:21:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.037+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d241ae8d0 con 0x7f5d24102320 2024-05-09T09:21:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.037+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d241aee50 con 0x7f5d24102320 2024-05-09T09:21:26.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.039+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d24107b80 con 0x7f5d24102320 2024-05-09T09:21:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.041+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d1c046070 con 0x7f5d24102320 2024-05-09T09:21:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.043+0000 7f5d20ff9640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d0c0a2ce0 0x7f5d0c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.043+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5d1c0ee870 con 0x7f5d24102320 2024-05-09T09:21:26.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.044+0000 7f5d237fe640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d0c0a2ce0 0x7f5d0c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.045+0000 7f5d237fe640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d0c0a2ce0 0x7f5d0c0a51a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d241017c0 tx=0x7f5d14009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.049+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d1c0bb3e0 con 0x7f5d24102320 2024-05-09T09:21:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 665.0 s, 749.4 tps, lat 133.184 ms stddev 67.993 2024-05-09T09:21:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.525+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d24062430 con 0x7f5d24102320 2024-05-09T09:21:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.527+0000 7f5d20ff9640 1 -- 172.21.15.64:0/2964763498 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11880 (secure 0 0 0) 0x7f5d1c0bace0 con 0x7f5d24102320 2024-05-09T09:21:26.528 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:21:26.528 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:20:50.564954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:21:26.529 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2024-05-09T09:21:26.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d0c0a2ce0 msgr2=0x7f5d0c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d0c0a2ce0 0x7f5d0c0a51a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d241017c0 tx=0x7f5d14009230 comp rx=0 tx=0).stop 2024-05-09T09:21:26.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 msgr2=0x7f5d241ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 0x7f5d241ae1c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006010 tx=0x7f5d1c00c460 comp rx=0 tx=0).stop 2024-05-09T09:21:26.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 shutdown_connections 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5d0c0a2ce0 0x7f5d0c0a51a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d24102320 0x7f5d241ae1c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d24101960 0x7f5d24070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 --2- 172.21.15.64:0/2964763498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d24100770 0x7f5d240702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 >> 172.21.15.64:0/2964763498 conn(0x7f5d240fbce0 msgr2=0x7f5d24104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.532+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 shutdown_connections 2024-05-09T09:21:26.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.533+0000 7f5d2a1b6640 1 -- 172.21.15.64:0/2964763498 wait complete. 2024-05-09T09:21:26.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:21:26.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.748+0000 7f8d01ffb640 1 -- 172.21.15.64:0/1987806059 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8030850 con 0x7f8d04105410 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.749+0000 7f8d097b0640 1 -- 172.21.15.64:0/1987806059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04105410 msgr2=0x7f8d04105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.749+0000 7f8d097b0640 1 --2- 172.21.15.64:0/1987806059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04105410 0x7f8d04105890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009860 tx=0x7f8cf802a340 comp rx=0 tx=0).stop 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.749+0000 7f8d097b0640 1 -- 172.21.15.64:0/1987806059 shutdown_connections 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.749+0000 7f8d097b0640 1 --2- 172.21.15.64:0/1987806059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04105410 0x7f8d04105890 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.749+0000 7f8d097b0640 1 --2- 172.21.15.64:0/1987806059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04102a60 0x7f8d04104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.749+0000 7f8d097b0640 1 --2- 172.21.15.64:0/1987806059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d041000f0 0x7f8d04102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.750+0000 7f8d097b0640 1 -- 172.21.15.64:0/1987806059 >> 172.21.15.64:0/1987806059 conn(0x7f8d040fbcf0 msgr2=0x7f8d040fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.750+0000 7f8d097b0640 1 -- 172.21.15.64:0/1987806059 shutdown_connections 2024-05-09T09:21:26.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.750+0000 7f8d097b0640 1 -- 172.21.15.64:0/1987806059 wait complete. 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 Processor -- start 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 -- start start 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d041000f0 0x7f8d04070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04102a60 0x7f8d04071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 0x7f8d04072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0406b3d0 con 0x7f8d041000f0 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0406b250 con 0x7f8d04105410 2024-05-09T09:21:26.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.751+0000 7f8d097b0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0406b550 con 0x7f8d04102a60 2024-05-09T09:21:26.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.752+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 0x7f8d04072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.752+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 0x7f8d04072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51568/0 (socket says 172.21.15.64:51568) 2024-05-09T09:21:26.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.752+0000 7f8d037fe640 1 -- 172.21.15.64:0/2444485576 learned_addr learned my addr 172.21.15.64:0/2444485576 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:26.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.752+0000 7f8d027fc640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04102a60 0x7f8d04071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.752+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d041000f0 0x7f8d04070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.752+0000 7f8d027fc640 1 -- 172.21.15.64:0/2444485576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 msgr2=0x7f8d04072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.753+0000 7f8d027fc640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 0x7f8d04072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.753+0000 7f8d027fc640 1 -- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d041000f0 msgr2=0x7f8d04070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.753+0000 7f8d027fc640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d041000f0 0x7f8d04070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.753+0000 7f8d027fc640 1 -- 172.21.15.64:0/2444485576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d040729d0 con 0x7f8d04102a60 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.753+0000 7f8d037fe640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 0x7f8d04072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:26.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.753+0000 7f8d027fc640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04102a60 0x7f8d04071320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00ef60 tx=0x7f8cfc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.754+0000 7f8cebfff640 1 -- 172.21.15.64:0/2444485576 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc01b070 con 0x7f8d04102a60 2024-05-09T09:21:26.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.754+0000 7f8cebfff640 1 -- 172.21.15.64:0/2444485576 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc00ba20 con 0x7f8d04102a60 2024-05-09T09:21:26.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.754+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d04072cc0 con 0x7f8d04102a60 2024-05-09T09:21:26.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.754+0000 7f8cebfff640 1 -- 172.21.15.64:0/2444485576 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc016b20 con 0x7f8d04102a60 2024-05-09T09:21:26.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.754+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d04073210 con 0x7f8d04102a60 2024-05-09T09:21:26.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.757+0000 7f8d0c59e740 1 Processor -- start 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 -- start start 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85dff40 0x5609b85e0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 0x5609b85e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5609b8544c40 con 0x5609b85e5aa0 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5609b84ec580 con 0x5609b85dff40 2024-05-09T09:21:26.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.758+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5609b8540c00 con 0x5609b85e0930 2024-05-09T09:21:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.759+0000 7f8cebfff640 1 -- 172.21.15.64:0/2444485576 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8cfc01e070 con 0x7f8d04102a60 2024-05-09T09:21:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.759+0000 7f8d02ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 0x5609b85e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.760+0000 7f8d02ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 0x5609b85e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57034/0 (socket says 172.21.15.64:57034) 2024-05-09T09:21:26.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.760+0000 7f8d02ffd640 1 -- 172.21.15.64:0/3870911470 learned_addr learned my addr 172.21.15.64:0/3870911470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.760+0000 7f8d027fc640 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8cebfff640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8cec0b3da0 0x7f8cec0b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8cebfff640 1 -- 172.21.15.64:0/2444485576 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8cfc0cf180 con 0x7f8d04102a60 2024-05-09T09:21:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8d027fc640 1 -- 172.21.15.64:0/3870911470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 msgr2=0x5609b85e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8d027fc640 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 0x5609b85e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8d027fc640 1 -- 172.21.15.64:0/3870911470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85dff40 msgr2=0x5609b85e0360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8d027fc640 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85dff40 0x5609b85e0360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8d027fc640 1 -- 172.21.15.64:0/3870911470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609b82583f0 con 0x5609b85e5aa0 2024-05-09T09:21:26.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.761+0000 7f8d027fc640 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85e7f30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc09b3a0 tx=0x7f8cfc09b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319a51694e453b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.762+0000 7f8d037fe640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8cec0b3da0 0x7f8cec0b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.762+0000 7f8ce97fa640 1 -- 172.21.15.64:0/3870911470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc09f9a0 con 0x5609b85e5aa0 2024-05-09T09:21:26.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.762+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 0x5609b85e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:26.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.762+0000 7f8ce97fa640 1 -- 172.21.15.64:0/3870911470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc0a4260 con 0x5609b85e5aa0 2024-05-09T09:21:26.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.762+0000 7f8ce97fa640 1 -- 172.21.15.64:0/3870911470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc09f9a0 con 0x5609b85e5aa0 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3870911470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 msgr2=0x5609b85e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85e7f30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc09b3a0 tx=0x7f8cfc09b310 comp rx=0 tx=0).stop 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d037fe640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8cec0b3da0 0x7f8cec0b6260 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80095f0 tx=0x7f8cf8030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3870911470 shutdown_connections 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85e7f30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e0930 0x5609b85e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3870911470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85dff40 0x5609b85e0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3870911470 >> 172.21.15.64:0/3870911470 conn(0x5609b85db5b0 msgr2=0x5609b85db9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3870911470 shutdown_connections 2024-05-09T09:21:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.763+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3870911470 wait complete. 2024-05-09T09:21:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 -- start start 2024-05-09T09:21:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 Processor -- start 2024-05-09T09:21:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b85f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5609b85e8a40 con 0x5609b85dff40 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5609b84ffc40 con 0x5609b85e0930 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5609b85e8bc0 con 0x5609b85e5aa0 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d0c59e740 10 client.0 _ll_register_callbacks cb 0x5609b8257be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d02ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d027fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.764+0000 7f8d02ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ea360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38810/0 (socket says 172.21.15.64:38810) 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.765+0000 7f8d02ffd640 1 -- 172.21.15.64:0/404934759 learned_addr learned my addr 172.21.15.64:0/404934759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.765+0000 7f8d027fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85ea8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51586/0 (socket says 172.21.15.64:51586) 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.765+0000 7f8d037fe640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b85f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.765+0000 7f8d02ffd640 1 -- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 msgr2=0x5609b85f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.765+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b85f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.765+0000 7f8d02ffd640 1 -- 172.21.15.64:0/404934759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 msgr2=0x5609b85ea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.766+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85ea8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.766+0000 7f8d02ffd640 1 -- 172.21.15.64:0/404934759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b83227d0 con 0x5609b85dff40 2024-05-09T09:21:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.766+0000 7f8d027fc640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85ea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.767+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ea360 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40126f0 tx=0x7f8cf4012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bae1ee61a2646c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.767+0000 7f8ce6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:21:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.768+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/404934759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4004050 con 0x5609b85dff40 2024-05-09T09:21:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.768+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cfc0a4260 con 0x5609b85dff40 2024-05-09T09:21:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.768+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/404934759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8cf4012e00 con 0x5609b85dff40 2024-05-09T09:21:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.768+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/404934759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8cf40041f0 con 0x5609b85dff40 2024-05-09T09:21:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.768+0000 7f8d0c59e740 10 client.18474 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.768+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5609b85f4cb0 con 0x5609b85dff40 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/404934759 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc0a4260 con 0x5609b85dff40 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/404934759 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f8cf4003920 con 0x5609b85dff40 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8d0c59e740 10 client.18474 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8d0c59e740 10 client.18474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8d0c59e740 20 client.18474 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b85f67a0 0x5609b85f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8d0c59e740 4 client.18474 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.769+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609b8552370 con 0x5609b85f67a0 2024-05-09T09:21:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.770+0000 7f8d027fc640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b85f67a0 0x5609b85f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.771+0000 7f8ce6ff5640 10 client.18474 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:21:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.771+0000 7f8d027fc640 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b85f67a0 0x5609b85f8c40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=831efbee41e1eed6 server_cookie=28ea18eb9e6c63d7 in_seq=0 out_seq=0 2024-05-09T09:21:26.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.772+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/404934759 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609b8552370 con 0x5609b85f67a0 2024-05-09T09:21:26.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.772+0000 7f8ce6ff5640 10 client.18474 handle_command_reply: tid=0 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 client.18474 shutdown 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 msgr2=0x5609b85ea360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ea360 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40126f0 tx=0x7f8cf4012bb0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 shutdown_connections 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b85f67a0 0x5609b85f8c40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b85f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85ea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/404934759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ea360 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 >> 172.21.15.64:0/404934759 conn(0x5609b85db5b0 msgr2=0x5609b85da7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8ce6ff5640 0 client.18474 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:21:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 shutdown_connections 2024-05-09T09:21:26.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.773+0000 7f8d0c59e740 1 -- 172.21.15.64:0/404934759 wait complete. 2024-05-09T09:21:26.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.774+0000 7f8d0c59e740 20 client.18474 trim_cache size 0 max 16384 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 Processor -- start 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 -- start start 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85dff40 0x5609b85de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85f1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf00125d0 con 0x5609b85e5aa0 2024-05-09T09:21:26.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf000fe70 con 0x5609b85e0930 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf000dcc0 con 0x5609b85dff40 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38814/0 (socket says 172.21.15.64:38814) 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d037fe640 1 -- 172.21.15.64:0/2753108009 learned_addr learned my addr 172.21.15.64:0/2753108009 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.801+0000 7f8d027fc640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d037fe640 1 -- 172.21.15.64:0/2753108009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85dff40 msgr2=0x5609b85de530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85dff40 0x5609b85de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d037fe640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85dff40 0x5609b85de530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d037fe640 1 -- 172.21.15.64:0/2753108009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 msgr2=0x5609b85f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d037fe640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d037fe640 1 -- 172.21.15.64:0/2753108009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609b85e0d10 con 0x5609b85e5aa0 2024-05-09T09:21:26.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d027fc640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.802+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85dff40 0x5609b85de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:21:26.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.803+0000 7f8d037fe640 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85dea70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8030e50 tx=0x7f8cf803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167e4da1223c2c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.803+0000 7f8ce8ff9640 1 -- 172.21.15.64:0/2753108009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf804e070 con 0x5609b85e5aa0 2024-05-09T09:21:26.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.803+0000 7f8ce8ff9640 1 -- 172.21.15.64:0/2753108009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cf8038040 con 0x5609b85e5aa0 2024-05-09T09:21:26.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.803+0000 7f8ce8ff9640 1 -- 172.21.15.64:0/2753108009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8037040 con 0x5609b85e5aa0 2024-05-09T09:21:26.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.805+0000 7f8d0c59e740 1 -- 172.21.15.64:0/2753108009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 msgr2=0x5609b85dea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.805+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85dea70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8030e50 tx=0x7f8cf803c000 comp rx=0 tx=0).stop 2024-05-09T09:21:26.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.806+0000 7f8d0c59e740 1 -- 172.21.15.64:0/2753108009 shutdown_connections 2024-05-09T09:21:26.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.806+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85e5aa0 0x5609b85dea70 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.806+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.806+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/2753108009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85dff40 0x5609b85de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.806+0000 7f8d0c59e740 1 -- 172.21.15.64:0/2753108009 >> 172.21.15.64:0/2753108009 conn(0x5609b85db5b0 msgr2=0x5609b85e11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.809+0000 7f8d0c59e740 1 -- 172.21.15.64:0/2753108009 shutdown_connections 2024-05-09T09:21:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.810+0000 7f8d0c59e740 1 -- 172.21.15.64:0/2753108009 wait complete. 2024-05-09T09:21:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.810+0000 7f8d0c59e740 1 -- start start 2024-05-09T09:21:26.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.810+0000 7f8d0c59e740 1 Processor -- start 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b866bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf000e330 con 0x5609b85dff40 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf000f7a0 con 0x5609b85e0930 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf000ea00 con 0x5609b85e5aa0 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d0c59e740 10 client.0 _ll_register_callbacks cb 0x7f8cf4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b866bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b866bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57074/0 (socket says 172.21.15.64:57074) 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 -- 172.21.15.64:0/3483472354 learned_addr learned my addr 172.21.15.64:0/3483472354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d027fc640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 -- 172.21.15.64:0/3483472354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 msgr2=0x5609b85eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 -- 172.21.15.64:0/3483472354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 msgr2=0x5609b85ee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ee7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.811+0000 7f8d037fe640 1 -- 172.21.15.64:0/3483472354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b866c300 con 0x5609b85e5aa0 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8d027fc640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8d037fe640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b866bdc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8cf802a930 tx=0x7f8cf80043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217e028429b6ebcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8ce6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/3483472354 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf805a030 con 0x5609b85e5aa0 2024-05-09T09:21:26.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf8038040 con 0x5609b85e5aa0 2024-05-09T09:21:26.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/3483472354 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8cf80037b0 con 0x5609b85e5aa0 2024-05-09T09:21:26.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.812+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/3483472354 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8cf8067050 con 0x5609b85e5aa0 2024-05-09T09:21:26.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.814+0000 7f8d0c59e740 10 client.28093 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:26.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.814+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5609b8656fe0 con 0x5609b85e5aa0 2024-05-09T09:21:26.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/3483472354 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cf8038040 con 0x5609b85e5aa0 2024-05-09T09:21:26.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/3483472354 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f8cf80537f0 con 0x5609b85e5aa0 2024-05-09T09:21:26.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8d0c59e740 10 client.28093 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8d0c59e740 10 client.28093 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:21:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8d0c59e740 20 client.28093 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b86579f0 0x5609b8659e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8d0c59e740 4 client.28093 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:21:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.815+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609b854c6c0 con 0x5609b86579f0 2024-05-09T09:21:26.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.816+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b86579f0 0x5609b8659e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:26.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8d02ffd640 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b86579f0 0x5609b8659e10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1e0223efc52d672 server_cookie=d727cf13107c01ad in_seq=0 out_seq=0 2024-05-09T09:21:26.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8ce6ff5640 10 client.28093 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:21:26.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8ce6ff5640 1 -- 172.21.15.64:0/3483472354 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609b854c6c0 con 0x5609b86579f0 2024-05-09T09:21:26.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8ce6ff5640 10 client.28093 handle_command_reply: tid=0 2024-05-09T09:21:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8d0c59e740 1 client.28093 shutdown 2024-05-09T09:21:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 msgr2=0x5609b866bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.819+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b866bdc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8cf802a930 tx=0x7f8cf80043f0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 shutdown_connections 2024-05-09T09:21:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x5609b86579f0 0x5609b8659e10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5609b85e5aa0 0x5609b866bdc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b85e0930 0x5609b85eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8d0c59e740 1 --2- 172.21.15.64:0/3483472354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609b85dff40 0x5609b85ee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 >> 172.21.15.64:0/3483472354 conn(0x5609b85db5b0 msgr2=0x5609b85d9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.820+0000 7f8ce6ff5640 0 client.28093 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:21:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.821+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 shutdown_connections 2024-05-09T09:21:26.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.821+0000 7f8d0c59e740 1 -- 172.21.15.64:0/3483472354 wait complete. 2024-05-09T09:21:26.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.821+0000 7f8d0c59e740 20 client.28093 trim_cache size 0 max 16384 2024-05-09T09:21:26.821 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:21:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:26 smithi139 ceph-mon[33599]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-05-09T09:21:26.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:26 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2964763498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.823+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8cec0b3da0 msgr2=0x7f8cec0b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.823+0000 7f8d097b0640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8cec0b3da0 0x7f8cec0b6260 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80095f0 tx=0x7f8cf8030450 comp rx=0 tx=0).stop 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04102a60 msgr2=0x7f8d04071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04102a60 0x7f8d04071320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00ef60 tx=0x7f8cfc00ef90 comp rx=0 tx=0).stop 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 shutdown_connections 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8cec0b3da0 0x7f8cec0b6260 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d04105410 0x7f8d04072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d04102a60 0x7f8d04071320 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 --2- 172.21.15.64:0/2444485576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d041000f0 0x7f8d04070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:26.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.824+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 >> 172.21.15.64:0/2444485576 conn(0x7f8d040fbcf0 msgr2=0x7f8d04100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.829+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 shutdown_connections 2024-05-09T09:21:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:26.830+0000 7f8d097b0640 1 -- 172.21.15.64:0/2444485576 wait complete. 2024-05-09T09:21:26.850 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:26 smithi064 ceph-mon[30149]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-05-09T09:21:26.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:26 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2964763498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:21:26.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:21:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:26 smithi179 ceph-mon[34086]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-05-09T09:21:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:26 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2964763498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:21:27.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 -- 172.21.15.64:0/2081640595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0075870 msgr2=0x7fc8f0073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/2081640595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0075870 0x7fc8f0073cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009900 tx=0x7fc8e402a420 comp rx=0 tx=0).stop 2024-05-09T09:21:27.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 -- 172.21.15.64:0/2081640595 shutdown_connections 2024-05-09T09:21:27.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/2081640595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/2081640595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f0074290 0x7fc8f0074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/2081640595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0075870 0x7fc8f0073cc0 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009900 tx=0x7fc8e402a420 comp rx=0 tx=0).stop 2024-05-09T09:21:27.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.056+0000 7fc8f6c37640 1 -- 172.21.15.64:0/2081640595 >> 172.21.15.64:0/2081640595 conn(0x7fc8f00fcde0 msgr2=0x7fc8f00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.058+0000 7fc8f6c37640 1 -- 172.21.15.64:0/2081640595 shutdown_connections 2024-05-09T09:21:27.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.058+0000 7fc8f6c37640 1 -- 172.21.15.64:0/2081640595 wait complete. 2024-05-09T09:21:27.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.059+0000 7fc8f6c37640 1 Processor -- start 2024-05-09T09:21:27.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.059+0000 7fc8f6c37640 1 -- start start 2024-05-09T09:21:27.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.059+0000 7fc8f6c37640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 0x7fc8f019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.059+0000 7fc8f6c37640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f49d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 0x7fc8f019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f6c37640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 0x7fc8f01a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f49d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 0x7fc8f019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57100/0 (socket says 172.21.15.64:57100) 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38840/0 (socket says 172.21.15.64:38840) 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f6c37640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f0102430 con 0x7fc8f0109730 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f6c37640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f01022b0 con 0x7fc8f01a3890 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f6c37640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f01025b0 con 0x7fc8f0074290 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f51d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 0x7fc8f01a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f51d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 0x7fc8f01a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51630/0 (socket says 172.21.15.64:51630) 2024-05-09T09:21:27.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.060+0000 7fc8f51d4640 1 -- 172.21.15.64:0/4081853520 learned_addr learned my addr 172.21.15.64:0/4081853520 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.061+0000 7fc8f51d4640 1 -- 172.21.15.64:0/4081853520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 msgr2=0x7fc8f019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.061+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 0x7fc8f019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.061+0000 7fc8f51d4640 1 -- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 msgr2=0x7fc8f019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.061+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.061+0000 7fc8f51d4640 1 -- 172.21.15.64:0/4081853520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f01a62f0 con 0x7fc8f01a3890 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.061+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 0x7fc8f01a5c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800efd0 tx=0x7fc8e800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.062+0000 7fc8edffb640 1 -- 172.21.15.64:0/4081853520 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8003b70 con 0x7fc8f01a3890 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.062+0000 7fc8edffb640 1 -- 172.21.15.64:0/4081853520 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8e8003d10 con 0x7fc8f01a3890 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.062+0000 7fc8effff640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.062+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 0x7fc8f019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.062+0000 7fc8edffb640 1 -- 172.21.15.64:0/4081853520 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80108c0 con 0x7fc8f01a3890 2024-05-09T09:21:27.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.062+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f01a65e0 con 0x7fc8f01a3890 2024-05-09T09:21:27.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.063+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f01a6a80 con 0x7fc8f01a3890 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.066+0000 7fc8f9a16740 1 Processor -- start 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 -- start start 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675cdf50 0x55bf675ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675ce940 0x55bf675d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf67521c40 con 0x55bf675cdf50 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf674c9580 con 0x55bf675d3ab0 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.067+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf6751dc00 con 0x55bf675ce940 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.068+0000 7fc8effff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.068+0000 7fc8effff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51634/0 (socket says 172.21.15.64:51634) 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.068+0000 7fc8effff640 1 -- 172.21.15.64:0/3483248058 learned_addr learned my addr 172.21.15.64:0/3483248058 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.069+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675ce940 0x55bf675d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8effff640 1 -- 172.21.15.64:0/3483248058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675ce940 msgr2=0x55bf675d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8edffb640 1 -- 172.21.15.64:0/4081853520 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8e80045f0 con 0x7fc8f01a3890 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8effff640 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675ce940 0x55bf675d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8effff640 1 -- 172.21.15.64:0/3483248058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675cdf50 msgr2=0x55bf675ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8effff640 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675cdf50 0x55bf675ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8effff640 1 -- 172.21.15.64:0/3483248058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf672353f0 con 0x55bf675d3ab0 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.070+0000 7fc8effff640 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675d5f40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000bc40 tx=0x7fc8e0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3694e5cdf8d552b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.071+0000 7fc8d6ffd640 1 -- 172.21.15.64:0/3483248058 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0004020 con 0x55bf675d3ab0 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.071+0000 7fc8edffb640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8d80a2c10 0x7fc8d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.071+0000 7fc8d6ffd640 1 -- 172.21.15.64:0/3483248058 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8e00041c0 con 0x55bf675d3ab0 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.072+0000 7fc8edffb640 1 -- 172.21.15.64:0/4081853520 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8e80cfab0 con 0x7fc8f01a3890 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.072+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8d80a2c10 0x7fc8d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8d6ffd640 1 -- 172.21.15.64:0/3483248058 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00126b0 con 0x55bf675d3ab0 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 -- 172.21.15.64:0/3483248058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 msgr2=0x55bf675d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675d5f40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000bc40 tx=0x7fc8e0010740 comp rx=0 tx=0).stop 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 -- 172.21.15.64:0/3483248058 shutdown_connections 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675d5f40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675ce940 0x55bf675d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/3483248058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675cdf50 0x55bf675ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.073+0000 7fc8f9a16740 1 -- 172.21.15.64:0/3483248058 >> 172.21.15.64:0/3483248058 conn(0x55bf675b85b0 msgr2=0x55bf675b89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.074+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8d80a2c10 0x7fc8d80a50d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006010 tx=0x7fc8e4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.074+0000 7fc8f9a16740 1 -- 172.21.15.64:0/3483248058 shutdown_connections 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.074+0000 7fc8f9a16740 1 -- 172.21.15.64:0/3483248058 wait complete. 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.074+0000 7fc8f9a16740 1 -- start start 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.074+0000 7fc8f9a16740 1 Processor -- start 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf675d6b00 con 0x55bf675ce940 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bf674dcc40 con 0x55bf675cdf50 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bf675d6c80 con 0x55bf675d3ab0 2024-05-09T09:21:27.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f51d4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f51d4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675f08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57130/0 (socket says 172.21.15.64:57130) 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1901296044 learned_addr learned my addr 172.21.15.64:0/1901296044 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.075+0000 7fc8f9a16740 10 client.0 _ll_register_callbacks cb 0x55bf67234be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8effff640 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1901296044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 msgr2=0x55bf675e9330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675e9330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1901296044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 msgr2=0x55bf675e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675e9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1901296044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf672ff7d0 con 0x55bf675d3ab0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.076+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675f08b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc8e809bc80 tx=0x7fc8e809c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79770640849c94fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8cffff640 1 -- 172.21.15.64:0/1901296044 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80b4020 con 0x55bf675d3ab0 2024-05-09T09:21:27.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e00041c0 con 0x55bf675d3ab0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8cffff640 1 -- 172.21.15.64:0/1901296044 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8e80babc0 con 0x55bf675d3ab0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8cffff640 1 -- 172.21.15.64:0/1901296044 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc8e80badc0 con 0x55bf675d3ab0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8f9a16740 10 client.28102 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.077+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bf675d98e0 con 0x55bf675d3ab0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8cffff640 1 -- 172.21.15.64:0/1901296044 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8e00041c0 con 0x55bf675d3ab0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8cffff640 1 -- 172.21.15.64:0/1901296044 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7fc8e8097cd0 con 0x55bf675d3ab0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f9a16740 10 client.28102 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f9a16740 10 client.28102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f9a16740 20 client.28102 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf675db050 0x55bf675dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f9a16740 4 client.28102 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf6752f370 con 0x55bf675db050 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.078+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf675db050 0x55bf675dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.079+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf675db050 0x55bf675dd4c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2043f98c1c8c315 server_cookie=48e702f8494d1510 in_seq=0 out_seq=0 2024-05-09T09:21:27.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.079+0000 7fc8cffff640 10 client.28102 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:21:27.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.079+0000 7fc8cffff640 1 -- 172.21.15.64:0/1901296044 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf6752f370 con 0x55bf675db050 2024-05-09T09:21:27.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.079+0000 7fc8cffff640 10 client.28102 handle_command_reply: tid=0 2024-05-09T09:21:27.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.079+0000 7fc8f9a16740 1 client.28102 shutdown 2024-05-09T09:21:27.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 msgr2=0x55bf675f08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675f08b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc8e809bc80 tx=0x7fc8e809c7c0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 shutdown_connections 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675f08b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf675db050 0x55bf675dd4c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1901296044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675e9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 >> 172.21.15.64:0/1901296044 conn(0x55bf675b85b0 msgr2=0x55bf675b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8cffff640 0 client.28102 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 shutdown_connections 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.080+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1901296044 wait complete. 2024-05-09T09:21:27.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.081+0000 7fc8f9a16740 20 client.28102 trim_cache size 0 max 16384 2024-05-09T09:21:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.115+0000 7fc8f9a16740 1 Processor -- start 2024-05-09T09:21:27.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.115+0000 7fc8f9a16740 1 -- start start 2024-05-09T09:21:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.116+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675cdf50 0x55bf675b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.116+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675f0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.116+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.116+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f006f820 con 0x55bf675ce940 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f9a16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f006d0c0 con 0x55bf675d3ab0 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51656/0 (socket says 172.21.15.64:51656) 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1295154968 learned_addr learned my addr 172.21.15.64:0/1295154968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1295154968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f0116870 con 0x55bf675cdf50 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8effff640 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675cdf50 0x55bf675b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1295154968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675cdf50 msgr2=0x55bf675b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675cdf50 0x55bf675b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1295154968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 msgr2=0x55bf675f0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.117+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675f0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.118+0000 7fc8f51d4640 1 -- 172.21.15.64:0/1295154968 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf675ced20 con 0x55bf675d3ab0 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.118+0000 7fc8f51d4640 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675b7460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8005c40 tx=0x7fc8e809a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147c7ce71fe6a670 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.118+0000 7fc8d67fc640 1 -- 172.21.15.64:0/1295154968 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80b4070 con 0x55bf675d3ab0 2024-05-09T09:21:27.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.118+0000 7fc8d67fc640 1 -- 172.21.15.64:0/1295154968 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8e80b6060 con 0x55bf675d3ab0 2024-05-09T09:21:27.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.119+0000 7fc8d67fc640 1 -- 172.21.15.64:0/1295154968 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e809b630 con 0x55bf675d3ab0 2024-05-09T09:21:27.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.120+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1295154968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 msgr2=0x55bf675b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.120+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675b7460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8005c40 tx=0x7fc8e809a490 comp rx=0 tx=0).stop 2024-05-09T09:21:27.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.122+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1295154968 shutdown_connections 2024-05-09T09:21:27.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.123+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675d3ab0 0x55bf675b7460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.123+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675f0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.123+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/1295154968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675cdf50 0x55bf675b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.123+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1295154968 >> 172.21.15.64:0/1295154968 conn(0x55bf675b85b0 msgr2=0x55bf675cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.125+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1295154968 shutdown_connections 2024-05-09T09:21:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.125+0000 7fc8f9a16740 1 -- 172.21.15.64:0/1295154968 wait complete. 2024-05-09T09:21:27.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.126+0000 7fc8f9a16740 1 -- start start 2024-05-09T09:21:27.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.126+0000 7fc8f9a16740 1 Processor -- start 2024-05-09T09:21:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f9a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675eea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f49d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f49d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675ee500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51660/0 (socket says 172.21.15.64:51660) 2024-05-09T09:21:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f49d3640 1 -- 172.21.15.64:0/269300916 learned_addr learned my addr 172.21.15.64:0/269300916 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675efa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.127+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f006b580 con 0x55bf675ce940 2024-05-09T09:21:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f006c9f0 con 0x55bf675cdf50 2024-05-09T09:21:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f006bc50 con 0x55bf675d3ab0 2024-05-09T09:21:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8f9a16740 10 client.0 _ll_register_callbacks cb 0x7fc8e8097be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8effff640 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8effff640 1 -- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 msgr2=0x55bf675efa50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8effff640 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675efa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8effff640 1 -- 172.21.15.64:0/269300916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 msgr2=0x55bf675ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8effff640 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675ee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.128+0000 7fc8effff640 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf675f0040 con 0x55bf675ce940 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.129+0000 7fc8effff640 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675eea40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x55bf675ea810 tx=0x7fc8e0013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4216c33fc07f393e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.129+0000 7fc8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:21:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.129+0000 7fc8cffff640 1 -- 172.21.15.64:0/269300916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0013c10 con 0x55bf675ce940 2024-05-09T09:21:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.130+0000 7fc8cffff640 1 -- 172.21.15.64:0/269300916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8e0013db0 con 0x55bf675ce940 2024-05-09T09:21:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.130+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e80b6060 con 0x55bf675ce940 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.130+0000 7fc8cffff640 1 -- 172.21.15.64:0/269300916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc8e000b8e0 con 0x55bf675ce940 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.130+0000 7fc8f9a16740 10 client.18492 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.130+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bf6766c630 con 0x55bf675ce940 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8cffff640 1 -- 172.21.15.64:0/269300916 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8e80b6060 con 0x55bf675ce940 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8cffff640 1 -- 172.21.15.64:0/269300916 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7fc8e000f040 con 0x55bf675ce940 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f9a16740 10 client.18492 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f9a16740 10 client.18492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f9a16740 20 client.18492 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf6766cb90 0x55bf6766ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f9a16740 4 client.18492 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf675296c0 con 0x55bf6766cb90 2024-05-09T09:21:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.131+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf6766cb90 0x55bf6766ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.133+0000 7fc8f49d3640 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf6766cb90 0x55bf6766ef80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a25020d3ec03f58 server_cookie=999d5cd96e028c7c in_seq=0 out_seq=0 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.133+0000 7fc8cffff640 10 client.18492 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.133+0000 7fc8cffff640 1 -- 172.21.15.64:0/269300916 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf675296c0 con 0x55bf6766cb90 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.133+0000 7fc8cffff640 10 client.18492 handle_command_reply: tid=0 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.133+0000 7fc8f9a16740 1 client.18492 shutdown 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 msgr2=0x55bf675eea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675eea40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x55bf675ea810 tx=0x7fc8e0013690 comp rx=0 tx=0).stop 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 shutdown_connections 2024-05-09T09:21:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf675d3ab0 0x55bf675efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55bf6766cb90 0x55bf6766ef80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8cffff640 0 client.18492 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf675ce940 0x55bf675eea40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 --2- 172.21.15.64:0/269300916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bf675cdf50 0x55bf675ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 >> 172.21.15.64:0/269300916 conn(0x55bf675b85b0 msgr2=0x55bf675bbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 shutdown_connections 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.135+0000 7fc8f9a16740 1 -- 172.21.15.64:0/269300916 wait complete. 2024-05-09T09:21:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.136+0000 7fc8f9a16740 20 client.18492 trim_cache size 0 max 16384 2024-05-09T09:21:27.138 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:21:27.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.140+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8d80a2c10 msgr2=0x7fc8d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.140+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8d80a2c10 0x7fc8d80a50d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006010 tx=0x7fc8e4005e80 comp rx=0 tx=0).stop 2024-05-09T09:21:27.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.140+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 msgr2=0x7fc8f01a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.140+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 0x7fc8f01a5c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800efd0 tx=0x7fc8e800c6a0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.143+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 shutdown_connections 2024-05-09T09:21:27.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.143+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8d80a2c10 0x7fc8d80a50d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.143+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f01a3890 0x7fc8f01a5c80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.143+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8f0109730 0x7fc8f019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.143+0000 7fc8f6c37640 1 --2- 172.21.15.64:0/4081853520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8f0074290 0x7fc8f019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.143+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 >> 172.21.15.64:0/4081853520 conn(0x7fc8f00fcde0 msgr2=0x7fc8f0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.144+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 shutdown_connections 2024-05-09T09:21:27.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.144+0000 7fc8f6c37640 1 -- 172.21.15.64:0/4081853520 wait complete. 2024-05-09T09:21:27.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:21:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 -- 172.21.15.64:0/3412669090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa4107e90 msgr2=0x7f8fa41082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/3412669090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa4107e90 0x7f8fa41082b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009900 tx=0x7f8fa002a3f0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 -- 172.21.15.64:0/3412669090 shutdown_connections 2024-05-09T09:21:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/3412669090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa41099a0 0x7f8fa41102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/3412669090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4108fe0 0x7f8fa4109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/3412669090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa4107e90 0x7f8fa41082b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 -- 172.21.15.64:0/3412669090 >> 172.21.15.64:0/3412669090 conn(0x7f8fa4076e30 msgr2=0x7f8fa4079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 -- 172.21.15.64:0/3412669090 shutdown_connections 2024-05-09T09:21:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.369+0000 7f8fac5e7640 1 -- 172.21.15.64:0/3412669090 wait complete. 2024-05-09T09:21:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.371+0000 7f8fac5e7640 1 Processor -- start 2024-05-09T09:21:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.371+0000 7f8fac5e7640 1 -- start start 2024-05-09T09:21:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8fac5e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 0x7f8fa419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8faa383640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 0x7f8fa419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8faa383640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 0x7f8fa419e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51686/0 (socket says 172.21.15.64:51686) 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8faa383640 1 -- 172.21.15.64:0/11260538 learned_addr learned my addr 172.21.15.64:0/11260538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa4108fe0 0x7f8fa419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa41099a0 0x7f8fa41a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.372+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4113a30 con 0x7f8fa4108fe0 2024-05-09T09:21:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8faab84640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa41099a0 0x7f8fa41a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa4108fe0 0x7f8fa419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41138b0 con 0x7f8fa4107e90 2024-05-09T09:21:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4113bb0 con 0x7f8fa41099a0 2024-05-09T09:21:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8faa383640 1 -- 172.21.15.64:0/11260538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa41099a0 msgr2=0x7f8fa41a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8faa383640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa41099a0 0x7f8fa41a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8faa383640 1 -- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa4108fe0 msgr2=0x7f8fa419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8faa383640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa4108fe0 0x7f8fa419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.373+0000 7f8faa383640 1 -- 172.21.15.64:0/11260538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa41a5cc0 con 0x7f8fa4107e90 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.374+0000 7f8faa383640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 0x7f8fa419e0a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00314e0 tx=0x7f8fa0003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.374+0000 7f8f9b7fe640 1 -- 172.21.15.64:0/11260538 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa003f070 con 0x7f8fa4107e90 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.374+0000 7f8f9b7fe640 1 -- 172.21.15.64:0/11260538 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa0031e40 con 0x7f8fa4107e90 2024-05-09T09:21:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.374+0000 7f8f9b7fe640 1 -- 172.21.15.64:0/11260538 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0035030 con 0x7f8fa4107e90 2024-05-09T09:21:27.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.375+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa41a5f50 con 0x7f8fa4107e90 2024-05-09T09:21:27.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.375+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa41a63b0 con 0x7f8fa4107e90 2024-05-09T09:21:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.378+0000 7f8f9b7fe640 1 -- 172.21.15.64:0/11260538 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fa0042070 con 0x7f8fa4107e90 2024-05-09T09:21:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 Processor -- start 2024-05-09T09:21:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 -- start start 2024-05-09T09:21:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e34360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e39430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e3bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a0a9d98c40 con 0x55a0a9e34930 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a0a9d40580 con 0x55a0a9e33f40 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.379+0000 7f8faf3d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a0a9d94c00 con 0x55a0a9e39aa0 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e34360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e34360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51692/0 (socket says 172.21.15.64:51692) 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 -- 172.21.15.64:0/1871448582 learned_addr learned my addr 172.21.15.64:0/1871448582 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faa383640 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e39430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 -- 172.21.15.64:0/1871448582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 msgr2=0x55a0a9e3bf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e3bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 -- 172.21.15.64:0/1871448582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 msgr2=0x55a0a9e39430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e39430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 -- 172.21.15.64:0/1871448582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a9aac3f0 con 0x55a0a9e33f40 2024-05-09T09:21:27.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.380+0000 7f8faab84640 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e34360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c007f40 tx=0x7f8f9c00f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d833ef451280229 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.381+0000 7f8f98bfe640 1 -- 172.21.15.64:0/1871448582 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c010900 con 0x55a0a9e33f40 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.381+0000 7f8f98bfe640 1 -- 172.21.15.64:0/1871448582 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f9c0183f0 con 0x55a0a9e33f40 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.381+0000 7f8f98bfe640 1 -- 172.21.15.64:0/1871448582 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c017600 con 0x55a0a9e33f40 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.381+0000 7f8f9b7fe640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8f900a2c10 0x7f8f900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.381+0000 7f8faf3d2740 1 -- 172.21.15.64:0/1871448582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 msgr2=0x55a0a9e34360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e34360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c007f40 tx=0x7f8f9c00f850 comp rx=0 tx=0).stop 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 -- 172.21.15.64:0/1871448582 shutdown_connections 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faa383640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8f900a2c10 0x7f8f900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e39430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/1871448582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e34360 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 -- 172.21.15.64:0/1871448582 >> 172.21.15.64:0/1871448582 conn(0x55a0a9e2f5b0 msgr2=0x55a0a9e2f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.382+0000 7f8faf3d2740 1 -- 172.21.15.64:0/1871448582 shutdown_connections 2024-05-09T09:21:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.383+0000 7f8faf3d2740 1 -- 172.21.15.64:0/1871448582 wait complete. 2024-05-09T09:21:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.383+0000 7f8faa383640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8f900a2c10 0x7f8f900a50d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00ed700 tx=0x7f8fa00f3210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.383+0000 7f8f9b7fe640 1 -- 172.21.15.64:0/11260538 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8fa00efc20 con 0x7f8fa4107e90 2024-05-09T09:21:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.383+0000 7f8faf3d2740 1 -- start start 2024-05-09T09:21:27.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.384+0000 7f8faf3d2740 1 Processor -- start 2024-05-09T09:21:27.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.384+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e34930 0x55a0a9e3e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57204/0 (socket says 172.21.15.64:57204) 2024-05-09T09:21:27.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 -- 172.21.15.64:0/467066619 learned_addr learned my addr 172.21.15.64:0/467066619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a0a9e3ca50 con 0x55a0a9e39aa0 2024-05-09T09:21:27.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a0a9d53c40 con 0x55a0a9e34930 2024-05-09T09:21:27.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a0a9e3cbd0 con 0x55a0a9e33f40 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faf3d2740 10 client.0 _ll_register_callbacks cb 0x55a0a9aabbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e34930 0x55a0a9e3e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 -- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e34930 msgr2=0x55a0a9e3e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e34930 0x55a0a9e3e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 -- 172.21.15.64:0/467066619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e39aa0 msgr2=0x55a0a9e458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.385+0000 7f8faab84640 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a9b767d0 con 0x55a0a9e33f40 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.386+0000 7f8faab84640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e360 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c007930 tx=0x7f8f9c005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fde648223b847fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.386+0000 7f8f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:27.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.386+0000 7f8f8dffb640 1 -- 172.21.15.64:0/467066619 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c02b060 con 0x55a0a9e33f40 2024-05-09T09:21:27.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.386+0000 7f8f8dffb640 1 -- 172.21.15.64:0/467066619 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8f9c004c40 con 0x55a0a9e33f40 2024-05-09T09:21:27.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.386+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f9c0183f0 con 0x55a0a9e33f40 2024-05-09T09:21:27.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.387+0000 7f8f8dffb640 1 -- 172.21.15.64:0/467066619 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8f9c032050 con 0x55a0a9e33f40 2024-05-09T09:21:27.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.387+0000 7f8faf3d2740 10 client.28114 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:27.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.387+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a0a9e48c30 con 0x55a0a9e33f40 2024-05-09T09:21:27.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.387+0000 7f8f8dffb640 1 -- 172.21.15.64:0/467066619 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f9c0183f0 con 0x55a0a9e33f40 2024-05-09T09:21:27.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8f8dffb640 1 -- 172.21.15.64:0/467066619 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f8f9c0042a0 con 0x55a0a9e33f40 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8faf3d2740 10 client.28114 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8faf3d2740 10 client.28114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8faf3d2740 20 client.28114 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9e4c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8faf3d2740 4 client.28114 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a9da6370 con 0x55a0a9e4a090 2024-05-09T09:21:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.389+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9e4c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.390+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9e4c480 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fe25059069e6ff server_cookie=f12c963787c7260 in_seq=0 out_seq=0 2024-05-09T09:21:27.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.390+0000 7f8f8dffb640 10 client.28114 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:27.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8f8dffb640 1 -- 172.21.15.64:0/467066619 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0a9da6370 con 0x55a0a9e4a090 2024-05-09T09:21:27.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8f8dffb640 10 client.28114 handle_command_reply: tid=0 2024-05-09T09:21:27.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 client.28114 shutdown 2024-05-09T09:21:27.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 msgr2=0x55a0a9e3e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e360 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c007930 tx=0x7f8f9c005b70 comp rx=0 tx=0).stop 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 shutdown_connections 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9e4c480 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e34930 0x55a0a9e3e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/467066619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e360 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.391+0000 7f8f8dffb640 0 client.28114 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.392+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 >> 172.21.15.64:0/467066619 conn(0x55a0a9e2f5b0 msgr2=0x55a0a9e2e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.392+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 shutdown_connections 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.392+0000 7f8faf3d2740 1 -- 172.21.15.64:0/467066619 wait complete. 2024-05-09T09:21:27.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.392+0000 7f8faf3d2740 20 client.28114 trim_cache size 0 max 16384 2024-05-09T09:21:27.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.418+0000 7f8faf3d2740 1 Processor -- start 2024-05-09T09:21:27.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.419+0000 7f8faf3d2740 1 -- start start 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.419+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.419+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e45870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.419+0000 7f8faab84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51718/0 (socket says 172.21.15.64:51718) 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 -- 172.21.15.64:0/3392995415 learned_addr learned my addr 172.21.15.64:0/3392995415 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e32c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e45870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41b32a0 con 0x55a0a9e34930 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41b0b40 con 0x55a0a9e33f40 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41aea40 con 0x55a0a9e39aa0 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e32c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 -- 172.21.15.64:0/3392995415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 msgr2=0x55a0a9e32c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e32c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 -- 172.21.15.64:0/3392995415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 msgr2=0x55a0a9e45870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e45870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.420+0000 7f8faab84640 1 -- 172.21.15.64:0/3392995415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a9e20d20 con 0x55a0a9e33f40 2024-05-09T09:21:27.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.421+0000 7f8faab84640 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c002230 tx=0x7f8f9c003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e34a8c90dcc875 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.421+0000 7f8f8ffff640 1 -- 172.21.15.64:0/3392995415 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c02b070 con 0x55a0a9e33f40 2024-05-09T09:21:27.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.421+0000 7f8f8ffff640 1 -- 172.21.15.64:0/3392995415 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f9c0183f0 con 0x55a0a9e33f40 2024-05-09T09:21:27.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.421+0000 7f8f8ffff640 1 -- 172.21.15.64:0/3392995415 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c010e10 con 0x55a0a9e33f40 2024-05-09T09:21:27.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.422+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 msgr2=0x55a0a9e3e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.422+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c002230 tx=0x7f8f9c003b80 comp rx=0 tx=0).stop 2024-05-09T09:21:27.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.423+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 shutdown_connections 2024-05-09T09:21:27.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.423+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e32c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.423+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e45870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.423+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/3392995415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e3e800 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.423+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 >> 172.21.15.64:0/3392995415 conn(0x55a0a9e2f5b0 msgr2=0x55a0a9e32560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.425+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 shutdown_connections 2024-05-09T09:21:27.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.425+0000 7f8faf3d2740 1 -- 172.21.15.64:0/3392995415 wait complete. 2024-05-09T09:21:27.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.425+0000 7f8faf3d2740 1 -- start start 2024-05-09T09:21:27.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 Processor -- start 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e42d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41af000 con 0x55a0a9e34930 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41b0470 con 0x55a0a9e33f40 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41af6d0 con 0x55a0a9e39aa0 2024-05-09T09:21:27.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faf3d2740 10 client.0 _ll_register_callbacks cb 0x7f8f9c007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faab84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e42d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.426+0000 7f8faab84640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8fa9b82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8fa9b82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38942/0 (socket says 172.21.15.64:38942) 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8fa9b82640 1 -- 172.21.15.64:0/2743261825 learned_addr learned my addr 172.21.15.64:0/2743261825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8faab84640 1 -- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 msgr2=0x55a0a9e42d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8faab84640 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e42d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8faab84640 1 -- 172.21.15.64:0/2743261825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 msgr2=0x55a0a9e432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8faab84640 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.427+0000 7f8faab84640 1 -- 172.21.15.64:0/2743261825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a9e448a0 con 0x55a0a9e39aa0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.428+0000 7f8faab84640 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e442b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c004810 tx=0x7f8f9c00fe00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c734b9ed9065a3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.428+0000 7f8f8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.428+0000 7f8f8dffb640 1 -- 172.21.15.64:0/2743261825 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c03c070 con 0x55a0a9e39aa0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.428+0000 7f8f8dffb640 1 -- 172.21.15.64:0/2743261825 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8f9c004a90 con 0x55a0a9e39aa0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.428+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f9c0183f0 con 0x55a0a9e39aa0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8f8dffb640 1 -- 172.21.15.64:0/2743261825 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8f9c02b070 con 0x55a0a9e39aa0 2024-05-09T09:21:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 10 client.28120 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a0a9e45ba0 con 0x55a0a9e39aa0 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8f8dffb640 1 -- 172.21.15.64:0/2743261825 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f9c0183f0 con 0x55a0a9e39aa0 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8f8dffb640 1 -- 172.21.15.64:0/2743261825 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f8f9c0247d0 con 0x55a0a9e39aa0 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 10 client.28120 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 10 client.28120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 20 client.28120 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9ec3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 4 client.28120 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.429+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0a9da06c0 con 0x55a0a9e4a090 2024-05-09T09:21:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.430+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9ec3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.430+0000 7f8fa9b82640 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9ec3bb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf6baab2ada881f server_cookie=681b9e02b9d8a741 in_seq=0 out_seq=0 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.430+0000 7f8f8dffb640 10 client.28120 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.431+0000 7f8f8dffb640 1 -- 172.21.15.64:0/2743261825 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0a9da06c0 con 0x55a0a9e4a090 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.431+0000 7f8f8dffb640 10 client.28120 handle_command_reply: tid=0 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.431+0000 7f8faf3d2740 1 client.28120 shutdown 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.431+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 msgr2=0x55a0a9e442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.431+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e442b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c004810 tx=0x7f8f9c00fe00 comp rx=0 tx=0).stop 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 shutdown_connections 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55a0a9e4a090 0x55a0a9ec3bb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0a9e39aa0 0x55a0a9e442b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0a9e34930 0x55a0a9e432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 --2- 172.21.15.64:0/2743261825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0a9e33f40 0x55a0a9e42d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 >> 172.21.15.64:0/2743261825 conn(0x55a0a9e2f5b0 msgr2=0x55a0a9e3e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8f8dffb640 0 client.28120 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 shutdown_connections 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.432+0000 7f8faf3d2740 1 -- 172.21.15.64:0/2743261825 wait complete. 2024-05-09T09:21:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.433+0000 7f8faf3d2740 20 client.28120 trim_cache size 0 max 16384 2024-05-09T09:21:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:21:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.434+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8f900a2c10 msgr2=0x7f8f900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.434+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8f900a2c10 0x7f8f900a50d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00ed700 tx=0x7f8fa00f3210 comp rx=0 tx=0).stop 2024-05-09T09:21:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.434+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 msgr2=0x7f8fa419e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.434+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 0x7f8fa419e0a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00314e0 tx=0x7f8fa0003c80 comp rx=0 tx=0).stop 2024-05-09T09:21:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 shutdown_connections 2024-05-09T09:21:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8f900a2c10 0x7f8f900a50d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fa41099a0 0x7f8fa41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa4108fe0 0x7f8fa419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 --2- 172.21.15.64:0/11260538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa4107e90 0x7f8fa419e0a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 >> 172.21.15.64:0/11260538 conn(0x7f8fa4076e30 msgr2=0x7f8fa410c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 shutdown_connections 2024-05-09T09:21:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:27.435+0000 7f8fac5e7640 1 -- 172.21.15.64:0/11260538 wait complete. 2024-05-09T09:21:28.455 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:21:28.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:21:28.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.657+0000 7fd955414640 1 -- 172.21.15.64:0/1996455059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 msgr2=0x7fd950102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.657+0000 7fd955414640 1 --2- 172.21.15.64:0/1996455059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd950102340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd94801acb0 tx=0x7fd94803b8e0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 -- 172.21.15.64:0/1996455059 shutdown_connections 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 --2- 172.21.15.64:0/1996455059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 0x7fd9501056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 --2- 172.21.15.64:0/1996455059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd950102880 0x7fd950104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 --2- 172.21.15.64:0/1996455059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd950102340 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 -- 172.21.15.64:0/1996455059 >> 172.21.15.64:0/1996455059 conn(0x7fd9500fbb30 msgr2=0x7fd9500fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 -- 172.21.15.64:0/1996455059 shutdown_connections 2024-05-09T09:21:28.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.658+0000 7fd955414640 1 -- 172.21.15.64:0/1996455059 wait complete. 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.659+0000 7fd955414640 1 Processor -- start 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.659+0000 7fd955414640 1 -- start start 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd955414640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd9501956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd955414640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd950102880 0x7fd950195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd94ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd9501956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd94ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd9501956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57248/0 (socket says 172.21.15.64:57248) 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd955414640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 0x7fd95019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd94ffff640 1 -- 172.21.15.64:0/3944461691 learned_addr learned my addr 172.21.15.64:0/3944461691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd950109040 con 0x7fd950102880 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd950108ec0 con 0x7fd950105230 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501091c0 con 0x7fd9500fff10 2024-05-09T09:21:28.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.660+0000 7fd94f7fe640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd950102880 0x7fd950195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd954c13640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 0x7fd95019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd94ffff640 1 -- 172.21.15.64:0/3944461691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 msgr2=0x7fd95019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd94ffff640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 0x7fd95019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd94ffff640 1 -- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd950102880 msgr2=0x7fd950195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd94ffff640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd950102880 0x7fd950195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd94ffff640 1 -- 172.21.15.64:0/3944461691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95019d2e0 con 0x7fd9500fff10 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.661+0000 7fd954c13640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 0x7fd95019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:28.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.662+0000 7fd94ffff640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd9501956c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd948005e30 tx=0x7fd948003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.662+0000 7fd94d7fa640 1 -- 172.21.15.64:0/3944461691 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948050070 con 0x7fd9500fff10 2024-05-09T09:21:28.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.662+0000 7fd94d7fa640 1 -- 172.21.15.64:0/3944461691 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9480044f0 con 0x7fd9500fff10 2024-05-09T09:21:28.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.662+0000 7fd94d7fa640 1 -- 172.21.15.64:0/3944461691 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948041cd0 con 0x7fd9500fff10 2024-05-09T09:21:28.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.662+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95019d570 con 0x7fd9500fff10 2024-05-09T09:21:28.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.662+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95019daf0 con 0x7fd9500fff10 2024-05-09T09:21:28.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.666+0000 7fd959385740 1 Processor -- start 2024-05-09T09:21:28.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 -- start start 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 0x55aeb7077af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb707cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb707d230 0x55aeb707f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aeb6fdb990 con 0x55aeb70780c0 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55aeb6f835b0 con 0x55aeb707d230 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd959385740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55aeb6fd7950 con 0x55aeb70776d0 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd954c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 0x55aeb7077af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd954c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 0x55aeb7077af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57262/0 (socket says 172.21.15.64:57262) 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd954c13640 1 -- 172.21.15.64:0/3572656886 learned_addr learned my addr 172.21.15.64:0/3572656886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd94d7fa640 1 -- 172.21.15.64:0/3944461691 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd94804c400 con 0x7fd9500fff10 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd94ffff640 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb707cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.667+0000 7fd94f7fe640 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb707d230 0x55aeb707f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.669+0000 7fd954c13640 1 -- 172.21.15.64:0/3572656886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb707d230 msgr2=0x55aeb707f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.669+0000 7fd954c13640 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb707d230 0x55aeb707f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.669+0000 7fd954c13640 1 -- 172.21.15.64:0/3572656886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 msgr2=0x55aeb707cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.669+0000 7fd954c13640 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb707cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.669+0000 7fd954c13640 1 -- 172.21.15.64:0/3572656886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb6db77e0 con 0x55aeb70776d0 2024-05-09T09:21:28.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.669+0000 7fd954c13640 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 0x55aeb7077af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd940011540 tx=0x7fd940011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c9eb67f7d497f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.671+0000 7fd94d7fa640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9380a2d90 0x7fd9380a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.671+0000 7fd9367fc640 1 -- 172.21.15.64:0/3572656886 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94000f5c0 con 0x55aeb70776d0 2024-05-09T09:21:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.671+0000 7fd94ffff640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9380a2d90 0x7fd9380a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.671+0000 7fd9367fc640 1 -- 172.21.15.64:0/3572656886 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd94000fc10 con 0x55aeb70776d0 2024-05-09T09:21:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.671+0000 7fd94d7fa640 1 -- 172.21.15.64:0/3944461691 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd948101530 con 0x7fd9500fff10 2024-05-09T09:21:28.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.671+0000 7fd9367fc640 1 -- 172.21.15.64:0/3572656886 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940019cb0 con 0x55aeb70776d0 2024-05-09T09:21:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.672+0000 7fd959385740 1 -- 172.21.15.64:0/3572656886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 msgr2=0x55aeb7077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.672+0000 7fd959385740 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 0x55aeb7077af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd940011540 tx=0x7fd940011a00 comp rx=0 tx=0).stop 2024-05-09T09:21:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.672+0000 7fd94ffff640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9380a2d90 0x7fd9380a5250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd9480c9e00 tx=0x7fd94801a5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.678+0000 7fd959385740 1 -- 172.21.15.64:0/3572656886 shutdown_connections 2024-05-09T09:21:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.678+0000 7fd959385740 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb707d230 0x55aeb707f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.678+0000 7fd959385740 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb707cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.678+0000 7fd959385740 1 --2- 172.21.15.64:0/3572656886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb70776d0 0x55aeb7077af0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.678+0000 7fd959385740 1 -- 172.21.15.64:0/3572656886 >> 172.21.15.64:0/3572656886 conn(0x55aeb703f0a0 msgr2=0x55aeb7073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.679+0000 7fd959385740 1 -- 172.21.15.64:0/3572656886 shutdown_connections 2024-05-09T09:21:28.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.680+0000 7fd959385740 1 -- 172.21.15.64:0/3572656886 wait complete. 2024-05-09T09:21:28.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.680+0000 7fd959385740 1 -- start start 2024-05-09T09:21:28.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 Processor -- start 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7081be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7082120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aeb70801e0 con 0x55aeb70780c0 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55aeb6f96b40 con 0x55aeb70776d0 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55aeb7080360 con 0x55aeb707d230 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.681+0000 7fd959385740 10 client.0 _ll_register_callbacks cb 0x55aeb6f8b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.682+0000 7fd94f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7082120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.682+0000 7fd94f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7082120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38970/0 (socket says 172.21.15.64:38970) 2024-05-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.682+0000 7fd94f7fe640 1 -- 172.21.15.64:0/368332375 learned_addr learned my addr 172.21.15.64:0/368332375 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.682+0000 7fd954c13640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7081be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.682+0000 7fd954c13640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.683+0000 7fd954c13640 1 -- 172.21.15.64:0/368332375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 msgr2=0x55aeb70890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.683+0000 7fd954c13640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.683+0000 7fd954c13640 1 -- 172.21.15.64:0/368332375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 msgr2=0x55aeb7082120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.683+0000 7fd954c13640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7082120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.683+0000 7fd954c13640 1 -- 172.21.15.64:0/368332375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb6e17f70 con 0x55aeb70776d0 2024-05-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.684+0000 7fd954c13640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7081be0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd940011540 tx=0x7fd940005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd919ceacf88049 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.684+0000 7fd92f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.684+0000 7fd92f7fe640 1 -- 172.21.15.64:0/368332375 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94002d060 con 0x55aeb70776d0 2024-05-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.684+0000 7fd92f7fe640 1 -- 172.21.15.64:0/368332375 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd94001f040 con 0x55aeb70776d0 2024-05-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.684+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94000fc10 con 0x55aeb70776d0 2024-05-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.685+0000 7fd92f7fe640 1 -- 172.21.15.64:0/368332375 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd940034050 con 0x55aeb70776d0 2024-05-09T09:21:28.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.687+0000 7fd959385740 10 client.27971 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:28.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.687+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55aeb6caf170 con 0x55aeb70776d0 2024-05-09T09:21:28.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.688+0000 7fd92f7fe640 1 -- 172.21.15.64:0/368332375 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd94000fc10 con 0x55aeb70776d0 2024-05-09T09:21:28.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd92f7fe640 1 -- 172.21.15.64:0/368332375 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7fd9400053e0 con 0x55aeb70776d0 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd959385740 10 client.27971 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd959385740 10 client.27971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd959385740 20 client.27971 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd959385740 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb709e410 0x55aeb70a0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd959385740 4 client.27971 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.689+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeb6fe90c0 con 0x55aeb709e410 2024-05-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.690+0000 7fd94f7fe640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb709e410 0x55aeb70a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.690+0000 7fd94f7fe640 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb709e410 0x55aeb70a0820 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723f44e3668c11b2 server_cookie=e667ed49c87af238 in_seq=0 out_seq=0 2024-05-09T09:21:28.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.690+0000 7fd92f7fe640 10 client.27971 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.691+0000 7fd92f7fe640 1 -- 172.21.15.64:0/368332375 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aeb6fe90c0 con 0x55aeb709e410 2024-05-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.691+0000 7fd92f7fe640 10 client.27971 handle_command_reply: tid=0 2024-05-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.692+0000 7fd959385740 1 client.27971 shutdown 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.692+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 msgr2=0x55aeb7081be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.692+0000 7fd959385740 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7081be0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd940011540 tx=0x7fd940005f90 comp rx=0 tx=0).stop 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 shutdown_connections 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd959385740 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb709e410 0x55aeb70a0820 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd959385740 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd92f7fe640 0 client.27971 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd959385740 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7082120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd959385740 1 --2- 172.21.15.64:0/368332375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7081be0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.693+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 >> 172.21.15.64:0/368332375 conn(0x55aeb703f0a0 msgr2=0x55aeb7072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.694+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 shutdown_connections 2024-05-09T09:21:28.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.694+0000 7fd959385740 1 -- 172.21.15.64:0/368332375 wait complete. 2024-05-09T09:21:28.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.695+0000 7fd959385740 20 client.27971 trim_cache size 0 max 16384 2024-05-09T09:21:28.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.737+0000 7fd959385740 1 Processor -- start 2024-05-09T09:21:28.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 -- start start 2024-05-09T09:21:28.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70776d0 0x55aeb7076800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70780c0 0x55aeb70a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501a20c0 con 0x55aeb70776d0 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd95019f960 con 0x55aeb70780c0 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd959385740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd950073580 con 0x55aeb707d230 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd954c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd954c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57284/0 (socket says 172.21.15.64:57284) 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.739+0000 7fd954c13640 1 -- 172.21.15.64:0/281602721 learned_addr learned my addr 172.21.15.64:0/281602721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70776d0 0x55aeb7076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 -- 172.21.15.64:0/281602721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70780c0 msgr2=0x55aeb70a60f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70780c0 0x55aeb70a60f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 -- 172.21.15.64:0/281602721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70776d0 msgr2=0x55aeb7076800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70776d0 0x55aeb7076800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 -- 172.21.15.64:0/281602721 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb707fb80 con 0x55aeb707d230 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd954c13640 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70a6630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd940009bb0 tx=0x7fd94000ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340431c0aeac9bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd935ffb640 1 -- 172.21.15.64:0/281602721 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94002d070 con 0x55aeb707d230 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd935ffb640 1 -- 172.21.15.64:0/281602721 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd94000fc10 con 0x55aeb707d230 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.740+0000 7fd935ffb640 1 -- 172.21.15.64:0/281602721 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9400288d0 con 0x55aeb707d230 2024-05-09T09:21:28.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 -- 172.21.15.64:0/281602721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 msgr2=0x55aeb70a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70a6630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd940009bb0 tx=0x7fd94000ffd0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 -- 172.21.15.64:0/281602721 shutdown_connections 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb70a6630 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70780c0 0x55aeb70a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 --2- 172.21.15.64:0/281602721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70776d0 0x55aeb7076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.741+0000 7fd959385740 1 -- 172.21.15.64:0/281602721 >> 172.21.15.64:0/281602721 conn(0x55aeb703f0a0 msgr2=0x55aeb70760f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.743+0000 7fd959385740 1 -- 172.21.15.64:0/281602721 shutdown_connections 2024-05-09T09:21:28.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.743+0000 7fd959385740 1 -- 172.21.15.64:0/281602721 wait complete. 2024-05-09T09:21:28.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 -- start start 2024-05-09T09:21:28.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 Processor -- start 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7087ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7088530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb7086600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd95019de40 con 0x55aeb70780c0 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd95019f290 con 0x55aeb70776d0 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd95019e4f0 con 0x55aeb707d230 2024-05-09T09:21:28.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.744+0000 7fd959385740 10 client.0 _ll_register_callbacks cb 0x7fd9440093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7087ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb7086600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7087ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51784/0 (socket says 172.21.15.64:51784) 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 -- 172.21.15.64:0/693250685 learned_addr learned my addr 172.21.15.64:0/693250685 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 -- 172.21.15.64:0/693250685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 msgr2=0x55aeb7086600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb7086600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 -- 172.21.15.64:0/693250685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 msgr2=0x55aeb7088530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7088530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.745+0000 7fd954c13640 1 -- 172.21.15.64:0/693250685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb7086e00 con 0x55aeb70776d0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.746+0000 7fd954c13640 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7087ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd940011540 tx=0x7fd940000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13db0b7ed46523f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.746+0000 7fd92f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.746+0000 7fd92f7fe640 1 -- 172.21.15.64:0/693250685 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94002d070 con 0x55aeb70776d0 2024-05-09T09:21:28.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.746+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94000fc10 con 0x55aeb70776d0 2024-05-09T09:21:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.746+0000 7fd92f7fe640 1 -- 172.21.15.64:0/693250685 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd94000f5c0 con 0x55aeb70776d0 2024-05-09T09:21:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.746+0000 7fd92f7fe640 1 -- 172.21.15.64:0/693250685 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd94000f7c0 con 0x55aeb70776d0 2024-05-09T09:21:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.747+0000 7fd959385740 10 client.27974 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.747+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55aeb71054a0 con 0x55aeb70776d0 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.748+0000 7fd92f7fe640 1 -- 172.21.15.64:0/693250685 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd94000fc10 con 0x55aeb70776d0 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.748+0000 7fd92f7fe640 1 -- 172.21.15.64:0/693250685 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7fd940046420 con 0x55aeb70776d0 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd959385740 10 client.27974 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd959385740 10 client.27974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd959385740 20 client.27974 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd959385740 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb7105dd0 0x55aeb71081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd959385740 4 client.27974 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:21:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aeb6fe3410 con 0x55aeb7105dd0 2024-05-09T09:21:28.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.749+0000 7fd94f7fe640 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb7105dd0 0x55aeb71081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.750+0000 7fd94f7fe640 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb7105dd0 0x55aeb71081f0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a95357369b6524 server_cookie=931acda63cfe0811 in_seq=0 out_seq=0 2024-05-09T09:21:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.752+0000 7fd92f7fe640 10 client.27974 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.752+0000 7fd92f7fe640 1 -- 172.21.15.64:0/693250685 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aeb6fe3410 con 0x55aeb7105dd0 2024-05-09T09:21:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.752+0000 7fd92f7fe640 10 client.27974 handle_command_reply: tid=0 2024-05-09T09:21:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.752+0000 7fd959385740 1 client.27974 shutdown 2024-05-09T09:21:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.752+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 msgr2=0x55aeb7087ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.752+0000 7fd959385740 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7087ff0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd940011540 tx=0x7fd940000bc0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 shutdown_connections 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55aeb7105dd0 0x55aeb71081f0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aeb707d230 0x55aeb7086600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aeb70780c0 0x55aeb7088530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 --2- 172.21.15.64:0/693250685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55aeb70776d0 0x55aeb7087ff0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 >> 172.21.15.64:0/693250685 conn(0x55aeb703f0a0 msgr2=0x55aeb7076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd92f7fe640 0 client.27974 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.753+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 shutdown_connections 2024-05-09T09:21:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.754+0000 7fd959385740 1 -- 172.21.15.64:0/693250685 wait complete. 2024-05-09T09:21:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.754+0000 7fd959385740 20 client.27974 trim_cache size 0 max 16384 2024-05-09T09:21:28.754 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:21:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:21:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "e6bbf302-351b-4ba7-9fa5-f48901065c2d", 2024-05-09T09:21:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:21:28.754 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:21:28.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9380a2d90 msgr2=0x7fd9380a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9380a2d90 0x7fd9380a5250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd9480c9e00 tx=0x7fd94801a5d0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 msgr2=0x7fd9501956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd9501956c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd948005e30 tx=0x7fd948003c10 comp rx=0 tx=0).stop 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 shutdown_connections 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd9380a2d90 0x7fd9380a5250 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd950105230 0x7fd95019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd950102880 0x7fd950195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 --2- 172.21.15.64:0/3944461691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9500fff10 0x7fd9501956c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 >> 172.21.15.64:0/3944461691 conn(0x7fd9500fbb30 msgr2=0x7fd9500fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.755+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 shutdown_connections 2024-05-09T09:21:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.756+0000 7fd955414640 1 -- 172.21.15.64:0/3944461691 wait complete. 2024-05-09T09:21:28.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:28 smithi064 ceph-mon[30149]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 677 op/s 2024-05-09T09:21:28.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:21:28.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:21:28.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:28 smithi139 ceph-mon[33599]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 677 op/s 2024-05-09T09:21:28.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:28 smithi179 ceph-mon[34086]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 677 op/s 2024-05-09T09:21:28.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.989+0000 7f4329c00640 1 -- 172.21.15.64:0/2496446717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324103430 msgr2=0x7f4324109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.989+0000 7f4329c00640 1 --2- 172.21.15.64:0/2496446717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324103430 0x7f4324109cc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f431c009ca0 tx=0x7f431c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.991+0000 7f4329c00640 1 -- 172.21.15.64:0/2496446717 shutdown_connections 2024-05-09T09:21:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.991+0000 7f4329c00640 1 --2- 172.21.15.64:0/2496446717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324103430 0x7f4324109cc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.991+0000 7f4329c00640 1 --2- 172.21.15.64:0/2496446717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324102a70 0x7f4324102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.991+0000 7f4329c00640 1 --2- 172.21.15.64:0/2496446717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4324101880 0x7f4324101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.991+0000 7f4329c00640 1 -- 172.21.15.64:0/2496446717 >> 172.21.15.64:0/2496446717 conn(0x7f43240fce10 msgr2=0x7f43240ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.993+0000 7f4329c00640 1 -- 172.21.15.64:0/2496446717 shutdown_connections 2024-05-09T09:21:28.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.993+0000 7f4329c00640 1 -- 172.21.15.64:0/2496446717 wait complete. 2024-05-09T09:21:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.995+0000 7f4329c00640 1 Processor -- start 2024-05-09T09:21:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.995+0000 7f4329c00640 1 -- start start 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.996+0000 7f4329c00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 0x7f43240755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.996+0000 7f4329c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324102a70 0x7f4324073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.996+0000 7f4329c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4324103430 0x7f4324074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.996+0000 7f43237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 0x7f43240755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.996+0000 7f43237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 0x7f43240755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51806/0 (socket says 172.21.15.64:51806) 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.996+0000 7f43237fe640 1 -- 172.21.15.64:0/3071561529 learned_addr learned my addr 172.21.15.64:0/3071561529 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f432410d3e0 con 0x7f4324102a70 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f432410d260 con 0x7f4324101880 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f4323fff640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4324103430 0x7f4324074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f432410d560 con 0x7f4324103430 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f431affd640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324102a70 0x7f4324073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f43237fe640 1 -- 172.21.15.64:0/3071561529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4324103430 msgr2=0x7f4324074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f43237fe640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4324103430 0x7f4324074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f43237fe640 1 -- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324102a70 msgr2=0x7f4324073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.997+0000 7f43237fe640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324102a70 0x7f4324073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.998+0000 7f43237fe640 1 -- 172.21.15.64:0/3071561529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240749f0 con 0x7f4324101880 2024-05-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.998+0000 7f431affd640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324102a70 0x7f4324073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.998+0000 7f43237fe640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 0x7f43240755e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f430c00e9c0 tx=0x7f430c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.998+0000 7f43217fa640 1 -- 172.21.15.64:0/3071561529 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00cde0 con 0x7f4324101880 2024-05-09T09:21:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.999+0000 7f43217fa640 1 -- 172.21.15.64:0/3071561529 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f430c0108e0 con 0x7f4324101880 2024-05-09T09:21:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.999+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43241a1b70 con 0x7f4324101880 2024-05-09T09:21:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.999+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241a2060 con 0x7f4324101880 2024-05-09T09:21:28.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:28.999+0000 7f43217fa640 1 -- 172.21.15.64:0/3071561529 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c018950 con 0x7f4324101880 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 Processor -- start 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 -- start start 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad45311d00 con 0x55ad453ade30 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad452b94c0 con 0x55ad453a8cc0 2024-05-09T09:21:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.004+0000 7f432c9ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad4530dcc0 con 0x55ad453a82d0 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57316/0 (socket says 172.21.15.64:57316) 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 -- 172.21.15.64:0/4222139208 learned_addr learned my addr 172.21.15.64:0/4222139208 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 -- 172.21.15.64:0/4222139208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 msgr2=0x55ad453ad7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453ad7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 -- 172.21.15.64:0/4222139208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 msgr2=0x55ad453c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.005+0000 7f4323fff640 1 -- 172.21.15.64:0/4222139208 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad450ef7d0 con 0x55ad453a82d0 2024-05-09T09:21:29.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.006+0000 7f43217fa640 1 -- 172.21.15.64:0/3071561529 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f430c018bd0 con 0x7f4324101880 2024-05-09T09:21:29.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.007+0000 7f43217fa640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43100a2d60 0x7f43100a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.008+0000 7f4323fff640 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453a86f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f431c02aeb0 tx=0x7f431c0093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0be1a3c1e63aff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:29.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.008+0000 7f43217fa640 1 -- 172.21.15.64:0/3071561529 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f430c0cf900 con 0x7f4324101880 2024-05-09T09:21:29.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.008+0000 7f43237fe640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43100a2d60 0x7f43100a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.008+0000 7f4319ffb640 1 -- 172.21.15.64:0/4222139208 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c031650 con 0x55ad453a82d0 2024-05-09T09:21:29.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.008+0000 7f4319ffb640 1 -- 172.21.15.64:0/4222139208 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f431c036030 con 0x55ad453a82d0 2024-05-09T09:21:29.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.008+0000 7f4319ffb640 1 -- 172.21.15.64:0/4222139208 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c004380 con 0x55ad453a82d0 2024-05-09T09:21:29.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 -- 172.21.15.64:0/4222139208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 msgr2=0x55ad453a86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:29.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f43237fe640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43100a2d60 0x7f43100a5220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f430c00efd0 tx=0x7f430c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453a86f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f431c02aeb0 tx=0x7f431c0093a0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 -- 172.21.15.64:0/4222139208 shutdown_connections 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 --2- 172.21.15.64:0/4222139208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453a86f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 -- 172.21.15.64:0/4222139208 >> 172.21.15.64:0/4222139208 conn(0x55ad453b0db0 msgr2=0x55ad453b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 -- 172.21.15.64:0/4222139208 shutdown_connections 2024-05-09T09:21:29.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.010+0000 7f432c9ed740 1 -- 172.21.15.64:0/4222139208 wait complete. 2024-05-09T09:21:29.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.011+0000 7f432c9ed740 1 -- start start 2024-05-09T09:21:29.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.012+0000 7f432c9ed740 1 Processor -- start 2024-05-09T09:21:29.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.013+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.013+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 0x55ad453b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.013+0000 7f4323fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.013+0000 7f4323fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51812/0 (socket says 172.21.15.64:51812) 2024-05-09T09:21:29.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.013+0000 7f4323fff640 1 -- 172.21.15.64:0/1124072963 learned_addr learned my addr 172.21.15.64:0/1124072963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:29.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.013+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.014+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad453b53e0 con 0x55ad453ade30 2024-05-09T09:21:29.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.014+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad452ccb90 con 0x55ad453a82d0 2024-05-09T09:21:29.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.014+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad453b5560 con 0x55ad453a8cc0 2024-05-09T09:21:29.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.014+0000 7f432c9ed740 10 client.0 _ll_register_callbacks cb 0x55ad45308da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:29.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.015+0000 7f4323fff640 1 -- 172.21.15.64:0/1124072963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 msgr2=0x55ad453b75c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:29.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.015+0000 7f4323fff640 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 0x55ad453b75c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.015+0000 7f4323fff640 1 -- 172.21.15.64:0/1124072963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 msgr2=0x55ad453be600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:29.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.015+0000 7f4323fff640 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.015+0000 7f4323fff640 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad450253f0 con 0x55ad453a82d0 2024-05-09T09:21:29.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.016+0000 7f4323fff640 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b7080 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f431c005c40 tx=0x7f431c003e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1aa034d3afdc0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:29.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.016+0000 7f4302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:29.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.016+0000 7f4302ffd640 1 -- 172.21.15.64:0/1124072963 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c004120 con 0x55ad453a82d0 2024-05-09T09:21:29.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.016+0000 7f4302ffd640 1 -- 172.21.15.64:0/1124072963 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f431c0042c0 con 0x55ad453a82d0 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.021+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f431c036030 con 0x55ad453a82d0 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.021+0000 7f4302ffd640 1 -- 172.21.15.64:0/1124072963 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f431c0044c0 con 0x55ad453a82d0 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 10 client.27983 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ad44fe50f0 con 0x55ad453a82d0 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f4302ffd640 1 -- 172.21.15.64:0/1124072963 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f431c004660 con 0x55ad453a82d0 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f4302ffd640 1 -- 172.21.15.64:0/1124072963 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f431c07a030 con 0x55ad453a82d0 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 10 client.27983 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 10 client.27983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:29.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 20 client.27983 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:29.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad453c2af0 0x55ad453c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 4 client.27983 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:29.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f431c036030 con 0x55ad453c2af0 2024-05-09T09:21:29.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.022+0000 7f431affd640 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad453c2af0 0x55ad453c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.024+0000 7f4302ffd640 10 client.27983 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:29.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.025+0000 7f431affd640 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad453c2af0 0x55ad453c4f00 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74b0f2de479b680 server_cookie=e793be4a8b18e0b6 in_seq=0 out_seq=0 2024-05-09T09:21:29.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.026+0000 7f4302ffd640 1 -- 172.21.15.64:0/1124072963 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f431c036030 con 0x55ad453c2af0 2024-05-09T09:21:29.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.026+0000 7f4302ffd640 10 client.27983 handle_command_reply: tid=0 2024-05-09T09:21:29.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.026+0000 7f432c9ed740 1 client.27983 shutdown 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 msgr2=0x55ad453b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b7080 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f431c005c40 tx=0x7f431c003e50 comp rx=0 tx=0).stop 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 shutdown_connections 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad453c2af0 0x55ad453c4f00 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453be600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 0x55ad453b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 --2- 172.21.15.64:0/1124072963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b7080 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 >> 172.21.15.64:0/1124072963 conn(0x55ad453b0db0 msgr2=0x55ad453a77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:29.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f4302ffd640 0 client.27983 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:29.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 shutdown_connections 2024-05-09T09:21:29.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.027+0000 7f432c9ed740 1 -- 172.21.15.64:0/1124072963 wait complete. 2024-05-09T09:21:29.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.028+0000 7f432c9ed740 20 client.27983 trim_cache size 0 max 16384 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.062+0000 7f432c9ed740 1 Processor -- start 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.062+0000 7f432c9ed740 1 -- start start 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.062+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 0x55ad453be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51824/0 (socket says 172.21.15.64:51824) 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 -- 172.21.15.64:0/334391013 learned_addr learned my addr 172.21.15.64:0/334391013 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f432c9ed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43040125e0 con 0x55ad453ade30 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f430400fe80 con 0x55ad453a82d0 2024-05-09T09:21:29.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f430400dcd0 con 0x55ad453a8cc0 2024-05-09T09:21:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 -- 172.21.15.64:0/334391013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 msgr2=0x55ad453be590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 0x55ad453be590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 -- 172.21.15.64:0/334391013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 msgr2=0x55ad453b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.063+0000 7f4323fff640 1 -- 172.21.15.64:0/334391013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad453b1a80 con 0x55ad453a82d0 2024-05-09T09:21:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.064+0000 7f4323fff640 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b6c90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55ad453b14d0 tx=0x7f431c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a12dce6508b50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.064+0000 7f43197fa640 1 -- 172.21.15.64:0/334391013 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c083040 con 0x55ad453a82d0 2024-05-09T09:21:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.064+0000 7f43197fa640 1 -- 172.21.15.64:0/334391013 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f431c036070 con 0x55ad453a82d0 2024-05-09T09:21:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.064+0000 7f43197fa640 1 -- 172.21.15.64:0/334391013 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0041a0 con 0x55ad453a82d0 2024-05-09T09:21:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.065+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 msgr2=0x55ad453b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:29.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.065+0000 7f432c9ed740 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b6c90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55ad453b14d0 tx=0x7f431c02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.066+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 shutdown_connections 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.066+0000 7f432c9ed740 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.066+0000 7f432c9ed740 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a8cc0 0x55ad453be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.066+0000 7f432c9ed740 1 --2- 172.21.15.64:0/334391013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a82d0 0x55ad453b6c90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.066+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 >> 172.21.15.64:0/334391013 conn(0x55ad453b0db0 msgr2=0x55ad453aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.066+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 shutdown_connections 2024-05-09T09:21:29.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.067+0000 7f432c9ed740 1 -- 172.21.15.64:0/334391013 wait complete. 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.067+0000 7f432c9ed740 1 -- start start 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.067+0000 7f432c9ed740 1 Processor -- start 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453bbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f430400e340 con 0x55ad453ade30 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f430400f7b0 con 0x55ad453a8cc0 2024-05-09T09:21:29.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f430400ea10 con 0x55ad453a82d0 2024-05-09T09:21:29.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f432c9ed740 10 client.0 _ll_register_callbacks cb 0x7f431c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:29.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f431affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453bbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f431affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453bbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51834/0 (socket says 172.21.15.64:51834) 2024-05-09T09:21:29.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.068+0000 7f431affd640 1 -- 172.21.15.64:0/3400232434 learned_addr learned my addr 172.21.15.64:0/3400232434 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:29.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.069+0000 7f431affd640 1 -- 172.21.15.64:0/3400232434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 msgr2=0x55ad453bb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:29.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.069+0000 7f431affd640 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.069+0000 7f431affd640 1 -- 172.21.15.64:0/3400232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 msgr2=0x55ad453bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:29.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.069+0000 7f431affd640 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.069+0000 7f431affd640 1 -- 172.21.15.64:0/3400232434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad453bd330 con 0x55ad453a8cc0 2024-05-09T09:21:29.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.070+0000 7f431affd640 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453bbd30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55ad453b1c80 tx=0x7f4314009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e85e8b5f8d0bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:29.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.070+0000 7f4302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:29.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.071+0000 7f4302ffd640 1 -- 172.21.15.64:0/3400232434 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314013e60 con 0x55ad453a8cc0 2024-05-09T09:21:29.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.071+0000 7f4302ffd640 1 -- 172.21.15.64:0/3400232434 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f431401b3b0 con 0x55ad453a8cc0 2024-05-09T09:21:29.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.071+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f431c036070 con 0x55ad453a8cc0 2024-05-09T09:21:29.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.071+0000 7f4302ffd640 1 -- 172.21.15.64:0/3400232434 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f431402a050 con 0x55ad453a8cc0 2024-05-09T09:21:29.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.071+0000 7f432c9ed740 10 client.27989 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.071+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ad453be6e0 con 0x55ad453a8cc0 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f4302ffd640 1 -- 172.21.15.64:0/3400232434 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f431c036070 con 0x55ad453a8cc0 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f4302ffd640 1 -- 172.21.15.64:0/3400232434 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f4314011c70 con 0x55ad453a8cc0 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f432c9ed740 10 client.27989 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f432c9ed740 10 client.27989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f432c9ed740 20 client.27989 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f432c9ed740 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad45429770 0x55ad4542bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f432c9ed740 4 client.27989 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:21:29.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.072+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad4531f430 con 0x55ad45429770 2024-05-09T09:21:29.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.074+0000 7f4323fff640 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad45429770 0x55ad4542bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.077+0000 7f4323fff640 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad45429770 0x55ad4542bb90 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae19faf79efb6ed server_cookie=ca452e77deca36a8 in_seq=0 out_seq=0 2024-05-09T09:21:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.077+0000 7f4302ffd640 10 client.27989 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.078+0000 7f4302ffd640 1 -- 172.21.15.64:0/3400232434 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ad4531f430 con 0x55ad45429770 2024-05-09T09:21:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.078+0000 7f4302ffd640 10 client.27989 handle_command_reply: tid=0 2024-05-09T09:21:29.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.078+0000 7f432c9ed740 1 client.27989 shutdown 2024-05-09T09:21:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.078+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 msgr2=0x55ad453bbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:29.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.078+0000 7f432c9ed740 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453bbd30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55ad453b1c80 tx=0x7f4314009d80 comp rx=0 tx=0).stop 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.080+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 shutdown_connections 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.080+0000 7f432c9ed740 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ad45429770 0x55ad4542bb90 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.080+0000 7f432c9ed740 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad453ade30 0x55ad453bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.080+0000 7f432c9ed740 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad453a8cc0 0x55ad453bbd30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.080+0000 7f432c9ed740 1 --2- 172.21.15.64:0/3400232434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad453a82d0 0x55ad453bb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.080+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 >> 172.21.15.64:0/3400232434 conn(0x55ad453b0db0 msgr2=0x55ad453b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:29.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.081+0000 7f4302ffd640 0 client.27989 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:29.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.082+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 shutdown_connections 2024-05-09T09:21:29.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.082+0000 7f432c9ed740 1 -- 172.21.15.64:0/3400232434 wait complete. 2024-05-09T09:21:29.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.082+0000 7f432c9ed740 20 client.27989 trim_cache size 0 max 16384 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: "e6bbf302-351b-4ba7-9fa5-f48901065c2d": { 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "e6bbf302-351b-4ba7-9fa5-f48901065c2d", 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:21:29.084 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43100a2d60 msgr2=0x7f43100a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43100a2d60 0x7f43100a5220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f430c00efd0 tx=0x7f430c01f000 comp rx=0 tx=0).stop 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 msgr2=0x7f43240755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 0x7f43240755e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f430c00e9c0 tx=0x7f430c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 shutdown_connections 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f43100a2d60 0x7f43100a5220 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4324103430 0x7f4324074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4324102a70 0x7f4324073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 --2- 172.21.15.64:0/3071561529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324101880 0x7f43240755e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 >> 172.21.15.64:0/3071561529 conn(0x7f43240fce10 msgr2=0x7f4324105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.085+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 shutdown_connections 2024-05-09T09:21:29.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:29.086+0000 7f4329c00640 1 -- 172.21.15.64:0/3071561529 wait complete. 2024-05-09T09:21:29.112 INFO:tasks.cephfs.filesystem:scrub status for tag:e6bbf302-351b-4ba7-9fa5-f48901065c2d - {'path': '/', 'tag': 'e6bbf302-351b-4ba7-9fa5-f48901065c2d', 'options': 'recursive,force'} 2024-05-09T09:21:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:29 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:21:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:29 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:29 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:21:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:29 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:21:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:29 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:29.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:29 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:29 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:21:29.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:29 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:29 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:21:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:29 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:21:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:29 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:29.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:29 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:30.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:29 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:21:30.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:29 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:30.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:29 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:21:30.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:29 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:21:30.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:29 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:30.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:29 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:30 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:21:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:30 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:30.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:30 smithi064 ceph-mon[30149]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.5 MiB/s wr, 879 op/s 2024-05-09T09:21:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:30 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:21:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:30 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:30 smithi179 ceph-mon[34086]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.5 MiB/s wr, 879 op/s 2024-05-09T09:21:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:30 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:21:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:30 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:21:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:30 smithi139 ceph-mon[33599]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.5 MiB/s wr, 879 op/s 2024-05-09T09:21:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 670.0 s, 723.6 tps, lat 138.090 ms stddev 69.851 2024-05-09T09:21:32.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:32 smithi064 ceph-mon[30149]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-05-09T09:21:32.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:32 smithi179 ceph-mon[34086]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-05-09T09:21:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:32 smithi139 ceph-mon[33599]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-05-09T09:21:34.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:34 smithi064 ceph-mon[30149]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.1 MiB/s wr, 803 op/s 2024-05-09T09:21:34.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:34 smithi179 ceph-mon[34086]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.1 MiB/s wr, 803 op/s 2024-05-09T09:21:35.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:34 smithi139 ceph-mon[33599]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.1 MiB/s wr, 803 op/s 2024-05-09T09:21:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 675.0 s, 785.6 tps, lat 127.693 ms stddev 85.779 2024-05-09T09:21:36.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:36 smithi064 ceph-mon[30149]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.8 MiB/s wr, 804 op/s 2024-05-09T09:21:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:36 smithi179 ceph-mon[34086]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.8 MiB/s wr, 804 op/s 2024-05-09T09:21:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:36 smithi139 ceph-mon[33599]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.8 MiB/s wr, 804 op/s 2024-05-09T09:21:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.355+0000 7f67f7996640 1 -- 172.21.15.64:0/1612316862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0100a60 msgr2=0x7f67f0100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.355+0000 7f67f7996640 1 --2- 172.21.15.64:0/1612316862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0100a60 0x7f67f0100ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009900 tx=0x7f67e002a410 comp rx=0 tx=0).stop 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.355+0000 7f67f7996640 1 -- 172.21.15.64:0/1612316862 shutdown_connections 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.355+0000 7f67f7996640 1 --2- 172.21.15.64:0/1612316862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67f0101400 0x7f67f0101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.356+0000 7f67f7996640 1 --2- 172.21.15.64:0/1612316862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0100a60 0x7f67f0100ec0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.356+0000 7f67f7996640 1 --2- 172.21.15.64:0/1612316862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100100 0x7f67f0100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.356+0000 7f67f7996640 1 -- 172.21.15.64:0/1612316862 >> 172.21.15.64:0/1612316862 conn(0x7f67f00fbd00 msgr2=0x7f67f00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.356+0000 7f67f7996640 1 -- 172.21.15.64:0/1612316862 shutdown_connections 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.356+0000 7f67f7996640 1 -- 172.21.15.64:0/1612316862 wait complete. 2024-05-09T09:21:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 Processor -- start 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 -- start start 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67f0100100 0x7f67f00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 0x7f67f0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0101400 0x7f67f01ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010f600 con 0x7f67f0100a60 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010f480 con 0x7f67f0101400 2024-05-09T09:21:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f7996640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010f780 con 0x7f67f0100100 2024-05-09T09:21:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f4f31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 0x7f67f0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f5732640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67f0100100 0x7f67f00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f4f31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 0x7f67f0070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35084/0 (socket says 172.21.15.64:35084) 2024-05-09T09:21:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.357+0000 7f67f4f31640 1 -- 172.21.15.64:0/1412861912 learned_addr learned my addr 172.21.15.64:0/1412861912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.358+0000 7f67f5f33640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0101400 0x7f67f01ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.358+0000 7f67f4f31640 1 -- 172.21.15.64:0/1412861912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67f0100100 msgr2=0x7f67f00700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.358+0000 7f67f4f31640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67f0100100 0x7f67f00700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.358+0000 7f67f4f31640 1 -- 172.21.15.64:0/1412861912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0101400 msgr2=0x7f67f01ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.358+0000 7f67f4f31640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0101400 0x7f67f01ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.358+0000 7f67f4f31640 1 -- 172.21.15.64:0/1412861912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f00737b0 con 0x7f67f0100a60 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.359+0000 7f67f4f31640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 0x7f67f0070630 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f67e00314e0 tx=0x7f67e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.359+0000 7f67ee7fc640 1 -- 172.21.15.64:0/1412861912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e003f070 con 0x7f67f0100a60 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.359+0000 7f67ee7fc640 1 -- 172.21.15.64:0/1412861912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e0031c40 con 0x7f67f0100a60 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.359+0000 7f67ee7fc640 1 -- 172.21.15.64:0/1412861912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0035030 con 0x7f67f0100a60 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.359+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f0106330 con 0x7f67f0100a60 2024-05-09T09:21:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.360+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01a5ed0 con 0x7f67f0100a60 2024-05-09T09:21:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.364+0000 7f67fa777740 1 Processor -- start 2024-05-09T09:21:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.364+0000 7f67fa777740 1 -- start start 2024-05-09T09:21:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.364+0000 7f67fa777740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 0x559373ef3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.364+0000 7f67fa777740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3c40 0x559373ef8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.364+0000 7f67f5f33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 0x559373ef3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.364+0000 7f67f5f33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 0x559373ef3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:58526/0 (socket says 172.21.15.64:58526) 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67fa777740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef8db0 0x559373efb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67fa777740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559373e57340 con 0x559373ef8db0 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67fa777740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559373dff290 con 0x559373ef3c40 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67fa777740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559373e53300 con 0x559373ef3250 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67f5f33640 1 -- 172.21.15.64:0/133725016 learned_addr learned my addr 172.21.15.64:0/133725016 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67f5f33640 1 -- 172.21.15.64:0/133725016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3c40 msgr2=0x559373ef8740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67f5f33640 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3c40 0x559373ef8740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67f5f33640 1 -- 172.21.15.64:0/133725016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef8db0 msgr2=0x559373efb240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67f5f33640 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef8db0 0x559373efb240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.365+0000 7f67f5f33640 1 -- 172.21.15.64:0/133725016 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559373e238e0 con 0x559373ef3250 2024-05-09T09:21:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.367+0000 7f67f5f33640 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 0x559373ef3670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009fc0 tx=0x7f67e8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1e00b54f0be52b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.367+0000 7f67d77fe640 1 -- 172.21.15.64:0/133725016 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8003b70 con 0x559373ef3250 2024-05-09T09:21:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.367+0000 7f67d77fe640 1 -- 172.21.15.64:0/133725016 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e8003d10 con 0x559373ef3250 2024-05-09T09:21:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.367+0000 7f67d77fe640 1 -- 172.21.15.64:0/133725016 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011a00 con 0x559373ef3250 2024-05-09T09:21:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.367+0000 7f67ee7fc640 1 -- 172.21.15.64:0/1412861912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67e0042070 con 0x7f67f0100a60 2024-05-09T09:21:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.368+0000 7f67fa777740 1 -- 172.21.15.64:0/133725016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 msgr2=0x559373ef3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.368+0000 7f67fa777740 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 0x559373ef3670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009fc0 tx=0x7f67e8010860 comp rx=0 tx=0).stop 2024-05-09T09:21:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 -- 172.21.15.64:0/133725016 shutdown_connections 2024-05-09T09:21:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef8db0 0x559373efb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3c40 0x559373ef8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 --2- 172.21.15.64:0/133725016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3250 0x559373ef3670 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 -- 172.21.15.64:0/133725016 >> 172.21.15.64:0/133725016 conn(0x559373eee8c0 msgr2=0x559373eeecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 -- 172.21.15.64:0/133725016 shutdown_connections 2024-05-09T09:21:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67fa777740 1 -- 172.21.15.64:0/133725016 wait complete. 2024-05-09T09:21:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.369+0000 7f67ee7fc640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f67d80a2d30 0x7f67d80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 -- start start 2024-05-09T09:21:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 Processor -- start 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67ee7fc640 1 -- 172.21.15.64:0/1412861912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67e00efb80 con 0x7f67f0100a60 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373efd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3c40 0x559373efda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef8db0 0x559373f04ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559373efbd60 con 0x559373ef3c40 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559373e12790 con 0x559373ef3250 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559373efbee0 con 0x559373ef8db0 2024-05-09T09:21:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67fa777740 10 client.0 _ll_register_callbacks cb 0x559373e4da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.370+0000 7f67f5f33640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f67d80a2d30 0x7f67d80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.371+0000 7f67f5732640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373efd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.371+0000 7f67f5f33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef8db0 0x559373f04ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.371+0000 7f67f5732640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373efd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36644/0 (socket says 172.21.15.64:36644) 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.371+0000 7f67f5732640 1 -- 172.21.15.64:0/2243284297 learned_addr learned my addr 172.21.15.64:0/2243284297 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.371+0000 7f67f4f31640 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3c40 0x559373efda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.372+0000 7f67f4f31640 1 -- 172.21.15.64:0/2243284297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef8db0 msgr2=0x559373f04ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.372+0000 7f67f4f31640 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef8db0 0x559373f04ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.372+0000 7f67f4f31640 1 -- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 msgr2=0x559373efd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.372+0000 7f67f4f31640 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373efd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.372+0000 7f67f4f31640 1 -- 172.21.15.64:0/2243284297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559373b6b210 con 0x559373ef3c40 2024-05-09T09:21:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.372+0000 7f67f4f31640 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3c40 0x559373efda90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f67e00ed290 tx=0x7f67e00bbbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce57af948449e82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67f5f33640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f67d80a2d30 0x7f67d80a51f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f67e8010d60 tx=0x7f67e8010fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67d4ff9640 1 -- 172.21.15.64:0/2243284297 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0049070 con 0x559373ef3c40 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e8003d10 con 0x559373ef3c40 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67d4ff9640 1 -- 172.21.15.64:0/2243284297 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67e00b8660 con 0x559373ef3c40 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67d4ff9640 1 -- 172.21.15.64:0/2243284297 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f67e00dc0b0 con 0x559373ef3c40 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67fa777740 10 client.18531 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559373c37620 con 0x559373ef3c40 2024-05-09T09:21:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.373+0000 7f67d4ff9640 1 -- 172.21.15.64:0/2243284297 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e8003d10 con 0x559373ef3c40 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67d4ff9640 1 -- 172.21.15.64:0/2243284297 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f67e00bc250 con 0x559373ef3c40 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67fa777740 10 client.18531 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67fa777740 10 client.18531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67fa777740 20 client.18531 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67fa777740 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373f09010 0x559373f0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67fa777740 4 client.18531 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559373e64a70 con 0x559373f09010 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.374+0000 7f67f5732640 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373f09010 0x559373f0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.375+0000 7f67f5732640 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373f09010 0x559373f0b400 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e4a5d494f77b49 server_cookie=a41186db6515c1c9 in_seq=0 out_seq=0 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.375+0000 7f67d4ff9640 10 client.18531 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.376+0000 7f67d4ff9640 1 -- 172.21.15.64:0/2243284297 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559373e64a70 con 0x559373f09010 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.376+0000 7f67d4ff9640 10 client.18531 handle_command_reply: tid=0 2024-05-09T09:21:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.376+0000 7f67fa777740 1 client.18531 shutdown 2024-05-09T09:21:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3c40 msgr2=0x559373efda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3c40 0x559373efda90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f67e00ed290 tx=0x7f67e00bbbf0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 shutdown_connections 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373f09010 0x559373f0b400 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef8db0 0x559373f04ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3c40 0x559373efda90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 --2- 172.21.15.64:0/2243284297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373efd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 >> 172.21.15.64:0/2243284297 conn(0x559373eee8c0 msgr2=0x559373eeda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67d4ff9640 0 client.18531 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 shutdown_connections 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 1 -- 172.21.15.64:0/2243284297 wait complete. 2024-05-09T09:21:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.377+0000 7f67fa777740 20 client.18531 trim_cache size 0 max 16384 2024-05-09T09:21:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.410+0000 7f67fa777740 1 Processor -- start 2024-05-09T09:21:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.410+0000 7f67fa777740 1 -- start start 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.411+0000 7f67fa777740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373eee8c0 0x559373ef1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.411+0000 7f67fa777740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373ef2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.411+0000 7f67fa777740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f04a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.411+0000 7f67fa777740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01c4210 con 0x559373eee8c0 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f5f33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f04a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373ef2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373ef2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36660/0 (socket says 172.21.15.64:36660) 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 -- 172.21.15.64:0/3169925694 learned_addr learned my addr 172.21.15.64:0/3169925694 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f5732640 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373eee8c0 0x559373ef1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01c1ab0 con 0x559373ef3250 2024-05-09T09:21:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01bfa50 con 0x559373ef3c40 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 -- 172.21.15.64:0/3169925694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 msgr2=0x559373f04a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f04a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 -- 172.21.15.64:0/3169925694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373eee8c0 msgr2=0x559373ef1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373eee8c0 0x559373ef1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.412+0000 7f67f4f31640 1 -- 172.21.15.64:0/3169925694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559373eecf00 con 0x559373ef3250 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.413+0000 7f67f4f31640 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373ef2290 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f67e00bbf80 tx=0x7f67e0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363e6beb9f456f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.413+0000 7f67d6ffd640 1 -- 172.21.15.64:0/3169925694 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0049070 con 0x559373ef3250 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.413+0000 7f67d6ffd640 1 -- 172.21.15.64:0/3169925694 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e00bbff0 con 0x559373ef3250 2024-05-09T09:21:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.413+0000 7f67d6ffd640 1 -- 172.21.15.64:0/3169925694 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e00b8660 con 0x559373ef3250 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 msgr2=0x559373ef2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373ef2290 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f67e00bbf80 tx=0x7f67e0009260 comp rx=0 tx=0).stop 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 shutdown_connections 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f04a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373ef3250 0x559373ef2290 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 --2- 172.21.15.64:0/3169925694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373eee8c0 0x559373ef1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 >> 172.21.15.64:0/3169925694 conn(0x559373ef88b0 msgr2=0x559373ef1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 shutdown_connections 2024-05-09T09:21:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.414+0000 7f67fa777740 1 -- 172.21.15.64:0/3169925694 wait complete. 2024-05-09T09:21:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.415+0000 7f67fa777740 1 -- start start 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.415+0000 7f67fa777740 1 Processor -- start 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.416+0000 7f67fa777740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373eee8c0 0x559373f024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.416+0000 7f67fa777740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3250 0x559373f02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.417+0000 7f67fa777740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f03a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.417+0000 7f67fa777740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01bff70 con 0x559373ef3250 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.417+0000 7f67fa777740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01c13e0 con 0x559373eee8c0 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.417+0000 7f67fa777740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01c0640 con 0x559373ef3c40 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.417+0000 7f67fa777740 10 client.0 _ll_register_callbacks cb 0x7f67e0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.418+0000 7f67f5f33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f03a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.418+0000 7f67f4f31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3250 0x559373f02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.418+0000 7f67f5f33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f03a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:58546/0 (socket says 172.21.15.64:58546) 2024-05-09T09:21:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.418+0000 7f67f5f33640 1 -- 172.21.15.64:0/128735317 learned_addr learned my addr 172.21.15.64:0/128735317 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.418+0000 7f67f5732640 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373eee8c0 0x559373f024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67f4f31640 1 -- 172.21.15.64:0/128735317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 msgr2=0x559373f03a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67f4f31640 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f03a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67f4f31640 1 -- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373eee8c0 msgr2=0x559373f024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67f4f31640 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373eee8c0 0x559373f024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67f4f31640 1 -- 172.21.15.64:0/128735317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559373f04020 con 0x559373ef3250 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67f4f31640 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3250 0x559373f02a20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005e00 tx=0x7f67e00319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661be5904a8abc9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67d4ff9640 1 -- 172.21.15.64:0/128735317 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e009f060 con 0x559373ef3250 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e00bbff0 con 0x559373ef3250 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.419+0000 7f67d4ff9640 1 -- 172.21.15.64:0/128735317 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67e00bb650 con 0x559373ef3250 2024-05-09T09:21:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.420+0000 7f67d4ff9640 1 -- 172.21.15.64:0/128735317 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f67e00a6050 con 0x559373ef3250 2024-05-09T09:21:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.421+0000 7f67fa777740 10 client.18537 fetch_fsmap learned FSMap version 46 2024-05-09T09:21:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.421+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559373f04020 con 0x559373ef3250 2024-05-09T09:21:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.421+0000 7f67d4ff9640 1 -- 172.21.15.64:0/128735317 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e00bbff0 con 0x559373ef3250 2024-05-09T09:21:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.421+0000 7f67d4ff9640 1 -- 172.21.15.64:0/128735317 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8350+0+0 (secure 0 0 0) 0x7f67e00cc070 con 0x559373ef3250 2024-05-09T09:21:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.423+0000 7f67fa777740 10 client.18537 fetch_fsmap finished waiting for FSMap version 46 2024-05-09T09:21:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.423+0000 7f67fa777740 10 client.18537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.423+0000 7f67fa777740 20 client.18537 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.423+0000 7f67fa777740 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373ef8db0 0x559373f71ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.423+0000 7f67fa777740 4 client.18537 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:21:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.423+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559373e5edc0 con 0x559373ef8db0 2024-05-09T09:21:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.424+0000 7f67f5732640 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373ef8db0 0x559373f71ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.425+0000 7f67f5732640 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373ef8db0 0x559373f71ed0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b44d95d1d220b27 server_cookie=dedabac4d8d04107 in_seq=0 out_seq=0 2024-05-09T09:21:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.425+0000 7f67d4ff9640 10 client.18537 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.465+0000 7f67d4ff9640 1 -- 172.21.15.64:0/128735317 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559373e5edc0 con 0x559373ef8db0 2024-05-09T09:21:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.465+0000 7f67d4ff9640 10 client.18537 handle_command_reply: tid=0 2024-05-09T09:21:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.467+0000 7f67fa777740 1 client.18537 shutdown 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3250 msgr2=0x559373f02a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3250 0x559373f02a20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005e00 tx=0x7f67e00319c0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 shutdown_connections 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559373ef8db0 0x559373f71ed0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559373ef3c40 0x559373f03a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559373ef3250 0x559373f02a20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 --2- 172.21.15.64:0/128735317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559373eee8c0 0x559373f024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 >> 172.21.15.64:0/128735317 conn(0x559373ef88b0 msgr2=0x559373ef1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.469+0000 7f67d4ff9640 0 client.18537 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.472+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 shutdown_connections 2024-05-09T09:21:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.472+0000 7f67fa777740 1 -- 172.21.15.64:0/128735317 wait complete. 2024-05-09T09:21:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.472+0000 7f67fa777740 20 client.18537 trim_cache size 0 max 16384 2024-05-09T09:21:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:21:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:21:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:21:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.479+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f67d80a2d30 msgr2=0x7f67d80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.479+0000 7f67f7996640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f67d80a2d30 0x7f67d80a51f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f67e8010d60 tx=0x7f67e8010fd0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 msgr2=0x7f67f0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 0x7f67f0070630 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f67e00314e0 tx=0x7f67e0003c10 comp rx=0 tx=0).stop 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 shutdown_connections 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f67d80a2d30 0x7f67d80a51f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67f0101400 0x7f67f01ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f0100a60 0x7f67f0070630 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 --2- 172.21.15.64:0/1412861912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67f0100100 0x7f67f00700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 >> 172.21.15.64:0/1412861912 conn(0x7f67f00fbd00 msgr2=0x7f67f00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 shutdown_connections 2024-05-09T09:21:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:37.480+0000 7f67f7996640 1 -- 172.21.15.64:0/1412861912 wait complete. 2024-05-09T09:21:38.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:38 smithi064 ceph-mon[30149]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.1 MiB/s wr, 715 op/s 2024-05-09T09:21:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:38 smithi179 ceph-mon[34086]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.1 MiB/s wr, 715 op/s 2024-05-09T09:21:39.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:38 smithi139 ceph-mon[33599]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.1 MiB/s wr, 715 op/s 2024-05-09T09:21:40.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:40 smithi064 ceph-mon[30149]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.4 MiB/s wr, 949 op/s 2024-05-09T09:21:40.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:21:40.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:40 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:21:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:40 smithi179 ceph-mon[34086]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.4 MiB/s wr, 949 op/s 2024-05-09T09:21:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:21:40.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:40 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:21:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:40 smithi139 ceph-mon[33599]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 8.4 MiB/s wr, 949 op/s 2024-05-09T09:21:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:21:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:40 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:21:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 680.0 s, 761.2 tps, lat 118.214 ms stddev 53.958 2024-05-09T09:21:41.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:41 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:21:41.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:41 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:21:42.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:41 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:21:42.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:42 smithi064 ceph-mon[30149]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 746 op/s 2024-05-09T09:21:42.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:42 smithi179 ceph-mon[34086]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 746 op/s 2024-05-09T09:21:43.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:42 smithi139 ceph-mon[33599]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 746 op/s 2024-05-09T09:21:44.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:44 smithi064 ceph-mon[30149]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.4 MiB/s wr, 867 op/s 2024-05-09T09:21:44.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:44 smithi179 ceph-mon[34086]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.4 MiB/s wr, 867 op/s 2024-05-09T09:21:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:44 smithi139 ceph-mon[33599]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.4 MiB/s wr, 867 op/s 2024-05-09T09:21:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 685.0 s, 807.2 tps, lat 136.313 ms stddev 121.395 2024-05-09T09:21:46.905 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:46 smithi064 ceph-mon[30149]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.2 MiB/s wr, 831 op/s 2024-05-09T09:21:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:46 smithi179 ceph-mon[34086]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.2 MiB/s wr, 831 op/s 2024-05-09T09:21:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:46 smithi139 ceph-mon[33599]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.2 MiB/s wr, 831 op/s 2024-05-09T09:21:48.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:48 smithi064 ceph-mon[30149]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.2 MiB/s wr, 716 op/s 2024-05-09T09:21:48.908 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:48 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:48.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:48 smithi179 ceph-mon[34086]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.2 MiB/s wr, 716 op/s 2024-05-09T09:21:48.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:48 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:49.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:48 smithi139 ceph-mon[33599]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.2 MiB/s wr, 716 op/s 2024-05-09T09:21:49.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:48 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:21:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:49.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:49.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:21:49.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:49.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:49.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:21:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:21:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:21:50.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:21:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 -- 172.21.15.64:0/3950294283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 msgr2=0x7fb9cc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/3950294283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 0x7fb9cc1092c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009ca0 tx=0x7fb9c402a8f0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 -- 172.21.15.64:0/3950294283 shutdown_connections 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/3950294283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 0x7fb9cc10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/3950294283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 0x7fb9cc1092c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/3950294283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 0x7fb9cc072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.649+0000 7fb9d26c7640 1 -- 172.21.15.64:0/3950294283 >> 172.21.15.64:0/3950294283 conn(0x7fb9cc06e6d0 msgr2=0x7fb9cc070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.650+0000 7fb9d26c7640 1 -- 172.21.15.64:0/3950294283 shutdown_connections 2024-05-09T09:21:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.650+0000 7fb9d26c7640 1 -- 172.21.15.64:0/3950294283 wait complete. 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 Processor -- start 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 -- start start 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 0x7fb9cc1a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 0x7fb9cc1a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 0x7fb9cc1add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc11c160 con 0x7fb9cc108e40 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc11bfe0 con 0x7fb9cc072b60 2024-05-09T09:21:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d26c7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc11c2e0 con 0x7fb9cc109800 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d1ec6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 0x7fb9cc1add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 0x7fb9cc1a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 0x7fb9cc1a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48666/0 (socket says 172.21.15.64:48666) 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.651+0000 7fb9d16c5640 1 -- 172.21.15.64:0/2490553238 learned_addr learned my addr 172.21.15.64:0/2490553238 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d1ec6640 1 -- 172.21.15.64:0/2490553238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 msgr2=0x7fb9cc1a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d1ec6640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 0x7fb9cc1a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d1ec6640 1 -- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 msgr2=0x7fb9cc1a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d1ec6640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 0x7fb9cc1a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d1ec6640 1 -- 172.21.15.64:0/2490553238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc1ae400 con 0x7fb9cc109800 2024-05-09T09:21:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d1ec6640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 0x7fb9cc1add90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800b3e0 tx=0x7fb9c800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9c27fc640 1 -- 172.21.15.64:0/2490553238 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8015020 con 0x7fb9cc109800 2024-05-09T09:21:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.652+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc1ae6f0 con 0x7fb9cc109800 2024-05-09T09:21:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.653+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1aecc0 con 0x7fb9cc109800 2024-05-09T09:21:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.653+0000 7fb9c27fc640 1 -- 172.21.15.64:0/2490553238 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c8011ce0 con 0x7fb9cc109800 2024-05-09T09:21:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.653+0000 7fb9c27fc640 1 -- 172.21.15.64:0/2490553238 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8010e00 con 0x7fb9cc109800 2024-05-09T09:21:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.657+0000 7fb9d663b740 1 Processor -- start 2024-05-09T09:21:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.657+0000 7fb9c27fc640 1 -- 172.21.15.64:0/2490553238 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb9c8018020 con 0x7fb9cc109800 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 -- start start 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55e3d0 0x555e1f55e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55edc0 0x555e1f5638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e1f4c2340 con 0x555e1f55e3d0 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e1f46a290 con 0x555e1f55edc0 2024-05-09T09:21:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d663b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555e1f4be300 con 0x555e1f563f30 2024-05-09T09:21:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d0ec4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55e3d0 0x555e1f55e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d0ec4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55e3d0 0x555e1f55e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44640/0 (socket says 172.21.15.64:44640) 2024-05-09T09:21:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d0ec4640 1 -- 172.21.15.64:0/3476996520 learned_addr learned my addr 172.21.15.64:0/3476996520 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.658+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55edc0 0x555e1f5638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d16c5640 1 -- 172.21.15.64:0/3476996520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 msgr2=0x555e1f5663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d16c5640 1 -- 172.21.15.64:0/3476996520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55e3d0 msgr2=0x555e1f55e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55e3d0 0x555e1f55e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d16c5640 1 -- 172.21.15.64:0/3476996520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1f48e8e0 con 0x555e1f55edc0 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9c27fc640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9b80a2c40 0x7fb9b80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.659+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55edc0 0x555e1f5638c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009a40 tx=0x7fb9bc009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f562c37878ea0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.660+0000 7fb9c27fc640 1 -- 172.21.15.64:0/2490553238 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9c80cea20 con 0x7fb9cc109800 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.660+0000 7fb9b77fe640 1 -- 172.21.15.64:0/3476996520 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc017070 con 0x555e1f55edc0 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.660+0000 7fb9b77fe640 1 -- 172.21.15.64:0/3476996520 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9bc013470 con 0x555e1f55edc0 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.660+0000 7fb9b77fe640 1 -- 172.21.15.64:0/3476996520 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc012630 con 0x555e1f55edc0 2024-05-09T09:21:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.660+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9b80a2c40 0x7fb9b80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 -- 172.21.15.64:0/3476996520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55edc0 msgr2=0x555e1f5638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55edc0 0x555e1f5638c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009a40 tx=0x7fb9bc009f00 comp rx=0 tx=0).stop 2024-05-09T09:21:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9b80a2c40 0x7fb9b80a5100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb9c402aee0 tx=0x7fb9c4005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 -- 172.21.15.64:0/3476996520 shutdown_connections 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55edc0 0x555e1f5638c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3476996520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55e3d0 0x555e1f55e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.661+0000 7fb9d663b740 1 -- 172.21.15.64:0/3476996520 >> 172.21.15.64:0/3476996520 conn(0x555e1f559a40 msgr2=0x555e1f559e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.662+0000 7fb9d663b740 1 -- 172.21.15.64:0/3476996520 shutdown_connections 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.662+0000 7fb9d663b740 1 -- 172.21.15.64:0/3476996520 wait complete. 2024-05-09T09:21:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.662+0000 7fb9d663b740 1 -- start start 2024-05-09T09:21:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.663+0000 7fb9d663b740 1 Processor -- start 2024-05-09T09:21:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.663+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f568640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.663+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f568b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f56fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f568640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f568640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48684/0 (socket says 172.21.15.64:48684) 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d0ec4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f568b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d0ec4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f568b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33024/0 (socket says 172.21.15.64:33024) 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d16c5640 1 -- 172.21.15.64:0/3647169037 learned_addr learned my addr 172.21.15.64:0/3647169037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e1f566ee0 con 0x555e1f563f30 2024-05-09T09:21:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e1f47d790 con 0x555e1f55e3d0 2024-05-09T09:21:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555e1f567060 con 0x555e1f55edc0 2024-05-09T09:21:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.664+0000 7fb9d1ec6640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f56fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d0ec4640 1 -- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 msgr2=0x555e1f568640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f568640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d0ec4640 1 -- 172.21.15.64:0/3647169037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 msgr2=0x555e1f56fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f56fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d0ec4640 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1f1d6210 con 0x555e1f55edc0 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f568640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d663b740 10 client.0 _ll_register_callbacks cb 0x555e1f4b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9bc013470 con 0x555e1f55edc0 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f568b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4006010 tx=0x7fb9c402adf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bd8c14157c82db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.665+0000 7fb9b4ff9640 10 client.28174 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.666+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/3647169037 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4059050 con 0x555e1f55edc0 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.666+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/3647169037 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9c402f640 con 0x555e1f55edc0 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.666+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/3647169037 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb9c404f070 con 0x555e1f55edc0 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.666+0000 7fb9d663b740 10 client.28174 fetch_fsmap learned FSMap version 47 2024-05-09T09:21:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.666+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=47}) -- 0x555e1f2a2620 con 0x555e1f55edc0 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/3647169037 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c405c020 con 0x555e1f55edc0 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d663b740 10 client.28174 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d663b740 10 client.28174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d663b740 20 client.28174 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f573d40 0x555e1f576200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d663b740 4 client.28174 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e1f4cfa70 con 0x555e1f573d40 2024-05-09T09:21:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.667+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f573d40 0x555e1f576200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.670+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f573d40 0x555e1f576200 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1ea7367c6df82b server_cookie=25fe030976fb6784 in_seq=0 out_seq=0 2024-05-09T09:21:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.670+0000 7fb9b4ff9640 10 client.28174 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.678+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/3647169037 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e1f4cfa70 con 0x555e1f573d40 2024-05-09T09:21:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.678+0000 7fb9b4ff9640 10 client.28174 handle_command_reply: tid=0 2024-05-09T09:21:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.678+0000 7fb9d663b740 1 client.28174 shutdown 2024-05-09T09:21:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.678+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 msgr2=0x555e1f568b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f568b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4006010 tx=0x7fb9c402adf0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 shutdown_connections 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f573d40 0x555e1f576200 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f56fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f568b80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 --2- 172.21.15.64:0/3647169037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f568640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 >> 172.21.15.64:0/3647169037 conn(0x555e1f559a40 msgr2=0x555e1f558ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9b4ff9640 0 client.28174 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 shutdown_connections 2024-05-09T09:21:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.679+0000 7fb9d663b740 1 -- 172.21.15.64:0/3647169037 wait complete. 2024-05-09T09:21:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.680+0000 7fb9d663b740 20 client.28174 trim_cache size 0 max 16384 2024-05-09T09:21:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.716+0000 7fb9d663b740 1 Processor -- start 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 -- start start 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f567490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f56fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f5679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b00124b0 con 0x555e1f563f30 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f567490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f567490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48696/0 (socket says 172.21.15.64:48696) 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d16c5640 1 -- 172.21.15.64:0/4033633065 learned_addr learned my addr 172.21.15.64:0/4033633065 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f56fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000fd50 con 0x555e1f55e3d0 2024-05-09T09:21:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.718+0000 7fb9d663b740 1 -- 172.21.15.64:0/4033633065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000dba0 con 0x555e1f55edc0 2024-05-09T09:21:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.719+0000 7fb9d16c5640 1 -- 172.21.15.64:0/4033633065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 msgr2=0x555e1f56fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.719+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f56fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.719+0000 7fb9d16c5640 1 -- 172.21.15.64:0/4033633065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 msgr2=0x555e1f5679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.719+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f5679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.719+0000 7fb9d16c5640 1 -- 172.21.15.64:0/4033633065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1f54b330 con 0x555e1f55e3d0 2024-05-09T09:21:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.719+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f567490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009fd0 tx=0x7fb9bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5eb7b68b63cc98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.720+0000 7fb9b6ffd640 1 -- 172.21.15.64:0/4033633065 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc017040 con 0x555e1f55e3d0 2024-05-09T09:21:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.720+0000 7fb9b6ffd640 1 -- 172.21.15.64:0/4033633065 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9bc004520 con 0x555e1f55e3d0 2024-05-09T09:21:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.720+0000 7fb9b6ffd640 1 -- 172.21.15.64:0/4033633065 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc003980 con 0x555e1f55e3d0 2024-05-09T09:21:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 -- 172.21.15.64:0/4033633065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 msgr2=0x555e1f567490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f567490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009fd0 tx=0x7fb9bc000bc0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 -- 172.21.15.64:0/4033633065 shutdown_connections 2024-05-09T09:21:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f563f30 0x555e1f5679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f55edc0 0x555e1f56fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 --2- 172.21.15.64:0/4033633065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f567490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 -- 172.21.15.64:0/4033633065 >> 172.21.15.64:0/4033633065 conn(0x555e1f559a40 msgr2=0x555e1f55ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 -- 172.21.15.64:0/4033633065 shutdown_connections 2024-05-09T09:21:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.721+0000 7fb9d663b740 1 -- 172.21.15.64:0/4033633065 wait complete. 2024-05-09T09:21:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.722+0000 7fb9d663b740 1 -- start start 2024-05-09T09:21:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.722+0000 7fb9d663b740 1 Processor -- start 2024-05-09T09:21:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.722+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f56ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.722+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55edc0 0x555e1f56cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d663b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d663b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000e210 con 0x555e1f55edc0 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d663b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000f680 con 0x555e1f55e3d0 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d663b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000e8e0 con 0x555e1f563f30 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f56ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d663b740 10 client.0 _ll_register_callbacks cb 0x7fb9c4009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f56ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48698/0 (socket says 172.21.15.64:48698) 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 -- 172.21.15.64:0/2924410103 learned_addr learned my addr 172.21.15.64:0/2924410103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55edc0 0x555e1f56cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 -- 172.21.15.64:0/2924410103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 msgr2=0x555e1f5d8370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5d8370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 -- 172.21.15.64:0/2924410103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55edc0 msgr2=0x555e1f56cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55edc0 0x555e1f56cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d16c5640 1 -- 172.21.15.64:0/2924410103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1f5d88b0 con 0x555e1f55e3d0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.723+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55edc0 0x555e1f56cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9d16c5640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f56ca40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc029040 tx=0x7fb9bc009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2770bebc18320219 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/2924410103 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc017070 con 0x555e1f55e3d0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9bc004520 con 0x555e1f55e3d0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/2924410103 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb9bc004320 con 0x555e1f55e3d0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/2924410103 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc005db0 con 0x555e1f55e3d0 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9d663b740 10 client.28019 fetch_fsmap learned FSMap version 47 2024-05-09T09:21:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.724+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555e1f5d9ac0 con 0x555e1f55e3d0 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.727+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/2924410103 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9bc004520 con 0x555e1f55e3d0 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.727+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/2924410103 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fb9bc003ce0 con 0x555e1f55e3d0 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d663b740 10 client.28019 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d663b740 10 client.28019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d663b740 20 client.28019 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d663b740 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f5da0e0 0x555e1f5dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d663b740 4 client.28019 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e1f4c9dc0 con 0x555e1f5da0e0 2024-05-09T09:21:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.728+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f5da0e0 0x555e1f5dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.729+0000 7fb9d0ec4640 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f5da0e0 0x555e1f5dc4d0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b285ed36244a92 server_cookie=179228a41b0c053c in_seq=0 out_seq=0 2024-05-09T09:21:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.729+0000 7fb9b4ff9640 10 client.28019 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.773+0000 7fb9b4ff9640 1 -- 172.21.15.64:0/2924410103 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e1f4c9dc0 con 0x555e1f5da0e0 2024-05-09T09:21:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.773+0000 7fb9b4ff9640 10 client.28019 handle_command_reply: tid=0 2024-05-09T09:21:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.773+0000 7fb9d663b740 1 client.28019 shutdown 2024-05-09T09:21:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.775+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 msgr2=0x555e1f56ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.775+0000 7fb9d663b740 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f56ca40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc029040 tx=0x7fb9bc009940 comp rx=0 tx=0).stop 2024-05-09T09:21:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 shutdown_connections 2024-05-09T09:21:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555e1f5da0e0 0x555e1f5dc4d0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e1f563f30 0x555e1f5d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e1f55edc0 0x555e1f56cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 --2- 172.21.15.64:0/2924410103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e1f55e3d0 0x555e1f56ca40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 >> 172.21.15.64:0/2924410103 conn(0x555e1f559a40 msgr2=0x555e1f567490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9b4ff9640 0 client.28019 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 shutdown_connections 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.776+0000 7fb9d663b740 1 -- 172.21.15.64:0/2924410103 wait complete. 2024-05-09T09:21:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.777+0000 7fb9d663b740 20 client.28019 trim_cache size 0 max 16384 2024-05-09T09:21:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:21:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:21:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:21:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:21:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.778+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9b80a2c40 msgr2=0x7fb9b80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.778+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9b80a2c40 0x7fb9b80a5100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb9c402aee0 tx=0x7fb9c4005eb0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.778+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 msgr2=0x7fb9cc1add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.778+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 0x7fb9cc1add90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800b3e0 tx=0x7fb9c800b8a0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.779+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 shutdown_connections 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.779+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb9b80a2c40 0x7fb9b80a5100 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.779+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9cc109800 0x7fb9cc1add90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.779+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9cc108e40 0x7fb9cc1a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.779+0000 7fb9d26c7640 1 --2- 172.21.15.64:0/2490553238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9cc072b60 0x7fb9cc1a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.779+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 >> 172.21.15.64:0/2490553238 conn(0x7fb9cc06e6d0 msgr2=0x7fb9cc070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.782+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 shutdown_connections 2024-05-09T09:21:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:21:50.782+0000 7fb9d26c7640 1 -- 172.21.15.64:0/2490553238 wait complete. 2024-05-09T09:21:50.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:50 smithi064 ceph-mon[30149]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.9 MiB/s wr, 909 op/s 2024-05-09T09:21:50.973 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:50 smithi139 ceph-mon[33599]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.9 MiB/s wr, 909 op/s 2024-05-09T09:21:50.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:50 smithi179 ceph-mon[34086]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.9 MiB/s wr, 909 op/s 2024-05-09T09:21:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 690.0 s, 726.4 tps, lat 137.413 ms stddev 52.416 2024-05-09T09:21:51.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:51 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:21:51.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:51 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:21:52.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:51 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:21:52.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:52 smithi179 ceph-mon[34086]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 723 op/s 2024-05-09T09:21:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:52 smithi139 ceph-mon[33599]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 723 op/s 2024-05-09T09:21:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:52 smithi064 ceph-mon[30149]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 723 op/s 2024-05-09T09:21:54.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:54 smithi179 ceph-mon[34086]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 796 op/s 2024-05-09T09:21:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:54 smithi139 ceph-mon[33599]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 796 op/s 2024-05-09T09:21:55.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:54 smithi064 ceph-mon[30149]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 796 op/s 2024-05-09T09:21:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 695.0 s, 737.0 tps, lat 135.659 ms stddev 56.939 2024-05-09T09:21:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:56 smithi179 ceph-mon[34086]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.6 MiB/s wr, 728 op/s 2024-05-09T09:21:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:56 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:21:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:56 smithi139 ceph-mon[33599]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.6 MiB/s wr, 728 op/s 2024-05-09T09:21:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:56 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:21:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:56 smithi064 ceph-mon[30149]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.6 MiB/s wr, 728 op/s 2024-05-09T09:21:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:56 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:21:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:21:58 smithi179 ceph-mon[34086]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 624 op/s 2024-05-09T09:21:59.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:21:58 smithi139 ceph-mon[33599]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 624 op/s 2024-05-09T09:21:59.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:21:59.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:21:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:21:58 smithi064 ceph-mon[30149]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 624 op/s 2024-05-09T09:21:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1491786998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc107dd0 msgr2=0x7fdfcc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1491786998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc107dd0 0x7fdfcc1081f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009860 tx=0x7fdfbc02a360 comp rx=0 tx=0).stop 2024-05-09T09:21:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1491786998 shutdown_connections 2024-05-09T09:21:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1491786998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfcc109930 0x7fdfcc1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1491786998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfcc108f70 0x7fdfcc1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1491786998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc107dd0 0x7fdfcc1081f0 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009860 tx=0x7fdfbc02a360 comp rx=0 tx=0).stop 2024-05-09T09:21:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.314+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1491786998 >> 172.21.15.64:0/1491786998 conn(0x7fdfcc076d00 msgr2=0x7fdfcc079160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.315+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1491786998 shutdown_connections 2024-05-09T09:21:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.316+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1491786998 wait complete. 2024-05-09T09:21:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.317+0000 7fdfd17ab640 1 Processor -- start 2024-05-09T09:21:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.317+0000 7fdfd17ab640 1 -- start start 2024-05-09T09:21:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfd17ab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfcc108f70 0x7fdfcc19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfd17ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfcc109930 0x7fdfcc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfd17ab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 0x7fdfcc1a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfd17ab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc113970 con 0x7fdfcc109930 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfd17ab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1137f0 con 0x7fdfcc108f70 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfd17ab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc113af0 con 0x7fdfcc1a3890 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfcb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 0x7fdfcc1a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfcc108f70 0x7fdfcc19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfcb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 0x7fdfcc1a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:51998/0 (socket says 172.21.15.64:51998) 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.318+0000 7fdfcb7fe640 1 -- 172.21.15.64:0/1629812843 learned_addr learned my addr 172.21.15.64:0/1629812843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.319+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfcc109930 0x7fdfcc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.319+0000 7fdfcb7fe640 1 -- 172.21.15.64:0/1629812843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfcc108f70 msgr2=0x7fdfcc19e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.319+0000 7fdfcb7fe640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfcc108f70 0x7fdfcc19e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.319+0000 7fdfcb7fe640 1 -- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfcc109930 msgr2=0x7fdfcc19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.319+0000 7fdfcb7fe640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfcc109930 0x7fdfcc19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.319+0000 7fdfcb7fe640 1 -- 172.21.15.64:0/1629812843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1a62f0 con 0x7fdfcc1a3890 2024-05-09T09:21:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.320+0000 7fdfcb7fe640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 0x7fdfcc1a5c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400eda0 tx=0x7fdfc400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.321+0000 7fdfb3fff640 1 -- 172.21.15.64:0/1629812843 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003b70 con 0x7fdfcc1a3890 2024-05-09T09:21:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.321+0000 7fdfb3fff640 1 -- 172.21.15.64:0/1629812843 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfc4003d10 con 0x7fdfcc1a3890 2024-05-09T09:21:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.321+0000 7fdfb3fff640 1 -- 172.21.15.64:0/1629812843 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40178a0 con 0x7fdfcc1a3890 2024-05-09T09:21:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.321+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1a65e0 con 0x7fdfcc1a3890 2024-05-09T09:21:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.321+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a6ae0 con 0x7fdfcc1a3890 2024-05-09T09:21:59.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.325+0000 7fdfd458b740 1 Processor -- start 2024-05-09T09:21:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.327+0000 7fdfb3fff640 1 -- 172.21.15.64:0/1629812843 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfc4004020 con 0x7fdfcc1a3890 2024-05-09T09:21:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.327+0000 7fdfd458b740 1 -- start start 2024-05-09T09:21:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.327+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f0b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.327+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f10610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f13110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfd458b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556ec5e6fd00 con 0x556ec5f0bb10 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfd458b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556ec5e174c0 con 0x556ec5f0b120 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfd458b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556ec5e6bcc0 con 0x556ec5f10c80 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f0b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f0b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36038/0 (socket says 172.21.15.64:36038) 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 -- 172.21.15.64:0/1602053076 learned_addr learned my addr 172.21.15.64:0/1602053076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f13110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f10610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 -- 172.21.15.64:0/1602053076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 msgr2=0x556ec5f13110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f13110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 -- 172.21.15.64:0/1602053076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 msgr2=0x556ec5f10610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f10610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfca7fc640 1 -- 172.21.15.64:0/1602053076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec5c4d7d0 con 0x556ec5f0b120 2024-05-09T09:21:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.328+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f10610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.329+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f0b540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00176c0 tx=0x7fdfc0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39a29ac2de6cb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.329+0000 7fdfb17fa640 1 -- 172.21.15.64:0/1602053076 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0012870 con 0x556ec5f0b120 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.329+0000 7fdfb17fa640 1 -- 172.21.15.64:0/1602053076 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfc001d070 con 0x556ec5f0b120 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfb17fa640 1 -- 172.21.15.64:0/1602053076 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0018a50 con 0x556ec5f0b120 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfb3fff640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfb40a2d90 0x7fdfb40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 -- 172.21.15.64:0/1602053076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 msgr2=0x556ec5f0b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f0b540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00176c0 tx=0x7fdfc0009490 comp rx=0 tx=0).stop 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 -- 172.21.15.64:0/1602053076 shutdown_connections 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfb40a2d90 0x7fdfb40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f13110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f10610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1602053076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f0b540 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 -- 172.21.15.64:0/1602053076 >> 172.21.15.64:0/1602053076 conn(0x556ec5f06790 msgr2=0x556ec5f06b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.330+0000 7fdfd458b740 1 -- 172.21.15.64:0/1602053076 shutdown_connections 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.331+0000 7fdfb3fff640 1 -- 172.21.15.64:0/1629812843 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfc40cf590 con 0x7fdfcc1a3890 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.331+0000 7fdfd458b740 1 -- 172.21.15.64:0/1602053076 wait complete. 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.332+0000 7fdfd458b740 1 -- start start 2024-05-09T09:21:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.332+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfb40a2d90 0x7fdfb40a5250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc035040 tx=0x7fdfbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.332+0000 7fdfd458b740 1 Processor -- start 2024-05-09T09:21:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.332+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 0x556ec5f264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0bb10 0x556ec5f269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f10c80 0x556ec5f2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfd458b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556ec5f24c30 con 0x556ec5f0b120 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfd458b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556ec5e2ab90 con 0x556ec5f10c80 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfd458b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556ec5f24db0 con 0x556ec5f0bb10 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfd458b740 10 client.0 _ll_register_callbacks cb 0x556ec5e66da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 0x556ec5f264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 0x556ec5f264b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50852/0 (socket says 172.21.15.64:50852) 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfcaffd640 1 -- 172.21.15.64:0/1770281996 learned_addr learned my addr 172.21.15.64:0/1770281996 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.333+0000 7fdfcb7fe640 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f10c80 0x556ec5f2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfcaffd640 1 -- 172.21.15.64:0/1770281996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0bb10 msgr2=0x556ec5f269f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0bb10 0x556ec5f269f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfcaffd640 1 -- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f10c80 msgr2=0x556ec5f2da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f10c80 0x556ec5f2da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfcaffd640 1 -- 172.21.15.64:0/1770281996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec5b833f0 con 0x556ec5f0b120 2024-05-09T09:21:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 0x556ec5f264b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02a8d0 tx=0x7fdfbc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459a9a70bbb98403 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.334+0000 7fdfaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:21:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.337+0000 7fdfaeff5640 1 -- 172.21.15.64:0/1770281996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc044070 con 0x556ec5f0b120 2024-05-09T09:21:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.337+0000 7fdfaeff5640 1 -- 172.21.15.64:0/1770281996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfbc0324e0 con 0x556ec5f0b120 2024-05-09T09:21:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.337+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc001d070 con 0x556ec5f0b120 2024-05-09T09:21:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.338+0000 7fdfaeff5640 1 -- 172.21.15.64:0/1770281996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc037040 con 0x556ec5f0b120 2024-05-09T09:21:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.339+0000 7fdfd458b740 10 client.18555 fetch_fsmap learned FSMap version 48 2024-05-09T09:21:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.339+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556ec5b430f0 con 0x556ec5f0b120 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfaeff5640 1 -- 172.21.15.64:0/1770281996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfc001d070 con 0x556ec5f0b120 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfaeff5640 1 -- 172.21.15.64:0/1770281996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fdfbc030800 con 0x556ec5f0b120 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfd458b740 10 client.18555 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfd458b740 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfd458b740 20 client.18555 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f34830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfd458b740 4 client.18555 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.340+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ec5e7d430 con 0x556ec5f32390 2024-05-09T09:21:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.341+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f34830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.342+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f34830 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d527c280f9958 server_cookie=7d99eab2919939d3 in_seq=0 out_seq=0 2024-05-09T09:21:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.342+0000 7fdfaeff5640 10 client.18555 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.343+0000 7fdfaeff5640 1 -- 172.21.15.64:0/1770281996 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ec5e7d430 con 0x556ec5f32390 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.343+0000 7fdfaeff5640 10 client.18555 handle_command_reply: tid=0 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.343+0000 7fdfd458b740 1 client.18555 shutdown 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 msgr2=0x556ec5f264b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 0x556ec5f264b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02a8d0 tx=0x7fdfbc005f90 comp rx=0 tx=0).stop 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 shutdown_connections 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f34830 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfaeff5640 0 client.18555 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f10c80 0x556ec5f2da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0bb10 0x556ec5f269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 --2- 172.21.15.64:0/1770281996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0b120 0x556ec5f264b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.344+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 >> 172.21.15.64:0/1770281996 conn(0x556ec5f06790 msgr2=0x556ec5f05a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.345+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 shutdown_connections 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.345+0000 7fdfd458b740 1 -- 172.21.15.64:0/1770281996 wait complete. 2024-05-09T09:21:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.345+0000 7fdfd458b740 20 client.18555 trim_cache size 0 max 16384 2024-05-09T09:21:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.388+0000 7fdfd458b740 1 Processor -- start 2024-05-09T09:21:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.388+0000 7fdfd458b740 1 -- start start 2024-05-09T09:21:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.388+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0b120 0x556ec5f260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.388+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 0x556ec5f26600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f10c80 0x556ec5f09470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfd458b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1aafe0 con 0x556ec5f10c80 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfd458b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a8880 con 0x556ec5f0bb10 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfd458b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc073330 con 0x556ec5f0b120 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 0x556ec5f26600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0b120 0x556ec5f260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 0x556ec5f26600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36052/0 (socket says 172.21.15.64:36052) 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0b120 0x556ec5f260c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52010/0 (socket says 172.21.15.64:52010) 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 -- 172.21.15.64:0/971598653 learned_addr learned my addr 172.21.15.64:0/971598653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfcb7fe640 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f10c80 0x556ec5f09470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 -- 172.21.15.64:0/971598653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0b120 msgr2=0x556ec5f260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0b120 0x556ec5f260c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 -- 172.21.15.64:0/971598653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f10c80 msgr2=0x556ec5f09470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f10c80 0x556ec5f09470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.389+0000 7fdfca7fc640 1 -- 172.21.15.64:0/971598653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec5ef7930 con 0x556ec5f0bb10 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.390+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 0x556ec5f26600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0006fd0 tx=0x7fdfc0012fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e18feac9d8c95f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.390+0000 7fdfb0ff9640 1 -- 172.21.15.64:0/971598653 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc002d070 con 0x556ec5f0bb10 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.390+0000 7fdfb0ff9640 1 -- 172.21.15.64:0/971598653 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfc001d070 con 0x556ec5f0bb10 2024-05-09T09:21:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.390+0000 7fdfb0ff9640 1 -- 172.21.15.64:0/971598653 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0003d50 con 0x556ec5f0bb10 2024-05-09T09:21:59.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.391+0000 7fdfd458b740 1 -- 172.21.15.64:0/971598653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 msgr2=0x556ec5f26600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.391+0000 7fdfd458b740 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 0x556ec5f26600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0006fd0 tx=0x7fdfc0012fd0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.393+0000 7fdfd458b740 1 -- 172.21.15.64:0/971598653 shutdown_connections 2024-05-09T09:21:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.393+0000 7fdfd458b740 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f10c80 0x556ec5f09470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.393+0000 7fdfd458b740 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0bb10 0x556ec5f26600 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.393+0000 7fdfd458b740 1 --2- 172.21.15.64:0/971598653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f0b120 0x556ec5f260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.393+0000 7fdfd458b740 1 -- 172.21.15.64:0/971598653 >> 172.21.15.64:0/971598653 conn(0x556ec5f06790 msgr2=0x556ec5f018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.394+0000 7fdfd458b740 1 -- 172.21.15.64:0/971598653 shutdown_connections 2024-05-09T09:21:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.396+0000 7fdfd458b740 1 -- 172.21.15.64:0/971598653 wait complete. 2024-05-09T09:21:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.396+0000 7fdfd458b740 1 -- start start 2024-05-09T09:21:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.397+0000 7fdfd458b740 1 Processor -- start 2024-05-09T09:21:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.397+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f2ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.397+0000 7fdfd458b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f2b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f2ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfcaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f2ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36058/0 (socket says 172.21.15.64:36058) 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfcaffd640 1 -- 172.21.15.64:0/4246668943 learned_addr learned my addr 172.21.15.64:0/4246668943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a6e00 con 0x556ec5f0bb10 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfcb7fe640 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a81b0 con 0x556ec5f0b120 2024-05-09T09:21:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.398+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a7410 con 0x556ec5f10c80 2024-05-09T09:21:59.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfd458b740 10 client.0 _ll_register_callbacks cb 0x7fdfbc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:21:59.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfcaffd640 1 -- 172.21.15.64:0/4246668943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 msgr2=0x556ec5f2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:21:59.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfcaffd640 1 -- 172.21.15.64:0/4246668943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 msgr2=0x556ec5f2b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f2b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfcaffd640 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec5f2c680 con 0x556ec5f0b120 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfcaffd640 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f2ab40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc030e10 tx=0x7fdfbc003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8828a7782a25d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc001d070 con 0x556ec5f0b120 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.399+0000 7fdfaeff5640 1 -- 172.21.15.64:0/4246668943 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc044040 con 0x556ec5f0b120 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.400+0000 7fdfaeff5640 1 -- 172.21.15.64:0/4246668943 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdfbc04b970 con 0x556ec5f0b120 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.400+0000 7fdfaeff5640 1 -- 172.21.15.64:0/4246668943 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc0044c0 con 0x556ec5f0b120 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.400+0000 7fdfd458b740 10 client.28037 fetch_fsmap learned FSMap version 48 2024-05-09T09:21:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.400+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=48}) -- 0x556ec5f98e20 con 0x556ec5f0b120 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfaeff5640 1 -- 172.21.15.64:0/4246668943 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fdfbc067020 con 0x556ec5f0b120 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfd458b740 10 client.28037 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfd458b740 10 client.28037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfd458b740 20 client.28037 populate_metadata read hostname 'smithi064' 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f9b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfd458b740 4 client.28037 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:21:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.401+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fdfc001d070 con 0x556ec5f32390 2024-05-09T09:21:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.402+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f9b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.404+0000 7fdfca7fc640 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f9b590 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f27d80ce51482f6 server_cookie=b498fc53fd5ab2b8 in_seq=0 out_seq=0 2024-05-09T09:21:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.404+0000 7fdfaeff5640 10 client.28037 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.404+0000 7fdfaeff5640 1 -- 172.21.15.64:0/4246668943 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fdfc001d070 con 0x556ec5f32390 2024-05-09T09:21:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.404+0000 7fdfaeff5640 10 client.28037 handle_command_reply: tid=0 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.404+0000 7fdfd458b740 1 client.28037 shutdown 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.404+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 msgr2=0x556ec5f2ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f2ab40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc030e10 tx=0x7fdfbc003f80 comp rx=0 tx=0).stop 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 shutdown_connections 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x556ec5f32390 0x556ec5f9b590 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ec5f10c80 0x556ec5f2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfaeff5640 0 client.28037 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ec5f0bb10 0x556ec5f2b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 --2- 172.21.15.64:0/4246668943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556ec5f0b120 0x556ec5f2ab40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.405+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 >> 172.21.15.64:0/4246668943 conn(0x556ec5f06790 msgr2=0x556ec5f260a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.406+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 shutdown_connections 2024-05-09T09:21:59.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.406+0000 7fdfd458b740 1 -- 172.21.15.64:0/4246668943 wait complete. 2024-05-09T09:21:59.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.407+0000 7fdfd458b740 20 client.28037 trim_cache size 0 max 16384 2024-05-09T09:21:59.407 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:21:59.408 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:21:59.408 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:21:59.408 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:21:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfb40a2d90 msgr2=0x7fdfb40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfb40a2d90 0x7fdfb40a5250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc035040 tx=0x7fdfbc000bc0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 msgr2=0x7fdfcc1a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 0x7fdfcc1a5c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400eda0 tx=0x7fdfc400c6e0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 shutdown_connections 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdfb40a2d90 0x7fdfb40a5250 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1a3890 0x7fdfcc1a5c80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfcc109930 0x7fdfcc19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.408+0000 7fdfd17ab640 1 --2- 172.21.15.64:0/1629812843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfcc108f70 0x7fdfcc19e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.409+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 >> 172.21.15.64:0/1629812843 conn(0x7fdfcc076d00 msgr2=0x7fdfcc10c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.410+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 shutdown_connections 2024-05-09T09:21:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.410+0000 7fdfd17ab640 1 -- 172.21.15.64:0/1629812843 wait complete. 2024-05-09T09:21:59.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:21:59.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:21:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 -- 172.21.15.64:0/286016607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a53c0 msgr2=0x7f68600ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 --2- 172.21.15.64:0/286016607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a53c0 0x7f68600ac510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6868066f90 tx=0x7f686808dc60 comp rx=0 tx=0).stop 2024-05-09T09:21:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 -- 172.21.15.64:0/286016607 shutdown_connections 2024-05-09T09:21:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 --2- 172.21.15.64:0/286016607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68600aca50 0x7f68600aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 --2- 172.21.15.64:0/286016607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a53c0 0x7f68600ac510 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 --2- 172.21.15.64:0/286016607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a49d0 0x7f68600a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 -- 172.21.15.64:0/286016607 >> 172.21.15.64:0/286016607 conn(0x7f68600a0520 msgr2=0x7f68600a2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:21:59.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 -- 172.21.15.64:0/286016607 shutdown_connections 2024-05-09T09:21:59.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.683+0000 7f686cc4c640 1 -- 172.21.15.64:0/286016607 wait complete. 2024-05-09T09:21:59.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.684+0000 7f686cc4c640 1 Processor -- start 2024-05-09T09:21:59.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.684+0000 7f686cc4c640 1 -- start start 2024-05-09T09:21:59.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686cc4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 0x7f686013f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686cc4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a53c0 0x7f686013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686cc4c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68600aca50 0x7f6860146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686cc4c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68600b4a30 con 0x7f68600a53c0 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686cc4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68600b48b0 con 0x7f68600a49d0 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686cc4c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f68600b4bb0 con 0x7f68600aca50 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f6865d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a53c0 0x7f686013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 0x7f686013f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 0x7f686013f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36072/0 (socket says 172.21.15.64:36072) 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f6865d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a53c0 0x7f686013f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50886/0 (socket says 172.21.15.64:50886) 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 -- 172.21.15.64:0/2091980984 learned_addr learned my addr 172.21.15.64:0/2091980984 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:21:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f6866d9d640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68600aca50 0x7f6860146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 -- 172.21.15.64:0/2091980984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68600aca50 msgr2=0x7f6860146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68600aca50 0x7f6860146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 -- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a53c0 msgr2=0x7f686013f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:21:59.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a53c0 0x7f686013f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:21:59.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.685+0000 7f686659c640 1 -- 172.21.15.64:0/2091980984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6860146d80 con 0x7f68600a49d0 2024-05-09T09:21:59.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.686+0000 7f686659c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 0x7f686013f160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9a0 tx=0x7f685800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.686+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685800cd80 con 0x7f68600a49d0 2024-05-09T09:21:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.686+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68580108e0 con 0x7f68600a49d0 2024-05-09T09:21:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.686+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858018950 con 0x7f68600a49d0 2024-05-09T09:21:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.686+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6860147070 con 0x7f68600a49d0 2024-05-09T09:21:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.686+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68600af7b0 con 0x7f68600a49d0 2024-05-09T09:21:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.690+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68600afa70 con 0x7f68600a49d0 2024-05-09T09:21:59.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.692+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6858010a90 con 0x7f68600a49d0 2024-05-09T09:21:59.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.693+0000 7f68577fe640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68500a2ce0 0x7f68500a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:21:59.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.693+0000 7f6865d9b640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68500a2ce0 0x7f68500a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:21:59.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.693+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68580cf6a0 con 0x7f68600a49d0 2024-05-09T09:21:59.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.694+0000 7f6865d9b640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68500a2ce0 0x7f68500a51a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6868050860 tx=0x7f6868090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:21:59.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:21:59.698+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6858053740 con 0x7f68600a49d0 2024-05-09T09:22:00.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.158+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68600afcc0 con 0x7f68600a49d0 2024-05-09T09:22:00.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.161+0000 7f68577fe640 1 -- 172.21.15.64:0/2091980984 <== mon.2 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11883 (secure 0 0 0) 0x7f685809c210 con 0x7f68600a49d0 2024-05-09T09:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:22:00.161 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:21:55.091434+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:22:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 48 2024-05-09T09:22:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68500a2ce0 msgr2=0x7f68500a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68500a2ce0 0x7f68500a51a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6868050860 tx=0x7f6868090040 comp rx=0 tx=0).stop 2024-05-09T09:22:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 msgr2=0x7f686013f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 0x7f686013f160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9a0 tx=0x7f685800ee60 comp rx=0 tx=0).stop 2024-05-09T09:22:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 shutdown_connections 2024-05-09T09:22:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68500a2ce0 0x7f68500a51a0 secure :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6868050860 tx=0x7f6868090040 comp rx=0 tx=0).stop 2024-05-09T09:22:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68600aca50 0x7f6860146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68600a53c0 0x7f686013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 --2- 172.21.15.64:0/2091980984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68600a49d0 0x7f686013f160 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.165+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 >> 172.21.15.64:0/2091980984 conn(0x7f68600a0520 msgr2=0x7f68600a1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.166+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 shutdown_connections 2024-05-09T09:22:00.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.166+0000 7f686cc4c640 1 -- 172.21.15.64:0/2091980984 wait complete. 2024-05-09T09:22:00.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:22:00.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.388+0000 7fc90a7fc640 1 -- 172.21.15.64:0/1178207375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc031e40 con 0x7fc90c0761f0 2024-05-09T09:22:00.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.389+0000 7fc912163640 1 -- 172.21.15.64:0/1178207375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c0761f0 msgr2=0x7fc90c076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.389+0000 7fc912163640 1 --2- 172.21.15.64:0/1178207375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c0761f0 0x7fc90c076610 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009ca0 tx=0x7fc8fc02a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.390+0000 7fc912163640 1 -- 172.21.15.64:0/1178207375 shutdown_connections 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.390+0000 7fc912163640 1 --2- 172.21.15.64:0/1178207375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 0x7fc90c078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.390+0000 7fc912163640 1 --2- 172.21.15.64:0/1178207375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c077430 0x7fc90c077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.390+0000 7fc912163640 1 --2- 172.21.15.64:0/1178207375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c0761f0 0x7fc90c076610 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.390+0000 7fc912163640 1 -- 172.21.15.64:0/1178207375 >> 172.21.15.64:0/1178207375 conn(0x7fc90c0ff020 msgr2=0x7fc90c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.390+0000 7fc912163640 1 -- 172.21.15.64:0/1178207375 shutdown_connections 2024-05-09T09:22:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.391+0000 7fc912163640 1 -- 172.21.15.64:0/1178207375 wait complete. 2024-05-09T09:22:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.392+0000 7fc912163640 1 Processor -- start 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 -- start start 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c0761f0 0x7fc90c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 0x7fc90c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 0x7fc90c1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c113a10 con 0x7fc90c077430 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c113890 con 0x7fc90c077dd0 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc912163640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c113b90 con 0x7fc90c0761f0 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc90affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 0x7fc90c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc90affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 0x7fc90c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50894/0 (socket says 172.21.15.64:50894) 2024-05-09T09:22:00.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.393+0000 7fc90affd640 1 -- 172.21.15.64:0/808678237 learned_addr learned my addr 172.21.15.64:0/808678237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c0761f0 0x7fc90c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90bfff640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 0x7fc90c1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90affd640 1 -- 172.21.15.64:0/808678237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c0761f0 msgr2=0x7fc90c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90affd640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c0761f0 0x7fc90c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90affd640 1 -- 172.21.15.64:0/808678237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 msgr2=0x7fc90c1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90affd640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 0x7fc90c1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90affd640 1 -- 172.21.15.64:0/808678237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c06b380 con 0x7fc90c077430 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c0761f0 0x7fc90c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.394+0000 7fc90bfff640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 0x7fc90c1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.395+0000 7fc90affd640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 0x7fc90c19e5b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e990 tx=0x7fc8f400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.395+0000 7fc908ff9640 1 -- 172.21.15.64:0/808678237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400cbe0 con 0x7fc90c077430 2024-05-09T09:22:00.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.395+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c06b670 con 0x7fc90c077430 2024-05-09T09:22:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.395+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c06bbc0 con 0x7fc90c077430 2024-05-09T09:22:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.395+0000 7fc908ff9640 1 -- 172.21.15.64:0/808678237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f400cd80 con 0x7fc90c077430 2024-05-09T09:22:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.396+0000 7fc908ff9640 1 -- 172.21.15.64:0/808678237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4011470 con 0x7fc90c077430 2024-05-09T09:22:00.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.399+0000 7fc914e79740 1 Processor -- start 2024-05-09T09:22:00.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 -- start start 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdf2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdf7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdf9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c50cd56c40 con 0x55c50cdf7aa0 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c50ccfe580 con 0x55c50cdf1f40 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc914e79740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c50cd52c00 con 0x55c50cdf2930 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdf2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdf2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36094/0 (socket says 172.21.15.64:36094) 2024-05-09T09:22:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.400+0000 7fc90bfff640 1 -- 172.21.15.64:0/1263854928 learned_addr learned my addr 172.21.15.64:0/1263854928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.401+0000 7fc908ff9640 1 -- 172.21.15.64:0/808678237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8f40046c0 con 0x7fc90c077430 2024-05-09T09:22:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.401+0000 7fc90affd640 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdf9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc908ff9640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f80a2d60 0x7fc8f80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc908ff9640 1 -- 172.21.15.64:0/808678237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f40cf390 con 0x7fc90c077430 2024-05-09T09:22:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc90bfff640 1 -- 172.21.15.64:0/1263854928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 msgr2=0x55c50cdf7430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc90bfff640 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdf7430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc90bfff640 1 -- 172.21.15.64:0/1263854928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 msgr2=0x55c50cdf9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc90bfff640 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdf9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.402+0000 7fc90bfff640 1 -- 172.21.15.64:0/1263854928 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50ca6a3f0 con 0x55c50cdf1f40 2024-05-09T09:22:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.404+0000 7fc90bfff640 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdf2360 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc90400cbf0 tx=0x7fc90400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9ef552937f30a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.404+0000 7fc90bfff640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f80a2d60 0x7fc8f80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.404+0000 7fc901ffb640 1 -- 172.21.15.64:0/1263854928 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904010710 con 0x55c50cdf1f40 2024-05-09T09:22:00.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.405+0000 7fc901ffb640 1 -- 172.21.15.64:0/1263854928 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc904010d60 con 0x55c50cdf1f40 2024-05-09T09:22:00.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.405+0000 7fc901ffb640 1 -- 172.21.15.64:0/1263854928 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400f3c0 con 0x55c50cdf1f40 2024-05-09T09:22:00.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.406+0000 7fc914e79740 1 -- 172.21.15.64:0/1263854928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 msgr2=0x55c50cdf2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.406+0000 7fc914e79740 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdf2360 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc90400cbf0 tx=0x7fc90400e720 comp rx=0 tx=0).stop 2024-05-09T09:22:00.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.406+0000 7fc90bfff640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f80a2d60 0x7fc8f80a5220 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc90400f010 tx=0x7fc90400f640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.407+0000 7fc914e79740 1 -- 172.21.15.64:0/1263854928 shutdown_connections 2024-05-09T09:22:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.407+0000 7fc914e79740 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdf9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.408+0000 7fc914e79740 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdf7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.408+0000 7fc914e79740 1 --2- 172.21.15.64:0/1263854928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdf2360 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.408+0000 7fc914e79740 1 -- 172.21.15.64:0/1263854928 >> 172.21.15.64:0/1263854928 conn(0x55c50cded5b0 msgr2=0x55c50cded9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.411+0000 7fc914e79740 1 -- 172.21.15.64:0/1263854928 shutdown_connections 2024-05-09T09:22:00.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.411+0000 7fc914e79740 1 -- 172.21.15.64:0/1263854928 wait complete. 2024-05-09T09:22:00.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.411+0000 7fc914e79740 1 -- start start 2024-05-09T09:22:00.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 Processor -- start 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50ce038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c50cdfaaa0 con 0x55c50cdf7aa0 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c50cd11c40 con 0x55c50cdf1f40 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c50cdfac20 con 0x55c50cdf2930 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc914e79740 10 client.0 _ll_register_callbacks cb 0x55c50ca69be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50ce038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50ce038f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50918/0 (socket says 172.21.15.64:50918) 2024-05-09T09:22:00.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.412+0000 7fc90bfff640 1 -- 172.21.15.64:0/3222827528 learned_addr learned my addr 172.21.15.64:0/3222827528 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.413+0000 7fc90affd640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.413+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.413+0000 7fc90affd640 1 -- 172.21.15.64:0/3222827528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 msgr2=0x55c50cdfc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.413+0000 7fc90affd640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.413+0000 7fc90affd640 1 -- 172.21.15.64:0/3222827528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 msgr2=0x55c50ce038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.413+0000 7fc90affd640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50ce038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc90affd640 1 -- 172.21.15.64:0/3222827528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50cb347d0 con 0x55c50cdf2930 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc90bfff640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50ce038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc90affd640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfc8b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40981d0 tx=0x7fc8f409b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f63400abc773a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3222827528 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f401f050 con 0x55c50cdf2930 2024-05-09T09:22:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3222827528 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f40d2d80 con 0x55c50cdf2930 2024-05-09T09:22:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.414+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc904010d60 con 0x55c50cdf2930 2024-05-09T09:22:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.415+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3222827528 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc8f409b800 con 0x55c50cdf2930 2024-05-09T09:22:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.415+0000 7fc914e79740 10 client.28207 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.415+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c50ce06cd0 con 0x55c50cdf2930 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.415+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3222827528 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc904010d60 con 0x55c50cdf2930 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3222827528 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fc8f400f040 con 0x55c50cdf2930 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc914e79740 10 client.28207 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc914e79740 10 client.28207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc914e79740 20 client.28207 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc914e79740 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce08920 0x55c50ce0ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc914e79740 4 client.28207 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.416+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50cd64370 con 0x55c50ce08920 2024-05-09T09:22:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.417+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce08920 0x55c50ce0ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.417+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce08920 0x55c50ce0ad60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=530ffb4eb0392bfd server_cookie=c9d441f1f51dc8d in_seq=0 out_seq=0 2024-05-09T09:22:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.417+0000 7fc8eaffd640 10 client.28207 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3222827528 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c50cd64370 con 0x55c50ce08920 2024-05-09T09:22:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc8eaffd640 10 client.28207 handle_command_reply: tid=0 2024-05-09T09:22:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 client.28207 shutdown 2024-05-09T09:22:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 msgr2=0x55c50cdfc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfc8b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40981d0 tx=0x7fc8f409b400 comp rx=0 tx=0).stop 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 shutdown_connections 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce08920 0x55c50ce0ad60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50ce038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfc8b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 --2- 172.21.15.64:0/3222827528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 >> 172.21.15.64:0/3222827528 conn(0x55c50cded5b0 msgr2=0x55c50cdec7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.418+0000 7fc8eaffd640 0 client.28207 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.419+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 shutdown_connections 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.419+0000 7fc914e79740 1 -- 172.21.15.64:0/3222827528 wait complete. 2024-05-09T09:22:00.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.419+0000 7fc914e79740 20 client.28207 trim_cache size 0 max 16384 2024-05-09T09:22:00.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.446+0000 7fc914e79740 1 Processor -- start 2024-05-09T09:22:00.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.446+0000 7fc914e79740 1 -- start start 2024-05-09T09:22:00.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc914e79740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc914e79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc90affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc90affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52082/0 (socket says 172.21.15.64:52082) 2024-05-09T09:22:00.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36120/0 (socket says 172.21.15.64:36120) 2024-05-09T09:22:00.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc90affd640 1 -- 172.21.15.64:0/1301801598 learned_addr learned my addr 172.21.15.64:0/1301801598 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.447+0000 7fc914e79740 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec012580 con 0x55c50cdf7aa0 2024-05-09T09:22:00.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec00fe20 con 0x55c50cdf1f40 2024-05-09T09:22:00.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec00dc70 con 0x55c50cdf2930 2024-05-09T09:22:00.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90bfff640 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90affd640 1 -- 172.21.15.64:0/1301801598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 msgr2=0x55c50cdfb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90affd640 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90affd640 1 -- 172.21.15.64:0/1301801598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 msgr2=0x55c50cdebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90affd640 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90affd640 1 -- 172.21.15.64:0/1301801598 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50cdded20 con 0x55c50cdf2930 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.448+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.449+0000 7fc90affd640 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfbb80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40d2ad0 tx=0x7fc8f4099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c1a51d49789192 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.449+0000 7fc9017fa640 1 -- 172.21.15.64:0/1301801598 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f401f050 con 0x55c50cdf2930 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.449+0000 7fc9017fa640 1 -- 172.21.15.64:0/1301801598 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f40a4470 con 0x55c50cdf2930 2024-05-09T09:22:00.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.449+0000 7fc9017fa640 1 -- 172.21.15.64:0/1301801598 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40d2de0 con 0x55c50cdf2930 2024-05-09T09:22:00.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.450+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 msgr2=0x55c50cdfbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.450+0000 7fc914e79740 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfbb80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40d2ad0 tx=0x7fc8f4099d10 comp rx=0 tx=0).stop 2024-05-09T09:22:00.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 shutdown_connections 2024-05-09T09:22:00.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf7aa0 0x55c50cdebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50cdfbb80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 --2- 172.21.15.64:0/1301801598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf1f40 0x55c50cdfb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 >> 172.21.15.64:0/1301801598 conn(0x55c50cded5b0 msgr2=0x55c50cdf3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 shutdown_connections 2024-05-09T09:22:00.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.451+0000 7fc914e79740 1 -- 172.21.15.64:0/1301801598 wait complete. 2024-05-09T09:22:00.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.452+0000 7fc914e79740 1 -- start start 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.452+0000 7fc914e79740 1 Processor -- start 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.452+0000 7fc914e79740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 0x55c50cdffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.452+0000 7fc914e79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50ce00010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc914e79740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf7aa0 0x55c50ce6ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50ce00010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50ce00010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52094/0 (socket says 172.21.15.64:52094) 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 0x55c50cdffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 0x55c50cdffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50934/0 (socket says 172.21.15.64:50934) 2024-05-09T09:22:00.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 -- 172.21.15.64:0/3391178573 learned_addr learned my addr 172.21.15.64:0/3391178573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec00e2e0 con 0x55c50cdf1f40 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec00f750 con 0x55c50cdf7aa0 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec00e9b0 con 0x55c50cdf2930 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc914e79740 10 client.0 _ll_register_callbacks cb 0x7fc8f409b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 -- 172.21.15.64:0/3391178573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf7aa0 msgr2=0x55c50ce6ce90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf7aa0 0x55c50ce6ce90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 -- 172.21.15.64:0/3391178573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 msgr2=0x55c50cdffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 0x55c50cdffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.453+0000 7fc90affd640 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50ce03090 con 0x55c50cdf2930 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.454+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 0x55c50cdffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.454+0000 7fc90affd640 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50ce00010 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e960 tx=0x7fc8f409cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=776ddaa3bca4e228 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.454+0000 7fc8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.454+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3391178573 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f401f070 con 0x55c50cdf2930 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.454+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3391178573 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f409a8c0 con 0x55c50cdf2930 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.455+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f40a4470 con 0x55c50cdf2930 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.455+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3391178573 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc8f409fbb0 con 0x55c50cdf2930 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.455+0000 7fc914e79740 10 client.28213 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.455+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c50ce6e3b0 con 0x55c50cdf2930 2024-05-09T09:22:00.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.455+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3391178573 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f40b4a00 con 0x55c50cdf2930 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3391178573 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fc8f40b4250 con 0x55c50cdf2930 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc914e79740 10 client.28213 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc914e79740 10 client.28213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc914e79740 20 client.28213 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc914e79740 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce6eda0 0x55c50ce711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc914e79740 4 client.28213 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc8f40a4470 con 0x55c50ce6eda0 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.456+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce6eda0 0x55c50ce711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.457+0000 7fc90b7fe640 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce6eda0 0x55c50ce711c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25d8e050eb7e086 server_cookie=e72075a3eb98ac9f in_seq=0 out_seq=0 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.457+0000 7fc8eaffd640 10 client.28213 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.457+0000 7fc8eaffd640 1 -- 172.21.15.64:0/3391178573 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc8f40a4470 con 0x55c50ce6eda0 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.457+0000 7fc8eaffd640 10 client.28213 handle_command_reply: tid=0 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.457+0000 7fc914e79740 1 client.28213 shutdown 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 msgr2=0x55c50ce00010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50ce00010 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e960 tx=0x7fc8f409cd60 comp rx=0 tx=0).stop 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 shutdown_connections 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c50ce6eda0 0x55c50ce711c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c50cdf7aa0 0x55c50ce6ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c50cdf2930 0x55c50ce00010 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 --2- 172.21.15.64:0/3391178573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c50cdf1f40 0x55c50cdffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 >> 172.21.15.64:0/3391178573 conn(0x55c50cded5b0 msgr2=0x55c50cdfb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.458+0000 7fc8eaffd640 0 client.28213 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:00.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.459+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 shutdown_connections 2024-05-09T09:22:00.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.459+0000 7fc914e79740 1 -- 172.21.15.64:0/3391178573 wait complete. 2024-05-09T09:22:00.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.459+0000 7fc914e79740 20 client.28213 trim_cache size 0 max 16384 2024-05-09T09:22:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:22:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f80a2d60 msgr2=0x7fc8f80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f80a2d60 0x7fc8f80a5220 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc90400f010 tx=0x7fc90400f640 comp rx=0 tx=0).stop 2024-05-09T09:22:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 msgr2=0x7fc90c19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 0x7fc90c19e5b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e990 tx=0x7fc8f400ee50 comp rx=0 tx=0).stop 2024-05-09T09:22:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 shutdown_connections 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f80a2d60 0x7fc8f80a5220 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90c077dd0 0x7fc90c1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90c077430 0x7fc90c19e5b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.462+0000 7fc912163640 1 --2- 172.21.15.64:0/808678237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90c0761f0 0x7fc90c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.463+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 >> 172.21.15.64:0/808678237 conn(0x7fc90c0ff020 msgr2=0x7fc90c100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.463+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 shutdown_connections 2024-05-09T09:22:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.463+0000 7fc912163640 1 -- 172.21.15.64:0/808678237 wait complete. 2024-05-09T09:22:00.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 -- 172.21.15.64:0/2228429451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 msgr2=0x7f41d810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 --2- 172.21.15.64:0/2228429451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 0x7f41d810d630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f41cc01a910 tx=0x7f41cc03b430 comp rx=0 tx=0).stop 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 -- 172.21.15.64:0/2228429451 shutdown_connections 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 --2- 172.21.15.64:0/2228429451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d810db70 0x7f41d810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 --2- 172.21.15.64:0/2228429451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 0x7f41d810d630 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 --2- 172.21.15.64:0/2228429451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d8108830 0x7f41d810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.704+0000 7f41de142640 1 -- 172.21.15.64:0/2228429451 >> 172.21.15.64:0/2228429451 conn(0x7f41d806e6c0 msgr2=0x7f41d8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.705+0000 7f41de142640 1 -- 172.21.15.64:0/2228429451 shutdown_connections 2024-05-09T09:22:00.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.705+0000 7f41de142640 1 -- 172.21.15.64:0/2228429451 wait complete. 2024-05-09T09:22:00.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 Processor -- start 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 -- start start 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 0x7f41d819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 0x7f41d819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d810db70 0x7f41d81a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81119f0 con 0x7f41d810db70 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8111870 con 0x7f41d810b1a0 2024-05-09T09:22:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.706+0000 7f41de142640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8111b70 con 0x7f41d8108830 2024-05-09T09:22:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 0x7f41d819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 0x7f41d819e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52108/0 (socket says 172.21.15.64:52108) 2024-05-09T09:22:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 -- 172.21.15.64:0/209315 learned_addr learned my addr 172.21.15.64:0/209315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d7fff640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d810db70 0x7f41d81a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 -- 172.21.15.64:0/209315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 msgr2=0x7f41d819e670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:22:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 0x7f41d819e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 -- 172.21.15.64:0/209315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d810db70 msgr2=0x7f41d81a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d810db70 0x7f41d81a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.709+0000 7f41d77fe640 1 -- 172.21.15.64:0/209315 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d81a5d00 con 0x7f41d8108830 2024-05-09T09:22:00.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.712+0000 7f41d77fe640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 0x7f41d819e130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f41c800d950 tx=0x7f41c800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.713+0000 7f41d4ff9640 1 -- 172.21.15.64:0/209315 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8017070 con 0x7f41d8108830 2024-05-09T09:22:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.713+0000 7f41d4ff9640 1 -- 172.21.15.64:0/209315 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41c8009c30 con 0x7f41d8108830 2024-05-09T09:22:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.713+0000 7f41d4ff9640 1 -- 172.21.15.64:0/209315 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800f620 con 0x7f41d8108830 2024-05-09T09:22:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.713+0000 7f41de142640 1 -- 172.21.15.64:0/209315 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d81a5ff0 con 0x7f41d8108830 2024-05-09T09:22:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.714+0000 7f41de142640 1 -- 172.21.15.64:0/209315 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81a6400 con 0x7f41d8108830 2024-05-09T09:22:00.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.718+0000 7f41e0e54740 1 Processor -- start 2024-05-09T09:22:00.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.718+0000 7f41d4ff9640 1 -- 172.21.15.64:0/209315 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41c80109a0 con 0x7f41d8108830 2024-05-09T09:22:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.719+0000 7f41d4ff9640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f41c00a2d60 0x7f41c00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41d4ff9640 1 -- 172.21.15.64:0/209315 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41c80cf0a0 con 0x7f41d8108830 2024-05-09T09:22:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 -- start start 2024-05-09T09:22:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a90f40 0x55d687a91360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a91930 0x55d687a96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a98f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d6879f5c40 con 0x55d687a91930 2024-05-09T09:22:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d68799d580 con 0x55d687a96aa0 2024-05-09T09:22:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.720+0000 7f41e0e54740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d6879f1c00 con 0x55d687a90f40 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.721+0000 7f41d7fff640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f41c00a2d60 0x7f41c00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.721+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a98f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.721+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a98f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36154/0 (socket says 172.21.15.64:36154) 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.721+0000 7f41d7fff640 1 -- 172.21.15.64:0/777053032 learned_addr learned my addr 172.21.15.64:0/777053032 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.721+0000 7f41d77fe640 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a90f40 0x55d687a91360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.722+0000 7f41d7fff640 1 -- 172.21.15.64:0/777053032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a90f40 msgr2=0x55d687a91360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.722+0000 7f41d7fff640 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a90f40 0x55d687a91360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.722+0000 7f41d7fff640 1 -- 172.21.15.64:0/777053032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a91930 msgr2=0x55d687a96430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.722+0000 7f41d7fff640 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a91930 0x55d687a96430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.722+0000 7f41d7fff640 1 -- 172.21.15.64:0/777053032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6877093f0 con 0x55d687a96aa0 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.722+0000 7f41d7fff640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f41c00a2d60 0x7f41c00a5220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009fd0 tx=0x7f41d0009d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.723+0000 7f41d7fff640 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a98f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41d0010800 tx=0x7f41d001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa77adcb0489d9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.723+0000 7f41d77fe640 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a90f40 0x55d687a91360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.723+0000 7f41bdffb640 1 -- 172.21.15.64:0/777053032 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0012030 con 0x55d687a96aa0 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.723+0000 7f41bdffb640 1 -- 172.21.15.64:0/777053032 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41d00121d0 con 0x55d687a96aa0 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.724+0000 7f41e0e54740 1 -- 172.21.15.64:0/777053032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 msgr2=0x55d687a98f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.724+0000 7f41e0e54740 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a98f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41d0010800 tx=0x7f41d001c000 comp rx=0 tx=0).stop 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.725+0000 7f41e0e54740 1 -- 172.21.15.64:0/777053032 shutdown_connections 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.725+0000 7f41e0e54740 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a98f30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.725+0000 7f41e0e54740 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a91930 0x55d687a96430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.725+0000 7f41e0e54740 1 --2- 172.21.15.64:0/777053032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a90f40 0x55d687a91360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.725+0000 7f41e0e54740 1 -- 172.21.15.64:0/777053032 >> 172.21.15.64:0/777053032 conn(0x55d687a8c5b0 msgr2=0x55d687a8c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.726+0000 7f41e0e54740 1 -- 172.21.15.64:0/777053032 shutdown_connections 2024-05-09T09:22:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.726+0000 7f41e0e54740 1 -- 172.21.15.64:0/777053032 wait complete. 2024-05-09T09:22:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.726+0000 7f41e0e54740 1 -- start start 2024-05-09T09:22:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.726+0000 7f41e0e54740 1 Processor -- start 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687a9b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687aa2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50968/0 (socket says 172.21.15.64:50968) 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41d77fe640 1 -- 172.21.15.64:0/634214029 learned_addr learned my addr 172.21.15.64:0/634214029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41d7fff640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687aa2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41d6ffd640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687a9b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d687a99a90 con 0x55d687a90f40 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d6879b0c40 con 0x55d687a96aa0 2024-05-09T09:22:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.727+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d687a99c10 con 0x55d687a91930 2024-05-09T09:22:00.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41e0e54740 10 client.0 _ll_register_callbacks cb 0x55d687708be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d7fff640 1 -- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 msgr2=0x55d687a9b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d7fff640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687a9b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d7fff640 1 -- 172.21.15.64:0/634214029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 msgr2=0x55d687a9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d7fff640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d7fff640 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6877d37d0 con 0x55d687a96aa0 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d77fe640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41d7fff640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687aa2820 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f41d0023000 tx=0x7f41d0010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a835e8d6372ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.728+0000 7f41b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.729+0000 7f41b6ffd640 1 -- 172.21.15.64:0/634214029 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d002e020 con 0x55d687a96aa0 2024-05-09T09:22:00.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.729+0000 7f41b6ffd640 1 -- 172.21.15.64:0/634214029 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41d0004bf0 con 0x55d687a96aa0 2024-05-09T09:22:00.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.729+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d00121d0 con 0x55d687a96aa0 2024-05-09T09:22:00.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.729+0000 7f41b6ffd640 1 -- 172.21.15.64:0/634214029 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f41d0039050 con 0x55d687a96aa0 2024-05-09T09:22:00.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.729+0000 7f41e0e54740 10 client.28064 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:00.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.729+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d687aa5ae0 con 0x55d687a96aa0 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.730+0000 7f41b6ffd640 1 -- 172.21.15.64:0/634214029 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41d00121d0 con 0x55d687a96aa0 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41b6ffd640 1 -- 172.21.15.64:0/634214029 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f41d001f070 con 0x55d687a96aa0 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41e0e54740 10 client.28064 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41e0e54740 10 client.28064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41e0e54740 20 client.28064 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41e0e54740 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687aa7000 0x55d687aa9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41e0e54740 4 client.28064 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d687a03370 con 0x55d687aa7000 2024-05-09T09:22:00.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.731+0000 7f41d77fe640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687aa7000 0x55d687aa9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.732+0000 7f41b6ffd640 10 client.28064 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:00.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.732+0000 7f41d77fe640 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687aa7000 0x55d687aa9410 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68a6c78b9853c46a server_cookie=26d5a7650e6bbd54 in_seq=0 out_seq=0 2024-05-09T09:22:00.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.733+0000 7f41b6ffd640 1 -- 172.21.15.64:0/634214029 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d687a03370 con 0x55d687aa7000 2024-05-09T09:22:00.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.733+0000 7f41b6ffd640 10 client.28064 handle_command_reply: tid=0 2024-05-09T09:22:00.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.733+0000 7f41e0e54740 1 client.28064 shutdown 2024-05-09T09:22:00.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.733+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 msgr2=0x55d687aa2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.733+0000 7f41e0e54740 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687aa2820 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f41d0023000 tx=0x7f41d0010f90 comp rx=0 tx=0).stop 2024-05-09T09:22:00.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 shutdown_connections 2024-05-09T09:22:00.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687aa2820 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687a9b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687aa7000 0x55d687aa9410 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 --2- 172.21.15.64:0/634214029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 >> 172.21.15.64:0/634214029 conn(0x55d687a8c5b0 msgr2=0x55d687a8b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41b6ffd640 0 client.28064 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 shutdown_connections 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 1 -- 172.21.15.64:0/634214029 wait complete. 2024-05-09T09:22:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.734+0000 7f41e0e54740 20 client.28064 trim_cache size 0 max 16384 2024-05-09T09:22:00.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 Processor -- start 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 -- start start 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687aa2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81b34c0 con 0x55d687a90f40 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81b0d60 con 0x55d687a96aa0 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.760+0000 7f41e0e54740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81aebb0 con 0x55d687a91930 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687aa2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a9b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36172/0 (socket says 172.21.15.64:36172) 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d6ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687aa2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52134/0 (socket says 172.21.15.64:52134) 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 -- 172.21.15.64:0/2346849382 learned_addr learned my addr 172.21.15.64:0/2346849382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d77fe640 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 -- 172.21.15.64:0/2346849382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 msgr2=0x55d687aa2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687aa2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 -- 172.21.15.64:0/2346849382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 msgr2=0x55d687a9a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 -- 172.21.15.64:0/2346849382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d687a7dd20 con 0x55d687a96aa0 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.761+0000 7f41d7fff640 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a9b2a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009cd0 tx=0x7f41d0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3979fa7008a6746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.762+0000 7f41bd7fa640 1 -- 172.21.15.64:0/2346849382 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d002e070 con 0x55d687a96aa0 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.762+0000 7f41bd7fa640 1 -- 172.21.15.64:0/2346849382 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41d000c9b0 con 0x55d687a96aa0 2024-05-09T09:22:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.762+0000 7f41bd7fa640 1 -- 172.21.15.64:0/2346849382 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0020ad0 con 0x55d687a96aa0 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 -- 172.21.15.64:0/2346849382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 msgr2=0x55d687a9b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a9b2a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009cd0 tx=0x7f41d0010800 comp rx=0 tx=0).stop 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 -- 172.21.15.64:0/2346849382 shutdown_connections 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a96aa0 0x55d687a9b2a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a91930 0x55d687aa2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 --2- 172.21.15.64:0/2346849382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687a9a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 -- 172.21.15.64:0/2346849382 >> 172.21.15.64:0/2346849382 conn(0x55d687a8c5b0 msgr2=0x55d687a8c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 -- 172.21.15.64:0/2346849382 shutdown_connections 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.763+0000 7f41e0e54740 1 -- 172.21.15.64:0/2346849382 wait complete. 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.764+0000 7f41e0e54740 1 -- start start 2024-05-09T09:22:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.764+0000 7f41e0e54740 1 Processor -- start 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687aa2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a91930 0x55d687a9ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687aa2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687aa2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50992/0 (socket says 172.21.15.64:50992) 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41d77fe640 1 -- 172.21.15.64:0/1875336044 learned_addr learned my addr 172.21.15.64:0/1875336044 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41d6ffd640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a91930 0x55d687a9ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a96aa0 0x55d687a9f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81af220 con 0x55d687a90f40 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81b0690 con 0x55d687a91930 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81af8f0 con 0x55d687a96aa0 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41e0e54740 10 client.0 _ll_register_callbacks cb 0x7f41d0009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.765+0000 7f41d7fff640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a96aa0 0x55d687a9f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d77fe640 1 -- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a96aa0 msgr2=0x55d687a9f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d77fe640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a96aa0 0x55d687a9f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d77fe640 1 -- 172.21.15.64:0/1875336044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a91930 msgr2=0x55d687a9ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d77fe640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a91930 0x55d687a9ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d77fe640 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d687a9f880 con 0x55d687a90f40 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d7fff640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a96aa0 0x55d687a9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:00.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.766+0000 7f41d6ffd640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a91930 0x55d687a9ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.770+0000 7f41d77fe640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687aa2360 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f41c8097ee0 tx=0x7f41c809bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d91e1e0b2dbf63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.770+0000 7f41b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:00.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.771+0000 7f41b6ffd640 1 -- 172.21.15.64:0/1875336044 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8020040 con 0x55d687a90f40 2024-05-09T09:22:00.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.771+0000 7f41b6ffd640 1 -- 172.21.15.64:0/1875336044 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f41c80a3cf0 con 0x55d687a90f40 2024-05-09T09:22:00.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.771+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d000c9b0 con 0x55d687a90f40 2024-05-09T09:22:00.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41b6ffd640 1 -- 172.21.15.64:0/1875336044 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f41c809f8c0 con 0x55d687a90f40 2024-05-09T09:22:00.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41e0e54740 10 client.18594 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:00.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d687b0c5b0 con 0x55d687a90f40 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41b6ffd640 1 -- 172.21.15.64:0/1875336044 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41d000c9b0 con 0x55d687a90f40 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41b6ffd640 1 -- 172.21.15.64:0/1875336044 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f41c8097260 con 0x55d687a90f40 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41e0e54740 10 client.18594 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41e0e54740 10 client.18594 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41e0e54740 20 client.18594 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.772+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687b0cf10 0x55d687b0f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.773+0000 7f41e0e54740 4 client.18594 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:22:00.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.773+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6879fd6c0 con 0x55d687b0cf10 2024-05-09T09:22:00.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.773+0000 7f41d6ffd640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687b0cf10 0x55d687b0f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.773+0000 7f41d6ffd640 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687b0cf10 0x55d687b0f330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d232c9433faf201d server_cookie=3ed564e75c52652b in_seq=0 out_seq=0 2024-05-09T09:22:00.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.774+0000 7f41b6ffd640 10 client.18594 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:00.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.774+0000 7f41b6ffd640 1 -- 172.21.15.64:0/1875336044 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6879fd6c0 con 0x55d687b0cf10 2024-05-09T09:22:00.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.774+0000 7f41b6ffd640 10 client.18594 handle_command_reply: tid=0 2024-05-09T09:22:00.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.774+0000 7f41e0e54740 1 client.18594 shutdown 2024-05-09T09:22:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.774+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 msgr2=0x55d687aa2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.774+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687aa2360 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f41c8097ee0 tx=0x7f41c809bcd0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 shutdown_connections 2024-05-09T09:22:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d687a96aa0 0x55d687a9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d687a91930 0x55d687a9ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55d687b0cf10 0x55d687b0f330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 --2- 172.21.15.64:0/1875336044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d687a90f40 0x55d687aa2360 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41b6ffd640 0 client.18594 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 >> 172.21.15.64:0/1875336044 conn(0x55d687a8c5b0 msgr2=0x55d687a9b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 shutdown_connections 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.775+0000 7f41e0e54740 1 -- 172.21.15.64:0/1875336044 wait complete. 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.776+0000 7f41e0e54740 20 client.18594 trim_cache size 0 max 16384 2024-05-09T09:22:00.776 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:22:00.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.778+0000 7f41de142640 1 -- 172.21.15.64:0/209315 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f41c00a2d60 msgr2=0x7f41c00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.778+0000 7f41de142640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f41c00a2d60 0x7f41c00a5220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009fd0 tx=0x7f41d0009d80 comp rx=0 tx=0).stop 2024-05-09T09:22:00.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.778+0000 7f41de142640 1 -- 172.21.15.64:0/209315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 msgr2=0x7f41d819e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.778+0000 7f41de142640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 0x7f41d819e130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f41c800d950 tx=0x7f41c800de10 comp rx=0 tx=0).stop 2024-05-09T09:22:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.779+0000 7f41de142640 1 -- 172.21.15.64:0/209315 shutdown_connections 2024-05-09T09:22:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.779+0000 7f41de142640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f41c00a2d60 0x7f41c00a5220 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.779+0000 7f41de142640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41d810db70 0x7f41d81a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.779+0000 7f41de142640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41d810b1a0 0x7f41d819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.779+0000 7f41de142640 1 --2- 172.21.15.64:0/209315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d8108830 0x7f41d819e130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.779+0000 7f41de142640 1 -- 172.21.15.64:0/209315 >> 172.21.15.64:0/209315 conn(0x7f41d806e6c0 msgr2=0x7f41d810bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.780+0000 7f41de142640 1 -- 172.21.15.64:0/209315 shutdown_connections 2024-05-09T09:22:00.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.780+0000 7f41de142640 1 -- 172.21.15.64:0/209315 wait complete. 2024-05-09T09:22:00.799 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:00 smithi064 ceph-mon[30149]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 833 op/s 2024-05-09T09:22:00.799 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:00 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2091980984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:00.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:22:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:00 smithi179 ceph-mon[34086]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 833 op/s 2024-05-09T09:22:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:00 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2091980984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.991+0000 7f06eea04640 1 -- 172.21.15.64:0/3974256409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 msgr2=0x7f06e0099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.991+0000 7f06eea04640 1 --2- 172.21.15.64:0/3974256409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0099d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009860 tx=0x7f06e402a340 comp rx=0 tx=0).stop 2024-05-09T09:22:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.992+0000 7f06eea04640 1 -- 172.21.15.64:0/3974256409 shutdown_connections 2024-05-09T09:22:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.992+0000 7f06eea04640 1 --2- 172.21.15.64:0/3974256409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0099d90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.992+0000 7f06eea04640 1 --2- 172.21.15.64:0/3974256409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e0096f60 0x7f06e00993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.992+0000 7f06eea04640 1 --2- 172.21.15.64:0/3974256409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06e00945f0 0x7f06e0096a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.992+0000 7f06eea04640 1 -- 172.21.15.64:0/3974256409 >> 172.21.15.64:0/3974256409 conn(0x7f06e00901d0 msgr2=0x7f06e0092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:00.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.994+0000 7f06eea04640 1 -- 172.21.15.64:0/3974256409 shutdown_connections 2024-05-09T09:22:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.995+0000 7f06eea04640 1 -- 172.21.15.64:0/3974256409 wait complete. 2024-05-09T09:22:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 Processor -- start 2024-05-09T09:22:00.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 -- start start 2024-05-09T09:22:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e00945f0 0x7f06e009fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06e0096f60 0x7f06e00a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06e00a7ef0 con 0x7f06e0096f60 2024-05-09T09:22:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06e00a7d70 con 0x7f06e00945f0 2024-05-09T09:22:00.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eea04640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06e00a8070 con 0x7f06e0099910 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.996+0000 7f06eda02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e00945f0 0x7f06e009fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06eda02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e00945f0 0x7f06e009fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36206/0 (socket says 172.21.15.64:36206) 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06eda02640 1 -- 172.21.15.64:0/1997371296 learned_addr learned my addr 172.21.15.64:0/1997371296 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ee203640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ed201640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06e0096f60 0x7f06e00a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ee203640 1 -- 172.21.15.64:0/1997371296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e00945f0 msgr2=0x7f06e009fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ee203640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e00945f0 0x7f06e009fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ee203640 1 -- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06e0096f60 msgr2=0x7f06e00a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ee203640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06e0096f60 0x7f06e00a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:00.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.997+0000 7f06ee203640 1 -- 172.21.15.64:0/1997371296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e00a07b0 con 0x7f06e0099910 2024-05-09T09:22:00.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.998+0000 7f06ee203640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0142600 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f06e402a860 tx=0x7f06e4005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:00.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.998+0000 7f06deffd640 1 -- 172.21.15.64:0/1997371296 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4040020 con 0x7f06e0099910 2024-05-09T09:22:00.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.998+0000 7f06deffd640 1 -- 172.21.15.64:0/1997371296 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e40046c0 con 0x7f06e0099910 2024-05-09T09:22:00.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.999+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e0142c00 con 0x7f06e0099910 2024-05-09T09:22:00.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.999+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e01430c0 con 0x7f06e0099910 2024-05-09T09:22:00.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:00.999+0000 7f06deffd640 1 -- 172.21.15.64:0/1997371296 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4037070 con 0x7f06e0099910 2024-05-09T09:22:01.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.003+0000 7f06f2970740 1 Processor -- start 2024-05-09T09:22:01.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.005+0000 7f06deffd640 1 -- 172.21.15.64:0/1997371296 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06e4043020 con 0x7f06e0099910 2024-05-09T09:22:01.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 -- start start 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 0x563a5b8be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8c3c90 0x563a5b8c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06ed201640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 0x563a5b8be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06ed201640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 0x563a5b8be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52160/0 (socket says 172.21.15.64:52160) 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06ed201640 1 -- 172.21.15.64:0/2001796967 learned_addr learned my addr 172.21.15.64:0/2001796967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a5b822c40 con 0x563a5b8c3c90 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563a5b7ca580 con 0x563a5b8beb20 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a5b81ec00 con 0x563a5b8be130 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06ed201640 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8c3c90 0x563a5b8c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.007+0000 7f06eda02640 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06ed201640 1 -- 172.21.15.64:0/2001796967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 msgr2=0x563a5b8c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06ed201640 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8c3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06ed201640 1 -- 172.21.15.64:0/2001796967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8c3c90 msgr2=0x563a5b8c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06ed201640 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8c3c90 0x563a5b8c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06ed201640 1 -- 172.21.15.64:0/2001796967 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5b5363f0 con 0x563a5b8be130 2024-05-09T09:22:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06eda02640 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:01.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06ed201640 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 0x563a5b8be550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f06e802d750 tx=0x7f06e8062d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0f6c8977038fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.008+0000 7f06d3fff640 1 -- 172.21.15.64:0/2001796967 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8073e60 con 0x563a5b8be130 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.009+0000 7f06d3fff640 1 -- 172.21.15.64:0/2001796967 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e806e6e0 con 0x563a5b8be130 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.009+0000 7f06deffd640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06d40a2d60 0x7f06d40a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.009+0000 7f06d3fff640 1 -- 172.21.15.64:0/2001796967 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e80696b0 con 0x563a5b8be130 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06eda02640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06d40a2d60 0x7f06d40a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06deffd640 1 -- 172.21.15.64:0/1997371296 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06e40ef530 con 0x7f06e0099910 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 msgr2=0x563a5b8be550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 0x563a5b8be550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f06e802d750 tx=0x7f06e8062d60 comp rx=0 tx=0).stop 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 shutdown_connections 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8c3c90 0x563a5b8c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 --2- 172.21.15.64:0/2001796967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8be130 0x563a5b8be550 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.010+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 >> 172.21.15.64:0/2001796967 conn(0x563a5b8b97a0 msgr2=0x563a5b8b9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.011+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 shutdown_connections 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.011+0000 7f06eda02640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06d40a2d60 0x7f06d40a5220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x563a5b8ba710 tx=0x7f06d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.011+0000 7f06f2970740 1 -- 172.21.15.64:0/2001796967 wait complete. 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.011+0000 7f06f2970740 1 -- start start 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.011+0000 7f06f2970740 1 Processor -- start 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8beb20 0x563a5b8d9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8c3c90 0x563a5b8e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a5b8d7b90 con 0x563a5b8be130 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563a5b7ddc40 con 0x563a5b8c3c90 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a5b8d7d10 con 0x563a5b8beb20 2024-05-09T09:22:01.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06f2970740 10 client.0 _ll_register_callbacks cb 0x563a5b535be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06eda02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06eda02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8d9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51024/0 (socket says 172.21.15.64:51024) 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.012+0000 7f06eda02640 1 -- 172.21.15.64:0/3805439493 learned_addr learned my addr 172.21.15.64:0/3805439493 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8beb20 0x563a5b8d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ee203640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8c3c90 0x563a5b8e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 -- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8c3c90 msgr2=0x563a5b8e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8c3c90 0x563a5b8e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 -- 172.21.15.64:0/3805439493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 msgr2=0x563a5b8d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 -- 172.21.15.64:0/3805439493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5b6007d0 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06eda02640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06ed201640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8beb20 0x563a5b8d9980 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f06e8050860 tx=0x7f06e8069660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd1674c1c6577d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.013+0000 7f06d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.014+0000 7f06d17fa640 1 -- 172.21.15.64:0/3805439493 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8086070 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.014+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e806e6e0 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.014+0000 7f06d17fa640 1 -- 172.21.15.64:0/3805439493 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06e8082430 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.014+0000 7f06d17fa640 1 -- 172.21.15.64:0/3805439493 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f06e8092050 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.014+0000 7f06f2970740 10 client.28240 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.014+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563a5b8e3d80 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.015+0000 7f06d17fa640 1 -- 172.21.15.64:0/3805439493 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e8082ae0 con 0x563a5b8beb20 2024-05-09T09:22:01.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.015+0000 7f06d17fa640 1 -- 172.21.15.64:0/3805439493 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f06e8067ce0 con 0x563a5b8beb20 2024-05-09T09:22:01.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.016+0000 7f06f2970740 10 client.28240 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:01.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.016+0000 7f06f2970740 10 client.28240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:01.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.016+0000 7f06f2970740 20 client.28240 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:01.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.016+0000 7f06f2970740 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b8e5960 0x563a5b8e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.016+0000 7f06f2970740 4 client.28240 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:01.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.016+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f06e806e6e0 con 0x563a5b8e5960 2024-05-09T09:22:01.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.018+0000 7f06eda02640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b8e5960 0x563a5b8e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.018+0000 7f06eda02640 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b8e5960 0x563a5b8e7d80 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3395dd1c1977f64e server_cookie=4344bd961ee3daf5 in_seq=0 out_seq=0 2024-05-09T09:22:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.018+0000 7f06d17fa640 10 client.28240 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.019+0000 7f06d17fa640 1 -- 172.21.15.64:0/3805439493 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f06e806e6e0 con 0x563a5b8e5960 2024-05-09T09:22:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.019+0000 7f06d17fa640 10 client.28240 handle_command_reply: tid=0 2024-05-09T09:22:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.019+0000 7f06f2970740 1 client.28240 shutdown 2024-05-09T09:22:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8beb20 msgr2=0x563a5b8d9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:01.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8beb20 0x563a5b8d9980 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f06e8050860 tx=0x7f06e8069660 comp rx=0 tx=0).stop 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 shutdown_connections 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b8e5960 0x563a5b8e7d80 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8c3c90 0x563a5b8e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8beb20 0x563a5b8d9980 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 --2- 172.21.15.64:0/3805439493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 >> 172.21.15.64:0/3805439493 conn(0x563a5b8b97a0 msgr2=0x563a5b8b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06d17fa640 0 client.28240 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 shutdown_connections 2024-05-09T09:22:01.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.020+0000 7f06f2970740 1 -- 172.21.15.64:0/3805439493 wait complete. 2024-05-09T09:22:01.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.021+0000 7f06f2970740 20 client.28240 trim_cache size 0 max 16384 2024-05-09T09:22:01.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.049+0000 7f06f2970740 1 Processor -- start 2024-05-09T09:22:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.049+0000 7f06f2970740 1 -- start start 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.049+0000 7f06f2970740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.049+0000 7f06f2970740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06f2970740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8c0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06ed201640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06ed201640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8e0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36230/0 (socket says 172.21.15.64:36230) 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06ed201640 1 -- 172.21.15.64:0/2484909842 learned_addr learned my addr 172.21.15.64:0/2484909842 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc0124c0 con 0x563a5b8be130 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06ee203640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc00fd60 con 0x563a5b8beb20 2024-05-09T09:22:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.050+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc00dbb0 con 0x563a5b8c3c90 2024-05-09T09:22:01.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.051+0000 7f06eda02640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.052+0000 7f06eda02640 1 -- 172.21.15.64:0/2484909842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 msgr2=0x563a5b8c0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.052+0000 7f06eda02640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8c0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.052+0000 7f06eda02640 1 -- 172.21.15.64:0/2484909842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 msgr2=0x563a5b8e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.052+0000 7f06eda02640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8e0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.052+0000 7f06eda02640 1 -- 172.21.15.64:0/2484909842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5b8c65e0 con 0x563a5b8be130 2024-05-09T09:22:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.053+0000 7f06eda02640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8bfee0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x563a5b8e8600 tx=0x7f06d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6d667311102d8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.054+0000 7f06ed201640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.054+0000 7f06ee203640 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.054+0000 7f06d37fe640 1 -- 172.21.15.64:0/2484909842 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d801b040 con 0x563a5b8be130 2024-05-09T09:22:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.054+0000 7f06d37fe640 1 -- 172.21.15.64:0/2484909842 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d802d070 con 0x563a5b8be130 2024-05-09T09:22:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.054+0000 7f06d37fe640 1 -- 172.21.15.64:0/2484909842 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80237c0 con 0x563a5b8be130 2024-05-09T09:22:01.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.055+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 msgr2=0x563a5b8bfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:01.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.055+0000 7f06f2970740 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8bfee0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x563a5b8e8600 tx=0x7f06d8012000 comp rx=0 tx=0).stop 2024-05-09T09:22:01.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.055+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 shutdown_connections 2024-05-09T09:22:01.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.055+0000 7f06f2970740 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.055+0000 7f06f2970740 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.055+0000 7f06f2970740 1 --2- 172.21.15.64:0/2484909842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8bfee0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.056+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 >> 172.21.15.64:0/2484909842 conn(0x563a5b8b97a0 msgr2=0x563a5b8d9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.056+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 shutdown_connections 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.056+0000 7f06f2970740 1 -- 172.21.15.64:0/2484909842 wait complete. 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.057+0000 7f06f2970740 1 -- start start 2024-05-09T09:22:01.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.057+0000 7f06f2970740 1 Processor -- start 2024-05-09T09:22:01.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06f2970740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06f2970740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06f2970740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8dd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06f2970740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc00e220 con 0x563a5b8be130 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06ed201640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06ed201640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8dd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36240/0 (socket says 172.21.15.64:36240) 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06ed201640 1 -- 172.21.15.64:0/2698070922 learned_addr learned my addr 172.21.15.64:0/2698070922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc00f690 con 0x563a5b8beb20 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc00e8f0 con 0x563a5b8c3c90 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.058+0000 7f06ee203640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06f2970740 10 client.0 _ll_register_callbacks cb 0x7f06e806e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06eda02640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06ed201640 1 -- 172.21.15.64:0/2698070922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 msgr2=0x563a5b8dd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06ed201640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8dd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06ed201640 1 -- 172.21.15.64:0/2698070922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 msgr2=0x563a5b8e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:01.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06ed201640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.059+0000 7f06ed201640 1 -- 172.21.15.64:0/2698070922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5b8e0b20 con 0x563a5b8beb20 2024-05-09T09:22:01.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.060+0000 7f06ed201640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8dd110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f06e8050860 tx=0x7f06e808b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667e2aeab07018a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:01.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.060+0000 7f06d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.062+0000 7f06d17fa640 1 -- 172.21.15.64:0/2698070922 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8086050 con 0x563a5b8beb20 2024-05-09T09:22:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.062+0000 7f06d17fa640 1 -- 172.21.15.64:0/2698070922 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f06e80730c0 con 0x563a5b8beb20 2024-05-09T09:22:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.062+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d802d070 con 0x563a5b8beb20 2024-05-09T09:22:01.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.062+0000 7f06d17fa640 1 -- 172.21.15.64:0/2698070922 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f06e8073e60 con 0x563a5b8beb20 2024-05-09T09:22:01.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.063+0000 7f06f2970740 10 client.28085 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:01.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.063+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563a5b94ae00 con 0x563a5b8beb20 2024-05-09T09:22:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.064+0000 7f06d17fa640 1 -- 172.21.15.64:0/2698070922 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e808f070 con 0x563a5b8beb20 2024-05-09T09:22:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.064+0000 7f06d17fa640 1 -- 172.21.15.64:0/2698070922 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f06e8067ce0 con 0x563a5b8beb20 2024-05-09T09:22:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.064+0000 7f06f2970740 10 client.28085 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.064+0000 7f06f2970740 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.064+0000 7f06f2970740 20 client.28085 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.065+0000 7f06f2970740 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b94b760 0x563a5b94db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.065+0000 7f06f2970740 4 client.28085 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:22:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.065+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f06d802d070 con 0x563a5b94b760 2024-05-09T09:22:01.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.066+0000 7f06eda02640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b94b760 0x563a5b94db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:01.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.067+0000 7f06eda02640 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b94b760 0x563a5b94db80 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8afb16e8ad2fdb server_cookie=ef22eb207e6d6053 in_seq=0 out_seq=0 2024-05-09T09:22:01.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.068+0000 7f06d17fa640 10 client.28085 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:01.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.068+0000 7f06d17fa640 1 -- 172.21.15.64:0/2698070922 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f06d802d070 con 0x563a5b94b760 2024-05-09T09:22:01.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.068+0000 7f06d17fa640 10 client.28085 handle_command_reply: tid=0 2024-05-09T09:22:01.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.068+0000 7f06f2970740 1 client.28085 shutdown 2024-05-09T09:22:01.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.069+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 msgr2=0x563a5b8dd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:01.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.069+0000 7f06f2970740 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8dd110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f06e8050860 tx=0x7f06e808b710 comp rx=0 tx=0).stop 2024-05-09T09:22:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:00 smithi139 ceph-mon[33599]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 833 op/s 2024-05-09T09:22:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:00 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2091980984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.074+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 shutdown_connections 2024-05-09T09:22:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.074+0000 7f06f2970740 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563a5b94b760 0x563a5b94db80 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.074+0000 7f06f2970740 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5b8c3c90 0x563a5b8dd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.074+0000 7f06f2970740 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563a5b8beb20 0x563a5b8dd110 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.074+0000 7f06f2970740 1 --2- 172.21.15.64:0/2698070922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a5b8be130 0x563a5b8e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.074+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 >> 172.21.15.64:0/2698070922 conn(0x563a5b8b97a0 msgr2=0x563a5b8bcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:01.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.075+0000 7f06d17fa640 0 client.28085 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:01.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.075+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 shutdown_connections 2024-05-09T09:22:01.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.075+0000 7f06f2970740 1 -- 172.21.15.64:0/2698070922 wait complete. 2024-05-09T09:22:01.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.076+0000 7f06f2970740 20 client.28085 trim_cache size 0 max 16384 2024-05-09T09:22:01.079 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:22:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.081+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06d40a2d60 msgr2=0x7f06d40a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.081+0000 7f06eea04640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06d40a2d60 0x7f06d40a5220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x563a5b8ba710 tx=0x7f06d8008040 comp rx=0 tx=0).stop 2024-05-09T09:22:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.081+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 msgr2=0x7f06e0142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:01.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.081+0000 7f06eea04640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0142600 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f06e402a860 tx=0x7f06e4005270 comp rx=0 tx=0).stop 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 shutdown_connections 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06d40a2d60 0x7f06d40a5220 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0099910 0x7f06e0142600 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06e0096f60 0x7f06e00a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 --2- 172.21.15.64:0/1997371296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e00945f0 0x7f06e009fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 >> 172.21.15.64:0/1997371296 conn(0x7f06e00901d0 msgr2=0x7f06e0097c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 shutdown_connections 2024-05-09T09:22:01.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:01.082+0000 7f06eea04640 1 -- 172.21.15.64:0/1997371296 wait complete. 2024-05-09T09:22:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 700.0 s, 719.6 tps, lat 138.813 ms stddev 79.349 2024-05-09T09:22:02.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:22:02.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:22:02.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 -- 172.21.15.64:0/2216104374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058108f60 msgr2=0x7fa0581093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 --2- 172.21.15.64:0/2216104374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058108f60 0x7fa0581093e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009900 tx=0x7fa04c02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 -- 172.21.15.64:0/2216104374 shutdown_connections 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 --2- 172.21.15.64:0/2216104374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa058109920 0x7fa0581101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 --2- 172.21.15.64:0/2216104374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058108f60 0x7fa0581093e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 --2- 172.21.15.64:0/2216104374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058107d70 0x7fa058108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.311+0000 7fa05c973640 1 -- 172.21.15.64:0/2216104374 >> 172.21.15.64:0/2216104374 conn(0x7fa058076d00 msgr2=0x7fa058079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.312+0000 7fa05c973640 1 -- 172.21.15.64:0/2216104374 shutdown_connections 2024-05-09T09:22:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.312+0000 7fa05c973640 1 -- 172.21.15.64:0/2216104374 wait complete. 2024-05-09T09:22:02.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.313+0000 7fa05c973640 1 Processor -- start 2024-05-09T09:22:02.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.313+0000 7fa05c973640 1 -- start start 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa05c973640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058107d70 0x7fa05819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa05c973640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa058108f60 0x7fa05819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa05c973640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 0x7fa0581a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa05c973640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa058113880 con 0x7fa058108f60 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa057fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 0x7fa0581a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa057fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 0x7fa0581a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52210/0 (socket says 172.21.15.64:52210) 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa057fff640 1 -- 172.21.15.64:0/49138237 learned_addr learned my addr 172.21.15.64:0/49138237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa058113700 con 0x7fa058107d70 2024-05-09T09:22:02.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.314+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa058113a00 con 0x7fa058109920 2024-05-09T09:22:02.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa0577fe640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058107d70 0x7fa05819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa057fff640 1 -- 172.21.15.64:0/49138237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058107d70 msgr2=0x7fa05819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa057fff640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058107d70 0x7fa05819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa057fff640 1 -- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa058108f60 msgr2=0x7fa05819e510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa057fff640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa058108f60 0x7fa05819e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa057fff640 1 -- 172.21.15.64:0/49138237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0581a5ba0 con 0x7fa058109920 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa0577fe640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058107d70 0x7fa05819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.315+0000 7fa057fff640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 0x7fa0581a5530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e970 tx=0x7fa05000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.316+0000 7fa054ff9640 1 -- 172.21.15.64:0/49138237 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05000ccc0 con 0x7fa058109920 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.316+0000 7fa054ff9640 1 -- 172.21.15.64:0/49138237 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa05000ce60 con 0x7fa058109920 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.316+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0581a5e90 con 0x7fa058109920 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.316+0000 7fa054ff9640 1 -- 172.21.15.64:0/49138237 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050010430 con 0x7fa058109920 2024-05-09T09:22:02.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.316+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a6350 con 0x7fa058109920 2024-05-09T09:22:02.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.320+0000 7fa0609c4740 1 Processor -- start 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa054ff9640 1 -- 172.21.15.64:0/49138237 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0500046c0 con 0x7fa058109920 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 -- start start 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c499ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c49eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b37c3fd990 con 0x55b37c4996b0 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b37c3a55b0 con 0x55b37c49a0a0 2024-05-09T09:22:02.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.322+0000 7fa0609c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b37c3f9950 con 0x55b37c49f210 2024-05-09T09:22:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c49eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c49eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36280/0 (socket says 172.21.15.64:36280) 2024-05-09T09:22:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 -- 172.21.15.64:0/200929596 learned_addr learned my addr 172.21.15.64:0/200929596 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa056ffd640 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c499ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa056ffd640 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 -- 172.21.15.64:0/200929596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 msgr2=0x55b37c4a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 -- 172.21.15.64:0/200929596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 msgr2=0x55b37c499ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c499ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa0577fe640 1 -- 172.21.15.64:0/200929596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37c1d97e0 con 0x55b37c49a0a0 2024-05-09T09:22:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.323+0000 7fa056ffd640 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:02.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.324+0000 7fa0577fe640 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c49eba0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa048011820 tx=0x7fa048011ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddea75d1a013f8d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.324+0000 7fa03dffb640 1 -- 172.21.15.64:0/200929596 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048011220 con 0x55b37c49a0a0 2024-05-09T09:22:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.325+0000 7fa03dffb640 1 -- 172.21.15.64:0/200929596 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa048019430 con 0x55b37c49a0a0 2024-05-09T09:22:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.325+0000 7fa03dffb640 1 -- 172.21.15.64:0/200929596 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048010670 con 0x55b37c49a0a0 2024-05-09T09:22:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.325+0000 7fa054ff9640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0400a2d60 0x7fa0400a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.325+0000 7fa054ff9640 1 -- 172.21.15.64:0/49138237 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa0500cf720 con 0x7fa058109920 2024-05-09T09:22:02.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.328+0000 7fa056ffd640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0400a2d60 0x7fa0400a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.329+0000 7fa0609c4740 1 -- 172.21.15.64:0/200929596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 msgr2=0x55b37c49eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.329+0000 7fa0609c4740 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c49eba0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa048011820 tx=0x7fa048011ce0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.330+0000 7fa056ffd640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0400a2d60 0x7fa0400a5220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa04c02a910 tx=0x7fa04c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.330+0000 7fa0609c4740 1 -- 172.21.15.64:0/200929596 shutdown_connections 2024-05-09T09:22:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.330+0000 7fa0609c4740 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.330+0000 7fa0609c4740 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c49eba0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.330+0000 7fa0609c4740 1 --2- 172.21.15.64:0/200929596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c499ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.330+0000 7fa0609c4740 1 -- 172.21.15.64:0/200929596 >> 172.21.15.64:0/200929596 conn(0x55b37c4610a0 msgr2=0x55b37c495130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.332+0000 7fa0609c4740 1 -- 172.21.15.64:0/200929596 shutdown_connections 2024-05-09T09:22:02.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.332+0000 7fa0609c4740 1 -- 172.21.15.64:0/200929596 wait complete. 2024-05-09T09:22:02.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.332+0000 7fa0609c4740 1 -- start start 2024-05-09T09:22:02.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.333+0000 7fa0609c4740 1 Processor -- start 2024-05-09T09:22:02.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.333+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c4a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.333+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c4a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.333+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4aae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa0609c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b37c4a21c0 con 0x55b37c4996b0 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa0609c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b37c3b8b40 con 0x55b37c49a0a0 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa0609c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b37c4a2340 con 0x55b37c49f210 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa056ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c4a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa0609c4740 10 client.0 _ll_register_callbacks cb 0x55b37c3ad0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa057fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.334+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c4a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c4a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51060/0 (socket says 172.21.15.64:51060) 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 -- 172.21.15.64:0/1840998860 learned_addr learned my addr 172.21.15.64:0/1840998860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 -- 172.21.15.64:0/1840998860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 msgr2=0x55b37c4aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 -- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 msgr2=0x55b37c4a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c4a3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.335+0000 7fa0577fe640 1 -- 172.21.15.64:0/1840998860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37c239f70 con 0x55b37c4996b0 2024-05-09T09:22:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.336+0000 7fa0577fe640 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c4a38f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa04800a010 tx=0x7fa048019b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2652035d0f575e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.336+0000 7fa036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.336+0000 7fa036ffd640 1 -- 172.21.15.64:0/1840998860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048027020 con 0x55b37c4996b0 2024-05-09T09:22:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.337+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa048019430 con 0x55b37c4996b0 2024-05-09T09:22:02.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.337+0000 7fa036ffd640 1 -- 172.21.15.64:0/1840998860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa048004c50 con 0x55b37c4996b0 2024-05-09T09:22:02.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.337+0000 7fa036ffd640 1 -- 172.21.15.64:0/1840998860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa048032050 con 0x55b37c4996b0 2024-05-09T09:22:02.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.338+0000 7fa0609c4740 10 client.18618 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:02.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.338+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b37c0d1170 con 0x55b37c4996b0 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.339+0000 7fa036ffd640 1 -- 172.21.15.64:0/1840998860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa048003970 con 0x55b37c4996b0 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.339+0000 7fa036ffd640 1 -- 172.21.15.64:0/1840998860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fa048004110 con 0x55b37c4996b0 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.339+0000 7fa0609c4740 10 client.18618 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.339+0000 7fa0609c4740 10 client.18618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.339+0000 7fa0609c4740 20 client.18618 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.339+0000 7fa0609c4740 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c4af7a0 0x55b37c4b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.340+0000 7fa0609c4740 4 client.18618 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:02.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.340+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa048019430 con 0x55b37c4af7a0 2024-05-09T09:22:02.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.340+0000 7fa056ffd640 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c4af7a0 0x55b37c4b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.342+0000 7fa036ffd640 10 client.18618 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.342+0000 7fa056ffd640 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c4af7a0 0x55b37c4b1bc0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b1db0e90857f8a server_cookie=b7723266f6d01022 in_seq=0 out_seq=0 2024-05-09T09:22:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.343+0000 7fa036ffd640 1 -- 172.21.15.64:0/1840998860 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa048019430 con 0x55b37c4af7a0 2024-05-09T09:22:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.343+0000 7fa036ffd640 10 client.18618 handle_command_reply: tid=0 2024-05-09T09:22:02.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.344+0000 7fa0609c4740 1 client.18618 shutdown 2024-05-09T09:22:02.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.346+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 msgr2=0x55b37c4a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.346+0000 7fa0609c4740 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c4a38f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa04800a010 tx=0x7fa048019b20 comp rx=0 tx=0).stop 2024-05-09T09:22:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.347+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 shutdown_connections 2024-05-09T09:22:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.347+0000 7fa0609c4740 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c4af7a0 0x55b37c4b1bc0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.347+0000 7fa0609c4740 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.347+0000 7fa0609c4740 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c49a0a0 0x55b37c4a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.347+0000 7fa0609c4740 1 --2- 172.21.15.64:0/1840998860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c4996b0 0x55b37c4a38f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.347+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 >> 172.21.15.64:0/1840998860 conn(0x55b37c4610a0 msgr2=0x55b37c494270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.349+0000 7fa036ffd640 0 client.18618 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.350+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 shutdown_connections 2024-05-09T09:22:02.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.351+0000 7fa0609c4740 1 -- 172.21.15.64:0/1840998860 wait complete. 2024-05-09T09:22:02.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.351+0000 7fa0609c4740 20 client.18618 trim_cache size 0 max 16384 2024-05-09T09:22:02.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.395+0000 7fa0609c4740 1 Processor -- start 2024-05-09T09:22:02.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.395+0000 7fa0609c4740 1 -- start start 2024-05-09T09:22:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 0x55b37c4a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49a0a0 0x55b37c4aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0609c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b3210 con 0x55b37c49a0a0 2024-05-09T09:22:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0609c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b0ab0 con 0x55b37c4996b0 2024-05-09T09:22:02.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0609c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581ae900 con 0x55b37c49f210 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 0x55b37c4a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa056ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49a0a0 0x55b37c4aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 0x55b37c4a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36294/0 (socket says 172.21.15.64:36294) 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 -- 172.21.15.64:0/2711008187 learned_addr learned my addr 172.21.15.64:0/2711008187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa057fff640 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 -- 172.21.15.64:0/2711008187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 msgr2=0x55b37c4949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 -- 172.21.15.64:0/2711008187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49a0a0 msgr2=0x55b37c4aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.396+0000 7fa0577fe640 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49a0a0 0x55b37c4aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.397+0000 7fa0577fe640 1 -- 172.21.15.64:0/2711008187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37c49a480 con 0x55b37c4996b0 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.397+0000 7fa057fff640 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:02.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.397+0000 7fa0577fe640 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 0x55b37c4a3d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa0480115e0 tx=0x7fa048009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57dfccdf6c88f7d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.399+0000 7fa03d7fa640 1 -- 172.21.15.64:0/2711008187 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048004c50 con 0x55b37c4996b0 2024-05-09T09:22:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.399+0000 7fa03d7fa640 1 -- 172.21.15.64:0/2711008187 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04803b070 con 0x55b37c4996b0 2024-05-09T09:22:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.399+0000 7fa03d7fa640 1 -- 172.21.15.64:0/2711008187 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048005570 con 0x55b37c4996b0 2024-05-09T09:22:02.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.400+0000 7fa0609c4740 1 -- 172.21.15.64:0/2711008187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 msgr2=0x55b37c4a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.400+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 0x55b37c4a3d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa0480115e0 tx=0x7fa048009990 comp rx=0 tx=0).stop 2024-05-09T09:22:02.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.401+0000 7fa0609c4740 1 -- 172.21.15.64:0/2711008187 shutdown_connections 2024-05-09T09:22:02.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.401+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49f210 0x55b37c4949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.401+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49a0a0 0x55b37c4aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.401+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2711008187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4996b0 0x55b37c4a3d90 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa0480115e0 tx=0x7fa048009990 comp rx=0 tx=0).stop 2024-05-09T09:22:02.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.401+0000 7fa0609c4740 1 -- 172.21.15.64:0/2711008187 >> 172.21.15.64:0/2711008187 conn(0x55b37c4610a0 msgr2=0x55b37c4a2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.403+0000 7fa0609c4740 1 -- 172.21.15.64:0/2711008187 shutdown_connections 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.404+0000 7fa0609c4740 1 -- 172.21.15.64:0/2711008187 wait complete. 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.404+0000 7fa0609c4740 1 -- start start 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.405+0000 7fa0609c4740 1 Processor -- start 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.405+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 0x55b37c4a81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.405+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49f210 0x55b37c4a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.405+0000 7fa0609c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4a9710 0x55b37c4a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.406+0000 7fa0609c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581aef70 con 0x55b37c49f210 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.406+0000 7fa0609c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b03e0 con 0x55b37c4a9710 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.406+0000 7fa0609c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581af640 con 0x55b37c49a0a0 2024-05-09T09:22:02.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.406+0000 7fa0609c4740 10 client.0 _ll_register_callbacks cb 0x7fa0480116a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 0x55b37c4a81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 0x55b37c4a81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52236/0 (socket says 172.21.15.64:52236) 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 -- 172.21.15.64:0/2887703930 learned_addr learned my addr 172.21.15.64:0/2887703930 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa056ffd640 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49f210 0x55b37c4a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 -- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4a9710 msgr2=0x55b37c4a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4a9710 0x55b37c4a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 -- 172.21.15.64:0/2887703930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49f210 msgr2=0x55b37c4a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49f210 0x55b37c4a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.407+0000 7fa0577fe640 1 -- 172.21.15.64:0/2887703930 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37c4aa0d0 con 0x55b37c49a0a0 2024-05-09T09:22:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.408+0000 7fa0577fe640 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 0x55b37c4a81c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa048002230 tx=0x7fa048004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d6c0a714ac80b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.408+0000 7fa036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.408+0000 7fa036ffd640 1 -- 172.21.15.64:0/2887703930 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048019430 con 0x55b37c49a0a0 2024-05-09T09:22:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.408+0000 7fa036ffd640 1 -- 172.21.15.64:0/2887703930 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa048002030 con 0x55b37c49a0a0 2024-05-09T09:22:02.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.408+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa04803b070 con 0x55b37c49a0a0 2024-05-09T09:22:02.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.411+0000 7fa036ffd640 1 -- 172.21.15.64:0/2887703930 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa04804b050 con 0x55b37c49a0a0 2024-05-09T09:22:02.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.411+0000 7fa0609c4740 10 client.28255 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:02.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.411+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b37c515dc0 con 0x55b37c49a0a0 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.412+0000 7fa036ffd640 1 -- 172.21.15.64:0/2887703930 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa048004e60 con 0x55b37c49a0a0 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa036ffd640 1 -- 172.21.15.64:0/2887703930 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fa048003970 con 0x55b37c49a0a0 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa0609c4740 10 client.28255 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa0609c4740 10 client.28255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa0609c4740 20 client.28255 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c5166c0 0x55b37c518ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa0609c4740 4 client.28255 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa04803b070 con 0x55b37c5166c0 2024-05-09T09:22:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.413+0000 7fa056ffd640 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c5166c0 0x55b37c518ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.414+0000 7fa036ffd640 10 client.28255 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.415+0000 7fa056ffd640 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c5166c0 0x55b37c518ae0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d259b528b5a6c7c server_cookie=fe3f02cebcaa93c4 in_seq=0 out_seq=0 2024-05-09T09:22:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.416+0000 7fa036ffd640 1 -- 172.21.15.64:0/2887703930 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa04803b070 con 0x55b37c5166c0 2024-05-09T09:22:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.416+0000 7fa036ffd640 10 client.28255 handle_command_reply: tid=0 2024-05-09T09:22:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.416+0000 7fa0609c4740 1 client.28255 shutdown 2024-05-09T09:22:02.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.420+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 msgr2=0x55b37c4a81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.420+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 0x55b37c4a81c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa048002230 tx=0x7fa048004980 comp rx=0 tx=0).stop 2024-05-09T09:22:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 shutdown_connections 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b37c5166c0 0x55b37c518ae0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b37c4a9710 0x55b37c4a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b37c49f210 0x55b37c4a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa0609c4740 1 --2- 172.21.15.64:0/2887703930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b37c49a0a0 0x55b37c4a81c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 >> 172.21.15.64:0/2887703930 conn(0x55b37c4610a0 msgr2=0x55b37c497df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.422+0000 7fa036ffd640 0 client.28255 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.423+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 shutdown_connections 2024-05-09T09:22:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.424+0000 7fa0609c4740 1 -- 172.21.15.64:0/2887703930 wait complete. 2024-05-09T09:22:02.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.425+0000 7fa0609c4740 20 client.28255 trim_cache size 0 max 16384 2024-05-09T09:22:02.424 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:22:02.424 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:22:02.425 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "e5f200de-7f32-47ae-b267-b2564d6259aa", 2024-05-09T09:22:02.425 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:22:02.425 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:22:02.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0400a2d60 msgr2=0x7fa0400a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0400a2d60 0x7fa0400a5220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa04c02a910 tx=0x7fa04c005d50 comp rx=0 tx=0).stop 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 msgr2=0x7fa0581a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 0x7fa0581a5530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e970 tx=0x7fa05000ed70 comp rx=0 tx=0).stop 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 shutdown_connections 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa0400a2d60 0x7fa0400a5220 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa058109920 0x7fa0581a5530 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa058108f60 0x7fa05819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 --2- 172.21.15.64:0/49138237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa058107d70 0x7fa05819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 >> 172.21.15.64:0/49138237 conn(0x7fa058076d00 msgr2=0x7fa05810c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 shutdown_connections 2024-05-09T09:22:02.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.426+0000 7fa05c973640 1 -- 172.21.15.64:0/49138237 wait complete. 2024-05-09T09:22:02.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:22:02.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 -- 172.21.15.64:0/1060893190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c102a70 msgr2=0x7fe18c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 --2- 172.21.15.64:0/1060893190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c102a70 0x7fe18c102ef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe180009860 tx=0x7fe18002a360 comp rx=0 tx=0).stop 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 -- 172.21.15.64:0/1060893190 shutdown_connections 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 --2- 172.21.15.64:0/1060893190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe18c103430 0x7fe18c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 --2- 172.21.15.64:0/1060893190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c102a70 0x7fe18c102ef0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 --2- 172.21.15.64:0/1060893190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c101880 0x7fe18c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 -- 172.21.15.64:0/1060893190 >> 172.21.15.64:0/1060893190 conn(0x7fe18c0fce10 msgr2=0x7fe18c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 -- 172.21.15.64:0/1060893190 shutdown_connections 2024-05-09T09:22:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.647+0000 7fe19302c640 1 -- 172.21.15.64:0/1060893190 wait complete. 2024-05-09T09:22:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 Processor -- start 2024-05-09T09:22:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 -- start start 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe18c101880 0x7fe18c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c103430 0x7fe18c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 0x7fe18c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10d390 con 0x7fe18c101880 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10d210 con 0x7fe18c103430 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.648+0000 7fe19302c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10d510 con 0x7fe18c19d270 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c103430 0x7fe18c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 0x7fe18c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe18c101880 0x7fe18c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 0x7fe18c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52266/0 (socket says 172.21.15.64:52266) 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c103430 0x7fe18c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36330/0 (socket says 172.21.15.64:36330) 2024-05-09T09:22:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 -- 172.21.15.64:0/2289200591 learned_addr learned my addr 172.21.15.64:0/2289200591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 -- 172.21.15.64:0/2289200591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c103430 msgr2=0x7fe18c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c103430 0x7fe18c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 -- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe18c101880 msgr2=0x7fe18c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe18c101880 0x7fe18c197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 -- 172.21.15.64:0/2289200591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c19fd00 con 0x7fe18c19d270 2024-05-09T09:22:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.649+0000 7fe1915c9640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 0x7fe18c19f660 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe18400eb10 tx=0x7fe18400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.650+0000 7fe189ffb640 1 -- 172.21.15.64:0/2289200591 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184004b20 con 0x7fe18c19d270 2024-05-09T09:22:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.650+0000 7fe189ffb640 1 -- 172.21.15.64:0/2289200591 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe184004cc0 con 0x7fe18c19d270 2024-05-09T09:22:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.651+0000 7fe189ffb640 1 -- 172.21.15.64:0/2289200591 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184005570 con 0x7fe18c19d270 2024-05-09T09:22:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.651+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c073ac0 con 0x7fe18c19d270 2024-05-09T09:22:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.651+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c073fc0 con 0x7fe18c19d270 2024-05-09T09:22:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.655+0000 7fe195e10740 1 Processor -- start 2024-05-09T09:22:02.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.656+0000 7fe189ffb640 1 -- 172.21.15.64:0/2289200591 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe184022020 con 0x7fe18c19d270 2024-05-09T09:22:02.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.657+0000 7fe189ffb640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1740a2d60 0x7fe1740a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.658+0000 7fe189ffb640 1 -- 172.21.15.64:0/2289200591 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1840cf4f0 con 0x7fe18c19d270 2024-05-09T09:22:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.658+0000 7fe195e10740 1 -- start start 2024-05-09T09:22:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe195e10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f539fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe195e10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe195e10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe195e10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5584f52fbd00 con 0x5584f53a0130 2024-05-09T09:22:02.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe195e10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5584f52a34c0 con 0x5584f53a52a0 2024-05-09T09:22:02.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe195e10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5584f52f7cc0 con 0x5584f539f740 2024-05-09T09:22:02.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.659+0000 7fe18bfff640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1740a2d60 0x7fe1740a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.660+0000 7fe1915c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.660+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f539fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.660+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f539fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52274/0 (socket says 172.21.15.64:52274) 2024-05-09T09:22:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.660+0000 7fe190dc8640 1 -- 172.21.15.64:0/1645020464 learned_addr learned my addr 172.21.15.64:0/1645020464 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.664+0000 7fe18bfff640 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.665+0000 7fe190dc8640 1 -- 172.21.15.64:0/1645020464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 msgr2=0x5584f53a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.665+0000 7fe190dc8640 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.665+0000 7fe190dc8640 1 -- 172.21.15.64:0/1645020464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 msgr2=0x5584f53a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.665+0000 7fe190dc8640 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.665+0000 7fe190dc8640 1 -- 172.21.15.64:0/1645020464 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f50d97d0 con 0x5584f539f740 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.665+0000 7fe190dc8640 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f539fb60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00dc60 tx=0x7fe17c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2ca565918c68af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.666+0000 7fe172ffd640 1 -- 172.21.15.64:0/1645020464 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c005020 con 0x5584f539f740 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.666+0000 7fe172ffd640 1 -- 172.21.15.64:0/1645020464 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe17c0047a0 con 0x5584f539f740 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.666+0000 7fe172ffd640 1 -- 172.21.15.64:0/1645020464 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0116a0 con 0x5584f539f740 2024-05-09T09:22:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.667+0000 7fe195e10740 1 -- 172.21.15.64:0/1645020464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 msgr2=0x5584f539fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.667+0000 7fe195e10740 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f539fb60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00dc60 tx=0x7fe17c00f730 comp rx=0 tx=0).stop 2024-05-09T09:22:02.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.669+0000 7fe18bfff640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1740a2d60 0x7fe1740a5220 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe180035040 tx=0x7fe1800044f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.670+0000 7fe195e10740 1 -- 172.21.15.64:0/1645020464 shutdown_connections 2024-05-09T09:22:02.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.670+0000 7fe195e10740 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.670+0000 7fe195e10740 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.670+0000 7fe195e10740 1 --2- 172.21.15.64:0/1645020464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f539fb60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.670+0000 7fe195e10740 1 -- 172.21.15.64:0/1645020464 >> 172.21.15.64:0/1645020464 conn(0x5584f539adb0 msgr2=0x5584f539b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.672+0000 7fe195e10740 1 -- 172.21.15.64:0/1645020464 shutdown_connections 2024-05-09T09:22:02.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.672+0000 7fe195e10740 1 -- 172.21.15.64:0/1645020464 wait complete. 2024-05-09T09:22:02.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.672+0000 7fe195e10740 1 -- start start 2024-05-09T09:22:02.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.672+0000 7fe195e10740 1 Processor -- start 2024-05-09T09:22:02.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 0x5584f53bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a52a0 0x5584f53c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5584f53b9260 con 0x5584f53a0130 2024-05-09T09:22:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5584f52b6b90 con 0x5584f539f740 2024-05-09T09:22:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5584f53b93e0 con 0x5584f53a52a0 2024-05-09T09:22:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe195e10740 10 client.0 _ll_register_callbacks cb 0x5584f52f2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 0x5584f53bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 0x5584f53bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36346/0 (socket says 172.21.15.64:36346) 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.673+0000 7fe190dc8640 1 -- 172.21.15.64:0/444383975 learned_addr learned my addr 172.21.15.64:0/444383975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe1915c9640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a52a0 0x5584f53c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe18bfff640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe190dc8640 1 -- 172.21.15.64:0/444383975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a52a0 msgr2=0x5584f53c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe190dc8640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a52a0 0x5584f53c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe190dc8640 1 -- 172.21.15.64:0/444383975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 msgr2=0x5584f53bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe190dc8640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.674+0000 7fe190dc8640 1 -- 172.21.15.64:0/444383975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f500f3f0 con 0x5584f539f740 2024-05-09T09:22:02.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe190dc8640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 0x5584f53bac10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe17c004f80 tx=0x7fe17c004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8134784f32f7ba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe16bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:02.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe16bfff640 1 -- 172.21.15.64:0/444383975 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c00dd50 con 0x5584f539f740 2024-05-09T09:22:02.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17c0047a0 con 0x5584f539f740 2024-05-09T09:22:02.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe16bfff640 1 -- 172.21.15.64:0/444383975 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe17c004040 con 0x5584f539f740 2024-05-09T09:22:02.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe16bfff640 1 -- 172.21.15.64:0/444383975 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe17c02f050 con 0x5584f539f740 2024-05-09T09:22:02.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe195e10740 10 client.28109 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:02.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.675+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5584f4fcf0f0 con 0x5584f539f740 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe16bfff640 1 -- 172.21.15.64:0/444383975 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe17c0047a0 con 0x5584f539f740 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe16bfff640 1 -- 172.21.15.64:0/444383975 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fe17c003780 con 0x5584f539f740 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe195e10740 10 client.28109 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe195e10740 10 client.28109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe195e10740 20 client.28109 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe195e10740 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f53c6de0 0x5584f53c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe195e10740 4 client.28109 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584f5309430 con 0x5584f53c6de0 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.676+0000 7fe18bfff640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f53c6de0 0x5584f53c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.677+0000 7fe18bfff640 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f53c6de0 0x5584f53c9260 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f07379d89a8f76 server_cookie=cf53c3a44b572035 in_seq=0 out_seq=0 2024-05-09T09:22:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.677+0000 7fe16bfff640 10 client.28109 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe16bfff640 1 -- 172.21.15.64:0/444383975 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584f5309430 con 0x5584f53c6de0 2024-05-09T09:22:02.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe16bfff640 10 client.28109 handle_command_reply: tid=0 2024-05-09T09:22:02.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 client.28109 shutdown 2024-05-09T09:22:02.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 msgr2=0x5584f53bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 0x5584f53bac10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe17c004f80 tx=0x7fe17c004b80 comp rx=0 tx=0).stop 2024-05-09T09:22:02.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 shutdown_connections 2024-05-09T09:22:02.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f53c6de0 0x5584f53c9260 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a52a0 0x5584f53c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a0130 0x5584f53bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 --2- 172.21.15.64:0/444383975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f539f740 0x5584f53bac10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.679+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 >> 172.21.15.64:0/444383975 conn(0x5584f539adb0 msgr2=0x5584f53917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.680+0000 7fe16bfff640 0 client.28109 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.680+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 shutdown_connections 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.680+0000 7fe195e10740 1 -- 172.21.15.64:0/444383975 wait complete. 2024-05-09T09:22:02.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.680+0000 7fe195e10740 20 client.28109 trim_cache size 0 max 16384 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.714+0000 7fe195e10740 1 Processor -- start 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.714+0000 7fe195e10740 1 -- start start 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe195e10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f538d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe195e10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a0130 0x5584f53c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe195e10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 0x5584f5391660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe195e10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1a2980 con 0x5584f53a52a0 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe195e10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1a0220 con 0x5584f53a0130 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe195e10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c071c60 con 0x5584f539f740 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe1915c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 0x5584f5391660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a0130 0x5584f53c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f538d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe1915c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 0x5584f5391660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51120/0 (socket says 172.21.15.64:51120) 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a0130 0x5584f53c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36362/0 (socket says 172.21.15.64:36362) 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.715+0000 7fe1915c9640 1 -- 172.21.15.64:0/1830900047 learned_addr learned my addr 172.21.15.64:0/1830900047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1915c9640 1 -- 172.21.15.64:0/1830900047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 msgr2=0x5584f538d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1915c9640 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f538d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1915c9640 1 -- 172.21.15.64:0/1830900047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a0130 msgr2=0x5584f53c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1915c9640 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a0130 0x5584f53c20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1915c9640 1 -- 172.21.15.64:0/1830900047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f539ba80 con 0x5584f53a52a0 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1915c9640 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 0x5584f5391660 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe18409fcf0 tx=0x7fe184013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814f714607efe06b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1727fc640 1 -- 172.21.15.64:0/1830900047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840b4070 con 0x5584f53a52a0 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1727fc640 1 -- 172.21.15.64:0/1830900047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1840a0460 con 0x5584f53a52a0 2024-05-09T09:22:02.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.716+0000 7fe1727fc640 1 -- 172.21.15.64:0/1830900047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840cf140 con 0x5584f53a52a0 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 -- 172.21.15.64:0/1830900047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 msgr2=0x5584f5391660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 0x5584f5391660 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe18409fcf0 tx=0x7fe184013f50 comp rx=0 tx=0).stop 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 -- 172.21.15.64:0/1830900047 shutdown_connections 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f53a52a0 0x5584f5391660 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a0130 0x5584f53c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 --2- 172.21.15.64:0/1830900047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f539f740 0x5584f538d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 -- 172.21.15.64:0/1830900047 >> 172.21.15.64:0/1830900047 conn(0x5584f539adb0 msgr2=0x5584f5390f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 -- 172.21.15.64:0/1830900047 shutdown_connections 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.718+0000 7fe195e10740 1 -- 172.21.15.64:0/1830900047 wait complete. 2024-05-09T09:22:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.719+0000 7fe195e10740 1 -- start start 2024-05-09T09:22:02.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.719+0000 7fe195e10740 1 Processor -- start 2024-05-09T09:22:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.720+0000 7fe195e10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 0x5584f53bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.720+0000 7fe195e10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 0x5584f53bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.720+0000 7fe195e10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.720+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 0x5584f53bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.720+0000 7fe190dc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 0x5584f53bf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51130/0 (socket says 172.21.15.64:51130) 2024-05-09T09:22:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.720+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 0x5584f53bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 0x5584f53bf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52298/0 (socket says 172.21.15.64:52298) 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe190dc8640 1 -- 172.21.15.64:0/2929775999 learned_addr learned my addr 172.21.15.64:0/2929775999 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c0722d0 con 0x5584f539f740 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c073740 con 0x5584f53a52a0 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c0729a0 con 0x5584f53a0130 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe195e10740 10 client.0 _ll_register_callbacks cb 0x7fe17c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe1915c9640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe190dc8640 1 -- 172.21.15.64:0/2929775999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 msgr2=0x5584f53bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe190dc8640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 0x5584f53bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe190dc8640 1 -- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 msgr2=0x5584f53c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe190dc8640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe190dc8640 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f53c0cf0 con 0x5584f539f740 2024-05-09T09:22:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.721+0000 7fe18bfff640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 0x5584f53bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.722+0000 7fe190dc8640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 0x5584f53bf1b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0059d0 tx=0x7fe17c00d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544b851982f1e2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.722+0000 7fe16bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.722+0000 7fe16bfff640 1 -- 172.21.15.64:0/2929775999 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c02c040 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.722+0000 7fe16bfff640 1 -- 172.21.15.64:0/2929775999 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe17c022070 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.723+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1840a0460 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.723+0000 7fe16bfff640 1 -- 172.21.15.64:0/2929775999 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe17c00d330 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.723+0000 7fe195e10740 10 client.18642 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.723+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5584f53c1fe0 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe16bfff640 1 -- 172.21.15.64:0/2929775999 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1840a0460 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe16bfff640 1 -- 172.21.15.64:0/2929775999 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fe17c003dc0 con 0x5584f539f740 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe195e10740 10 client.18642 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe195e10740 10 client.18642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:02.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe195e10740 20 client.18642 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:02.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe195e10740 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f542d7f0 0x5584f542fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:02.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe195e10740 4 client.18642 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:22:02.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584f5303780 con 0x5584f542d7f0 2024-05-09T09:22:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.724+0000 7fe18bfff640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f542d7f0 0x5584f542fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe16bfff640 10 client.18642 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe18bfff640 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f542d7f0 0x5584f542fc10 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c52ece7df68c7e server_cookie=d0499120da8b3cae in_seq=0 out_seq=0 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe16bfff640 1 -- 172.21.15.64:0/2929775999 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5584f5303780 con 0x5584f542d7f0 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe16bfff640 10 client.18642 handle_command_reply: tid=0 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe195e10740 1 client.18642 shutdown 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 msgr2=0x5584f53bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.725+0000 7fe195e10740 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 0x5584f53bf1b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0059d0 tx=0x7fe17c00d830 comp rx=0 tx=0).stop 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 shutdown_connections 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5584f542d7f0 0x5584f542fc10 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584f53a52a0 0x5584f53c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584f53a0130 0x5584f53bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe16bfff640 0 client.18642 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 --2- 172.21.15.64:0/2929775999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584f539f740 0x5584f53bf1b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 >> 172.21.15.64:0/2929775999 conn(0x5584f539adb0 msgr2=0x5584f53917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 shutdown_connections 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 1 -- 172.21.15.64:0/2929775999 wait complete. 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.726+0000 7fe195e10740 20 client.18642 trim_cache size 0 max 16384 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "e5f200de-7f32-47ae-b267-b2564d6259aa": { 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "e5f200de-7f32-47ae-b267-b2564d6259aa", 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:22:02.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:22:02.729 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:22:02.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1740a2d60 msgr2=0x7fe1740a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1740a2d60 0x7fe1740a5220 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe180035040 tx=0x7fe1800044f0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 msgr2=0x7fe18c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:02.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 0x7fe18c19f660 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe18400eb10 tx=0x7fe18400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 shutdown_connections 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1740a2d60 0x7fe1740a5220 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c19d270 0x7fe18c19f660 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c103430 0x7fe18c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 --2- 172.21.15.64:0/2289200591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe18c101880 0x7fe18c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 >> 172.21.15.64:0/2289200591 conn(0x7fe18c0fce10 msgr2=0x7fe18c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.727+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 shutdown_connections 2024-05-09T09:22:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:02.728+0000 7fe19302c640 1 -- 172.21.15.64:0/2289200591 wait complete. 2024-05-09T09:22:02.749 INFO:tasks.cephfs.filesystem:scrub status for tag:e5f200de-7f32-47ae-b267-b2564d6259aa - {'path': '/', 'tag': 'e5f200de-7f32-47ae-b267-b2564d6259aa', 'options': 'recursive,force'} 2024-05-09T09:22:02.749 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:02 smithi064 ceph-mon[30149]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 669 op/s 2024-05-09T09:22:02.749 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:02 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:22:02.749 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:02 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:02 smithi179 ceph-mon[34086]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 669 op/s 2024-05-09T09:22:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:02 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:22:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:02 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:02 smithi139 ceph-mon[33599]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 669 op/s 2024-05-09T09:22:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:02 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:22:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:02 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 -- 172.21.15.64:0/2982614875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc102aa0 msgr2=0x7f8bbc102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/2982614875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc102aa0 0x7f8bbc102f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009900 tx=0x7f8bac02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 -- 172.21.15.64:0/2982614875 shutdown_connections 2024-05-09T09:22:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/2982614875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/2982614875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc102aa0 0x7f8bbc102f20 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009900 tx=0x7f8bac02a3f0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/2982614875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bbc1018b0 0x7f8bbc101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.944+0000 7f8bc32ae640 1 -- 172.21.15.64:0/2982614875 >> 172.21.15.64:0/2982614875 conn(0x7f8bbc0fce20 msgr2=0x7f8bbc0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.946+0000 7f8bc32ae640 1 -- 172.21.15.64:0/2982614875 shutdown_connections 2024-05-09T09:22:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.946+0000 7f8bc32ae640 1 -- 172.21.15.64:0/2982614875 wait complete. 2024-05-09T09:22:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.947+0000 7f8bc32ae640 1 Processor -- start 2024-05-09T09:22:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.947+0000 7f8bc32ae640 1 -- start start 2024-05-09T09:22:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.948+0000 7f8bc32ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bbc1018b0 0x7f8bbc19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.948+0000 7f8bc32ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc32ae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc1a37f0 0x7f8bbc1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc32ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10d410 con 0x7f8bbc103460 2024-05-09T09:22:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc32ae640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10d290 con 0x7f8bbc1a37f0 2024-05-09T09:22:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc32ae640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10d590 con 0x7f8bbc1018b0 2024-05-09T09:22:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc0849640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc0849640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51144/0 (socket says 172.21.15.64:51144) 2024-05-09T09:22:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.949+0000 7f8bc0849640 1 -- 172.21.15.64:0/1331205167 learned_addr learned my addr 172.21.15.64:0/1331205167 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.950+0000 7f8bc0849640 1 -- 172.21.15.64:0/1331205167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bbc1018b0 msgr2=0x7f8bbc19e250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:22:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.950+0000 7f8bc0849640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bbc1018b0 0x7f8bbc19e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.950+0000 7f8bc0849640 1 -- 172.21.15.64:0/1331205167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc1a37f0 msgr2=0x7f8bbc1a5be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:22:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.950+0000 7f8bc0849640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc1a37f0 0x7f8bbc1a5be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.950+0000 7f8bc0849640 1 -- 172.21.15.64:0/1331205167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc1a6250 con 0x7f8bbc103460 2024-05-09T09:22:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.950+0000 7f8bc0849640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc19e790 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005ab0 tx=0x7f8bac003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.951+0000 7f8bb27fc640 1 -- 172.21.15.64:0/1331205167 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac03f020 con 0x7f8bbc103460 2024-05-09T09:22:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.951+0000 7f8bb27fc640 1 -- 172.21.15.64:0/1331205167 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bac031c30 con 0x7f8bbc103460 2024-05-09T09:22:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.951+0000 7f8bb27fc640 1 -- 172.21.15.64:0/1331205167 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac035070 con 0x7f8bbc103460 2024-05-09T09:22:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.951+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc1a64e0 con 0x7f8bbc103460 2024-05-09T09:22:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.952+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc1a68c0 con 0x7f8bbc103460 2024-05-09T09:22:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.956+0000 7f8bc5fb7740 1 Processor -- start 2024-05-09T09:22:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.957+0000 7f8bb27fc640 1 -- 172.21.15.64:0/1331205167 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8bac042020 con 0x7f8bbc103460 2024-05-09T09:22:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.958+0000 7f8bb27fc640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8ba80a2e20 0x7f8ba80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.959+0000 7f8bb27fc640 1 -- 172.21.15.64:0/1331205167 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bac0efc70 con 0x7f8bbc103460 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 -- start start 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8250 0x563bc6ea8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8c40 0x563bc6ead740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6eb0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bc6e0c340 con 0x563bc6eaddb0 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563bc6db4290 con 0x563bc6ea8c40 2024-05-09T09:22:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.962+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563bc6e08300 con 0x563bc6ea8250 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc104a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8250 0x563bc6ea8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc104a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8250 0x563bc6ea8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52322/0 (socket says 172.21.15.64:52322) 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc104a640 1 -- 172.21.15.64:0/2618703722 learned_addr learned my addr 172.21.15.64:0/2618703722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6eb0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8ba80a2e20 0x7f8ba80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 -- 172.21.15.64:0/2618703722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8250 msgr2=0x563bc6ea8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8250 0x563bc6ea8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 -- 172.21.15.64:0/2618703722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8c40 msgr2=0x563bc6ead740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8c40 0x563bc6ead740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.963+0000 7f8bc184b640 1 -- 172.21.15.64:0/2618703722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc6dd88e0 con 0x563bc6eaddb0 2024-05-09T09:22:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.964+0000 7f8bc184b640 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6eb0240 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4013540 tx=0x7f8bb4013a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f843653b404c867e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.964+0000 7f8bc184b640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8ba80a2e20 0x7f8ba80a52e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4020000 tx=0x7f8bb4019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.965+0000 7f8ba77fe640 1 -- 172.21.15.64:0/2618703722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4023070 con 0x563bc6eaddb0 2024-05-09T09:22:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.965+0000 7f8ba77fe640 1 -- 172.21.15.64:0/2618703722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bb4010d60 con 0x563bc6eaddb0 2024-05-09T09:22:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.965+0000 7f8ba77fe640 1 -- 172.21.15.64:0/2618703722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400ec50 con 0x563bc6eaddb0 2024-05-09T09:22:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.968+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/2618703722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 msgr2=0x563bc6eb0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.968+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6eb0240 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4013540 tx=0x7f8bb4013a00 comp rx=0 tx=0).stop 2024-05-09T09:22:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.974+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/2618703722 shutdown_connections 2024-05-09T09:22:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.974+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6eb0240 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.974+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8c40 0x563bc6ead740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.974+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/2618703722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8250 0x563bc6ea8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.974+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/2618703722 >> 172.21.15.64:0/2618703722 conn(0x563bc6ea38c0 msgr2=0x563bc6ea3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:03.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:03 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:22:03.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:03 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:03.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:03 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:22:03.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:03 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:22:03.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:03 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:03.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:03 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.981+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/2618703722 shutdown_connections 2024-05-09T09:22:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.981+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/2618703722 wait complete. 2024-05-09T09:22:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 -- start start 2024-05-09T09:22:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 Processor -- start 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8250 0x563bc6ec35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6eaddb0 0x563bc6ecab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bc6ec1d70 con 0x563bc6ea8250 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563bc6dc7790 con 0x563bc6eaddb0 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.982+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563bc6ec1ef0 con 0x563bc6ea8c40 2024-05-09T09:22:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc5fb7740 10 client.0 _ll_register_callbacks cb 0x563bc6e02a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52336/0 (socket says 172.21.15.64:52336) 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 -- 172.21.15.64:0/132093418 learned_addr learned my addr 172.21.15.64:0/132093418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 -- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6eaddb0 msgr2=0x563bc6ecab70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6eaddb0 0x563bc6ecab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 -- 172.21.15.64:0/132093418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8250 msgr2=0x563bc6ec35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8250 0x563bc6ec35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.983+0000 7f8bc0849640 1 -- 172.21.15.64:0/132093418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc6b20210 con 0x563bc6ea8c40 2024-05-09T09:22:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.984+0000 7f8bc0849640 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec3b30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0b8540 tx=0x7f8bac0bbce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afca5b9b5d61cef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.984+0000 7f8ba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.984+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/132093418 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0bc7e0 con 0x563bc6ea8c40 2024-05-09T09:22:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.984+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/132093418 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bac0c0120 con 0x563bc6ea8c40 2024-05-09T09:22:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.987+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb4010d60 con 0x563bc6ea8c40 2024-05-09T09:22:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.990+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/132093418 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8bac0bc0e0 con 0x563bc6ea8c40 2024-05-09T09:22:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.991+0000 7f8bc5fb7740 10 client.28282 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.991+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563bc6bec620 con 0x563bc6ea8c40 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.991+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/132093418 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bb4010d60 con 0x563bc6ea8c40 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/132093418 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f8bac03b460 con 0x563bc6ea8c40 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8bc5fb7740 10 client.28282 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8bc5fb7740 10 client.28282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8bc5fb7740 20 client.28282 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6ecf340 0x563bc6ed17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8bc5fb7740 4 client.28282 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.992+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc6e19a70 con 0x563bc6ecf340 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.993+0000 7f8bc104a640 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6ecf340 0x563bc6ed17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.993+0000 7f8bc104a640 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6ecf340 0x563bc6ed17a0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5948f47a2432bd18 server_cookie=5b6da08286d4919a in_seq=0 out_seq=0 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.993+0000 7f8ba4ff9640 10 client.28282 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/132093418 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bc6e19a70 con 0x563bc6ecf340 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8ba4ff9640 10 client.28282 handle_command_reply: tid=0 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 client.28282 shutdown 2024-05-09T09:22:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 msgr2=0x563bc6ec3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec3b30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0b8540 tx=0x7f8bac0bbce0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 shutdown_connections 2024-05-09T09:22:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6ecf340 0x563bc6ed17a0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6eaddb0 0x563bc6ecab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec3b30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/132093418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8250 0x563bc6ec35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8ba4ff9640 0 client.28282 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 >> 172.21.15.64:0/132093418 conn(0x563bc6ea38c0 msgr2=0x563bc6ea2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 shutdown_connections 2024-05-09T09:22:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.994+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/132093418 wait complete. 2024-05-09T09:22:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:03.995+0000 7f8bc5fb7740 20 client.28282 trim_cache size 0 max 16384 2024-05-09T09:22:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.026+0000 7f8bc5fb7740 1 Processor -- start 2024-05-09T09:22:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.027+0000 7f8bc5fb7740 1 -- start start 2024-05-09T09:22:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.027+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6eaa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.027+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8c40 0x563bc6ecab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.027+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6eaddb0 0x563bc6eaa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6eaa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6eaa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36398/0 (socket says 172.21.15.64:36398) 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 -- 172.21.15.64:0/1910630369 learned_addr learned my addr 172.21.15.64:0/1910630369 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc1ab060 con 0x563bc6ea8c40 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc1a8900 con 0x563bc6ea8250 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc073630 con 0x563bc6eaddb0 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc0849640 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8c40 0x563bc6ecab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc184b640 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6eaddb0 0x563bc6eaa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 -- 172.21.15.64:0/1910630369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6eaddb0 msgr2=0x563bc6eaa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6eaddb0 0x563bc6eaa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 -- 172.21.15.64:0/1910630369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8c40 msgr2=0x563bc6ecab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.028+0000 7f8bc104a640 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8c40 0x563bc6ecab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.029+0000 7f8bc104a640 1 -- 172.21.15.64:0/1910630369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc6ea4310 con 0x563bc6ea8250 2024-05-09T09:22:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.029+0000 7f8bc104a640 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6eaa000 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009ea0 tx=0x7f8bb8009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8c3c5cef738e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.029+0000 7f8ba6ffd640 1 -- 172.21.15.64:0/1910630369 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8019310 con 0x563bc6ea8250 2024-05-09T09:22:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.029+0000 7f8ba6ffd640 1 -- 172.21.15.64:0/1910630369 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bb8005100 con 0x563bc6ea8250 2024-05-09T09:22:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.029+0000 7f8ba6ffd640 1 -- 172.21.15.64:0/1910630369 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80126b0 con 0x563bc6ea8250 2024-05-09T09:22:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 msgr2=0x563bc6eaa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6eaa000 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009ea0 tx=0x7f8bb8009a00 comp rx=0 tx=0).stop 2024-05-09T09:22:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 shutdown_connections 2024-05-09T09:22:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6eaddb0 0x563bc6eaa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6ea8c40 0x563bc6ecab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/1910630369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6eaa000 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.030+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 >> 172.21.15.64:0/1910630369 conn(0x563bc6ea38c0 msgr2=0x563bc6ec35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.031+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 shutdown_connections 2024-05-09T09:22:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.031+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/1910630369 wait complete. 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.031+0000 7f8bc5fb7740 1 -- start start 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.031+0000 7f8bc5fb7740 1 Processor -- start 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6ec7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6ec9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc1a6dc0 con 0x563bc6eaddb0 2024-05-09T09:22:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc1a8230 con 0x563bc6ea8250 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc1a7490 con 0x563bc6ea8c40 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc5fb7740 10 client.0 _ll_register_callbacks cb 0x7f8bac005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc104a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6ec7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc104a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6ec7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:36402/0 (socket says 172.21.15.64:36402) 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc104a640 1 -- 172.21.15.64:0/493449645 learned_addr learned my addr 172.21.15.64:0/493449645 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52362/0 (socket says 172.21.15.64:52362) 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc184b640 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6ec9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 -- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 msgr2=0x563bc6ec7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6ec7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 -- 172.21.15.64:0/493449645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 msgr2=0x563bc6ec9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6ec9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.032+0000 7f8bc0849640 1 -- 172.21.15.64:0/493449645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc6ec9860 con 0x563bc6ea8c40 2024-05-09T09:22:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8bc0849640 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec8260 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006010 tx=0x7f8bac0bbd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312c9a1f09c31fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8ba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/493449645 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0bbeb0 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb8005100 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/493449645 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bac0c48c0 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/493449645 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8bac0bc7e0 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8bc5fb7740 10 client.28288 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.033+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563bc6ecabe0 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.034+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/493449645 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bb8005100 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/493449645 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f8bac0bb530 con 0x563bc6ea8c40 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc5fb7740 10 client.28288 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc5fb7740 10 client.28288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc5fb7740 20 client.28288 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6f35d10 0x563bc6f38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc5fb7740 4 client.28288 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:22:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bc6e13dc0 con 0x563bc6f35d10 2024-05-09T09:22:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.035+0000 7f8bc104a640 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6f35d10 0x563bc6f38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.037+0000 7f8bc104a640 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6f35d10 0x563bc6f38130 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7234a225f2320e60 server_cookie=db9300df31f410d3 in_seq=0 out_seq=0 2024-05-09T09:22:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.037+0000 7f8ba4ff9640 10 client.28288 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.054+0000 7f8ba4ff9640 1 -- 172.21.15.64:0/493449645 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bc6e13dc0 con 0x563bc6f35d10 2024-05-09T09:22:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.054+0000 7f8ba4ff9640 10 client.28288 handle_command_reply: tid=0 2024-05-09T09:22:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.055+0000 7f8bc5fb7740 1 client.28288 shutdown 2024-05-09T09:22:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.056+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 msgr2=0x563bc6ec8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.056+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec8260 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006010 tx=0x7f8bac0bbd50 comp rx=0 tx=0).stop 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.058+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 shutdown_connections 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.058+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563bc6f35d10 0x563bc6f38130 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.058+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bc6eaddb0 0x563bc6ec9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.058+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563bc6ea8c40 0x563bc6ec8260 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.058+0000 7f8bc5fb7740 1 --2- 172.21.15.64:0/493449645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563bc6ea8250 0x563bc6ec7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.058+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 >> 172.21.15.64:0/493449645 conn(0x563bc6ea38c0 msgr2=0x563bc6ea2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.059+0000 7f8ba4ff9640 0 client.28288 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.059+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 shutdown_connections 2024-05-09T09:22:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.059+0000 7f8bc5fb7740 1 -- 172.21.15.64:0/493449645 wait complete. 2024-05-09T09:22:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.060+0000 7f8bc5fb7740 20 client.28288 trim_cache size 0 max 16384 2024-05-09T09:22:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:22:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:22:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:22:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:22:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.062+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8ba80a2e20 msgr2=0x7f8ba80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.062+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8ba80a2e20 0x7f8ba80a52e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4020000 tx=0x7f8bb4019000 comp rx=0 tx=0).stop 2024-05-09T09:22:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.062+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 msgr2=0x7f8bbc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.062+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc19e790 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005ab0 tx=0x7f8bac003b90 comp rx=0 tx=0).stop 2024-05-09T09:22:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 shutdown_connections 2024-05-09T09:22:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8ba80a2e20 0x7f8ba80a52e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bbc1a37f0 0x7f8bbc1a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bbc103460 0x7f8bbc19e790 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 --2- 172.21.15.64:0/1331205167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bbc1018b0 0x7f8bbc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 >> 172.21.15.64:0/1331205167 conn(0x7f8bbc0fce20 msgr2=0x7f8bbc103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.064+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 shutdown_connections 2024-05-09T09:22:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:04.065+0000 7f8bc32ae640 1 -- 172.21.15.64:0/1331205167 wait complete. 2024-05-09T09:22:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:03 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:22:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:03 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:03 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:22:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:03 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:22:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:03 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:04.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:03 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:04.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:03 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:22:04.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:03 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:04.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:03 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:22:04.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:03 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:22:04.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:03 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:04.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:03 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:04.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:04 smithi179 ceph-mon[34086]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 752 op/s 2024-05-09T09:22:05.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:04 smithi139 ceph-mon[33599]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 752 op/s 2024-05-09T09:22:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:04 smithi064 ceph-mon[30149]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 752 op/s 2024-05-09T09:22:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 705.0 s, 789.4 tps, lat 126.668 ms stddev 68.879 2024-05-09T09:22:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:06 smithi179 ceph-mon[34086]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-05-09T09:22:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:06 smithi139 ceph-mon[33599]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-05-09T09:22:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:06 smithi064 ceph-mon[30149]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.8 MiB/s wr, 807 op/s 2024-05-09T09:22:08.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:08 smithi179 ceph-mon[34086]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-05-09T09:22:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:08 smithi139 ceph-mon[33599]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-05-09T09:22:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:08 smithi064 ceph-mon[30149]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.1 MiB/s wr, 735 op/s 2024-05-09T09:22:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:10 smithi179 ceph-mon[34086]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 969 op/s 2024-05-09T09:22:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:10 smithi139 ceph-mon[33599]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 969 op/s 2024-05-09T09:22:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:10 smithi064 ceph-mon[30149]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 969 op/s 2024-05-09T09:22:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 710.0 s, 783.3 tps, lat 127.824 ms stddev 94.166 2024-05-09T09:22:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:12 smithi179 ceph-mon[34086]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.4 MiB/s wr, 763 op/s 2024-05-09T09:22:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:12 smithi139 ceph-mon[33599]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.4 MiB/s wr, 763 op/s 2024-05-09T09:22:13.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:12 smithi064 ceph-mon[30149]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.4 MiB/s wr, 763 op/s 2024-05-09T09:22:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:14 smithi179 ceph-mon[34086]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.2 MiB/s wr, 853 op/s 2024-05-09T09:22:15.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:14 smithi139 ceph-mon[33599]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.2 MiB/s wr, 853 op/s 2024-05-09T09:22:15.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:14 smithi064 ceph-mon[30149]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.2 MiB/s wr, 853 op/s 2024-05-09T09:22:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:15 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:22:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:15 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:22:15.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:15 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:22:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:15 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:22:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:15 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:22:16.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:15 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:22:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:15 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:22:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:15 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:22:16.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:15 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:22:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 715.0 s, 764.2 tps, lat 131.042 ms stddev 70.525 2024-05-09T09:22:17.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:16 smithi139 ceph-mon[33599]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.1 MiB/s wr, 819 op/s 2024-05-09T09:22:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:16 smithi064 ceph-mon[30149]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.1 MiB/s wr, 819 op/s 2024-05-09T09:22:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.222+0000 7f46a17f1640 1 -- 172.21.15.64:0/2994556629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c072b60 msgr2=0x7f469c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.222+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2994556629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c072b60 0x7f469c072f80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002a420 comp rx=0 tx=0).stop 2024-05-09T09:22:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.223+0000 7f46a17f1640 1 -- 172.21.15.64:0/2994556629 shutdown_connections 2024-05-09T09:22:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.223+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2994556629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f469c109800 0x7f469c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.223+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2994556629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f469c108e40 0x7f469c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.223+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2994556629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c072b60 0x7f469c072f80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.223+0000 7f46a17f1640 1 -- 172.21.15.64:0/2994556629 >> 172.21.15.64:0/2994556629 conn(0x7f469c06e6d0 msgr2=0x7f469c070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.224+0000 7f46a17f1640 1 -- 172.21.15.64:0/2994556629 shutdown_connections 2024-05-09T09:22:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.225+0000 7f46a17f1640 1 -- 172.21.15.64:0/2994556629 wait complete. 2024-05-09T09:22:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.226+0000 7f46a17f1640 1 Processor -- start 2024-05-09T09:22:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.226+0000 7f46a17f1640 1 -- start start 2024-05-09T09:22:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.226+0000 7f46a17f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f469c072b60 0x7f469c1a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.226+0000 7f46a17f1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 0x7f469c1a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f46a17f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f469c109800 0x7f469c1add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f46a17f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f469c11c0c0 con 0x7f469c109800 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f469b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 0x7f469c1a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f469b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 0x7f469c1a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36346/0 (socket says 172.21.15.64:36346) 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f469b7fe640 1 -- 172.21.15.64:0/2619933531 learned_addr learned my addr 172.21.15.64:0/2619933531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f469c109800 0x7f469c1add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f469bfff640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f469c072b60 0x7f469c1a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f469c11bf40 con 0x7f469c072b60 2024-05-09T09:22:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.227+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f469c11c240 con 0x7f469c108e40 2024-05-09T09:22:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.228+0000 7f469b7fe640 1 -- 172.21.15.64:0/2619933531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f469c072b60 msgr2=0x7f469c1a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.228+0000 7f469b7fe640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f469c072b60 0x7f469c1a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.228+0000 7f469b7fe640 1 -- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f469c109800 msgr2=0x7f469c1add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.228+0000 7f469b7fe640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f469c109800 0x7f469c1add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.228+0000 7f469b7fe640 1 -- 172.21.15.64:0/2619933531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c1ae3a0 con 0x7f469c108e40 2024-05-09T09:22:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.228+0000 7f469b7fe640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 0x7f469c1a6cc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b470 tx=0x7f468c00b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.229+0000 7f46997fa640 1 -- 172.21.15.64:0/2619933531 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c015020 con 0x7f469c108e40 2024-05-09T09:22:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.229+0000 7f46997fa640 1 -- 172.21.15.64:0/2619933531 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468c011d40 con 0x7f469c108e40 2024-05-09T09:22:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.229+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c1ae690 con 0x7f469c108e40 2024-05-09T09:22:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.229+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c1aec60 con 0x7f469c108e40 2024-05-09T09:22:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.229+0000 7f46997fa640 1 -- 172.21.15.64:0/2619933531 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c00f340 con 0x7f469c108e40 2024-05-09T09:22:17.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:16 smithi179 ceph-mon[34086]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.1 MiB/s wr, 819 op/s 2024-05-09T09:22:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.234+0000 7f46997fa640 1 -- 172.21.15.64:0/2619933531 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f468c018020 con 0x7f469c108e40 2024-05-09T09:22:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 Processor -- start 2024-05-09T09:22:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 -- start start 2024-05-09T09:22:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d2cf408340 con 0x55d2cf4a9f30 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d2cf3b0290 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f46a5835740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d2cf404300 con 0x55d2cf4a43d0 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.235+0000 7f469bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:48798/0 (socket says 172.21.15.64:48798) 2024-05-09T09:22:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 -- 172.21.15.64:0/722424354 learned_addr learned my addr 172.21.15.64:0/722424354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 -- 172.21.15.64:0/722424354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a43d0 msgr2=0x55d2cf4a47f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4a47f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 -- 172.21.15.64:0/722424354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 msgr2=0x55d2cf4ac3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4ac3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.236+0000 7f469bfff640 1 -- 172.21.15.64:0/722424354 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cf3d48e0 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f46997fa640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46840a2d90 0x7f46840a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f469bfff640 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4a98c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f4690005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777c849ac1db689d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f46997fa640 1 -- 172.21.15.64:0/2619933531 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f468c0cfd50 con 0x7f469c108e40 2024-05-09T09:22:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f46827fc640 1 -- 172.21.15.64:0/722424354 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690035030 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f46827fc640 1 -- 172.21.15.64:0/722424354 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4690040070 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f46827fc640 1 -- 172.21.15.64:0/722424354 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690031b80 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.237+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46840a2d90 0x7f46840a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.238+0000 7f46a5835740 1 -- 172.21.15.64:0/722424354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 msgr2=0x55d2cf4a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.238+0000 7f46a5835740 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4a98c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f4690005fd0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.241+0000 7f46a5835740 1 -- 172.21.15.64:0/722424354 shutdown_connections 2024-05-09T09:22:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.241+0000 7f46a5835740 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.241+0000 7f46a5835740 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4a98c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.241+0000 7f46a5835740 1 --2- 172.21.15.64:0/722424354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.241+0000 7f46a5835740 1 -- 172.21.15.64:0/722424354 >> 172.21.15.64:0/722424354 conn(0x55d2cf49fa40 msgr2=0x55d2cf49fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.244+0000 7f46a5835740 1 -- 172.21.15.64:0/722424354 shutdown_connections 2024-05-09T09:22:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.246+0000 7f46a5835740 1 -- 172.21.15.64:0/722424354 wait complete. 2024-05-09T09:22:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.246+0000 7f46a5835740 1 -- start start 2024-05-09T09:22:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.247+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46840a2d90 0x7f46840a5250 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4694005c40 tx=0x7f4694005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.247+0000 7f46a5835740 1 Processor -- start 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.248+0000 7f46a5835740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.248+0000 7f46a5835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.248+0000 7f46a5835740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a5835740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d2cf4ace80 con 0x55d2cf4a9f30 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a0ff0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a0ff0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54810/0 (socket says 172.21.15.64:54810) 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a0ff0640 1 -- 172.21.15.64:0/4172216175 learned_addr learned my addr 172.21.15.64:0/4172216175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d2cf3c3790 con 0x55d2cf4a43d0 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d2cf4ad000 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.249+0000 7f46a5835740 10 client.0 _ll_register_callbacks cb 0x55d2cf3fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f46a0ff0640 1 -- 172.21.15.64:0/4172216175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 msgr2=0x55d2cf4aebf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4aebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f46a0ff0640 1 -- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 msgr2=0x55d2cf4ae6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4ae6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f46a0ff0640 1 -- 172.21.15.64:0/4172216175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cf11c210 con 0x55d2cf4a9f30 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b5c30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4694012450 tx=0x7f4694004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8825ee94d01dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.250+0000 7f467b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.251+0000 7f467b7fe640 1 -- 172.21.15.64:0/4172216175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694023020 con 0x55d2cf4a9f30 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.251+0000 7f467b7fe640 1 -- 172.21.15.64:0/4172216175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f469401f890 con 0x55d2cf4a9f30 2024-05-09T09:22:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.251+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4690040070 con 0x55d2cf4a9f30 2024-05-09T09:22:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.252+0000 7f467b7fe640 1 -- 172.21.15.64:0/4172216175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f469401fa90 con 0x55d2cf4a9f30 2024-05-09T09:22:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.252+0000 7f46a5835740 10 client.18666 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.252+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d2cf1e8620 con 0x55d2cf4a9f30 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.253+0000 7f467b7fe640 1 -- 172.21.15.64:0/4172216175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f469401fc30 con 0x55d2cf4a9f30 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.253+0000 7f467b7fe640 1 -- 172.21.15.64:0/4172216175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f4694030020 con 0x55d2cf4a9f30 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.253+0000 7f46a5835740 10 client.18666 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.254+0000 7f46a5835740 10 client.18666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.254+0000 7f46a5835740 20 client.18666 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.254+0000 7f46a5835740 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf4ba250 0x55d2cf4bc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.254+0000 7f46a5835740 4 client.18666 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.254+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4690040070 con 0x55d2cf4ba250 2024-05-09T09:22:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.254+0000 7f469bfff640 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf4ba250 0x55d2cf4bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.255+0000 7f467b7fe640 10 client.18666 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.255+0000 7f469bfff640 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf4ba250 0x55d2cf4bc710 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d5fde204c8c2ad server_cookie=dd761d037500a12a in_seq=0 out_seq=0 2024-05-09T09:22:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.256+0000 7f467b7fe640 1 -- 172.21.15.64:0/4172216175 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4690040070 con 0x55d2cf4ba250 2024-05-09T09:22:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.256+0000 7f467b7fe640 10 client.18666 handle_command_reply: tid=0 2024-05-09T09:22:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.256+0000 7f46a5835740 1 client.18666 shutdown 2024-05-09T09:22:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.257+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 msgr2=0x55d2cf4b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.257+0000 7f46a5835740 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b5c30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4694012450 tx=0x7f4694004a40 comp rx=0 tx=0).stop 2024-05-09T09:22:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 shutdown_connections 2024-05-09T09:22:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f46a5835740 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf4ba250 0x55d2cf4bc710 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f46a5835740 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b5c30 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f46a5835740 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f46a5835740 1 --2- 172.21.15.64:0/4172216175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 >> 172.21.15.64:0/4172216175 conn(0x55d2cf49fa40 msgr2=0x55d2cf49eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.260+0000 7f467b7fe640 0 client.18666 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.263+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 shutdown_connections 2024-05-09T09:22:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.263+0000 7f46a5835740 1 -- 172.21.15.64:0/4172216175 wait complete. 2024-05-09T09:22:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.266+0000 7f46a5835740 20 client.18666 trim_cache size 0 max 16384 2024-05-09T09:22:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.306+0000 7f46a5835740 1 Processor -- start 2024-05-09T09:22:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.307+0000 7f46a5835740 1 -- start start 2024-05-09T09:22:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.307+0000 7f46a5835740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.307+0000 7f46a5835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.308+0000 7f46a5835740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:36380/0 (socket says 172.21.15.64:36380) 2024-05-09T09:22:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f469c1bbab0 con 0x55d2cf4a9f30 2024-05-09T09:22:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 -- 172.21.15.64:0/4056525488 learned_addr learned my addr 172.21.15.64:0/4056525488 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f469c1b9350 con 0x55d2cf4a43d0 2024-05-09T09:22:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f469c1b71a0 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469bfff640 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f46a0ff0640 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 -- 172.21.15.64:0/4056525488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 msgr2=0x55d2cf4bc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4bc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 -- 172.21.15.64:0/4056525488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 msgr2=0x55d2cf4a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.309+0000 7f469b7fe640 1 -- 172.21.15.64:0/4056525488 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cf491330 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.310+0000 7f469b7fe640 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5bf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f468c006fd0 tx=0x7f468c09bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cb59227b682952 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.311+0000 7f4681ffb640 1 -- 172.21.15.64:0/4056525488 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0a0570 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.311+0000 7f4681ffb640 1 -- 172.21.15.64:0/4056525488 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468c0a4e30 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.311+0000 7f4681ffb640 1 -- 172.21.15.64:0/4056525488 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0cd640 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.312+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 msgr2=0x55d2cf4b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.312+0000 7f46a5835740 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5bf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f468c006fd0 tx=0x7f468c09bbe0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.312+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 shutdown_connections 2024-05-09T09:22:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.312+0000 7f46a5835740 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.313+0000 7f46a5835740 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5bf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.313+0000 7f46a5835740 1 --2- 172.21.15.64:0/4056525488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4bc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.313+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 >> 172.21.15.64:0/4056525488 conn(0x55d2cf49fa40 msgr2=0x55d2cf4a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.313+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 shutdown_connections 2024-05-09T09:22:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.313+0000 7f46a5835740 1 -- 172.21.15.64:0/4056525488 wait complete. 2024-05-09T09:22:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.314+0000 7f46a5835740 1 -- start start 2024-05-09T09:22:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.314+0000 7f46a5835740 1 Processor -- start 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f469c1b7810 con 0x55d2cf4a43d0 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f469c1b8c80 con 0x55d2cf4a9f30 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f469c1b7ee0 con 0x55d2cf4a4dc0 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.315+0000 7f46a5835740 10 client.0 _ll_register_callbacks cb 0x7f4694016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54818/0 (socket says 172.21.15.64:54818) 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 -- 172.21.15.64:0/709062830 learned_addr learned my addr 172.21.15.64:0/709062830 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469b7fe640 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 -- 172.21.15.64:0/709062830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 msgr2=0x55d2cf4b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 -- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a9f30 msgr2=0x55d2cf4b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.316+0000 7f469bfff640 1 -- 172.21.15.64:0/709062830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cf4b43c0 con 0x55d2cf4a43d0 2024-05-09T09:22:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.317+0000 7f469bfff640 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4b5580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4690005ab0 tx=0x7f4690004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ed18b6d832411b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.317+0000 7f467b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.318+0000 7f467b7fe640 1 -- 172.21.15.64:0/709062830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690035040 con 0x55d2cf4a43d0 2024-05-09T09:22:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.318+0000 7f467b7fe640 1 -- 172.21.15.64:0/709062830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4690003ac0 con 0x55d2cf4a43d0 2024-05-09T09:22:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.318+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c0a4e30 con 0x55d2cf4a43d0 2024-05-09T09:22:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.319+0000 7f467b7fe640 1 -- 172.21.15.64:0/709062830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4690061050 con 0x55d2cf4a43d0 2024-05-09T09:22:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.320+0000 7f46a5835740 10 client.18672 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.320+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d2cf5213e0 con 0x55d2cf4a43d0 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.322+0000 7f467b7fe640 1 -- 172.21.15.64:0/709062830 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468c0a4e30 con 0x55d2cf4a43d0 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.322+0000 7f467b7fe640 1 -- 172.21.15.64:0/709062830 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f4690051180 con 0x55d2cf4a43d0 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.323+0000 7f46a5835740 10 client.18672 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.323+0000 7f46a5835740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.323+0000 7f46a5835740 20 client.18672 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.323+0000 7f46a5835740 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf521930 0x55d2cf523d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.323+0000 7f46a5835740 4 client.18672 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:22:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.323+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2cf415a70 con 0x55d2cf521930 2024-05-09T09:22:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.325+0000 7f469b7fe640 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf521930 0x55d2cf523d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.327+0000 7f469b7fe640 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf521930 0x55d2cf523d20 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2acad4e829d0f40 server_cookie=ed0a31e0bc7c11ac in_seq=0 out_seq=0 2024-05-09T09:22:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.327+0000 7f467b7fe640 10 client.18672 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.350+0000 7f467b7fe640 1 -- 172.21.15.64:0/709062830 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2cf415a70 con 0x55d2cf521930 2024-05-09T09:22:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.351+0000 7f467b7fe640 10 client.18672 handle_command_reply: tid=0 2024-05-09T09:22:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.351+0000 7f46a5835740 1 client.18672 shutdown 2024-05-09T09:22:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.351+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 msgr2=0x55d2cf4b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.351+0000 7f46a5835740 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4b5580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4690005ab0 tx=0x7f4690004500 comp rx=0 tx=0).stop 2024-05-09T09:22:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.353+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 shutdown_connections 2024-05-09T09:22:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.353+0000 7f46a5835740 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55d2cf521930 0x55d2cf523d20 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.353+0000 7f46a5835740 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2cf4a9f30 0x55d2cf4b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.353+0000 7f46a5835740 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d2cf4a4dc0 0x55d2cf4b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.353+0000 7f46a5835740 1 --2- 172.21.15.64:0/709062830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2cf4a43d0 0x55d2cf4b5580 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.353+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 >> 172.21.15.64:0/709062830 conn(0x55d2cf49fa40 msgr2=0x55d2cf49e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.355+0000 7f467b7fe640 0 client.18672 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.355+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 shutdown_connections 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.355+0000 7f46a5835740 1 -- 172.21.15.64:0/709062830 wait complete. 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.355+0000 7f46a5835740 20 client.18672 trim_cache size 0 max 16384 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:22:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:22:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.356+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46840a2d90 msgr2=0x7f46840a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.356+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46840a2d90 0x7f46840a5250 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4694005c40 tx=0x7f4694005bd0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.356+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 msgr2=0x7f469c1a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.356+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 0x7f469c1a6cc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b470 tx=0x7f468c00b930 comp rx=0 tx=0).stop 2024-05-09T09:22:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.358+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 shutdown_connections 2024-05-09T09:22:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.358+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f46840a2d90 0x7f46840a5250 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.358+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f469c109800 0x7f469c1add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.358+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469c108e40 0x7f469c1a6cc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.358+0000 7f46a17f1640 1 --2- 172.21.15.64:0/2619933531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f469c072b60 0x7f469c1a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.358+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 >> 172.21.15.64:0/2619933531 conn(0x7f469c06e6d0 msgr2=0x7f469c070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.361+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 shutdown_connections 2024-05-09T09:22:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:17.362+0000 7f46a17f1640 1 -- 172.21.15.64:0/2619933531 wait complete. 2024-05-09T09:22:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:18 smithi139 ceph-mon[33599]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 664 op/s 2024-05-09T09:22:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:18 smithi064 ceph-mon[30149]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 664 op/s 2024-05-09T09:22:19.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:18 smithi179 ceph-mon[34086]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 664 op/s 2024-05-09T09:22:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:20 smithi139 ceph-mon[33599]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 885 op/s 2024-05-09T09:22:21.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:20 smithi064 ceph-mon[30149]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 885 op/s 2024-05-09T09:22:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:20 smithi179 ceph-mon[34086]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 885 op/s 2024-05-09T09:22:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 720.0 s, 771.0 tps, lat 129.238 ms stddev 55.315 2024-05-09T09:22:23.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:22 smithi139 ceph-mon[33599]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-05-09T09:22:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:22 smithi064 ceph-mon[30149]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-05-09T09:22:23.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:22 smithi179 ceph-mon[34086]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-05-09T09:22:25.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:24 smithi179 ceph-mon[34086]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 779 op/s 2024-05-09T09:22:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:24 smithi139 ceph-mon[33599]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 779 op/s 2024-05-09T09:22:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:24 smithi064 ceph-mon[30149]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 779 op/s 2024-05-09T09:22:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 725.0 s, 737.4 tps, lat 135.540 ms stddev 61.486 2024-05-09T09:22:27.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:26 smithi139 ceph-mon[33599]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 739 op/s 2024-05-09T09:22:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:26 smithi064 ceph-mon[30149]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 739 op/s 2024-05-09T09:22:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:26 smithi179 ceph-mon[34086]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 739 op/s 2024-05-09T09:22:29.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:28 smithi139 ceph-mon[33599]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 642 op/s 2024-05-09T09:22:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:28 smithi064 ceph-mon[30149]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 642 op/s 2024-05-09T09:22:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:28 smithi179 ceph-mon[34086]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 642 op/s 2024-05-09T09:22:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 -- 172.21.15.64:0/4066620089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8408101030 msgr2=0x7f84081014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 --2- 172.21.15.64:0/4066620089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8408101030 0x7f84081014b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ca0 tx=0x7f840002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 -- 172.21.15.64:0/4066620089 shutdown_connections 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 --2- 172.21.15.64:0/4066620089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8408101030 0x7f84081014b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 --2- 172.21.15.64:0/4066620089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408068200 0x7f8408100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 --2- 172.21.15.64:0/4066620089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84080ff900 0x7f84080ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.521+0000 7f840759e640 1 -- 172.21.15.64:0/4066620089 >> 172.21.15.64:0/4066620089 conn(0x7f84080fae90 msgr2=0x7f84080fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.522+0000 7f840759e640 1 -- 172.21.15.64:0/4066620089 shutdown_connections 2024-05-09T09:22:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.522+0000 7f840759e640 1 -- 172.21.15.64:0/4066620089 wait complete. 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.523+0000 7f840759e640 1 Processor -- start 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.523+0000 7f840759e640 1 -- start start 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408068200 0x7f840819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84080ff900 0x7f840819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 0x7f84081a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840759e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84081139f0 con 0x7f8408068200 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840759e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8408113870 con 0x7f84080ff900 2024-05-09T09:22:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840759e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8408113b70 con 0x7f8408101030 2024-05-09T09:22:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 0x7f84081a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 0x7f84081a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49848/0 (socket says 172.21.15.64:49848) 2024-05-09T09:22:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 -- 172.21.15.64:0/2654269534 learned_addr learned my addr 172.21.15.64:0/2654269534 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8405d9b640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84080ff900 0x7f840819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f840659c640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408068200 0x7f840819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 -- 172.21.15.64:0/2654269534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84080ff900 msgr2=0x7f840819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84080ff900 0x7f840819e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 -- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408068200 msgr2=0x7f840819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408068200 0x7f840819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.524+0000 7f8406d9d640 1 -- 172.21.15.64:0/2654269534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84081a5950 con 0x7f8408101030 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.525+0000 7f8406d9d640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 0x7f84081a52b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8400003f10 tx=0x7f8400003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.525+0000 7f83f77fe640 1 -- 172.21.15.64:0/2654269534 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004240 con 0x7f8408101030 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.525+0000 7f83f77fe640 1 -- 172.21.15.64:0/2654269534 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84000305c0 con 0x7f8408101030 2024-05-09T09:22:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.525+0000 7f83f77fe640 1 -- 172.21.15.64:0/2654269534 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840003c670 con 0x7f8408101030 2024-05-09T09:22:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.525+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84081a5be0 con 0x7f8408101030 2024-05-09T09:22:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.526+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a6020 con 0x7f8408101030 2024-05-09T09:22:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.530+0000 7f83f77fe640 1 -- 172.21.15.64:0/2654269534 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8400030770 con 0x7f8408101030 2024-05-09T09:22:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.532+0000 7f83f77fe640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83f00a2cd0 0x7f83f00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.532+0000 7f83f77fe640 1 -- 172.21.15.64:0/2654269534 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f84000f1060 con 0x7f8408101030 2024-05-09T09:22:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.534+0000 7f840659c640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83f00a2cd0 0x7f83f00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.534+0000 7f840fb6c740 1 Processor -- start 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 -- start start 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd8087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd80d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd80df30 0x5592dd8103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5592dd76c340 con 0x5592dd808dc0 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5592dd714290 con 0x5592dd80df30 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840fb6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5592dd768300 con 0x5592dd8083d0 2024-05-09T09:22:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840659c640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83f00a2cd0 0x7f83f00a5190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f83f80058c0 tx=0x7f83f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.536+0000 7f840659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd8087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd8087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49852/0 (socket says 172.21.15.64:49852) 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 -- 172.21.15.64:0/2790587262 learned_addr learned my addr 172.21.15.64:0/2790587262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f8405d9b640 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd80d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 -- 172.21.15.64:0/2790587262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd80df30 msgr2=0x5592dd8103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd80df30 0x5592dd8103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 -- 172.21.15.64:0/2790587262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 msgr2=0x5592dd80d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd80d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 -- 172.21.15.64:0/2790587262 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592dd7388e0 con 0x5592dd8083d0 2024-05-09T09:22:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f840659c640 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd8087f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83f800ee10 tx=0x7f83f8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70cecc526e89c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.537+0000 7f83f4ff9640 1 -- 172.21.15.64:0/2790587262 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8021070 con 0x5592dd8083d0 2024-05-09T09:22:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.538+0000 7f83f4ff9640 1 -- 172.21.15.64:0/2790587262 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f80054d0 con 0x5592dd8083d0 2024-05-09T09:22:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.538+0000 7f83f4ff9640 1 -- 172.21.15.64:0/2790587262 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f801d8c0 con 0x5592dd8083d0 2024-05-09T09:22:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.539+0000 7f840fb6c740 1 -- 172.21.15.64:0/2790587262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 msgr2=0x5592dd8087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.539+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd8087f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83f800ee10 tx=0x7f83f8011000 comp rx=0 tx=0).stop 2024-05-09T09:22:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 -- 172.21.15.64:0/2790587262 shutdown_connections 2024-05-09T09:22:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd80df30 0x5592dd8103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd80d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2790587262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd8087f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 -- 172.21.15.64:0/2790587262 >> 172.21.15.64:0/2790587262 conn(0x5592dd803a40 msgr2=0x5592dd803e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 -- 172.21.15.64:0/2790587262 shutdown_connections 2024-05-09T09:22:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 -- 172.21.15.64:0/2790587262 wait complete. 2024-05-09T09:22:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 -- start start 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.540+0000 7f840fb6c740 1 Processor -- start 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd823750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd808dc0 0x5592dd823c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 0x5592dd82ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5592dd821ef0 con 0x5592dd80df30 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5592dd727790 con 0x5592dd808dc0 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5592dd822070 con 0x5592dd8083d0 2024-05-09T09:22:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f840fb6c740 10 client.0 _ll_register_callbacks cb 0x5592dd762a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 0x5592dd82ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 0x5592dd82ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54464/0 (socket says 172.21.15.64:54464) 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.541+0000 7f8406d9d640 1 -- 172.21.15.64:0/3614716787 learned_addr learned my addr 172.21.15.64:0/3614716787 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f8406d9d640 1 -- 172.21.15.64:0/3614716787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 msgr2=0x5592dd823750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f8406d9d640 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd823750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f8406d9d640 1 -- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd808dc0 msgr2=0x5592dd823c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f8406d9d640 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd808dc0 0x5592dd823c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f8406d9d640 1 -- 172.21.15.64:0/3614716787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592dd480210 con 0x5592dd80df30 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f8406d9d640 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 0x5592dd82ac60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f84000bd460 tx=0x7f84000bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440ab116a3b70065 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.542+0000 7f83edffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.544+0000 7f83edffb640 1 -- 172.21.15.64:0/3614716787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000d6060 con 0x5592dd80df30 2024-05-09T09:22:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.544+0000 7f83edffb640 1 -- 172.21.15.64:0/3614716787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f84000c5cb0 con 0x5592dd80df30 2024-05-09T09:22:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.544+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f80054d0 con 0x5592dd80df30 2024-05-09T09:22:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.545+0000 7f83edffb640 1 -- 172.21.15.64:0/3614716787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f84000bd4d0 con 0x5592dd80df30 2024-05-09T09:22:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.545+0000 7f840fb6c740 10 client.18681 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.545+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5592dd54c620 con 0x5592dd80df30 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f83edffb640 1 -- 172.21.15.64:0/3614716787 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f80054d0 con 0x5592dd80df30 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f83edffb640 1 -- 172.21.15.64:0/3614716787 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f84000d6200 con 0x5592dd80df30 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840fb6c740 10 client.18681 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840fb6c740 10 client.18681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840fb6c740 20 client.18681 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840fb6c740 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd82f150 0x5592dd8315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840fb6c740 4 client.18681 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592dd779a70 con 0x5592dd82f150 2024-05-09T09:22:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.546+0000 7f840659c640 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd82f150 0x5592dd8315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.549+0000 7f840659c640 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd82f150 0x5592dd8315b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768ad2adcdf1b223 server_cookie=e5b09e07c5ef4268 in_seq=0 out_seq=0 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.549+0000 7f83edffb640 10 client.18681 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.550+0000 7f83edffb640 1 -- 172.21.15.64:0/3614716787 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592dd779a70 con 0x5592dd82f150 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.550+0000 7f83edffb640 10 client.18681 handle_command_reply: tid=0 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.550+0000 7f840fb6c740 1 client.18681 shutdown 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 msgr2=0x5592dd82ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 0x5592dd82ac60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f84000bd460 tx=0x7f84000bd0d0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 shutdown_connections 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd82f150 0x5592dd8315b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd80df30 0x5592dd82ac60 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd808dc0 0x5592dd823c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 --2- 172.21.15.64:0/3614716787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd8083d0 0x5592dd823750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 >> 172.21.15.64:0/3614716787 conn(0x5592dd803a40 msgr2=0x5592dd802ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f83edffb640 0 client.18681 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 shutdown_connections 2024-05-09T09:22:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 1 -- 172.21.15.64:0/3614716787 wait complete. 2024-05-09T09:22:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.551+0000 7f840fb6c740 20 client.18681 trim_cache size 0 max 16384 2024-05-09T09:22:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.585+0000 7f840fb6c740 1 Processor -- start 2024-05-09T09:22:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.585+0000 7f840fb6c740 1 -- start start 2024-05-09T09:22:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd80a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd82ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd80a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840fb6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84081aa760 con 0x5592dd808dc0 2024-05-09T09:22:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840fb6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84081a8000 con 0x5592dd8083d0 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840fb6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8408073600 con 0x5592dd80df30 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd80a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd80a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51816/0 (socket says 172.21.15.64:51816) 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 -- 172.21.15.64:0/2042897688 learned_addr learned my addr 172.21.15.64:0/2042897688 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f8406d9d640 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd80a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 -- 172.21.15.64:0/2042897688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 msgr2=0x5592dd80a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd80a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 -- 172.21.15.64:0/2042897688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 msgr2=0x5592dd82ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd82ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.586+0000 7f840659c640 1 -- 172.21.15.64:0/2042897688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592dd831c30 con 0x5592dd8083d0 2024-05-09T09:22:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.587+0000 7f840659c640 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd80a120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f83f800ee10 tx=0x7f83f8004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abf69e14e8312ea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.587+0000 7f83effff640 1 -- 172.21.15.64:0/2042897688 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8035020 con 0x5592dd8083d0 2024-05-09T09:22:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.587+0000 7f83effff640 1 -- 172.21.15.64:0/2042897688 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f8031870 con 0x5592dd8083d0 2024-05-09T09:22:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.587+0000 7f83effff640 1 -- 172.21.15.64:0/2042897688 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8021040 con 0x5592dd8083d0 2024-05-09T09:22:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 -- 172.21.15.64:0/2042897688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 msgr2=0x5592dd80a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd80a120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f83f800ee10 tx=0x7f83f8004a10 comp rx=0 tx=0).stop 2024-05-09T09:22:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 -- 172.21.15.64:0/2042897688 shutdown_connections 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd80a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd82ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 --2- 172.21.15.64:0/2042897688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd80a120 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.588+0000 7f840fb6c740 1 -- 172.21.15.64:0/2042897688 >> 172.21.15.64:0/2042897688 conn(0x5592dd803a40 msgr2=0x5592dd823750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.589+0000 7f840fb6c740 1 -- 172.21.15.64:0/2042897688 shutdown_connections 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.589+0000 7f840fb6c740 1 -- 172.21.15.64:0/2042897688 wait complete. 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.589+0000 7f840fb6c740 1 -- start start 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.590+0000 7f840fb6c740 1 Processor -- start 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.590+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd8284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.590+0000 7f840fb6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd828a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd8284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd8284d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51832/0 (socket says 172.21.15.64:51832) 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840659c640 1 -- 172.21.15.64:0/199128180 learned_addr learned my addr 172.21.15.64:0/199128180 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd829a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84081a64c0 con 0x5592dd808dc0 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84081a7930 con 0x5592dd8083d0 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f8406d9d640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd829a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f8405d9b640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd828a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f84081a6b90 con 0x5592dd80df30 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.591+0000 7f840fb6c740 10 client.0 _ll_register_callbacks cb 0x7f8400030d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.592+0000 7f840659c640 1 -- 172.21.15.64:0/199128180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 msgr2=0x5592dd829a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.592+0000 7f840659c640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd829a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.592+0000 7f840659c640 1 -- 172.21.15.64:0/199128180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 msgr2=0x5592dd828a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.592+0000 7f840659c640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd828a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.592+0000 7f840659c640 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592dd82a010 con 0x5592dd8083d0 2024-05-09T09:22:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.593+0000 7f840659c640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd8284d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83f80315f0 tx=0x7f83f8031620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b189f5946a8ebdba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.596+0000 7f83edffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.596+0000 7f83edffb640 1 -- 172.21.15.64:0/199128180 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8014050 con 0x5592dd8083d0 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.596+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f8031870 con 0x5592dd8083d0 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.596+0000 7f83edffb640 1 -- 172.21.15.64:0/199128180 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f83f801da80 con 0x5592dd8083d0 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.596+0000 7f83edffb640 1 -- 172.21.15.64:0/199128180 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f83f804c050 con 0x5592dd8083d0 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.597+0000 7f840fb6c740 10 client.28145 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.597+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5592dd895880 con 0x5592dd8083d0 2024-05-09T09:22:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.598+0000 7f83edffb640 1 -- 172.21.15.64:0/199128180 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f8031870 con 0x5592dd8083d0 2024-05-09T09:22:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.598+0000 7f83edffb640 1 -- 172.21.15.64:0/199128180 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f83f803a9e0 con 0x5592dd8083d0 2024-05-09T09:22:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f840fb6c740 10 client.28145 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f840fb6c740 10 client.28145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f840fb6c740 20 client.28145 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd896030 0x5592dd898450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f840fb6c740 4 client.28145 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:22:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592dd773dc0 con 0x5592dd896030 2024-05-09T09:22:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.599+0000 7f8405d9b640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd896030 0x5592dd898450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.600+0000 7f8405d9b640 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd896030 0x5592dd898450 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61da4bea7df49de7 server_cookie=25bfc43b8ad9d5b8 in_seq=0 out_seq=0 2024-05-09T09:22:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.600+0000 7f83edffb640 10 client.28145 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.622+0000 7f83edffb640 1 -- 172.21.15.64:0/199128180 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592dd773dc0 con 0x5592dd896030 2024-05-09T09:22:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.622+0000 7f83edffb640 10 client.28145 handle_command_reply: tid=0 2024-05-09T09:22:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.622+0000 7f840fb6c740 1 client.28145 shutdown 2024-05-09T09:22:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.622+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 msgr2=0x5592dd8284d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.622+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd8284d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83f80315f0 tx=0x7f83f8031620 comp rx=0 tx=0).stop 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 shutdown_connections 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5592dd896030 0x5592dd898450 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592dd80df30 0x5592dd829a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592dd808dc0 0x5592dd828a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 --2- 172.21.15.64:0/199128180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5592dd8083d0 0x5592dd8284d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 >> 172.21.15.64:0/199128180 conn(0x5592dd803a40 msgr2=0x5592dd823750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f83edffb640 0 client.28145 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 shutdown_connections 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 1 -- 172.21.15.64:0/199128180 wait complete. 2024-05-09T09:22:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.623+0000 7f840fb6c740 20 client.28145 trim_cache size 0 max 16384 2024-05-09T09:22:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:22:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:22:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:22:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.624+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83f00a2cd0 msgr2=0x7f83f00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83f00a2cd0 0x7f83f00a5190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f83f80058c0 tx=0x7f83f8005850 comp rx=0 tx=0).stop 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 msgr2=0x7f84081a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 0x7f84081a52b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8400003f10 tx=0x7f8400003f40 comp rx=0 tx=0).stop 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 shutdown_connections 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83f00a2cd0 0x7f83f00a5190 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408101030 0x7f84081a52b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84080ff900 0x7f840819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 --2- 172.21.15.64:0/2654269534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408068200 0x7f840819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 >> 172.21.15.64:0/2654269534 conn(0x7f84080fae90 msgr2=0x7f8408102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 shutdown_connections 2024-05-09T09:22:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:30.625+0000 7f840759e640 1 -- 172.21.15.64:0/2654269534 wait complete. 2024-05-09T09:22:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:30 smithi139 ceph-mon[33599]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 872 op/s 2024-05-09T09:22:31.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:30 smithi064 ceph-mon[30149]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 872 op/s 2024-05-09T09:22:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:30 smithi179 ceph-mon[34086]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 872 op/s 2024-05-09T09:22:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 730.0 s, 751.8 tps, lat 133.054 ms stddev 46.705 2024-05-09T09:22:32.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:22:32.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:22:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 -- 172.21.15.64:0/3606174877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c108fe0 msgr2=0x7fe66c109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 --2- 172.21.15.64:0/3606174877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c108fe0 0x7fe66c109460 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009900 tx=0x7fe65c02a440 comp rx=0 tx=0).stop 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 -- 172.21.15.64:0/3606174877 shutdown_connections 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 --2- 172.21.15.64:0/3606174877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c1099a0 0x7fe66c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 --2- 172.21.15.64:0/3606174877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c108fe0 0x7fe66c109460 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 --2- 172.21.15.64:0/3606174877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 0x7fe66c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.949+0000 7fe673712640 1 -- 172.21.15.64:0/3606174877 >> 172.21.15.64:0/3606174877 conn(0x7fe66c076e30 msgr2=0x7fe66c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.950+0000 7fe673712640 1 -- 172.21.15.64:0/3606174877 shutdown_connections 2024-05-09T09:22:32.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.950+0000 7fe673712640 1 -- 172.21.15.64:0/3606174877 wait complete. 2024-05-09T09:22:32.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.951+0000 7fe673712640 1 Processor -- start 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.951+0000 7fe673712640 1 -- start start 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.951+0000 7fe673712640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 0x7fe66c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.951+0000 7fe673712640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c108fe0 0x7fe66c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.951+0000 7fe673712640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c1099a0 0x7fe66c1a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 0x7fe66c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 0x7fe66c19e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49896/0 (socket says 172.21.15.64:49896) 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe673712640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c113a60 con 0x7fe66c1099a0 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe673712640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1138e0 con 0x7fe66c108fe0 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe673712640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c113be0 con 0x7fe66c107e90 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe6714ae640 1 -- 172.21.15.64:0/630474122 learned_addr learned my addr 172.21.15.64:0/630474122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:32.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe671caf640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c1099a0 0x7fe66c1a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.952+0000 7fe670cad640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c108fe0 0x7fe66c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.953+0000 7fe671caf640 1 -- 172.21.15.64:0/630474122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 msgr2=0x7fe66c19e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.953+0000 7fe671caf640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 0x7fe66c19e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.953+0000 7fe671caf640 1 -- 172.21.15.64:0/630474122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c108fe0 msgr2=0x7fe66c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.953+0000 7fe671caf640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c108fe0 0x7fe66c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.953+0000 7fe671caf640 1 -- 172.21.15.64:0/630474122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c1a5d90 con 0x7fe66c1099a0 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.953+0000 7fe671caf640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c1099a0 0x7fe66c1a5720 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe66400ca30 tx=0x7fe66400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.954+0000 7fe6627fc640 1 -- 172.21.15.64:0/630474122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664007d90 con 0x7fe66c1099a0 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.954+0000 7fe6627fc640 1 -- 172.21.15.64:0/630474122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe664005020 con 0x7fe66c1099a0 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.954+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c1a6080 con 0x7fe66c1099a0 2024-05-09T09:22:32.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.954+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1a6540 con 0x7fe66c1099a0 2024-05-09T09:22:32.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.955+0000 7fe6627fc640 1 -- 172.21.15.64:0/630474122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400f690 con 0x7fe66c1099a0 2024-05-09T09:22:32.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.959+0000 7fe676416740 1 Processor -- start 2024-05-09T09:22:32.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.960+0000 7fe676416740 1 -- start start 2024-05-09T09:22:32.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.960+0000 7fe676416740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe676416740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2f7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe670cad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe6627fc640 1 -- 172.21.15.64:0/630474122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe664004a00 con 0x7fe66c1099a0 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe670cad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54506/0 (socket says 172.21.15.64:54506) 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe670cad640 1 -- 172.21.15.64:0/2374286733 learned_addr learned my addr 172.21.15.64:0/2374286733 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe676416740 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed2f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b4ed256d00 con 0x55b4ed2f1f30 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe6714ae640 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b4ed1fe4c0 con 0x55b4ed2f7a90 2024-05-09T09:22:32.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.961+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b4ed252cc0 con 0x55b4ed2f2920 2024-05-09T09:22:32.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe6627fc640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6580a2c40 0x7fe6580a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe670cad640 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed2f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe670cad640 1 -- 172.21.15.64:0/2374286733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 msgr2=0x55b4ed2f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe670cad640 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe670cad640 1 -- 172.21.15.64:0/2374286733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 msgr2=0x55b4ed2f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe670cad640 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed2f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe670cad640 1 -- 172.21.15.64:0/2374286733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ed0347d0 con 0x55b4ed2f1f30 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.962+0000 7fe6627fc640 1 -- 172.21.15.64:0/630474122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe6640cf4d0 con 0x7fe66c1099a0 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.963+0000 7fe670cad640 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2f2350 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009900 tx=0x7fe65c009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684bffd589e3e771 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.964+0000 7fe671caf640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6580a2c40 0x7fe6580a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.964+0000 7fe6577fe640 1 -- 172.21.15.64:0/2374286733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c035020 con 0x55b4ed2f1f30 2024-05-09T09:22:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.964+0000 7fe6577fe640 1 -- 172.21.15.64:0/2374286733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe65c047070 con 0x55b4ed2f1f30 2024-05-09T09:22:32.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.964+0000 7fe6577fe640 1 -- 172.21.15.64:0/2374286733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c031930 con 0x55b4ed2f1f30 2024-05-09T09:22:32.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.965+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 msgr2=0x55b4ed2f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.965+0000 7fe676416740 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2f2350 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009900 tx=0x7fe65c009460 comp rx=0 tx=0).stop 2024-05-09T09:22:32.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.965+0000 7fe671caf640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6580a2c40 0x7fe6580a5100 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe66409b6f0 tx=0x7fe664097d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.966+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 shutdown_connections 2024-05-09T09:22:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.966+0000 7fe676416740 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed2f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.966+0000 7fe676416740 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.966+0000 7fe676416740 1 --2- 172.21.15.64:0/2374286733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2f2350 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.966+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 >> 172.21.15.64:0/2374286733 conn(0x55b4ed2ed5a0 msgr2=0x55b4ed2ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.967+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 shutdown_connections 2024-05-09T09:22:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.967+0000 7fe676416740 1 -- 172.21.15.64:0/2374286733 wait complete. 2024-05-09T09:22:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.967+0000 7fe676416740 1 -- start start 2024-05-09T09:22:32.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.968+0000 7fe676416740 1 Processor -- start 2024-05-09T09:22:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.968+0000 7fe676416740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe676416740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe676416740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed3037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe676416740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b4ed2fa9e0 con 0x55b4ed2f1f30 2024-05-09T09:22:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe676416740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b4ed211b90 con 0x55b4ed2f7a90 2024-05-09T09:22:32.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe676416740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b4ed2fab60 con 0x55b4ed2f2920 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe676416740 10 client.0 _ll_register_callbacks cb 0x55b4ed24dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49910/0 (socket says 172.21.15.64:49910) 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe6714ae640 1 -- 172.21.15.64:0/3660031599 learned_addr learned my addr 172.21.15.64:0/3660031599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe670cad640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed3037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.969+0000 7fe670cad640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe6714ae640 1 -- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 msgr2=0x55b4ed3037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe6714ae640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed3037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe6714ae640 1 -- 172.21.15.64:0/3660031599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 msgr2=0x55b4ed2fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:32.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe6714ae640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe6714ae640 1 -- 172.21.15.64:0/3660031599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ecf6a3f0 con 0x55b4ed2f2920 2024-05-09T09:22:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe6714ae640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc780 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe668009fb0 tx=0x7fe6680116f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38385df11888460 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.970+0000 7fe654ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.971+0000 7fe654ff9640 1 -- 172.21.15.64:0/3660031599 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668019070 con 0x55b4ed2f2920 2024-05-09T09:22:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.971+0000 7fe654ff9640 1 -- 172.21.15.64:0/3660031599 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe668015870 con 0x55b4ed2f2920 2024-05-09T09:22:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.971+0000 7fe670cad640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed3037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.974+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe65c047070 con 0x55b4ed2f2920 2024-05-09T09:22:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.974+0000 7fe654ff9640 1 -- 172.21.15.64:0/3660031599 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe668023050 con 0x55b4ed2f2920 2024-05-09T09:22:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.974+0000 7fe676416740 10 client.28321 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.974+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b4ecf2a0f0 con 0x55b4ed2f2920 2024-05-09T09:22:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe654ff9640 1 -- 172.21.15.64:0/3660031599 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe65c047070 con 0x55b4ed2f2920 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe654ff9640 1 -- 172.21.15.64:0/3660031599 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fe66801eaf0 con 0x55b4ed2f2920 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe676416740 10 client.28321 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe676416740 10 client.28321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe676416740 20 client.28321 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe676416740 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed308110 0x55b4ed30a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe676416740 4 client.28321 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.975+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4ed264430 con 0x55b4ed308110 2024-05-09T09:22:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.976+0000 7fe670cad640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed308110 0x55b4ed30a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.978+0000 7fe670cad640 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed308110 0x55b4ed30a550 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a87a77635e97fa0 server_cookie=6b2b7b973ad1b4d2 in_seq=0 out_seq=0 2024-05-09T09:22:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.978+0000 7fe654ff9640 10 client.28321 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.979+0000 7fe654ff9640 1 -- 172.21.15.64:0/3660031599 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4ed264430 con 0x55b4ed308110 2024-05-09T09:22:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.979+0000 7fe654ff9640 10 client.28321 handle_command_reply: tid=0 2024-05-09T09:22:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.979+0000 7fe676416740 1 client.28321 shutdown 2024-05-09T09:22:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 msgr2=0x55b4ed2fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc780 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe668009fb0 tx=0x7fe6680116f0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 shutdown_connections 2024-05-09T09:22:32.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed308110 0x55b4ed30a550 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed3037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc780 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 --2- 172.21.15.64:0/3660031599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:32.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 >> 172.21.15.64:0/3660031599 conn(0x55b4ed2ed5a0 msgr2=0x55b4ed2ec7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:32.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.981+0000 7fe654ff9640 0 client.28321 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:32.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.982+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 shutdown_connections 2024-05-09T09:22:32.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.982+0000 7fe676416740 1 -- 172.21.15.64:0/3660031599 wait complete. 2024-05-09T09:22:32.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:32.982+0000 7fe676416740 20 client.28321 trim_cache size 0 max 16384 2024-05-09T09:22:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.024+0000 7fe676416740 1 Processor -- start 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.024+0000 7fe676416740 1 -- start start 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.024+0000 7fe676416740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.024+0000 7fe676416740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.025+0000 7fe676416740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed303780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.025+0000 7fe676416740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b3120 con 0x55b4ed2f2920 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.025+0000 7fe676416740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b09c0 con 0x55b4ed2f7a90 2024-05-09T09:22:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.025+0000 7fe676416740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1ae810 con 0x55b4ed2f1f30 2024-05-09T09:22:33.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.026+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.026+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54528/0 (socket says 172.21.15.64:54528) 2024-05-09T09:22:33.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.026+0000 7fe6714ae640 1 -- 172.21.15.64:0/2780431078 learned_addr learned my addr 172.21.15.64:0/2780431078 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:33.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.027+0000 7fe670cad640 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.027+0000 7fe670cad640 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed303780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.027+0000 7fe670cad640 1 -- 172.21.15.64:0/2780431078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 msgr2=0x55b4ed303780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:33.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.028+0000 7fe670cad640 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed303780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.028+0000 7fe670cad640 1 -- 172.21.15.64:0/2780431078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 msgr2=0x55b4ed2fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:33.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.028+0000 7fe670cad640 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.029+0000 7fe670cad640 1 -- 172.21.15.64:0/2780431078 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ed2fac20 con 0x55b4ed2f1f30 2024-05-09T09:22:33.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.030+0000 7fe670cad640 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55b4ed30ae00 tx=0x7fe65c02a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fefd6c2363caf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.030+0000 7fe6714ae640 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:33.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.032+0000 7fe656ffd640 1 -- 172.21.15.64:0/2780431078 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c035040 con 0x55b4ed2f1f30 2024-05-09T09:22:33.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.032+0000 7fe656ffd640 1 -- 172.21.15.64:0/2780431078 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe65c047070 con 0x55b4ed2f1f30 2024-05-09T09:22:33.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.032+0000 7fe656ffd640 1 -- 172.21.15.64:0/2780431078 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c054940 con 0x55b4ed2f1f30 2024-05-09T09:22:33.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 -- 172.21.15.64:0/2780431078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f1f30 msgr2=0x55b4ed2fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55b4ed30ae00 tx=0x7fe65c02a440 comp rx=0 tx=0).stop 2024-05-09T09:22:33.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 -- 172.21.15.64:0/2780431078 shutdown_connections 2024-05-09T09:22:33.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f7a90 0x55b4ed303780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f2920 0x55b4ed2fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 --2- 172.21.15.64:0/2780431078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f1f30 0x55b4ed2fc220 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.033+0000 7fe676416740 1 -- 172.21.15.64:0/2780431078 >> 172.21.15.64:0/2780431078 conn(0x55b4ed2ed5a0 msgr2=0x55b4ed2f3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.035+0000 7fe676416740 1 -- 172.21.15.64:0/2780431078 shutdown_connections 2024-05-09T09:22:33.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.035+0000 7fe676416740 1 -- 172.21.15.64:0/2780431078 wait complete. 2024-05-09T09:22:33.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.035+0000 7fe676416740 1 -- start start 2024-05-09T09:22:33.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 Processor -- start 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f1f30 0x55b4ed3005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed300b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f7a90 0x55b4ed37d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1aee80 con 0x55b4ed2f7a90 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b02f0 con 0x55b4ed2f1f30 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1af550 con 0x55b4ed2f2920 2024-05-09T09:22:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.036+0000 7fe676416740 10 client.0 _ll_register_callbacks cb 0x7fe668009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.038+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed300b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.038+0000 7fe6714ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed300b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49920/0 (socket says 172.21.15.64:49920) 2024-05-09T09:22:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.038+0000 7fe6714ae640 1 -- 172.21.15.64:0/3100082499 learned_addr learned my addr 172.21.15.64:0/3100082499 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.038+0000 7fe670cad640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f1f30 0x55b4ed3005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.038+0000 7fe6714ae640 1 -- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f1f30 msgr2=0x55b4ed3005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe6714ae640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f1f30 0x55b4ed3005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe6714ae640 1 -- 172.21.15.64:0/3100082499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f7a90 msgr2=0x55b4ed37d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe6714ae640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f7a90 0x55b4ed37d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe6714ae640 1 -- 172.21.15.64:0/3100082499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ed37dc10 con 0x55b4ed2f2920 2024-05-09T09:22:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe6714ae640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed300b30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe668014520 tx=0x7fe6680148b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=788cbd6add9e45d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe654ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe670cad640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f1f30 0x55b4ed3005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.039+0000 7fe654ff9640 1 -- 172.21.15.64:0/3100082499 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668019040 con 0x55b4ed2f2920 2024-05-09T09:22:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.040+0000 7fe654ff9640 1 -- 172.21.15.64:0/3100082499 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe668014950 con 0x55b4ed2f2920 2024-05-09T09:22:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.040+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe65c047070 con 0x55b4ed2f2920 2024-05-09T09:22:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.042+0000 7fe654ff9640 1 -- 172.21.15.64:0/3100082499 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe668038050 con 0x55b4ed2f2920 2024-05-09T09:22:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.043+0000 7fe676416740 10 client.28327 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.043+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b4ed37ee40 con 0x55b4ed2f2920 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.043+0000 7fe654ff9640 1 -- 172.21.15.64:0/3100082499 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe65c047070 con 0x55b4ed2f2920 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.043+0000 7fe654ff9640 1 -- 172.21.15.64:0/3100082499 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fe6680037a0 con 0x55b4ed2f2920 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.043+0000 7fe676416740 10 client.28327 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.044+0000 7fe676416740 10 client.28327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.044+0000 7fe676416740 20 client.28327 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.044+0000 7fe676416740 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed37f8d0 0x55b4ed381cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.044+0000 7fe676416740 4 client.28327 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.044+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4ed25e780 con 0x55b4ed37f8d0 2024-05-09T09:22:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.044+0000 7fe670cad640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed37f8d0 0x55b4ed381cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.045+0000 7fe670cad640 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed37f8d0 0x55b4ed381cf0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0fe30f6fc8bf343 server_cookie=13499f9a3ff48799 in_seq=0 out_seq=0 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.045+0000 7fe654ff9640 10 client.28327 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.046+0000 7fe654ff9640 1 -- 172.21.15.64:0/3100082499 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b4ed25e780 con 0x55b4ed37f8d0 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.046+0000 7fe654ff9640 10 client.28327 handle_command_reply: tid=0 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.046+0000 7fe676416740 1 client.28327 shutdown 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.046+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 msgr2=0x55b4ed300b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed300b30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe668014520 tx=0x7fe6680148b0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 shutdown_connections 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b4ed37f8d0 0x55b4ed381cf0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4ed2f7a90 0x55b4ed37d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4ed2f2920 0x55b4ed300b30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 --2- 172.21.15.64:0/3100082499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b4ed2f1f30 0x55b4ed3005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 >> 172.21.15.64:0/3100082499 conn(0x55b4ed2ed5a0 msgr2=0x55b4ed2fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.047+0000 7fe654ff9640 0 client.28327 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.048+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 shutdown_connections 2024-05-09T09:22:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.048+0000 7fe676416740 1 -- 172.21.15.64:0/3100082499 wait complete. 2024-05-09T09:22:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.050+0000 7fe676416740 20 client.28327 trim_cache size 0 max 16384 2024-05-09T09:22:33.052 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:22:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:22:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:22:33.052 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:22:33.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.053+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6580a2c40 msgr2=0x7fe6580a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.053+0000 7fe673712640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6580a2c40 0x7fe6580a5100 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe66409b6f0 tx=0x7fe664097d80 comp rx=0 tx=0).stop 2024-05-09T09:22:33.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.053+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c1099a0 msgr2=0x7fe66c1a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.053+0000 7fe673712640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c1099a0 0x7fe66c1a5720 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe66400ca30 tx=0x7fe66400cef0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.054+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 shutdown_connections 2024-05-09T09:22:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.054+0000 7fe673712640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe6580a2c40 0x7fe6580a5100 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.054+0000 7fe673712640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe66c1099a0 0x7fe66c1a5720 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.054+0000 7fe673712640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c108fe0 0x7fe66c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.054+0000 7fe673712640 1 --2- 172.21.15.64:0/630474122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c107e90 0x7fe66c19e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.054+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 >> 172.21.15.64:0/630474122 conn(0x7fe66c076e30 msgr2=0x7fe66c10c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.056+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 shutdown_connections 2024-05-09T09:22:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.057+0000 7fe673712640 1 -- 172.21.15.64:0/630474122 wait complete. 2024-05-09T09:22:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:32 smithi139 ceph-mon[33599]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 651 op/s 2024-05-09T09:22:33.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:32 smithi064 ceph-mon[30149]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 651 op/s 2024-05-09T09:22:33.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:22:33.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:22:33.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:32 smithi179 ceph-mon[34086]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 651 op/s 2024-05-09T09:22:33.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.273+0000 7f06177fe640 1 -- 172.21.15.64:0/4025596206 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c031880 con 0x7f061810b1a0 2024-05-09T09:22:33.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.274+0000 7f061ed16640 1 -- 172.21.15.64:0/4025596206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 msgr2=0x7f061810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.274+0000 7f061ed16640 1 --2- 172.21.15.64:0/4025596206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f061810d630 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f060c009900 tx=0x7f060c02a400 comp rx=0 tx=0).stop 2024-05-09T09:22:33.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.276+0000 7f061ed16640 1 -- 172.21.15.64:0/4025596206 shutdown_connections 2024-05-09T09:22:33.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.276+0000 7f061ed16640 1 --2- 172.21.15.64:0/4025596206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061810db70 0x7f061810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.276+0000 7f061ed16640 1 --2- 172.21.15.64:0/4025596206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f061810d630 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.276+0000 7f061ed16640 1 --2- 172.21.15.64:0/4025596206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0618108830 0x7f061810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.276+0000 7f061ed16640 1 -- 172.21.15.64:0/4025596206 >> 172.21.15.64:0/4025596206 conn(0x7f061806e6d0 msgr2=0x7f0618070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.277+0000 7f061ed16640 1 -- 172.21.15.64:0/4025596206 shutdown_connections 2024-05-09T09:22:33.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.277+0000 7f061ed16640 1 -- 172.21.15.64:0/4025596206 wait complete. 2024-05-09T09:22:33.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.277+0000 7f061ed16640 1 Processor -- start 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.277+0000 7f061ed16640 1 -- start start 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061ed16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0618108830 0x7f06181a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061ed16640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f06181a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061ed16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061810db70 0x7f06181ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061ed16640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f061811c220 con 0x7f061810db70 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061ed16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f061811c0a0 con 0x7f0618108830 2024-05-09T09:22:33.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061ed16640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f061811c3a0 con 0x7f061810b1a0 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f0617fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f06181a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f0617fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f06181a6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49946/0 (socket says 172.21.15.64:49946) 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061cab2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0618108830 0x7f06181a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f0617fff640 1 -- 172.21.15.64:0/4187753763 learned_addr learned my addr 172.21.15.64:0/4187753763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.278+0000 7f061d2b3640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061810db70 0x7f06181ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0617fff640 1 -- 172.21.15.64:0/4187753763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0618108830 msgr2=0x7f06181a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0617fff640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0618108830 0x7f06181a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0617fff640 1 -- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061810db70 msgr2=0x7f06181ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0617fff640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061810db70 0x7f06181ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0617fff640 1 -- 172.21.15.64:0/4187753763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0618110090 con 0x7f061810b1a0 2024-05-09T09:22:33.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0617fff640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f06181a6e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f060c0314e0 tx=0x7f060c003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c03f070 con 0x7f061810b1a0 2024-05-09T09:22:33.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f060c003e40 con 0x7f061810b1a0 2024-05-09T09:22:33.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.279+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c035030 con 0x7f061810b1a0 2024-05-09T09:22:33.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.282+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0618110320 con 0x7f061810b1a0 2024-05-09T09:22:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.282+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0618110790 con 0x7f061810b1a0 2024-05-09T09:22:33.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.284+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06180711d0 con 0x7f061810b1a0 2024-05-09T09:22:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.287+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f060c042070 con 0x7f061810b1a0 2024-05-09T09:22:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.288+0000 7f0615ffb640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06000a2ce0 0x7f06000a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.288+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f060c0eff00 con 0x7f061810b1a0 2024-05-09T09:22:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.291+0000 7f061cab2640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06000a2ce0 0x7f06000a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.292+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f060c049030 con 0x7f061810b1a0 2024-05-09T09:22:33.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.292+0000 7f061cab2640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06000a2ce0 0x7f06000a51a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f060800bbe0 tx=0x7f060800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.742+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f061802d3e0 con 0x7f061810b1a0 2024-05-09T09:22:33.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.743+0000 7f0615ffb640 1 -- 172.21.15.64:0/4187753763 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11883 (secure 0 0 0) 0x7f060c0bca70 con 0x7f061810b1a0 2024-05-09T09:22:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:22:33.743 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:21:55.091434+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24503,24496]},"id":1}]} 2024-05-09T09:22:33.744 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 48 2024-05-09T09:22:33.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.748+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06000a2ce0 msgr2=0x7f06000a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.748+0000 7f061ed16640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06000a2ce0 0x7f06000a51a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f060800bbe0 tx=0x7f060800b390 comp rx=0 tx=0).stop 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.748+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 msgr2=0x7f06181a6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.748+0000 7f061ed16640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f06181a6e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f060c0314e0 tx=0x7f060c003cd0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 shutdown_connections 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06000a2ce0 0x7f06000a51a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061810db70 0x7f06181ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f061810b1a0 0x7f06181a6e20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 --2- 172.21.15.64:0/4187753763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0618108830 0x7f06181a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 >> 172.21.15.64:0/4187753763 conn(0x7f061806e6d0 msgr2=0x7f0618109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 shutdown_connections 2024-05-09T09:22:33.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.749+0000 7f061ed16640 1 -- 172.21.15.64:0/4187753763 wait complete. 2024-05-09T09:22:33.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:22:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.965+0000 7f204eaee640 1 -- 172.21.15.64:0/617636060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048107e90 msgr2=0x7f20481082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.965+0000 7f204eaee640 1 --2- 172.21.15.64:0/617636060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048107e90 0x7f20481082b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2040009be0 tx=0x7f204002a6d0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 -- 172.21.15.64:0/617636060 shutdown_connections 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 --2- 172.21.15.64:0/617636060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20481099a0 0x7f20481102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 --2- 172.21.15.64:0/617636060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048108fe0 0x7f2048109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 --2- 172.21.15.64:0/617636060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048107e90 0x7f20481082b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 -- 172.21.15.64:0/617636060 >> 172.21.15.64:0/617636060 conn(0x7f2048076e00 msgr2=0x7f2048079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 -- 172.21.15.64:0/617636060 shutdown_connections 2024-05-09T09:22:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.969+0000 7f204eaee640 1 -- 172.21.15.64:0/617636060 wait complete. 2024-05-09T09:22:33.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 Processor -- start 2024-05-09T09:22:33.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 -- start start 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048107e90 0x7f2048195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 0x7f20481963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20481099a0 0x7f204819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f204806dd80 con 0x7f2048107e90 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f204806dc00 con 0x7f20481099a0 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f204eaee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f204806df00 con 0x7f2048108fe0 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f2047fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 0x7f20481963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f2047fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 0x7f20481963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49978/0 (socket says 172.21.15.64:49978) 2024-05-09T09:22:33.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.971+0000 7f2047fff640 1 -- 172.21.15.64:0/1249704361 learned_addr learned my addr 172.21.15.64:0/1249704361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f204d08b640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20481099a0 0x7f204819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f204c88a640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048107e90 0x7f2048195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f2047fff640 1 -- 172.21.15.64:0/1249704361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20481099a0 msgr2=0x7f204819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f2047fff640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20481099a0 0x7f204819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f2047fff640 1 -- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048107e90 msgr2=0x7f2048195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f2047fff640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048107e90 0x7f2048195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f2047fff640 1 -- 172.21.15.64:0/1249704361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204819d560 con 0x7f2048108fe0 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f204c88a640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048107e90 0x7f2048195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:33.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.972+0000 7f2047fff640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 0x7f20481963b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f203801fa70 tx=0x7f203801ff30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.973+0000 7f2045ffb640 1 -- 172.21.15.64:0/1249704361 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203801dc50 con 0x7f2048108fe0 2024-05-09T09:22:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.973+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204819d850 con 0x7f2048108fe0 2024-05-09T09:22:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.974+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204819dda0 con 0x7f2048108fe0 2024-05-09T09:22:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.977+0000 7f2045ffb640 1 -- 172.21.15.64:0/1249704361 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f203801ddf0 con 0x7f2048108fe0 2024-05-09T09:22:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.977+0000 7f2045ffb640 1 -- 172.21.15.64:0/1249704361 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038021440 con 0x7f2048108fe0 2024-05-09T09:22:33.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.977+0000 7f20517f6740 1 Processor -- start 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.978+0000 7f20517f6740 1 -- start start 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f20517f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd12593370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f20517f6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd12598440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f20517f6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd1259af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f20517f6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd124e6c40 con 0x55cd12598ab0 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f20517f6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cd1248e580 con 0x55cd12593940 2024-05-09T09:22:33.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f20517f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cd124e2c00 con 0x55cd12592f50 2024-05-09T09:22:33.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f204c88a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd12598440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f204c88a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd12598440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51928/0 (socket says 172.21.15.64:51928) 2024-05-09T09:22:33.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.979+0000 7f204c88a640 1 -- 172.21.15.64:0/4045151746 learned_addr learned my addr 172.21.15.64:0/4045151746 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:33.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.980+0000 7f204d08b640 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd12593370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.981+0000 7f204d08b640 1 -- 172.21.15.64:0/4045151746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 msgr2=0x55cd12598440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:33.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.981+0000 7f204d08b640 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd12598440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.981+0000 7f204d08b640 1 -- 172.21.15.64:0/4045151746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 msgr2=0x55cd1259af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:33.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.981+0000 7f204d08b640 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd1259af40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.981+0000 7f204d08b640 1 -- 172.21.15.64:0/4045151746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd121fa3f0 con 0x55cd12592f50 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.981+0000 7f204d08b640 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd12593370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f20300045b0 tx=0x7f203000bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5abf175004678f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.982+0000 7f203effd640 1 -- 172.21.15.64:0/4045151746 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030003bd0 con 0x55cd12592f50 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.982+0000 7f203effd640 1 -- 172.21.15.64:0/4045151746 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2030015070 con 0x55cd12592f50 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.982+0000 7f203effd640 1 -- 172.21.15.64:0/4045151746 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300106a0 con 0x55cd12592f50 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 -- 172.21.15.64:0/4045151746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 msgr2=0x55cd12593370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd12593370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f20300045b0 tx=0x7f203000bb10 comp rx=0 tx=0).stop 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 -- 172.21.15.64:0/4045151746 shutdown_connections 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd1259af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd12598440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 --2- 172.21.15.64:0/4045151746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd12593370 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.983+0000 7f20517f6740 1 -- 172.21.15.64:0/4045151746 >> 172.21.15.64:0/4045151746 conn(0x55cd1258e5c0 msgr2=0x55cd1258e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.984+0000 7f2045ffb640 1 -- 172.21.15.64:0/1249704361 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f20380215e0 con 0x7f2048108fe0 2024-05-09T09:22:33.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.984+0000 7f20517f6740 1 -- 172.21.15.64:0/4045151746 shutdown_connections 2024-05-09T09:22:33.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.985+0000 7f20517f6740 1 -- 172.21.15.64:0/4045151746 wait complete. 2024-05-09T09:22:33.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.985+0000 7f2045ffb640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20340a2ca0 0x7f20340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.985+0000 7f20517f6740 1 -- start start 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.985+0000 7f204d08b640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20340a2ca0 0x7f20340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.985+0000 7f20517f6740 1 Processor -- start 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.985+0000 7f2045ffb640 1 -- 172.21.15.64:0/1249704361 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20380e0ad0 con 0x7f2048108fe0 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd1259d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd125a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd1259ba60 con 0x55cd12593940 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cd124a1c40 con 0x55cd12598ab0 2024-05-09T09:22:33.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cd1259bbe0 con 0x55cd12592f50 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f20517f6740 10 client.0 _ll_register_callbacks cb 0x55cd121f9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f204d08b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd125a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f204d08b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd125a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51936/0 (socket says 172.21.15.64:51936) 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.986+0000 7f204d08b640 1 -- 172.21.15.64:0/4080352472 learned_addr learned my addr 172.21.15.64:0/4080352472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20340a2ca0 0x7f20340a5160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2030004370 tx=0x7f2030011b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 -- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 msgr2=0x55cd1259d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259d440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 -- 172.21.15.64:0/4080352472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 msgr2=0x55cd1259d980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd1259d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 -- 172.21.15.64:0/4080352472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd122c47d0 con 0x55cd12598ab0 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.987+0000 7f204d08b640 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd125a49c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2030019000 tx=0x7f203000bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea0bd781b6ee32f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.988+0000 7f2027fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:33.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.988+0000 7f2027fff640 1 -- 172.21.15.64:0/4080352472 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203002b040 con 0x55cd12598ab0 2024-05-09T09:22:33.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.988+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2030015070 con 0x55cd12598ab0 2024-05-09T09:22:33.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.988+0000 7f2027fff640 1 -- 172.21.15.64:0/4080352472 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2030005020 con 0x55cd12598ab0 2024-05-09T09:22:33.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.988+0000 7f2027fff640 1 -- 172.21.15.64:0/4080352472 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f203003c050 con 0x55cd12598ab0 2024-05-09T09:22:33.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.988+0000 7f20517f6740 10 client.28172 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:33.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cd1257fef0 con 0x55cd12598ab0 2024-05-09T09:22:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f2027fff640 1 -- 172.21.15.64:0/4080352472 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2030015070 con 0x55cd12598ab0 2024-05-09T09:22:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f2027fff640 1 -- 172.21.15.64:0/4080352472 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f2030025480 con 0x55cd12598ab0 2024-05-09T09:22:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 10 client.28172 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 10 client.28172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:22:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 20 client.28172 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:33.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12581a70 0x55cd12583f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 4 client.28172 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.989+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd124f4370 con 0x55cd12581a70 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.990+0000 7f204c88a640 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12581a70 0x55cd12583f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.991+0000 7f204c88a640 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12581a70 0x55cd12583f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41855d2baf3b7ae1 server_cookie=14601f7113e04037 in_seq=0 out_seq=0 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.991+0000 7f2027fff640 10 client.28172 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.991+0000 7f2027fff640 1 -- 172.21.15.64:0/4080352472 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd124f4370 con 0x55cd12581a70 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.991+0000 7f2027fff640 10 client.28172 handle_command_reply: tid=0 2024-05-09T09:22:33.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.992+0000 7f20517f6740 1 client.28172 shutdown 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 msgr2=0x55cd125a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd125a49c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2030019000 tx=0x7f203000bb10 comp rx=0 tx=0).stop 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 shutdown_connections 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12581a70 0x55cd12583f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd125a49c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd1259d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 --2- 172.21.15.64:0/4080352472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 >> 172.21.15.64:0/4080352472 conn(0x55cd1258e5c0 msgr2=0x55cd1257c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:33.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f2027fff640 0 client.28172 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:33.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.993+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 shutdown_connections 2024-05-09T09:22:33.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.994+0000 7f20517f6740 1 -- 172.21.15.64:0/4080352472 wait complete. 2024-05-09T09:22:33.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:33.994+0000 7f20517f6740 20 client.28172 trim_cache size 0 max 16384 2024-05-09T09:22:34.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.022+0000 7f20517f6740 1 Processor -- start 2024-05-09T09:22:34.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.025+0000 7f20517f6740 1 -- start start 2024-05-09T09:22:34.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.025+0000 7f20517f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.025+0000 7f20517f6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd125a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.025+0000 7f20517f6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd12591340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.026+0000 7f20517f6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20280125d0 con 0x55cd12598ab0 2024-05-09T09:22:34.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.026+0000 7f20517f6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f202800fe70 con 0x55cd12593940 2024-05-09T09:22:34.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.026+0000 7f20517f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f202800dcc0 con 0x55cd12592f50 2024-05-09T09:22:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.028+0000 7f204d08b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd12591340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.028+0000 7f204d08b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd12591340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54590/0 (socket says 172.21.15.64:54590) 2024-05-09T09:22:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.028+0000 7f204d08b640 1 -- 172.21.15.64:0/2116609370 learned_addr learned my addr 172.21.15.64:0/2116609370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.028+0000 7f204c88a640 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f204d08b640 1 -- 172.21.15.64:0/2116609370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 msgr2=0x55cd1259ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f204d08b640 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f204d08b640 1 -- 172.21.15.64:0/2116609370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 msgr2=0x55cd125a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f2047fff640 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd125a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f204d08b640 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd125a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f204d08b640 1 -- 172.21.15.64:0/2116609370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1256ed20 con 0x55cd12598ab0 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.029+0000 7f204d08b640 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd12591340 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f20300039c0 tx=0x7f203000b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0fe8d34cc6ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.030+0000 7f203e7fc640 1 -- 172.21.15.64:0/2116609370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030044020 con 0x55cd12598ab0 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.030+0000 7f203e7fc640 1 -- 172.21.15.64:0/2116609370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2030015070 con 0x55cd12598ab0 2024-05-09T09:22:34.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.030+0000 7f203e7fc640 1 -- 172.21.15.64:0/2116609370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203002b040 con 0x55cd12598ab0 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 -- 172.21.15.64:0/2116609370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 msgr2=0x55cd12591340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd12591340 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f20300039c0 tx=0x7f203000b440 comp rx=0 tx=0).stop 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 -- 172.21.15.64:0/2116609370 shutdown_connections 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12598ab0 0x55cd12591340 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12593940 0x55cd125a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 --2- 172.21.15.64:0/2116609370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd1259ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.031+0000 7f20517f6740 1 -- 172.21.15.64:0/2116609370 >> 172.21.15.64:0/2116609370 conn(0x55cd1258e5c0 msgr2=0x55cd1259c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.032+0000 7f20517f6740 1 -- 172.21.15.64:0/2116609370 shutdown_connections 2024-05-09T09:22:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.032+0000 7f20517f6740 1 -- 172.21.15.64:0/2116609370 wait complete. 2024-05-09T09:22:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.032+0000 7f20517f6740 1 -- start start 2024-05-09T09:22:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.032+0000 7f20517f6740 1 Processor -- start 2024-05-09T09:22:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd125a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd125a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd1261eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f202800e330 con 0x55cd12593940 2024-05-09T09:22:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f202800f7a0 con 0x55cd12598ab0 2024-05-09T09:22:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f202800ea00 con 0x55cd12592f50 2024-05-09T09:22:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f20517f6740 10 client.0 _ll_register_callbacks cb 0x7f2030028040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:34.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f2047fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd125a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f2047fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd125a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54600/0 (socket says 172.21.15.64:54600) 2024-05-09T09:22:34.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f2047fff640 1 -- 172.21.15.64:0/3582470346 learned_addr learned my addr 172.21.15.64:0/3582470346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f204d08b640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd1261eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.033+0000 7f204c88a640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd125a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.034+0000 7f204d08b640 1 -- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 msgr2=0x55cd125a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.034+0000 7f204d08b640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd125a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.034+0000 7f204d08b640 1 -- 172.21.15.64:0/3582470346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 msgr2=0x55cd125a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.034+0000 7f204d08b640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd125a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.034+0000 7f204d08b640 1 -- 172.21.15.64:0/3582470346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1261f1e0 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.034+0000 7f204d08b640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd1261eca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2030025a60 tx=0x7f2030011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f354b066accb71a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.035+0000 7f2027fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.035+0000 7f2027fff640 1 -- 172.21.15.64:0/3582470346 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203004d020 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.035+0000 7f2027fff640 1 -- 172.21.15.64:0/3582470346 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2030005040 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.035+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2030015070 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.036+0000 7f2027fff640 1 -- 172.21.15.64:0/3582470346 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2030055050 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.037+0000 7f20517f6740 10 client.28175 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.037+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cd12620310 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f2027fff640 1 -- 172.21.15.64:0/3582470346 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2030015070 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f2027fff640 1 -- 172.21.15.64:0/3582470346 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f2030049450 con 0x55cd12598ab0 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f20517f6740 10 client.28175 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f20517f6740 10 client.28175 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:22:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f20517f6740 20 client.28175 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:34.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f20517f6740 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12620c60 0x55cd12623080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f20517f6740 4 client.28175 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:22:34.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd124ee6c0 con 0x55cd12620c60 2024-05-09T09:22:34.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.038+0000 7f204c88a640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12620c60 0x55cd12623080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.039+0000 7f2027fff640 10 client.28175 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:34.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.041+0000 7f204c88a640 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12620c60 0x55cd12623080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8f339246c90038c server_cookie=7422bdb7d921be8f in_seq=0 out_seq=0 2024-05-09T09:22:34.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.041+0000 7f2027fff640 1 -- 172.21.15.64:0/3582470346 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd124ee6c0 con 0x55cd12620c60 2024-05-09T09:22:34.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.041+0000 7f2027fff640 10 client.28175 handle_command_reply: tid=0 2024-05-09T09:22:34.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.041+0000 7f20517f6740 1 client.28175 shutdown 2024-05-09T09:22:34.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 msgr2=0x55cd1261eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd1261eca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2030025a60 tx=0x7f2030011e60 comp rx=0 tx=0).stop 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 shutdown_connections 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd12620c60 0x55cd12623080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd12598ab0 0x55cd1261eca0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd12593940 0x55cd125a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 --2- 172.21.15.64:0/3582470346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd12592f50 0x55cd125a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 >> 172.21.15.64:0/3582470346 conn(0x55cd1258e5c0 msgr2=0x55cd1257bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f2027fff640 0 client.28175 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 shutdown_connections 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.042+0000 7f20517f6740 1 -- 172.21.15.64:0/3582470346 wait complete. 2024-05-09T09:22:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.043+0000 7f20517f6740 20 client.28175 trim_cache size 0 max 16384 2024-05-09T09:22:34.046 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:22:34.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.048+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20340a2ca0 msgr2=0x7f20340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.048+0000 7f204eaee640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20340a2ca0 0x7f20340a5160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2030004370 tx=0x7f2030011b00 comp rx=0 tx=0).stop 2024-05-09T09:22:34.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.048+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 msgr2=0x7f20481963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.048+0000 7f204eaee640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 0x7f20481963b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f203801fa70 tx=0x7f203801ff30 comp rx=0 tx=0).stop 2024-05-09T09:22:34.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.051+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 shutdown_connections 2024-05-09T09:22:34.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.051+0000 7f204eaee640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f20340a2ca0 0x7f20340a5160 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.051+0000 7f204eaee640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20481099a0 0x7f204819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.051+0000 7f204eaee640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2048108fe0 0x7f20481963b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.051+0000 7f204eaee640 1 --2- 172.21.15.64:0/1249704361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2048107e90 0x7f2048195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.051+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 >> 172.21.15.64:0/1249704361 conn(0x7f2048076e00 msgr2=0x7f20480708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.056+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 shutdown_connections 2024-05-09T09:22:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.056+0000 7f204eaee640 1 -- 172.21.15.64:0/1249704361 wait complete. 2024-05-09T09:22:34.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:33 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4187753763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:34.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:33 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4187753763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:34.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:22:34.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:33 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4187753763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:22:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 -- 172.21.15.64:0/643595273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97701100b0 msgr2=0x7f97701124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 --2- 172.21.15.64:0/643595273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97701100b0 0x7f97701124a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9768009890 tx=0x7f976802a380 comp rx=0 tx=0).stop 2024-05-09T09:22:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 -- 172.21.15.64:0/643595273 shutdown_connections 2024-05-09T09:22:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 --2- 172.21.15.64:0/643595273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97701100b0 0x7f97701124a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 --2- 172.21.15.64:0/643595273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9770109190 0x7f977010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 --2- 172.21.15.64:0/643595273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9770108830 0x7f9770108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.300+0000 7f9774c15640 1 -- 172.21.15.64:0/643595273 >> 172.21.15.64:0/643595273 conn(0x7f977006ea60 msgr2=0x7f9770070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.301+0000 7f9774c15640 1 -- 172.21.15.64:0/643595273 shutdown_connections 2024-05-09T09:22:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.301+0000 7f9774c15640 1 -- 172.21.15.64:0/643595273 wait complete. 2024-05-09T09:22:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.302+0000 7f9774c15640 1 Processor -- start 2024-05-09T09:22:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.302+0000 7f9774c15640 1 -- start start 2024-05-09T09:22:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f9774c15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 0x7f97701a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f9774c15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9770109190 0x7f97701a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f976e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 0x7f97701a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f9774c15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97701100b0 0x7f97701a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f9774c15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9770115b90 con 0x7f9770109190 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f9774c15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9770115a10 con 0x7f9770108830 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f9774c15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9770115d10 con 0x7f97701100b0 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f976e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 0x7f97701a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51980/0 (socket says 172.21.15.64:51980) 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.303+0000 7f976e59c640 1 -- 172.21.15.64:0/3311863014 learned_addr learned my addr 172.21.15.64:0/3311863014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976ed9d640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97701100b0 0x7f97701a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976e59c640 1 -- 172.21.15.64:0/3311863014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97701100b0 msgr2=0x7f97701a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976e59c640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97701100b0 0x7f97701a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976e59c640 1 -- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9770109190 msgr2=0x7f97701a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976e59c640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9770109190 0x7f97701a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976e59c640 1 -- 172.21.15.64:0/3311863014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97701099c0 con 0x7f9770108830 2024-05-09T09:22:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f976e59c640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 0x7f97701a02f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9760009d50 tx=0x7f976000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.304+0000 7f975f7fe640 1 -- 172.21.15.64:0/3311863014 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760003b70 con 0x7f9770108830 2024-05-09T09:22:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.305+0000 7f975f7fe640 1 -- 172.21.15.64:0/3311863014 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9760003d10 con 0x7f9770108830 2024-05-09T09:22:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.305+0000 7f975f7fe640 1 -- 172.21.15.64:0/3311863014 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600108f0 con 0x7f9770108830 2024-05-09T09:22:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.305+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9770109cb0 con 0x7f9770108830 2024-05-09T09:22:34.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.305+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977010a1b0 con 0x7f9770108830 2024-05-09T09:22:34.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.309+0000 7f975f7fe640 1 -- 172.21.15.64:0/3311863014 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9760003ec0 con 0x7f9770108830 2024-05-09T09:22:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.310+0000 7f975f7fe640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97580a2c90 0x7f97580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.310+0000 7f975f7fe640 1 -- 172.21.15.64:0/3311863014 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f97600cfaf0 con 0x7f9770108830 2024-05-09T09:22:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.311+0000 7f976dd9b640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97580a2c90 0x7f97580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.312+0000 7f976dd9b640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97580a2c90 0x7f97580a5150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f97701a17b0 tx=0x7f9764009660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.313+0000 7f9777a03740 1 Processor -- start 2024-05-09T09:22:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 -- start start 2024-05-09T09:22:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531bdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558253114c40 con 0x5582531b8750 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5582530bc580 con 0x5582531be2b0 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f9777a03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558253110c00 con 0x5582531b9140 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f976ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f976ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531c0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51982/0 (socket says 172.21.15.64:51982) 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f976ed9d640 1 -- 172.21.15.64:0/2084527980 learned_addr learned my addr 172.21.15.64:0/2084527980 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f976e59c640 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.314+0000 7f976dd9b640 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531bdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.315+0000 7f976ed9d640 1 -- 172.21.15.64:0/2084527980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 msgr2=0x5582531bdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.315+0000 7f976ed9d640 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531bdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.315+0000 7f976ed9d640 1 -- 172.21.15.64:0/2084527980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 msgr2=0x5582531b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.315+0000 7f976ed9d640 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.315+0000 7f976ed9d640 1 -- 172.21.15.64:0/2084527980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558252e283f0 con 0x5582531be2b0 2024-05-09T09:22:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.315+0000 7f976ed9d640 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531c0740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f97680057c0 tx=0x7f9768005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd4af4ae12e6ae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.317+0000 7f975cff9640 1 -- 172.21.15.64:0/2084527980 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768037040 con 0x5582531be2b0 2024-05-09T09:22:34.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.317+0000 7f975cff9640 1 -- 172.21.15.64:0/2084527980 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9768038040 con 0x5582531be2b0 2024-05-09T09:22:34.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.317+0000 7f975cff9640 1 -- 172.21.15.64:0/2084527980 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680038e0 con 0x5582531be2b0 2024-05-09T09:22:34.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 -- 172.21.15.64:0/2084527980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 msgr2=0x5582531c0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531c0740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f97680057c0 tx=0x7f9768005b20 comp rx=0 tx=0).stop 2024-05-09T09:22:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 -- 172.21.15.64:0/2084527980 shutdown_connections 2024-05-09T09:22:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531c0740 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531bdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 --2- 172.21.15.64:0/2084527980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.318+0000 7f9777a03740 1 -- 172.21.15.64:0/2084527980 >> 172.21.15.64:0/2084527980 conn(0x5582531b3dc0 msgr2=0x5582531b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.322+0000 7f9777a03740 1 -- 172.21.15.64:0/2084527980 shutdown_connections 2024-05-09T09:22:34.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.322+0000 7f9777a03740 1 -- 172.21.15.64:0/2084527980 wait complete. 2024-05-09T09:22:34.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 -- start start 2024-05-09T09:22:34.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 Processor -- start 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5582531c1250 con 0x5582531b8750 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5582530cfc40 con 0x5582531be2b0 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5582531c13d0 con 0x5582531b9140 2024-05-09T09:22:34.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.323+0000 7f9777a03740 10 client.0 _ll_register_callbacks cb 0x558252e27be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.324+0000 7f976e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.324+0000 7f976e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54634/0 (socket says 172.21.15.64:54634) 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.324+0000 7f976e59c640 1 -- 172.21.15.64:0/1472413185 learned_addr learned my addr 172.21.15.64:0/1472413185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.324+0000 7f976ed9d640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.324+0000 7f976dd9b640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f976ed9d640 1 -- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 msgr2=0x5582531c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f976ed9d640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f976ed9d640 1 -- 172.21.15.64:0/1472413185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 msgr2=0x5582531c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f976ed9d640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f976ed9d640 1 -- 172.21.15.64:0/1472413185 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558252ef27d0 con 0x5582531be2b0 2024-05-09T09:22:34.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f976ed9d640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531ca0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9768009860 tx=0x7f9768030ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4c41dad1bfe520 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.325+0000 7f9755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.326+0000 7f9755ffb640 1 -- 172.21.15.64:0/1472413185 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768037040 con 0x5582531be2b0 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.326+0000 7f9755ffb640 1 -- 172.21.15.64:0/1472413185 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9768005030 con 0x5582531be2b0 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.326+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768038040 con 0x5582531be2b0 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.326+0000 7f9755ffb640 1 -- 172.21.15.64:0/1472413185 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9768005230 con 0x5582531be2b0 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.326+0000 7f9777a03740 10 client.28187 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.326+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5582531cd410 con 0x5582531be2b0 2024-05-09T09:22:34.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9755ffb640 1 -- 172.21.15.64:0/1472413185 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9768038040 con 0x5582531be2b0 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9755ffb640 1 -- 172.21.15.64:0/1472413185 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f9768032440 con 0x5582531be2b0 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9777a03740 10 client.28187 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9777a03740 10 client.28187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9777a03740 20 client.28187 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9777a03740 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ce760 0x5582531d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9777a03740 4 client.28187 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558253122370 con 0x5582531ce760 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.327+0000 7f976e59c640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ce760 0x5582531d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.328+0000 7f9755ffb640 10 client.28187 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:34.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.328+0000 7f976e59c640 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ce760 0x5582531d0b50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f154b4221ac9353 server_cookie=e45c74abd2713873 in_seq=0 out_seq=0 2024-05-09T09:22:34.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.329+0000 7f9755ffb640 1 -- 172.21.15.64:0/1472413185 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558253122370 con 0x5582531ce760 2024-05-09T09:22:34.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.329+0000 7f9755ffb640 10 client.28187 handle_command_reply: tid=0 2024-05-09T09:22:34.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.329+0000 7f9777a03740 1 client.28187 shutdown 2024-05-09T09:22:34.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.331+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 msgr2=0x5582531ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.331+0000 7f9777a03740 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531ca0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9768009860 tx=0x7f9768030ec0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 shutdown_connections 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9777a03740 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531be2b0 0x5582531ca0a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9777a03740 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9777a03740 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ce760 0x5582531d0b50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9777a03740 1 --2- 172.21.15.64:0/1472413185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 >> 172.21.15.64:0/1472413185 conn(0x5582531b3dc0 msgr2=0x5582531aa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.334+0000 7f9755ffb640 0 client.28187 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:34.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.335+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 shutdown_connections 2024-05-09T09:22:34.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.335+0000 7f9777a03740 1 -- 172.21.15.64:0/1472413185 wait complete. 2024-05-09T09:22:34.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.335+0000 7f9777a03740 20 client.28187 trim_cache size 0 max 16384 2024-05-09T09:22:34.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.368+0000 7f9777a03740 1 Processor -- start 2024-05-09T09:22:34.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.368+0000 7f9777a03740 1 -- start start 2024-05-09T09:22:34.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.368+0000 7f9777a03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.368+0000 7f9777a03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.368+0000 7f9777a03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f9777a03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97701b34c0 con 0x5582531be2b0 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f9777a03740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97701b0d60 con 0x5582531b8750 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f976dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f976dd9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50036/0 (socket says 172.21.15.64:50036) 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f976dd9b640 1 -- 172.21.15.64:0/617385745 learned_addr learned my addr 172.21.15.64:0/617385745 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f976ed9d640 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f9777a03740 1 -- 172.21.15.64:0/617385745 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97701aebb0 con 0x5582531b9140 2024-05-09T09:22:34.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.369+0000 7f976e59c640 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.370+0000 7f976ed9d640 1 -- 172.21.15.64:0/617385745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 msgr2=0x5582531ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.370+0000 7f976ed9d640 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.370+0000 7f976ed9d640 1 -- 172.21.15.64:0/617385745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 msgr2=0x5582531c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.370+0000 7f976ed9d640 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.370+0000 7f976ed9d640 1 -- 172.21.15.64:0/617385745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582531c0b00 con 0x5582531be2b0 2024-05-09T09:22:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.371+0000 7f976ed9d640 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c3060 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9768032a00 tx=0x7f9768005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ba93985777c5a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.372+0000 7f9757fff640 1 -- 172.21.15.64:0/617385745 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768037070 con 0x5582531be2b0 2024-05-09T09:22:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.372+0000 7f9757fff640 1 -- 172.21.15.64:0/617385745 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9768005850 con 0x5582531be2b0 2024-05-09T09:22:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.372+0000 7f9757fff640 1 -- 172.21.15.64:0/617385745 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976804b070 con 0x5582531be2b0 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.373+0000 7f9777a03740 1 -- 172.21.15.64:0/617385745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 msgr2=0x5582531c3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.373+0000 7f9777a03740 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c3060 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9768032a00 tx=0x7f9768005b20 comp rx=0 tx=0).stop 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.374+0000 7f9777a03740 1 -- 172.21.15.64:0/617385745 shutdown_connections 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.374+0000 7f9777a03740 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c3060 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.374+0000 7f9777a03740 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.374+0000 7f9777a03740 1 --2- 172.21.15.64:0/617385745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.374+0000 7f9777a03740 1 -- 172.21.15.64:0/617385745 >> 172.21.15.64:0/617385745 conn(0x5582531b3dc0 msgr2=0x5582531d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.378+0000 7f9777a03740 1 -- 172.21.15.64:0/617385745 shutdown_connections 2024-05-09T09:22:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.379+0000 7f9777a03740 1 -- 172.21.15.64:0/617385745 wait complete. 2024-05-09T09:22:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.379+0000 7f9777a03740 1 -- start start 2024-05-09T09:22:34.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.379+0000 7f9777a03740 1 Processor -- start 2024-05-09T09:22:34.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.380+0000 7f9777a03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.380+0000 7f9777a03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.380+0000 7f9777a03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.380+0000 7f9777a03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97701af220 con 0x5582531be2b0 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f9777a03740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97701b0690 con 0x5582531b8750 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f9777a03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97701af8f0 con 0x5582531b9140 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f9777a03740 10 client.0 _ll_register_callbacks cb 0x7f9760009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f976ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f976ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54646/0 (socket says 172.21.15.64:54646) 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f976ed9d640 1 -- 172.21.15.64:0/4180026327 learned_addr learned my addr 172.21.15.64:0/4180026327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f976e59c640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.381+0000 7f976dd9b640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976e59c640 1 -- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 msgr2=0x5582531c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976e59c640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976e59c640 1 -- 172.21.15.64:0/4180026327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 msgr2=0x5582531c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976e59c640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976e59c640 1 -- 172.21.15.64:0/4180026327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582531c8ed0 con 0x5582531b8750 2024-05-09T09:22:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976ed9d640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:34.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976dd9b640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:34.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f976e59c640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c7390 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9760009d20 tx=0x7f97600bc160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=162d0694f3627e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f9755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:34.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.382+0000 7f9755ffb640 1 -- 172.21.15.64:0/4180026327 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600a0310 con 0x5582531b8750 2024-05-09T09:22:34.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.383+0000 7f9755ffb640 1 -- 172.21.15.64:0/4180026327 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f97600045f0 con 0x5582531b8750 2024-05-09T09:22:34.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.383+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768005850 con 0x5582531b8750 2024-05-09T09:22:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.389+0000 7f9755ffb640 1 -- 172.21.15.64:0/4180026327 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f97600a4bd0 con 0x5582531b8750 2024-05-09T09:22:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.389+0000 7f9777a03740 10 client.28193 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.389+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5582531ca240 con 0x5582531b8750 2024-05-09T09:22:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9755ffb640 1 -- 172.21.15.64:0/4180026327 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9768005850 con 0x5582531b8750 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9755ffb640 1 -- 172.21.15.64:0/4180026327 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f9760085020 con 0x5582531b8750 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9777a03740 10 client.28193 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9777a03740 10 client.28193 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9777a03740 20 client.28193 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9777a03740 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ab5a0 0x5582531ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9777a03740 4 client.28193 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:22:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.390+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582531c2910 con 0x5582531ab5a0 2024-05-09T09:22:34.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.393+0000 7f976dd9b640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ab5a0 0x5582531ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.394+0000 7f976dd9b640 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ab5a0 0x5582531ad9c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a31067d987e505d3 server_cookie=e7d9de1601a33451 in_seq=0 out_seq=0 2024-05-09T09:22:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.394+0000 7f9755ffb640 10 client.28193 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:34.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.396+0000 7f9755ffb640 1 -- 172.21.15.64:0/4180026327 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582531c2910 con 0x5582531ab5a0 2024-05-09T09:22:34.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.396+0000 7f9755ffb640 10 client.28193 handle_command_reply: tid=0 2024-05-09T09:22:34.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.397+0000 7f9777a03740 1 client.28193 shutdown 2024-05-09T09:22:34.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.400+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 msgr2=0x5582531c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.400+0000 7f9777a03740 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c7390 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9760009d20 tx=0x7f97600bc160 comp rx=0 tx=0).stop 2024-05-09T09:22:34.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 shutdown_connections 2024-05-09T09:22:34.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9777a03740 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5582531ab5a0 0x5582531ad9c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9777a03740 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582531be2b0 0x5582531c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9777a03740 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582531b9140 0x5582531c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9777a03740 1 --2- 172.21.15.64:0/4180026327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582531b8750 0x5582531c7390 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 >> 172.21.15.64:0/4180026327 conn(0x5582531b3dc0 msgr2=0x5582531d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.401+0000 7f9755ffb640 0 client.28193 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:22:34.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.402+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 shutdown_connections 2024-05-09T09:22:34.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.403+0000 7f9777a03740 1 -- 172.21.15.64:0/4180026327 wait complete. 2024-05-09T09:22:34.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.405+0000 7f9777a03740 20 client.28193 trim_cache size 0 max 16384 2024-05-09T09:22:34.406 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:22:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97580a2c90 msgr2=0x7f97580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97580a2c90 0x7f97580a5150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f97701a17b0 tx=0x7f9764009660 comp rx=0 tx=0).stop 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 msgr2=0x7f97701a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 0x7f97701a02f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9760009d50 tx=0x7f976000c6a0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 shutdown_connections 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97580a2c90 0x7f97580a5150 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97701100b0 0x7f97701a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9770109190 0x7f97701a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 --2- 172.21.15.64:0/3311863014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9770108830 0x7f97701a02f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.409+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 >> 172.21.15.64:0/3311863014 conn(0x7f977006ea60 msgr2=0x7f977010bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.410+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 shutdown_connections 2024-05-09T09:22:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.411+0000 7f9774c15640 1 -- 172.21.15.64:0/3311863014 wait complete. 2024-05-09T09:22:34.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/186844650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a6710 msgr2=0x7f3dc00aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/186844650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a6710 0x7f3dc00aafd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3db801a910 tx=0x7f3db803b430 comp rx=0 tx=0).stop 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/186844650 shutdown_connections 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/186844650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a6710 0x7f3dc00aafd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/186844650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dc00a5d70 0x7f3dc00a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/186844650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a4c40 0x7f3dc00a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.631+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/186844650 >> 172.21.15.64:0/186844650 conn(0x7f3dc00a00d0 msgr2=0x7f3dc00a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.632+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/186844650 shutdown_connections 2024-05-09T09:22:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.632+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/186844650 wait complete. 2024-05-09T09:22:34.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.633+0000 7f3dcc8ed640 1 Processor -- start 2024-05-09T09:22:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.633+0000 7f3dcc8ed640 1 -- start start 2024-05-09T09:22:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dcc8ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dc00a4c40 0x7f3dc00b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dcc8ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a5d70 0x7f3dc00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dcc8ed640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 0x7f3dc0164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dcc8ed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc00ae6f0 con 0x7f3dc00a4c40 2024-05-09T09:22:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dcc8ed640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc00ae570 con 0x7f3dc00a5d70 2024-05-09T09:22:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dcc8ed640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc00ae870 con 0x7f3dc00a6710 2024-05-09T09:22:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 0x7f3dc0164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 0x7f3dc0164270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44734/0 (socket says 172.21.15.64:44734) 2024-05-09T09:22:34.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 -- 172.21.15.64:0/971645237 learned_addr learned my addr 172.21.15.64:0/971645237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc659c640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a5d70 0x7f3dc00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 -- 172.21.15.64:0/971645237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a5d70 msgr2=0x7f3dc00b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a5d70 0x7f3dc00b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 -- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dc00a4c40 msgr2=0x7f3dc00b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dc00a4c40 0x7f3dc00b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.634+0000 7f3dc759e640 1 -- 172.21.15.64:0/971645237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc0164910 con 0x7f3dc00a6710 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.635+0000 7f3dc759e640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 0x7f3dc0164270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3db80048d0 tx=0x7f3db8004900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.635+0000 7f3daffff640 1 -- 172.21.15.64:0/971645237 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8046070 con 0x7f3dc00a6710 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.635+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc0164ba0 con 0x7f3dc00a6710 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.635+0000 7f3daffff640 1 -- 172.21.15.64:0/971645237 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db8013070 con 0x7f3dc00a6710 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.635+0000 7f3daffff640 1 -- 172.21.15.64:0/971645237 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db800ecd0 con 0x7f3dc00a6710 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.635+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc01650e0 con 0x7f3dc00a6710 2024-05-09T09:22:34.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.639+0000 7f3dcfdfc740 1 Processor -- start 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.640+0000 7f3daffff640 1 -- 172.21.15.64:0/971645237 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3db800c040 con 0x7f3dc00a6710 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.641+0000 7f3daffff640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3db00a2c50 0x7f3db00a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.641+0000 7f3dcfdfc740 1 -- start start 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3daffff640 1 -- 172.21.15.64:0/971645237 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3db80ef5a0 con 0x7f3dc00a6710 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2306550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f230b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f230bc90 0x5568f230e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568f226ac40 con 0x5568f230bc90 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5568f2212580 con 0x5568f2306b20 2024-05-09T09:22:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.642+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5568f2266c00 con 0x5568f2306130 2024-05-09T09:22:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2306550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2306550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44748/0 (socket says 172.21.15.64:44748) 2024-05-09T09:22:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc6d9d640 1 -- 172.21.15.64:0/1698925379 learned_addr learned my addr 172.21.15.64:0/1698925379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc659c640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3db00a2c50 0x7f3db00a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc659c640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f230b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc6d9d640 1 -- 172.21.15.64:0/1698925379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 msgr2=0x5568f230b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f230b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc759e640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f230bc90 0x5568f230e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.643+0000 7f3dc6d9d640 1 -- 172.21.15.64:0/1698925379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f230bc90 msgr2=0x5568f230e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f230bc90 0x5568f230e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dc6d9d640 1 -- 172.21.15.64:0/1698925379 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f1f7e3f0 con 0x5568f2306130 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dc759e640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f230bc90 0x5568f230e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2306550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80673a0 tx=0x7f3dc8072c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0884bfca64be4f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dc659c640 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f230b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dad7fa640 1 -- 172.21.15.64:0/1698925379 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8070800 con 0x5568f2306130 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dad7fa640 1 -- 172.21.15.64:0/1698925379 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dc8079020 con 0x5568f2306130 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.644+0000 7f3dad7fa640 1 -- 172.21.15.64:0/1698925379 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8074cc0 con 0x5568f2306130 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.645+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1698925379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 msgr2=0x5568f2306550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.645+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2306550 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80673a0 tx=0x7f3dc8072c20 comp rx=0 tx=0).stop 2024-05-09T09:22:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.645+0000 7f3dc659c640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3db00a2c50 0x7f3db00a5110 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5568f2306f20 tx=0x7f3dbc00b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.645+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1698925379 shutdown_connections 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f230bc90 0x5568f230e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f230b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1698925379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2306550 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1698925379 >> 172.21.15.64:0/1698925379 conn(0x5568f23017a0 msgr2=0x5568f2301ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1698925379 shutdown_connections 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1698925379 wait complete. 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 -- start start 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.646+0000 7f3dcfdfc740 1 Processor -- start 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f23104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2317a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568f230ec40 con 0x5568f2306130 2024-05-09T09:22:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5568f2225c40 con 0x5568f2306b20 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5568f230edc0 con 0x5568f230bc90 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dcfdfc740 10 client.0 _ll_register_callbacks cb 0x5568f1f7dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58270/0 (socket says 172.21.15.64:58270) 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.647+0000 7f3dc659c640 1 -- 172.21.15.64:0/2256004821 learned_addr learned my addr 172.21.15.64:0/2256004821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.648+0000 7f3dc659c640 1 -- 172.21.15.64:0/2256004821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 msgr2=0x5568f2317a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.648+0000 7f3dc659c640 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2317a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.648+0000 7f3dc659c640 1 -- 172.21.15.64:0/2256004821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 msgr2=0x5568f23104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.648+0000 7f3dc659c640 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f23104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.649+0000 7f3dc659c640 1 -- 172.21.15.64:0/2256004821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f20487d0 con 0x5568f2306b20 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.649+0000 7f3dc659c640 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23109e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc012f30 tx=0x7f3dbc00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d1a2423ca3580c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.649+0000 7f3daaff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.653+0000 7f3daaff5640 1 -- 172.21.15.64:0/2256004821 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc010ba0 con 0x5568f2306b20 2024-05-09T09:22:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.653+0000 7f3daaff5640 1 -- 172.21.15.64:0/2256004821 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3dbc010d40 con 0x5568f2306b20 2024-05-09T09:22:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.653+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc8079020 con 0x5568f2306b20 2024-05-09T09:22:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.653+0000 7f3daaff5640 1 -- 172.21.15.64:0/2256004821 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3dbc02f050 con 0x5568f2306b20 2024-05-09T09:22:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.654+0000 7f3dcfdfc740 10 client.28205 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.654+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5568f231adc0 con 0x5568f2306b20 2024-05-09T09:22:34.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.654+0000 7f3daaff5640 1 -- 172.21.15.64:0/2256004821 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dc8079020 con 0x5568f2306b20 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3daaff5640 1 -- 172.21.15.64:0/2256004821 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f3dbc020410 con 0x5568f2306b20 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3dcfdfc740 10 client.28205 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3dcfdfc740 10 client.28205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3dcfdfc740 20 client.28205 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f231c550 0x5568f231e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3dcfdfc740 4 client.28205 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.655+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568f2278370 con 0x5568f231c550 2024-05-09T09:22:34.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.656+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f231c550 0x5568f231e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.656+0000 7f3daaff5640 10 client.28205 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:34.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.656+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f231c550 0x5568f231e9f0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b030c40de30e9d server_cookie=af6c745134e1d38b in_seq=0 out_seq=0 2024-05-09T09:22:34.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.658+0000 7f3daaff5640 1 -- 172.21.15.64:0/2256004821 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568f2278370 con 0x5568f231c550 2024-05-09T09:22:34.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.658+0000 7f3daaff5640 10 client.28205 handle_command_reply: tid=0 2024-05-09T09:22:34.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.658+0000 7f3dcfdfc740 1 client.28205 shutdown 2024-05-09T09:22:34.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 msgr2=0x5568f23109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23109e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc012f30 tx=0x7f3dbc00b390 comp rx=0 tx=0).stop 2024-05-09T09:22:34.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 shutdown_connections 2024-05-09T09:22:34.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f231c550 0x5568f231e9f0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2317a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23109e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2256004821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f23104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.659+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 >> 172.21.15.64:0/2256004821 conn(0x5568f23017a0 msgr2=0x5568f23009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.660+0000 7f3daaff5640 0 client.28205 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:34.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.660+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 shutdown_connections 2024-05-09T09:22:34.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.660+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2256004821 wait complete. 2024-05-09T09:22:34.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.660+0000 7f3dcfdfc740 20 client.28205 trim_cache size 0 max 16384 2024-05-09T09:22:34.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.686+0000 7f3dcfdfc740 1 Processor -- start 2024-05-09T09:22:34.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.686+0000 7f3dcfdfc740 1 -- start start 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f2307ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2308420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0172010 con 0x5568f2306130 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016f8b0 con 0x5568f2306b20 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dcfdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016d700 con 0x5568f230bc90 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f2307ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f2307ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59930/0 (socket says 172.21.15.64:59930) 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dc6d9d640 1 -- 172.21.15.64:0/2863872864 learned_addr learned my addr 172.21.15.64:0/2863872864 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.687+0000 7f3dc759e640 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2308420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc759e640 1 -- 172.21.15.64:0/2863872864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 msgr2=0x5568f23179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc759e640 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc759e640 1 -- 172.21.15.64:0/2863872864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 msgr2=0x5568f2307ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc759e640 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f2307ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc759e640 1 -- 172.21.15.64:0/2863872864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f231abc0 con 0x5568f230bc90 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f2307ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:22:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.688+0000 7f3dc759e640 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2308420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3db80ecd60 tx=0x7f3db8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851f2e3defaf6c04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.690+0000 7f3dacff9640 1 -- 172.21.15.64:0/2863872864 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80d4070 con 0x5568f230bc90 2024-05-09T09:22:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.690+0000 7f3dacff9640 1 -- 172.21.15.64:0/2863872864 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db80054d0 con 0x5568f230bc90 2024-05-09T09:22:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.690+0000 7f3dacff9640 1 -- 172.21.15.64:0/2863872864 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80c46e0 con 0x5568f230bc90 2024-05-09T09:22:34.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2863872864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 msgr2=0x5568f2308420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2308420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3db80ecd60 tx=0x7f3db8005fd0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2863872864 shutdown_connections 2024-05-09T09:22:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f230bc90 0x5568f2308420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f2306b20 0x5568f23179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/2863872864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306130 0x5568f2307ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.691+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2863872864 >> 172.21.15.64:0/2863872864 conn(0x5568f23017a0 msgr2=0x5568f23104a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.692+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2863872864 shutdown_connections 2024-05-09T09:22:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.692+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/2863872864 wait complete. 2024-05-09T09:22:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.693+0000 7f3dcfdfc740 1 -- start start 2024-05-09T09:22:34.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.693+0000 7f3dcfdfc740 1 Processor -- start 2024-05-09T09:22:34.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.694+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2314750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.694+0000 7f3dcfdfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306b20 0x5568f2314c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.694+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2314750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2314750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44772/0 (socket says 172.21.15.64:44772) 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dc6d9d640 1 -- 172.21.15.64:0/1180811210 learned_addr learned my addr 172.21.15.64:0/1180811210 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dc659c640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306b20 0x5568f2314c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f230bc90 0x5568f2380b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016dd70 con 0x5568f2306b20 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016f1e0 con 0x5568f230bc90 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dc759e640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f230bc90 0x5568f2380b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.695+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc016e440 con 0x5568f2306130 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dcfdfc740 10 client.0 _ll_register_callbacks cb 0x7f3dbc012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dc659c640 1 -- 172.21.15.64:0/1180811210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 msgr2=0x5568f2314750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dc659c640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2314750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dc659c640 1 -- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f230bc90 msgr2=0x5568f2380b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dc659c640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f230bc90 0x5568f2380b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dc659c640 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f23152a0 con 0x5568f2306b20 2024-05-09T09:22:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.696+0000 7f3dc659c640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306b20 0x5568f2314c90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00efc0 tx=0x7f3dbc020690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7397681be40c2ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:34.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.697+0000 7f3daaff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:34.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.697+0000 7f3daaff5640 1 -- 172.21.15.64:0/1180811210 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc039020 con 0x5568f2306b20 2024-05-09T09:22:34.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.697+0000 7f3daaff5640 1 -- 172.21.15.64:0/1180811210 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3dbc010aa0 con 0x5568f2306b20 2024-05-09T09:22:34.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.698+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db80054d0 con 0x5568f2306b20 2024-05-09T09:22:34.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.699+0000 7f3daaff5640 1 -- 172.21.15.64:0/1180811210 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3dbc010ce0 con 0x5568f2306b20 2024-05-09T09:22:34.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.699+0000 7f3dcfdfc740 10 client.18738 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:34.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.699+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5568f2382160 con 0x5568f2306b20 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.702+0000 7f3daaff5640 1 -- 172.21.15.64:0/1180811210 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db80054d0 con 0x5568f2306b20 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.702+0000 7f3daaff5640 1 -- 172.21.15.64:0/1180811210 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f3dbc029430 con 0x5568f2306b20 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.702+0000 7f3dcfdfc740 10 client.18738 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.702+0000 7f3dcfdfc740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.702+0000 7f3dcfdfc740 20 client.18738 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.702+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f2382b20 0x5568f2384f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.703+0000 7f3dcfdfc740 4 client.18738 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:22:34.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.703+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568f22726c0 con 0x5568f2382b20 2024-05-09T09:22:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.703+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f2382b20 0x5568f2384f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.704+0000 7f3dc6d9d640 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f2382b20 0x5568f2384f40 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f406d8500b7a9f server_cookie=9e67c93a65fb8870 in_seq=0 out_seq=0 2024-05-09T09:22:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.705+0000 7f3daaff5640 10 client.18738 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.705+0000 7f3daaff5640 1 -- 172.21.15.64:0/1180811210 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568f22726c0 con 0x5568f2382b20 2024-05-09T09:22:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.705+0000 7f3daaff5640 10 client.18738 handle_command_reply: tid=0 2024-05-09T09:22:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.705+0000 7f3dcfdfc740 1 client.18738 shutdown 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306b20 msgr2=0x5568f2314c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306b20 0x5568f2314c90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00efc0 tx=0x7f3dbc020690 comp rx=0 tx=0).stop 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 shutdown_connections 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5568f2382b20 0x5568f2384f40 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5568f230bc90 0x5568f2380b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568f2306b20 0x5568f2314c90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 --2- 172.21.15.64:0/1180811210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f2306130 0x5568f2314750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 >> 172.21.15.64:0/1180811210 conn(0x5568f23017a0 msgr2=0x5568f23104a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3daaff5640 0 client.18738 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 shutdown_connections 2024-05-09T09:22:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.706+0000 7f3dcfdfc740 1 -- 172.21.15.64:0/1180811210 wait complete. 2024-05-09T09:22:34.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.709+0000 7f3dcfdfc740 20 client.18738 trim_cache size 0 max 16384 2024-05-09T09:22:34.710 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:22:34.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.711+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3db00a2c50 msgr2=0x7f3db00a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.711+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3db00a2c50 0x7f3db00a5110 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5568f2306f20 tx=0x7f3dbc00b770 comp rx=0 tx=0).stop 2024-05-09T09:22:34.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.711+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 msgr2=0x7f3dc0164270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:34.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.711+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 0x7f3dc0164270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3db80048d0 tx=0x7f3db8004900 comp rx=0 tx=0).stop 2024-05-09T09:22:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.715+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 shutdown_connections 2024-05-09T09:22:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.715+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3db00a2c50 0x7f3db00a5110 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.715+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dc00a6710 0x7f3dc0164270 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.715+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc00a5d70 0x7f3dc00b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.715+0000 7f3dcc8ed640 1 --2- 172.21.15.64:0/971645237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dc00a4c40 0x7f3dc00b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.715+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 >> 172.21.15.64:0/971645237 conn(0x7f3dc00a00d0 msgr2=0x7f3dc00a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:34.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.718+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 shutdown_connections 2024-05-09T09:22:34.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:34.721+0000 7f3dcc8ed640 1 -- 172.21.15.64:0/971645237 wait complete. 2024-05-09T09:22:35.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:34 smithi139 ceph-mon[33599]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 766 op/s 2024-05-09T09:22:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:34 smithi064 ceph-mon[30149]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 766 op/s 2024-05-09T09:22:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:34 smithi179 ceph-mon[34086]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 766 op/s 2024-05-09T09:22:35.745 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:22:35.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:22:35.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:22:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.940+0000 7fc90e3c1640 1 -- 172.21.15.64:0/1866530897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90810db70 msgr2=0x7fc90810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.940+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/1866530897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90810db70 0x7fc90810e020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009ca0 tx=0x7fc8f802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.941+0000 7fc90e3c1640 1 -- 172.21.15.64:0/1866530897 shutdown_connections 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.941+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/1866530897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc90810db70 0x7fc90810e020 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.941+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/1866530897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810b1a0 0x7fc90810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.941+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/1866530897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc908108830 0x7fc90810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.941+0000 7fc90e3c1640 1 -- 172.21.15.64:0/1866530897 >> 172.21.15.64:0/1866530897 conn(0x7fc90806e6c0 msgr2=0x7fc908070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.942+0000 7fc90e3c1640 1 -- 172.21.15.64:0/1866530897 shutdown_connections 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.942+0000 7fc90e3c1640 1 -- 172.21.15.64:0/1866530897 wait complete. 2024-05-09T09:22:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 Processor -- start 2024-05-09T09:22:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 -- start start 2024-05-09T09:22:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc908108830 0x7fc9081a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90810b1a0 0x7fc9081a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810db70 0x7fc9081ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc90811c1a0 con 0x7fc90810db70 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc90811c020 con 0x7fc908108830 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.943+0000 7fc90e3c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc90811c320 con 0x7fc90810b1a0 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.945+0000 7fc90c95e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810db70 0x7fc9081ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.945+0000 7fc90c95e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810db70 0x7fc9081ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59966/0 (socket says 172.21.15.64:59966) 2024-05-09T09:22:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.945+0000 7fc90c95e640 1 -- 172.21.15.64:0/3275840553 learned_addr learned my addr 172.21.15.64:0/3275840553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:35.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.946+0000 7fc907fff640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc908108830 0x7fc9081a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc9077fe640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90810b1a0 0x7fc9081a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc907fff640 1 -- 172.21.15.64:0/3275840553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90810b1a0 msgr2=0x7fc9081a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc907fff640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90810b1a0 0x7fc9081a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc907fff640 1 -- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810db70 msgr2=0x7fc9081ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc907fff640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810db70 0x7fc9081ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc907fff640 1 -- 172.21.15.64:0/3275840553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc908110090 con 0x7fc908108830 2024-05-09T09:22:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.947+0000 7fc907fff640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc908108830 0x7fc9081a6890 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e9f0 tx=0x7fc90000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:35.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.948+0000 7fc9057fa640 1 -- 172.21.15.64:0/3275840553 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000cd80 con 0x7fc908108830 2024-05-09T09:22:35.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.948+0000 7fc9057fa640 1 -- 172.21.15.64:0/3275840553 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc900005020 con 0x7fc908108830 2024-05-09T09:22:35.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.948+0000 7fc9057fa640 1 -- 172.21.15.64:0/3275840553 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900004a00 con 0x7fc908108830 2024-05-09T09:22:35.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.948+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc908110380 con 0x7fc908108830 2024-05-09T09:22:35.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.949+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc908110880 con 0x7fc908108830 2024-05-09T09:22:35.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.953+0000 7fc9057fa640 1 -- 172.21.15.64:0/3275840553 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc900004c60 con 0x7fc908108830 2024-05-09T09:22:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.955+0000 7fc9111af740 1 Processor -- start 2024-05-09T09:22:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.955+0000 7fc9111af740 1 -- start start 2024-05-09T09:22:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.955+0000 7fc9057fa640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f00a2d90 0x7fc8f00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.956+0000 7fc9057fa640 1 -- 172.21.15.64:0/3275840553 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc9000cf2f0 con 0x7fc908108830 2024-05-09T09:22:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.956+0000 7fc9111af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271fee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.956+0000 7fc9111af740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ff39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.956+0000 7fc9111af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ff64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.956+0000 7fc9111af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b271f52990 con 0x55b271fee4c0 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.957+0000 7fc9111af740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b271efa5b0 con 0x55b271feeeb0 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.957+0000 7fc9111af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b271f4e950 con 0x55b271ff4020 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.957+0000 7fc907fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271fee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.957+0000 7fc907fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271fee8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59982/0 (socket says 172.21.15.64:59982) 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.957+0000 7fc907fff640 1 -- 172.21.15.64:0/2041539380 learned_addr learned my addr 172.21.15.64:0/2041539380 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc907fff640 1 -- 172.21.15.64:0/2041539380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 msgr2=0x55b271ff64b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc907fff640 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ff64b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc907fff640 1 -- 172.21.15.64:0/2041539380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 msgr2=0x55b271ff39b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc907fff640 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ff39b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc907fff640 1 -- 172.21.15.64:0/2041539380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b271d2e7e0 con 0x55b271fee4c0 2024-05-09T09:22:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc907fff640 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271fee8e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc900098130 tx=0x7fc900005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc2aa9743935205 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.958+0000 7fc90c95e640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f00a2d90 0x7fc8f00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.962+0000 7fc8ee7fc640 1 -- 172.21.15.64:0/2041539380 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9000b4070 con 0x55b271fee4c0 2024-05-09T09:22:35.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.962+0000 7fc8ee7fc640 1 -- 172.21.15.64:0/2041539380 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc90009b760 con 0x55b271fee4c0 2024-05-09T09:22:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.963+0000 7fc9111af740 1 -- 172.21.15.64:0/2041539380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 msgr2=0x55b271fee8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.963+0000 7fc9111af740 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271fee8e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc900098130 tx=0x7fc900005540 comp rx=0 tx=0).stop 2024-05-09T09:22:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.964+0000 7fc9111af740 1 -- 172.21.15.64:0/2041539380 shutdown_connections 2024-05-09T09:22:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.965+0000 7fc9111af740 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ff64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.965+0000 7fc9111af740 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ff39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.965+0000 7fc9111af740 1 --2- 172.21.15.64:0/2041539380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271fee8e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.965+0000 7fc9111af740 1 -- 172.21.15.64:0/2041539380 >> 172.21.15.64:0/2041539380 conn(0x55b271fb60a0 msgr2=0x55b271fe9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.965+0000 7fc90c95e640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f00a2d90 0x7fc8f00a5250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8031aa0 tx=0x7fc8f8031450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:35.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.968+0000 7fc9111af740 1 -- 172.21.15.64:0/2041539380 shutdown_connections 2024-05-09T09:22:35.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.968+0000 7fc9111af740 1 -- 172.21.15.64:0/2041539380 wait complete. 2024-05-09T09:22:35.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.969+0000 7fc9111af740 1 -- start start 2024-05-09T09:22:35.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.969+0000 7fc9111af740 1 Processor -- start 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ff8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271ffff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b271ff7020 con 0x55b271ff4020 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b271f0db40 con 0x55b271feeeb0 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b271ff71a0 con 0x55b271fee4c0 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc9111af740 10 client.0 _ll_register_callbacks cb 0x55b271f020f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc907fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc907fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44802/0 (socket says 172.21.15.64:44802) 2024-05-09T09:22:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.970+0000 7fc907fff640 1 -- 172.21.15.64:0/1427364832 learned_addr learned my addr 172.21.15.64:0/1427364832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.971+0000 7fc907fff640 1 -- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 msgr2=0x55b271ff8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.971+0000 7fc907fff640 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ff8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.971+0000 7fc907fff640 1 -- 172.21.15.64:0/1427364832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 msgr2=0x55b271ffff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.971+0000 7fc907fff640 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271ffff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.971+0000 7fc907fff640 1 -- 172.21.15.64:0/1427364832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b271d8ef70 con 0x55b271fee4c0 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.972+0000 7fc907fff640 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff89b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc90000c6a0 tx=0x7fc9000a41f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e9b0f1b918ec7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.972+0000 7fc8e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.972+0000 7fc8e77fe640 1 -- 172.21.15.64:0/1427364832 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9000b9060 con 0x55b271fee4c0 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.972+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc90009b760 con 0x55b271fee4c0 2024-05-09T09:22:35.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.972+0000 7fc8e77fe640 1 -- 172.21.15.64:0/1427364832 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc900004230 con 0x55b271fee4c0 2024-05-09T09:22:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.973+0000 7fc8e77fe640 1 -- 172.21.15.64:0/1427364832 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc9000b4070 con 0x55b271fee4c0 2024-05-09T09:22:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.973+0000 7fc9111af740 10 client.28387 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:35.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.973+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b271c26170 con 0x55b271fee4c0 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc8e77fe640 1 -- 172.21.15.64:0/1427364832 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc90009b760 con 0x55b271fee4c0 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc8e77fe640 1 -- 172.21.15.64:0/1427364832 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fc90009a620 con 0x55b271fee4c0 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc9111af740 10 client.28387 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc9111af740 10 client.28387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc9111af740 20 client.28387 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc9111af740 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b272004670 0x55b272006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc9111af740 4 client.28387 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.974+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b271f600c0 con 0x55b272004670 2024-05-09T09:22:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.975+0000 7fc9077fe640 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b272004670 0x55b272006af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:35.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.976+0000 7fc9077fe640 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b272004670 0x55b272006af0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c12cd40fc7858b5 server_cookie=543262c49f1c7743 in_seq=0 out_seq=0 2024-05-09T09:22:35.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.976+0000 7fc8e77fe640 10 client.28387 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:35.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.977+0000 7fc8e77fe640 1 -- 172.21.15.64:0/1427364832 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b271f600c0 con 0x55b272004670 2024-05-09T09:22:35.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.977+0000 7fc8e77fe640 10 client.28387 handle_command_reply: tid=0 2024-05-09T09:22:35.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.977+0000 7fc9111af740 1 client.28387 shutdown 2024-05-09T09:22:35.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 msgr2=0x55b271ff89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff89b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc90000c6a0 tx=0x7fc9000a41f0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 shutdown_connections 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b272004670 0x55b272006af0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271ffff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ff8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 --2- 172.21.15.64:0/1427364832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff89b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:35.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.978+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 >> 172.21.15.64:0/1427364832 conn(0x55b271fb60a0 msgr2=0x55b271fe9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:35.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.981+0000 7fc8e77fe640 0 client.28387 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:35.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.981+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 shutdown_connections 2024-05-09T09:22:35.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.981+0000 7fc9111af740 1 -- 172.21.15.64:0/1427364832 wait complete. 2024-05-09T09:22:35.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:35.982+0000 7fc9111af740 20 client.28387 trim_cache size 0 max 16384 2024-05-09T09:22:36.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.022+0000 7fc9111af740 1 Processor -- start 2024-05-09T09:22:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.022+0000 7fc9111af740 1 -- start start 2024-05-09T09:22:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.022+0000 7fc9111af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9111af740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271fffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9111af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271fece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9111af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081c44e0 con 0x55b271ff4020 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9111af740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081c1d80 con 0x55b271feeeb0 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9111af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081bfca0 con 0x55b271fee4c0 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc907fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271fffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc907fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44816/0 (socket says 172.21.15.64:44816) 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc907fff640 1 -- 172.21.15.64:0/466360988 learned_addr learned my addr 172.21.15.64:0/466360988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc90c95e640 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271fece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 -- 172.21.15.64:0/466360988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 msgr2=0x55b271ff8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 -- 172.21.15.64:0/466360988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 msgr2=0x55b271fece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271fece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 -- 172.21.15.64:0/466360988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b271ff72e0 con 0x55b271feeeb0 2024-05-09T09:22:36.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.023+0000 7fc9077fe640 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271fffec0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55b271feadf0 tx=0x7fc8fc009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862de401c4f3021c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.025+0000 7fc8edffb640 1 -- 172.21.15.64:0/466360988 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc004050 con 0x55b271feeeb0 2024-05-09T09:22:36.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.025+0000 7fc8edffb640 1 -- 172.21.15.64:0/466360988 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8fc013dc0 con 0x55b271feeeb0 2024-05-09T09:22:36.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.025+0000 7fc8edffb640 1 -- 172.21.15.64:0/466360988 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc019730 con 0x55b271feeeb0 2024-05-09T09:22:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 -- 172.21.15.64:0/466360988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 msgr2=0x55b271fffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271fffec0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55b271feadf0 tx=0x7fc8fc009630 comp rx=0 tx=0).stop 2024-05-09T09:22:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 -- 172.21.15.64:0/466360988 shutdown_connections 2024-05-09T09:22:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271ff4020 0x55b271fece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271fffec0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 --2- 172.21.15.64:0/466360988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271fee4c0 0x55b271ff8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.026+0000 7fc9111af740 1 -- 172.21.15.64:0/466360988 >> 172.21.15.64:0/466360988 conn(0x55b271fb60a0 msgr2=0x55b271ff85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.027+0000 7fc9111af740 1 -- 172.21.15.64:0/466360988 shutdown_connections 2024-05-09T09:22:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.027+0000 7fc9111af740 1 -- 172.21.15.64:0/466360988 wait complete. 2024-05-09T09:22:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.027+0000 7fc9111af740 1 -- start start 2024-05-09T09:22:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 Processor -- start 2024-05-09T09:22:36.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271ffd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ffd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ffe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081c0240 con 0x55b271fee4c0 2024-05-09T09:22:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081c16b0 con 0x55b271feeeb0 2024-05-09T09:22:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081c0910 con 0x55b271ff4020 2024-05-09T09:22:36.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.028+0000 7fc9111af740 10 client.0 _ll_register_callbacks cb 0x7fc9000b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ffd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ffd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58344/0 (socket says 172.21.15.64:58344) 2024-05-09T09:22:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 -- 172.21.15.64:0/3768465466 learned_addr learned my addr 172.21.15.64:0/3768465466 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc907fff640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271ffd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc90c95e640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ffe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 -- 172.21.15.64:0/3768465466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 msgr2=0x55b271ffe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ffe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 -- 172.21.15.64:0/3768465466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 msgr2=0x55b271ffd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271ffd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.029+0000 7fc9077fe640 1 -- 172.21.15.64:0/3768465466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b271ffed80 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc9077fe640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ffd7d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55b272007360 tx=0x7fc8fc005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c57bd09175566 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc8e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3768465466 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc013b20 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3768465466 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8fc004050 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8fc013dc0 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3768465466 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc8fc031050 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc9111af740 10 client.28223 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.030+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b272000160 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3768465466 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8fc013dc0 con 0x55b271feeeb0 2024-05-09T09:22:36.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3768465466 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fc8fc00f040 con 0x55b271feeeb0 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc9111af740 10 client.28223 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc9111af740 10 client.28223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc9111af740 20 client.28223 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc9111af740 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b27206b380 0x55b27206d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc9111af740 4 client.28223 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.031+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b271f5a410 con 0x55b27206b380 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.032+0000 7fc907fff640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b27206b380 0x55b27206d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.032+0000 7fc907fff640 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b27206b380 0x55b27206d7a0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c164c378783369 server_cookie=c497340d335b0ce6 in_seq=0 out_seq=0 2024-05-09T09:22:36.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.032+0000 7fc8e77fe640 10 client.28223 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:36.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.033+0000 7fc8e77fe640 1 -- 172.21.15.64:0/3768465466 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b271f5a410 con 0x55b27206b380 2024-05-09T09:22:36.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.033+0000 7fc8e77fe640 10 client.28223 handle_command_reply: tid=0 2024-05-09T09:22:36.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.034+0000 7fc9111af740 1 client.28223 shutdown 2024-05-09T09:22:36.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 msgr2=0x55b271ffd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ffd7d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55b272007360 tx=0x7fc8fc005c50 comp rx=0 tx=0).stop 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 shutdown_connections 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55b27206b380 0x55b27206d7a0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b271ff4020 0x55b271ffe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b271feeeb0 0x55b271ffd7d0 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55b272007360 tx=0x7fc8fc005c50 comp rx=0 tx=0).stop 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc8e77fe640 0 client.28223 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 --2- 172.21.15.64:0/3768465466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b271fee4c0 0x55b271ffd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.035+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 >> 172.21.15.64:0/3768465466 conn(0x55b271fb60a0 msgr2=0x55b271fe87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.036+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 shutdown_connections 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.036+0000 7fc9111af740 1 -- 172.21.15.64:0/3768465466 wait complete. 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.036+0000 7fc9111af740 20 client.28223 trim_cache size 0 max 16384 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "86fc48f1-f1aa-49a3-8424-a3f634554e56", 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:22:36.037 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:22:36.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f00a2d90 msgr2=0x7fc8f00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f00a2d90 0x7fc8f00a5250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8031aa0 tx=0x7fc8f8031450 comp rx=0 tx=0).stop 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc908108830 msgr2=0x7fc9081a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc908108830 0x7fc9081a6890 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e9f0 tx=0x7fc90000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 shutdown_connections 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8f00a2d90 0x7fc8f00a5250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90810db70 0x7fc9081ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc90810b1a0 0x7fc9081a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 --2- 172.21.15.64:0/3275840553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc908108830 0x7fc9081a6890 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.038+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 >> 172.21.15.64:0/3275840553 conn(0x7fc90806e6c0 msgr2=0x7fc90810c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.039+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 shutdown_connections 2024-05-09T09:22:36.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.039+0000 7fc90e3c1640 1 -- 172.21.15.64:0/3275840553 wait complete. 2024-05-09T09:22:36.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:22:36.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:22:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.276+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2795077030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94100770 msgr2=0x7f5a94100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.276+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2795077030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94100770 0x7f5a94100b90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c01a910 tx=0x7f5a7c03b420 comp rx=0 tx=0).stop 2024-05-09T09:22:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2795077030 shutdown_connections 2024-05-09T09:22:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2795077030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a94102320 0x7f5a94108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2795077030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94101960 0x7f5a94101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2795077030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94100770 0x7f5a94100b90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2795077030 >> 172.21.15.64:0/2795077030 conn(0x7f5a940fbce0 msgr2=0x7f5a940fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2795077030 shutdown_connections 2024-05-09T09:22:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.277+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2795077030 wait complete. 2024-05-09T09:22:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.278+0000 7f5a99cc4640 1 Processor -- start 2024-05-09T09:22:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.278+0000 7f5a99cc4640 1 -- start start 2024-05-09T09:22:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.278+0000 7f5a99cc4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 0x7f5a9406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.278+0000 7f5a99cc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a94101960 0x7f5a9406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.278+0000 7f5a99cc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94102320 0x7f5a9406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.278+0000 7f5a99cc4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94071030 con 0x7f5a94101960 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.279+0000 7f5a99cc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94070eb0 con 0x7f5a94102320 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.279+0000 7f5a99cc4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940711b0 con 0x7f5a94100770 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.279+0000 7f5a937fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 0x7f5a9406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.279+0000 7f5a937fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 0x7f5a9406eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44858/0 (socket says 172.21.15.64:44858) 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.279+0000 7f5a937fe640 1 -- 172.21.15.64:0/2650218134 learned_addr learned my addr 172.21.15.64:0/2650218134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a937fe640 1 -- 172.21.15.64:0/2650218134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94102320 msgr2=0x7f5a9406b6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94102320 0x7f5a9406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a937fe640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94102320 0x7f5a9406b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a937fe640 1 -- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a94101960 msgr2=0x7f5a9406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a937fe640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a94101960 0x7f5a9406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a937fe640 1 -- 172.21.15.64:0/2650218134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9406bfa0 con 0x7f5a94100770 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a937fe640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 0x7f5a9406eb70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0057c0 tx=0x7f5a7c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.280+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94102320 0x7f5a9406b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.281+0000 7f5a90ff9640 1 -- 172.21.15.64:0/2650218134 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c04f070 con 0x7f5a94100770 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.281+0000 7f5a90ff9640 1 -- 172.21.15.64:0/2650218134 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a7c004b50 con 0x7f5a94100770 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.281+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9406c230 con 0x7f5a94100770 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.281+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a94191490 con 0x7f5a94100770 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.281+0000 7f5a90ff9640 1 -- 172.21.15.64:0/2650218134 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c046030 con 0x7f5a94100770 2024-05-09T09:22:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.285+0000 7f5a9c9d8740 1 Processor -- start 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.286+0000 7f5a9c9d8740 1 -- start start 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.287+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c23d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.287+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c242420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.287+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 0x55dc6c244f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.287+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dc6c1a1d00 con 0x55dc6c242a90 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.287+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dc6c1494c0 con 0x55dc6c23d920 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.287+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dc6c19dcc0 con 0x55dc6c23cf30 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 0x55dc6c244f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 0x55dc6c244f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60036/0 (socket says 172.21.15.64:60036) 2024-05-09T09:22:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 -- 172.21.15.64:0/3441081866 learned_addr learned my addr 172.21.15.64:0/3441081866 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a937fe640 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c242420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 -- 172.21.15.64:0/3441081866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 msgr2=0x55dc6c23d350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c23d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 -- 172.21.15.64:0/3441081866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 msgr2=0x55dc6c242420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c242420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a92ffd640 1 -- 172.21.15.64:0/3441081866 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6bf7f7d0 con 0x55dc6c242a90 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.288+0000 7f5a90ff9640 1 -- 172.21.15.64:0/2650218134 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a7c052070 con 0x7f5a94100770 2024-05-09T09:22:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.289+0000 7f5a92ffd640 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 0x55dc6c244f20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009b20 tx=0x7f5a8400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb350ab9e1d5a8b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.290+0000 7f5a90ff9640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a800a2d90 0x7f5a800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.290+0000 7f5a89ffb640 1 -- 172.21.15.64:0/3441081866 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400ed80 con 0x55dc6c242a90 2024-05-09T09:22:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.290+0000 7f5a89ffb640 1 -- 172.21.15.64:0/3441081866 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a840108e0 con 0x55dc6c242a90 2024-05-09T09:22:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.290+0000 7f5a89ffb640 1 -- 172.21.15.64:0/3441081866 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84018970 con 0x55dc6c242a90 2024-05-09T09:22:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.290+0000 7f5a90ff9640 1 -- 172.21.15.64:0/2650218134 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a7c100710 con 0x7f5a94100770 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a800a2d90 0x7f5a800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/3441081866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 msgr2=0x55dc6c244f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 0x55dc6c244f20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009b20 tx=0x7f5a8400e350 comp rx=0 tx=0).stop 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a800a2d90 0x7f5a800a5250 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55dc6c23e180 tx=0x7f5a8c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/3441081866 shutdown_connections 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c242a90 0x55dc6c244f20 secure :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009b20 tx=0x7f5a8400e350 comp rx=0 tx=0).stop 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c242420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/3441081866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c23d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.291+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/3441081866 >> 172.21.15.64:0/3441081866 conn(0x55dc6c2385a0 msgr2=0x55dc6c2389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.292+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/3441081866 shutdown_connections 2024-05-09T09:22:36.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.292+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/3441081866 wait complete. 2024-05-09T09:22:36.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.292+0000 7f5a9c9d8740 1 -- start start 2024-05-09T09:22:36.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.293+0000 7f5a9c9d8740 1 Processor -- start 2024-05-09T09:22:36.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.294+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c247680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.294+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c247bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.294+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c24cbf0 0x55dc6c24efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.294+0000 7f5a92ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c247bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.294+0000 7f5a92ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c247bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58374/0 (socket says 172.21.15.64:58374) 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.294+0000 7f5a92ffd640 1 -- 172.21.15.64:0/1283105720 learned_addr learned my addr 172.21.15.64:0/1283105720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.295+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dc6c245a40 con 0x55dc6c24cbf0 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.295+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dc6c15cb90 con 0x55dc6c23d920 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.295+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55dc6c245bc0 con 0x55dc6c23cf30 2024-05-09T09:22:36.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.295+0000 7f5a9c9d8740 10 client.0 _ll_register_callbacks cb 0x55dc6c198da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.295+0000 7f5a937fe640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c247680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.295+0000 7f5a92ffd640 1 -- 172.21.15.64:0/1283105720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 msgr2=0x55dc6c247680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a92ffd640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c247680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a92ffd640 1 -- 172.21.15.64:0/1283105720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c24cbf0 msgr2=0x55dc6c24efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a92ffd640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c24cbf0 0x55dc6c24efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a92ffd640 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6beb53f0 con 0x55dc6c23d920 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a937fe640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c247680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a92ffd640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c247bc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5a8401a040 tx=0x7f5a84010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8f875ebdc48780 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.296+0000 7f5a72ffd640 1 -- 172.21.15.64:0/1283105720 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84026020 con 0x55dc6c23d920 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.297+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a840108e0 con 0x55dc6c23d920 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.297+0000 7f5a72ffd640 1 -- 172.21.15.64:0/1283105720 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a84005040 con 0x55dc6c23d920 2024-05-09T09:22:36.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.297+0000 7f5a72ffd640 1 -- 172.21.15.64:0/1283105720 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5a84005240 con 0x55dc6c23d920 2024-05-09T09:22:36.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.298+0000 7f5a9c9d8740 10 client.28232 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.298+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dc6be750f0 con 0x55dc6c23d920 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.298+0000 7f5a72ffd640 1 -- 172.21.15.64:0/1283105720 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a840108e0 con 0x55dc6c23d920 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.298+0000 7f5a72ffd640 1 -- 172.21.15.64:0/1283105720 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f5a840186f0 con 0x55dc6c23d920 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.298+0000 7f5a9c9d8740 10 client.28232 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.299+0000 7f5a9c9d8740 10 client.28232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.299+0000 7f5a9c9d8740 20 client.28232 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.299+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c253be0 0x55dc6c256080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.299+0000 7f5a9c9d8740 4 client.28232 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.299+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc6c1af430 con 0x55dc6c253be0 2024-05-09T09:22:36.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.299+0000 7f5a937fe640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c253be0 0x55dc6c256080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.300+0000 7f5a937fe640 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c253be0 0x55dc6c256080 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ce811647c18ff6 server_cookie=4890bbf30a04fe46 in_seq=0 out_seq=0 2024-05-09T09:22:36.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.300+0000 7f5a72ffd640 10 client.28232 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.301+0000 7f5a72ffd640 1 -- 172.21.15.64:0/1283105720 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc6c1af430 con 0x55dc6c253be0 2024-05-09T09:22:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.301+0000 7f5a72ffd640 10 client.28232 handle_command_reply: tid=0 2024-05-09T09:22:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.301+0000 7f5a9c9d8740 1 client.28232 shutdown 2024-05-09T09:22:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 msgr2=0x55dc6c247bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c247bc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5a8401a040 tx=0x7f5a84010660 comp rx=0 tx=0).stop 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 shutdown_connections 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c253be0 0x55dc6c256080 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c24cbf0 0x55dc6c24efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c23d920 0x55dc6c247bc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/1283105720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c247680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 >> 172.21.15.64:0/1283105720 conn(0x55dc6c2385a0 msgr2=0x55dc6c2377e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.302+0000 7f5a72ffd640 0 client.28232 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:36.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.304+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 shutdown_connections 2024-05-09T09:22:36.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.304+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/1283105720 wait complete. 2024-05-09T09:22:36.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.305+0000 7f5a9c9d8740 20 client.28232 trim_cache size 0 max 16384 2024-05-09T09:22:36.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.348+0000 7f5a9c9d8740 1 Processor -- start 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 -- start start 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23cf30 0x55dc6c233670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23d920 0x55dc6c247680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c247bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a740124b0 con 0x55dc6c23cf30 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7400fd50 con 0x55dc6c24cbf0 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.349+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7400dba0 con 0x55dc6c23d920 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.350+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c247bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.350+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c247bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58386/0 (socket says 172.21.15.64:58386) 2024-05-09T09:22:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.350+0000 7f5a93fff640 1 -- 172.21.15.64:0/462320524 learned_addr learned my addr 172.21.15.64:0/462320524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.351+0000 7f5a93fff640 1 -- 172.21.15.64:0/462320524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23d920 msgr2=0x55dc6c247680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.351+0000 7f5a93fff640 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23d920 0x55dc6c247680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.351+0000 7f5a93fff640 1 -- 172.21.15.64:0/462320524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23cf30 msgr2=0x55dc6c233670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.351+0000 7f5a93fff640 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23cf30 0x55dc6c233670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.351+0000 7f5a93fff640 1 -- 172.21.15.64:0/462320524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6c248520 con 0x55dc6c24cbf0 2024-05-09T09:22:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 735.0 s, 661.2 tps, lat 150.380 ms stddev 62.700 2024-05-09T09:22:36.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.354+0000 7f5a93fff640 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c247bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c011540 tx=0x7f5a8c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90d95ff51b1934e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.357+0000 7f5a897fa640 1 -- 172.21.15.64:0/462320524 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c022070 con 0x55dc6c24cbf0 2024-05-09T09:22:36.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.358+0000 7f5a897fa640 1 -- 172.21.15.64:0/462320524 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a8c009bb0 con 0x55dc6c24cbf0 2024-05-09T09:22:36.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.358+0000 7f5a897fa640 1 -- 172.21.15.64:0/462320524 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c010b40 con 0x55dc6c24cbf0 2024-05-09T09:22:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.359+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/462320524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 msgr2=0x55dc6c247bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.359+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c247bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c011540 tx=0x7f5a8c004ab0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.362+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/462320524 shutdown_connections 2024-05-09T09:22:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.362+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c247bc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.362+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23d920 0x55dc6c247680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.362+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/462320524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23cf30 0x55dc6c233670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.362+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/462320524 >> 172.21.15.64:0/462320524 conn(0x55dc6c2385a0 msgr2=0x55dc6c255bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.367+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/462320524 shutdown_connections 2024-05-09T09:22:36.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.369+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/462320524 wait complete. 2024-05-09T09:22:36.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.369+0000 7f5a9c9d8740 1 -- start start 2024-05-09T09:22:36.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.370+0000 7f5a9c9d8740 1 Processor -- start 2024-05-09T09:22:36.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.370+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c2642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.370+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23d920 0x55dc6c264810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a9c9d8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c2b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7400e210 con 0x55dc6c23d920 2024-05-09T09:22:36.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a9c9d8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7400f680 con 0x55dc6c24cbf0 2024-05-09T09:22:36.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c2b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c2b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58390/0 (socket says 172.21.15.64:58390) 2024-05-09T09:22:36.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 -- 172.21.15.64:0/2588059977 learned_addr learned my addr 172.21.15.64:0/2588059977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:36.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7400e8e0 con 0x55dc6c23cf30 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a9c9d8740 10 client.0 _ll_register_callbacks cb 0x7f5a84009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 -- 172.21.15.64:0/2588059977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 msgr2=0x55dc6c2642d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c2642d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 -- 172.21.15.64:0/2588059977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23d920 msgr2=0x55dc6c264810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.371+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23d920 0x55dc6c264810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.372+0000 7f5a93fff640 1 -- 172.21.15.64:0/2588059977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6c2b8210 con 0x55dc6c24cbf0 2024-05-09T09:22:36.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.372+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc6c1a9780 con 0x55dc6c24cbf0 2024-05-09T09:22:36.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.377+0000 7f5a93fff640 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c2b7cd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c010e90 tx=0x7f5a8c010ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e171c6b863975292 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:36.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.377+0000 7f5a72ffd640 10 client.28238 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:36.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.378+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2588059977 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c036060 con 0x55dc6c24cbf0 2024-05-09T09:22:36.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.378+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2588059977 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5a8c027660 con 0x55dc6c24cbf0 2024-05-09T09:22:36.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.378+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2588059977 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5a8c022050 con 0x55dc6c24cbf0 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.378+0000 7f5a9c9d8740 10 client.28238 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.379+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=48}) -- 0x55dc6c2b9260 con 0x55dc6c24cbf0 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.379+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2588059977 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f5a8c0278c0 con 0x55dc6c24cbf0 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.379+0000 7f5a9c9d8740 10 client.28238 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.379+0000 7f5a9c9d8740 10 client.28238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.380+0000 7f5a9c9d8740 20 client.28238 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:36.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.380+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c2b9ac0 0x55dc6c2bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:36.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.380+0000 7f5a9c9d8740 4 client.28238 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:22:36.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.380+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc6c1266f0 con 0x55dc6c2b9ac0 2024-05-09T09:22:36.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.380+0000 7f5a937fe640 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c2b9ac0 0x55dc6c2bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:36.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.381+0000 7f5a937fe640 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c2b9ac0 0x55dc6c2bbee0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be3fa94b9210c55 server_cookie=73ef9c8c2060a3a2 in_seq=0 out_seq=0 2024-05-09T09:22:36.386 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:22:36.386 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:22:36.386 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout: "86fc48f1-f1aa-49a3-8424-a3f634554e56": { 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "86fc48f1-f1aa-49a3-8424-a3f634554e56", 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:22:36.387 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.382+0000 7f5a72ffd640 10 client.28238 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.382+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2588059977 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc6c1266f0 con 0x55dc6c2b9ac0 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.382+0000 7f5a72ffd640 10 client.28238 handle_command_reply: tid=0 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.382+0000 7f5a9c9d8740 1 client.28238 shutdown 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 msgr2=0x55dc6c2b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c2b7cd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c010e90 tx=0x7f5a8c010ec0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 shutdown_connections 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55dc6c2b9ac0 0x55dc6c2bbee0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc6c24cbf0 0x55dc6c2b7cd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc6c23d920 0x55dc6c264810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 --2- 172.21.15.64:0/2588059977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dc6c23cf30 0x55dc6c2642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 >> 172.21.15.64:0/2588059977 conn(0x55dc6c2385a0 msgr2=0x55dc6c23f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a72ffd640 0 client.28238 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 shutdown_connections 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 1 -- 172.21.15.64:0/2588059977 wait complete. 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.384+0000 7f5a9c9d8740 20 client.28238 trim_cache size 0 max 16384 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.387+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a800a2d90 msgr2=0x7f5a800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.387+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a800a2d90 0x7f5a800a5250 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55dc6c23e180 tx=0x7f5a8c009500 comp rx=0 tx=0).stop 2024-05-09T09:22:36.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.387+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 msgr2=0x7f5a9406eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 0x7f5a9406eb70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0057c0 tx=0x7f5a7c0048f0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 shutdown_connections 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5a800a2d90 0x7f5a800a5250 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a94102320 0x7f5a9406b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a94101960 0x7f5a9406f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 --2- 172.21.15.64:0/2650218134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a94100770 0x7f5a9406eb70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.388+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 >> 172.21.15.64:0/2650218134 conn(0x7f5a940fbce0 msgr2=0x7f5a94102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:36.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.389+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 shutdown_connections 2024-05-09T09:22:36.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:22:36.389+0000 7f5a99cc4640 1 -- 172.21.15.64:0/2650218134 wait complete. 2024-05-09T09:22:36.415 INFO:tasks.cephfs.filesystem:scrub status for tag:86fc48f1-f1aa-49a3-8424-a3f634554e56 - {'path': '/', 'tag': '86fc48f1-f1aa-49a3-8424-a3f634554e56', 'options': 'recursive,force'} 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 715 op/s 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:22:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:36 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 715 op/s 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:22:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:36 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 715 op/s 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:37.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:36 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:22:39.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:38 smithi139 ceph-mon[33599]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 638 op/s 2024-05-09T09:22:39.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:38 smithi064 ceph-mon[30149]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 638 op/s 2024-05-09T09:22:39.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:38 smithi179 ceph-mon[34086]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 638 op/s 2024-05-09T09:22:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:40 smithi139 ceph-mon[33599]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.4 MiB/s wr, 903 op/s 2024-05-09T09:22:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:40 smithi064 ceph-mon[30149]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.4 MiB/s wr, 903 op/s 2024-05-09T09:22:41.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:40 smithi179 ceph-mon[34086]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.4 MiB/s wr, 903 op/s 2024-05-09T09:22:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 740.0 s, 650.0 tps, lat 155.571 ms stddev 84.143 2024-05-09T09:22:43.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:42 smithi139 ceph-mon[33599]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.7 MiB/s wr, 713 op/s 2024-05-09T09:22:43.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:42 smithi064 ceph-mon[30149]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.7 MiB/s wr, 713 op/s 2024-05-09T09:22:43.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:42 smithi179 ceph-mon[34086]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.7 MiB/s wr, 713 op/s 2024-05-09T09:22:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.792+0000 7f68b6031640 1 -- 172.21.15.64:0/2120168051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109a10 msgr2=0x7f68b0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.792+0000 7f68b6031640 1 --2- 172.21.15.64:0/2120168051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109a10 0x7f68b0110330 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.794+0000 7f68b6031640 1 -- 172.21.15.64:0/2120168051 shutdown_connections 2024-05-09T09:22:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.794+0000 7f68b6031640 1 --2- 172.21.15.64:0/2120168051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109a10 0x7f68b0110330 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.794+0000 7f68b6031640 1 --2- 172.21.15.64:0/2120168051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0109050 0x7f68b01094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.794+0000 7f68b6031640 1 --2- 172.21.15.64:0/2120168051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b0107e70 0x7f68b0108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.794+0000 7f68b6031640 1 -- 172.21.15.64:0/2120168051 >> 172.21.15.64:0/2120168051 conn(0x7f68b0076e80 msgr2=0x7f68b00792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.795+0000 7f68b6031640 1 -- 172.21.15.64:0/2120168051 shutdown_connections 2024-05-09T09:22:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.795+0000 7f68b6031640 1 -- 172.21.15.64:0/2120168051 wait complete. 2024-05-09T09:22:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.795+0000 7f68b6031640 1 Processor -- start 2024-05-09T09:22:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 -- start start 2024-05-09T09:22:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 0x7f68b0080240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109050 0x7f68b0080780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b0109a10 0x7f68b0081770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68b007a980 con 0x7f68b0109050 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68b007a800 con 0x7f68b0109a10 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68b6031640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f68b007ab00 con 0x7f68b0107e70 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 0x7f68b0080240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 0x7f68b0080240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44894/0 (socket says 172.21.15.64:44894) 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68af7fe640 1 -- 172.21.15.64:0/2086950597 learned_addr learned my addr 172.21.15.64:0/2086950597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109050 0x7f68b0080780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.796+0000 7f68affff640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b0109a10 0x7f68b0081770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68af7fe640 1 -- 172.21.15.64:0/2086950597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b0109a10 msgr2=0x7f68b0081770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68af7fe640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b0109a10 0x7f68b0081770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68af7fe640 1 -- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109050 msgr2=0x7f68b0080780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68af7fe640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109050 0x7f68b0080780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68af7fe640 1 -- 172.21.15.64:0/2086950597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b0081de0 con 0x7f68b0107e70 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68af7fe640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 0x7f68b0080240 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009f20 tx=0x7f68a000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.797+0000 7f68acff9640 1 -- 172.21.15.64:0/2086950597 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a000cd00 con 0x7f68b0107e70 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.798+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b00820d0 con 0x7f68b0107e70 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.798+0000 7f68acff9640 1 -- 172.21.15.64:0/2086950597 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a00035f0 con 0x7f68b0107e70 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.798+0000 7f68acff9640 1 -- 172.21.15.64:0/2086950597 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a00296d0 con 0x7f68b0107e70 2024-05-09T09:22:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.798+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b0082620 con 0x7f68b0107e70 2024-05-09T09:22:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.801+0000 7f68b8e18740 1 Processor -- start 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 -- start start 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250821670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821c40 0x563250826740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 0x563250829240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563250785340 con 0x563250821c40 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56325072d290 con 0x563250821250 2024-05-09T09:22:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.802+0000 7f68b8e18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563250781300 con 0x563250826db0 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 0x563250829240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 0x563250829240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44906/0 (socket says 172.21.15.64:44906) 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 -- 172.21.15.64:0/2160088870 learned_addr learned my addr 172.21.15.64:0/2160088870 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 -- 172.21.15.64:0/2160088870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 msgr2=0x563250821670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250821670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 -- 172.21.15.64:0/2160088870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821c40 msgr2=0x563250826740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821c40 0x563250826740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.803+0000 7f68aeffd640 1 -- 172.21.15.64:0/2160088870 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632507518e0 con 0x563250826db0 2024-05-09T09:22:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.804+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 0x563250829240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f68a4003a20 tx=0x7f68a4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f42e57344ffe52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.804+0000 7f6895ffb640 1 -- 172.21.15.64:0/2160088870 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4004b20 con 0x563250826db0 2024-05-09T09:22:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.804+0000 7f6895ffb640 1 -- 172.21.15.64:0/2160088870 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a4004cc0 con 0x563250826db0 2024-05-09T09:22:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.804+0000 7f6895ffb640 1 -- 172.21.15.64:0/2160088870 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4012740 con 0x563250826db0 2024-05-09T09:22:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.804+0000 7f68acff9640 1 -- 172.21.15.64:0/2086950597 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68a00037a0 con 0x7f68b0107e70 2024-05-09T09:22:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.805+0000 7f68acff9640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68980a2df0 0x7f68980a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.805+0000 7f68b8e18740 1 -- 172.21.15.64:0/2160088870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 msgr2=0x563250829240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.805+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 0x563250829240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f68a4003a20 tx=0x7f68a4009f90 comp rx=0 tx=0).stop 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68acff9640 1 -- 172.21.15.64:0/2086950597 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68a00e06d0 con 0x7f68b0107e70 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68affff640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68980a2df0 0x7f68980a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68b8e18740 1 -- 172.21.15.64:0/2160088870 shutdown_connections 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250826db0 0x563250829240 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821c40 0x563250826740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2160088870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250821670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68b8e18740 1 -- 172.21.15.64:0/2160088870 >> 172.21.15.64:0/2160088870 conn(0x56325081c8c0 msgr2=0x56325081ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.806+0000 7f68b8e18740 1 -- 172.21.15.64:0/2160088870 shutdown_connections 2024-05-09T09:22:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.807+0000 7f68b8e18740 1 -- 172.21.15.64:0/2160088870 wait complete. 2024-05-09T09:22:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.807+0000 7f68affff640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68980a2df0 0x7f68980a52b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005ab0 tx=0x7f68a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.807+0000 7f68b8e18740 1 -- start start 2024-05-09T09:22:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.807+0000 7f68b8e18740 1 Processor -- start 2024-05-09T09:22:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821250 0x56325082b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821c40 0x56325082b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x563250832a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563250829d60 con 0x563250826db0 2024-05-09T09:22:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563250740790 con 0x563250821c40 2024-05-09T09:22:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563250829ee0 con 0x563250821250 2024-05-09T09:22:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68b8e18740 10 client.0 _ll_register_callbacks cb 0x56325077ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821250 0x56325082b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821250 0x56325082b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44914/0 (socket says 172.21.15.64:44914) 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68af7fe640 1 -- 172.21.15.64:0/16709005 learned_addr learned my addr 172.21.15.64:0/16709005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68affff640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x563250832a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.808+0000 7f68aeffd640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821c40 0x56325082b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f68affff640 1 -- 172.21.15.64:0/16709005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821250 msgr2=0x56325082b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f68affff640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821250 0x56325082b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f68affff640 1 -- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821c40 msgr2=0x56325082b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f68affff640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821c40 0x56325082b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f68affff640 1 -- 172.21.15.64:0/16709005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563250499210 con 0x563250826db0 2024-05-09T09:22:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f68affff640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x563250832a20 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f68a802a9e0 tx=0x7f68a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31f931aaf6a7e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.809+0000 7f688effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:22:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.810+0000 7f688effd640 1 -- 172.21.15.64:0/16709005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a804d070 con 0x563250826db0 2024-05-09T09:22:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.810+0000 7f688effd640 1 -- 172.21.15.64:0/16709005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68a8031780 con 0x563250826db0 2024-05-09T09:22:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.810+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a4004cc0 con 0x563250826db0 2024-05-09T09:22:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.810+0000 7f688effd640 1 -- 172.21.15.64:0/16709005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f68a8059050 con 0x563250826db0 2024-05-09T09:22:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.811+0000 7f68b8e18740 10 client.18768 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.811+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563250565620 con 0x563250826db0 2024-05-09T09:22:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f688effd640 1 -- 172.21.15.64:0/16709005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a4004cc0 con 0x563250826db0 2024-05-09T09:22:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f688effd640 1 -- 172.21.15.64:0/16709005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f68a8049b10 con 0x563250826db0 2024-05-09T09:22:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f68b8e18740 10 client.18768 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f68b8e18740 10 client.18768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f68b8e18740 20 client.18768 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f68b8e18740 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563250837430 0x563250839890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f68b8e18740 4 client.18768 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.814+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563250792a70 con 0x563250837430 2024-05-09T09:22:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.815+0000 7f68af7fe640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563250837430 0x563250839890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.816+0000 7f688effd640 10 client.18768 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.816+0000 7f68af7fe640 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563250837430 0x563250839890 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536ae725d1309f26 server_cookie=34c305550c1ac2fd in_seq=0 out_seq=0 2024-05-09T09:22:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.818+0000 7f688effd640 1 -- 172.21.15.64:0/16709005 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563250792a70 con 0x563250837430 2024-05-09T09:22:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.818+0000 7f688effd640 10 client.18768 handle_command_reply: tid=0 2024-05-09T09:22:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.818+0000 7f68b8e18740 1 client.18768 shutdown 2024-05-09T09:22:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.820+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 msgr2=0x563250832a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.820+0000 7f68b8e18740 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x563250832a20 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f68a802a9e0 tx=0x7f68a8005fd0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 shutdown_connections 2024-05-09T09:22:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f68b8e18740 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563250837430 0x563250839890 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f68b8e18740 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x563250832a20 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f68b8e18740 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821c40 0x56325082b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f68b8e18740 1 --2- 172.21.15.64:0/16709005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821250 0x56325082b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 >> 172.21.15.64:0/16709005 conn(0x56325081c8c0 msgr2=0x56325081ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.822+0000 7f688effd640 0 client.18768 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.824+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 shutdown_connections 2024-05-09T09:22:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.827+0000 7f68b8e18740 1 -- 172.21.15.64:0/16709005 wait complete. 2024-05-09T09:22:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.827+0000 7f68b8e18740 20 client.18768 trim_cache size 0 max 16384 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.862+0000 7f68b8e18740 1 Processor -- start 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 -- start start 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821250 0x563250829c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x5632508329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250826db0 0x56325081cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0125d0 con 0x563250821250 2024-05-09T09:22:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f689c00fe70 con 0x563250826db0 2024-05-09T09:22:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.863+0000 7f68b8e18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f689c00dcc0 con 0x563250821c40 2024-05-09T09:22:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x5632508329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x5632508329e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:44928/0 (socket says 172.21.15.64:44928) 2024-05-09T09:22:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 -- 172.21.15.64:0/2804904688 learned_addr learned my addr 172.21.15.64:0/2804904688 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 -- 172.21.15.64:0/2804904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250826db0 msgr2=0x56325081cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250826db0 0x56325081cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 -- 172.21.15.64:0/2804904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821250 msgr2=0x563250829c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821250 0x563250829c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.864+0000 7f68aeffd640 1 -- 172.21.15.64:0/2804904688 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632508228e0 con 0x563250821c40 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.865+0000 7f68aeffd640 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x5632508329e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x56325081d830 tx=0x7f68a4009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5c58e89e3202a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.865+0000 7f68957fa640 1 -- 172.21.15.64:0/2804904688 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4005030 con 0x563250821c40 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.865+0000 7f68957fa640 1 -- 172.21.15.64:0/2804904688 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a40046e0 con 0x563250821c40 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.865+0000 7f68957fa640 1 -- 172.21.15.64:0/2804904688 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40126b0 con 0x563250821c40 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.866+0000 7f68b8e18740 1 -- 172.21.15.64:0/2804904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 msgr2=0x5632508329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.866+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x5632508329e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x56325081d830 tx=0x7f68a4009710 comp rx=0 tx=0).stop 2024-05-09T09:22:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.866+0000 7f68b8e18740 1 -- 172.21.15.64:0/2804904688 shutdown_connections 2024-05-09T09:22:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.867+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250826db0 0x56325081cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.867+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x5632508329e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.867+0000 7f68b8e18740 1 --2- 172.21.15.64:0/2804904688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250821250 0x563250829c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.867+0000 7f68b8e18740 1 -- 172.21.15.64:0/2804904688 >> 172.21.15.64:0/2804904688 conn(0x56325081c8c0 msgr2=0x563250826470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.870+0000 7f68b8e18740 1 -- 172.21.15.64:0/2804904688 shutdown_connections 2024-05-09T09:22:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.871+0000 7f68b8e18740 1 -- 172.21.15.64:0/2804904688 wait complete. 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.871+0000 7f68b8e18740 1 -- start start 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.871+0000 7f68b8e18740 1 Processor -- start 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250832830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x56325082f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x56325082f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f689c00e330 con 0x563250826db0 2024-05-09T09:22:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f689c00f7a0 con 0x563250821250 2024-05-09T09:22:43.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f689c00ea00 con 0x563250821c40 2024-05-09T09:22:43.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68b8e18740 10 client.0 _ll_register_callbacks cb 0x7f68a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250832830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250832830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58448/0 (socket says 172.21.15.64:58448) 2024-05-09T09:22:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68af7fe640 1 -- 172.21.15.64:0/361892486 learned_addr learned my addr 172.21.15.64:0/361892486 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.872+0000 7f68aeffd640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x56325082f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68affff640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x56325082f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68af7fe640 1 -- 172.21.15.64:0/361892486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 msgr2=0x56325082f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68af7fe640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x56325082f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68af7fe640 1 -- 172.21.15.64:0/361892486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 msgr2=0x56325082f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68af7fe640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x56325082f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68af7fe640 1 -- 172.21.15.64:0/361892486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325089ba00 con 0x563250821250 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68affff640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x56325082f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f68af7fe640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250832830 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f68a000c6a0 tx=0x7f68a00ac560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5190816585fdb21a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.873+0000 7f688effd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.874+0000 7f688effd640 1 -- 172.21.15.64:0/361892486 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a00c9030 con 0x563250821250 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.874+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a40046e0 con 0x563250821250 2024-05-09T09:22:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.874+0000 7f688effd640 1 -- 172.21.15.64:0/361892486 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f68a00aabe0 con 0x563250821250 2024-05-09T09:22:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.874+0000 7f688effd640 1 -- 172.21.15.64:0/361892486 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f68a00bd050 con 0x563250821250 2024-05-09T09:22:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.874+0000 7f68b8e18740 10 client.28250 fetch_fsmap learned FSMap version 48 2024-05-09T09:22:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.874+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56325089cd60 con 0x563250821250 2024-05-09T09:22:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.875+0000 7f688effd640 1 -- 172.21.15.64:0/361892486 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a40046e0 con 0x563250821250 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f688effd640 1 -- 172.21.15.64:0/361892486 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f68a00a91b0 con 0x563250821250 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f68b8e18740 10 client.28250 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f68b8e18740 10 client.28250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f68b8e18740 20 client.28250 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f68b8e18740 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56325089d6d0 0x56325089faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f68b8e18740 4 client.28250 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.876+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56325078cdc0 con 0x56325089d6d0 2024-05-09T09:22:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.877+0000 7f68aeffd640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56325089d6d0 0x56325089faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.877+0000 7f68aeffd640 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56325089d6d0 0x56325089faf0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204c8034fbb8911a server_cookie=ad6b48c90744a42d in_seq=0 out_seq=0 2024-05-09T09:22:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.877+0000 7f688effd640 10 client.28250 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.914+0000 7f688effd640 1 -- 172.21.15.64:0/361892486 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56325078cdc0 con 0x56325089d6d0 2024-05-09T09:22:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.914+0000 7f688effd640 10 client.28250 handle_command_reply: tid=0 2024-05-09T09:22:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.914+0000 7f68b8e18740 1 client.28250 shutdown 2024-05-09T09:22:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.915+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 msgr2=0x563250832830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.915+0000 7f68b8e18740 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250832830 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f68a000c6a0 tx=0x7f68a00ac560 comp rx=0 tx=0).stop 2024-05-09T09:22:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.916+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 shutdown_connections 2024-05-09T09:22:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.916+0000 7f68b8e18740 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56325089d6d0 0x56325089faf0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.916+0000 7f68b8e18740 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563250826db0 0x56325082f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.916+0000 7f68b8e18740 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563250821c40 0x56325082f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.916+0000 7f68b8e18740 1 --2- 172.21.15.64:0/361892486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563250821250 0x563250832830 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.916+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 >> 172.21.15.64:0/361892486 conn(0x56325081c8c0 msgr2=0x563250822e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.918+0000 7f688effd640 0 client.28250 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.918+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 shutdown_connections 2024-05-09T09:22:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.921+0000 7f68b8e18740 1 -- 172.21.15.64:0/361892486 wait complete. 2024-05-09T09:22:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.921+0000 7f68b8e18740 20 client.28250 trim_cache size 0 max 16384 2024-05-09T09:22:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:22:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:22:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:22:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:22:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68980a2df0 msgr2=0x7f68980a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68980a2df0 0x7f68980a52b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005ab0 tx=0x7f68a8009260 comp rx=0 tx=0).stop 2024-05-09T09:22:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 msgr2=0x7f68b0080240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 0x7f68b0080240 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009f20 tx=0x7f68a000ee50 comp rx=0 tx=0).stop 2024-05-09T09:22:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 shutdown_connections 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f68980a2df0 0x7f68980a52b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b0109a10 0x7f68b0081770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0109050 0x7f68b0080780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 --2- 172.21.15.64:0/2086950597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b0107e70 0x7f68b0080240 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.923+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 >> 172.21.15.64:0/2086950597 conn(0x7f68b0076e80 msgr2=0x7f68b010c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.924+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 shutdown_connections 2024-05-09T09:22:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:43.924+0000 7f68b6031640 1 -- 172.21.15.64:0/2086950597 wait complete. 2024-05-09T09:22:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:44 smithi139 ceph-mon[33599]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.8 MiB/s wr, 822 op/s 2024-05-09T09:22:45.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:44 smithi064 ceph-mon[30149]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.8 MiB/s wr, 822 op/s 2024-05-09T09:22:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:44 smithi179 ceph-mon[34086]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.8 MiB/s wr, 822 op/s 2024-05-09T09:22:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 745.0 s, 773.8 tps, lat 126.911 ms stddev 96.596 2024-05-09T09:22:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:46 smithi139 ceph-mon[33599]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 800 op/s 2024-05-09T09:22:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:46 smithi064 ceph-mon[30149]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 800 op/s 2024-05-09T09:22:47.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:46 smithi179 ceph-mon[34086]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 800 op/s 2024-05-09T09:22:49.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:48 smithi139 ceph-mon[33599]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.9 MiB/s wr, 721 op/s 2024-05-09T09:22:49.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:48 smithi064 ceph-mon[30149]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.9 MiB/s wr, 721 op/s 2024-05-09T09:22:49.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:48 smithi179 ceph-mon[34086]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.9 MiB/s wr, 721 op/s 2024-05-09T09:22:50.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:50.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:50.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.8 MiB/s wr, 933 op/s 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:51.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.8 MiB/s wr, 933 op/s 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:22:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 7.8 MiB/s wr, 933 op/s 2024-05-09T09:22:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:22:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:22:51.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:22:51.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:22:51.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:22:51.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:22:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 750.0 s, 711.4 tps, lat 142.461 ms stddev 91.568 2024-05-09T09:22:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:52 smithi139 ceph-mon[33599]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-05-09T09:22:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:52 smithi064 ceph-mon[30149]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-05-09T09:22:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:52 smithi179 ceph-mon[34086]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-05-09T09:22:55.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:54 smithi064 ceph-mon[30149]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 751 op/s 2024-05-09T09:22:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:54 smithi179 ceph-mon[34086]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 751 op/s 2024-05-09T09:22:55.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:54 smithi139 ceph-mon[33599]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 751 op/s 2024-05-09T09:22:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 755.0 s, 731.8 tps, lat 136.650 ms stddev 70.627 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dce7fc640 1 -- 172.21.15.64:0/4025458268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8031e50 con 0x7f2dd00fff40 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 -- 172.21.15.64:0/4025458268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00fff40 msgr2=0x7f2dd0100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 --2- 172.21.15.64:0/4025458268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00fff40 0x7f2dd0100360 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009ca0 tx=0x7f2dc802a8d0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 -- 172.21.15.64:0/4025458268 shutdown_connections 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 --2- 172.21.15.64:0/4025458268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dd0101240 0x7f2dd01016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 --2- 172.21.15.64:0/4025458268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd01008a0 0x7f2dd0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 --2- 172.21.15.64:0/4025458268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00fff40 0x7f2dd0100360 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.086+0000 7f2dd482f640 1 -- 172.21.15.64:0/4025458268 >> 172.21.15.64:0/4025458268 conn(0x7f2dd00fbb60 msgr2=0x7f2dd00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.087+0000 7f2dd482f640 1 -- 172.21.15.64:0/4025458268 shutdown_connections 2024-05-09T09:22:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.087+0000 7f2dd482f640 1 -- 172.21.15.64:0/4025458268 wait complete. 2024-05-09T09:22:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 Processor -- start 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 -- start start 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dd00fff40 0x7f2dd0106540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 0x7f2dd0106a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101240 0x7f2dd010ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0103200 con 0x7f2dd01008a0 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0103080 con 0x7f2dd0101240 2024-05-09T09:22:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.088+0000 7f2dd482f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0103380 con 0x7f2dd00fff40 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 0x7f2dd0106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 0x7f2dd0106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49630/0 (socket says 172.21.15.64:49630) 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 -- 172.21.15.64:0/1186225151 learned_addr learned my addr 172.21.15.64:0/1186225151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dcffff640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101240 0x7f2dd010ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 -- 172.21.15.64:0/1186225151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dd00fff40 msgr2=0x7f2dd0106540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dd00fff40 0x7f2dd0106540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 -- 172.21.15.64:0/1186225151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101240 msgr2=0x7f2dd010ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101240 0x7f2dd010ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.089+0000 7f2dceffd640 1 -- 172.21.15.64:0/1186225151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd010c100 con 0x7f2dd01008a0 2024-05-09T09:22:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.090+0000 7f2dceffd640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 0x7f2dd0106a80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b4a0 tx=0x7f2dc000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.090+0000 7f2dccff9640 1 -- 172.21.15.64:0/1186225151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0014070 con 0x7f2dd01008a0 2024-05-09T09:22:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.090+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01ae9c0 con 0x7f2dd01008a0 2024-05-09T09:22:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.090+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01aedf0 con 0x7f2dd01008a0 2024-05-09T09:22:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.091+0000 7f2dccff9640 1 -- 172.21.15.64:0/1186225151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc0010ac0 con 0x7f2dd01008a0 2024-05-09T09:22:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.092+0000 7f2dccff9640 1 -- 172.21.15.64:0/1186225151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0017c00 con 0x7f2dd01008a0 2024-05-09T09:22:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.094+0000 7f2dd8795740 1 Processor -- start 2024-05-09T09:22:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 -- start start 2024-05-09T09:22:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e5dc0 0x5642f04ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04eaf30 0x5642f04ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5642f0449340 con 0x5642f04e5dc0 2024-05-09T09:22:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5642f03f1290 con 0x5642f04eaf30 2024-05-09T09:22:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dd8795740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5642f0445300 con 0x5642f04e53d0 2024-05-09T09:22:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04e57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:46970/0 (socket says 172.21.15.64:46970) 2024-05-09T09:22:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dcffff640 1 -- 172.21.15.64:0/3148721075 learned_addr learned my addr 172.21.15.64:0/3148721075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.095+0000 7f2dcf7fe640 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e5dc0 0x5642f04ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dcffff640 1 -- 172.21.15.64:0/3148721075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04eaf30 msgr2=0x5642f04ed3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:22:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dcffff640 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04eaf30 0x5642f04ed3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dcffff640 1 -- 172.21.15.64:0/3148721075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e5dc0 msgr2=0x5642f04ea8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dcffff640 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e5dc0 0x5642f04ea8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dcffff640 1 -- 172.21.15.64:0/3148721075 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f04158e0 con 0x5642f04e53d0 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dcffff640 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04e57f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40099c0 tx=0x7f2dc4009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1370fdba71b76f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2dccff9640 1 -- 172.21.15.64:0/1186225151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2dc0017070 con 0x7f2dd01008a0 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2db5ffb640 1 -- 172.21.15.64:0/3148721075 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4017070 con 0x5642f04e53d0 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2db5ffb640 1 -- 172.21.15.64:0/3148721075 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc4013470 con 0x5642f04e53d0 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.096+0000 7f2db5ffb640 1 -- 172.21.15.64:0/3148721075 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4012630 con 0x5642f04e53d0 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.097+0000 7f2dd8795740 1 -- 172.21.15.64:0/3148721075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 msgr2=0x5642f04e57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.097+0000 7f2dd8795740 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04e57f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40099c0 tx=0x7f2dc4009e80 comp rx=0 tx=0).stop 2024-05-09T09:22:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 -- 172.21.15.64:0/3148721075 shutdown_connections 2024-05-09T09:22:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04eaf30 0x5642f04ed3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e5dc0 0x5642f04ea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 --2- 172.21.15.64:0/3148721075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04e57f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 -- 172.21.15.64:0/3148721075 >> 172.21.15.64:0/3148721075 conn(0x5642f04e0a40 msgr2=0x5642f04e0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 -- 172.21.15.64:0/3148721075 shutdown_connections 2024-05-09T09:22:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.098+0000 7f2dd8795740 1 -- 172.21.15.64:0/3148721075 wait complete. 2024-05-09T09:22:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.099+0000 7f2dd8795740 1 -- start start 2024-05-09T09:22:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.099+0000 7f2dd8795740 1 Processor -- start 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.099+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 0x5642f04efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04eaf30 0x5642f04f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 0x5642f04efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 0x5642f04efc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51258/0 (socket says 172.21.15.64:51258) 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dcf7fe640 1 -- 172.21.15.64:0/2818834858 learned_addr learned my addr 172.21.15.64:0/2818834858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dcffff640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5642f04edee0 con 0x5642f04eaf30 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04eaf30 0x5642f04f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5642f0404790 con 0x5642f04e5dc0 2024-05-09T09:22:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.100+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5642f04ee060 con 0x5642f04e53d0 2024-05-09T09:22:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.101+0000 7f2dd8795740 10 client.0 _ll_register_callbacks cb 0x5642f043fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.101+0000 7f2dccff9640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2db80a2c20 0x7f2db80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcf7fe640 1 -- 172.21.15.64:0/2818834858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 msgr2=0x5642f04ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcf7fe640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcf7fe640 1 -- 172.21.15.64:0/2818834858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04eaf30 msgr2=0x5642f04f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcf7fe640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04eaf30 0x5642f04f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcf7fe640 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f015d210 con 0x5642f04e5dc0 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcffff640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dccff9640 1 -- 172.21.15.64:0/1186225151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2dc00ce630 con 0x7f2dd01008a0 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04eaf30 0x5642f04f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:22:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.102+0000 7f2dcffff640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2db80a2c20 0x7f2db80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2dcf7fe640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 0x5642f04efc10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8031980 tx=0x7f2dc8031460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e216525446988253 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2daeffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2dcffff640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2db80a2c20 0x7f2db80a50e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009780 tx=0x7f2dc401e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2daeffd640 1 -- 172.21.15.64:0/2818834858 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003b70 con 0x5642f04e5dc0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc4013470 con 0x5642f04e5dc0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2daeffd640 1 -- 172.21.15.64:0/2818834858 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2dc8003d10 con 0x5642f04e5dc0 2024-05-09T09:22:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.103+0000 7f2daeffd640 1 -- 172.21.15.64:0/2818834858 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2dc8004040 con 0x5642f04e5dc0 2024-05-09T09:22:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.108+0000 7f2dd8795740 10 client.28256 fetch_fsmap learned FSMap version 49 2024-05-09T09:22:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.108+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5642f0229620 con 0x5642f04e5dc0 2024-05-09T09:22:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.108+0000 7f2daeffd640 1 -- 172.21.15.64:0/2818834858 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc4013470 con 0x5642f04e5dc0 2024-05-09T09:22:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.108+0000 7f2daeffd640 1 -- 172.21.15.64:0/2818834858 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f2dc8042070 con 0x5642f04e5dc0 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.110+0000 7f2dd8795740 10 client.28256 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.110+0000 7f2dd8795740 10 client.28256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.110+0000 7f2dd8795740 20 client.28256 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.110+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04faf90 0x5642f04fd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.110+0000 7f2dd8795740 4 client.28256 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.110+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642f0456a70 con 0x5642f04faf90 2024-05-09T09:22:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.111+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04faf90 0x5642f04fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.111+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04faf90 0x5642f04fd380 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb11596ed3cb6a58 server_cookie=426ec1ede141e1e4 in_seq=0 out_seq=0 2024-05-09T09:22:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.111+0000 7f2daeffd640 10 client.28256 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.112+0000 7f2daeffd640 1 -- 172.21.15.64:0/2818834858 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642f0456a70 con 0x5642f04faf90 2024-05-09T09:22:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.112+0000 7f2daeffd640 10 client.28256 handle_command_reply: tid=0 2024-05-09T09:22:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 client.28256 shutdown 2024-05-09T09:22:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 msgr2=0x5642f04efc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 0x5642f04efc10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8031980 tx=0x7f2dc8031460 comp rx=0 tx=0).stop 2024-05-09T09:22:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 shutdown_connections 2024-05-09T09:22:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04faf90 0x5642f04fd380 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04eaf30 0x5642f04f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.113+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e5dc0 0x5642f04efc10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.114+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2818834858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e53d0 0x5642f04ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.114+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 >> 172.21.15.64:0/2818834858 conn(0x5642f04e0a40 msgr2=0x5642f04dfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.114+0000 7f2daeffd640 0 client.28256 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.114+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 shutdown_connections 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.114+0000 7f2dd8795740 1 -- 172.21.15.64:0/2818834858 wait complete. 2024-05-09T09:22:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.114+0000 7f2dd8795740 20 client.28256 trim_cache size 0 max 16384 2024-05-09T09:22:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.150+0000 7f2dd8795740 1 Processor -- start 2024-05-09T09:22:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.150+0000 7f2dd8795740 1 -- start start 2024-05-09T09:22:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.150+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04ef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49644/0 (socket says 172.21.15.64:49644) 2024-05-09T09:22:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:51270/0 (socket says 172.21.15.64:51270) 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dcf7fe640 1 -- 172.21.15.64:0/2156064105 learned_addr learned my addr 172.21.15.64:0/2156064105 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.151+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.152+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0125e0 con 0x5642f04e0a40 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.152+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc00fe50 con 0x5642f04e53d0 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.152+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc00dca0 con 0x5642f04e5dc0 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.152+0000 7f2dcffff640 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2dceffd640 1 -- 172.21.15.64:0/2156064105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 msgr2=0x5642f04df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04df410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2dceffd640 1 -- 172.21.15.64:0/2156064105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 msgr2=0x5642f04ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04ef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2dceffd640 1 -- 172.21.15.64:0/2156064105 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f04e5010 con 0x5642f04e53d0 2024-05-09T09:22:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2dceffd640 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f6c10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000be60 tx=0x7f2dc00d0a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ee7a2b9f755449 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:56 smithi064 ceph-mon[30149]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.2 MiB/s wr, 712 op/s 2024-05-09T09:22:57.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:56 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:22:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.153+0000 7f2db57fa640 1 -- 172.21.15.64:0/2156064105 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc001e040 con 0x5642f04e53d0 2024-05-09T09:22:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.154+0000 7f2db57fa640 1 -- 172.21.15.64:0/2156064105 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc009ee50 con 0x5642f04e53d0 2024-05-09T09:22:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.154+0000 7f2db57fa640 1 -- 172.21.15.64:0/2156064105 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00cbf20 con 0x5642f04e53d0 2024-05-09T09:22:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.155+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 msgr2=0x5642f04f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.155+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f6c10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000be60 tx=0x7f2dc00d0a80 comp rx=0 tx=0).stop 2024-05-09T09:22:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.155+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 shutdown_connections 2024-05-09T09:22:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.155+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.155+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f6c10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.155+0000 7f2dd8795740 1 --2- 172.21.15.64:0/2156064105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.156+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 >> 172.21.15.64:0/2156064105 conn(0x5642f04eaa00 msgr2=0x5642f04ee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.156+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 shutdown_connections 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.156+0000 7f2dd8795740 1 -- 172.21.15.64:0/2156064105 wait complete. 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.156+0000 7f2dd8795740 1 -- start start 2024-05-09T09:22:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 Processor -- start 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc00e310 con 0x5642f04e0a40 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc00f780 con 0x5642f04e53d0 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.157+0000 7f2dd8795740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc00e9e0 con 0x5642f04e5dc0 2024-05-09T09:22:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dd8795740 10 client.0 _ll_register_callbacks cb 0x7f2dc8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:22:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:46992/0 (socket says 172.21.15.64:46992) 2024-05-09T09:22:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 -- 172.21.15.64:0/853820437 learned_addr learned my addr 172.21.15.64:0/853820437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 -- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 msgr2=0x5642f04f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 -- 172.21.15.64:0/853820437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 msgr2=0x5642f04f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.158+0000 7f2dcffff640 1 -- 172.21.15.64:0/853820437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f04f60e0 con 0x5642f04e5dc0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2dcffff640 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04f5af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40099c0 tx=0x7f2dc4009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee473bf35448b570 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2daeffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2daeffd640 1 -- 172.21.15.64:0/853820437 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4017070 con 0x5642f04e5dc0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2daeffd640 1 -- 172.21.15.64:0/853820437 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2dc4005990 con 0x5642f04e5dc0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc009ee50 con 0x5642f04e5dc0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2daeffd640 1 -- 172.21.15.64:0/853820437 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2dc4037050 con 0x5642f04e5dc0 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2dd8795740 10 client.28426 fetch_fsmap learned FSMap version 49 2024-05-09T09:22:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.159+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5642f05617e0 con 0x5642f04e5dc0 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.160+0000 7f2daeffd640 1 -- 172.21.15.64:0/853820437 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc009ee50 con 0x5642f04e5dc0 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2daeffd640 1 -- 172.21.15.64:0/853820437 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f2dc4006700 con 0x5642f04e5dc0 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dd8795740 10 client.28426 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dd8795740 10 client.28426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dd8795740 20 client.28426 populate_metadata read hostname 'smithi064' 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dd8795740 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04eaf30 0x5642f0563fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dd8795740 4 client.28426 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642f0450dc0 con 0x5642f04eaf30 2024-05-09T09:22:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.161+0000 7f2dcf7fe640 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04eaf30 0x5642f0563fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:22:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.163+0000 7f2dcf7fe640 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04eaf30 0x5642f0563fe0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a8a6a9320d171 server_cookie=1e96338533c8961e in_seq=0 out_seq=0 2024-05-09T09:22:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.163+0000 7f2daeffd640 10 client.28426 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.192+0000 7f2daeffd640 1 -- 172.21.15.64:0/853820437 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642f0450dc0 con 0x5642f04eaf30 2024-05-09T09:22:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.192+0000 7f2daeffd640 10 client.28426 handle_command_reply: tid=0 2024-05-09T09:22:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.192+0000 7f2dd8795740 1 client.28426 shutdown 2024-05-09T09:22:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.195+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 msgr2=0x5642f04f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.195+0000 7f2dd8795740 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04f5af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40099c0 tx=0x7f2dc4009e80 comp rx=0 tx=0).stop 2024-05-09T09:22:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 shutdown_connections 2024-05-09T09:22:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2dd8795740 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5642f04eaf30 0x5642f0563fe0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2dd8795740 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642f04e5dc0 0x5642f04f5af0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2dd8795740 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5642f04e53d0 0x5642f04f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2dd8795740 1 --2- 172.21.15.64:0/853820437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5642f04e0a40 0x5642f04f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 >> 172.21.15.64:0/853820437 conn(0x5642f04eaa00 msgr2=0x5642f04e37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.198+0000 7f2daeffd640 0 client.28426 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:22:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.201+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 shutdown_connections 2024-05-09T09:22:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.203+0000 7f2dd8795740 1 -- 172.21.15.64:0/853820437 wait complete. 2024-05-09T09:22:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.203+0000 7f2dd8795740 20 client.28426 trim_cache size 0 max 16384 2024-05-09T09:22:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:22:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:22:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:22:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:22:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2db80a2c20 msgr2=0x7f2db80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2db80a2c20 0x7f2db80a50e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009780 tx=0x7f2dc401e040 comp rx=0 tx=0).stop 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 msgr2=0x7f2dd0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 0x7f2dd0106a80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000b4a0 tx=0x7f2dc000b960 comp rx=0 tx=0).stop 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 shutdown_connections 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2db80a2c20 0x7f2db80a50e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101240 0x7f2dd010ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd01008a0 0x7f2dd0106a80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 --2- 172.21.15.64:0/1186225151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2dd00fff40 0x7f2dd0106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.205+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 >> 172.21.15.64:0/1186225151 conn(0x7f2dd00fbb60 msgr2=0x7f2dd00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:22:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.206+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 shutdown_connections 2024-05-09T09:22:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:22:57.207+0000 7f2dd482f640 1 -- 172.21.15.64:0/1186225151 wait complete. 2024-05-09T09:22:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:56 smithi179 ceph-mon[34086]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.2 MiB/s wr, 712 op/s 2024-05-09T09:22:57.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:56 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:22:57.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:56 smithi139 ceph-mon[33599]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.2 MiB/s wr, 712 op/s 2024-05-09T09:22:57.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:56 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:22:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:22:58 smithi064 ceph-mon[30149]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.8 MiB/s wr, 633 op/s 2024-05-09T09:22:59.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:22:58 smithi179 ceph-mon[34086]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.8 MiB/s wr, 633 op/s 2024-05-09T09:22:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:22:58 smithi139 ceph-mon[33599]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.8 MiB/s wr, 633 op/s 2024-05-09T09:23:01.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:00 smithi064 ceph-mon[30149]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.2 MiB/s wr, 818 op/s 2024-05-09T09:23:01.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:00 smithi179 ceph-mon[34086]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.2 MiB/s wr, 818 op/s 2024-05-09T09:23:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:00 smithi139 ceph-mon[33599]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.2 MiB/s wr, 818 op/s 2024-05-09T09:23:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 760.0 s, 731.0 tps, lat 137.771 ms stddev 56.119 2024-05-09T09:23:03.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:02 smithi064 ceph-mon[30149]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 622 op/s 2024-05-09T09:23:03.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:02 smithi179 ceph-mon[34086]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 622 op/s 2024-05-09T09:23:03.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:02 smithi139 ceph-mon[33599]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 622 op/s 2024-05-09T09:23:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:04 smithi064 ceph-mon[30149]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 745 op/s 2024-05-09T09:23:05.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:04 smithi179 ceph-mon[34086]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 745 op/s 2024-05-09T09:23:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:04 smithi139 ceph-mon[33599]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 745 op/s 2024-05-09T09:23:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 765.0 s, 868.0 tps, lat 114.258 ms stddev 55.042 2024-05-09T09:23:06.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:23:06.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 -- 172.21.15.64:0/3077186520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30103a90 msgr2=0x7faf30103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 --2- 172.21.15.64:0/3077186520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30103a90 0x7faf30103eb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faf20009860 tx=0x7faf2002a340 comp rx=0 tx=0).stop 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 -- 172.21.15.64:0/3077186520 shutdown_connections 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 --2- 172.21.15.64:0/3077186520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf3010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 --2- 172.21.15.64:0/3077186520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30104c80 0x7faf30105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 --2- 172.21.15.64:0/3077186520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30103a90 0x7faf30103eb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 -- 172.21.15.64:0/3077186520 >> 172.21.15.64:0/3077186520 conn(0x7faf300ff020 msgr2=0x7faf30101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 -- 172.21.15.64:0/3077186520 shutdown_connections 2024-05-09T09:23:06.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.615+0000 7faf357dc640 1 -- 172.21.15.64:0/3077186520 wait complete. 2024-05-09T09:23:06.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.617+0000 7faf357dc640 1 Processor -- start 2024-05-09T09:23:06.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.617+0000 7faf357dc640 1 -- start start 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf357dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30103a90 0x7faf30199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf357dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30104c80 0x7faf3019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf357dc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf301a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf357dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf3010f590 con 0x7faf30103a90 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf2f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf301a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf2f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf301a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42152/0 (socket says 172.21.15.64:42152) 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.618+0000 7faf2f7fe640 1 -- 172.21.15.64:0/1357203432 learned_addr learned my addr 172.21.15.64:0/1357203432 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.619+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf3010f410 con 0x7faf30104c80 2024-05-09T09:23:06.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.619+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf3010f710 con 0x7faf30105640 2024-05-09T09:23:06.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.620+0000 7faf2f7fe640 1 -- 172.21.15.64:0/1357203432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30104c80 msgr2=0x7faf3019a050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:23:06.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.620+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30104c80 0x7faf3019a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.620+0000 7faf2f7fe640 1 -- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30103a90 msgr2=0x7faf30199b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:23:06.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.620+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30103a90 0x7faf30199b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.620+0000 7faf2f7fe640 1 -- 172.21.15.64:0/1357203432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf301a1730 con 0x7faf30105640 2024-05-09T09:23:06.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.620+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf301a10c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faf2800eb10 tx=0x7faf2800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.621+0000 7faf2cff9640 1 -- 172.21.15.64:0/1357203432 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28004b20 con 0x7faf30105640 2024-05-09T09:23:06.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.621+0000 7faf2cff9640 1 -- 172.21.15.64:0/1357203432 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf28004cc0 con 0x7faf30105640 2024-05-09T09:23:06.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.621+0000 7faf2cff9640 1 -- 172.21.15.64:0/1357203432 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28010720 con 0x7faf30105640 2024-05-09T09:23:06.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.621+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf301a1a20 con 0x7faf30105640 2024-05-09T09:23:06.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.621+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf301a1f10 con 0x7faf30105640 2024-05-09T09:23:06.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.625+0000 7faf2cff9640 1 -- 172.21.15.64:0/1357203432 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf28005020 con 0x7faf30105640 2024-05-09T09:23:06.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.627+0000 7faf385c8740 1 Processor -- start 2024-05-09T09:23:06.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.628+0000 7faf2cff9640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf1c0a2d60 0x7faf1c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.629+0000 7faf2cff9640 1 -- 172.21.15.64:0/1357203432 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faf280cfae0 con 0x7faf30105640 2024-05-09T09:23:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf385c8740 1 -- start start 2024-05-09T09:23:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf385c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821b5f30 0x55cf821b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf385c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf385c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821bba90 0x55cf821bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf2effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821b5f30 0x55cf821b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf2effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821b5f30 0x55cf821b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59172/0 (socket says 172.21.15.64:59172) 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.630+0000 7faf2effd640 1 -- 172.21.15.64:0/2862578805 learned_addr learned my addr 172.21.15.64:0/2862578805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.631+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821bba90 0x55cf821bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.631+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cf8211ad00 con 0x55cf821b5f30 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.631+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf820c24c0 con 0x55cf821b6920 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.631+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cf82116cc0 con 0x55cf821bba90 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.631+0000 7faf27fff640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf1c0a2d60 0x7faf1c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.631+0000 7faf27fff640 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2862578805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 msgr2=0x55cf821bb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821bb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2862578805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821b5f30 msgr2=0x55cf821b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821b5f30 0x55cf821b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2862578805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf81ef87d0 con 0x55cf821bba90 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821bba90 0x55cf821bdf20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faf2809bc50 tx=0x7faf28005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb14a70b858cfce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf257fa640 1 -- 172.21.15.64:0/2862578805 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2809bf50 con 0x55cf821bba90 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf257fa640 1 -- 172.21.15.64:0/2862578805 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf280a0300 con 0x55cf821bba90 2024-05-09T09:23:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.632+0000 7faf257fa640 1 -- 172.21.15.64:0/2862578805 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280a4730 con 0x55cf821bba90 2024-05-09T09:23:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf27fff640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf1c0a2d60 0x7faf1c0a5220 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faf3019b020 tx=0x7faf180127c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821bba90 msgr2=0x55cf821bdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821bba90 0x55cf821bdf20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faf2809bc50 tx=0x7faf28005560 comp rx=0 tx=0).stop 2024-05-09T09:23:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 shutdown_connections 2024-05-09T09:23:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821bba90 0x55cf821bdf20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821bb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 --2- 172.21.15.64:0/2862578805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821b5f30 0x55cf821b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 >> 172.21.15.64:0/2862578805 conn(0x55cf821b15a0 msgr2=0x55cf821b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.634+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 shutdown_connections 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.635+0000 7faf385c8740 1 -- 172.21.15.64:0/2862578805 wait complete. 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.635+0000 7faf385c8740 1 -- start start 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.635+0000 7faf385c8740 1 Processor -- start 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b5f30 0x55cf821c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821c0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cf821bea40 con 0x55cf821bba90 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf820d5b90 con 0x55cf821b6920 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cf821bebc0 con 0x55cf821b5f30 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821c0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53438/0 (socket says 172.21.15.64:53438) 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf27fff640 1 -- 172.21.15.64:0/3756995907 learned_addr learned my addr 172.21.15.64:0/3756995907 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf2effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b5f30 0x55cf821c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf385c8740 10 client.0 _ll_register_callbacks cb 0x55cf82111da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.636+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf27fff640 1 -- 172.21.15.64:0/3756995907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b5f30 msgr2=0x55cf821c0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf27fff640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b5f30 0x55cf821c0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf27fff640 1 -- 172.21.15.64:0/3756995907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 msgr2=0x55cf821c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf27fff640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf27fff640 1 -- 172.21.15.64:0/3756995907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf81e2e3f0 con 0x55cf821b6920 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:06.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf27fff640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821c0990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faf180108d0 tx=0x7faf180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c084ca2019d42337 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf167fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.637+0000 7faf167fc640 1 -- 172.21.15.64:0/3756995907 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1800ece0 con 0x55cf821b6920 2024-05-09T09:23:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.638+0000 7faf167fc640 1 -- 172.21.15.64:0/3756995907 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faf18004030 con 0x55cf821b6920 2024-05-09T09:23:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.638+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf280a0300 con 0x55cf821b6920 2024-05-09T09:23:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.639+0000 7faf167fc640 1 -- 172.21.15.64:0/3756995907 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faf18010e50 con 0x55cf821b6920 2024-05-09T09:23:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.643+0000 7faf385c8740 10 client.28268 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.643+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cf81dee0f0 con 0x55cf821b6920 2024-05-09T09:23:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.643+0000 7faf167fc640 1 -- 172.21.15.64:0/3756995907 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf180054d0 con 0x55cf821b6920 2024-05-09T09:23:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf167fc640 1 -- 172.21.15.64:0/3756995907 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7faf180048c0 con 0x55cf821b6920 2024-05-09T09:23:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf385c8740 10 client.28268 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf385c8740 10 client.28268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf385c8740 20 client.28268 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf385c8740 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf821cc2b0 0x55cf821ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf385c8740 4 client.28268 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faf280a0300 con 0x55cf821cc2b0 2024-05-09T09:23:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.644+0000 7faf2effd640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf821cc2b0 0x55cf821ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.645+0000 7faf2effd640 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf821cc2b0 0x55cf821ce750 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3621adfae276f2 server_cookie=3d63f45be4f202d in_seq=0 out_seq=0 2024-05-09T09:23:06.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.645+0000 7faf167fc640 10 client.28268 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf167fc640 1 -- 172.21.15.64:0/3756995907 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7faf280a0300 con 0x55cf821cc2b0 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf167fc640 10 client.28268 handle_command_reply: tid=0 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf385c8740 1 client.28268 shutdown 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 msgr2=0x55cf821c0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf385c8740 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821c0990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faf180108d0 tx=0x7faf180046c0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 shutdown_connections 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.646+0000 7faf385c8740 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf821cc2b0 0x55cf821ce750 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf167fc640 0 client.28268 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:06.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b6920 0x55cf821c0990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 1 --2- 172.21.15.64:0/3756995907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b5f30 0x55cf821c0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 >> 172.21.15.64:0/3756995907 conn(0x55cf821b15a0 msgr2=0x55cf821b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 shutdown_connections 2024-05-09T09:23:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 1 -- 172.21.15.64:0/3756995907 wait complete. 2024-05-09T09:23:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.647+0000 7faf385c8740 20 client.28268 trim_cache size 0 max 16384 2024-05-09T09:23:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.680+0000 7faf385c8740 1 Processor -- start 2024-05-09T09:23:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.680+0000 7faf385c8740 1 -- start start 2024-05-09T09:23:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.680+0000 7faf385c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 0x55cf821b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.680+0000 7faf385c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.680+0000 7faf385c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf385c8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf301aee50 con 0x55cf821bba90 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf385c8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf301ac6c0 con 0x55cf821b5f30 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf385c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf301aa510 con 0x55cf821b6920 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42188/0 (socket says 172.21.15.64:42188) 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 -- 172.21.15.64:0/3231788599 learned_addr learned my addr 172.21.15.64:0/3231788599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 -- 172.21.15.64:0/3231788599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 msgr2=0x55cf821b4590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 0x55cf821b4590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 -- 172.21.15.64:0/3231788599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 msgr2=0x55cf821b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:06.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.681+0000 7faf27fff640 1 -- 172.21.15.64:0/3231788599 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf821acf60 con 0x55cf821b6920 2024-05-09T09:23:06.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.682+0000 7faf27fff640 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c7990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faf18009a20 tx=0x7faf18005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61bf85a8c2c388b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.682+0000 7faf24ff9640 1 -- 172.21.15.64:0/3231788599 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18038070 con 0x55cf821b6920 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.682+0000 7faf24ff9640 1 -- 172.21.15.64:0/3231788599 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf18010e50 con 0x55cf821b6920 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.682+0000 7faf24ff9640 1 -- 172.21.15.64:0/3231788599 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18005510 con 0x55cf821b6920 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.684+0000 7faf385c8740 1 -- 172.21.15.64:0/3231788599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 msgr2=0x55cf821c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.684+0000 7faf385c8740 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c7990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faf18009a20 tx=0x7faf18005970 comp rx=0 tx=0).stop 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.684+0000 7faf385c8740 1 -- 172.21.15.64:0/3231788599 shutdown_connections 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.684+0000 7faf385c8740 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.684+0000 7faf385c8740 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c7990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.684+0000 7faf385c8740 1 --2- 172.21.15.64:0/3231788599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 0x55cf821b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.685+0000 7faf385c8740 1 -- 172.21.15.64:0/3231788599 >> 172.21.15.64:0/3231788599 conn(0x55cf821b15a0 msgr2=0x55cf821bef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.685+0000 7faf385c8740 1 -- 172.21.15.64:0/3231788599 shutdown_connections 2024-05-09T09:23:06.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.685+0000 7faf385c8740 1 -- 172.21.15.64:0/3231788599 wait complete. 2024-05-09T09:23:06.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.686+0000 7faf385c8740 1 -- start start 2024-05-09T09:23:06.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.686+0000 7faf385c8740 1 Processor -- start 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 0x55cf821c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf301aab80 con 0x55cf821bba90 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf301abff0 con 0x55cf821b5f30 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf301ab250 con 0x55cf821b6920 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf385c8740 10 client.0 _ll_register_callbacks cb 0x7faf18009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59208/0 (socket says 172.21.15.64:59208) 2024-05-09T09:23:06.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.687+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2795962506 learned_addr learned my addr 172.21.15.64:0/2795962506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:06.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.688+0000 7faf27fff640 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.688+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2795962506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 msgr2=0x55cf821c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:06.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.688+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.688+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 msgr2=0x55cf821c52b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.688+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 0x55cf821c52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.688+0000 7faf2f7fe640 1 -- 172.21.15.64:0/2795962506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf821c6df0 con 0x55cf821bba90 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c6800 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faf280a4690 tx=0x7faf280a4bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cde532b7cf782c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf167fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf167fc640 1 -- 172.21.15.64:0/2795962506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280cd270 con 0x55cf821bba90 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf18010e50 con 0x55cf821bba90 2024-05-09T09:23:06.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf167fc640 1 -- 172.21.15.64:0/2795962506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faf280a4730 con 0x55cf821bba90 2024-05-09T09:23:06.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf167fc640 1 -- 172.21.15.64:0/2795962506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faf280a4930 con 0x55cf821bba90 2024-05-09T09:23:06.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf385c8740 10 client.18798 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:06.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.689+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cf82232bd0 con 0x55cf821bba90 2024-05-09T09:23:06.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf167fc640 1 -- 172.21.15.64:0/2795962506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf18010e50 con 0x55cf821bba90 2024-05-09T09:23:06.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf167fc640 1 -- 172.21.15.64:0/2795962506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7faf280b9070 con 0x55cf821bba90 2024-05-09T09:23:06.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf385c8740 10 client.18798 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:06.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf385c8740 10 client.18798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:06.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf385c8740 20 client.18798 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:06.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf385c8740 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf822331a0 0x55cf82235590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf385c8740 4 client.18798 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:23:06.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.690+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf82128430 con 0x55cf822331a0 2024-05-09T09:23:06.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.691+0000 7faf2effd640 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf822331a0 0x55cf82235590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.691+0000 7faf167fc640 10 client.18798 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.692+0000 7faf2effd640 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf822331a0 0x55cf82235590 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e052bb0f2dae6ab server_cookie=f10edb3fc880e9e7 in_seq=0 out_seq=0 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.692+0000 7faf167fc640 1 -- 172.21.15.64:0/2795962506 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cf82128430 con 0x55cf822331a0 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.692+0000 7faf167fc640 10 client.18798 handle_command_reply: tid=0 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.692+0000 7faf385c8740 1 client.18798 shutdown 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 msgr2=0x55cf821c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c6800 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faf280a4690 tx=0x7faf280a4bc0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 shutdown_connections 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55cf822331a0 0x55cf82235590 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf821bba90 0x55cf821c6800 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf821b6920 0x55cf821c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf167fc640 0 client.18798 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 --2- 172.21.15.64:0/2795962506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf821b5f30 0x55cf821c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.693+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 >> 172.21.15.64:0/2795962506 conn(0x55cf821b15a0 msgr2=0x55cf821aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.694+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 shutdown_connections 2024-05-09T09:23:06.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.694+0000 7faf385c8740 1 -- 172.21.15.64:0/2795962506 wait complete. 2024-05-09T09:23:06.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.694+0000 7faf385c8740 20 client.18798 trim_cache size 0 max 16384 2024-05-09T09:23:06.694 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:23:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:23:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:23:06.694 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf1c0a2d60 msgr2=0x7faf1c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf1c0a2d60 0x7faf1c0a5220 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faf3019b020 tx=0x7faf180127c0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 msgr2=0x7faf301a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf301a10c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faf2800eb10 tx=0x7faf2800c2d0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 shutdown_connections 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7faf1c0a2d60 0x7faf1c0a5220 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30105640 0x7faf301a10c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf30104c80 0x7faf3019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 --2- 172.21.15.64:0/1357203432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf30103a90 0x7faf30199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.695+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 >> 172.21.15.64:0/1357203432 conn(0x7faf300ff020 msgr2=0x7faf30107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.696+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 shutdown_connections 2024-05-09T09:23:06.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.696+0000 7faf357dc640 1 -- 172.21.15.64:0/1357203432 wait complete. 2024-05-09T09:23:06.713 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:23:06.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.915+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/1263992188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 msgr2=0x7f3e9c100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.915+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/1263992188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 0x7f3e9c100340 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009900 tx=0x7f3e8c02a400 comp rx=0 tx=0).stop 2024-05-09T09:23:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/1263992188 shutdown_connections 2024-05-09T09:23:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/1263992188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c101220 0x7f3e9c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/1263992188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c100880 0x7f3e9c100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/1263992188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 0x7f3e9c100340 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/1263992188 >> 172.21.15.64:0/1263992188 conn(0x7f3e9c0fbb40 msgr2=0x7f3e9c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/1263992188 shutdown_connections 2024-05-09T09:23:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.916+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/1263992188 wait complete. 2024-05-09T09:23:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 Processor -- start 2024-05-09T09:23:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 -- start start 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 0x7f3e9c071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 0x7f3e9c071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c101220 0x7f3e9c0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c06bb50 con 0x7f3e9c100880 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c06b9d0 con 0x7f3e9c101220 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3ea0bdd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c06bcd0 con 0x7f3e9c0fff20 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3e9affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 0x7f3e9c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c101220 0x7f3e9c0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3e9affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 0x7f3e9c071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59232/0 (socket says 172.21.15.64:59232) 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.917+0000 7f3e9affd640 1 -- 172.21.15.64:0/3576198326 learned_addr learned my addr 172.21.15.64:0/3576198326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e9affd640 1 -- 172.21.15.64:0/3576198326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 msgr2=0x7f3e9c071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e9affd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 0x7f3e9c071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e9affd640 1 -- 172.21.15.64:0/3576198326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c101220 msgr2=0x7f3e9c0728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e9affd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c101220 0x7f3e9c0728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e9affd640 1 -- 172.21.15.64:0/3576198326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c072f20 con 0x7f3e9c100880 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e9affd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 0x7f3e9c071870 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c890 tx=0x7f3e9000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.918+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004220 con 0x7f3e9c100880 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.919+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c073210 con 0x7f3e9c100880 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.919+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c0737d0 con 0x7f3e9c100880 2024-05-09T09:23:06.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.919+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e900043c0 con 0x7f3e9c100880 2024-05-09T09:23:06.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.919+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9001d060 con 0x7f3e9c100880 2024-05-09T09:23:06.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.920+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e880052c0 con 0x7f3e9c100880 2024-05-09T09:23:06.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.922+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e900079b0 con 0x7f3e9c100880 2024-05-09T09:23:06.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.923+0000 7f3e98ff9640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e840a2c60 0x7f3e840a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:06.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.923+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e900cf120 con 0x7f3e9c100880 2024-05-09T09:23:06.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.924+0000 7f3e9b7fe640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e840a2c60 0x7f3e840a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:06.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.924+0000 7f3e9b7fe640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e840a2c60 0x7f3e840a5120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0096c0 tx=0x7f3e8c005760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:06.930+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e90014030 con 0x7f3e9c100880 2024-05-09T09:23:07.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:06 smithi064 ceph-mon[30149]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 737 op/s 2024-05-09T09:23:07.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:06 smithi179 ceph-mon[34086]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 737 op/s 2024-05-09T09:23:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:06 smithi139 ceph-mon[33599]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 737 op/s 2024-05-09T09:23:07.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.396+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e880055b0 con 0x7f3e9c100880 2024-05-09T09:23:07.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.398+0000 7f3e98ff9640 1 -- 172.21.15.64:0/3576198326 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11885 (secure 0 0 0) 0x7f3e9009bcf0 con 0x7f3e9c100880 2024-05-09T09:23:07.400 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:23:07.400 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:22:55.565953+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:23:07.400 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 49 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.404+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e840a2c60 msgr2=0x7f3e840a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.404+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e840a2c60 0x7f3e840a5120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0096c0 tx=0x7f3e8c005760 comp rx=0 tx=0).stop 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.404+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 msgr2=0x7f3e9c071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.404+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 0x7f3e9c071870 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c890 tx=0x7f3e9000cd50 comp rx=0 tx=0).stop 2024-05-09T09:23:07.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.407+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 shutdown_connections 2024-05-09T09:23:07.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.407+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3e840a2c60 0x7f3e840a5120 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.407+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c101220 0x7f3e9c0728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.407+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e9c100880 0x7f3e9c071870 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.407+0000 7f3ea0bdd640 1 --2- 172.21.15.64:0/3576198326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9c0fff20 0x7f3e9c071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.407+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 >> 172.21.15.64:0/3576198326 conn(0x7f3e9c0fbb40 msgr2=0x7f3e9c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.411+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 shutdown_connections 2024-05-09T09:23:07.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.411+0000 7f3ea0bdd640 1 -- 172.21.15.64:0/3576198326 wait complete. 2024-05-09T09:23:07.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:23:07.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 -- 172.21.15.64:0/1721577537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8107d70 msgr2=0x7f64e8108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 --2- 172.21.15.64:0/1721577537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8107d70 0x7f64e8108190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009900 tx=0x7f64d802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 -- 172.21.15.64:0/1721577537 shutdown_connections 2024-05-09T09:23:07.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 --2- 172.21.15.64:0/1721577537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64e8109920 0x7f64e81101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 --2- 172.21.15.64:0/1721577537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8108f60 0x7f64e81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 --2- 172.21.15.64:0/1721577537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8107d70 0x7f64e8108190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 -- 172.21.15.64:0/1721577537 >> 172.21.15.64:0/1721577537 conn(0x7f64e8076d00 msgr2=0x7f64e8079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 -- 172.21.15.64:0/1721577537 shutdown_connections 2024-05-09T09:23:07.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.639+0000 7f64ed210640 1 -- 172.21.15.64:0/1721577537 wait complete. 2024-05-09T09:23:07.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.641+0000 7f64ed210640 1 Processor -- start 2024-05-09T09:23:07.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.641+0000 7f64ed210640 1 -- start start 2024-05-09T09:23:07.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.642+0000 7f64ed210640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 0x7f64e8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 0x7f64e8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 0x7f64e8072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59256/0 (socket says 172.21.15.64:59256) 2024-05-09T09:23:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64ed210640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64e8108f60 0x7f64e80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64e7fff640 1 -- 172.21.15.64:0/2709696527 learned_addr learned my addr 172.21.15.64:0/2709696527 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8109920 0x7f64e80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64df7fe640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64e8108f60 0x7f64e80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.643+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64e806bb50 con 0x7f64e8107d70 2024-05-09T09:23:07.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.644+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64e806b9d0 con 0x7f64e8109920 2024-05-09T09:23:07.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.644+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64e806bcd0 con 0x7f64e8108f60 2024-05-09T09:23:07.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.644+0000 7f64eca0f640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8109920 0x7f64e80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64e7fff640 1 -- 172.21.15.64:0/2709696527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64e8108f60 msgr2=0x7f64e80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64e7fff640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64e8108f60 0x7f64e80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64e7fff640 1 -- 172.21.15.64:0/2709696527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8109920 msgr2=0x7f64e80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64e7fff640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8109920 0x7f64e80712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64e7fff640 1 -- 172.21.15.64:0/2709696527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e8071960 con 0x7f64e8107d70 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64eca0f640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8109920 0x7f64e80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:07.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.645+0000 7f64e7fff640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 0x7f64e8072c80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005ab0 tx=0x7f64d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.646+0000 7f64e5ffb640 1 -- 172.21.15.64:0/2709696527 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d803f070 con 0x7f64e8107d70 2024-05-09T09:23:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.646+0000 7f64e5ffb640 1 -- 172.21.15.64:0/2709696527 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64d8031c30 con 0x7f64e8107d70 2024-05-09T09:23:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.646+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e8071bf0 con 0x7f64e8107d70 2024-05-09T09:23:07.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.647+0000 7f64e5ffb640 1 -- 172.21.15.64:0/2709696527 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8035030 con 0x7f64e8107d70 2024-05-09T09:23:07.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.647+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e80720e0 con 0x7f64e8107d70 2024-05-09T09:23:07.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 Processor -- start 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 -- start start 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894362550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c894367620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c89436a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c8942c6c40 con 0x55c894362130 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c89426e580 con 0x55c894367c90 2024-05-09T09:23:07.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.651+0000 7f64f117d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c8942c2c00 con 0x55c894362b20 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894362550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894362550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59270/0 (socket says 172.21.15.64:59270) 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64eca0f640 1 -- 172.21.15.64:0/1898186879 learned_addr learned my addr 172.21.15.64:0/1898186879 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c894367620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c89436a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64eca0f640 1 -- 172.21.15.64:0/1898186879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 msgr2=0x55c894367620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.652+0000 7f64eca0f640 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c894367620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.653+0000 7f64eca0f640 1 -- 172.21.15.64:0/1898186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 msgr2=0x55c89436a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.653+0000 7f64eca0f640 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c89436a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.653+0000 7f64eca0f640 1 -- 172.21.15.64:0/1898186879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c893fda3f0 con 0x55c894362130 2024-05-09T09:23:07.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.653+0000 7f64eca0f640 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894362550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bcf0 tx=0x7f64e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7855fa05c7cd1b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.653+0000 7f64de7fc640 1 -- 172.21.15.64:0/1898186879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0015070 con 0x55c894362130 2024-05-09T09:23:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.653+0000 7f64e5ffb640 1 -- 172.21.15.64:0/2709696527 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64d8042070 con 0x7f64e8107d70 2024-05-09T09:23:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.654+0000 7f64de7fc640 1 -- 172.21.15.64:0/1898186879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e00118f0 con 0x55c894362130 2024-05-09T09:23:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.654+0000 7f64de7fc640 1 -- 172.21.15.64:0/1898186879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e00107d0 con 0x55c894362130 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 -- 172.21.15.64:0/1898186879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 msgr2=0x55c894362550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894362550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bcf0 tx=0x7f64e000f730 comp rx=0 tx=0).stop 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64e5ffb640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f64d40a2d90 0x7f64d40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64e5ffb640 1 -- 172.21.15.64:0/2709696527 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64d80efbc0 con 0x7f64e8107d70 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 -- 172.21.15.64:0/1898186879 shutdown_connections 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c89436a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c894367620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 --2- 172.21.15.64:0/1898186879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894362550 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.655+0000 7f64f117d740 1 -- 172.21.15.64:0/1898186879 >> 172.21.15.64:0/1898186879 conn(0x55c89435d7a0 msgr2=0x55c89435dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.656+0000 7f64f117d740 1 -- 172.21.15.64:0/1898186879 shutdown_connections 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.656+0000 7f64eca0f640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f64d40a2d90 0x7f64d40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.656+0000 7f64f117d740 1 -- 172.21.15.64:0/1898186879 wait complete. 2024-05-09T09:23:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.657+0000 7f64f117d740 1 -- start start 2024-05-09T09:23:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.657+0000 7f64f117d740 1 Processor -- start 2024-05-09T09:23:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.658+0000 7f64f117d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c89436c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.658+0000 7f64f117d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362b20 0x55c89436c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.658+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c89436c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.658+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c89436c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42264/0 (socket says 172.21.15.64:42264) 2024-05-09T09:23:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64eca0f640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f64d40a2d90 0x7f64d40a5250 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bcf0 tx=0x7f64e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64e7fff640 1 -- 172.21.15.64:0/1534817400 learned_addr learned my addr 172.21.15.64:0/1534817400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894373930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c89436ac40 con 0x55c894362b20 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c894281c40 con 0x55c894367c90 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c89436adc0 con 0x55c894362130 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64f117d740 10 client.0 _ll_register_callbacks cb 0x55c893fd9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.659+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362b20 0x55c89436c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64e7fff640 1 -- 172.21.15.64:0/1534817400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 msgr2=0x55c894373930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894373930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64e7fff640 1 -- 172.21.15.64:0/1534817400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362b20 msgr2=0x55c89436c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362b20 0x55c89436c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64e7fff640 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8940a47d0 con 0x55c894362130 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362b20 0x55c89436c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c89436c3b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64d80bbfc0 tx=0x7f64d80bbb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8f4882cf8611e81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64c77fe640 1 -- 172.21.15.64:0/1534817400 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8049040 con 0x55c894362130 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64c77fe640 1 -- 172.21.15.64:0/1534817400 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64d80b86a0 con 0x55c894362130 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.660+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e00118f0 con 0x55c894362130 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.661+0000 7f64c77fe640 1 -- 172.21.15.64:0/1534817400 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f64d80f4040 con 0x55c894362130 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.661+0000 7f64f117d740 10 client.28456 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.661+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c89436acb0 con 0x55c894362130 2024-05-09T09:23:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.661+0000 7f64c77fe640 1 -- 172.21.15.64:0/1534817400 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64d8031de0 con 0x55c894362130 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64c77fe640 1 -- 172.21.15.64:0/1534817400 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f64d80c4810 con 0x55c894362130 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64f117d740 10 client.28456 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64f117d740 10 client.28456 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64f117d740 20 client.28456 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c894378230 0x55c89437a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64f117d740 4 client.28456 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.662+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64e00118f0 con 0x55c894378230 2024-05-09T09:23:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.664+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c894378230 0x55c89437a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.664+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c894378230 0x55c89437a6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff4a4ae04fce5e65 server_cookie=30fe55e04f7769f3 in_seq=0 out_seq=0 2024-05-09T09:23:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.664+0000 7f64c77fe640 10 client.28456 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.666+0000 7f64c77fe640 1 -- 172.21.15.64:0/1534817400 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f64e00118f0 con 0x55c894378230 2024-05-09T09:23:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.666+0000 7f64c77fe640 10 client.28456 handle_command_reply: tid=0 2024-05-09T09:23:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.666+0000 7f64f117d740 1 client.28456 shutdown 2024-05-09T09:23:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.667+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 msgr2=0x55c89436c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.667+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c89436c3b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64d80bbfc0 tx=0x7f64d80bbb70 comp rx=0 tx=0).stop 2024-05-09T09:23:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.667+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 shutdown_connections 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c894378230 0x55c89437a6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894373930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362b20 0x55c89436c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 --2- 172.21.15.64:0/1534817400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c89436c3b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 >> 172.21.15.64:0/1534817400 conn(0x55c89435d7a0 msgr2=0x55c89435c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64c77fe640 0 client.28456 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 shutdown_connections 2024-05-09T09:23:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.668+0000 7f64f117d740 1 -- 172.21.15.64:0/1534817400 wait complete. 2024-05-09T09:23:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.669+0000 7f64f117d740 20 client.28456 trim_cache size 0 max 16384 2024-05-09T09:23:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.698+0000 7f64f117d740 1 Processor -- start 2024-05-09T09:23:07.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 -- start start 2024-05-09T09:23:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894360750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c8943738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894360c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81bb960 con 0x55c894362130 2024-05-09T09:23:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b91d0 con 0x55c894367c90 2024-05-09T09:23:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64f117d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b7020 con 0x55c894362b20 2024-05-09T09:23:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.699+0000 7f64eca0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894360c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.700+0000 7f64df7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c8943738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.700+0000 7f64df7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c8943738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42278/0 (socket says 172.21.15.64:42278) 2024-05-09T09:23:07.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.700+0000 7f64df7fe640 1 -- 172.21.15.64:0/3133326145 learned_addr learned my addr 172.21.15.64:0/3133326145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.701+0000 7f64df7fe640 1 -- 172.21.15.64:0/3133326145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 msgr2=0x55c894360c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.701+0000 7f64df7fe640 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894360c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.701+0000 7f64df7fe640 1 -- 172.21.15.64:0/3133326145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 msgr2=0x55c894360750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.701+0000 7f64df7fe640 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894360750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.701+0000 7f64df7fe640 1 -- 172.21.15.64:0/3133326145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c894362f00 con 0x55c894362b20 2024-05-09T09:23:07.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.701+0000 7f64df7fe640 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c8943738c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f64e80729f0 tx=0x7f64d0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19651f5f16490a29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.702+0000 7f64ddffb640 1 -- 172.21.15.64:0/3133326145 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0025070 con 0x55c894362b20 2024-05-09T09:23:07.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.702+0000 7f64ddffb640 1 -- 172.21.15.64:0/3133326145 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64d000fd20 con 0x55c894362b20 2024-05-09T09:23:07.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.702+0000 7f64ddffb640 1 -- 172.21.15.64:0/3133326145 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0004420 con 0x55c894362b20 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.703+0000 7f64f117d740 1 -- 172.21.15.64:0/3133326145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 msgr2=0x55c8943738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.703+0000 7f64f117d740 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c8943738c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f64e80729f0 tx=0x7f64d0009500 comp rx=0 tx=0).stop 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.704+0000 7f64f117d740 1 -- 172.21.15.64:0/3133326145 shutdown_connections 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.704+0000 7f64f117d740 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894367c90 0x55c894360c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.704+0000 7f64f117d740 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362b20 0x55c8943738c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.704+0000 7f64f117d740 1 --2- 172.21.15.64:0/3133326145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894362130 0x55c894360750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.704+0000 7f64f117d740 1 -- 172.21.15.64:0/3133326145 >> 172.21.15.64:0/3133326145 conn(0x55c89435d7a0 msgr2=0x55c894363da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.705+0000 7f64f117d740 1 -- 172.21.15.64:0/3133326145 shutdown_connections 2024-05-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.705+0000 7f64f117d740 1 -- 172.21.15.64:0/3133326145 wait complete. 2024-05-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.705+0000 7f64f117d740 1 -- start start 2024-05-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.706+0000 7f64f117d740 1 Processor -- start 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.707+0000 7f64f117d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c894370ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.707+0000 7f64f117d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894362b20 0x55c894371220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.707+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c894370ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.707+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c894370ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42280/0 (socket says 172.21.15.64:42280) 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.707+0000 7f64e7fff640 1 -- 172.21.15.64:0/1646195 learned_addr learned my addr 172.21.15.64:0/1646195 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.708+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894362b20 0x55c894371220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.708+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894367c90 0x55c894372230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.708+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b7690 con 0x55c894367c90 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.708+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b8b00 con 0x55c894362b20 2024-05-09T09:23:07.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.708+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b7d60 con 0x55c894362130 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.708+0000 7f64f117d740 10 client.0 _ll_register_callbacks cb 0x7f64d80b83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64e7fff640 1 -- 172.21.15.64:0/1646195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894362b20 msgr2=0x55c894371220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894362b20 0x55c894371220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64e7fff640 1 -- 172.21.15.64:0/1646195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894367c90 msgr2=0x55c894372230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894367c90 0x55c894372230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64e7fff640 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c894372820 con 0x55c894362130 2024-05-09T09:23:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64e7fff640 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c894370ce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f64d8000c00 tx=0x7f64d802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f077ca38cffcb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d000fd20 con 0x55c894362130 2024-05-09T09:23:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64c77fe640 10 client.28462 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.709+0000 7f64c77fe640 1 -- 172.21.15.64:0/1646195 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8049070 con 0x55c894362130 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.710+0000 7f64c77fe640 1 -- 172.21.15.64:0/1646195 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f64d8031980 con 0x55c894362130 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.710+0000 7f64c77fe640 1 -- 172.21.15.64:0/1646195 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f64d80bb990 con 0x55c894362130 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.710+0000 7f64f117d740 10 client.28462 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.710+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=49}) -- 0x55c8943739e0 con 0x55c894362130 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64c77fe640 1 -- 172.21.15.64:0/1646195 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f64d80bb480 con 0x55c894362130 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64f117d740 10 client.28462 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64f117d740 10 client.28462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64f117d740 20 client.28462 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c8943debc0 0x55c8943e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64f117d740 4 client.28462 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:23:07.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f64d000fd20 con 0x55c8943debc0 2024-05-09T09:23:07.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.711+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c8943debc0 0x55c8943e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.712+0000 7f64df7fe640 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c8943debc0 0x55c8943e0fe0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a78450992180503f server_cookie=c281dc502d49f721 in_seq=0 out_seq=0 2024-05-09T09:23:07.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.712+0000 7f64c77fe640 10 client.28462 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:07.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.712+0000 7f64c77fe640 1 -- 172.21.15.64:0/1646195 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f64d000fd20 con 0x55c8943debc0 2024-05-09T09:23:07.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.712+0000 7f64c77fe640 10 client.28462 handle_command_reply: tid=0 2024-05-09T09:23:07.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.713+0000 7f64f117d740 1 client.28462 shutdown 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.713+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 msgr2=0x55c894370ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.713+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c894370ce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f64d8000c00 tx=0x7f64d802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.713+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 shutdown_connections 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55c8943debc0 0x55c8943e0fe0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c894367c90 0x55c894372230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c894362b20 0x55c894371220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64f117d740 1 --2- 172.21.15.64:0/1646195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c894362130 0x55c894370ce0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 >> 172.21.15.64:0/1646195 conn(0x55c89435d7a0 msgr2=0x55c894360750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64c77fe640 0 client.28462 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.714+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 shutdown_connections 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.715+0000 7f64f117d740 1 -- 172.21.15.64:0/1646195 wait complete. 2024-05-09T09:23:07.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.715+0000 7f64f117d740 20 client.28462 trim_cache size 0 max 16384 2024-05-09T09:23:07.716 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:23:07.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.717+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f64d40a2d90 msgr2=0x7f64d40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f64d40a2d90 0x7f64d40a5250 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bcf0 tx=0x7f64e0000bc0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 msgr2=0x7f64e8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 0x7f64e8072c80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005ab0 tx=0x7f64d8003c20 comp rx=0 tx=0).stop 2024-05-09T09:23:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 shutdown_connections 2024-05-09T09:23:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f64d40a2d90 0x7f64d40a5250 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e8109920 0x7f64e80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64e8108f60 0x7f64e80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 --2- 172.21.15.64:0/2709696527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8107d70 0x7f64e8072c80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.718+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 >> 172.21.15.64:0/2709696527 conn(0x7f64e8076d00 msgr2=0x7f64e810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.719+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 shutdown_connections 2024-05-09T09:23:07.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.719+0000 7f64ed210640 1 -- 172.21.15.64:0/2709696527 wait complete. 2024-05-09T09:23:07.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:23:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.943+0000 7fb365168640 1 -- 172.21.15.64:0/2227398654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c035020 con 0x7fb360077dd0 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 -- 172.21.15.64:0/2227398654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077dd0 msgr2=0x7fb360078670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2227398654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077dd0 0x7fb360078670 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009900 tx=0x7fb35c02a440 comp rx=0 tx=0).stop 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 -- 172.21.15.64:0/2227398654 shutdown_connections 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2227398654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077dd0 0x7fb360078670 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2227398654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077430 0x7fb360077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2227398654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3600761f0 0x7fb360076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 -- 172.21.15.64:0/2227398654 >> 172.21.15.64:0/2227398654 conn(0x7fb3600ff020 msgr2=0x7fb360101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 -- 172.21.15.64:0/2227398654 shutdown_connections 2024-05-09T09:23:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.944+0000 7fb3683ce640 1 -- 172.21.15.64:0/2227398654 wait complete. 2024-05-09T09:23:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 Processor -- start 2024-05-09T09:23:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 -- start start 2024-05-09T09:23:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3600761f0 0x7fb36019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077430 0x7fb36019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 0x7fb3601a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb360113a70 con 0x7fb360077430 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3601138f0 con 0x7fb360077dd0 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb3683ce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb360113bf0 con 0x7fb3600761f0 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb365969640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077430 0x7fb36019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb36696b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 0x7fb3601a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb36616a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3600761f0 0x7fb36019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb36696b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 0x7fb3601a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53526/0 (socket says 172.21.15.64:53526) 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.945+0000 7fb36696b640 1 -- 172.21.15.64:0/2220452633 learned_addr learned my addr 172.21.15.64:0/2220452633 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb36696b640 1 -- 172.21.15.64:0/2220452633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3600761f0 msgr2=0x7fb36019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb36696b640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3600761f0 0x7fb36019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb36696b640 1 -- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077430 msgr2=0x7fb36019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb36696b640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077430 0x7fb36019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb36696b640 1 -- 172.21.15.64:0/2220452633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3601a5ce0 con 0x7fb360077dd0 2024-05-09T09:23:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb36696b640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 0x7fb3601a5670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb35c005ec0 tx=0x7fb35c004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.946+0000 7fb3577fe640 1 -- 172.21.15.64:0/2220452633 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c040070 con 0x7fb360077dd0 2024-05-09T09:23:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.947+0000 7fb3577fe640 1 -- 172.21.15.64:0/2220452633 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb35c031c00 con 0x7fb360077dd0 2024-05-09T09:23:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.947+0000 7fb3577fe640 1 -- 172.21.15.64:0/2220452633 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c035020 con 0x7fb360077dd0 2024-05-09T09:23:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.947+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3601a5f70 con 0x7fb360077dd0 2024-05-09T09:23:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.947+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3601a6400 con 0x7fb360077dd0 2024-05-09T09:23:07.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.951+0000 7fb3577fe640 1 -- 172.21.15.64:0/2220452633 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb35c043070 con 0x7fb360077dd0 2024-05-09T09:23:07.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.952+0000 7fb36b1b9740 1 Processor -- start 2024-05-09T09:23:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.953+0000 7fb3577fe640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb34c0a2d60 0x7fb34c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 -- start start 2024-05-09T09:23:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb3577fe640 1 -- 172.21.15.64:0/2220452633 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb35c0f0010 con 0x7fb360077dd0 2024-05-09T09:23:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975164e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751653430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x559751655f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597515b2c40 con 0x55975164df40 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55975155a580 con 0x559751653aa0 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.954+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597515aec00 con 0x55975164e930 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb365969640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb34c0a2d60 0x7fb34c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975164e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975164e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59326/0 (socket says 172.21.15.64:59326) 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 -- 172.21.15.64:0/2575241229 learned_addr learned my addr 172.21.15.64:0/2575241229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36696b640 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x559751655f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb365969640 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751653430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 -- 172.21.15.64:0/2575241229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 msgr2=0x559751653430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751653430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 -- 172.21.15.64:0/2575241229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 msgr2=0x559751655f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x559751655f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36616a640 1 -- 172.21.15.64:0/2575241229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597512c63f0 con 0x55975164df40 2024-05-09T09:23:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.955+0000 7fb36696b640 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x559751655f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.956+0000 7fb365969640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb34c0a2d60 0x7fb34c0a5220 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb35000afd0 tx=0x7fb350013460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.956+0000 7fb36616a640 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975164e360 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb35800b580 tx=0x7fb3580096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54138b844e0c7ea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.958+0000 7fb354bfe640 1 -- 172.21.15.64:0/2575241229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35800a990 con 0x55975164df40 2024-05-09T09:23:07.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.959+0000 7fb354bfe640 1 -- 172.21.15.64:0/2575241229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb358019070 con 0x55975164df40 2024-05-09T09:23:07.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.960+0000 7fb354bfe640 1 -- 172.21.15.64:0/2575241229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358013380 con 0x55975164df40 2024-05-09T09:23:07.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.960+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2575241229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 msgr2=0x55975164e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.960+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975164e360 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb35800b580 tx=0x7fb3580096b0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.961+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2575241229 shutdown_connections 2024-05-09T09:23:07.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.961+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x559751655f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.961+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751653430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.961+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2575241229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975164e360 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.961+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2575241229 >> 172.21.15.64:0/2575241229 conn(0x5597516495b0 msgr2=0x5597516499b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.962+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2575241229 shutdown_connections 2024-05-09T09:23:07.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.962+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2575241229 wait complete. 2024-05-09T09:23:07.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.962+0000 7fb36b1b9740 1 -- start start 2024-05-09T09:23:07.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.963+0000 7fb36b1b9740 1 Processor -- start 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.963+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x559751658310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.963+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751658850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.963+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x559751658310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559751656aa0 con 0x55975164df40 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55975156dc40 con 0x559751653aa0 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559751656c20 con 0x55975164e930 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x559751658310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59334/0 (socket says 172.21.15.64:59334) 2024-05-09T09:23:07.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 -- 172.21.15.64:0/2174166411 learned_addr learned my addr 172.21.15.64:0/2174166411 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb365969640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751658850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36b1b9740 10 client.0 _ll_register_callbacks cb 0x5597512c5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36696b640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 -- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 msgr2=0x559751658850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751658850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 -- 172.21.15.64:0/2174166411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 msgr2=0x55975165f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36616a640 1 -- 172.21.15.64:0/2174166411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597513907d0 con 0x55975164df40 2024-05-09T09:23:07.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.964+0000 7fb36696b640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.965+0000 7fb36616a640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x559751658310 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb358009aa0 tx=0x7fb358015f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a55137d259865a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.965+0000 7fb349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.965+0000 7fb349ffb640 1 -- 172.21.15.64:0/2174166411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358014830 con 0x55975164df40 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.965+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb358019070 con 0x55975164df40 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.965+0000 7fb349ffb640 1 -- 172.21.15.64:0/2174166411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb35800a790 con 0x55975164df40 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.966+0000 7fb349ffb640 1 -- 172.21.15.64:0/2174166411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb35800a990 con 0x55975164df40 2024-05-09T09:23:07.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.966+0000 7fb36b1b9740 10 client.18825 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:07.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.966+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559751662c60 con 0x55975164df40 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb349ffb640 1 -- 172.21.15.64:0/2174166411 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb358019070 con 0x55975164df40 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb349ffb640 1 -- 172.21.15.64:0/2174166411 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fb35801e400 con 0x55975164df40 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb36b1b9740 10 client.18825 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb36b1b9740 10 client.18825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb36b1b9740 20 client.18825 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x559751664010 0x559751666440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb36b1b9740 4 client.18825 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597515c0370 con 0x559751664010 2024-05-09T09:23:07.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.968+0000 7fb365969640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x559751664010 0x559751666440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.971+0000 7fb349ffb640 10 client.18825 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:07.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.971+0000 7fb365969640 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x559751664010 0x559751666440 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=368bbf9d4773b747 server_cookie=adc05871a6de64b9 in_seq=0 out_seq=0 2024-05-09T09:23:07.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.972+0000 7fb349ffb640 1 -- 172.21.15.64:0/2174166411 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597515c0370 con 0x559751664010 2024-05-09T09:23:07.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.972+0000 7fb349ffb640 10 client.18825 handle_command_reply: tid=0 2024-05-09T09:23:07.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.972+0000 7fb36b1b9740 1 client.18825 shutdown 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.973+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 msgr2=0x559751658310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.973+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x559751658310 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb358009aa0 tx=0x7fb358015f30 comp rx=0 tx=0).stop 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.973+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 shutdown_connections 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.973+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.973+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x559751658850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.974+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x559751664010 0x559751666440 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.974+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2174166411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x559751658310 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.974+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 >> 172.21.15.64:0/2174166411 conn(0x5597516495b0 msgr2=0x5597516487b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:07.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.974+0000 7fb349ffb640 0 client.18825 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.974+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 shutdown_connections 2024-05-09T09:23:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.974+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2174166411 wait complete. 2024-05-09T09:23:07.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:07.975+0000 7fb36b1b9740 20 client.18825 trim_cache size 0 max 16384 2024-05-09T09:23:08.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.008+0000 7fb36b1b9740 1 Processor -- start 2024-05-09T09:23:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.009+0000 7fb36b1b9740 1 -- start start 2024-05-09T09:23:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.009+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164df40 0x559751657f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.009+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 0x55975165f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.009+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559751653aa0 0x559751658460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb344012530 con 0x559751653aa0 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400fda0 con 0x55975164e930 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 0x55975165f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400dbf0 con 0x55975164df40 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 0x55975165f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53548/0 (socket says 172.21.15.64:53548) 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 -- 172.21.15.64:0/4103410740 learned_addr learned my addr 172.21.15.64:0/4103410740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb36696b640 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559751653aa0 0x559751658460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 -- 172.21.15.64:0/4103410740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164df40 msgr2=0x559751657f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164df40 0x559751657f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 -- 172.21.15.64:0/4103410740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559751653aa0 msgr2=0x559751658460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559751653aa0 0x559751658460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.010+0000 7fb365969640 1 -- 172.21.15.64:0/4103410740 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975163ad20 con 0x55975164e930 2024-05-09T09:23:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.011+0000 7fb365969640 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 0x55975165f820 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb36019f5d0 tx=0x7fb35000a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1534b0c817b1c937 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.011+0000 7fb34bfff640 1 -- 172.21.15.64:0/4103410740 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35002abc0 con 0x55975164e930 2024-05-09T09:23:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.011+0000 7fb34bfff640 1 -- 172.21.15.64:0/4103410740 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5597515c0370 con 0x55975164e930 2024-05-09T09:23:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.011+0000 7fb34bfff640 1 -- 172.21.15.64:0/4103410740 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350020bd0 con 0x55975164e930 2024-05-09T09:23:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.012+0000 7fb36b1b9740 1 -- 172.21.15.64:0/4103410740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 msgr2=0x55975165f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.012+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 0x55975165f820 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb36019f5d0 tx=0x7fb35000a2d0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.015+0000 7fb36b1b9740 1 -- 172.21.15.64:0/4103410740 shutdown_connections 2024-05-09T09:23:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.015+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559751653aa0 0x559751658460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.015+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55975164e930 0x55975165f820 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.015+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/4103410740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164df40 0x559751657f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.015+0000 7fb36b1b9740 1 -- 172.21.15.64:0/4103410740 >> 172.21.15.64:0/4103410740 conn(0x5597516495b0 msgr2=0x559751649990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.017+0000 7fb36b1b9740 1 -- 172.21.15.64:0/4103410740 shutdown_connections 2024-05-09T09:23:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.017+0000 7fb36b1b9740 1 -- 172.21.15.64:0/4103410740 wait complete. 2024-05-09T09:23:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.017+0000 7fb36b1b9740 1 -- start start 2024-05-09T09:23:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.017+0000 7fb36b1b9740 1 Processor -- start 2024-05-09T09:23:08.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975165c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x55975165ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400e260 con 0x55975164df40 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400f6d0 con 0x559751653aa0 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400e930 con 0x55975164e930 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36b1b9740 10 client.0 _ll_register_callbacks cb 0x7fb35c0b8110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36616a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975165c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36616a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975165c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59352/0 (socket says 172.21.15.64:59352) 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb36616a640 1 -- 172.21.15.64:0/2639248506 learned_addr learned my addr 172.21.15.64:0/2639248506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.018+0000 7fb365969640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x55975165ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36616a640 1 -- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 msgr2=0x55975165ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36616a640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x55975165ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36616a640 1 -- 172.21.15.64:0/2639248506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 msgr2=0x55975165de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36696b640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36616a640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36616a640 1 -- 172.21.15.64:0/2639248506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975165e480 con 0x55975164df40 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb36616a640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975165c940 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb358009910 tx=0x7fb358009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad028824ce69085a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.019+0000 7fb349ffb640 1 -- 172.21.15.64:0/2639248506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35803b060 con 0x55975164df40 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.020+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5597515c0370 con 0x55975164df40 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.020+0000 7fb349ffb640 1 -- 172.21.15.64:0/2639248506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb35801e400 con 0x55975164df40 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.020+0000 7fb349ffb640 1 -- 172.21.15.64:0/2639248506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb358042050 con 0x55975164df40 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.020+0000 7fb36b1b9740 10 client.18831 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.020+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55975165f780 con 0x55975164df40 2024-05-09T09:23:08.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.021+0000 7fb349ffb640 1 -- 172.21.15.64:0/2639248506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb358019050 con 0x55975164df40 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.021+0000 7fb349ffb640 1 -- 172.21.15.64:0/2639248506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fb358014c60 con 0x55975164df40 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.021+0000 7fb36b1b9740 10 client.18831 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.021+0000 7fb36b1b9740 10 client.18831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.021+0000 7fb36b1b9740 20 client.18831 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.021+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5597516ca940 0x5597516ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.022+0000 7fb36b1b9740 4 client.18831 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:23:08.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.022+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597515c0370 con 0x5597516ca940 2024-05-09T09:23:08.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.023+0000 7fb365969640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5597516ca940 0x5597516ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.023+0000 7fb365969640 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5597516ca940 0x5597516ccd60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=313e4c209cf758b4 server_cookie=12c20391b3e8d173 in_seq=0 out_seq=0 2024-05-09T09:23:08.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.023+0000 7fb349ffb640 10 client.18831 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:08.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.023+0000 7fb349ffb640 1 -- 172.21.15.64:0/2639248506 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597515c0370 con 0x5597516ca940 2024-05-09T09:23:08.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.023+0000 7fb349ffb640 10 client.18831 handle_command_reply: tid=0 2024-05-09T09:23:08.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.023+0000 7fb36b1b9740 1 client.18831 shutdown 2024-05-09T09:23:08.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.030+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 msgr2=0x55975165c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.030+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975165c940 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb358009910 tx=0x7fb358009940 comp rx=0 tx=0).stop 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 shutdown_connections 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559751653aa0 0x55975165de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55975164e930 0x55975165ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5597516ca940 0x5597516ccd60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 --2- 172.21.15.64:0/2639248506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55975164df40 0x55975165c940 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb349ffb640 0 client.18831 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 >> 172.21.15.64:0/2639248506 conn(0x5597516495b0 msgr2=0x559751652e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 shutdown_connections 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 1 -- 172.21.15.64:0/2639248506 wait complete. 2024-05-09T09:23:08.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.031+0000 7fb36b1b9740 20 client.18831 trim_cache size 0 max 16384 2024-05-09T09:23:08.032 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.032+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb34c0a2d60 msgr2=0x7fb34c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.032+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb34c0a2d60 0x7fb34c0a5220 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb35000afd0 tx=0x7fb350013460 comp rx=0 tx=0).stop 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.032+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 msgr2=0x7fb3601a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.032+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 0x7fb3601a5670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb35c005ec0 tx=0x7fb35c004b80 comp rx=0 tx=0).stop 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 shutdown_connections 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb34c0a2d60 0x7fb34c0a5220 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360077dd0 0x7fb3601a5670 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb360077430 0x7fb36019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 --2- 172.21.15.64:0/2220452633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3600761f0 0x7fb36019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 >> 172.21.15.64:0/2220452633 conn(0x7fb3600ff020 msgr2=0x7fb360100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 shutdown_connections 2024-05-09T09:23:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.033+0000 7fb3683ce640 1 -- 172.21.15.64:0/2220452633 wait complete. 2024-05-09T09:23:08.054 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:07 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3576198326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:08.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:23:08.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:07 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3576198326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:08.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/4031973032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 msgr2=0x7fe1dc09deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/4031973032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 0x7fe1dc09deb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe1e406b220 tx=0x7fe1e40960a0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/4031973032 shutdown_connections 2024-05-09T09:23:08.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/4031973032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 0x7fe1dc09deb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/4031973032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc099130 0x7fe1dc09b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/4031973032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc096800 0x7fe1dc098bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.260+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/4031973032 >> 172.21.15.64:0/4031973032 conn(0x7fe1dc0901d0 msgr2=0x7fe1dc092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.261+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/4031973032 shutdown_connections 2024-05-09T09:23:08.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.261+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/4031973032 wait complete. 2024-05-09T09:23:08.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 Processor -- start 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 -- start start 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc096800 0x7fe1dc0a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 0x7fe1dc0a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 0x7fe1dc09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc0a3aa0 con 0x7fe1dc096800 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc0a3920 con 0x7fe1dc09ba60 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.262+0000 7fe1eb6ee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc0a3c20 con 0x7fe1dc099130 2024-05-09T09:23:08.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 0x7fe1dc0a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 0x7fe1dc0a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42324/0 (socket says 172.21.15.64:42324) 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 -- 172.21.15.64:0/890850039 learned_addr learned my addr 172.21.15.64:0/890850039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e948a640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc096800 0x7fe1dc0a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 0x7fe1dc09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 -- 172.21.15.64:0/890850039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 msgr2=0x7fe1dc09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 0x7fe1dc09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 -- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc096800 msgr2=0x7fe1dc0a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc096800 0x7fe1dc0a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.263+0000 7fe1e8c89640 1 -- 172.21.15.64:0/890850039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc09fb80 con 0x7fe1dc099130 2024-05-09T09:23:08.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.264+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 0x7fe1dc0a2c90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400e9a0 tx=0x7fe1d400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.264+0000 7fe1da7fc640 1 -- 172.21.15.64:0/890850039 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400cd80 con 0x7fe1dc099130 2024-05-09T09:23:08.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.264+0000 7fe1da7fc640 1 -- 172.21.15.64:0/890850039 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1d40035f0 con 0x7fe1dc099130 2024-05-09T09:23:08.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.264+0000 7fe1da7fc640 1 -- 172.21.15.64:0/890850039 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4010690 con 0x7fe1dc099130 2024-05-09T09:23:08.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.264+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc13a480 con 0x7fe1dc099130 2024-05-09T09:23:08.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.264+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc13a910 con 0x7fe1dc099130 2024-05-09T09:23:08.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.268+0000 7fe1ee3f7740 1 Processor -- start 2024-05-09T09:23:08.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 -- start start 2024-05-09T09:23:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f68550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f6d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f70120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5eccc40 con 0x55bbb5f68130 2024-05-09T09:23:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5e74580 con 0x55bbb5f6dc90 2024-05-09T09:23:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.270+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5ec8c00 con 0x55bbb5f68b20 2024-05-09T09:23:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.271+0000 7fe1da7fc640 1 -- 172.21.15.64:0/890850039 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe1d4004000 con 0x7fe1dc099130 2024-05-09T09:23:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.271+0000 7fe1e9c8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f68550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.271+0000 7fe1e9c8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f68550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59374/0 (socket says 172.21.15.64:59374) 2024-05-09T09:23:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.271+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/1619549418 learned_addr learned my addr 172.21.15.64:0/1619549418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.271+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f70120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.271+0000 7fe1e948a640 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f6d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/1619549418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 msgr2=0x55bbb5f6d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f6d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/1619549418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 msgr2=0x55bbb5f70120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f70120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/1619549418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb5be03f0 con 0x55bbb5f68130 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f68550 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4050dd0 tx=0x7fe1e409dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28048863e348eab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1cf7fe640 1 -- 172.21.15.64:0/1619549418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e409d650 con 0x55bbb5f68130 2024-05-09T09:23:08.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.272+0000 7fe1cf7fe640 1 -- 172.21.15.64:0/1619549418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1e409a040 con 0x55bbb5f68130 2024-05-09T09:23:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.273+0000 7fe1da7fc640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1d00a2d10 0x7fe1d00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.273+0000 7fe1da7fc640 1 -- 172.21.15.64:0/890850039 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1d40cf380 con 0x7fe1dc099130 2024-05-09T09:23:08.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.273+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1619549418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 msgr2=0x55bbb5f68550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.273+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f68550 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4050dd0 tx=0x7fe1e409dc80 comp rx=0 tx=0).stop 2024-05-09T09:23:08.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.274+0000 7fe1e948a640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1d00a2d10 0x7fe1d00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.275+0000 7fe1e948a640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1d00a2d10 0x7fe1d00a51d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55bbb5f68f20 tx=0x7fe1e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.275+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1619549418 shutdown_connections 2024-05-09T09:23:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.275+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f70120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.275+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f6d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.275+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1619549418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f68550 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.275+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1619549418 >> 172.21.15.64:0/1619549418 conn(0x55bbb5f637a0 msgr2=0x55bbb5f63ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.276+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1619549418 shutdown_connections 2024-05-09T09:23:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.278+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1619549418 wait complete. 2024-05-09T09:23:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.278+0000 7fe1ee3f7740 1 -- start start 2024-05-09T09:23:08.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.278+0000 7fe1ee3f7740 1 Processor -- start 2024-05-09T09:23:08.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f72530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f72a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f79ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5f70bd0 con 0x55bbb5f6dc90 2024-05-09T09:23:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5e87c40 con 0x55bbb5f68130 2024-05-09T09:23:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5f70d50 con 0x55bbb5f68b20 2024-05-09T09:23:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1ee3f7740 10 client.0 _ll_register_callbacks cb 0x55bbb5bdfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:08.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1e9c8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f79ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1e9c8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f79ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59380/0 (socket says 172.21.15.64:59380) 2024-05-09T09:23:08.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.279+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/1093995367 learned_addr learned my addr 172.21.15.64:0/1093995367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f72a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e948a640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f72530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e8c89640 1 -- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 msgr2=0x55bbb5f72530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f72530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e8c89640 1 -- 172.21.15.64:0/1093995367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f6dc90 msgr2=0x55bbb5f79ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f79ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.280+0000 7fe1e8c89640 1 -- 172.21.15.64:0/1093995367 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb5caa7d0 con 0x55bbb5f68b20 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.281+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f72a70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40ccbe0 tx=0x7fe1d4097470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d81897db0ce4ce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.281+0000 7fe1ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.281+0000 7fe1ccff9640 1 -- 172.21.15.64:0/1093995367 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d409b430 con 0x55bbb5f68b20 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.281+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e409a040 con 0x55bbb5f68b20 2024-05-09T09:23:08.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.281+0000 7fe1ccff9640 1 -- 172.21.15.64:0/1093995367 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1d40d2d40 con 0x55bbb5f68b20 2024-05-09T09:23:08.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.282+0000 7fe1ccff9640 1 -- 172.21.15.64:0/1093995367 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe1d409b250 con 0x55bbb5f68b20 2024-05-09T09:23:08.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.282+0000 7fe1ee3f7740 10 client.28480 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:08.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.282+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bbb5f7cf40 con 0x55bbb5f68b20 2024-05-09T09:23:08.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.282+0000 7fe1ccff9640 1 -- 172.21.15.64:0/1093995367 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1e409a040 con 0x55bbb5f68b20 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ccff9640 1 -- 172.21.15.64:0/1093995367 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe1d40b4af0 con 0x55bbb5f68b20 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ee3f7740 10 client.28480 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ee3f7740 10 client.28480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ee3f7740 20 client.28480 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5f7ea30 0x55bbb5f80eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ee3f7740 4 client.28480 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb5eda370 con 0x55bbb5f7ea30 2024-05-09T09:23:08.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.283+0000 7fe1e948a640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5f7ea30 0x55bbb5f80eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.284+0000 7fe1e948a640 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5f7ea30 0x55bbb5f80eb0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eefeb14bea5ceb41 server_cookie=55be1d1e91dc64dd in_seq=0 out_seq=0 2024-05-09T09:23:08.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.284+0000 7fe1ccff9640 10 client.28480 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.285+0000 7fe1ccff9640 1 -- 172.21.15.64:0/1093995367 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbb5eda370 con 0x55bbb5f7ea30 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.285+0000 7fe1ccff9640 10 client.28480 handle_command_reply: tid=0 2024-05-09T09:23:08.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.285+0000 7fe1ee3f7740 1 client.28480 shutdown 2024-05-09T09:23:08.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.287+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 msgr2=0x55bbb5f72a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.287+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f72a70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40ccbe0 tx=0x7fe1d4097470 comp rx=0 tx=0).stop 2024-05-09T09:23:08.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 shutdown_connections 2024-05-09T09:23:08.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5f7ea30 0x55bbb5f80eb0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f79ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f72a70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/1093995367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f72530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 >> 172.21.15.64:0/1093995367 conn(0x55bbb5f637a0 msgr2=0x55bbb5f629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.288+0000 7fe1ccff9640 0 client.28480 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:08.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.294+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 shutdown_connections 2024-05-09T09:23:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.294+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/1093995367 wait complete. 2024-05-09T09:23:08.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.294+0000 7fe1ee3f7740 20 client.28480 trim_cache size 0 max 16384 2024-05-09T09:23:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.320+0000 7fe1ee3f7740 1 Processor -- start 2024-05-09T09:23:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.320+0000 7fe1ee3f7740 1 -- start start 2024-05-09T09:23:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.321+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f6d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.321+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68b20 0x55bbb5f79a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1e8c89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68b20 0x55bbb5f79a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1e948a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f6d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1e948a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f6d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53590/0 (socket says 172.21.15.64:53590) 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1e948a640 1 -- 172.21.15.64:0/538369815 learned_addr learned my addr 172.21.15.64:0/538369815 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f72530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c8012490 con 0x55bbb5f68b20 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c800fd00 con 0x55bbb5f68130 2024-05-09T09:23:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c800db50 con 0x55bbb5f6dc90 2024-05-09T09:23:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.322+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f72530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:07 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3576198326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/538369815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 msgr2=0x55bbb5f6d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f6d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/538369815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68b20 msgr2=0x55bbb5f79a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68b20 0x55bbb5f79a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1e9c8b640 1 -- 172.21.15.64:0/538369815 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb5f81530 con 0x55bbb5f6dc90 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f72530 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe1e402d780 tx=0x7fe1e4067600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae5efe29038ee46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.323+0000 7fe1ceffd640 1 -- 172.21.15.64:0/538369815 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e409d450 con 0x55bbb5f6dc90 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.324+0000 7fe1ceffd640 1 -- 172.21.15.64:0/538369815 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1e409a040 con 0x55bbb5f6dc90 2024-05-09T09:23:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.324+0000 7fe1ceffd640 1 -- 172.21.15.64:0/538369815 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4069110 con 0x55bbb5f6dc90 2024-05-09T09:23:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f6dc90 msgr2=0x55bbb5f72530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f72530 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe1e402d780 tx=0x7fe1e4067600 comp rx=0 tx=0).stop 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 shutdown_connections 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f72530 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68b20 0x55bbb5f79a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/538369815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f68130 0x55bbb5f6d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 >> 172.21.15.64:0/538369815 conn(0x55bbb5f637a0 msgr2=0x55bbb5f69f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 shutdown_connections 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.325+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/538369815 wait complete. 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.326+0000 7fe1ee3f7740 1 -- start start 2024-05-09T09:23:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.326+0000 7fe1ee3f7740 1 Processor -- start 2024-05-09T09:23:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f76ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f77010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f77fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c800e1c0 con 0x55bbb5f68130 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c800f630 con 0x55bbb5f6dc90 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c800e890 con 0x55bbb5f68b20 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1e948a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f76ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1ee3f7740 10 client.0 _ll_register_callbacks cb 0x7fe1d400ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1e948a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f76ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59390/0 (socket says 172.21.15.64:59390) 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.327+0000 7fe1e948a640 1 -- 172.21.15.64:0/2076836193 learned_addr learned my addr 172.21.15.64:0/2076836193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f77fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f77010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e948a640 1 -- 172.21.15.64:0/2076836193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 msgr2=0x55bbb5f77010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e948a640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f77010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e948a640 1 -- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 msgr2=0x55bbb5f77fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e948a640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f77fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e948a640 1 -- 172.21.15.64:0/2076836193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb5f785c0 con 0x55bbb5f68130 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f77010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.328+0000 7fe1e9c8b640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f77fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.329+0000 7fe1e948a640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f76ad0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf4e612c6895a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.329+0000 7fe1ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.329+0000 7fe1ccff9640 1 -- 172.21.15.64:0/2076836193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0036060 con 0x55bbb5f68130 2024-05-09T09:23:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.329+0000 7fe1ccff9640 1 -- 172.21.15.64:0/2076836193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe1e0015a00 con 0x55bbb5f68130 2024-05-09T09:23:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.329+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e409a040 con 0x55bbb5f68130 2024-05-09T09:23:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.330+0000 7fe1ccff9640 1 -- 172.21.15.64:0/2076836193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe1e003d050 con 0x55bbb5f68130 2024-05-09T09:23:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.330+0000 7fe1ee3f7740 10 client.18849 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.330+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bbb5f79990 con 0x55bbb5f68130 2024-05-09T09:23:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.330+0000 7fe1ccff9640 1 -- 172.21.15.64:0/2076836193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1e409a040 con 0x55bbb5f68130 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ccff9640 1 -- 172.21.15.64:0/2076836193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe1e0031ce0 con 0x55bbb5f68130 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ee3f7740 10 client.18849 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ee3f7740 10 client.18849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ee3f7740 20 client.18849 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5fe5510 0x55bbb5fe7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ee3f7740 4 client.18849 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.331+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbb5ed46c0 con 0x55bbb5fe5510 2024-05-09T09:23:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.332+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5fe5510 0x55bbb5fe7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:08.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.332+0000 7fe1ccff9640 10 client.18849 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:08.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.333+0000 7fe1e8c89640 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5fe5510 0x55bbb5fe7930 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa53d07b9fec321a server_cookie=5443d32c10abebc0 in_seq=0 out_seq=0 2024-05-09T09:23:08.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.333+0000 7fe1ccff9640 1 -- 172.21.15.64:0/2076836193 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbb5ed46c0 con 0x55bbb5fe5510 2024-05-09T09:23:08.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.333+0000 7fe1ccff9640 10 client.18849 handle_command_reply: tid=0 2024-05-09T09:23:08.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.333+0000 7fe1ee3f7740 1 client.18849 shutdown 2024-05-09T09:23:08.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 msgr2=0x55bbb5f76ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f76ad0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0013000 comp rx=0 tx=0).stop 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 shutdown_connections 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55bbb5fe5510 0x55bbb5fe7930 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbb5f6dc90 0x55bbb5f77fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbb5f68b20 0x55bbb5f77010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 --2- 172.21.15.64:0/2076836193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbb5f68130 0x55bbb5f76ad0 secure :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0013000 comp rx=0 tx=0).stop 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 >> 172.21.15.64:0/2076836193 conn(0x55bbb5f637a0 msgr2=0x55bbb5f72470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.334+0000 7fe1ccff9640 0 client.18849 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.335+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 shutdown_connections 2024-05-09T09:23:08.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.335+0000 7fe1ee3f7740 1 -- 172.21.15.64:0/2076836193 wait complete. 2024-05-09T09:23:08.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.337+0000 7fe1ee3f7740 20 client.18849 trim_cache size 0 max 16384 2024-05-09T09:23:08.337 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:23:08.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1d00a2d10 msgr2=0x7fe1d00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1d00a2d10 0x7fe1d00a51d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55bbb5f68f20 tx=0x7fe1e0008040 comp rx=0 tx=0).stop 2024-05-09T09:23:08.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 msgr2=0x7fe1dc0a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:08.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 0x7fe1dc0a2c90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400e9a0 tx=0x7fe1d400ee60 comp rx=0 tx=0).stop 2024-05-09T09:23:08.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 shutdown_connections 2024-05-09T09:23:08.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe1d00a2d10 0x7fe1d00a51d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1dc09ba60 0x7fe1dc09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1dc099130 0x7fe1dc0a2c90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 --2- 172.21.15.64:0/890850039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc096800 0x7fe1dc0a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:08.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.339+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 >> 172.21.15.64:0/890850039 conn(0x7fe1dc0901d0 msgr2=0x7fe1dc097140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:08.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.341+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 shutdown_connections 2024-05-09T09:23:08.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:08.341+0000 7fe1eb6ee640 1 -- 172.21.15.64:0/890850039 wait complete. 2024-05-09T09:23:09.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:08 smithi064 ceph-mon[30149]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-05-09T09:23:09.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:08 smithi179 ceph-mon[34086]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-05-09T09:23:09.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:08 smithi139 ceph-mon[33599]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-05-09T09:23:09.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:23:09.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:09.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:23:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 -- 172.21.15.64:0/100632466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10dd60 msgr2=0x7f8bcc110220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 --2- 172.21.15.64:0/100632466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10dd60 0x7f8bcc110220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009be0 tx=0x7f8bc402c650 comp rx=0 tx=0).stop 2024-05-09T09:23:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 -- 172.21.15.64:0/100632466 shutdown_connections 2024-05-09T09:23:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 --2- 172.21.15.64:0/100632466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10dd60 0x7f8bcc110220 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 --2- 172.21.15.64:0/100632466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc10b280 0x7f8bcc10d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 --2- 172.21.15.64:0/100632466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 0x7f8bcc10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.557+0000 7f8bd4509640 1 -- 172.21.15.64:0/100632466 >> 172.21.15.64:0/100632466 conn(0x7f8bcc06e880 msgr2=0x7f8bcc070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.561+0000 7f8bd4509640 1 -- 172.21.15.64:0/100632466 shutdown_connections 2024-05-09T09:23:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.561+0000 7f8bd4509640 1 -- 172.21.15.64:0/100632466 wait complete. 2024-05-09T09:23:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.563+0000 7f8bd4509640 1 Processor -- start 2024-05-09T09:23:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.563+0000 7f8bd4509640 1 -- start start 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd4509640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 0x7f8bcc1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd4509640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 0x7f8bcc1a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd4509640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc10dd60 0x7f8bcc1adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd4509640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc11c080 con 0x7f8bcc1088c0 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd4509640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc11bf00 con 0x7f8bcc10b280 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd4509640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc11c200 con 0x7f8bcc10dd60 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 0x7f8bcc1a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 0x7f8bcc1a6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53608/0 (socket says 172.21.15.64:53608) 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/155352284 learned_addr learned my addr 172.21.15.64:0/155352284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 0x7f8bcc1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/155352284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc10dd60 msgr2=0x7f8bcc1adaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc10dd60 0x7f8bcc1adaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 msgr2=0x7f8bcc1a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 0x7f8bcc1a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.564+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/155352284 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc1ae110 con 0x7f8bcc10b280 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.565+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 0x7f8bcc1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.565+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 0x7f8bcc1a6a30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800ea60 tx=0x7f8bc800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.565+0000 7f8bc37fe640 1 -- 172.21.15.64:0/155352284 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800cd80 con 0x7f8bcc10b280 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.565+0000 7f8bc37fe640 1 -- 172.21.15.64:0/155352284 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bc80054d0 con 0x7f8bcc10b280 2024-05-09T09:23:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.566+0000 7f8bc37fe640 1 -- 172.21.15.64:0/155352284 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8004e60 con 0x7f8bcc10b280 2024-05-09T09:23:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.566+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1ae400 con 0x7f8bcc10b280 2024-05-09T09:23:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.566+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc1ae950 con 0x7f8bcc10b280 2024-05-09T09:23:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.571+0000 7f8bd720c740 1 Processor -- start 2024-05-09T09:23:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.573+0000 7f8bc37fe640 1 -- 172.21.15.64:0/155352284 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8bc8005020 con 0x7f8bcc10b280 2024-05-09T09:23:09.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.574+0000 7f8bc37fe640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8bb80a2d40 0x7f8bb80a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.574+0000 7f8bc37fe640 1 -- 172.21.15.64:0/155352284 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc80cf2a0 con 0x7f8bcc10b280 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 -- start start 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8b46b0 0x55e9ad8b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad818990 con 0x55e9ad8ba210 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad7c05b0 con 0x55e9ad8b46b0 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd720c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad814950 con 0x55e9ad8b50a0 2024-05-09T09:23:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.580+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8bb80a2d40 0x7f8bb80a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.581+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.581+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59408/0 (socket says 172.21.15.64:59408) 2024-05-09T09:23:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.581+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/706954234 learned_addr learned my addr 172.21.15.64:0/706954234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.581+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/706954234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8b46b0 msgr2=0x55e9ad8b4ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8b46b0 0x55e9ad8b4ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/706954234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 msgr2=0x55e9ad8bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/706954234 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ad5f47e0 con 0x55e9ad8b50a0 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bd1aa4640 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8b9ba0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8bc809b6a0 tx=0x7f8bc809b310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8339caba830a8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.582+0000 7f8bc0bfe640 1 -- 172.21.15.64:0/706954234 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8016300 con 0x55e9ad8b50a0 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.583+0000 7f8bc0bfe640 1 -- 172.21.15.64:0/706954234 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bc809be10 con 0x55e9ad8b50a0 2024-05-09T09:23:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.583+0000 7f8bc0bfe640 1 -- 172.21.15.64:0/706954234 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8004830 con 0x55e9ad8b50a0 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.584+0000 7f8bd720c740 1 -- 172.21.15.64:0/706954234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 msgr2=0x55e9ad8b9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.584+0000 7f8bd720c740 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8b9ba0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8bc809b6a0 tx=0x7f8bc809b310 comp rx=0 tx=0).stop 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.585+0000 7f8bd720c740 1 -- 172.21.15.64:0/706954234 shutdown_connections 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.585+0000 7f8bd720c740 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.585+0000 7f8bd720c740 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8b9ba0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.585+0000 7f8bd720c740 1 --2- 172.21.15.64:0/706954234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8b46b0 0x55e9ad8b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.585+0000 7f8bd720c740 1 -- 172.21.15.64:0/706954234 >> 172.21.15.64:0/706954234 conn(0x55e9ad87c0a0 msgr2=0x55e9ad8b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.586+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8bb80a2d40 0x7f8bb80a5200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009be0 tx=0x7f8bc4032680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.587+0000 7f8bd720c740 1 -- 172.21.15.64:0/706954234 shutdown_connections 2024-05-09T09:23:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.588+0000 7f8bd720c740 1 -- 172.21.15.64:0/706954234 wait complete. 2024-05-09T09:23:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.588+0000 7f8bd720c740 1 -- start start 2024-05-09T09:23:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 Processor -- start 2024-05-09T09:23:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad8bd210 con 0x55e9ad8ba210 2024-05-09T09:23:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad7d3b40 con 0x55e9ad8c4310 2024-05-09T09:23:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad8bd390 con 0x55e9ad8b50a0 2024-05-09T09:23:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.589+0000 7f8bd720c740 10 client.0 _ll_register_callbacks cb 0x55e9ad7c80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53628/0 (socket says 172.21.15.64:53628) 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/2106007813 learned_addr learned my addr 172.21.15.64:0/2106007813 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/2106007813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 msgr2=0x55e9ad8beda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8beda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/2106007813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 msgr2=0x55e9ad8bf2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bf2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.592+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/2106007813 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ad654f70 con 0x55e9ad8c4310 2024-05-09T09:23:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.593+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8c6700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4032ba0 tx=0x7f8bc4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9925aa850b27c214 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.593+0000 7f8bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.594+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/2106007813 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc404f020 con 0x55e9ad8c4310 2024-05-09T09:23:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.594+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/2106007813 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc4004510 con 0x55e9ad8c4310 2024-05-09T09:23:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.594+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc809be10 con 0x55e9ad8c4310 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.594+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/2106007813 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8bc4038030 con 0x55e9ad8c4310 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.594+0000 7f8bd720c740 10 client.28322 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.594+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e9ad4ec170 con 0x55e9ad8c4310 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/2106007813 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bc809be10 con 0x55e9ad8c4310 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/2106007813 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f8bc4042430 con 0x55e9ad8c4310 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bd720c740 10 client.28322 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bd720c740 10 client.28322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bd720c740 20 client.28322 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bd720c740 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad8cb120 0x55e9ad8cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bd720c740 4 client.28322 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.595+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ad8260c0 con 0x55e9ad8cb120 2024-05-09T09:23:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.596+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad8cb120 0x55e9ad8cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.598+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad8cb120 0x55e9ad8cd5a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6971954f9a7903c server_cookie=b4eeb31d72aaa5cf in_seq=0 out_seq=0 2024-05-09T09:23:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.598+0000 7f8bb5ffb640 10 client.28322 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.599+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/2106007813 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9ad8260c0 con 0x55e9ad8cb120 2024-05-09T09:23:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.599+0000 7f8bb5ffb640 10 client.28322 handle_command_reply: tid=0 2024-05-09T09:23:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 client.28322 shutdown 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 msgr2=0x55e9ad8c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8c6700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4032ba0 tx=0x7f8bc4005fd0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 shutdown_connections 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad8cb120 0x55e9ad8cd5a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8c6700 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 --2- 172.21.15.64:0/2106007813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 >> 172.21.15.64:0/2106007813 conn(0x55e9ad87c0a0 msgr2=0x55e9ad8af270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.600+0000 7f8bb5ffb640 0 client.28322 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.601+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 shutdown_connections 2024-05-09T09:23:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.601+0000 7f8bd720c740 1 -- 172.21.15.64:0/2106007813 wait complete. 2024-05-09T09:23:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.601+0000 7f8bd720c740 20 client.28322 trim_cache size 0 max 16384 2024-05-09T09:23:09.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.636+0000 7f8bd720c740 1 Processor -- start 2024-05-09T09:23:09.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.636+0000 7f8bd720c740 1 -- start start 2024-05-09T09:23:09.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd720c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1bb850 con 0x55e9ad8ba210 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd720c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b90c0 con 0x55e9ad8c4310 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd720c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b6f10 con 0x55e9ad8b50a0 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd1aa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd1aa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59412/0 (socket says 172.21.15.64:59412) 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.637+0000 7f8bd1aa4640 1 -- 172.21.15.64:0/3240436266 learned_addr learned my addr 172.21.15.64:0/3240436266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3240436266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 msgr2=0x55e9ad8bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8bd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3240436266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 msgr2=0x55e9ad8bdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3240436266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ad8c95a0 con 0x55e9ad8c4310 2024-05-09T09:23:09.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.638+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8ae9e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40428d0 tx=0x7f8bc4032fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c760e87338f9673e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.639+0000 7f8bb7fff640 1 -- 172.21.15.64:0/3240436266 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4063070 con 0x55e9ad8c4310 2024-05-09T09:23:09.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.639+0000 7f8bb7fff640 1 -- 172.21.15.64:0/3240436266 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bc40047c0 con 0x55e9ad8c4310 2024-05-09T09:23:09.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.639+0000 7f8bb7fff640 1 -- 172.21.15.64:0/3240436266 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc404f030 con 0x55e9ad8c4310 2024-05-09T09:23:09.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.640+0000 7f8bd720c740 1 -- 172.21.15.64:0/3240436266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 msgr2=0x55e9ad8ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.640+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8ae9e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40428d0 tx=0x7f8bc4032fa0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.640+0000 7f8bd720c740 1 -- 172.21.15.64:0/3240436266 shutdown_connections 2024-05-09T09:23:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.641+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8c4310 0x55e9ad8ae9e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.641+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8ba210 0x55e9ad8bdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.641+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3240436266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.641+0000 7f8bd720c740 1 -- 172.21.15.64:0/3240436266 >> 172.21.15.64:0/3240436266 conn(0x55e9ad87c0a0 msgr2=0x55e9ad8afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.642+0000 7f8bd720c740 1 -- 172.21.15.64:0/3240436266 shutdown_connections 2024-05-09T09:23:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.642+0000 7f8bd720c740 1 -- 172.21.15.64:0/3240436266 wait complete. 2024-05-09T09:23:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.642+0000 7f8bd720c740 1 -- start start 2024-05-09T09:23:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 Processor -- start 2024-05-09T09:23:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8ba210 0x55e9ad8dc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 0x55e9ad8dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b7580 con 0x55e9ad8b50a0 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b89f0 con 0x55e9ad8ba210 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.643+0000 7f8bd720c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b7c50 con 0x55e9ad8c4310 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.644+0000 7f8bd720c740 10 client.0 _ll_register_callbacks cb 0x7f8bc8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.644+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 0x55e9ad8dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.644+0000 7f8bd2aa6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 0x55e9ad8dd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42402/0 (socket says 172.21.15.64:42402) 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.644+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3407166462 learned_addr learned my addr 172.21.15.64:0/3407166462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.645+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.645+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8ba210 msgr2=0x55e9ad8dc810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.645+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8ba210 0x55e9ad8dc810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.645+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3407166462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8b50a0 msgr2=0x55e9ad8dc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.645+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8dc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.645+0000 7f8bd2aa6640 1 -- 172.21.15.64:0/3407166462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ad8dde10 con 0x55e9ad8c4310 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.647+0000 7f8bd2aa6640 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 0x55e9ad8dd820 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40429c0 tx=0x7f8bc4042450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4823adeb9c51192b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.647+0000 7f8bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.647+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/3407166462 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc406c060 con 0x55e9ad8c4310 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.647+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/3407166462 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc4004310 con 0x55e9ad8c4310 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.648+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc40047c0 con 0x55e9ad8c4310 2024-05-09T09:23:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.648+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/3407166462 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8bc404f070 con 0x55e9ad8c4310 2024-05-09T09:23:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.651+0000 7f8bd720c740 10 client.28498 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.651+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e9ad9313e0 con 0x55e9ad8c4310 2024-05-09T09:23:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.652+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/3407166462 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bc4004550 con 0x55e9ad8c4310 2024-05-09T09:23:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.652+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/3407166462 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f8bc4067570 con 0x55e9ad8c4310 2024-05-09T09:23:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.655+0000 7f8bd720c740 10 client.28498 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.655+0000 7f8bd720c740 10 client.28498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.655+0000 7f8bd720c740 20 client.28498 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.655+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad931d00 0x55e9ad934120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.655+0000 7f8bd720c740 4 client.28498 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:23:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.655+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f8bc40047c0 con 0x55e9ad931d00 2024-05-09T09:23:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.656+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad931d00 0x55e9ad934120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.657+0000 7f8bd22a5640 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad931d00 0x55e9ad934120 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4df7a82a44dbcf server_cookie=645981544f0f0d75 in_seq=0 out_seq=0 2024-05-09T09:23:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.658+0000 7f8bb5ffb640 10 client.28498 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.659+0000 7f8bb5ffb640 1 -- 172.21.15.64:0/3407166462 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f8bc40047c0 con 0x55e9ad931d00 2024-05-09T09:23:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.659+0000 7f8bb5ffb640 10 client.28498 handle_command_reply: tid=0 2024-05-09T09:23:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.660+0000 7f8bd720c740 1 client.28498 shutdown 2024-05-09T09:23:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.662+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 msgr2=0x55e9ad8dd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.662+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 0x55e9ad8dd820 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40429c0 tx=0x7f8bc4042450 comp rx=0 tx=0).stop 2024-05-09T09:23:09.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 shutdown_connections 2024-05-09T09:23:09.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e9ad931d00 0x55e9ad934120 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9ad8c4310 0x55e9ad8dd820 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e9ad8ba210 0x55e9ad8dc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bd720c740 1 --2- 172.21.15.64:0/3407166462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9ad8b50a0 0x55e9ad8dc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 >> 172.21.15.64:0/3407166462 conn(0x55e9ad87c0a0 msgr2=0x55e9ad8bd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.665+0000 7f8bb5ffb640 0 client.28498 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.666+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 shutdown_connections 2024-05-09T09:23:09.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.666+0000 7f8bd720c740 1 -- 172.21.15.64:0/3407166462 wait complete. 2024-05-09T09:23:09.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.668+0000 7f8bd720c740 20 client.28498 trim_cache size 0 max 16384 2024-05-09T09:23:09.668 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:23:09.668 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:23:09.668 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "a3cac930-318f-4f97-9d36-e36e2409737d", 2024-05-09T09:23:09.668 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:23:09.668 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8bb80a2d40 msgr2=0x7f8bb80a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8bb80a2d40 0x7f8bb80a5200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009be0 tx=0x7f8bc4032680 comp rx=0 tx=0).stop 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 msgr2=0x7f8bcc1a6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 0x7f8bcc1a6a30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800ea60 tx=0x7f8bc800ef20 comp rx=0 tx=0).stop 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 shutdown_connections 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f8bb80a2d40 0x7f8bb80a5200 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bcc10dd60 0x7f8bcc1adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc10b280 0x7f8bcc1a6a30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 --2- 172.21.15.64:0/155352284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bcc1088c0 0x7f8bcc1a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.670+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 >> 172.21.15.64:0/155352284 conn(0x7f8bcc06e880 msgr2=0x7f8bcc10bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.671+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 shutdown_connections 2024-05-09T09:23:09.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.671+0000 7f8bd4509640 1 -- 172.21.15.64:0/155352284 wait complete. 2024-05-09T09:23:09.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:23:09.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:23:09.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.901+0000 7f0acd27b640 1 -- 172.21.15.64:0/2383346370 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4031880 con 0x7f0ac8100100 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 -- 172.21.15.64:0/2383346370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 msgr2=0x7f0ac8100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/2383346370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac8100520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009900 tx=0x7f0ac402a400 comp rx=0 tx=0).stop 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 -- 172.21.15.64:0/2383346370 shutdown_connections 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/2383346370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 0x7f0ac8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/2383346370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8100a60 0x7f0ac8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/2383346370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac8100520 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.902+0000 7f0ad04e1640 1 -- 172.21.15.64:0/2383346370 >> 172.21.15.64:0/2383346370 conn(0x7f0ac80fbd20 msgr2=0x7f0ac80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.904+0000 7f0ad04e1640 1 -- 172.21.15.64:0/2383346370 shutdown_connections 2024-05-09T09:23:09.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.905+0000 7f0ad04e1640 1 -- 172.21.15.64:0/2383346370 wait complete. 2024-05-09T09:23:09.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.905+0000 7f0ad04e1640 1 Processor -- start 2024-05-09T09:23:09.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 -- start start 2024-05-09T09:23:09.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8100a60 0x7f0ac819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 0x7f0ac81a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81033c0 con 0x7f0ac8101400 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8103240 con 0x7f0ac8100a60 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ad04e1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8103540 con 0x7f0ac8100100 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ace27d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ace27d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:42412/0 (socket says 172.21.15.64:42412) 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0ace27d640 1 -- 172.21.15.64:0/3161381642 learned_addr learned my addr 172.21.15.64:0/3161381642 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 0x7f0ac81a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.906+0000 7f0acda7c640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8100a60 0x7f0ac819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.907+0000 7f0acea7e640 1 -- 172.21.15.64:0/3161381642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 msgr2=0x7f0ac819e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.907+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac819e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.907+0000 7f0acea7e640 1 -- 172.21.15.64:0/3161381642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8100a60 msgr2=0x7f0ac819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.907+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8100a60 0x7f0ac819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.907+0000 7f0acea7e640 1 -- 172.21.15.64:0/3161381642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac81a5d30 con 0x7f0ac8101400 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.907+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 0x7f0ac81a56c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e970 tx=0x7f0ab800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.908+0000 7f0abf7fe640 1 -- 172.21.15.64:0/3161381642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800ccc0 con 0x7f0ac8101400 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.908+0000 7f0abf7fe640 1 -- 172.21.15.64:0/3161381642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab8005020 con 0x7f0ac8101400 2024-05-09T09:23:09.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.908+0000 7f0abf7fe640 1 -- 172.21.15.64:0/3161381642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8004a00 con 0x7f0ac8101400 2024-05-09T09:23:09.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.908+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac81a5fc0 con 0x7f0ac8101400 2024-05-09T09:23:09.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.908+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac81a6430 con 0x7f0ac8101400 2024-05-09T09:23:09.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.912+0000 7f0abf7fe640 1 -- 172.21.15.64:0/3161381642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ab80051d0 con 0x7f0ac8101400 2024-05-09T09:23:09.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.914+0000 7f0abf7fe640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2d10 0x7f0ab40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.915+0000 7f0abf7fe640 1 -- 172.21.15.64:0/3161381642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ab80cf270 con 0x7f0ac8101400 2024-05-09T09:23:09.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.915+0000 7f0ace27d640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2d10 0x7f0ab40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.915+0000 7f0ad31f3740 1 Processor -- start 2024-05-09T09:23:09.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.915+0000 7f0ad31f3740 1 -- start start 2024-05-09T09:23:09.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a709350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a70e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a710f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55947a66dd00 con 0x55947a708f30 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55947a6154c0 con 0x55947a70ea90 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55947a669cc0 con 0x55947a709920 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ace27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a709350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ace27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a709350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59454/0 (socket says 172.21.15.64:59454) 2024-05-09T09:23:09.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0ace27d640 1 -- 172.21.15.64:0/3531661560 learned_addr learned my addr 172.21.15.64:0/3531661560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a710f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.916+0000 7f0acda7c640 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a70e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.917+0000 7f0acea7e640 1 -- 172.21.15.64:0/3531661560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 msgr2=0x55947a70e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.917+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a70e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.917+0000 7f0acea7e640 1 -- 172.21.15.64:0/3531661560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 msgr2=0x55947a709350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.917+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a709350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.917+0000 7f0acea7e640 1 -- 172.21.15.64:0/3531661560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947a44b7d0 con 0x55947a70ea90 2024-05-09T09:23:09.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.917+0000 7f0acea7e640 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a710f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c6a0 tx=0x7f0ab8005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6fd4c67035393aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.918+0000 7f0abcbfe640 1 -- 172.21.15.64:0/3531661560 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80b4070 con 0x55947a70ea90 2024-05-09T09:23:09.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.918+0000 7f0abcbfe640 1 -- 172.21.15.64:0/3531661560 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab809b740 con 0x55947a70ea90 2024-05-09T09:23:09.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.918+0000 7f0ace27d640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2d10 0x7f0ab40a51d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40096c0 tx=0x7f0ac40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 -- 172.21.15.64:0/3531661560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 msgr2=0x55947a710f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a710f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c6a0 tx=0x7f0ab8005730 comp rx=0 tx=0).stop 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 -- 172.21.15.64:0/3531661560 shutdown_connections 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a710f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a70e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/3531661560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a709350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.919+0000 7f0ad31f3740 1 -- 172.21.15.64:0/3531661560 >> 172.21.15.64:0/3531661560 conn(0x55947a7045a0 msgr2=0x55947a7049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.920+0000 7f0ad31f3740 1 -- 172.21.15.64:0/3531661560 shutdown_connections 2024-05-09T09:23:09.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.921+0000 7f0ad31f3740 1 -- 172.21.15.64:0/3531661560 wait complete. 2024-05-09T09:23:09.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.922+0000 7f0ad31f3740 1 -- start start 2024-05-09T09:23:09.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.922+0000 7f0ad31f3740 1 Processor -- start 2024-05-09T09:23:09.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a713400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a713940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a71a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55947a711a90 con 0x55947a708f30 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55947a628b90 con 0x55947a70ea90 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55947a711c10 con 0x55947a709920 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ad31f3740 10 client.0 _ll_register_callbacks cb 0x55947a664da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ace27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a713400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ace27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a713400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59458/0 (socket says 172.21.15.64:59458) 2024-05-09T09:23:09.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.923+0000 7f0ace27d640 1 -- 172.21.15.64:0/1419052270 learned_addr learned my addr 172.21.15.64:0/1419052270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ace27d640 1 -- 172.21.15.64:0/1419052270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 msgr2=0x55947a713940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ace27d640 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a713940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ace27d640 1 -- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 msgr2=0x55947a71a980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ace27d640 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a71a980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ace27d640 1 -- 172.21.15.64:0/1419052270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947a3813f0 con 0x55947a708f30 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ace27d640 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a713400 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40056a0 tx=0x7f0ac4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d603068169d85c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.924+0000 7f0ab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.925+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/1419052270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4035050 con 0x55947a708f30 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.925+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/1419052270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ac4032790 con 0x55947a708f30 2024-05-09T09:23:09.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.925+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab809b740 con 0x55947a708f30 2024-05-09T09:23:09.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.930+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/1419052270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0ac4004e50 con 0x55947a708f30 2024-05-09T09:23:09.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.930+0000 7f0ad31f3740 10 client.18873 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:09.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.930+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55947a3410f0 con 0x55947a708f30 2024-05-09T09:23:09.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.932+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/1419052270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac40044c0 con 0x55947a708f30 2024-05-09T09:23:09.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.932+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/1419052270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0ac4003e10 con 0x55947a708f30 2024-05-09T09:23:09.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.933+0000 7f0ad31f3740 10 client.18873 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:09.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.933+0000 7f0ad31f3740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:09.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.933+0000 7f0ad31f3740 20 client.18873 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:09.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.933+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a71f220 0x55947a7216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.933+0000 7f0ad31f3740 4 client.18873 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:09.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.933+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0ab809b740 con 0x55947a71f220 2024-05-09T09:23:09.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.934+0000 7f0acda7c640 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a71f220 0x55947a7216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.935+0000 7f0ab1ffb640 10 client.18873 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.935+0000 7f0acda7c640 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a71f220 0x55947a7216e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4645c398053d5a server_cookie=636d60ca3627069f in_seq=0 out_seq=0 2024-05-09T09:23:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.936+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/1419052270 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0ab809b740 con 0x55947a71f220 2024-05-09T09:23:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.936+0000 7f0ab1ffb640 10 client.18873 handle_command_reply: tid=0 2024-05-09T09:23:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.936+0000 7f0ad31f3740 1 client.18873 shutdown 2024-05-09T09:23:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 msgr2=0x55947a713400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a713400 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40056a0 tx=0x7f0ac4009260 comp rx=0 tx=0).stop 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 shutdown_connections 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a71f220 0x55947a7216e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ab1ffb640 0 client.18873 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a70ea90 0x55947a71a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a713940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/1419052270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a708f30 0x55947a713400 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 >> 172.21.15.64:0/1419052270 conn(0x55947a7045a0 msgr2=0x55947a7037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.937+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 shutdown_connections 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.938+0000 7f0ad31f3740 1 -- 172.21.15.64:0/1419052270 wait complete. 2024-05-09T09:23:09.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.938+0000 7f0ad31f3740 20 client.18873 trim_cache size 0 max 16384 2024-05-09T09:23:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.986+0000 7f0ad31f3740 1 Processor -- start 2024-05-09T09:23:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 -- start start 2024-05-09T09:23:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a708f30 0x55947a7074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a71a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a7079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b34d0 con 0x55947a70ea90 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b0d40 con 0x55947a708f30 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.987+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81aeb90 con 0x55947a709920 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a7079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a7079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59462/0 (socket says 172.21.15.64:59462) 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 -- 172.21.15.64:0/4095072387 learned_addr learned my addr 172.21.15.64:0/4095072387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 -- 172.21.15.64:0/4095072387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 msgr2=0x55947a71a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a71a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 -- 172.21.15.64:0/4095072387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a708f30 msgr2=0x55947a7074b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a708f30 0x55947a7074b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.989+0000 7f0acea7e640 1 -- 172.21.15.64:0/4095072387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947a711d90 con 0x55947a70ea90 2024-05-09T09:23:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.990+0000 7f0acea7e640 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a7079f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80ccd50 tx=0x7f0ab8097b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8a06ee1f75302b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.990+0000 7f0ab3fff640 1 -- 172.21.15.64:0/4095072387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80b9070 con 0x55947a70ea90 2024-05-09T09:23:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.990+0000 7f0ab3fff640 1 -- 172.21.15.64:0/4095072387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab80a43b0 con 0x55947a70ea90 2024-05-09T09:23:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.991+0000 7f0ad31f3740 1 -- 172.21.15.64:0/4095072387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 msgr2=0x55947a7079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.991+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a7079f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80ccd50 tx=0x7f0ab8097b10 comp rx=0 tx=0).stop 2024-05-09T09:23:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 -- 172.21.15.64:0/4095072387 shutdown_connections 2024-05-09T09:23:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a7079f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a709920 0x55947a71a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/4095072387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a708f30 0x55947a7074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 -- 172.21.15.64:0/4095072387 >> 172.21.15.64:0/4095072387 conn(0x55947a7045a0 msgr2=0x55947a70ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 -- 172.21.15.64:0/4095072387 shutdown_connections 2024-05-09T09:23:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 -- 172.21.15.64:0/4095072387 wait complete. 2024-05-09T09:23:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 -- start start 2024-05-09T09:23:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.992+0000 7f0ad31f3740 1 Processor -- start 2024-05-09T09:23:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a708f30 0x55947a718900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a709920 0x55947a718e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a719e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81af200 con 0x55947a70ea90 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81b0670 con 0x55947a709920 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81af8d0 con 0x55947a708f30 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0ad31f3740 10 client.0 _ll_register_callbacks cb 0x7f0ac403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0acda7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a709920 0x55947a718e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0acda7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a709920 0x55947a718e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53676/0 (socket says 172.21.15.64:53676) 2024-05-09T09:23:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0acda7c640 1 -- 172.21.15.64:0/215976113 learned_addr learned my addr 172.21.15.64:0/215976113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.993+0000 7f0acea7e640 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a719e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0acea7e640 1 -- 172.21.15.64:0/215976113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a708f30 msgr2=0x55947a718900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0acea7e640 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a708f30 0x55947a718900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0acea7e640 1 -- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a709920 msgr2=0x55947a718e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0acea7e640 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a709920 0x55947a718e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0acea7e640 1 -- 172.21.15.64:0/215976113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947a71a3f0 con 0x55947a70ea90 2024-05-09T09:23:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0acea7e640 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a719e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0ab809fcb0 tx=0x7f0ab8005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e3aa8c32179a7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.994+0000 7f0ab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.995+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/215976113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80b9070 con 0x55947a70ea90 2024-05-09T09:23:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.995+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab80a43b0 con 0x55947a70ea90 2024-05-09T09:23:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.995+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/215976113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0ab8097490 con 0x55947a70ea90 2024-05-09T09:23:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.995+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/215976113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0ab80976d0 con 0x55947a70ea90 2024-05-09T09:23:09.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 10 client.18879 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:09.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55947a786120 con 0x55947a70ea90 2024-05-09T09:23:09.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/215976113 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab80a43b0 con 0x55947a70ea90 2024-05-09T09:23:09.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/215976113 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab80a0c80 con 0x55947a70ea90 2024-05-09T09:23:09.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 10 client.18879 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:09.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 20 client.18879 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a7869a0 0x55947a788dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 4 client.18879 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.996+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55947a67b430 con 0x55947a7869a0 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.997+0000 7f0ace27d640 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a7869a0 0x55947a788dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.997+0000 7f0ab1ffb640 10 client.18879 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.998+0000 7f0ace27d640 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a7869a0 0x55947a788dc0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29dbbe958c498ff server_cookie=b210a890ff15b0c1 in_seq=0 out_seq=0 2024-05-09T09:23:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.998+0000 7f0ab1ffb640 1 -- 172.21.15.64:0/215976113 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55947a67b430 con 0x55947a7869a0 2024-05-09T09:23:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.998+0000 7f0ab1ffb640 10 client.18879 handle_command_reply: tid=0 2024-05-09T09:23:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.998+0000 7f0ad31f3740 1 client.18879 shutdown 2024-05-09T09:23:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.999+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 msgr2=0x55947a719e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:09.999+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a719e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0ab809fcb0 tx=0x7f0ab8005730 comp rx=0 tx=0).stop 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 shutdown_connections 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55947a7869a0 0x55947a788dc0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55947a70ea90 0x55947a719e00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55947a709920 0x55947a718e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ad31f3740 1 --2- 172.21.15.64:0/215976113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55947a708f30 0x55947a718900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 >> 172.21.15.64:0/215976113 conn(0x55947a7045a0 msgr2=0x55947a702f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.000+0000 7f0ab1ffb640 0 client.18879 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.005+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 shutdown_connections 2024-05-09T09:23:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.007+0000 7f0ad31f3740 1 -- 172.21.15.64:0/215976113 wait complete. 2024-05-09T09:23:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.007+0000 7f0ad31f3740 20 client.18879 trim_cache size 0 max 16384 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: "a3cac930-318f-4f97-9d36-e36e2409737d": { 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "a3cac930-318f-4f97-9d36-e36e2409737d", 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:23:10.008 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:23:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2d10 msgr2=0x7f0ab40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2d10 0x7f0ab40a51d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40096c0 tx=0x7f0ac40314a0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 msgr2=0x7f0ac81a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 0x7f0ac81a56c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e970 tx=0x7f0ab800ee30 comp rx=0 tx=0).stop 2024-05-09T09:23:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 shutdown_connections 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f0ab40a2d10 0x7f0ab40a51d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8101400 0x7f0ac81a56c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac8100a60 0x7f0ac819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 --2- 172.21.15.64:0/3161381642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ac8100100 0x7f0ac819e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 >> 172.21.15.64:0/3161381642 conn(0x7f0ac80fbd20 msgr2=0x7f0ac80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.009+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 shutdown_connections 2024-05-09T09:23:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:10.010+0000 7f0ad04e1640 1 -- 172.21.15.64:0/3161381642 wait complete. 2024-05-09T09:23:10.033 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:09 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:23:10.034 INFO:tasks.cephfs.filesystem:scrub status for tag:a3cac930-318f-4f97-9d36-e36e2409737d - {'path': '/', 'tag': 'a3cac930-318f-4f97-9d36-e36e2409737d', 'options': 'recursive,force'} 2024-05-09T09:23:10.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:09 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:23:10.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:09 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.363+0000 7f6ae473b640 1 -- 172.21.15.64:0/2325746804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc103940 msgr2=0x7f6adc105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.363+0000 7f6ae473b640 1 --2- 172.21.15.64:0/2325746804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc103940 0x7f6adc105d30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009900 tx=0x7f6ad002a400 comp rx=0 tx=0).stop 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 -- 172.21.15.64:0/2325746804 shutdown_connections 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 --2- 172.21.15.64:0/2325746804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc106350 0x7f6adc108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 --2- 172.21.15.64:0/2325746804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc103940 0x7f6adc105d30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 --2- 172.21.15.64:0/2325746804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 0x7f6adc103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 -- 172.21.15.64:0/2325746804 >> 172.21.15.64:0/2325746804 conn(0x7f6adc0fb050 msgr2=0x7f6adc0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 -- 172.21.15.64:0/2325746804 shutdown_connections 2024-05-09T09:23:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.364+0000 7f6ae473b640 1 -- 172.21.15.64:0/2325746804 wait complete. 2024-05-09T09:23:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.365+0000 7f6ae473b640 1 Processor -- start 2024-05-09T09:23:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.366+0000 7f6ae473b640 1 -- start start 2024-05-09T09:23:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.366+0000 7f6ae473b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 0x7f6adc1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.366+0000 7f6ae473b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 0x7f6adc196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae473b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc106350 0x7f6adc19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae1cd6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 0x7f6adc196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae1cd6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 0x7f6adc196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53700/0 (socket says 172.21.15.64:53700) 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae1cd6640 1 -- 172.21.15.64:0/4061431753 learned_addr learned my addr 172.21.15.64:0/4061431753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 0x7f6adc1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10c370 con 0x7f6adc101010 2024-05-09T09:23:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10c1f0 con 0x7f6adc103940 2024-05-09T09:23:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10c4f0 con 0x7f6adc106350 2024-05-09T09:23:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.367+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc106350 0x7f6adc19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.368+0000 7f6ae24d7640 1 -- 172.21.15.64:0/4061431753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc106350 msgr2=0x7f6adc19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.368+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc106350 0x7f6adc19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.368+0000 7f6ae24d7640 1 -- 172.21.15.64:0/4061431753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 msgr2=0x7f6adc196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.368+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 0x7f6adc196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.368+0000 7f6ae24d7640 1 -- 172.21.15.64:0/4061431753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc19e3e0 con 0x7f6adc101010 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.368+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 0x7f6adc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.369+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 0x7f6adc1967c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400db10 tx=0x7f6ad400e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.370+0000 7f6adb7fe640 1 -- 172.21.15.64:0/4061431753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400e8c0 con 0x7f6adc101010 2024-05-09T09:23:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.370+0000 7f6adb7fe640 1 -- 172.21.15.64:0/4061431753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad40094c0 con 0x7f6adc101010 2024-05-09T09:23:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.370+0000 7f6adb7fe640 1 -- 172.21.15.64:0/4061431753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400e8c0 con 0x7f6adc101010 2024-05-09T09:23:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.373+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc0ffd20 con 0x7f6adc101010 2024-05-09T09:23:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.373+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc100270 con 0x7f6adc101010 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.378+0000 7f6ae7446740 1 Processor -- start 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 -- start start 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c1e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1ec50 0x5596f0c23750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 0x5596f0c26250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5596f0b71340 con 0x5596f0c1ec50 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5596f0b19290 con 0x5596f0c23dc0 2024-05-09T09:23:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae7446740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5596f0b6d300 con 0x5596f0c1e260 2024-05-09T09:23:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae1cd6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 0x5596f0c26250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae1cd6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 0x5596f0c26250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53706/0 (socket says 172.21.15.64:53706) 2024-05-09T09:23:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.379+0000 7f6ae1cd6640 1 -- 172.21.15.64:0/4017663430 learned_addr learned my addr 172.21.15.64:0/4017663430 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.381+0000 7f6ae1cd6640 1 -- 172.21.15.64:0/4017663430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 msgr2=0x5596f0c1e680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.381+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c1e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.381+0000 7f6ae1cd6640 1 -- 172.21.15.64:0/4017663430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1ec50 msgr2=0x5596f0c23750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.381+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1ec50 0x5596f0c23750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.381+0000 7f6ae1cd6640 1 -- 172.21.15.64:0/4017663430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f0b3d8e0 con 0x5596f0c23dc0 2024-05-09T09:23:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.381+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 0x5596f0c26250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004640 tx=0x7f6ad0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b3a05e08fc9ff29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.382+0000 7f6ad8ff9640 1 -- 172.21.15.64:0/4017663430 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0035040 con 0x5596f0c23dc0 2024-05-09T09:23:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.383+0000 7f6ad8ff9640 1 -- 172.21.15.64:0/4017663430 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad0004bf0 con 0x5596f0c23dc0 2024-05-09T09:23:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.383+0000 7f6adb7fe640 1 -- 172.21.15.64:0/4061431753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ad401d020 con 0x7f6adc101010 2024-05-09T09:23:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.384+0000 7f6ae7446740 1 -- 172.21.15.64:0/4017663430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 msgr2=0x5596f0c26250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.384+0000 7f6ae7446740 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 0x5596f0c26250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0004640 tx=0x7f6ad0005040 comp rx=0 tx=0).stop 2024-05-09T09:23:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6adb7fe640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ac40a3100 0x7f6ac40a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6adb7fe640 1 -- 172.21.15.64:0/4061431753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ad400b2e0 con 0x7f6adc101010 2024-05-09T09:23:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6ae7446740 1 -- 172.21.15.64:0/4017663430 shutdown_connections 2024-05-09T09:23:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6ae7446740 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c23dc0 0x5596f0c26250 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6ae7446740 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1ec50 0x5596f0c23750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6ae7446740 1 --2- 172.21.15.64:0/4017663430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c1e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.385+0000 7f6ae7446740 1 -- 172.21.15.64:0/4017663430 >> 172.21.15.64:0/4017663430 conn(0x5596f0c088c0 msgr2=0x5596f0c08cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.386+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ac40a3100 0x7f6ac40a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.386+0000 7f6ae7446740 1 -- 172.21.15.64:0/4017663430 shutdown_connections 2024-05-09T09:23:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.386+0000 7f6ae7446740 1 -- 172.21.15.64:0/4017663430 wait complete. 2024-05-09T09:23:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.386+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ac40a3100 0x7f6ac40a55c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x5596f0c1f4b0 tx=0x7f6acc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.386+0000 7f6ae7446740 1 -- start start 2024-05-09T09:23:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.387+0000 7f6ae7446740 1 Processor -- start 2024-05-09T09:23:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 0x5596f0c284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c23dc0 0x5596f0c2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5596f0c26d60 con 0x5596f0c1e260 2024-05-09T09:23:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5596f0b2c790 con 0x5596f0c1ec50 2024-05-09T09:23:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5596f0c26ee0 con 0x5596f0c23dc0 2024-05-09T09:23:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae7446740 10 client.0 _ll_register_callbacks cb 0x5596f0b67a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae24d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 0x5596f0c284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae24d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 0x5596f0c284b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59516/0 (socket says 172.21.15.64:59516) 2024-05-09T09:23:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.388+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2495961171 learned_addr learned my addr 172.21.15.64:0/2495961171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.389+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c23dc0 0x5596f0c2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.389+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2495961171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c23dc0 msgr2=0x5596f0c2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.389+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c23dc0 0x5596f0c2fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.389+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 msgr2=0x5596f0c289f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.389+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c289f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.390+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2495961171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f0885210 con 0x5596f0c1e260 2024-05-09T09:23:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.390+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c23dc0 0x5596f0c2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.391+0000 7f6ac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.391+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 0x5596f0c284b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40990e0 tx=0x7f6ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7b259229d55157 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.394+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/2495961171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad409c070 con 0x5596f0c1e260 2024-05-09T09:23:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.394+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/2495961171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ad4008040 con 0x5596f0c1e260 2024-05-09T09:23:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.395+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad0004bf0 con 0x5596f0c1e260 2024-05-09T09:23:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.396+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/2495961171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6ad409c070 con 0x5596f0c1e260 2024-05-09T09:23:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.396+0000 7f6ae7446740 10 client.18885 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.396+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5596f0951620 con 0x5596f0c1e260 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.396+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/2495961171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad0004bf0 con 0x5596f0c1e260 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/2495961171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6ad409c970 con 0x5596f0c1e260 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae7446740 10 client.18885 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae7446740 10 client.18885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae7446740 20 client.18885 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0c3cc90 0x5596f0c3f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae7446740 4 client.18885 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f0b7ea70 con 0x5596f0c3cc90 2024-05-09T09:23:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.397+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0c3cc90 0x5596f0c3f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.398+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0c3cc90 0x5596f0c3f110 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6a713fa4b93cbf server_cookie=c5b21e2b4af014c in_seq=0 out_seq=0 2024-05-09T09:23:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.401+0000 7f6ac1ffb640 10 client.18885 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.402+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/2495961171 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596f0b7ea70 con 0x5596f0c3cc90 2024-05-09T09:23:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.402+0000 7f6ac1ffb640 10 client.18885 handle_command_reply: tid=0 2024-05-09T09:23:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.402+0000 7f6ae7446740 1 client.18885 shutdown 2024-05-09T09:23:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.402+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 msgr2=0x5596f0c284b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.402+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 0x5596f0c284b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40990e0 tx=0x7f6ad4004a40 comp rx=0 tx=0).stop 2024-05-09T09:23:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 shutdown_connections 2024-05-09T09:23:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0c3cc90 0x5596f0c3f110 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c23dc0 0x5596f0c2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2495961171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c1e260 0x5596f0c284b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 >> 172.21.15.64:0/2495961171 conn(0x5596f0c088c0 msgr2=0x5596f0c07a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ac1ffb640 0 client.18885 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 shutdown_connections 2024-05-09T09:23:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 1 -- 172.21.15.64:0/2495961171 wait complete. 2024-05-09T09:23:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.403+0000 7f6ae7446740 20 client.18885 trim_cache size 0 max 16384 2024-05-09T09:23:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.435+0000 7f6ae7446740 1 Processor -- start 2024-05-09T09:23:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 -- start start 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c28ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c2f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c0b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1a23d0 con 0x5596f0c23dc0 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc19fc40 con 0x5596f0c1ec50 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae7446740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc072ea0 con 0x5596f0c1e260 2024-05-09T09:23:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae2cd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c0b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.436+0000 7f6ae2cd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c0b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59532/0 (socket says 172.21.15.64:59532) 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.437+0000 7f6ae2cd8640 1 -- 172.21.15.64:0/2767111674 learned_addr learned my addr 172.21.15.64:0/2767111674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.437+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c28ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.437+0000 7f6ae1cd6640 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c2f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.438+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2767111674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 msgr2=0x5596f0c2f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.438+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c2f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.438+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2767111674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 msgr2=0x5596f0c0b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.438+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c0b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.438+0000 7f6ae24d7640 1 -- 172.21.15.64:0/2767111674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f0c06f00 con 0x5596f0c1e260 2024-05-09T09:23:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.438+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c28ae0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40a3330 tx=0x7f6ad4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b753b316d322ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.439+0000 7f6ac3fff640 1 -- 172.21.15.64:0/2767111674 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad409c070 con 0x5596f0c1e260 2024-05-09T09:23:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.439+0000 7f6ac3fff640 1 -- 172.21.15.64:0/2767111674 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad40a5380 con 0x5596f0c1e260 2024-05-09T09:23:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.439+0000 7f6ac3fff640 1 -- 172.21.15.64:0/2767111674 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40870b0 con 0x5596f0c1e260 2024-05-09T09:23:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.440+0000 7f6ae7446740 1 -- 172.21.15.64:0/2767111674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 msgr2=0x5596f0c28ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.441+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c28ae0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40a3330 tx=0x7f6ad4000f40 comp rx=0 tx=0).stop 2024-05-09T09:23:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.441+0000 7f6ae7446740 1 -- 172.21.15.64:0/2767111674 shutdown_connections 2024-05-09T09:23:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.441+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c0b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.441+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1ec50 0x5596f0c2f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.441+0000 7f6ae7446740 1 --2- 172.21.15.64:0/2767111674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1e260 0x5596f0c28ae0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.441+0000 7f6ae7446740 1 -- 172.21.15.64:0/2767111674 >> 172.21.15.64:0/2767111674 conn(0x5596f0c088c0 msgr2=0x5596f0c23470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.444+0000 7f6ae7446740 1 -- 172.21.15.64:0/2767111674 shutdown_connections 2024-05-09T09:23:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.444+0000 7f6ae7446740 1 -- 172.21.15.64:0/2767111674 wait complete. 2024-05-09T09:23:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.445+0000 7f6ae7446740 1 -- start start 2024-05-09T09:23:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.445+0000 7f6ae7446740 1 Processor -- start 2024-05-09T09:23:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 0x5596f0c2f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1ec50 0x5596f0c2c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae7446740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c2cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae24d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 0x5596f0c2f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae24d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 0x5596f0c2f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:53742/0 (socket says 172.21.15.64:53742) 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae1cd6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1ec50 0x5596f0c2c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.446+0000 7f6ae24d7640 1 -- 172.21.15.64:0/1440422957 learned_addr learned my addr 172.21.15.64:0/1440422957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc073550 con 0x5596f0c23dc0 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc19f570 con 0x5596f0c1e260 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc19e7d0 con 0x5596f0c1ec50 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae7446740 10 client.0 _ll_register_callbacks cb 0x7f6ad4098a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c2cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae2cd8640 1 -- 172.21.15.64:0/1440422957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1ec50 msgr2=0x5596f0c2c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1ec50 0x5596f0c2c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae2cd8640 1 -- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 msgr2=0x5596f0c2f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 0x5596f0c2f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.447+0000 7f6ae2cd8640 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f0ca1330 con 0x5596f0c23dc0 2024-05-09T09:23:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.448+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 0x5596f0c2f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.448+0000 7f6ae2cd8640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c2cad0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6acc004620 tx=0x7f6acc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db77d3c7c218fb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.448+0000 7f6ac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.448+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/1440422957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc005020 con 0x5596f0c23dc0 2024-05-09T09:23:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.448+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/1440422957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6acc009040 con 0x5596f0c23dc0 2024-05-09T09:23:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.449+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad40a5380 con 0x5596f0c23dc0 2024-05-09T09:23:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.449+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/1440422957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6acc0051c0 con 0x5596f0c23dc0 2024-05-09T09:23:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.449+0000 7f6ae7446740 10 client.18891 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.449+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5596f0ca2650 con 0x5596f0c23dc0 2024-05-09T09:23:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/1440422957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad40a5380 con 0x5596f0c23dc0 2024-05-09T09:23:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/1440422957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6acc01c6e0 con 0x5596f0c23dc0 2024-05-09T09:23:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae7446740 10 client.18891 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae7446740 10 client.18891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae7446740 20 client.18891 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae7446740 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0ca3090 0x5596f0ca54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae7446740 4 client.18891 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596f0b78dc0 con 0x5596f0ca3090 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.450+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0ca3090 0x5596f0ca54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.451+0000 7f6ac1ffb640 10 client.18891 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.451+0000 7f6ae24d7640 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0ca3090 0x5596f0ca54b0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7176833df16c668d server_cookie=569f51cc90eaaa10 in_seq=0 out_seq=0 2024-05-09T09:23:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.471+0000 7f6ac1ffb640 1 -- 172.21.15.64:0/1440422957 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596f0b78dc0 con 0x5596f0ca3090 2024-05-09T09:23:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.471+0000 7f6ac1ffb640 10 client.18891 handle_command_reply: tid=0 2024-05-09T09:23:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.471+0000 7f6ae7446740 1 client.18891 shutdown 2024-05-09T09:23:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.471+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 msgr2=0x5596f0c2cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.471+0000 7f6ae7446740 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c2cad0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6acc004620 tx=0x7f6acc004730 comp rx=0 tx=0).stop 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 shutdown_connections 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ae7446740 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5596f0ca3090 0x5596f0ca54b0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ac1ffb640 0 client.18891 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ae7446740 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596f0c23dc0 0x5596f0c2cad0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ae7446740 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596f0c1ec50 0x5596f0c2c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ae7446740 1 --2- 172.21.15.64:0/1440422957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5596f0c1e260 0x5596f0c2f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.472+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 >> 172.21.15.64:0/1440422957 conn(0x5596f0c088c0 msgr2=0x5596f0c0b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.473+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 shutdown_connections 2024-05-09T09:23:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.473+0000 7f6ae7446740 1 -- 172.21.15.64:0/1440422957 wait complete. 2024-05-09T09:23:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.473+0000 7f6ae7446740 20 client.18891 trim_cache size 0 max 16384 2024-05-09T09:23:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:23:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:23:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:23:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:23:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ac40a3100 msgr2=0x7f6ac40a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ac40a3100 0x7f6ac40a55c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x5596f0c1f4b0 tx=0x7f6acc005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 msgr2=0x7f6adc1967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 0x7f6adc1967c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400db10 tx=0x7f6ad400e0c0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 shutdown_connections 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6ac40a3100 0x7f6ac40a55c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6adc106350 0x7f6adc19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6adc103940 0x7f6adc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 --2- 172.21.15.64:0/4061431753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6adc101010 0x7f6adc1967c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.474+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 >> 172.21.15.64:0/4061431753 conn(0x7f6adc0fb050 msgr2=0x7f6adc101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.475+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 shutdown_connections 2024-05-09T09:23:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:10.475+0000 7f6ae473b640 1 -- 172.21.15.64:0/4061431753 wait complete. 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 879 op/s 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:10 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 879 op/s 2024-05-09T09:23:11.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:23:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:23:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:23:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:10 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 879 op/s 2024-05-09T09:23:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:23:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:23:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:23:11.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:10 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 770.0 s, 685.7 tps, lat 144.740 ms stddev 86.794 2024-05-09T09:23:13.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:12 smithi064 ceph-mon[30149]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 698 op/s 2024-05-09T09:23:13.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:12 smithi179 ceph-mon[34086]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 698 op/s 2024-05-09T09:23:13.326 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:12 smithi139 ceph-mon[33599]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 698 op/s 2024-05-09T09:23:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:14 smithi064 ceph-mon[30149]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 842 op/s 2024-05-09T09:23:15.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:14 smithi179 ceph-mon[34086]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 842 op/s 2024-05-09T09:23:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:14 smithi139 ceph-mon[33599]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 842 op/s 2024-05-09T09:23:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 775.0 s, 749.1 tps, lat 135.132 ms stddev 94.804 2024-05-09T09:23:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:16 smithi064 ceph-mon[30149]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.4 MiB/s wr, 824 op/s 2024-05-09T09:23:17.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:16 smithi179 ceph-mon[34086]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.4 MiB/s wr, 824 op/s 2024-05-09T09:23:17.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:16 smithi139 ceph-mon[33599]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.4 MiB/s wr, 824 op/s 2024-05-09T09:23:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:18 smithi064 ceph-mon[30149]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.6 MiB/s wr, 725 op/s 2024-05-09T09:23:19.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:18 smithi179 ceph-mon[34086]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.6 MiB/s wr, 725 op/s 2024-05-09T09:23:19.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:18 smithi139 ceph-mon[33599]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.6 MiB/s wr, 725 op/s 2024-05-09T09:23:21.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:20 smithi179 ceph-mon[34086]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.9 MiB/s wr, 974 op/s 2024-05-09T09:23:21.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:20 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:23:21.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:20 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:23:21.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:20 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:23:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:20 smithi139 ceph-mon[33599]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.9 MiB/s wr, 974 op/s 2024-05-09T09:23:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:20 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:23:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:20 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:23:21.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:20 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:23:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 780.0 s, 879.0 tps, lat 113.807 ms stddev 47.237 2024-05-09T09:23:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:20 smithi064 ceph-mon[30149]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.9 MiB/s wr, 974 op/s 2024-05-09T09:23:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:20 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:23:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:20 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:23:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:20 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:23:23.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:22 smithi179 ceph-mon[34086]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.6 MiB/s wr, 722 op/s 2024-05-09T09:23:23.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:22 smithi139 ceph-mon[33599]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.6 MiB/s wr, 722 op/s 2024-05-09T09:23:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:22 smithi064 ceph-mon[30149]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.6 MiB/s wr, 722 op/s 2024-05-09T09:23:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.638+0000 7fe769454640 1 -- 172.21.15.64:0/1668035656 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754031880 con 0x7fe764102340 2024-05-09T09:23:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.638+0000 7fe76c6ba640 1 -- 172.21.15.64:0/1668035656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764102340 msgr2=0x7fe764108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.638+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/1668035656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764102340 0x7fe764108bd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe754009900 tx=0x7fe75402a420 comp rx=0 tx=0).stop 2024-05-09T09:23:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.641+0000 7fe76c6ba640 1 -- 172.21.15.64:0/1668035656 shutdown_connections 2024-05-09T09:23:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.641+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/1668035656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764102340 0x7fe764108bd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.641+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/1668035656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe764101980 0x7fe764101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.641+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/1668035656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764100790 0x7fe764100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.641+0000 7fe76c6ba640 1 -- 172.21.15.64:0/1668035656 >> 172.21.15.64:0/1668035656 conn(0x7fe7640fbd00 msgr2=0x7fe7640fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.642+0000 7fe76c6ba640 1 -- 172.21.15.64:0/1668035656 shutdown_connections 2024-05-09T09:23:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.642+0000 7fe76c6ba640 1 -- 172.21.15.64:0/1668035656 wait complete. 2024-05-09T09:23:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 Processor -- start 2024-05-09T09:23:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 -- start start 2024-05-09T09:23:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe764100790 0x7fe7640734e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764101980 0x7fe76406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 0x7fe764070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe76406b720 con 0x7fe764100790 2024-05-09T09:23:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe76406b5a0 con 0x7fe764102340 2024-05-09T09:23:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.643+0000 7fe76c6ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe76406b8a0 con 0x7fe764101980 2024-05-09T09:23:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 0x7fe764070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 0x7fe764070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39940/0 (socket says 172.21.15.64:39940) 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 -- 172.21.15.64:0/3712205504 learned_addr learned my addr 172.21.15.64:0/3712205504 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe769c55640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764101980 0x7fe76406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76a456640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe764100790 0x7fe7640734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 -- 172.21.15.64:0/3712205504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764101980 msgr2=0x7fe76406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764101980 0x7fe76406ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 -- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe764100790 msgr2=0x7fe7640734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe764100790 0x7fe7640734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 -- 172.21.15.64:0/3712205504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641ae580 con 0x7fe764102340 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.644+0000 7fe76ac57640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 0x7fe764070600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe754003d60 tx=0x7fe754003d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.645+0000 7fe75b7fe640 1 -- 172.21.15.64:0/3712205504 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75403f070 con 0x7fe764102340 2024-05-09T09:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.645+0000 7fe75b7fe640 1 -- 172.21.15.64:0/3712205504 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe754031e40 con 0x7fe764102340 2024-05-09T09:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.645+0000 7fe75b7fe640 1 -- 172.21.15.64:0/3712205504 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754035030 con 0x7fe764102340 2024-05-09T09:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.645+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641ae810 con 0x7fe764102340 2024-05-09T09:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.645+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641aed30 con 0x7fe764102340 2024-05-09T09:23:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.649+0000 7fe75b7fe640 1 -- 172.21.15.64:0/3712205504 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe754042070 con 0x7fe764102340 2024-05-09T09:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.650+0000 7fe76f3c7740 1 Processor -- start 2024-05-09T09:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.650+0000 7fe76f3c7740 1 -- start start 2024-05-09T09:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.651+0000 7fe75b7fe640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7500a2d80 0x7fe7500a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.651+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd04ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.651+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd04f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.651+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd04f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ebd0452340 con 0x55ebd04ee250 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ebd03fa290 con 0x55ebd04eec40 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ebd044e300 con 0x55ebd04f3db0 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe75b7fe640 1 -- 172.21.15.64:0/3712205504 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe7540f0280 con 0x7fe764102340 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe76ac57640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd04f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd04f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe76ac57640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd04f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50814/0 (socket says 172.21.15.64:50814) 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe76ac57640 1 -- 172.21.15.64:0/2181923378 learned_addr learned my addr 172.21.15.64:0/2181923378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7500a2d80 0x7fe7500a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 -- 172.21.15.64:0/2181923378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 msgr2=0x55ebd04f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd04f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 -- 172.21.15.64:0/2181923378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 msgr2=0x55ebd04ee670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd04ee670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.652+0000 7fe769c55640 1 -- 172.21.15.64:0/2181923378 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd041e8e0 con 0x55ebd04eec40 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.653+0000 7fe769c55640 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd04f3740 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00b510 tx=0x7fe75c00b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e6a05126b68d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.653+0000 7fe758ff9640 1 -- 172.21.15.64:0/2181923378 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0087f0 con 0x55ebd04eec40 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.653+0000 7fe758ff9640 1 -- 172.21.15.64:0/2181923378 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c01c020 con 0x55ebd04eec40 2024-05-09T09:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.653+0000 7fe769c55640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7500a2d80 0x7fe7500a5240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe75c01d000 tx=0x7fe75c016850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.654+0000 7fe758ff9640 1 -- 172.21.15.64:0/2181923378 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c01b040 con 0x55ebd04eec40 2024-05-09T09:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2181923378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 msgr2=0x55ebd04f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd04f3740 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00b510 tx=0x7fe75c00b9d0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2181923378 shutdown_connections 2024-05-09T09:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd04f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd04f3740 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2181923378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd04ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.655+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2181923378 >> 172.21.15.64:0/2181923378 conn(0x55ebd04e98c0 msgr2=0x55ebd04e9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.658+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2181923378 shutdown_connections 2024-05-09T09:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.658+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2181923378 wait complete. 2024-05-09T09:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.658+0000 7fe76f3c7740 1 -- start start 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 Processor -- start 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04ee250 0x55ebd0509470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04eec40 0x55ebd05099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd05109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ebd04f6e00 con 0x55ebd04eec40 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ebd040d790 con 0x55ebd04ee250 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ebd04f6f80 con 0x55ebd04f3db0 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76ac57640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd05109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76ac57640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd05109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50830/0 (socket says 172.21.15.64:50830) 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76ac57640 1 -- 172.21.15.64:0/1789575246 learned_addr learned my addr 172.21.15.64:0/1789575246 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76a456640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04ee250 0x55ebd0509470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.659+0000 7fe76f3c7740 10 client.0 _ll_register_callbacks cb 0x55ebd0448a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76ac57640 1 -- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04ee250 msgr2=0x55ebd0509470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76ac57640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04ee250 0x55ebd0509470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76ac57640 1 -- 172.21.15.64:0/1789575246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04eec40 msgr2=0x55ebd05099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76ac57640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04eec40 0x55ebd05099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76ac57640 1 -- 172.21.15.64:0/1789575246 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd0166210 con 0x55ebd04f3db0 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76a456640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04ee250 0x55ebd0509470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe76ac57640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd05109f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe754009850 tx=0x7fe7540bc740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ba4336703fb8df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.660+0000 7fe74dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.661+0000 7fe74dffb640 1 -- 172.21.15.64:0/1789575246 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754049040 con 0x55ebd04f3db0 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.661+0000 7fe74dffb640 1 -- 172.21.15.64:0/1789575246 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe75403b5f0 con 0x55ebd04f3db0 2024-05-09T09:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.661+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c01c020 con 0x55ebd04f3db0 2024-05-09T09:23:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.666+0000 7fe74dffb640 1 -- 172.21.15.64:0/1789575246 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe75403be50 con 0x55ebd04f3db0 2024-05-09T09:23:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.666+0000 7fe76f3c7740 10 client.28534 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.666+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ebd0232620 con 0x55ebd04f3db0 2024-05-09T09:23:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.666+0000 7fe74dffb640 1 -- 172.21.15.64:0/1789575246 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c01c020 con 0x55ebd04f3db0 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.666+0000 7fe74dffb640 1 -- 172.21.15.64:0/1789575246 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe7540a7110 con 0x55ebd04f3db0 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76f3c7740 10 client.28534 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76f3c7740 10 client.28534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76f3c7740 20 client.28534 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd04fb500 0x55ebd04fd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76f3c7740 4 client.28534 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd045fa70 con 0x55ebd04fb500 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.667+0000 7fe76a456640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd04fb500 0x55ebd04fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.668+0000 7fe74dffb640 10 client.28534 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.668+0000 7fe76a456640 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd04fb500 0x55ebd04fd980 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a251be68c016099 server_cookie=e9e94bee7f4ec083 in_seq=0 out_seq=0 2024-05-09T09:23:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.669+0000 7fe74dffb640 1 -- 172.21.15.64:0/1789575246 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebd045fa70 con 0x55ebd04fb500 2024-05-09T09:23:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.669+0000 7fe74dffb640 10 client.28534 handle_command_reply: tid=0 2024-05-09T09:23:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.669+0000 7fe76f3c7740 1 client.28534 shutdown 2024-05-09T09:23:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 msgr2=0x55ebd05109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd05109f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe754009850 tx=0x7fe7540bc740 comp rx=0 tx=0).stop 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 shutdown_connections 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd04fb500 0x55ebd04fd980 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04f3db0 0x55ebd05109f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04eec40 0x55ebd05099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/1789575246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04ee250 0x55ebd0509470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 >> 172.21.15.64:0/1789575246 conn(0x55ebd04e98c0 msgr2=0x55ebd04e8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe74dffb640 0 client.28534 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 shutdown_connections 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 1 -- 172.21.15.64:0/1789575246 wait complete. 2024-05-09T09:23:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.670+0000 7fe76f3c7740 20 client.28534 trim_cache size 0 max 16384 2024-05-09T09:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.700+0000 7fe76f3c7740 1 Processor -- start 2024-05-09T09:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.701+0000 7fe76f3c7740 1 -- start start 2024-05-09T09:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.702+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04ee250 0x55ebd04ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.702+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd05109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.702+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04f3db0 0x55ebd04ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.702+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b33a0 con 0x55ebd04f3db0 2024-05-09T09:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.702+0000 7fe769c55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd05109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.703+0000 7fe769c55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd05109b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39976/0 (socket says 172.21.15.64:39976) 2024-05-09T09:23:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.703+0000 7fe769c55640 1 -- 172.21.15.64:0/2221415728 learned_addr learned my addr 172.21.15.64:0/2221415728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.703+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b0c10 con 0x55ebd04eec40 2024-05-09T09:23:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.703+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641a9ec0 con 0x55ebd04ee250 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe769c55640 1 -- 172.21.15.64:0/2221415728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04ee250 msgr2=0x55ebd04ec870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe769c55640 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04ee250 0x55ebd04ec870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe769c55640 1 -- 172.21.15.64:0/2221415728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04f3db0 msgr2=0x55ebd04ecdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe769c55640 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04f3db0 0x55ebd04ecdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe769c55640 1 -- 172.21.15.64:0/2221415728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd04e7f00 con 0x55ebd04eec40 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe769c55640 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd05109b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00bed0 tx=0x7fe75c022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52968f776af0fa5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.704+0000 7fe74ffff640 1 -- 172.21.15.64:0/2221415728 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c01b040 con 0x55ebd04eec40 2024-05-09T09:23:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.705+0000 7fe74ffff640 1 -- 172.21.15.64:0/2221415728 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c02f070 con 0x55ebd04eec40 2024-05-09T09:23:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.705+0000 7fe74ffff640 1 -- 172.21.15.64:0/2221415728 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c021cd0 con 0x55ebd04eec40 2024-05-09T09:23:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 msgr2=0x55ebd05109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd05109b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00bed0 tx=0x7fe75c022000 comp rx=0 tx=0).stop 2024-05-09T09:23:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 shutdown_connections 2024-05-09T09:23:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04f3db0 0x55ebd04ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04eec40 0x55ebd05109b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2221415728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04ee250 0x55ebd04ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 >> 172.21.15.64:0/2221415728 conn(0x55ebd04e98c0 msgr2=0x55ebd0509470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 shutdown_connections 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.706+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2221415728 wait complete. 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 -- start start 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 Processor -- start 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd0510910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04eec40 0x55ebd050ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04f3db0 0x55ebd050f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641af0d0 con 0x55ebd04ee250 2024-05-09T09:23:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b0540 con 0x55ebd04f3db0 2024-05-09T09:23:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641af7a0 con 0x55ebd04eec40 2024-05-09T09:23:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.707+0000 7fe76f3c7740 10 client.0 _ll_register_callbacks cb 0x7fe7540ed8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.708+0000 7fe76a456640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd0510910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.708+0000 7fe76a456640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd0510910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34384/0 (socket says 172.21.15.64:34384) 2024-05-09T09:23:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.708+0000 7fe76a456640 1 -- 172.21.15.64:0/2852934340 learned_addr learned my addr 172.21.15.64:0/2852934340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.708+0000 7fe769c55640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04eec40 0x55ebd050ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.708+0000 7fe76ac57640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04f3db0 0x55ebd050f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe769c55640 1 -- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04f3db0 msgr2=0x55ebd050f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe769c55640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04f3db0 0x55ebd050f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe769c55640 1 -- 172.21.15.64:0/2852934340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 msgr2=0x55ebd0510910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe769c55640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd0510910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe769c55640 1 -- 172.21.15.64:0/2852934340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd050fc40 con 0x55ebd04eec40 2024-05-09T09:23:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe769c55640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04eec40 0x55ebd050ef20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe75c024040 tx=0x7fe75c009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2e103d81a2970c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.709+0000 7fe74dffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.710+0000 7fe74dffb640 1 -- 172.21.15.64:0/2852934340 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c008e60 con 0x55ebd04eec40 2024-05-09T09:23:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.710+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c02f070 con 0x55ebd04eec40 2024-05-09T09:23:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.710+0000 7fe74dffb640 1 -- 172.21.15.64:0/2852934340 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe75c009bb0 con 0x55ebd04eec40 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.710+0000 7fe74dffb640 1 -- 172.21.15.64:0/2852934340 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe75c043050 con 0x55ebd04eec40 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.710+0000 7fe76f3c7740 10 client.28537 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.710+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ebd056a9b0 con 0x55ebd04eec40 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe74dffb640 1 -- 172.21.15.64:0/2852934340 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c02f070 con 0x55ebd04eec40 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe74dffb640 1 -- 172.21.15.64:0/2852934340 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe75c035800 con 0x55ebd04eec40 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76f3c7740 10 client.28537 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76f3c7740 10 client.28537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76f3c7740 20 client.28537 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd056b2e0 0x55ebd056d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76f3c7740 4 client.28537 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:23:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebd0459dc0 con 0x55ebd056b2e0 2024-05-09T09:23:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.711+0000 7fe76a456640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd056b2e0 0x55ebd056d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.712+0000 7fe76a456640 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd056b2e0 0x55ebd056d700 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e163416cf1396a2 server_cookie=c1f3f22ee5752460 in_seq=0 out_seq=0 2024-05-09T09:23:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.712+0000 7fe74dffb640 10 client.28537 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.734+0000 7fe74dffb640 1 -- 172.21.15.64:0/2852934340 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebd0459dc0 con 0x55ebd056b2e0 2024-05-09T09:23:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.734+0000 7fe74dffb640 10 client.28537 handle_command_reply: tid=0 2024-05-09T09:23:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.734+0000 7fe76f3c7740 1 client.28537 shutdown 2024-05-09T09:23:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.735+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04eec40 msgr2=0x55ebd050ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.735+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04eec40 0x55ebd050ef20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe75c024040 tx=0x7fe75c009710 comp rx=0 tx=0).stop 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 shutdown_connections 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ebd056b2e0 0x55ebd056d700 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ebd04f3db0 0x55ebd050f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebd04eec40 0x55ebd050ef20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe76f3c7740 1 --2- 172.21.15.64:0/2852934340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebd04ee250 0x55ebd0510910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 >> 172.21.15.64:0/2852934340 conn(0x55ebd04e98c0 msgr2=0x55ebd0508270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.736+0000 7fe74dffb640 0 client.28537 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.738+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 shutdown_connections 2024-05-09T09:23:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.738+0000 7fe76f3c7740 1 -- 172.21.15.64:0/2852934340 wait complete. 2024-05-09T09:23:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.738+0000 7fe76f3c7740 20 client.28537 trim_cache size 0 max 16384 2024-05-09T09:23:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:23:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:23:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:23:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:23:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.742+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7500a2d80 msgr2=0x7fe7500a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.742+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7500a2d80 0x7fe7500a5240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe75c01d000 tx=0x7fe75c016850 comp rx=0 tx=0).stop 2024-05-09T09:23:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.742+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 msgr2=0x7fe764070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.742+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 0x7fe764070600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe754003d60 tx=0x7fe754003d90 comp rx=0 tx=0).stop 2024-05-09T09:23:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 shutdown_connections 2024-05-09T09:23:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe7500a2d80 0x7fe7500a5240 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe764102340 0x7fe764070600 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe764101980 0x7fe76406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 --2- 172.21.15.64:0/3712205504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe764100790 0x7fe7640734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 >> 172.21.15.64:0/3712205504 conn(0x7fe7640fbd00 msgr2=0x7fe7640fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 shutdown_connections 2024-05-09T09:23:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:23.746+0000 7fe76c6ba640 1 -- 172.21.15.64:0/3712205504 wait complete. 2024-05-09T09:23:25.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:24 smithi179 ceph-mon[34086]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.0 MiB/s wr, 868 op/s 2024-05-09T09:23:25.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:24 smithi139 ceph-mon[33599]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.0 MiB/s wr, 868 op/s 2024-05-09T09:23:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:24 smithi064 ceph-mon[30149]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.0 MiB/s wr, 868 op/s 2024-05-09T09:23:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 785.0 s, 745.8 tps, lat 133.368 ms stddev 75.182 2024-05-09T09:23:27.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:26 smithi179 ceph-mon[34086]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 801 op/s 2024-05-09T09:23:27.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:26 smithi139 ceph-mon[33599]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 801 op/s 2024-05-09T09:23:27.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:26 smithi064 ceph-mon[30149]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 801 op/s 2024-05-09T09:23:29.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:28 smithi179 ceph-mon[34086]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 MiB/s wr, 704 op/s 2024-05-09T09:23:29.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:28 smithi139 ceph-mon[33599]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 MiB/s wr, 704 op/s 2024-05-09T09:23:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:28 smithi064 ceph-mon[30149]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 MiB/s wr, 704 op/s 2024-05-09T09:23:31.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:30 smithi179 ceph-mon[34086]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 906 op/s 2024-05-09T09:23:31.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:30 smithi139 ceph-mon[33599]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 906 op/s 2024-05-09T09:23:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 790.0 s, 760.2 tps, lat 131.885 ms stddev 115.713 2024-05-09T09:23:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:30 smithi064 ceph-mon[30149]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 906 op/s 2024-05-09T09:23:33.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:32 smithi179 ceph-mon[34086]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 652 op/s 2024-05-09T09:23:33.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:32 smithi139 ceph-mon[33599]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 652 op/s 2024-05-09T09:23:33.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:32 smithi064 ceph-mon[30149]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 652 op/s 2024-05-09T09:23:35.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:34 smithi179 ceph-mon[34086]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 827 op/s 2024-05-09T09:23:35.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:34 smithi139 ceph-mon[33599]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 827 op/s 2024-05-09T09:23:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:34 smithi064 ceph-mon[30149]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 827 op/s 2024-05-09T09:23:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 795.0 s, 878.2 tps, lat 113.841 ms stddev 53.385 2024-05-09T09:23:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb1ffb640 1 -- 172.21.15.64:0/2151375914 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40308e0 con 0x7fdcac0a6710 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2151375914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 msgr2=0x7fdcac0aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2151375914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac0aafd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009860 tx=0x7fdca402a380 comp rx=0 tx=0).stop 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2151375914 shutdown_connections 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2151375914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac0aafd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2151375914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcac0a5d70 0x7fdcac0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2151375914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a4c40 0x7fdcac0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.910+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2151375914 >> 172.21.15.64:0/2151375914 conn(0x7fdcac0a00d0 msgr2=0x7fdcac0a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.911+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2151375914 shutdown_connections 2024-05-09T09:23:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.911+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2151375914 wait complete. 2024-05-09T09:23:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.912+0000 7fdcb3fff640 1 Processor -- start 2024-05-09T09:23:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.912+0000 7fdcb3fff640 1 -- start start 2024-05-09T09:23:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcac0a4c40 0x7fdcac13ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a5d70 0x7fdcac13b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb3fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac0ae880 con 0x7fdcac0a4c40 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb3fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac0ae700 con 0x7fdcac0a5d70 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb3fff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac0aea00 con 0x7fdcac0a6710 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a5d70 0x7fdcac13b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a5d70 0x7fdcac13b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57070/0 (socket says 172.21.15.64:57070) 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb27fc640 1 -- 172.21.15.64:0/2420154304 learned_addr learned my addr 172.21.15.64:0/2420154304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.913+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcac0a4c40 0x7fdcac13ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb37fe640 1 -- 172.21.15.64:0/2420154304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a5d70 msgr2=0x7fdcac13b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a5d70 0x7fdcac13b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb37fe640 1 -- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcac0a4c40 msgr2=0x7fdcac13ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcac0a4c40 0x7fdcac13ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb37fe640 1 -- 172.21.15.64:0/2420154304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac1429b0 con 0x7fdcac0a6710 2024-05-09T09:23:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac142310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdca402a8a0 tx=0x7fdca4032e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb8907640 1 -- 172.21.15.64:0/2420154304 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4040020 con 0x7fdcac0a6710 2024-05-09T09:23:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac142c40 con 0x7fdcac0a6710 2024-05-09T09:23:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.914+0000 7fdcb8907640 1 -- 172.21.15.64:0/2420154304 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdca4038040 con 0x7fdcac0a6710 2024-05-09T09:23:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.915+0000 7fdcb8907640 1 -- 172.21.15.64:0/2420154304 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4037070 con 0x7fdcac0a6710 2024-05-09T09:23:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.915+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac143100 con 0x7fdcac0a6710 2024-05-09T09:23:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.918+0000 7fdcbc610740 1 Processor -- start 2024-05-09T09:23:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.919+0000 7fdcb8907640 1 -- 172.21.15.64:0/2420154304 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdca4043020 con 0x7fdcac0a6710 2024-05-09T09:23:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 -- start start 2024-05-09T09:23:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcb8907640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc9c0a2b90 0x7fdc9c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 0x5639caff4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caff98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff9f60 0x5639caffc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639caf58340 con 0x5639caff4df0 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5639caf00290 con 0x5639caff9f60 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcbc610740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5639caf54300 con 0x5639caff4400 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.920+0000 7fdcb8907640 1 -- 172.21.15.64:0/2420154304 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdca40efc10 con 0x7fdcac0a6710 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc9c0a2b90 0x7fdc9c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff9f60 0x5639caffc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 0x5639caff4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caff98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 0x5639caff4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49836/0 (socket says 172.21.15.64:49836) 2024-05-09T09:23:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 -- 172.21.15.64:0/2220212703 learned_addr learned my addr 172.21.15.64:0/2220212703 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 -- 172.21.15.64:0/2220212703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff9f60 msgr2=0x5639caffc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff9f60 0x5639caffc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 -- 172.21.15.64:0/2220212703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 msgr2=0x5639caff98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caff98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.921+0000 7fdcb2ffd640 1 -- 172.21.15.64:0/2220212703 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639caf248e0 con 0x5639caff4400 2024-05-09T09:23:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.922+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc9c0a2b90 0x7fdc9c0a5050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdca800afd0 tx=0x7fdca800a9b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.922+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 0x5639caff4820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4065080 tx=0x7fdcb404fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b976b4975c72c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.922+0000 7fdc99ffb640 1 -- 172.21.15.64:0/2220212703 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb408b640 con 0x5639caff4400 2024-05-09T09:23:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.922+0000 7fdc99ffb640 1 -- 172.21.15.64:0/2220212703 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcb408bc90 con 0x5639caff4400 2024-05-09T09:23:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.922+0000 7fdc99ffb640 1 -- 172.21.15.64:0/2220212703 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb407dbd0 con 0x5639caff4400 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 -- 172.21.15.64:0/2220212703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 msgr2=0x5639caff4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 0x5639caff4820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4065080 tx=0x7fdcb404fcf0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 -- 172.21.15.64:0/2220212703 shutdown_connections 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff9f60 0x5639caffc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caff98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2220212703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4400 0x5639caff4820 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.923+0000 7fdcbc610740 1 -- 172.21.15.64:0/2220212703 >> 172.21.15.64:0/2220212703 conn(0x5639cafefa70 msgr2=0x5639cafefe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.924+0000 7fdcbc610740 1 -- 172.21.15.64:0/2220212703 shutdown_connections 2024-05-09T09:23:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.924+0000 7fdcbc610740 1 -- 172.21.15.64:0/2220212703 wait complete. 2024-05-09T09:23:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.924+0000 7fdcbc610740 1 -- start start 2024-05-09T09:23:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.924+0000 7fdcbc610740 1 Processor -- start 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639caffe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639caffcf10 con 0x5639caff4df0 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5639caf13790 con 0x5639caff4400 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5639caffd090 con 0x5639caff9f60 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46544/0 (socket says 172.21.15.64:46544) 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcb27fc640 1 -- 172.21.15.64:0/2653790269 learned_addr learned my addr 172.21.15.64:0/2653790269 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.925+0000 7fdcbc610740 10 client.0 _ll_register_callbacks cb 0x5639caf4ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb27fc640 1 -- 172.21.15.64:0/2653790269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 msgr2=0x5639cb005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb27fc640 1 -- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 msgr2=0x5639caffe5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639caffe5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb27fc640 1 -- 172.21.15.64:0/2653790269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cac6c210 con 0x5639caff4df0 2024-05-09T09:23:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb005b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdca800ac70 tx=0x7fdca8006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b746367de84480f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.926+0000 7fdc92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.927+0000 7fdc92ffd640 1 -- 172.21.15.64:0/2653790269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8024020 con 0x5639caff4df0 2024-05-09T09:23:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.927+0000 7fdc92ffd640 1 -- 172.21.15.64:0/2653790269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdca8016cc0 con 0x5639caff4df0 2024-05-09T09:23:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.927+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb408bc90 con 0x5639caff4df0 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.927+0000 7fdc92ffd640 1 -- 172.21.15.64:0/2653790269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fdca802f050 con 0x5639caff4df0 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.927+0000 7fdcbc610740 10 client.18906 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.927+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5639cad38620 con 0x5639caff4df0 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdc92ffd640 1 -- 172.21.15.64:0/2653790269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdca8018e30 con 0x5639caff4df0 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdc92ffd640 1 -- 172.21.15.64:0/2653790269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fdca801f070 con 0x5639caff4df0 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdcbc610740 10 client.18906 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdcbc610740 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdcbc610740 20 client.18906 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb00a140 0x5639cb00c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdcbc610740 4 client.18906 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.928+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdcb408bc90 con 0x5639cb00a140 2024-05-09T09:23:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.929+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb00a140 0x5639cb00c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.930+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb00a140 0x5639cb00c600 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c359ad035d208f server_cookie=93e0adf737fa5346 in_seq=0 out_seq=0 2024-05-09T09:23:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.930+0000 7fdc92ffd640 10 client.18906 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.931+0000 7fdc92ffd640 1 -- 172.21.15.64:0/2653790269 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fdcb408bc90 con 0x5639cb00a140 2024-05-09T09:23:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.931+0000 7fdc92ffd640 10 client.18906 handle_command_reply: tid=0 2024-05-09T09:23:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.931+0000 7fdcbc610740 1 client.18906 shutdown 2024-05-09T09:23:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.936+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 msgr2=0x5639caffeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.936+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdca800ac70 tx=0x7fdca8006e10 comp rx=0 tx=0).stop 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.939+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 shutdown_connections 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.939+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb00a140 0x5639cb00c600 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.939+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb005b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.939+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.939+0000 7fdcbc610740 1 --2- 172.21.15.64:0/2653790269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639caffe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.939+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 >> 172.21.15.64:0/2653790269 conn(0x5639cafefa70 msgr2=0x5639cafeebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.940+0000 7fdc92ffd640 0 client.18906 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.941+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 shutdown_connections 2024-05-09T09:23:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.942+0000 7fdcbc610740 1 -- 172.21.15.64:0/2653790269 wait complete. 2024-05-09T09:23:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.943+0000 7fdcbc610740 20 client.18906 trim_cache size 0 max 16384 2024-05-09T09:23:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.976+0000 7fdcbc610740 1 Processor -- start 2024-05-09T09:23:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.976+0000 7fdcbc610740 1 -- start start 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639caffe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff9f60 0x5639cb005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49854/0 (socket says 172.21.15.64:49854) 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcb27fc640 1 -- 172.21.15.64:0/1577494802 learned_addr learned my addr 172.21.15.64:0/1577494802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac14ffb0 con 0x5639caff9f60 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac14d820 con 0x5639caff4400 2024-05-09T09:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.977+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac14b670 con 0x5639caff4df0 2024-05-09T09:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.978+0000 7fdcb27fc640 1 -- 172.21.15.64:0/1577494802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 msgr2=0x5639caffe5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.978+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639caffe5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.978+0000 7fdcb27fc640 1 -- 172.21.15.64:0/1577494802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff9f60 msgr2=0x5639cb005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.978+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff9f60 0x5639cb005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.978+0000 7fdcb27fc640 1 -- 172.21.15.64:0/1577494802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cafff450 con 0x5639caff4df0 2024-05-09T09:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.978+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdca8006a50 tx=0x7fdca8014370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c976ef7a19115cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.979+0000 7fdc997fa640 1 -- 172.21.15.64:0/1577494802 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8024070 con 0x5639caff4df0 2024-05-09T09:23:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.979+0000 7fdc997fa640 1 -- 172.21.15.64:0/1577494802 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdca8005e40 con 0x5639caff4df0 2024-05-09T09:23:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.979+0000 7fdc997fa640 1 -- 172.21.15.64:0/1577494802 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca801c070 con 0x5639caff4df0 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 msgr2=0x5639caffeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdca8006a50 tx=0x7fdca8014370 comp rx=0 tx=0).stop 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 shutdown_connections 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff9f60 0x5639cb005af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff4df0 0x5639caffeaf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 --2- 172.21.15.64:0/1577494802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639caffe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.981+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 >> 172.21.15.64:0/1577494802 conn(0x5639cafefa70 msgr2=0x5639cb00c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.986+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 shutdown_connections 2024-05-09T09:23:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.987+0000 7fdcbc610740 1 -- 172.21.15.64:0/1577494802 wait complete. 2024-05-09T09:23:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.987+0000 7fdcbc610740 1 -- start start 2024-05-09T09:23:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.987+0000 7fdcbc610740 1 Processor -- start 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639cb002a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639cb002fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac14bce0 con 0x5639caff4df0 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac14d150 con 0x5639caff4400 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac14c3b0 con 0x5639caff9f60 2024-05-09T09:23:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.989+0000 7fdcbc610740 10 client.0 _ll_register_callbacks cb 0x7fdca800abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.990+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639cb002fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.990+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639cb002fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46564/0 (socket says 172.21.15.64:46564) 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.990+0000 7fdcb27fc640 1 -- 172.21.15.64:0/3323715495 learned_addr learned my addr 172.21.15.64:0/3323715495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.991+0000 7fdcb27fc640 1 -- 172.21.15.64:0/3323715495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 msgr2=0x5639cb06e770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.991+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb06e770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.991+0000 7fdcb27fc640 1 -- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 msgr2=0x5639cb002a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.991+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639cb002a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.991+0000 7fdcb27fc640 1 -- 172.21.15.64:0/3323715495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cb06ecb0 con 0x5639caff4df0 2024-05-09T09:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.992+0000 7fdcb27fc640 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639cb002fd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdca8018010 tx=0x7fdca8018ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77b19c087c5bc8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.992+0000 7fdc92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.992+0000 7fdc92ffd640 1 -- 172.21.15.64:0/3323715495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca801c070 con 0x5639caff4df0 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.992+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639caf65a70 con 0x5639caff4df0 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.992+0000 7fdc92ffd640 1 -- 172.21.15.64:0/3323715495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdca802b720 con 0x5639caff4df0 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.993+0000 7fdc92ffd640 1 -- 172.21.15.64:0/3323715495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fdca8043050 con 0x5639caff4df0 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.993+0000 7fdcbc610740 10 client.18909 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.993+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5639cb06ff40 con 0x5639caff4df0 2024-05-09T09:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdc92ffd640 1 -- 172.21.15.64:0/3323715495 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5639caf65a70 con 0x5639caff4df0 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdc92ffd640 1 -- 172.21.15.64:0/3323715495 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fdca801f040 con 0x5639caff4df0 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdcbc610740 10 client.18909 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdcbc610740 10 client.18909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdcbc610740 20 client.18909 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdcbc610740 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb070910 0x5639cb072d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdcbc610740 4 client.18909 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.994+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639caf5fdc0 con 0x5639cb070910 2024-05-09T09:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:36.998+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb070910 0x5639cb072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.002+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb070910 0x5639cb072d30 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a5f71c516f65b5 server_cookie=7c81d325ec9542f1 in_seq=0 out_seq=0 2024-05-09T09:23:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.002+0000 7fdc92ffd640 10 client.18909 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.049+0000 7fdc92ffd640 1 -- 172.21.15.64:0/3323715495 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639caf5fdc0 con 0x5639cb070910 2024-05-09T09:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.049+0000 7fdc92ffd640 10 client.18909 handle_command_reply: tid=0 2024-05-09T09:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.049+0000 7fdcbc610740 1 client.18909 shutdown 2024-05-09T09:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.049+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 msgr2=0x5639cb002fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.049+0000 7fdcbc610740 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639cb002fd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdca8018010 tx=0x7fdca8018ee0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 shutdown_connections 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5639cb070910 0x5639cb072d30 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639caff9f60 0x5639cb06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639caff4df0 0x5639cb002fd0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 --2- 172.21.15.64:0/3323715495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639caff4400 0x5639cb002a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 >> 172.21.15.64:0/3323715495 conn(0x5639cafefa70 msgr2=0x5639caff5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdc92ffd640 0 client.18909 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 shutdown_connections 2024-05-09T09:23:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.050+0000 7fdcbc610740 1 -- 172.21.15.64:0/3323715495 wait complete. 2024-05-09T09:23:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.051+0000 7fdcbc610740 20 client.18909 trim_cache size 0 max 16384 2024-05-09T09:23:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:23:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:23:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:23:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.052+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc9c0a2b90 msgr2=0x7fdc9c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.052+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc9c0a2b90 0x7fdc9c0a5050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdca800afd0 tx=0x7fdca800a9b0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.052+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 msgr2=0x7fdcac142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.052+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac142310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdca402a8a0 tx=0x7fdca4032e30 comp rx=0 tx=0).stop 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 shutdown_connections 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc9c0a2b90 0x7fdc9c0a5050 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcac0a6710 0x7fdcac142310 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcac0a5d70 0x7fdcac13b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/2420154304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcac0a4c40 0x7fdcac13ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 >> 172.21.15.64:0/2420154304 conn(0x7fdcac0a00d0 msgr2=0x7fdcac0a8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 shutdown_connections 2024-05-09T09:23:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:37.053+0000 7fdcb3fff640 1 -- 172.21.15.64:0/2420154304 wait complete. 2024-05-09T09:23:37.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:36 smithi179 ceph-mon[34086]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 779 op/s 2024-05-09T09:23:37.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:36 smithi139 ceph-mon[33599]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 779 op/s 2024-05-09T09:23:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:36 smithi064 ceph-mon[30149]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 779 op/s 2024-05-09T09:23:39.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:38 smithi179 ceph-mon[34086]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-05-09T09:23:39.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:38 smithi139 ceph-mon[33599]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-05-09T09:23:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:38 smithi064 ceph-mon[30149]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-05-09T09:23:40.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:23:40.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 -- 172.21.15.64:0/3548643435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406f2b0 msgr2=0x7f89b406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 --2- 172.21.15.64:0/3548643435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406f2b0 0x7f89b406fb50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01acb0 tx=0x7f89ac03b900 comp rx=0 tx=0).stop 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 -- 172.21.15.64:0/3548643435 shutdown_connections 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 --2- 172.21.15.64:0/3548643435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406f2b0 0x7f89b406fb50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 --2- 172.21.15.64:0/3548643435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406e910 0x7f89b406ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 --2- 172.21.15.64:0/3548643435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406d6d0 0x7f89b406daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 -- 172.21.15.64:0/3548643435 >> 172.21.15.64:0/3548643435 conn(0x7f89b406c2a0 msgr2=0x7f89b406c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 -- 172.21.15.64:0/3548643435 shutdown_connections 2024-05-09T09:23:40.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.221+0000 7f89bb134640 1 -- 172.21.15.64:0/3548643435 wait complete. 2024-05-09T09:23:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.222+0000 7f89bb134640 1 Processor -- start 2024-05-09T09:23:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.222+0000 7f89bb134640 1 -- start start 2024-05-09T09:23:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.222+0000 7f89bb134640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406d6d0 0x7f89b419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.222+0000 7f89bb134640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406e910 0x7f89b419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.222+0000 7f89bb134640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 0x7f89b41a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89bb134640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b410da00 con 0x7f89b406d6d0 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89bb134640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89b410d880 con 0x7f89b406e910 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89bb134640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89b410db80 con 0x7f89b406f2b0 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89b96d1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 0x7f89b41a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89b96d1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 0x7f89b41a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49878/0 (socket says 172.21.15.64:49878) 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89b96d1640 1 -- 172.21.15.64:0/69951805 learned_addr learned my addr 172.21.15.64:0/69951805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89abfff640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406e910 0x7f89b419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406d6d0 0x7f89b419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89abfff640 1 -- 172.21.15.64:0/69951805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 msgr2=0x7f89b41a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89abfff640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 0x7f89b41a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89abfff640 1 -- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406d6d0 msgr2=0x7f89b419e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89abfff640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406d6d0 0x7f89b419e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89abfff640 1 -- 172.21.15.64:0/69951805 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b41a5d40 con 0x7f89b406e910 2024-05-09T09:23:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89b96d1640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 0x7f89b41a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.223+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406d6d0 0x7f89b419e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.224+0000 7f89abfff640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406e910 0x7f89b419e660 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d720 tx=0x7f89a400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.224+0000 7f89a9ffb640 1 -- 172.21.15.64:0/69951805 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4017070 con 0x7f89b406e910 2024-05-09T09:23:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.224+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b41a5fd0 con 0x7f89b406e910 2024-05-09T09:23:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.224+0000 7f89a9ffb640 1 -- 172.21.15.64:0/69951805 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89a400de30 con 0x7f89b406e910 2024-05-09T09:23:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.224+0000 7f89a9ffb640 1 -- 172.21.15.64:0/69951805 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a40047a0 con 0x7f89b406e910 2024-05-09T09:23:40.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.224+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41a6570 con 0x7f89b406e910 2024-05-09T09:23:40.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.228+0000 7f89bde41740 1 Processor -- start 2024-05-09T09:23:40.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.228+0000 7f89bde41740 1 -- start start 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89bde41740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1ff30 0x56321ae20350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89bde41740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 0x56321ae25420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89bde41740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae25a90 0x56321ae27f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89bde41740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56321ad84d00 con 0x56321ae1ff30 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89bde41740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56321ad2c4c0 con 0x56321ae20920 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89bde41740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56321ad80cc0 con 0x56321ae25a90 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89b8ed0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 0x56321ae25420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89b8ed0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 0x56321ae25420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57090/0 (socket says 172.21.15.64:57090) 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.229+0000 7f89b8ed0640 1 -- 172.21.15.64:0/1117482123 learned_addr learned my addr 172.21.15.64:0/1117482123 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.230+0000 7f89b8ed0640 1 -- 172.21.15.64:0/1117482123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae25a90 msgr2=0x56321ae27f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.230+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae25a90 0x56321ae27f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.230+0000 7f89b8ed0640 1 -- 172.21.15.64:0/1117482123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1ff30 msgr2=0x56321ae20350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.230+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1ff30 0x56321ae20350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.230+0000 7f89b8ed0640 1 -- 172.21.15.64:0/1117482123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321ab627d0 con 0x56321ae20920 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.230+0000 7f89a9ffb640 1 -- 172.21.15.64:0/69951805 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89a4009950 con 0x7f89b406e910 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.231+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 0x56321ae25420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009b60 tx=0x7f89b00090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fec6b123550fc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.231+0000 7f899affd640 1 -- 172.21.15.64:0/1117482123 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017020 con 0x56321ae20920 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.231+0000 7f899affd640 1 -- 172.21.15.64:0/1117482123 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b000ebe0 con 0x56321ae20920 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.231+0000 7f89a9ffb640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f899c0a2c60 0x7f899c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.232+0000 7f899affd640 1 -- 172.21.15.64:0/1117482123 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000f040 con 0x56321ae20920 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.232+0000 7f89bde41740 1 -- 172.21.15.64:0/1117482123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 msgr2=0x56321ae25420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.232+0000 7f89bde41740 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 0x56321ae25420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009b60 tx=0x7f89b00090e0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.232+0000 7f89a9ffb640 1 -- 172.21.15.64:0/69951805 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f89a40cee60 con 0x7f89b406e910 2024-05-09T09:23:40.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.233+0000 7f89bde41740 1 -- 172.21.15.64:0/1117482123 shutdown_connections 2024-05-09T09:23:40.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.233+0000 7f89bde41740 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae25a90 0x56321ae27f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.233+0000 7f89bde41740 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae20920 0x56321ae25420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.233+0000 7f89bde41740 1 --2- 172.21.15.64:0/1117482123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1ff30 0x56321ae20350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.233+0000 7f89bde41740 1 -- 172.21.15.64:0/1117482123 >> 172.21.15.64:0/1117482123 conn(0x56321ae1b5a0 msgr2=0x56321ae1b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.233+0000 7f89b96d1640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f899c0a2c60 0x7f899c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.234+0000 7f89bde41740 1 -- 172.21.15.64:0/1117482123 shutdown_connections 2024-05-09T09:23:40.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.235+0000 7f89b96d1640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f899c0a2c60 0x7f899c0a5120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x56321ae21180 tx=0x7f89ac01a5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.238+0000 7f89bde41740 1 -- 172.21.15.64:0/1117482123 wait complete. 2024-05-09T09:23:40.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.239+0000 7f89bde41740 1 -- start start 2024-05-09T09:23:40.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.239+0000 7f89bde41740 1 Processor -- start 2024-05-09T09:23:40.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89bde41740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89bde41740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89bde41740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae25a90 0x56321ae317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89bde41740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56321ae28a90 con 0x56321ae25a90 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89b96d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae25a90 0x56321ae317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89abfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89abfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae2a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49892/0 (socket says 172.21.15.64:49892) 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89abfff640 1 -- 172.21.15.64:0/2711251626 learned_addr learned my addr 172.21.15.64:0/2711251626 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.240+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56321ad3fb90 con 0x56321ae1ff30 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56321ae28c10 con 0x56321ae20920 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89bde41740 10 client.0 _ll_register_callbacks cb 0x56321ad7bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89b8ed0640 1 -- 172.21.15.64:0/2711251626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 msgr2=0x56321ae2a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae2a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89b8ed0640 1 -- 172.21.15.64:0/2711251626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae25a90 msgr2=0x56321ae317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae25a90 0x56321ae317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89b8ed0640 1 -- 172.21.15.64:0/2711251626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321aa983f0 con 0x56321ae1ff30 2024-05-09T09:23:40.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89abfff640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:40.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.241+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x56321ae1c470 tx=0x7f89b0009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7650764b600a219 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.242+0000 7f8993fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:40.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.242+0000 7f8993fff640 1 -- 172.21.15.64:0/2711251626 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017030 con 0x56321ae1ff30 2024-05-09T09:23:40.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.242+0000 7f8993fff640 1 -- 172.21.15.64:0/2711251626 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f89b001c840 con 0x56321ae1ff30 2024-05-09T09:23:40.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.242+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b000ebe0 con 0x56321ae1ff30 2024-05-09T09:23:40.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.248+0000 7f8993fff640 1 -- 172.21.15.64:0/2711251626 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f89b001ca40 con 0x56321ae1ff30 2024-05-09T09:23:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.249+0000 7f89bde41740 10 client.28385 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.249+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56321aa580f0 con 0x56321ae1ff30 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f8993fff640 1 -- 172.21.15.64:0/2711251626 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b001cbe0 con 0x56321ae1ff30 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f8993fff640 1 -- 172.21.15.64:0/2711251626 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f89b0012050 con 0x56321ae1ff30 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f89bde41740 10 client.28385 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f89bde41740 10 client.28385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f89bde41740 20 client.28385 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f89bde41740 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae35c60 0x56321ae38050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f89bde41740 4 client.28385 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:40.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.255+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f89b000ebe0 con 0x56321ae35c60 2024-05-09T09:23:40.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.259+0000 7f89abfff640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae35c60 0x56321ae38050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.260+0000 7f89abfff640 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae35c60 0x56321ae38050 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb44ddfcdfa81b3 server_cookie=69509f1332b668c3 in_seq=0 out_seq=0 2024-05-09T09:23:40.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.261+0000 7f8993fff640 10 client.28385 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:40.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.263+0000 7f8993fff640 1 -- 172.21.15.64:0/2711251626 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f89b000ebe0 con 0x56321ae35c60 2024-05-09T09:23:40.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.263+0000 7f8993fff640 10 client.28385 handle_command_reply: tid=0 2024-05-09T09:23:40.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.263+0000 7f89bde41740 1 client.28385 shutdown 2024-05-09T09:23:40.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.263+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 msgr2=0x56321ae2a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.263+0000 7f89bde41740 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x56321ae1c470 tx=0x7f89b0009f80 comp rx=0 tx=0).stop 2024-05-09T09:23:40.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 shutdown_connections 2024-05-09T09:23:40.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f89bde41740 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae35c60 0x56321ae38050 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f8993fff640 0 client.28385 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:40.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f89bde41740 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae25a90 0x56321ae317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f89bde41740 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f89bde41740 1 --2- 172.21.15.64:0/2711251626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a230 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.265+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 >> 172.21.15.64:0/2711251626 conn(0x56321ae1b5a0 msgr2=0x56321ae1a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.269+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 shutdown_connections 2024-05-09T09:23:40.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.269+0000 7f89bde41740 1 -- 172.21.15.64:0/2711251626 wait complete. 2024-05-09T09:23:40.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.272+0000 7f89bde41740 20 client.28385 trim_cache size 0 max 16384 2024-05-09T09:23:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.311+0000 7f89bde41740 1 Processor -- start 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.311+0000 7f89bde41740 1 -- start start 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.311+0000 7f89bde41740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1b5a0 0x56321ae29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.312+0000 7f89bde41740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.312+0000 7f89bde41740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.312+0000 7f89bde41740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b3410 con 0x56321ae1b5a0 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.312+0000 7f89bde41740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b0c80 con 0x56321ae1ff30 2024-05-09T09:23:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.312+0000 7f89bde41740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41aead0 con 0x56321ae20920 2024-05-09T09:23:40.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.313+0000 7f89b96d1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.313+0000 7f89b96d1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae1e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49900/0 (socket says 172.21.15.64:49900) 2024-05-09T09:23:40.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.313+0000 7f89b96d1640 1 -- 172.21.15.64:0/223078859 learned_addr learned my addr 172.21.15.64:0/223078859 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89abfff640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1b5a0 0x56321ae29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89b96d1640 1 -- 172.21.15.64:0/223078859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 msgr2=0x56321ae2a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89b96d1640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89b96d1640 1 -- 172.21.15.64:0/223078859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1b5a0 msgr2=0x56321ae29e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89abfff640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89b96d1640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1b5a0 0x56321ae29e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.314+0000 7f89b96d1640 1 -- 172.21.15.64:0/223078859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321ae24df0 con 0x56321ae20920 2024-05-09T09:23:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.315+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1b5a0 0x56321ae29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:23:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.315+0000 7f89b96d1640 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae1e590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f89ac040e10 tx=0x7f89ac050000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8136d615e96932cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.318+0000 7f899a7fc640 1 -- 172.21.15.64:0/223078859 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac045050 con 0x56321ae20920 2024-05-09T09:23:40.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.318+0000 7f899a7fc640 1 -- 172.21.15.64:0/223078859 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89ac0568f0 con 0x56321ae20920 2024-05-09T09:23:40.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.319+0000 7f899a7fc640 1 -- 172.21.15.64:0/223078859 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac059620 con 0x56321ae20920 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 -- 172.21.15.64:0/223078859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 msgr2=0x56321ae1e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae1e590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f89ac040e10 tx=0x7f89ac050000 comp rx=0 tx=0).stop 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 -- 172.21.15.64:0/223078859 shutdown_connections 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae20920 0x56321ae1e590 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 --2- 172.21.15.64:0/223078859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae1b5a0 0x56321ae29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.322+0000 7f89bde41740 1 -- 172.21.15.64:0/223078859 >> 172.21.15.64:0/223078859 conn(0x56321ae254e0 msgr2=0x56321ae21b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.326+0000 7f89bde41740 1 -- 172.21.15.64:0/223078859 shutdown_connections 2024-05-09T09:23:40.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.326+0000 7f89bde41740 1 -- 172.21.15.64:0/223078859 wait complete. 2024-05-09T09:23:40.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.327+0000 7f89bde41740 1 -- start start 2024-05-09T09:23:40.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.327+0000 7f89bde41740 1 Processor -- start 2024-05-09T09:23:40.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89bde41740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 0x56321ae2ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89bde41740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89bde41740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae20920 0x56321ae30230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89b8ed0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 0x56321ae2ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89b8ed0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 0x56321ae2ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49904/0 (socket says 172.21.15.64:49904) 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89b8ed0640 1 -- 172.21.15.64:0/4242885459 learned_addr learned my addr 172.21.15.64:0/4242885459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41af140 con 0x56321ae20920 2024-05-09T09:23:40.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b05b0 con 0x56321ae1ff30 2024-05-09T09:23:40.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41af810 con 0x56321ae1b5a0 2024-05-09T09:23:40.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.328+0000 7f89abfff640 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89b8ed0640 1 -- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 msgr2=0x56321ae2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89b8ed0640 1 -- 172.21.15.64:0/4242885459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae20920 msgr2=0x56321ae30230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae20920 0x56321ae30230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89b8ed0640 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321ae30820 con 0x56321ae1b5a0 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89bde41740 10 client.0 _ll_register_callbacks cb 0x7f89b00099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f89b8ed0640 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 0x56321ae2ece0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f89b00314f0 tx=0x7f89b0009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e8d5680e8348d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f8993fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f8993fff640 1 -- 172.21.15.64:0/4242885459 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017040 con 0x56321ae1b5a0 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.329+0000 7f8993fff640 1 -- 172.21.15.64:0/4242885459 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f89b001cbe0 con 0x56321ae1b5a0 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.330+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac0568f0 con 0x56321ae1b5a0 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.330+0000 7f8993fff640 1 -- 172.21.15.64:0/4242885459 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f89b001c850 con 0x56321ae1b5a0 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.330+0000 7f89bde41740 10 client.28567 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.330+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=49}) -- 0x56321ae9c770 con 0x56321ae1b5a0 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.330+0000 7f8993fff640 1 -- 172.21.15.64:0/4242885459 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f89b0012040 con 0x56321ae1b5a0 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89bde41740 10 client.28567 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89bde41740 10 client.28567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89bde41740 20 client.28567 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89bde41740 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae25a90 0x56321ae9f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89bde41740 4 client.28567 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:23:40.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f89ac0568f0 con 0x56321ae25a90 2024-05-09T09:23:40.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.331+0000 7f89abfff640 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae25a90 0x56321ae9f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.332+0000 7f8993fff640 10 client.28567 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.332+0000 7f89abfff640 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae25a90 0x56321ae9f150 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e60d01752788e1b server_cookie=82b62040fb2ab30 in_seq=0 out_seq=0 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.332+0000 7f8993fff640 1 -- 172.21.15.64:0/4242885459 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f89ac0568f0 con 0x56321ae25a90 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.332+0000 7f8993fff640 10 client.28567 handle_command_reply: tid=0 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.332+0000 7f89bde41740 1 client.28567 shutdown 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 msgr2=0x56321ae2ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 0x56321ae2ece0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f89b00314f0 tx=0x7f89b0009ad0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 shutdown_connections 2024-05-09T09:23:40.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56321ae25a90 0x56321ae9f150 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56321ae20920 0x56321ae30230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f8993fff640 0 client.28567 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56321ae1ff30 0x56321ae2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 --2- 172.21.15.64:0/4242885459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56321ae1b5a0 0x56321ae2ece0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.333+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 >> 172.21.15.64:0/4242885459 conn(0x56321ae254e0 msgr2=0x56321ae29e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.334+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 shutdown_connections 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.334+0000 7f89bde41740 1 -- 172.21.15.64:0/4242885459 wait complete. 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.334+0000 7f89bde41740 20 client.28567 trim_cache size 0 max 16384 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:23:40.336 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:23:40.337 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:23:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.337+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f899c0a2c60 msgr2=0x7f899c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.337+0000 7f89bb134640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f899c0a2c60 0x7f899c0a5120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x56321ae21180 tx=0x7f89ac01a5d0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.337+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406e910 msgr2=0x7f89b419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.337+0000 7f89bb134640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406e910 0x7f89b419e660 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d720 tx=0x7f89a400dbe0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.337+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 shutdown_connections 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f899c0a2c60 0x7f899c0a5120 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b406f2b0 0x7f89b41a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89b406e910 0x7f89b419e660 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 --2- 172.21.15.64:0/69951805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b406d6d0 0x7f89b419e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 >> 172.21.15.64:0/69951805 conn(0x7f89b406c2a0 msgr2=0x7f89b410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 shutdown_connections 2024-05-09T09:23:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.338+0000 7f89bb134640 1 -- 172.21.15.64:0/69951805 wait complete. 2024-05-09T09:23:40.361 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:23:40.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:23:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 -- 172.21.15.64:0/2251610378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 msgr2=0x7f26e810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 --2- 172.21.15.64:0/2251610378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e810e020 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009860 tx=0x7f26d802a340 comp rx=0 tx=0).stop 2024-05-09T09:23:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 -- 172.21.15.64:0/2251610378 shutdown_connections 2024-05-09T09:23:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 --2- 172.21.15.64:0/2251610378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e810e020 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 --2- 172.21.15.64:0/2251610378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26e810b1a0 0x7f26e810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 --2- 172.21.15.64:0/2251610378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e8108830 0x7f26e810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.551+0000 7f26eca1c640 1 -- 172.21.15.64:0/2251610378 >> 172.21.15.64:0/2251610378 conn(0x7f26e806e6d0 msgr2=0x7f26e8070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.552+0000 7f26eca1c640 1 -- 172.21.15.64:0/2251610378 shutdown_connections 2024-05-09T09:23:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.552+0000 7f26eca1c640 1 -- 172.21.15.64:0/2251610378 wait complete. 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.552+0000 7f26eca1c640 1 Processor -- start 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.552+0000 7f26eca1c640 1 -- start start 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26eca1c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26e8108830 0x7f26e8114f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26eca1c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e810b1a0 0x7f26e8115490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26eca1c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e81b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26eca1c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26e811c190 con 0x7f26e8108830 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26eca1c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26e811c010 con 0x7f26e810b1a0 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26eca1c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f26e811c310 con 0x7f26e810db70 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26e6d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e81b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26e5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e810b1a0 0x7f26e8115490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26e5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e810b1a0 0x7f26e8115490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57138/0 (socket says 172.21.15.64:57138) 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26e5d9b640 1 -- 172.21.15.64:0/466709127 learned_addr learned my addr 172.21.15.64:0/466709127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26e6d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e81b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49928/0 (socket says 172.21.15.64:49928) 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.553+0000 7f26e659c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26e8108830 0x7f26e8114f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.554+0000 7f26e6d9d640 1 -- 172.21.15.64:0/466709127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e810b1a0 msgr2=0x7f26e8115490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.554+0000 7f26e6d9d640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e810b1a0 0x7f26e8115490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.554+0000 7f26e6d9d640 1 -- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26e8108830 msgr2=0x7f26e8114f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.554+0000 7f26e6d9d640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26e8108830 0x7f26e8114f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:40.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.554+0000 7f26e6d9d640 1 -- 172.21.15.64:0/466709127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e81b6de0 con 0x7f26e810db70 2024-05-09T09:23:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.554+0000 7f26e6d9d640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e81b68a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009780 tx=0x7f26d8030d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.555+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8040020 con 0x7f26e810db70 2024-05-09T09:23:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.555+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26d8038040 con 0x7f26e810db70 2024-05-09T09:23:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.555+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81b6fb0 con 0x7f26e810db70 2024-05-09T09:23:40.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.555+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81b7360 con 0x7f26e810db70 2024-05-09T09:23:40.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.556+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8040020 con 0x7f26e810db70 2024-05-09T09:23:40.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.557+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e80711d0 con 0x7f26e810db70 2024-05-09T09:23:40.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.561+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26d8043020 con 0x7f26e810db70 2024-05-09T09:23:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.562+0000 7f26d77fe640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26d00a2f80 0x7f26d00a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:40.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.562+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f26d8030760 con 0x7f26e810db70 2024-05-09T09:23:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.562+0000 7f26e659c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26d00a2f80 0x7f26d00a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:40.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.563+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26d80bb320 con 0x7f26e810db70 2024-05-09T09:23:40.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:40.563+0000 7f26e659c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26d00a2f80 0x7f26d00a5440 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005fd0 tx=0x7f26e0009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.017+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26e802d370 con 0x7f26e810db70 2024-05-09T09:23:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.019+0000 7f26d77fe640 1 -- 172.21.15.64:0/466709127 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11885 (secure 0 0 0) 0x7f26d80bb320 con 0x7f26e810db70 2024-05-09T09:23:41.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:23:41.020 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:22:55.565953+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:23:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 49 2024-05-09T09:23:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.022+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26d00a2f80 msgr2=0x7f26d00a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.022+0000 7f26eca1c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26d00a2f80 0x7f26d00a5440 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005fd0 tx=0x7f26e0009680 comp rx=0 tx=0).stop 2024-05-09T09:23:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.022+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 msgr2=0x7f26e81b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.022+0000 7f26eca1c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e81b68a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009780 tx=0x7f26d8030d80 comp rx=0 tx=0).stop 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.022+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 shutdown_connections 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.023+0000 7f26eca1c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f26d00a2f80 0x7f26d00a5440 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.023+0000 7f26eca1c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e810db70 0x7f26e81b68a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.023+0000 7f26eca1c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e810b1a0 0x7f26e8115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.023+0000 7f26eca1c640 1 --2- 172.21.15.64:0/466709127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26e8108830 0x7f26e8114f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.023+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 >> 172.21.15.64:0/466709127 conn(0x7f26e806e6d0 msgr2=0x7f26e8109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.025+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 shutdown_connections 2024-05-09T09:23:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.027+0000 7f26eca1c640 1 -- 172.21.15.64:0/466709127 wait complete. 2024-05-09T09:23:41.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:23:41.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:40 smithi179 ceph-mon[34086]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.2 MiB/s wr, 935 op/s 2024-05-09T09:23:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.248+0000 7f92a8f10640 1 -- 172.21.15.64:0/3699146493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 msgr2=0x7f92a410dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.248+0000 7f92a8f10640 1 --2- 172.21.15.64:0/3699146493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 0x7f92a410dd00 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f929800aa10 tx=0x7f929802b750 comp rx=0 tx=0).stop 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 -- 172.21.15.64:0/3699146493 shutdown_connections 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 --2- 172.21.15.64:0/3699146493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a410e370 0x7f92a4110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 --2- 172.21.15.64:0/3699146493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 0x7f92a410dd00 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 --2- 172.21.15.64:0/3699146493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a410b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 -- 172.21.15.64:0/3699146493 >> 172.21.15.64:0/3699146493 conn(0x7f92a406ea60 msgr2=0x7f92a4070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 -- 172.21.15.64:0/3699146493 shutdown_connections 2024-05-09T09:23:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.249+0000 7f92a8f10640 1 -- 172.21.15.64:0/3699146493 wait complete. 2024-05-09T09:23:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.250+0000 7f92a8f10640 1 Processor -- start 2024-05-09T09:23:41.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.251+0000 7f92a8f10640 1 -- start start 2024-05-09T09:23:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.251+0000 7f92a8f10640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a41a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.251+0000 7f92a8f10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 0x7f92a41a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.251+0000 7f92a8f10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a410e370 0x7f92a41a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a41a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a41a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49944/0 (socket says 172.21.15.64:49944) 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 -- 172.21.15.64:0/1924507901 learned_addr learned my addr 172.21.15.64:0/1924507901 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a8f10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4113a60 con 0x7f92a410b910 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41138e0 con 0x7f92a410e370 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4113be0 con 0x7f92a406a670 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a410e370 0x7f92a41a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 0x7f92a41a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 -- 172.21.15.64:0/1924507901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a410e370 msgr2=0x7f92a41a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a410e370 0x7f92a41a9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 -- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 msgr2=0x7f92a41a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 0x7f92a41a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.252+0000 7f92a259c640 1 -- 172.21.15.64:0/1924507901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a41aa210 con 0x7f92a406a670 2024-05-09T09:23:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.253+0000 7f92a259c640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a41a25f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f929400d770 tx=0x7f929400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.253+0000 7f92937fe640 1 -- 172.21.15.64:0/1924507901 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294017070 con 0x7f92a406a670 2024-05-09T09:23:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.253+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a41aa500 con 0x7f92a406a670 2024-05-09T09:23:41.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.253+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41aaaf0 con 0x7f92a406a670 2024-05-09T09:23:41.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.254+0000 7f92937fe640 1 -- 172.21.15.64:0/1924507901 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9294009c30 con 0x7f92a406a670 2024-05-09T09:23:41.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.254+0000 7f92937fe640 1 -- 172.21.15.64:0/1924507901 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294003d00 con 0x7f92a406a670 2024-05-09T09:23:41.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.257+0000 7f92abc1d740 1 Processor -- start 2024-05-09T09:23:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 -- start start 2024-05-09T09:23:41.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c14140 0x55caf9c18c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c192b0 0x55caf9c1b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55caf9b6fc40 con 0x55caf9c192b0 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55caf9b17580 con 0x55caf9c14140 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92abc1d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55caf9b6bc00 con 0x55caf9c13750 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.260+0000 7f92937fe640 1 -- 172.21.15.64:0/1924507901 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9294010460 con 0x7f92a406a670 2024-05-09T09:23:41.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c13b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49952/0 (socket says 172.21.15.64:49952) 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 -- 172.21.15.64:0/4138722343 learned_addr learned my addr 172.21.15.64:0/4138722343 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c192b0 0x55caf9c1b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 -- 172.21.15.64:0/4138722343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c14140 msgr2=0x55caf9c18c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c14140 0x55caf9c18c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 -- 172.21.15.64:0/4138722343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c192b0 msgr2=0x55caf9c1b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.261+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c192b0 0x55caf9c1b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.262+0000 7f92a2d9d640 1 -- 172.21.15.64:0/4138722343 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf98833f0 con 0x55caf9c13750 2024-05-09T09:23:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.262+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c13b70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b500 tx=0x7f929c009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ac2be01e3c1ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.263+0000 7f92937fe640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f928c0a2c70 0x7f928c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.263+0000 7f9290ff9640 1 -- 172.21.15.64:0/4138722343 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c00a970 con 0x55caf9c13750 2024-05-09T09:23:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.263+0000 7f9290ff9640 1 -- 172.21.15.64:0/4138722343 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929c019070 con 0x55caf9c13750 2024-05-09T09:23:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.263+0000 7f9290ff9640 1 -- 172.21.15.64:0/4138722343 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c013380 con 0x55caf9c13750 2024-05-09T09:23:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.263+0000 7f92937fe640 1 -- 172.21.15.64:0/1924507901 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f92940cf040 con 0x7f92a406a670 2024-05-09T09:23:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 -- 172.21.15.64:0/4138722343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 msgr2=0x55caf9c13b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c13b70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b500 tx=0x7f929c009630 comp rx=0 tx=0).stop 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 -- 172.21.15.64:0/4138722343 shutdown_connections 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c192b0 0x55caf9c1b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c14140 0x55caf9c18c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 --2- 172.21.15.64:0/4138722343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c13b70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.265+0000 7f92abc1d740 1 -- 172.21.15.64:0/4138722343 >> 172.21.15.64:0/4138722343 conn(0x55caf9c0edc0 msgr2=0x55caf9c0f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.266+0000 7f92a259c640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f928c0a2c70 0x7f928c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.267+0000 7f92a259c640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f928c0a2c70 0x7f928c0a5130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f92940a3bf0 tx=0x7f929400de80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.267+0000 7f92abc1d740 1 -- 172.21.15.64:0/4138722343 shutdown_connections 2024-05-09T09:23:41.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.268+0000 7f92abc1d740 1 -- 172.21.15.64:0/4138722343 wait complete. 2024-05-09T09:23:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.268+0000 7f92abc1d740 1 -- start start 2024-05-09T09:23:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.268+0000 7f92abc1d740 1 Processor -- start 2024-05-09T09:23:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c1dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c25150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55caf9c1c260 con 0x55caf9c14140 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55caf9b2ac40 con 0x55caf9c192b0 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55caf9c1c3e0 con 0x55caf9c13750 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92abc1d740 10 client.0 _ll_register_callbacks cb 0x55caf9882be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c1dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c1dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49964/0 (socket says 172.21.15.64:49964) 2024-05-09T09:23:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 -- 172.21.15.64:0/2987222672 learned_addr learned my addr 172.21.15.64:0/2987222672 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c25150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 -- 172.21.15.64:0/2987222672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 msgr2=0x55caf9c25150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c25150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 -- 172.21.15.64:0/2987222672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 msgr2=0x55caf9c1e110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.270+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c1e110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.271+0000 7f92a2d9d640 1 -- 172.21.15.64:0/2987222672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf994d7d0 con 0x55caf9c13750 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.271+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c1dbd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f929c004b30 tx=0x7f929c004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=527165e5dbed81a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.271+0000 7f9289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.271+0000 7f9289ffb640 1 -- 172.21.15.64:0/2987222672 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c027030 con 0x55caf9c13750 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.272+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c019070 con 0x55caf9c13750 2024-05-09T09:23:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.272+0000 7f9289ffb640 1 -- 172.21.15.64:0/2987222672 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f929c00a770 con 0x55caf9c13750 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.272+0000 7f9289ffb640 1 -- 172.21.15.64:0/2987222672 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f929c034050 con 0x55caf9c13750 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.272+0000 7f92abc1d740 10 client.28585 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.272+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55caf9c284f0 con 0x55caf9c13750 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f9289ffb640 1 -- 172.21.15.64:0/2987222672 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929c019070 con 0x55caf9c13750 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f9289ffb640 1 -- 172.21.15.64:0/2987222672 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f929c01d980 con 0x55caf9c13750 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f92abc1d740 10 client.28585 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f92abc1d740 10 client.28585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f92abc1d740 20 client.28585 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f92abc1d740 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c29950 0x55caf9c2bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f92abc1d740 4 client.28585 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:41.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.273+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf9b7d370 con 0x55caf9c29950 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.274+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c29950 0x55caf9c2bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.274+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c29950 0x55caf9c2bd40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=882aff5b5d1bb885 server_cookie=93d8619a7680a95b in_seq=0 out_seq=0 2024-05-09T09:23:41.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.274+0000 7f9289ffb640 10 client.28585 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.276+0000 7f9289ffb640 1 -- 172.21.15.64:0/2987222672 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caf9b7d370 con 0x55caf9c29950 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.276+0000 7f9289ffb640 10 client.28585 handle_command_reply: tid=0 2024-05-09T09:23:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.276+0000 7f92abc1d740 1 client.28585 shutdown 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.276+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 msgr2=0x55caf9c1dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.276+0000 7f92abc1d740 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c1dbd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f929c004b30 tx=0x7f929c004a80 comp rx=0 tx=0).stop 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 shutdown_connections 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c29950 0x55caf9c2bd40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c25150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 --2- 172.21.15.64:0/2987222672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c1dbd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f9289ffb640 0 client.28585 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 >> 172.21.15.64:0/2987222672 conn(0x55caf9c0edc0 msgr2=0x55caf9c057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 shutdown_connections 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 1 -- 172.21.15.64:0/2987222672 wait complete. 2024-05-09T09:23:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.277+0000 7f92abc1d740 20 client.28585 trim_cache size 0 max 16384 2024-05-09T09:23:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.304+0000 7f92abc1d740 1 Processor -- start 2024-05-09T09:23:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.304+0000 7f92abc1d740 1 -- start start 2024-05-09T09:23:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.304+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 0x55caf9c1d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.304+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 0x55caf9c1d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c14140 0x55caf9c250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 0x55caf9c1d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46668/0 (socket says 172.21.15.64:46668) 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92a2d9d640 1 -- 172.21.15.64:0/72278756 learned_addr learned my addr 172.21.15.64:0/72278756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92abc1d740 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c1dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c14140 0x55caf9c250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9284012580 con 0x55caf9c13750 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f928400fdf0 con 0x55caf9c192b0 2024-05-09T09:23:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.305+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f928400dc40 con 0x55caf9c14140 2024-05-09T09:23:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.306+0000 7f92a2d9d640 1 -- 172.21.15.64:0/72278756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c14140 msgr2=0x55caf9c250e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.306+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c14140 0x55caf9c250e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.306+0000 7f92a2d9d640 1 -- 172.21.15.64:0/72278756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 msgr2=0x55caf9c1dd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.306+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c1dd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.306+0000 7f92a2d9d640 1 -- 172.21.15.64:0/72278756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf9bf7d20 con 0x55caf9c13750 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.306+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 0x55caf9c1d7e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f929c01df70 tx=0x7f929c00a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d997f9f12ad843ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.307+0000 7f928bfff640 1 -- 172.21.15.64:0/72278756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c027030 con 0x55caf9c13750 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.307+0000 7f928bfff640 1 -- 172.21.15.64:0/72278756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929c00be50 con 0x55caf9c13750 2024-05-09T09:23:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.307+0000 7f928bfff640 1 -- 172.21.15.64:0/72278756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0148e0 con 0x55caf9c13750 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 msgr2=0x55caf9c1d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 0x55caf9c1d7e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f929c01df70 tx=0x7f929c00a2e0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 shutdown_connections 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c1dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c14140 0x55caf9c250e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 --2- 172.21.15.64:0/72278756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c13750 0x55caf9c1d7e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.308+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 >> 172.21.15.64:0/72278756 conn(0x55caf9c0edc0 msgr2=0x55caf9c15540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.309+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 shutdown_connections 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.309+0000 7f92abc1d740 1 -- 172.21.15.64:0/72278756 wait complete. 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.309+0000 7f92abc1d740 1 -- start start 2024-05-09T09:23:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.309+0000 7f92abc1d740 1 Processor -- start 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c21e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c21e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c21e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49980/0 (socket says 172.21.15.64:49980) 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92a1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92a1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46684/0 (socket says 172.21.15.64:46684) 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92a1d9b640 1 -- 172.21.15.64:0/438141532 learned_addr learned my addr 172.21.15.64:0/438141532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f928400e2b0 con 0x55caf9c14140 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f928400f720 con 0x55caf9c192b0 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f928400e980 con 0x55caf9c13750 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.310+0000 7f92abc1d740 10 client.0 _ll_register_callbacks cb 0x7f929800a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a1d9b640 1 -- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 msgr2=0x55caf9c21e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c21e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a1d9b640 1 -- 172.21.15.64:0/438141532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 msgr2=0x55caf9c8de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c8de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a1d9b640 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf9c229f0 con 0x55caf9c14140 2024-05-09T09:23:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f92a1d9b640 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c223b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9298005ab0 tx=0x7f9298005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5646b123206233 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.311+0000 7f9289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.312+0000 7f9289ffb640 1 -- 172.21.15.64:0/438141532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298036040 con 0x55caf9c14140 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.312+0000 7f9289ffb640 1 -- 172.21.15.64:0/438141532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9298004450 con 0x55caf9c14140 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.312+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c00be50 con 0x55caf9c14140 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.313+0000 7f9289ffb640 1 -- 172.21.15.64:0/438141532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f92980046e0 con 0x55caf9c14140 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.313+0000 7f92abc1d740 10 client.18939 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:41.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.313+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55caf9c8f4e0 con 0x55caf9c14140 2024-05-09T09:23:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f9289ffb640 1 -- 172.21.15.64:0/438141532 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9298048070 con 0x55caf9c14140 2024-05-09T09:23:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f9289ffb640 1 -- 172.21.15.64:0/438141532 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f929803b040 con 0x55caf9c14140 2024-05-09T09:23:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92abc1d740 10 client.18939 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92abc1d740 10 client.18939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:23:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92abc1d740 20 client.18939 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92abc1d740 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c8ff20 0x55caf9c92340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92abc1d740 4 client.18939 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f929c00be50 con 0x55caf9c8ff20 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.314+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c8ff20 0x55caf9c92340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.315+0000 7f9289ffb640 10 client.18939 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.315+0000 7f92a2d9d640 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c8ff20 0x55caf9c92340 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78cba3e39fbcd00f server_cookie=2ba95725468bcfc7 in_seq=0 out_seq=0 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f9289ffb640 1 -- 172.21.15.64:0/438141532 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f929c00be50 con 0x55caf9c8ff20 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f9289ffb640 10 client.18939 handle_command_reply: tid=0 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f92abc1d740 1 client.18939 shutdown 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 msgr2=0x55caf9c223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f92abc1d740 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c223b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9298005ab0 tx=0x7f9298005510 comp rx=0 tx=0).stop 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 shutdown_connections 2024-05-09T09:23:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f92abc1d740 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55caf9c8ff20 0x55caf9c92340 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.316+0000 7f92abc1d740 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55caf9c192b0 0x55caf9c8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f92abc1d740 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55caf9c14140 0x55caf9c223b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f9289ffb640 0 client.18939 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f92abc1d740 1 --2- 172.21.15.64:0/438141532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caf9c13750 0x55caf9c21e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 >> 172.21.15.64:0/438141532 conn(0x55caf9c0edc0 msgr2=0x55caf9c1d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 shutdown_connections 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f92abc1d740 1 -- 172.21.15.64:0/438141532 wait complete. 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.317+0000 7f92abc1d740 20 client.18939 trim_cache size 0 max 16384 2024-05-09T09:23:41.320 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:23:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.318+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f928c0a2c70 msgr2=0x7f928c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.318+0000 7f92a8f10640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f928c0a2c70 0x7f928c0a5130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f92940a3bf0 tx=0x7f929400de80 comp rx=0 tx=0).stop 2024-05-09T09:23:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.318+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 msgr2=0x7f92a41a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a41a25f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f929400d770 tx=0x7f929400dc30 comp rx=0 tx=0).stop 2024-05-09T09:23:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 shutdown_connections 2024-05-09T09:23:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f928c0a2c70 0x7f928c0a5130 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a410e370 0x7f92a41a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92a410b910 0x7f92a41a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 --2- 172.21.15.64:0/1924507901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92a406a670 0x7f92a41a25f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.319+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 >> 172.21.15.64:0/1924507901 conn(0x7f92a406ea60 msgr2=0x7f92a4109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.320+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 shutdown_connections 2024-05-09T09:23:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.320+0000 7f92a8f10640 1 -- 172.21.15.64:0/1924507901 wait complete. 2024-05-09T09:23:41.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:40 smithi139 ceph-mon[33599]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.2 MiB/s wr, 935 op/s 2024-05-09T09:23:41.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:40 smithi064 ceph-mon[30149]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.2 MiB/s wr, 935 op/s 2024-05-09T09:23:41.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:23:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 800.0 s, 712.4 tps, lat 140.497 ms stddev 101.881 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 -- 172.21.15.64:0/2777953771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c00793f0 msgr2=0x7fc6c007b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/2777953771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c00793f0 0x7fc6c007b880 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009860 tx=0x7fc6b402a340 comp rx=0 tx=0).stop 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 -- 172.21.15.64:0/2777953771 shutdown_connections 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/2777953771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c00793f0 0x7fc6c007b880 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/2777953771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00765f0 0x7fc6c0078a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/2777953771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c0073710 0x7fc6c0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.541+0000 7fc6c5aca640 1 -- 172.21.15.64:0/2777953771 >> 172.21.15.64:0/2777953771 conn(0x7fc6c006e6c0 msgr2=0x7fc6c0070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.542+0000 7fc6c5aca640 1 -- 172.21.15.64:0/2777953771 shutdown_connections 2024-05-09T09:23:41.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.542+0000 7fc6c5aca640 1 -- 172.21.15.64:0/2777953771 wait complete. 2024-05-09T09:23:41.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 Processor -- start 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 -- start start 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c0073710 0x7fc6c012aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c00765f0 0x7fc6c012afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 0x7fc6c0131910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c007eba0 con 0x7fc6c0073710 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c007ea20 con 0x7fc6c00765f0 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.543+0000 7fc6c5aca640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c007ed20 con 0x7fc6c00793f0 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c0073710 0x7fc6c012aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6bffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 0x7fc6c0131910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6bffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 0x7fc6c0131910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:49994/0 (socket says 172.21.15.64:49994) 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c0073710 0x7fc6c012aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46702/0 (socket says 172.21.15.64:46702) 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6bffff640 1 -- 172.21.15.64:0/4094640130 learned_addr learned my addr 172.21.15.64:0/4094640130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c00765f0 0x7fc6c012afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 -- 172.21.15.64:0/4094640130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 msgr2=0x7fc6c0131910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 0x7fc6c0131910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 -- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c0073710 msgr2=0x7fc6c012aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c0073710 0x7fc6c012aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 -- 172.21.15.64:0/4094640130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c0131f80 con 0x7fc6c00765f0 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6bffff640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 0x7fc6c0131910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.544+0000 7fc6beffd640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c00765f0 0x7fc6c012afa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000d720 tx=0x7fc6b000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.545+0000 7fc6bcff9640 1 -- 172.21.15.64:0/4094640130 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0017070 con 0x7fc6c00765f0 2024-05-09T09:23:41.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.545+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c0132270 con 0x7fc6c00765f0 2024-05-09T09:23:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.545+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01327c0 con 0x7fc6c00765f0 2024-05-09T09:23:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.545+0000 7fc6bcff9640 1 -- 172.21.15.64:0/4094640130 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b000de30 con 0x7fc6c00765f0 2024-05-09T09:23:41.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.545+0000 7fc6bcff9640 1 -- 172.21.15.64:0/4094640130 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00047a0 con 0x7fc6c00765f0 2024-05-09T09:23:41.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.550+0000 7fc6bcff9640 1 -- 172.21.15.64:0/4094640130 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc6b0004940 con 0x7fc6c00765f0 2024-05-09T09:23:41.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.550+0000 7fc6c87d7740 1 Processor -- start 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.550+0000 7fc6c87d7740 1 -- start start 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c3529360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c352e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3530f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6c87d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5593c348dc40 con 0x5593c3528f40 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6c87d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5593c3435580 con 0x5593c3529930 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6c87d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5593c3489c00 con 0x5593c352eaa0 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3530f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3530f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50010/0 (socket says 172.21.15.64:50010) 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 -- 172.21.15.64:0/626279267 learned_addr learned my addr 172.21.15.64:0/626279267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c352e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6bffff640 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c3529360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 -- 172.21.15.64:0/626279267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 msgr2=0x5593c352e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c352e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 -- 172.21.15.64:0/626279267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 msgr2=0x5593c3529360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c3529360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.551+0000 7fc6beffd640 1 -- 172.21.15.64:0/626279267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c31a13f0 con 0x5593c352eaa0 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.552+0000 7fc6beffd640 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3530f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0016000 tx=0x7fc6b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f589d0ca7fb1c808 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.552+0000 7fc6a5ffb640 1 -- 172.21.15.64:0/626279267 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00db470 con 0x5593c352eaa0 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.552+0000 7fc6a5ffb640 1 -- 172.21.15.64:0/626279267 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b00e0070 con 0x5593c352eaa0 2024-05-09T09:23:41.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.552+0000 7fc6a5ffb640 1 -- 172.21.15.64:0/626279267 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00da980 con 0x5593c352eaa0 2024-05-09T09:23:41.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.553+0000 7fc6bcff9640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6a80a2d60 0x7fc6a80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.553+0000 7fc6bffff640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6a80a2d60 0x7fc6a80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.553+0000 7fc6bcff9640 1 -- 172.21.15.64:0/4094640130 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc6b00cfea0 con 0x7fc6c00765f0 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.553+0000 7fc6c87d7740 1 -- 172.21.15.64:0/626279267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 msgr2=0x5593c3530f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.553+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3530f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0016000 tx=0x7fc6b0014000 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.554+0000 7fc6c87d7740 1 -- 172.21.15.64:0/626279267 shutdown_connections 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.554+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3530f30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.554+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c352e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.554+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/626279267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c3529360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.554+0000 7fc6c87d7740 1 -- 172.21.15.64:0/626279267 >> 172.21.15.64:0/626279267 conn(0x5593c35245b0 msgr2=0x5593c35249b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.554+0000 7fc6bffff640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6a80a2d60 0x7fc6a80a5220 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5593c352a190 tx=0x7fc6b4030440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.559+0000 7fc6c87d7740 1 -- 172.21.15.64:0/626279267 shutdown_connections 2024-05-09T09:23:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.560+0000 7fc6c87d7740 1 -- 172.21.15.64:0/626279267 wait complete. 2024-05-09T09:23:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.560+0000 7fc6c87d7740 1 -- start start 2024-05-09T09:23:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 Processor -- start 2024-05-09T09:23:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3533310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3533850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 0x5593c353a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5593c3531a50 con 0x5593c352eaa0 2024-05-09T09:23:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5593c3448c40 con 0x5593c3528f40 2024-05-09T09:23:41.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5593c3531bd0 con 0x5593c3529930 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6c87d7740 10 client.0 _ll_register_callbacks cb 0x5593c31a0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3533310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3533310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57196/0 (socket says 172.21.15.64:57196) 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/567520085 learned_addr learned my addr 172.21.15.64:0/567520085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.561+0000 7fc6beffd640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3533850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 msgr2=0x5593c3533850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3533850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/567520085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 msgr2=0x5593c353a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 0x5593c353a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/567520085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c326b7d0 con 0x5593c3528f40 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6beffd640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3533850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3533310 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005fd0 tx=0x7fc6b800d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4ccd24c52d9b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.562+0000 7fc69effd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:41.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.563+0000 7fc69effd640 1 -- 172.21.15.64:0/567520085 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b801c020 con 0x5593c3528f40 2024-05-09T09:23:41.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.563+0000 7fc69effd640 1 -- 172.21.15.64:0/567520085 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc6b8018660 con 0x5593c3528f40 2024-05-09T09:23:41.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.563+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b00e0070 con 0x5593c3528f40 2024-05-09T09:23:41.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.563+0000 7fc69effd640 1 -- 172.21.15.64:0/567520085 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc6b8018860 con 0x5593c3528f40 2024-05-09T09:23:41.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.563+0000 7fc6c87d7740 10 client.28415 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:41.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.563+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5593c353dc70 con 0x5593c3528f40 2024-05-09T09:23:41.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.564+0000 7fc69effd640 1 -- 172.21.15.64:0/567520085 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b80173c0 con 0x5593c3528f40 2024-05-09T09:23:41.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc69effd640 1 -- 172.21.15.64:0/567520085 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc6b800b040 con 0x5593c3528f40 2024-05-09T09:23:41.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc6c87d7740 10 client.28415 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:41.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc6c87d7740 10 client.28415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:23:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc6c87d7740 20 client.28415 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c353f330 0x5593c35417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc6c87d7740 4 client.28415 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.565+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc6b00e0070 con 0x5593c353f330 2024-05-09T09:23:41.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.567+0000 7fc6beffd640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c353f330 0x5593c35417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.569+0000 7fc6beffd640 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c353f330 0x5593c35417d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca1f8e4b92f32424 server_cookie=78038930003afb39 in_seq=0 out_seq=0 2024-05-09T09:23:41.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.569+0000 7fc69effd640 10 client.28415 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:41.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.571+0000 7fc69effd640 1 -- 172.21.15.64:0/567520085 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc6b00e0070 con 0x5593c353f330 2024-05-09T09:23:41.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.571+0000 7fc69effd640 10 client.28415 handle_command_reply: tid=0 2024-05-09T09:23:41.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.571+0000 7fc6c87d7740 1 client.28415 shutdown 2024-05-09T09:23:41.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 msgr2=0x5593c3533310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3533310 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005fd0 tx=0x7fc6b800d7b0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 shutdown_connections 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c353f330 0x5593c35417d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 0x5593c353a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3533850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/567520085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3533310 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.572+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 >> 172.21.15.64:0/567520085 conn(0x5593c35245b0 msgr2=0x5593c35237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.573+0000 7fc69effd640 0 client.28415 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.573+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 shutdown_connections 2024-05-09T09:23:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.574+0000 7fc6c87d7740 1 -- 172.21.15.64:0/567520085 wait complete. 2024-05-09T09:23:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.574+0000 7fc6c87d7740 20 client.28415 trim_cache size 0 max 16384 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.605+0000 7fc6c87d7740 1 Processor -- start 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.605+0000 7fc6c87d7740 1 -- start start 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c35460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c353a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3546610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c353a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c353a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57208/0 (socket says 172.21.15.64:57208) 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6beffd640 1 -- 172.21.15.64:0/1885769268 learned_addr learned my addr 172.21.15.64:0/1885769268 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6bffff640 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3546610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0125b0 con 0x5593c3528f40 2024-05-09T09:23:41.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c35460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac00fe20 con 0x5593c3529930 2024-05-09T09:23:41.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.606+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac00dc70 con 0x5593c352eaa0 2024-05-09T09:23:41.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/1885769268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 msgr2=0x5593c3546610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3546610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/1885769268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 msgr2=0x5593c353a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c353a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/1885769268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c3515d20 con 0x5593c3528f40 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c35460d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8023040 tx=0x7fc6b80204a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fd92d705723c86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.607+0000 7fc6a57fa640 1 -- 172.21.15.64:0/1885769268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8018c00 con 0x5593c3528f40 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.608+0000 7fc6a57fa640 1 -- 172.21.15.64:0/1885769268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b8003e20 con 0x5593c3528f40 2024-05-09T09:23:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.608+0000 7fc6a57fa640 1 -- 172.21.15.64:0/1885769268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8020d60 con 0x5593c3528f40 2024-05-09T09:23:41.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.609+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 msgr2=0x5593c35460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.609+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c35460d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8023040 tx=0x7fc6b80204a0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.610+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 shutdown_connections 2024-05-09T09:23:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.610+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c352eaa0 0x5593c3546610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.610+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3529930 0x5593c353a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.610+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/1885769268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c3528f40 0x5593c35460d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.610+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 >> 172.21.15.64:0/1885769268 conn(0x5593c35245b0 msgr2=0x5593c35237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.611+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 shutdown_connections 2024-05-09T09:23:41.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.611+0000 7fc6c87d7740 1 -- 172.21.15.64:0/1885769268 wait complete. 2024-05-09T09:23:41.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.612+0000 7fc6c87d7740 1 -- start start 2024-05-09T09:23:41.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.612+0000 7fc6c87d7740 1 Processor -- start 2024-05-09T09:23:41.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.613+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3537550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.613+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3537a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6c87d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 0x5593c35a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3537550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3537550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57212/0 (socket says 172.21.15.64:57212) 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3537a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3537a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50036/0 (socket says 172.21.15.64:50036) 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/2288822428 learned_addr learned my addr 172.21.15.64:0/2288822428 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac00e2e0 con 0x5593c352eaa0 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac00f750 con 0x5593c3528f40 2024-05-09T09:23:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.614+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac00e9b0 con 0x5593c3529930 2024-05-09T09:23:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6c87d7740 10 client.0 _ll_register_callbacks cb 0x7fc6b8009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 msgr2=0x5593c3537a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3537a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/2288822428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 msgr2=0x5593c35a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 0x5593c35a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6bf7fe640 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c3538080 con 0x5593c3528f40 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6bf7fe640 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3537550 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009af0 tx=0x7fc6b8018c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ca0efbbc15cff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc69effd640 10 client.28421 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b8003e20 con 0x5593c3528f40 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.615+0000 7fc69effd640 1 -- 172.21.15.64:0/2288822428 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8009280 con 0x5593c3528f40 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.616+0000 7fc69effd640 1 -- 172.21.15.64:0/2288822428 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc6b8020c70 con 0x5593c3528f40 2024-05-09T09:23:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.616+0000 7fc69effd640 1 -- 172.21.15.64:0/2288822428 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc6b8020e70 con 0x5593c3528f40 2024-05-09T09:23:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.617+0000 7fc6c87d7740 10 client.28421 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.617+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=49}) -- 0x5593c35a4df0 con 0x5593c3528f40 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.623+0000 7fc69effd640 1 -- 172.21.15.64:0/2288822428 <== mon.2 v2:172.21.15.139:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc6b800b070 con 0x5593c3528f40 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.623+0000 7fc6c87d7740 10 client.28421 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.623+0000 7fc6c87d7740 10 client.28421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.623+0000 7fc6c87d7740 20 client.28421 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.623+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c35a5680 0x5593c35a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.624+0000 7fc6c87d7740 4 client.28421 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:23:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.624+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc6b8003e20 con 0x5593c35a5680 2024-05-09T09:23:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.624+0000 7fc6beffd640 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c35a5680 0x5593c35a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.625+0000 7fc6beffd640 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c35a5680 0x5593c35a7aa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9db565bae980865e server_cookie=f8f597cba3af167a in_seq=0 out_seq=0 2024-05-09T09:23:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.625+0000 7fc69effd640 10 client.28421 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.626+0000 7fc69effd640 1 -- 172.21.15.64:0/2288822428 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc6b8003e20 con 0x5593c35a5680 2024-05-09T09:23:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.626+0000 7fc69effd640 10 client.28421 handle_command_reply: tid=0 2024-05-09T09:23:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.626+0000 7fc6c87d7740 1 client.28421 shutdown 2024-05-09T09:23:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.626+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 msgr2=0x5593c3537550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.626+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3537550 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009af0 tx=0x7fc6b8018c30 comp rx=0 tx=0).stop 2024-05-09T09:23:41.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 shutdown_connections 2024-05-09T09:23:41.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x5593c35a5680 0x5593c35a7aa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593c352eaa0 0x5593c35a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593c3529930 0x5593c3537a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 --2- 172.21.15.64:0/2288822428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593c3528f40 0x5593c3537550 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc69effd640 0 client.28421 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:23:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 >> 172.21.15.64:0/2288822428 conn(0x5593c35245b0 msgr2=0x5593c3527330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 shutdown_connections 2024-05-09T09:23:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 1 -- 172.21.15.64:0/2288822428 wait complete. 2024-05-09T09:23:41.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.628+0000 7fc6c87d7740 20 client.28421 trim_cache size 0 max 16384 2024-05-09T09:23:41.629 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:23:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.629+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6a80a2d60 msgr2=0x7fc6a80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.629+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6a80a2d60 0x7fc6a80a5220 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5593c352a190 tx=0x7fc6b4030440 comp rx=0 tx=0).stop 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.629+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c00765f0 msgr2=0x7fc6c012afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.629+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c00765f0 0x7fc6c012afa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000d720 tx=0x7fc6b000dbe0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 shutdown_connections 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6a80a2d60 0x7fc6a80a5220 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6c00793f0 0x7fc6c0131910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6c00765f0 0x7fc6c012afa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 --2- 172.21.15.64:0/4094640130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c0073710 0x7fc6c012aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 >> 172.21.15.64:0/4094640130 conn(0x7fc6c006e6c0 msgr2=0x7fc6c0076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 shutdown_connections 2024-05-09T09:23:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.630+0000 7fc6c5aca640 1 -- 172.21.15.64:0/4094640130 wait complete. 2024-05-09T09:23:41.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:23:41.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.879+0000 7fe503815640 1 -- 172.21.15.64:0/1274907337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 msgr2=0x7fe4fc100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.879+0000 7fe503815640 1 --2- 172.21.15.64:0/1274907337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc100b90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009900 tx=0x7fe4f802a3f0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.880+0000 7fe503815640 1 -- 172.21.15.64:0/1274907337 shutdown_connections 2024-05-09T09:23:41.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.880+0000 7fe503815640 1 --2- 172.21.15.64:0/1274907337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4fc102320 0x7fe4fc0689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.880+0000 7fe503815640 1 --2- 172.21.15.64:0/1274907337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4fc101960 0x7fe4fc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.880+0000 7fe503815640 1 --2- 172.21.15.64:0/1274907337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc100b90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.880+0000 7fe503815640 1 -- 172.21.15.64:0/1274907337 >> 172.21.15.64:0/1274907337 conn(0x7fe4fc0fbce0 msgr2=0x7fe4fc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.882+0000 7fe503815640 1 -- 172.21.15.64:0/1274907337 shutdown_connections 2024-05-09T09:23:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.882+0000 7fe503815640 1 -- 172.21.15.64:0/1274907337 wait complete. 2024-05-09T09:23:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 Processor -- start 2024-05-09T09:23:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 -- start start 2024-05-09T09:23:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc107640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4fc101960 0x7fe4fc107b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4fc102320 0x7fe4fc10ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe5015b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe5015b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46758/0 (socket says 172.21.15.64:46758) 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe5015b1640 1 -- 172.21.15.64:0/3995837323 learned_addr learned my addr 172.21.15.64:0/3995837323 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc10f610 con 0x7fe4fc100770 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc10f490 con 0x7fe4fc102320 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc10f790 con 0x7fe4fc101960 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.883+0000 7fe500db0640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4fc101960 0x7fe4fc107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.884+0000 7fe501db2640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4fc102320 0x7fe4fc10ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.884+0000 7fe5015b1640 1 -- 172.21.15.64:0/3995837323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4fc101960 msgr2=0x7fe4fc107b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.884+0000 7fe5015b1640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4fc101960 0x7fe4fc107b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.884+0000 7fe5015b1640 1 -- 172.21.15.64:0/3995837323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4fc102320 msgr2=0x7fe4fc10ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.884+0000 7fe5015b1640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4fc102320 0x7fe4fc10ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.884+0000 7fe5015b1640 1 -- 172.21.15.64:0/3995837323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc19d570 con 0x7fe4fc100770 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.885+0000 7fe5015b1640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc107640 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80314e0 tx=0x7fe4f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.885+0000 7fe4f27fc640 1 -- 172.21.15.64:0/3995837323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f803f070 con 0x7fe4fc100770 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.885+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc19d800 con 0x7fe4fc100770 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.885+0000 7fe4f27fc640 1 -- 172.21.15.64:0/3995837323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4f8031c40 con 0x7fe4fc100770 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.885+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc19dcf0 con 0x7fe4fc100770 2024-05-09T09:23:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.885+0000 7fe4f27fc640 1 -- 172.21.15.64:0/3995837323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8035030 con 0x7fe4fc100770 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 Processor -- start 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 -- start start 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af32930 0x56245af37430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56245ae96c40 con 0x56245af31f40 2024-05-09T09:23:41.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56245ae3e580 con 0x56245af32930 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.889+0000 7fe506604740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56245ae92c00 con 0x56245af37aa0 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.890+0000 7fe4f27fc640 1 -- 172.21.15.64:0/3995837323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe4f8042070 con 0x7fe4fc100770 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.890+0000 7fe501db2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.890+0000 7fe501db2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af32360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46762/0 (socket says 172.21.15.64:46762) 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.890+0000 7fe501db2640 1 -- 172.21.15.64:0/1027483772 learned_addr learned my addr 172.21.15.64:0/1027483772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe501db2640 1 -- 172.21.15.64:0/1027483772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 msgr2=0x56245af39f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe501db2640 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe501db2640 1 -- 172.21.15.64:0/1027483772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af32930 msgr2=0x56245af37430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe501db2640 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af32930 0x56245af37430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe4f27fc640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe4e80a2d60 0x7fe4e80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe501db2640 1 -- 172.21.15.64:0/1027483772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245abaa3f0 con 0x56245af31f40 2024-05-09T09:23:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.891+0000 7fe4f27fc640 1 -- 172.21.15.64:0/3995837323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe4f80efa10 con 0x7fe4fc100770 2024-05-09T09:23:41.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.892+0000 7fe501db2640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe4e80a2d60 0x7fe4e80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.894+0000 7fe501db2640 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af32360 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009fc0 tx=0x7fe4f4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e39fce4c9348c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.896+0000 7fe501db2640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe4e80a2d60 0x7fe4e80a5220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4021000 tx=0x7fe4f4022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.897+0000 7fe4e77fe640 1 -- 172.21.15.64:0/1027483772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f401d8f0 con 0x56245af31f40 2024-05-09T09:23:41.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.897+0000 7fe4e77fe640 1 -- 172.21.15.64:0/1027483772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4f401c420 con 0x56245af31f40 2024-05-09T09:23:41.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.897+0000 7fe4e77fe640 1 -- 172.21.15.64:0/1027483772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f401b610 con 0x56245af31f40 2024-05-09T09:23:41.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.898+0000 7fe506604740 1 -- 172.21.15.64:0/1027483772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 msgr2=0x56245af32360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.898+0000 7fe506604740 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af32360 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009fc0 tx=0x7fe4f4010860 comp rx=0 tx=0).stop 2024-05-09T09:23:41.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.899+0000 7fe506604740 1 -- 172.21.15.64:0/1027483772 shutdown_connections 2024-05-09T09:23:41.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.899+0000 7fe506604740 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.899+0000 7fe506604740 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af32930 0x56245af37430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.899+0000 7fe506604740 1 --2- 172.21.15.64:0/1027483772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af32360 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.899+0000 7fe506604740 1 -- 172.21.15.64:0/1027483772 >> 172.21.15.64:0/1027483772 conn(0x56245af2d5b0 msgr2=0x56245af2d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.902+0000 7fe506604740 1 -- 172.21.15.64:0/1027483772 shutdown_connections 2024-05-09T09:23:41.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.902+0000 7fe506604740 1 -- 172.21.15.64:0/1027483772 wait complete. 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.903+0000 7fe506604740 1 -- start start 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.903+0000 7fe506604740 1 Processor -- start 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.904+0000 7fe506604740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe506604740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af32930 0x56245af3c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe506604740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af43980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe506604740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56245af3aa50 con 0x56245af31f40 2024-05-09T09:23:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe506604740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56245ae51c40 con 0x56245af37aa0 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe506604740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56245af3abd0 con 0x56245af32930 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe506604740 10 client.0 _ll_register_callbacks cb 0x56245aba9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af43980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af43980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57244/0 (socket says 172.21.15.64:57244) 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 -- 172.21.15.64:0/698817159 learned_addr learned my addr 172.21.15.64:0/698817159 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 -- 172.21.15.64:0/698817159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af32930 msgr2=0x56245af3c940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af32930 0x56245af3c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 -- 172.21.15.64:0/698817159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 msgr2=0x56245af3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.905+0000 7fe501db2640 1 -- 172.21.15.64:0/698817159 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245ac747d0 con 0x56245af37aa0 2024-05-09T09:23:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.912+0000 7fe501db2640 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af43980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40092d0 tx=0x7fe4f4011500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b91b372c71c870 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.912+0000 7fe4e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.913+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/698817159 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4037060 con 0x56245af37aa0 2024-05-09T09:23:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.913+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/698817159 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe4f401d8f0 con 0x56245af37aa0 2024-05-09T09:23:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.913+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f401c420 con 0x56245af37aa0 2024-05-09T09:23:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.915+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/698817159 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe4f40115a0 con 0x56245af37aa0 2024-05-09T09:23:41.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.917+0000 7fe506604740 10 client.28430 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:41.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.917+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56245af46cf0 con 0x56245af37aa0 2024-05-09T09:23:41.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.919+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/698817159 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4f401c420 con 0x56245af37aa0 2024-05-09T09:23:41.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.919+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/698817159 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f4032c10 con 0x56245af37aa0 2024-05-09T09:23:41.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.920+0000 7fe506604740 10 client.28430 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:41.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.920+0000 7fe506604740 10 client.28430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:41.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.920+0000 7fe506604740 20 client.28430 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:41.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.920+0000 7fe506604740 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245af482a0 0x56245af4a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.920+0000 7fe506604740 4 client.28430 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:41.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.920+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56245aea4370 con 0x56245af482a0 2024-05-09T09:23:41.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.923+0000 7fe5015b1640 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245af482a0 0x56245af4a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.924+0000 7fe5015b1640 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245af482a0 0x56245af4a6e0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0a4fa4d7bd348e server_cookie=ba72162e46570161 in_seq=0 out_seq=0 2024-05-09T09:23:41.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.924+0000 7fe4e4ff9640 10 client.28430 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:41.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.925+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/698817159 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56245aea4370 con 0x56245af482a0 2024-05-09T09:23:41.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.925+0000 7fe4e4ff9640 10 client.28430 handle_command_reply: tid=0 2024-05-09T09:23:41.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.925+0000 7fe506604740 1 client.28430 shutdown 2024-05-09T09:23:41.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.926+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 msgr2=0x56245af43980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.926+0000 7fe506604740 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af43980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40092d0 tx=0x7fe4f4011500 comp rx=0 tx=0).stop 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.927+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 shutdown_connections 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.927+0000 7fe506604740 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245af482a0 0x56245af4a6e0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.927+0000 7fe506604740 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af43980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.928+0000 7fe506604740 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af32930 0x56245af3c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.928+0000 7fe506604740 1 --2- 172.21.15.64:0/698817159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af31f40 0x56245af3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.928+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 >> 172.21.15.64:0/698817159 conn(0x56245af2d5b0 msgr2=0x56245af2c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.928+0000 7fe4e4ff9640 0 client.28430 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.929+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 shutdown_connections 2024-05-09T09:23:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.930+0000 7fe506604740 1 -- 172.21.15.64:0/698817159 wait complete. 2024-05-09T09:23:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.931+0000 7fe506604740 20 client.28430 trim_cache size 0 max 16384 2024-05-09T09:23:41.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.966+0000 7fe506604740 1 Processor -- start 2024-05-09T09:23:41.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.966+0000 7fe506604740 1 -- start start 2024-05-09T09:23:41.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe506604740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af31f40 0x56245af4a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe506604740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af43940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe506604740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe500db0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af43940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe5015b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af31f40 0x56245af4a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe500db0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af43940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46780/0 (socket says 172.21.15.64:46780) 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe5015b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af31f40 0x56245af4a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50076/0 (socket says 172.21.15.64:50076) 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe500db0640 1 -- 172.21.15.64:0/2202372352 learned_addr learned my addr 172.21.15.64:0/2202372352 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e00125d0 con 0x56245af32930 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e000fe40 con 0x56245af37aa0 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.967+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e000dc90 con 0x56245af31f40 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe501db2640 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe500db0640 1 -- 172.21.15.64:0/2202372352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af31f40 msgr2=0x56245af4a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe500db0640 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af31f40 0x56245af4a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe500db0640 1 -- 172.21.15.64:0/2202372352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 msgr2=0x56245af30560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe500db0640 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af30560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe500db0640 1 -- 172.21.15.64:0/2202372352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245af1ed20 con 0x56245af32930 2024-05-09T09:23:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.968+0000 7fe500db0640 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af43940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009bf0 tx=0x7fe4ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817080f5fc85796d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.969+0000 7fe4e6ffd640 1 -- 172.21.15.64:0/2202372352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec005020 con 0x56245af32930 2024-05-09T09:23:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.969+0000 7fe4e6ffd640 1 -- 172.21.15.64:0/2202372352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4ec0047a0 con 0x56245af32930 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.970+0000 7fe4e6ffd640 1 -- 172.21.15.64:0/2202372352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec019690 con 0x56245af32930 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.970+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 msgr2=0x56245af43940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.970+0000 7fe506604740 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af43940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009bf0 tx=0x7fe4ec011810 comp rx=0 tx=0).stop 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 shutdown_connections 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af37aa0 0x56245af30560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af43940 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 --2- 172.21.15.64:0/2202372352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af31f40 0x56245af4a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 >> 172.21.15.64:0/2202372352 conn(0x56245af2d5b0 msgr2=0x56245af33d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 shutdown_connections 2024-05-09T09:23:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 -- 172.21.15.64:0/2202372352 wait complete. 2024-05-09T09:23:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 -- start start 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.971+0000 7fe506604740 1 Processor -- start 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af31f40 0x56245af41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e000e300 con 0x56245af32930 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e000f770 con 0x56245af31f40 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e000e9d0 con 0x56245af37aa0 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe506604740 10 client.0 _ll_register_callbacks cb 0x7fe4f4020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe501db2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe501db2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50090/0 (socket says 172.21.15.64:50090) 2024-05-09T09:23:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe501db2640 1 -- 172.21.15.64:0/2871520392 learned_addr learned my addr 172.21.15.64:0/2871520392 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.972+0000 7fe5015b1640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af31f40 0x56245af41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe500db0640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe501db2640 1 -- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af31f40 msgr2=0x56245af41480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe501db2640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af31f40 0x56245af41480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe501db2640 1 -- 172.21.15.64:0/2871520392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 msgr2=0x56245af419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe501db2640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe501db2640 1 -- 172.21.15.64:0/2871520392 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245af42fc0 con 0x56245af37aa0 2024-05-09T09:23:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe501db2640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af429d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40092a0 tx=0x7fe4f4028820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4889a1e7e5a3d611 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe4e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/2871520392 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f404a060 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.973+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ec0047a0 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/2871520392 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe4f401da00 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/2871520392 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe4f4051050 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe506604740 10 client.28621 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56245afbfc90 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/2871520392 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4ec0047a0 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/2871520392 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f401dc00 con 0x56245af37aa0 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe506604740 10 client.28621 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe506604740 10 client.28621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.974+0000 7fe506604740 20 client.28621 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.975+0000 7fe506604740 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245afc0670 0x56245afc2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.975+0000 7fe506604740 4 client.28621 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:23:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.975+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56245ae9e6c0 con 0x56245afc0670 2024-05-09T09:23:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.975+0000 7fe5015b1640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245afc0670 0x56245afc2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.977+0000 7fe5015b1640 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245afc0670 0x56245afc2a90 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab3007e9100a62b server_cookie=cc995a501d8e6d7b in_seq=0 out_seq=0 2024-05-09T09:23:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.977+0000 7fe4e4ff9640 10 client.28621 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.977+0000 7fe4e4ff9640 1 -- 172.21.15.64:0/2871520392 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56245ae9e6c0 con 0x56245afc0670 2024-05-09T09:23:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.977+0000 7fe4e4ff9640 10 client.28621 handle_command_reply: tid=0 2024-05-09T09:23:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.977+0000 7fe506604740 1 client.28621 shutdown 2024-05-09T09:23:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.980+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 msgr2=0x56245af429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.980+0000 7fe506604740 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af429d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40092a0 tx=0x7fe4f4028820 comp rx=0 tx=0).stop 2024-05-09T09:23:41.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 shutdown_connections 2024-05-09T09:23:41.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe506604740 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56245afc0670 0x56245afc2a90 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe506604740 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56245af37aa0 0x56245af429d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe506604740 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56245af32930 0x56245af419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe506604740 1 --2- 172.21.15.64:0/2871520392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56245af31f40 0x56245af41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 >> 172.21.15.64:0/2871520392 conn(0x56245af2d5b0 msgr2=0x56245af30560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.982+0000 7fe4e4ff9640 0 client.28621 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:41.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.985+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 shutdown_connections 2024-05-09T09:23:41.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.985+0000 7fe506604740 1 -- 172.21.15.64:0/2871520392 wait complete. 2024-05-09T09:23:41.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.987+0000 7fe506604740 20 client.28621 trim_cache size 0 max 16384 2024-05-09T09:23:41.988 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:23:41.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.990+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe4e80a2d60 msgr2=0x7fe4e80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.990+0000 7fe503815640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe4e80a2d60 0x7fe4e80a5220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4021000 tx=0x7fe4f4022000 comp rx=0 tx=0).stop 2024-05-09T09:23:41.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.990+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 msgr2=0x7fe4fc107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:41.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.990+0000 7fe503815640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc107640 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80314e0 tx=0x7fe4f8003c10 comp rx=0 tx=0).stop 2024-05-09T09:23:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.993+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 shutdown_connections 2024-05-09T09:23:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.993+0000 7fe503815640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe4e80a2d60 0x7fe4e80a5220 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.993+0000 7fe503815640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4fc102320 0x7fe4fc10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.994+0000 7fe503815640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4fc101960 0x7fe4fc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.994+0000 7fe503815640 1 --2- 172.21.15.64:0/3995837323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4fc100770 0x7fe4fc107640 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.994+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 >> 172.21.15.64:0/3995837323 conn(0x7fe4fc0fbce0 msgr2=0x7fe4fc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.997+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 shutdown_connections 2024-05-09T09:23:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:41.997+0000 7fe503815640 1 -- 172.21.15.64:0/3995837323 wait complete. 2024-05-09T09:23:42.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:41 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/466709127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:42.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:41 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/466709127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:42.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:41 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/466709127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:23:43.023 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:23:43.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:23:43.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:23:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 -- 172.21.15.64:0/2011692255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009c370 msgr2=0x7fb55009c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2011692255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009c370 0x7fb55009c790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb554009900 tx=0x7fb55402a340 comp rx=0 tx=0).stop 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 -- 172.21.15.64:0/2011692255 shutdown_connections 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2011692255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009df20 0x7fb5500a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2011692255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb55009d560 0x7fb55009d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2011692255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009c370 0x7fb55009c790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.217+0000 7fb55f10e640 1 -- 172.21.15.64:0/2011692255 >> 172.21.15.64:0/2011692255 conn(0x7fb55000b310 msgr2=0x7fb55000b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.218+0000 7fb55f10e640 1 -- 172.21.15.64:0/2011692255 shutdown_connections 2024-05-09T09:23:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.218+0000 7fb55f10e640 1 -- 172.21.15.64:0/2011692255 wait complete. 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.219+0000 7fb55f10e640 1 Processor -- start 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.219+0000 7fb55f10e640 1 -- start start 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.219+0000 7fb55f10e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 0x7fb5501325c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55f10e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009d560 0x7fb550132b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 0x7fb5501325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 0x7fb5501325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46810/0 (socket says 172.21.15.64:46810) 2024-05-09T09:23:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55e10c640 1 -- 172.21.15.64:0/2544487665 learned_addr learned my addr 172.21.15.64:0/2544487665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb55009df20 0x7fb550139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500a7f00 con 0x7fb55009c370 2024-05-09T09:23:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500a7d80 con 0x7fb55009df20 2024-05-09T09:23:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500a8080 con 0x7fb55009d560 2024-05-09T09:23:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.220+0000 7fb55e90d640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb55009df20 0x7fb550139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.221+0000 7fb55d90b640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009d560 0x7fb550132b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.221+0000 7fb55e10c640 1 -- 172.21.15.64:0/2544487665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009d560 msgr2=0x7fb550132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.221+0000 7fb55e10c640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009d560 0x7fb550132b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.221+0000 7fb55e10c640 1 -- 172.21.15.64:0/2544487665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb55009df20 msgr2=0x7fb550139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.221+0000 7fb55e10c640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb55009df20 0x7fb550139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.221+0000 7fb55e10c640 1 -- 172.21.15.64:0/2544487665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55013a1e0 con 0x7fb55009c370 2024-05-09T09:23:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.222+0000 7fb55e10c640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 0x7fb5501325c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb554034040 tx=0x7fb554032960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.222+0000 7fb54f7fe640 1 -- 172.21.15.64:0/2544487665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554040020 con 0x7fb55009c370 2024-05-09T09:23:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.222+0000 7fb54f7fe640 1 -- 172.21.15.64:0/2544487665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb554003aa0 con 0x7fb55009c370 2024-05-09T09:23:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.222+0000 7fb54f7fe640 1 -- 172.21.15.64:0/2544487665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55403cbe0 con 0x7fb55009c370 2024-05-09T09:23:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.222+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55013a470 con 0x7fb55009c370 2024-05-09T09:23:43.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.222+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55013a8b0 con 0x7fb55009c370 2024-05-09T09:23:43.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.226+0000 7fb54f7fe640 1 -- 172.21.15.64:0/2544487665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb554043020 con 0x7fb55009c370 2024-05-09T09:23:43.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.228+0000 7fb54f7fe640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5440a2d90 0x7fb5440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.228+0000 7fb54f7fe640 1 -- 172.21.15.64:0/2544487665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5540efc40 con 0x7fb55009c370 2024-05-09T09:23:43.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.229+0000 7fb55d90b640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5440a2d90 0x7fb5440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:42 smithi179 ceph-mon[34086]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 734 op/s 2024-05-09T09:23:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.230+0000 7fb563083740 1 Processor -- start 2024-05-09T09:23:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.230+0000 7fb55d90b640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5440a2d90 0x7fb5440a5250 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb550133ad0 tx=0x7fb54800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.230+0000 7fb563083740 1 -- start start 2024-05-09T09:23:43.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb563083740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f06b0 0x560b115f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb563083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b115f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb563083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b115f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb55d90b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b115f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb55d90b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b115f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50118/0 (socket says 172.21.15.64:50118) 2024-05-09T09:23:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb55e90d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b115f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb55e90d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b115f86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57280/0 (socket says 172.21.15.64:57280) 2024-05-09T09:23:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.231+0000 7fb55d90b640 1 -- 172.21.15.64:0/94050004 learned_addr learned my addr 172.21.15.64:0/94050004 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b11554990 con 0x560b115f06b0 2024-05-09T09:23:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560b114fc5b0 con 0x560b115f6210 2024-05-09T09:23:43.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560b11550950 con 0x560b115f10a0 2024-05-09T09:23:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb55d90b640 1 -- 172.21.15.64:0/94050004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 msgr2=0x560b115f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb55d90b640 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b115f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb55d90b640 1 -- 172.21.15.64:0/94050004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f06b0 msgr2=0x560b115f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb55d90b640 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f06b0 0x560b115f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb55d90b640 1 -- 172.21.15.64:0/94050004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b113307e0 con 0x560b115f10a0 2024-05-09T09:23:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.232+0000 7fb55d90b640 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b115f5ba0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b6e0 tx=0x7fb548013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63928e96c78181a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.233+0000 7fb54cbfe640 1 -- 172.21.15.64:0/94050004 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548005220 con 0x560b115f10a0 2024-05-09T09:23:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.233+0000 7fb54cbfe640 1 -- 172.21.15.64:0/94050004 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb548011e50 con 0x560b115f10a0 2024-05-09T09:23:43.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.233+0000 7fb54cbfe640 1 -- 172.21.15.64:0/94050004 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54801e7e0 con 0x560b115f10a0 2024-05-09T09:23:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.234+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 msgr2=0x560b115f5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.234+0000 7fb563083740 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b115f5ba0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b6e0 tx=0x7fb548013000 comp rx=0 tx=0).stop 2024-05-09T09:23:43.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 shutdown_connections 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b115f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b115f5ba0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 --2- 172.21.15.64:0/94050004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f06b0 0x560b115f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 >> 172.21.15.64:0/94050004 conn(0x560b115b80a0 msgr2=0x560b115ec130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 shutdown_connections 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 -- 172.21.15.64:0/94050004 wait complete. 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.235+0000 7fb563083740 1 -- start start 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 Processor -- start 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b1160b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b1160bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b11612f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b115f91c0 con 0x560b115f10a0 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560b1150fb40 con 0x560b115f6210 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560b115f9340 con 0x560b115f06b0 2024-05-09T09:23:43.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb563083740 10 client.0 _ll_register_callbacks cb 0x560b115040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b1160b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b1160b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50120/0 (socket says 172.21.15.64:50120) 2024-05-09T09:23:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.236+0000 7fb55e10c640 1 -- 172.21.15.64:0/3762380566 learned_addr learned my addr 172.21.15.64:0/3762380566 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.238+0000 7fb55e90d640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b11612f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.238+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b1160bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.239+0000 7fb55e10c640 1 -- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 msgr2=0x560b11612f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.239+0000 7fb55e10c640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b11612f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.239+0000 7fb55e10c640 1 -- 172.21.15.64:0/3762380566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 msgr2=0x560b1160bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.239+0000 7fb55e10c640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b1160bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.239+0000 7fb55e10c640 1 -- 172.21.15.64:0/3762380566 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b11390f70 con 0x560b115f06b0 2024-05-09T09:23:43.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.241+0000 7fb55e10c640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b1160b9f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb5540b8480 tx=0x7fb554009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f0f132cf159782 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.241+0000 7fb541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:43.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.242+0000 7fb541ffb640 1 -- 172.21.15.64:0/3762380566 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540d5060 con 0x560b115f06b0 2024-05-09T09:23:43.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.242+0000 7fb541ffb640 1 -- 172.21.15.64:0/3762380566 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5540c00d0 con 0x560b115f06b0 2024-05-09T09:23:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.245+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb548011e50 con 0x560b115f06b0 2024-05-09T09:23:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.245+0000 7fb541ffb640 1 -- 172.21.15.64:0/3762380566 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb5540ba1f0 con 0x560b115f06b0 2024-05-09T09:23:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.246+0000 7fb563083740 10 client.28633 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.246+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560b11228170 con 0x560b115f06b0 2024-05-09T09:23:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb541ffb640 1 -- 172.21.15.64:0/3762380566 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb548011e50 con 0x560b115f06b0 2024-05-09T09:23:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb541ffb640 1 -- 172.21.15.64:0/3762380566 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fb554003c50 con 0x560b115f06b0 2024-05-09T09:23:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb563083740 10 client.28633 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb563083740 10 client.28633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb563083740 20 client.28633 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb563083740 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b115fe110 0x560b11600570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb563083740 4 client.28633 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.247+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b115620c0 con 0x560b115fe110 2024-05-09T09:23:43.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.249+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b115fe110 0x560b11600570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.252+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b115fe110 0x560b11600570 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96a1dc965ed8420 server_cookie=40a4955ee803902 in_seq=0 out_seq=0 2024-05-09T09:23:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.252+0000 7fb541ffb640 10 client.28633 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.253+0000 7fb541ffb640 1 -- 172.21.15.64:0/3762380566 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b115620c0 con 0x560b115fe110 2024-05-09T09:23:43.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.253+0000 7fb541ffb640 10 client.28633 handle_command_reply: tid=0 2024-05-09T09:23:43.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.253+0000 7fb563083740 1 client.28633 shutdown 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.256+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 msgr2=0x560b1160b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.256+0000 7fb563083740 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b1160b9f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb5540b8480 tx=0x7fb554009670 comp rx=0 tx=0).stop 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 shutdown_connections 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b115fe110 0x560b11600570 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b11612f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b1160bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 --2- 172.21.15.64:0/3762380566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b1160b9f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 >> 172.21.15.64:0/3762380566 conn(0x560b115b80a0 msgr2=0x560b115eb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb541ffb640 0 client.28633 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 shutdown_connections 2024-05-09T09:23:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.257+0000 7fb563083740 1 -- 172.21.15.64:0/3762380566 wait complete. 2024-05-09T09:23:43.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.258+0000 7fb563083740 20 client.28633 trim_cache size 0 max 16384 2024-05-09T09:23:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.293+0000 7fb563083740 1 Processor -- start 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb563083740 1 -- start start 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb563083740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 0x560b115ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb563083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b11612f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 0x560b115ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 0x560b115ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57290/0 (socket says 172.21.15.64:57290) 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb55e10c640 1 -- 172.21.15.64:0/3246314407 learned_addr learned my addr 172.21.15.64:0/3246314407 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.294+0000 7fb563083740 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f6210 0x560b115ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb550147880 con 0x560b115f6210 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5501450f0 con 0x560b115f06b0 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb550142f40 con 0x560b115f10a0 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b11612f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb55e90d640 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f6210 0x560b115ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb55d90b640 1 -- 172.21.15.64:0/3246314407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 msgr2=0x560b115ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 0x560b115ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb55d90b640 1 -- 172.21.15.64:0/3246314407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f6210 msgr2=0x560b115ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.295+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f6210 0x560b115ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.296+0000 7fb55d90b640 1 -- 172.21.15.64:0/3246314407 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b115f5630 con 0x560b115f10a0 2024-05-09T09:23:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.296+0000 7fb55e10c640 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 0x560b115ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.296+0000 7fb55d90b640 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b11612f30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b9c0 tx=0x7fb548010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb26beec2d1a1ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.296+0000 7fb543fff640 1 -- 172.21.15.64:0/3246314407 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54803d070 con 0x560b115f10a0 2024-05-09T09:23:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.296+0000 7fb543fff640 1 -- 172.21.15.64:0/3246314407 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb548011e50 con 0x560b115f10a0 2024-05-09T09:23:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.296+0000 7fb543fff640 1 -- 172.21.15.64:0/3246314407 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548037070 con 0x560b115f10a0 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.297+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 msgr2=0x560b11612f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.298+0000 7fb563083740 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b11612f30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b9c0 tx=0x7fb548010fd0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.298+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 shutdown_connections 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.298+0000 7fb563083740 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f6210 0x560b115ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.298+0000 7fb563083740 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f10a0 0x560b11612f30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.298+0000 7fb563083740 1 --2- 172.21.15.64:0/3246314407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f06b0 0x560b115ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.298+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 >> 172.21.15.64:0/3246314407 conn(0x560b115b80a0 msgr2=0x560b115ebbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.300+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 shutdown_connections 2024-05-09T09:23:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.301+0000 7fb563083740 1 -- 172.21.15.64:0/3246314407 wait complete. 2024-05-09T09:23:43.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.302+0000 7fb563083740 1 -- start start 2024-05-09T09:23:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.302+0000 7fb563083740 1 Processor -- start 2024-05-09T09:23:43.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.303+0000 7fb563083740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b11612070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.303+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b11612070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.303+0000 7fb55e10c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b11612070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50140/0 (socket says 172.21.15.64:50140) 2024-05-09T09:23:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.303+0000 7fb55e10c640 1 -- 172.21.15.64:0/2841507133 learned_addr learned my addr 172.21.15.64:0/2841507133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.303+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b116125b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.304+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b1168d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.304+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5501435b0 con 0x560b115f10a0 2024-05-09T09:23:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.304+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb550144a20 con 0x560b115f6210 2024-05-09T09:23:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.304+0000 7fb55e90d640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b1168d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.304+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb550143c80 con 0x560b115f06b0 2024-05-09T09:23:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.304+0000 7fb563083740 10 client.0 _ll_register_callbacks cb 0x7fb554032ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.305+0000 7fb55d90b640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b116125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.305+0000 7fb55e10c640 1 -- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 msgr2=0x560b1168d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.305+0000 7fb55e10c640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b1168d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.305+0000 7fb55e10c640 1 -- 172.21.15.64:0/2841507133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 msgr2=0x560b116125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.305+0000 7fb55e10c640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b116125b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.305+0000 7fb55e10c640 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1168d8f0 con 0x560b115f06b0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.306+0000 7fb55e10c640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b11612070 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb5540c4a60 tx=0x7fb5540c0300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1455a09631edc584 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.306+0000 7fb541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.306+0000 7fb541ffb640 1 -- 172.21.15.64:0/2841507133 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540d5040 con 0x560b115f06b0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.306+0000 7fb541ffb640 1 -- 172.21.15.64:0/2841507133 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb5540035f0 con 0x560b115f06b0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.306+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb548011e50 con 0x560b115f06b0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.306+0000 7fb541ffb640 1 -- 172.21.15.64:0/2841507133 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb5540c9080 con 0x560b115f06b0 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.307+0000 7fb563083740 10 client.28639 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.307+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560b1168ecd0 con 0x560b115f06b0 2024-05-09T09:23:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.307+0000 7fb541ffb640 1 -- 172.21.15.64:0/2841507133 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb548011e50 con 0x560b115f06b0 2024-05-09T09:23:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.308+0000 7fb541ffb640 1 -- 172.21.15.64:0/2841507133 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fb5540d01a0 con 0x560b115f06b0 2024-05-09T09:23:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.309+0000 7fb563083740 10 client.28639 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.309+0000 7fb563083740 10 client.28639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.309+0000 7fb563083740 20 client.28639 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.309+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b1168f4c0 0x560b116918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.310+0000 7fb563083740 4 client.28639 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:23:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.310+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b1155c410 con 0x560b1168f4c0 2024-05-09T09:23:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.310+0000 7fb55d90b640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b1168f4c0 0x560b116918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.311+0000 7fb55d90b640 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b1168f4c0 0x560b116918e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93429735a4a2788f server_cookie=4585750de37b0c4f in_seq=0 out_seq=0 2024-05-09T09:23:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.311+0000 7fb541ffb640 10 client.28639 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.313+0000 7fb541ffb640 1 -- 172.21.15.64:0/2841507133 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b1155c410 con 0x560b1168f4c0 2024-05-09T09:23:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.313+0000 7fb541ffb640 10 client.28639 handle_command_reply: tid=0 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.313+0000 7fb563083740 1 client.28639 shutdown 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.313+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 msgr2=0x560b11612070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.313+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b11612070 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb5540c4a60 tx=0x7fb5540c0300 comp rx=0 tx=0).stop 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 shutdown_connections 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x560b1168f4c0 0x560b116918e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b115f6210 0x560b1168d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b115f10a0 0x560b116125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb563083740 1 --2- 172.21.15.64:0/2841507133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b115f06b0 0x560b11612070 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 >> 172.21.15.64:0/2841507133 conn(0x560b115b80a0 msgr2=0x560b115ef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.314+0000 7fb541ffb640 0 client.28639 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.315+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 shutdown_connections 2024-05-09T09:23:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.315+0000 7fb563083740 1 -- 172.21.15.64:0/2841507133 wait complete. 2024-05-09T09:23:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.315+0000 7fb563083740 20 client.28639 trim_cache size 0 max 16384 2024-05-09T09:23:43.318 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:23:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:23:43.318 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "69f73b96-5f68-4a22-a8a2-08ea4ecd6ed0", 2024-05-09T09:23:43.319 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:23:43.319 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:23:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.319+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5440a2d90 msgr2=0x7fb5440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.319+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5440a2d90 0x7fb5440a5250 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb550133ad0 tx=0x7fb54800b390 comp rx=0 tx=0).stop 2024-05-09T09:23:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.319+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 msgr2=0x7fb5501325c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.319+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 0x7fb5501325c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb554034040 tx=0x7fb554032960 comp rx=0 tx=0).stop 2024-05-09T09:23:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.320+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 shutdown_connections 2024-05-09T09:23:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.320+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fb5440a2d90 0x7fb5440a5250 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.320+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb55009df20 0x7fb550139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.320+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb55009d560 0x7fb550132b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.320+0000 7fb55f10e640 1 --2- 172.21.15.64:0/2544487665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb55009c370 0x7fb5501325c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.320+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 >> 172.21.15.64:0/2544487665 conn(0x7fb55000b310 msgr2=0x7fb5500a0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.321+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 shutdown_connections 2024-05-09T09:23:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.321+0000 7fb55f10e640 1 -- 172.21.15.64:0/2544487665 wait complete. 2024-05-09T09:23:43.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:42 smithi139 ceph-mon[33599]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 734 op/s 2024-05-09T09:23:43.343 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:42 smithi064 ceph-mon[30149]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 734 op/s 2024-05-09T09:23:43.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:23:43.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:23:43.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.554+0000 7f6629a74640 1 -- 172.21.15.64:0/1403800899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c096830 msgr2=0x7f661c09d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.554+0000 7f6629a74640 1 --2- 172.21.15.64:0/1403800899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c096830 0x7f661c09d0c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f661800d910 tx=0x7f661802a160 comp rx=0 tx=0).stop 2024-05-09T09:23:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.555+0000 7f6629a74640 1 -- 172.21.15.64:0/1403800899 shutdown_connections 2024-05-09T09:23:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.555+0000 7f6629a74640 1 --2- 172.21.15.64:0/1403800899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c096830 0x7f661c09d0c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.555+0000 7f6629a74640 1 --2- 172.21.15.64:0/1403800899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c095e70 0x7f661c0962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.555+0000 7f6629a74640 1 --2- 172.21.15.64:0/1403800899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661c094c80 0x7f661c0950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.555+0000 7f6629a74640 1 -- 172.21.15.64:0/1403800899 >> 172.21.15.64:0/1403800899 conn(0x7f661c0901d0 msgr2=0x7f661c092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.556+0000 7f6629a74640 1 -- 172.21.15.64:0/1403800899 shutdown_connections 2024-05-09T09:23:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.556+0000 7f6629a74640 1 -- 172.21.15.64:0/1403800899 wait complete. 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.557+0000 7f6629a74640 1 Processor -- start 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.557+0000 7f6629a74640 1 -- start start 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.557+0000 7f6629a74640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661c094c80 0x7f661c13ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.557+0000 7f6629a74640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c095e70 0x7f661c13b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629a74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 0x7f661c142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629a74640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f661c09f650 con 0x7f661c095e70 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629a74640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f661c09f4d0 con 0x7f661c096830 2024-05-09T09:23:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629a74640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f661c09f7d0 con 0x7f661c094c80 2024-05-09T09:23:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629273640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 0x7f661c142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6623fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c095e70 0x7f661c13b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6623fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c095e70 0x7f661c13b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46886/0 (socket says 172.21.15.64:46886) 2024-05-09T09:23:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629273640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 0x7f661c142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57318/0 (socket says 172.21.15.64:57318) 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.558+0000 7f6629273640 1 -- 172.21.15.64:0/3883822291 learned_addr learned my addr 172.21.15.64:0/3883822291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.559+0000 7f6628a72640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661c094c80 0x7f661c13ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.559+0000 7f6629273640 1 -- 172.21.15.64:0/3883822291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661c094c80 msgr2=0x7f661c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.559+0000 7f6629273640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661c094c80 0x7f661c13ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.559+0000 7f6629273640 1 -- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c095e70 msgr2=0x7f661c13b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.559+0000 7f6629273640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c095e70 0x7f661c13b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.559+0000 7f6629273640 1 -- 172.21.15.64:0/3883822291 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c142930 con 0x7f661c096830 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.560+0000 7f6629273640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 0x7f661c142290 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f661800c690 tx=0x7f661800c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.560+0000 7f6621ffb640 1 -- 172.21.15.64:0/3883822291 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661803f020 con 0x7f661c096830 2024-05-09T09:23:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.560+0000 7f6621ffb640 1 -- 172.21.15.64:0/3883822291 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f661800cd10 con 0x7f661c096830 2024-05-09T09:23:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.561+0000 7f6621ffb640 1 -- 172.21.15.64:0/3883822291 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661802c5e0 con 0x7f661c096830 2024-05-09T09:23:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.561+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c142bc0 con 0x7f661c096830 2024-05-09T09:23:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.561+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c143030 con 0x7f661c096830 2024-05-09T09:23:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.565+0000 7f662d9e2740 1 Processor -- start 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f6621ffb640 1 -- 172.21.15.64:0/3883822291 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6618042020 con 0x7f661c096830 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 -- start start 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c73500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c7b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a51bd7d00 con 0x564a51c73ad0 2024-05-09T09:23:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564a51b7f4c0 con 0x564a51c730e0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.568+0000 7f662d9e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a51bd3cc0 con 0x564a51c78c40 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c73500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c73500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57324/0 (socket says 172.21.15.64:57324) 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 -- 172.21.15.64:0/224505820 learned_addr learned my addr 172.21.15.64:0/224505820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c7b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6628a72640 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 -- 172.21.15.64:0/224505820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 msgr2=0x564a51c7b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c7b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 -- 172.21.15.64:0/224505820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 msgr2=0x564a51c785d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c785d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6623fff640 1 -- 172.21.15.64:0/224505820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a519b57d0 con 0x564a51c730e0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.569+0000 7f6628a72640 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c785d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.570+0000 7f6623fff640 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c73500 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6610013880 tx=0x7f6610005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8e662f1f74372d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.570+0000 7f660effd640 1 -- 172.21.15.64:0/224505820 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610013e00 con 0x564a51c730e0 2024-05-09T09:23:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.570+0000 7f660effd640 1 -- 172.21.15.64:0/224505820 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6610013170 con 0x564a51c730e0 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.570+0000 7f660effd640 1 -- 172.21.15.64:0/224505820 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610004d20 con 0x564a51c730e0 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.570+0000 7f6621ffb640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66140a2ab0 0x7f66140a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.570+0000 7f6621ffb640 1 -- 172.21.15.64:0/3883822291 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66180eea50 con 0x7f661c096830 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 -- 172.21.15.64:0/224505820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 msgr2=0x564a51c73500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c73500 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6610013880 tx=0x7f6610005f90 comp rx=0 tx=0).stop 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f6628a72640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66140a2ab0 0x7f66140a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 -- 172.21.15.64:0/224505820 shutdown_connections 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c7b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c785d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 --2- 172.21.15.64:0/224505820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c73500 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 -- 172.21.15.64:0/224505820 >> 172.21.15.64:0/224505820 conn(0x564a51c6e750 msgr2=0x564a51c6eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 -- 172.21.15.64:0/224505820 shutdown_connections 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 -- 172.21.15.64:0/224505820 wait complete. 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f6628a72640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66140a2ab0 0x7f66140a4f70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f662406b220 tx=0x7f66240670e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.571+0000 7f662d9e2740 1 -- start start 2024-05-09T09:23:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 Processor -- start 2024-05-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c84a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a51c7bbf0 con 0x564a51c730e0 2024-05-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564a51b92b90 con 0x564a51c73ad0 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a51c7bd70 con 0x564a51c78c40 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f662d9e2740 10 client.0 _ll_register_callbacks cb 0x564a51bceda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6629273640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c84a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6629273640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c84a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50178/0 (socket says 172.21.15.64:50178) 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6629273640 1 -- 172.21.15.64:0/4031440181 learned_addr learned my addr 172.21.15.64:0/4031440181 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6623fff640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6628a72640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6629273640 1 -- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 msgr2=0x564a51c7da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.572+0000 7f6629273640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c7da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6629273640 1 -- 172.21.15.64:0/4031440181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 msgr2=0x564a51c7d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6629273640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c7d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6629273640 1 -- 172.21.15.64:0/4031440181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a518eb3f0 con 0x564a51c78c40 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6628a72640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6629273640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c84a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f661802d010 tx=0x7f661802a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf087f9691cbb67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6603fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6603fff640 1 -- 172.21.15.64:0/4031440181 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661800d520 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6610013170 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.573+0000 7f6603fff640 1 -- 172.21.15.64:0/4031440181 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f661800c860 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.574+0000 7f6603fff640 1 -- 172.21.15.64:0/4031440181 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f66180bb5c0 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.574+0000 7f662d9e2740 10 client.28648 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.574+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564a518ab0f0 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.574+0000 7f6603fff640 1 -- 172.21.15.64:0/4031440181 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6610013170 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.574+0000 7f6603fff640 1 -- 172.21.15.64:0/4031440181 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f66180d9b00 con 0x564a51c78c40 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.575+0000 7f662d9e2740 10 client.28648 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.575+0000 7f662d9e2740 10 client.28648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.575+0000 7f662d9e2740 20 client.28648 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.575+0000 7f662d9e2740 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51c893b0 0x564a51c8b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.575+0000 7f662d9e2740 4 client.28648 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.575+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a51be5430 con 0x564a51c893b0 2024-05-09T09:23:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.576+0000 7f6628a72640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51c893b0 0x564a51c8b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.576+0000 7f6603fff640 10 client.28648 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.576+0000 7f6628a72640 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51c893b0 0x564a51c8b830 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb99606bb6dc0c6 server_cookie=b62fe9d72e87f5c in_seq=0 out_seq=0 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.577+0000 7f6603fff640 1 -- 172.21.15.64:0/4031440181 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a51be5430 con 0x564a51c893b0 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.577+0000 7f6603fff640 10 client.28648 handle_command_reply: tid=0 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.577+0000 7f662d9e2740 1 client.28648 shutdown 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 msgr2=0x564a51c84a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c84a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f661802d010 tx=0x7f661802a160 comp rx=0 tx=0).stop 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 shutdown_connections 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51c893b0 0x564a51c8b830 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c84a80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c7da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 --2- 172.21.15.64:0/4031440181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 >> 172.21.15.64:0/4031440181 conn(0x564a51c6e750 msgr2=0x564a51c6da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f6603fff640 0 client.28648 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 shutdown_connections 2024-05-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 1 -- 172.21.15.64:0/4031440181 wait complete. 2024-05-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.578+0000 7f662d9e2740 20 client.28648 trim_cache size 0 max 16384 2024-05-09T09:23:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.612+0000 7f662d9e2740 1 Processor -- start 2024-05-09T09:23:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.619+0000 7f662d9e2740 1 -- start start 2024-05-09T09:23:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.619+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c6eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.619+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c84a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f662d9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f661c14fe60 con 0x564a51c73ad0 2024-05-09T09:23:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f662d9e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f661c14d6d0 con 0x564a51c730e0 2024-05-09T09:23:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f662d9e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f661c14b520 con 0x564a51c78c40 2024-05-09T09:23:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f6628a72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c6eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f6628a72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c6eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57352/0 (socket says 172.21.15.64:57352) 2024-05-09T09:23:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f6628a72640 1 -- 172.21.15.64:0/1305008858 learned_addr learned my addr 172.21.15.64:0/1305008858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.620+0000 7f6629273640 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.625+0000 7f6628a72640 1 -- 172.21.15.64:0/1305008858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 msgr2=0x564a51c716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.625+0000 7f6628a72640 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.625+0000 7f6628a72640 1 -- 172.21.15.64:0/1305008858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 msgr2=0x564a51c84a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.625+0000 7f6628a72640 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c84a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.625+0000 7f6628a72640 1 -- 172.21.15.64:0/1305008858 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a51c6e580 con 0x564a51c730e0 2024-05-09T09:23:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.626+0000 7f6628a72640 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c6eb30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6624050ef0 tx=0x7f6624069050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b0db64dd65d70f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.629+0000 7f660e7fc640 1 -- 172.21.15.64:0/1305008858 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624093070 con 0x564a51c730e0 2024-05-09T09:23:43.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.629+0000 7f660e7fc640 1 -- 172.21.15.64:0/1305008858 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f662408e850 con 0x564a51c730e0 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.630+0000 7f662d9e2740 1 -- 172.21.15.64:0/1305008858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 msgr2=0x564a51c6eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.630+0000 7f662d9e2740 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c6eb30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6624050ef0 tx=0x7f6624069050 comp rx=0 tx=0).stop 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.631+0000 7f662d9e2740 1 -- 172.21.15.64:0/1305008858 shutdown_connections 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.631+0000 7f662d9e2740 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.631+0000 7f662d9e2740 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c73ad0 0x564a51c84a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.631+0000 7f662d9e2740 1 --2- 172.21.15.64:0/1305008858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c730e0 0x564a51c6eb30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.631+0000 7f662d9e2740 1 -- 172.21.15.64:0/1305008858 >> 172.21.15.64:0/1305008858 conn(0x564a51c6e750 msgr2=0x564a51c74fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.632+0000 7f662d9e2740 1 -- 172.21.15.64:0/1305008858 shutdown_connections 2024-05-09T09:23:43.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.632+0000 7f662d9e2740 1 -- 172.21.15.64:0/1305008858 wait complete. 2024-05-09T09:23:43.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 -- start start 2024-05-09T09:23:43.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 Processor -- start 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c82190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f661c14bb90 con 0x564a51c730e0 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f661c14d000 con 0x564a51c73ad0 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f661c14c260 con 0x564a51c78c40 2024-05-09T09:23:43.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.633+0000 7f662d9e2740 10 client.0 _ll_register_callbacks cb 0x7f661802cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:43.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.634+0000 7f6629273640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.634+0000 7f6629273640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c836e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50192/0 (socket says 172.21.15.64:50192) 2024-05-09T09:23:43.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.634+0000 7f6629273640 1 -- 172.21.15.64:0/226026351 learned_addr learned my addr 172.21.15.64:0/226026351 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:43.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.637+0000 7f6629273640 1 -- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 msgr2=0x564a51c826d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:23:43.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.637+0000 7f6629273640 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c826d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.637+0000 7f6629273640 1 -- 172.21.15.64:0/226026351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 msgr2=0x564a51c82190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:43.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.637+0000 7f6629273640 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c82190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.637+0000 7f6629273640 1 -- 172.21.15.64:0/226026351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a51c83cd0 con 0x564a51c78c40 2024-05-09T09:23:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.639+0000 7f6629273640 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c836e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f66180b6ba0 tx=0x7f661800c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab46029e55a9d5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.639+0000 7f6603fff640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.639+0000 7f6603fff640 1 -- 172.21.15.64:0/226026351 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66180d7070 con 0x564a51c78c40 2024-05-09T09:23:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.640+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662408e850 con 0x564a51c78c40 2024-05-09T09:23:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.640+0000 7f6603fff640 1 -- 172.21.15.64:0/226026351 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f66180ba310 con 0x564a51c78c40 2024-05-09T09:23:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.640+0000 7f6603fff640 1 -- 172.21.15.64:0/226026351 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f66180b70d0 con 0x564a51c78c40 2024-05-09T09:23:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.640+0000 7f662d9e2740 10 client.28654 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.640+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564a51cf8290 con 0x564a51c78c40 2024-05-09T09:23:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.641+0000 7f6603fff640 1 -- 172.21.15.64:0/226026351 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f662408e850 con 0x564a51c78c40 2024-05-09T09:23:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f6603fff640 1 -- 172.21.15.64:0/226026351 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f66180c0400 con 0x564a51c78c40 2024-05-09T09:23:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f662d9e2740 10 client.28654 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f662d9e2740 10 client.28654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f662d9e2740 20 client.28654 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f662d9e2740 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51cf8b80 0x564a51cfafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f662d9e2740 4 client.28654 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:23:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.642+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a51bdf780 con 0x564a51cf8b80 2024-05-09T09:23:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.643+0000 7f6628a72640 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51cf8b80 0x564a51cfafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:43.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.643+0000 7f6628a72640 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51cf8b80 0x564a51cfafc0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f37cda8b45f5ff0 server_cookie=b58891c75c160191 in_seq=0 out_seq=0 2024-05-09T09:23:43.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.645+0000 7f6603fff640 10 client.28654 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.645+0000 7f6603fff640 1 -- 172.21.15.64:0/226026351 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564a51bdf780 con 0x564a51cf8b80 2024-05-09T09:23:43.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.645+0000 7f6603fff640 10 client.28654 handle_command_reply: tid=0 2024-05-09T09:23:43.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.645+0000 7f662d9e2740 1 client.28654 shutdown 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.649+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 msgr2=0x564a51c836e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.649+0000 7f662d9e2740 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c836e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f66180b6ba0 tx=0x7f661800c9f0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.650+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 shutdown_connections 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.650+0000 7f662d9e2740 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x564a51cf8b80 0x564a51cfafc0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.650+0000 7f662d9e2740 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a51c78c40 0x564a51c836e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.650+0000 7f662d9e2740 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564a51c73ad0 0x564a51c826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.650+0000 7f662d9e2740 1 --2- 172.21.15.64:0/226026351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a51c730e0 0x564a51c82190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.650+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 >> 172.21.15.64:0/226026351 conn(0x564a51c6e750 msgr2=0x564a51c6d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.651+0000 7f6603fff640 0 client.28654 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:43.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.651+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 shutdown_connections 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: "69f73b96-5f68-4a22-a8a2-08ea4ecd6ed0": { 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "69f73b96-5f68-4a22-a8a2-08ea4ecd6ed0", 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:23:43.652 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:23:43.653 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.651+0000 7f662d9e2740 1 -- 172.21.15.64:0/226026351 wait complete. 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.651+0000 7f662d9e2740 20 client.28654 trim_cache size 0 max 16384 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.653+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66140a2ab0 msgr2=0x7f66140a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.653+0000 7f6629a74640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66140a2ab0 0x7f66140a4f70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f662406b220 tx=0x7f66240670e0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.653+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 msgr2=0x7f661c142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.653+0000 7f6629a74640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 0x7f661c142290 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f661800c690 tx=0x7f661800c6c0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 shutdown_connections 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f66140a2ab0 0x7f66140a4f70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c096830 0x7f661c142290 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f661c095e70 0x7f661c13b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 --2- 172.21.15.64:0/3883822291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661c094c80 0x7f661c13ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 >> 172.21.15.64:0/3883822291 conn(0x7f661c0901d0 msgr2=0x7f661c099170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 shutdown_connections 2024-05-09T09:23:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:23:43.654+0000 7f6629a74640 1 -- 172.21.15.64:0/3883822291 wait complete. 2024-05-09T09:23:43.675 INFO:tasks.cephfs.filesystem:scrub status for tag:69f73b96-5f68-4a22-a8a2-08ea4ecd6ed0 - {'path': '/', 'tag': '69f73b96-5f68-4a22-a8a2-08ea4ecd6ed0', 'options': 'recursive,force'} 2024-05-09T09:23:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:43 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:23:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:43 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:23:44.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:43 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:43 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:23:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:43 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:23:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:43 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:43 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:23:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:43 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:23:44.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:43 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:44 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:23:45.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:44 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:44 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:23:45.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:44 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:44 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:44 smithi179 ceph-mon[34086]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 844 op/s 2024-05-09T09:23:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:44 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:23:45.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:44 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:44 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:23:45.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:44 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:44 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:44 smithi139 ceph-mon[33599]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 844 op/s 2024-05-09T09:23:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:44 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:23:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:44 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:44 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:23:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:44 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:44 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:23:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:44 smithi064 ceph-mon[30149]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 844 op/s 2024-05-09T09:23:46.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 805.0 s, 726.4 tps, lat 137.375 ms stddev 81.783 2024-05-09T09:23:47.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:46 smithi139 ceph-mon[33599]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 819 op/s 2024-05-09T09:23:47.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:46 smithi064 ceph-mon[30149]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 819 op/s 2024-05-09T09:23:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:46 smithi179 ceph-mon[34086]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 819 op/s 2024-05-09T09:23:49.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:48 smithi139 ceph-mon[33599]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.2 MiB/s wr, 726 op/s 2024-05-09T09:23:49.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:48 smithi064 ceph-mon[30149]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.2 MiB/s wr, 726 op/s 2024-05-09T09:23:49.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:48 smithi179 ceph-mon[34086]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.2 MiB/s wr, 726 op/s 2024-05-09T09:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.228+0000 7f15e7b29640 1 -- 172.21.15.64:0/3035963643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 msgr2=0x7f15e0101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.228+0000 7f15e7b29640 1 --2- 172.21.15.64:0/3035963643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 0x7f15e0101aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009ca0 tx=0x7f15d802a8d0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.230+0000 7f15e7b29640 1 -- 172.21.15.64:0/3035963643 shutdown_connections 2024-05-09T09:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.230+0000 7f15e7b29640 1 --2- 172.21.15.64:0/3035963643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 0x7f15e0101aa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.230+0000 7f15e7b29640 1 --2- 172.21.15.64:0/3035963643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e0100860 0x7f15e0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.230+0000 7f15e7b29640 1 --2- 172.21.15.64:0/3035963643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e00ff730 0x7f15e00ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.230+0000 7f15e7b29640 1 -- 172.21.15.64:0/3035963643 >> 172.21.15.64:0/3035963643 conn(0x7f15e00fb050 msgr2=0x7f15e00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.233+0000 7f15e7b29640 1 -- 172.21.15.64:0/3035963643 shutdown_connections 2024-05-09T09:23:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.234+0000 7f15e7b29640 1 -- 172.21.15.64:0/3035963643 wait complete. 2024-05-09T09:23:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.235+0000 7f15e7b29640 1 Processor -- start 2024-05-09T09:23:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.236+0000 7f15e7b29640 1 -- start start 2024-05-09T09:23:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.236+0000 7f15e7b29640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e00ff730 0x7f15e0110be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.236+0000 7f15e7b29640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 0x7f15e0111120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.236+0000 7f15e7b29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 0x7f15e01adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e7b29640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0104c00 con 0x7f15e0100860 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e7b29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0104a80 con 0x7f15e0101200 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e7b29640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0104d80 con 0x7f15e00ff730 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e58c5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e00ff730 0x7f15e0110be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 0x7f15e0111120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 0x7f15e0111120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38546/0 (socket says 172.21.15.64:38546) 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e58c5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e00ff730 0x7f15e0110be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33472/0 (socket says 172.21.15.64:33472) 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 -- 172.21.15.64:0/2378960709 learned_addr learned my addr 172.21.15.64:0/2378960709 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e60c6640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 0x7f15e01adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 -- 172.21.15.64:0/2378960709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e00ff730 msgr2=0x7f15e0110be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e00ff730 0x7f15e0110be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 -- 172.21.15.64:0/2378960709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 msgr2=0x7f15e01adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 0x7f15e01adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.237+0000 7f15e50c4640 1 -- 172.21.15.64:0/2378960709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e01ae550 con 0x7f15e0100860 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.238+0000 7f15e50c4640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 0x7f15e0111120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b450 tx=0x7f15dc00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.238+0000 7f15d6ffd640 1 -- 172.21.15.64:0/2378960709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc015020 con 0x7f15e0100860 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.238+0000 7f15d6ffd640 1 -- 172.21.15.64:0/2378960709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15dc011ce0 con 0x7f15e0100860 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.238+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e01ae840 con 0x7f15e0100860 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.238+0000 7f15d6ffd640 1 -- 172.21.15.64:0/2378960709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc010e20 con 0x7f15e0100860 2024-05-09T09:23:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.238+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e01aedb0 con 0x7f15e0100860 2024-05-09T09:23:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.243+0000 7f15d6ffd640 1 -- 172.21.15.64:0/2378960709 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15dc018020 con 0x7f15e0100860 2024-05-09T09:23:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.244+0000 7f15d6ffd640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15cc0a2db0 0x7f15cc0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.245+0000 7f15d6ffd640 1 -- 172.21.15.64:0/2378960709 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15dc0cec90 con 0x7f15e0100860 2024-05-09T09:23:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.245+0000 7f15ea83c740 1 Processor -- start 2024-05-09T09:23:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15e58c5640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15cc0a2db0 0x7f15cc0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 -- start start 2024-05-09T09:23:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144250 0x5561ac144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 0x5561ac149740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac149db0 0x5561ac14c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561ac0a8340 con 0x5561ac144250 2024-05-09T09:23:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5561ac050290 con 0x5561ac144c40 2024-05-09T09:23:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15ea83c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5561ac0a4300 con 0x5561ac149db0 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15e58c5640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15cc0a2db0 0x7f15cc0a5270 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009990 tx=0x7f15d0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.246+0000 7f15e50c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 0x5561ac149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 0x5561ac149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47488/0 (socket says 172.21.15.64:47488) 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 -- 172.21.15.64:0/4112393281 learned_addr learned my addr 172.21.15.64:0/4112393281 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e58c5640 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144250 0x5561ac144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 -- 172.21.15.64:0/4112393281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac149db0 msgr2=0x5561ac14c240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac149db0 0x5561ac14c240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 -- 172.21.15.64:0/4112393281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144250 msgr2=0x5561ac144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144250 0x5561ac144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 -- 172.21.15.64:0/4112393281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ac0748e0 con 0x5561ac144c40 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.247+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 0x5561ac149740 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0092f0 tx=0x7f15dc0974d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddefbfa7d66c8ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.248+0000 7f15cbfff640 1 -- 172.21.15.64:0/4112393281 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0b4070 con 0x5561ac144c40 2024-05-09T09:23:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.248+0000 7f15cbfff640 1 -- 172.21.15.64:0/4112393281 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15dc09f4b0 con 0x5561ac144c40 2024-05-09T09:23:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.248+0000 7f15cbfff640 1 -- 172.21.15.64:0/4112393281 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0bb0b0 con 0x5561ac144c40 2024-05-09T09:23:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 -- 172.21.15.64:0/4112393281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 msgr2=0x5561ac149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 0x5561ac149740 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0092f0 tx=0x7f15dc0974d0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 -- 172.21.15.64:0/4112393281 shutdown_connections 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac149db0 0x5561ac14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144c40 0x5561ac149740 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4112393281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144250 0x5561ac144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 -- 172.21.15.64:0/4112393281 >> 172.21.15.64:0/4112393281 conn(0x5561ac13f8c0 msgr2=0x5561ac13fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 -- 172.21.15.64:0/4112393281 shutdown_connections 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.249+0000 7f15ea83c740 1 -- 172.21.15.64:0/4112393281 wait complete. 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 -- start start 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 Processor -- start 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac14e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac14e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561ac14cd60 con 0x5561ac144c40 2024-05-09T09:23:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5561ac063790 con 0x5561ac149db0 2024-05-09T09:23:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5561ac14cee0 con 0x5561ac144250 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.250+0000 7f15ea83c740 10 client.0 _ll_register_callbacks cb 0x5561ac09ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47504/0 (socket says 172.21.15.64:47504) 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 -- 172.21.15.64:0/3755122094 learned_addr learned my addr 172.21.15.64:0/3755122094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e58c5640 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac14e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 -- 172.21.15.64:0/3755122094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 msgr2=0x5561ac14e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac14e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 -- 172.21.15.64:0/3755122094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 msgr2=0x5561ac14e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac14e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 -- 172.21.15.64:0/3755122094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561abdbc210 con 0x5561ac149db0 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15e60c6640 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac155a10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009bf0 tx=0x7f15d8031d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402ad5dd77d3314c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.251+0000 7f15c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.252+0000 7f15c97fa640 1 -- 172.21.15.64:0/3755122094 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8031660 con 0x5561ac149db0 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.252+0000 7f15c97fa640 1 -- 172.21.15.64:0/3755122094 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15d8030a10 con 0x5561ac149db0 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.252+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc09f4b0 con 0x5561ac149db0 2024-05-09T09:23:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.252+0000 7f15c97fa640 1 -- 172.21.15.64:0/3755122094 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f15d8030c10 con 0x5561ac149db0 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15ea83c740 10 client.28478 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5561abe88620 con 0x5561ac149db0 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15c97fa640 1 -- 172.21.15.64:0/3755122094 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15dc09f4b0 con 0x5561ac149db0 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15c97fa640 1 -- 172.21.15.64:0/3755122094 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f15d80042d0 con 0x5561ac149db0 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15ea83c740 10 client.28478 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15ea83c740 10 client.28478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15ea83c740 20 client.28478 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.254+0000 7f15ea83c740 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac16b420 0x5561ac16d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.255+0000 7f15ea83c740 4 client.28478 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:23:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.255+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ac0b5a70 con 0x5561ac16b420 2024-05-09T09:23:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.255+0000 7f15e58c5640 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac16b420 0x5561ac16d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.256+0000 7f15e58c5640 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac16b420 0x5561ac16d8a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a19e17ea9df3614 server_cookie=345bbb3b93544ab4 in_seq=0 out_seq=0 2024-05-09T09:23:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.256+0000 7f15c97fa640 10 client.28478 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.257+0000 7f15c97fa640 1 -- 172.21.15.64:0/3755122094 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561ac0b5a70 con 0x5561ac16b420 2024-05-09T09:23:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.257+0000 7f15c97fa640 10 client.28478 handle_command_reply: tid=0 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.257+0000 7f15ea83c740 1 client.28478 shutdown 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 msgr2=0x5561ac155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac155a10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009bf0 tx=0x7f15d8031d80 comp rx=0 tx=0).stop 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 shutdown_connections 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac16b420 0x5561ac16d8a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac155a10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac14e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 --2- 172.21.15.64:0/3755122094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac14e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 >> 172.21.15.64:0/3755122094 conn(0x5561ac13f8c0 msgr2=0x5561ac13ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15c97fa640 0 client.28478 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 shutdown_connections 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.258+0000 7f15ea83c740 1 -- 172.21.15.64:0/3755122094 wait complete. 2024-05-09T09:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.259+0000 7f15ea83c740 20 client.28478 trim_cache size 0 max 16384 2024-05-09T09:23:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.293+0000 7f15ea83c740 1 Processor -- start 2024-05-09T09:23:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.293+0000 7f15ea83c740 1 -- start start 2024-05-09T09:23:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.293+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac142f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.293+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.293+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac1731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15ea83c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01b3460 con 0x5561ac144c40 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15ea83c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01b0cd0 con 0x5561ac149db0 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15ea83c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0073500 con 0x5561ac144250 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac1731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac1731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47506/0 (socket says 172.21.15.64:47506) 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 -- 172.21.15.64:0/2973143904 learned_addr learned my addr 172.21.15.64:0/2973143904 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e50c4640 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e58c5640 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 -- 172.21.15.64:0/2973143904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 msgr2=0x5561ac142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 -- 172.21.15.64:0/2973143904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 msgr2=0x5561ac1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.294+0000 7f15e60c6640 1 -- 172.21.15.64:0/2973143904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ac16df20 con 0x5561ac149db0 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.295+0000 7f15e60c6640 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac1731b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15d802a910 tx=0x7f15d80093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ddcaaf5ed4b82f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.296+0000 7f15cb7fe640 1 -- 172.21.15.64:0/2973143904 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8030db0 con 0x5561ac149db0 2024-05-09T09:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.296+0000 7f15cb7fe640 1 -- 172.21.15.64:0/2973143904 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15d8036070 con 0x5561ac149db0 2024-05-09T09:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.296+0000 7f15cb7fe640 1 -- 172.21.15.64:0/2973143904 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d804eba0 con 0x5561ac149db0 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.297+0000 7f15ea83c740 1 -- 172.21.15.64:0/2973143904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 msgr2=0x5561ac1731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.297+0000 7f15ea83c740 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac1731b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15d802a910 tx=0x7f15d80093e0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.298+0000 7f15ea83c740 1 -- 172.21.15.64:0/2973143904 shutdown_connections 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.298+0000 7f15ea83c740 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac149db0 0x5561ac1731b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.298+0000 7f15ea83c740 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac144c40 0x5561ac1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.298+0000 7f15ea83c740 1 --2- 172.21.15.64:0/2973143904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144250 0x5561ac142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.298+0000 7f15ea83c740 1 -- 172.21.15.64:0/2973143904 >> 172.21.15.64:0/2973143904 conn(0x5561ac13f8c0 msgr2=0x5561ac142840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.301+0000 7f15ea83c740 1 -- 172.21.15.64:0/2973143904 shutdown_connections 2024-05-09T09:23:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.301+0000 7f15ea83c740 1 -- 172.21.15.64:0/2973143904 wait complete. 2024-05-09T09:23:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.302+0000 7f15ea83c740 1 -- start start 2024-05-09T09:23:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.302+0000 7f15ea83c740 1 Processor -- start 2024-05-09T09:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144250 0x5561ac1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144c40 0x5561ac153700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac149db0 0x5561ac154710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01af190 con 0x5561ac149db0 2024-05-09T09:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01b0600 con 0x5561ac144250 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01af860 con 0x5561ac144c40 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15ea83c740 10 client.0 _ll_register_callbacks cb 0x7f15d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15e58c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144250 0x5561ac1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15e58c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144250 0x5561ac1531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47516/0 (socket says 172.21.15.64:47516) 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15e58c5640 1 -- 172.21.15.64:0/4222614770 learned_addr learned my addr 172.21.15.64:0/4222614770 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144c40 0x5561ac153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.303+0000 7f15e60c6640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac149db0 0x5561ac154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e50c4640 1 -- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144250 msgr2=0x5561ac1531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144250 0x5561ac1531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e50c4640 1 -- 172.21.15.64:0/4222614770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac149db0 msgr2=0x5561ac154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac149db0 0x5561ac154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e50c4640 1 -- 172.21.15.64:0/4222614770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ac154d00 con 0x5561ac144c40 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e60c6640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac149db0 0x5561ac154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15e50c4640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144c40 0x5561ac153700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5561ac14c9b0 tx=0x7f15dc09ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130da39c7caab5ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15c97fa640 1 -- 172.21.15.64:0/4222614770 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0b4040 con 0x5561ac144c40 2024-05-09T09:23:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.305+0000 7f15c97fa640 1 -- 172.21.15.64:0/4222614770 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f15dc09f170 con 0x5561ac144c40 2024-05-09T09:23:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.306+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d8036070 con 0x5561ac144c40 2024-05-09T09:23:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.306+0000 7f15c97fa640 1 -- 172.21.15.64:0/4222614770 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f15dc0a3d70 con 0x5561ac144c40 2024-05-09T09:23:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.306+0000 7f15ea83c740 10 client.28666 fetch_fsmap learned FSMap version 49 2024-05-09T09:23:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.306+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5561ac1d1b40 con 0x5561ac144c40 2024-05-09T09:23:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.310+0000 7f15c97fa640 1 -- 172.21.15.64:0/4222614770 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15dc09f4b0 con 0x5561ac144c40 2024-05-09T09:23:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.311+0000 7f15c97fa640 1 -- 172.21.15.64:0/4222614770 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f15dc09a620 con 0x5561ac144c40 2024-05-09T09:23:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.312+0000 7f15ea83c740 10 client.28666 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:23:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.312+0000 7f15ea83c740 10 client.28666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:23:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.312+0000 7f15ea83c740 20 client.28666 populate_metadata read hostname 'smithi064' 2024-05-09T09:23:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.312+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac1d2160 0x5561ac1d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:23:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.312+0000 7f15ea83c740 4 client.28666 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:23:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.312+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f15d8036070 con 0x5561ac1d2160 2024-05-09T09:23:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.314+0000 7f15e58c5640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac1d2160 0x5561ac1d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:23:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.315+0000 7f15e58c5640 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac1d2160 0x5561ac1d4550 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6e85051004a3ef server_cookie=b35dca8a57c430b6 in_seq=0 out_seq=0 2024-05-09T09:23:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.315+0000 7f15c97fa640 10 client.28666 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.356+0000 7f15c97fa640 1 -- 172.21.15.64:0/4222614770 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f15d8036070 con 0x5561ac1d2160 2024-05-09T09:23:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.356+0000 7f15c97fa640 10 client.28666 handle_command_reply: tid=0 2024-05-09T09:23:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.356+0000 7f15ea83c740 1 client.28666 shutdown 2024-05-09T09:23:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.362+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144c40 msgr2=0x5561ac153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.362+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144c40 0x5561ac153700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5561ac14c9b0 tx=0x7f15dc09ad70 comp rx=0 tx=0).stop 2024-05-09T09:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 shutdown_connections 2024-05-09T09:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5561ac1d2160 0x5561ac1d4550 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561ac149db0 0x5561ac154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15c97fa640 0 client.28666 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561ac144c40 0x5561ac153700 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 --2- 172.21.15.64:0/4222614770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5561ac144250 0x5561ac1531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 >> 172.21.15.64:0/4222614770 conn(0x5561ac13f8c0 msgr2=0x5561ac142840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.365+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 shutdown_connections 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.366+0000 7f15ea83c740 1 -- 172.21.15.64:0/4222614770 wait complete. 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.366+0000 7f15ea83c740 20 client.28666 trim_cache size 0 max 16384 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15cc0a2db0 msgr2=0x7f15cc0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15cc0a2db0 0x7f15cc0a5270 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009990 tx=0x7f15d0009110 comp rx=0 tx=0).stop 2024-05-09T09:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 msgr2=0x7f15e0111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 0x7f15e0111120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b450 tx=0x7f15dc00b850 comp rx=0 tx=0).stop 2024-05-09T09:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 shutdown_connections 2024-05-09T09:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f15cc0a2db0 0x7f15cc0a5270 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e0101200 0x7f15e01adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15e0100860 0x7f15e0111120 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 --2- 172.21.15.64:0/2378960709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e00ff730 0x7f15e0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.368+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 >> 172.21.15.64:0/2378960709 conn(0x7f15e00fb050 msgr2=0x7f15e00fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.369+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 shutdown_connections 2024-05-09T09:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:23:50.369+0000 7f15e7b29640 1 -- 172.21.15.64:0/2378960709 wait complete. 2024-05-09T09:23:51.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:50 smithi139 ceph-mon[33599]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 8.1 MiB/s wr, 969 op/s 2024-05-09T09:23:51.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:50 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 810.0 s, 722.4 tps, lat 138.055 ms stddev 96.928 2024-05-09T09:23:51.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:50 smithi064 ceph-mon[30149]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 8.1 MiB/s wr, 969 op/s 2024-05-09T09:23:51.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:50 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:51.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:50 smithi179 ceph-mon[34086]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 8.1 MiB/s wr, 969 op/s 2024-05-09T09:23:51.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:50 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:23:53.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:53 smithi139 ceph-mon[33599]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.7 MiB/s wr, 688 op/s 2024-05-09T09:23:53.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:53 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:53.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:53 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:53.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:53 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:23:53.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:53 smithi064 ceph-mon[30149]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.7 MiB/s wr, 688 op/s 2024-05-09T09:23:53.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:53 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:53.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:53 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:53.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:53 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:23:53.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:53 smithi179 ceph-mon[34086]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.7 MiB/s wr, 688 op/s 2024-05-09T09:23:53.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:53 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:23:53.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:53 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:23:53.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:53 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:23:55.280 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:55 smithi179 ceph-mon[34086]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.9 MiB/s wr, 821 op/s 2024-05-09T09:23:55.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:55 smithi139 ceph-mon[33599]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.9 MiB/s wr, 821 op/s 2024-05-09T09:23:55.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:55 smithi064 ceph-mon[30149]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.9 MiB/s wr, 821 op/s 2024-05-09T09:23:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:56 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:23:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:56 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:23:56.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:56 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:23:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 815.0 s, 712.2 tps, lat 140.623 ms stddev 100.968 2024-05-09T09:23:56.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:56 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:23:56.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:56 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:23:56.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:56 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:23:56.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:56 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:23:56.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:56 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:23:56.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:56 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:23:57.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:57 smithi139 ceph-mon[33599]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 796 op/s 2024-05-09T09:23:57.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:57 smithi064 ceph-mon[30149]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 796 op/s 2024-05-09T09:23:57.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:57 smithi179 ceph-mon[34086]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 796 op/s 2024-05-09T09:23:59.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:23:59 smithi139 ceph-mon[33599]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.4 MiB/s wr, 658 op/s 2024-05-09T09:23:59.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:23:59 smithi064 ceph-mon[30149]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.4 MiB/s wr, 658 op/s 2024-05-09T09:23:59.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:23:59 smithi179 ceph-mon[34086]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.4 MiB/s wr, 658 op/s 2024-05-09T09:24:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:01 smithi139 ceph-mon[33599]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.9 MiB/s wr, 836 op/s 2024-05-09T09:24:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 820.0 s, 709.6 tps, lat 140.864 ms stddev 76.012 2024-05-09T09:24:01.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:01 smithi064 ceph-mon[30149]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.9 MiB/s wr, 836 op/s 2024-05-09T09:24:01.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:01 smithi179 ceph-mon[34086]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.9 MiB/s wr, 836 op/s 2024-05-09T09:24:03.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:03 smithi139 ceph-mon[33599]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 582 op/s 2024-05-09T09:24:03.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:03 smithi064 ceph-mon[30149]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 582 op/s 2024-05-09T09:24:03.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:03 smithi179 ceph-mon[34086]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 582 op/s 2024-05-09T09:24:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 -- 172.21.15.64:0/1854276117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538109b30 msgr2=0x7fe538112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1854276117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538109b30 0x7fe538112410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe530009be0 tx=0x7fe53002a6c0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 -- 172.21.15.64:0/1854276117 shutdown_connections 2024-05-09T09:24:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1854276117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538109b30 0x7fe538112410 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1854276117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109190 0x7fe5381095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1854276117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538108830 0x7fe538108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 -- 172.21.15.64:0/1854276117 >> 172.21.15.64:0/1854276117 conn(0x7fe53806e6d0 msgr2=0x7fe538070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 -- 172.21.15.64:0/1854276117 shutdown_connections 2024-05-09T09:24:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.532+0000 7fe53daa5640 1 -- 172.21.15.64:0/1854276117 wait complete. 2024-05-09T09:24:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.533+0000 7fe53daa5640 1 Processor -- start 2024-05-09T09:24:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 -- start start 2024-05-09T09:24:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 0x7fe5381a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538109190 0x7fe5381a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109b30 0x7fe538079a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe538113810 con 0x7fe538109190 2024-05-09T09:24:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe538113690 con 0x7fe538109b30 2024-05-09T09:24:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe53daa5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe538113990 con 0x7fe538108830 2024-05-09T09:24:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe5377fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 0x7fe5381a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe5377fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 0x7fe5381a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:56932/0 (socket says 172.21.15.64:56932) 2024-05-09T09:24:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe5377fe640 1 -- 172.21.15.64:0/1197274027 learned_addr learned my addr 172.21.15.64:0/1197274027 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.534+0000 7fe52ffff640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538109190 0x7fe5381a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe537fff640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109b30 0x7fe538079a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe5377fe640 1 -- 172.21.15.64:0/1197274027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109b30 msgr2=0x7fe538079a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe5377fe640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109b30 0x7fe538079a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe5377fe640 1 -- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538109190 msgr2=0x7fe5381a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe5377fe640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538109190 0x7fe5381a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe5377fe640 1 -- 172.21.15.64:0/1197274027 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53807a130 con 0x7fe538108830 2024-05-09T09:24:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe537fff640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109b30 0x7fe538079a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.535+0000 7fe5377fe640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 0x7fe5381a6690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e9c0 tx=0x7fe52800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.536+0000 7fe5357fa640 1 -- 172.21.15.64:0/1197274027 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800ccc0 con 0x7fe538108830 2024-05-09T09:24:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.536+0000 7fe5357fa640 1 -- 172.21.15.64:0/1197274027 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5280054d0 con 0x7fe538108830 2024-05-09T09:24:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.536+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53807a420 con 0x7fe538108830 2024-05-09T09:24:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.536+0000 7fe5357fa640 1 -- 172.21.15.64:0/1197274027 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528010430 con 0x7fe538108830 2024-05-09T09:24:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.536+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53807a970 con 0x7fe538108830 2024-05-09T09:24:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.542+0000 7fe5357fa640 1 -- 172.21.15.64:0/1197274027 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe5280046c0 con 0x7fe538108830 2024-05-09T09:24:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.546+0000 7fe5407b1740 1 Processor -- start 2024-05-09T09:24:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.546+0000 7fe5357fa640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5240a2d50 0x7fe5240a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.546+0000 7fe5357fa640 1 -- 172.21.15.64:0/1197274027 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5280cf630 con 0x7fe538108830 2024-05-09T09:24:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 -- start start 2024-05-09T09:24:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497cc92820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497cc978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497cc9a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56497cbf6340 con 0x56497cc97f60 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56497cb9e290 con 0x56497cc92df0 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe5407b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56497cbf2300 con 0x56497cc92400 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe537fff640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5240a2d50 0x7fe5240a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe537fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497cc9a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe537fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497cc9a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33056/0 (socket says 172.21.15.64:33056) 2024-05-09T09:24:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.547+0000 7fe537fff640 1 -- 172.21.15.64:0/3941706951 learned_addr learned my addr 172.21.15.64:0/3941706951 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe52ffff640 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497cc978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe5377fe640 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497cc92820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe537fff640 1 -- 172.21.15.64:0/3941706951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 msgr2=0x56497cc92820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe537fff640 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497cc92820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe537fff640 1 -- 172.21.15.64:0/3941706951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 msgr2=0x56497cc978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe537fff640 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497cc978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.548+0000 7fe537fff640 1 -- 172.21.15.64:0/3941706951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497cbc28e0 con 0x56497cc97f60 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.549+0000 7fe537fff640 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497cc9a3f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe530030850 tx=0x7fe530042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13904da7ff73c048 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.549+0000 7fe537fff640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5240a2d50 0x7fe5240a5210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe530030620 tx=0x7fe530030650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.549+0000 7fe52dffb640 1 -- 172.21.15.64:0/3941706951 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530051070 con 0x56497cc97f60 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.550+0000 7fe52dffb640 1 -- 172.21.15.64:0/3941706951 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe530037040 con 0x56497cc97f60 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.550+0000 7fe52dffb640 1 -- 172.21.15.64:0/3941706951 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530036050 con 0x56497cc97f60 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 -- 172.21.15.64:0/3941706951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 msgr2=0x56497cc9a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497cc9a3f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe530030850 tx=0x7fe530042000 comp rx=0 tx=0).stop 2024-05-09T09:24:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 -- 172.21.15.64:0/3941706951 shutdown_connections 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497cc9a3f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497cc978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 --2- 172.21.15.64:0/3941706951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497cc92820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 -- 172.21.15.64:0/3941706951 >> 172.21.15.64:0/3941706951 conn(0x56497cc8da70 msgr2=0x56497cc8de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 -- 172.21.15.64:0/3941706951 shutdown_connections 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.551+0000 7fe5407b1740 1 -- 172.21.15.64:0/3941706951 wait complete. 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.552+0000 7fe5407b1740 1 -- start start 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.552+0000 7fe5407b1740 1 Processor -- start 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497ccad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497ccadba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56497ccabec0 con 0x56497cc92400 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56497cbb1790 con 0x56497cc97f60 2024-05-09T09:24:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56497ccac040 con 0x56497cc92df0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe5407b1740 10 client.0 _ll_register_callbacks cb 0x56497cbeca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.553+0000 7fe537fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe537fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58228/0 (socket says 172.21.15.64:58228) 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe537fff640 1 -- 172.21.15.64:0/1666471459 learned_addr learned my addr 172.21.15.64:0/1666471459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe5377fe640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497ccad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe52ffff640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497ccadba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe52ffff640 1 -- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 msgr2=0x56497ccb4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe52ffff640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe52ffff640 1 -- 172.21.15.64:0/1666471459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 msgr2=0x56497ccad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe52ffff640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497ccad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.554+0000 7fe52ffff640 1 -- 172.21.15.64:0/1666471459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497c90a210 con 0x56497cc92df0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.555+0000 7fe52ffff640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497ccadba0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x56497cc931f0 tx=0x7fe520012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51c705ec7bb8d5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.555+0000 7fe516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.555+0000 7fe516ffd640 1 -- 172.21.15.64:0/1666471459 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004030 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.555+0000 7fe516ffd640 1 -- 172.21.15.64:0/1666471459 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5200041d0 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.555+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe530037040 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe516ffd640 1 -- 172.21.15.64:0/1666471459 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe5200043d0 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe5407b1740 10 client.28675 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56497c9d6620 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe516ffd640 1 -- 172.21.15.64:0/1666471459 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe52000fd40 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe516ffd640 1 -- 172.21.15.64:0/1666471459 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe520003c60 con 0x56497cc92df0 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe5407b1740 10 client.28675 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.556+0000 7fe5407b1740 10 client.28675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.557+0000 7fe5407b1740 20 client.28675 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.557+0000 7fe5407b1740 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497ccb9270 0x56497ccbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.557+0000 7fe5407b1740 4 client.28675 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.557+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe530037040 con 0x56497ccb9270 2024-05-09T09:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.557+0000 7fe5377fe640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497ccb9270 0x56497ccbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.563+0000 7fe5377fe640 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497ccb9270 0x56497ccbb6c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a99e828d24c2eeb server_cookie=d1fdaafd81d68daa in_seq=0 out_seq=0 2024-05-09T09:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.563+0000 7fe516ffd640 10 client.28675 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe516ffd640 1 -- 172.21.15.64:0/1666471459 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fe530037040 con 0x56497ccb9270 2024-05-09T09:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe516ffd640 10 client.28675 handle_command_reply: tid=0 2024-05-09T09:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 client.28675 shutdown 2024-05-09T09:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 msgr2=0x56497ccadba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497ccadba0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x56497cc931f0 tx=0x7fe520012b60 comp rx=0 tx=0).stop 2024-05-09T09:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 shutdown_connections 2024-05-09T09:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497ccb9270 0x56497ccbb6c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497ccadba0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 --2- 172.21.15.64:0/1666471459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497ccad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.566+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 >> 172.21.15.64:0/1666471459 conn(0x56497cc8da70 msgr2=0x56497cc8cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.567+0000 7fe516ffd640 0 client.28675 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.568+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 shutdown_connections 2024-05-09T09:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.568+0000 7fe5407b1740 1 -- 172.21.15.64:0/1666471459 wait complete. 2024-05-09T09:24:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.569+0000 7fe5407b1740 20 client.28675 trim_cache size 0 max 16384 2024-05-09T09:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.602+0000 7fe5407b1740 1 Processor -- start 2024-05-09T09:24:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.603+0000 7fe5407b1740 1 -- start start 2024-05-09T09:24:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.603+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497cc940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497cc945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe5407b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe53807efa0 con 0x56497cc92400 2024-05-09T09:24:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe5407b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe53807c810 con 0x56497cc97f60 2024-05-09T09:24:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe5407b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe53811efa0 con 0x56497cc92df0 2024-05-09T09:24:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe537fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe537fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58230/0 (socket says 172.21.15.64:58230) 2024-05-09T09:24:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.604+0000 7fe537fff640 1 -- 172.21.15.64:0/2566911829 learned_addr learned my addr 172.21.15.64:0/2566911829 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe5377fe640 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497cc940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe537fff640 1 -- 172.21.15.64:0/2566911829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 msgr2=0x56497cc945f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe537fff640 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497cc945f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe537fff640 1 -- 172.21.15.64:0/2566911829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 msgr2=0x56497cc940b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe537fff640 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497cc940b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe537fff640 1 -- 172.21.15.64:0/2566911829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497cc8e7e0 con 0x56497cc97f60 2024-05-09T09:24:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe537fff640 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4ba0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe5300307b0 tx=0x7fe5300304b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc41e1eda1aae718 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.605+0000 7fe52d7fa640 1 -- 172.21.15.64:0/2566911829 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530051040 con 0x56497cc97f60 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.606+0000 7fe52d7fa640 1 -- 172.21.15.64:0/2566911829 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe530030ba0 con 0x56497cc97f60 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.606+0000 7fe52d7fa640 1 -- 172.21.15.64:0/2566911829 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530058460 con 0x56497cc97f60 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.606+0000 7fe5407b1740 1 -- 172.21.15.64:0/2566911829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 msgr2=0x56497ccb4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4ba0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe5300307b0 tx=0x7fe5300304b0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 -- 172.21.15.64:0/2566911829 shutdown_connections 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc97f60 0x56497ccb4ba0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92df0 0x56497cc945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 --2- 172.21.15.64:0/2566911829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc92400 0x56497cc940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 -- 172.21.15.64:0/2566911829 >> 172.21.15.64:0/2566911829 conn(0x56497cc8da70 msgr2=0x56497cc8de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 -- 172.21.15.64:0/2566911829 shutdown_connections 2024-05-09T09:24:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 -- 172.21.15.64:0/2566911829 wait complete. 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 -- start start 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.607+0000 7fe5407b1740 1 Processor -- start 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497ccb2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497ccb2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497ccb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe53807acd0 con 0x56497cc97f60 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe53807c140 con 0x56497cc92df0 2024-05-09T09:24:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe53807b3a0 con 0x56497cc92400 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5407b1740 10 client.0 _ll_register_callbacks cb 0x7fe5200097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497ccb2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497ccb2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:58232/0 (socket says 172.21.15.64:58232) 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 -- 172.21.15.64:0/566306908 learned_addr learned my addr 172.21.15.64:0/566306908 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe537fff640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497ccb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5377fe640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497ccb2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 -- 172.21.15.64:0/566306908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 msgr2=0x56497ccb2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497ccb2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 -- 172.21.15.64:0/566306908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 msgr2=0x56497ccb3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497ccb3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe52ffff640 1 -- 172.21.15.64:0/566306908 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497ccb4240 con 0x56497cc92df0 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe537fff640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497ccb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.608+0000 7fe5377fe640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497ccb2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe52ffff640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497ccb2c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe520010af0 tx=0x7fe52000fa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c261be0f3485f536 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe516ffd640 1 -- 172.21.15.64:0/566306908 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52002d020 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe530030ba0 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe516ffd640 1 -- 172.21.15.64:0/566306908 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe520029420 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe516ffd640 1 -- 172.21.15.64:0/566306908 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe52000fe40 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe5407b1740 10 client.28499 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.609+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56497cd1fc50 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe516ffd640 1 -- 172.21.15.64:0/566306908 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe530030ba0 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe516ffd640 1 -- 172.21.15.64:0/566306908 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe52001c420 con 0x56497cc92df0 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe5407b1740 10 client.28499 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe5407b1740 10 client.28499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe5407b1740 20 client.28499 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe5407b1740 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497cd20680 0x56497cd22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe5407b1740 4 client.28499 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:24:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.610+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56497cc03a70 con 0x56497cd20680 2024-05-09T09:24:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.612+0000 7fe5377fe640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497cd20680 0x56497cd22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.613+0000 7fe5377fe640 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497cd20680 0x56497cd22aa0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1aea1131923f1 server_cookie=bc60b79771ea66b6 in_seq=0 out_seq=0 2024-05-09T09:24:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.613+0000 7fe516ffd640 10 client.28499 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe516ffd640 1 -- 172.21.15.64:0/566306908 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56497cc03a70 con 0x56497cd20680 2024-05-09T09:24:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe516ffd640 10 client.28499 handle_command_reply: tid=0 2024-05-09T09:24:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 client.28499 shutdown 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 msgr2=0x56497ccb2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497ccb2c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe520010af0 tx=0x7fe52000fa40 comp rx=0 tx=0).stop 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 shutdown_connections 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56497cd20680 0x56497cd22aa0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56497cc97f60 0x56497ccb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56497cc92df0 0x56497ccb2c40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 --2- 172.21.15.64:0/566306908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56497cc92400 0x56497ccb2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 >> 172.21.15.64:0/566306908 conn(0x56497cc8da70 msgr2=0x56497ccad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe516ffd640 0 client.28499 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 shutdown_connections 2024-05-09T09:24:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 1 -- 172.21.15.64:0/566306908 wait complete. 2024-05-09T09:24:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.627+0000 7fe5407b1740 20 client.28499 trim_cache size 0 max 16384 2024-05-09T09:24:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:24:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5240a2d50 msgr2=0x7fe5240a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5240a2d50 0x7fe5240a5210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe530030620 tx=0x7fe530030650 comp rx=0 tx=0).stop 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 msgr2=0x7fe5381a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 0x7fe5381a6690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e9c0 tx=0x7fe52800ee80 comp rx=0 tx=0).stop 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 shutdown_connections 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe5240a2d50 0x7fe5240a5210 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe538109b30 0x7fe538079a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538109190 0x7fe5381a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 --2- 172.21.15.64:0/1197274027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe538108830 0x7fe5381a6690 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.628+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 >> 172.21.15.64:0/1197274027 conn(0x7fe53806e6d0 msgr2=0x7fe53806f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.629+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 shutdown_connections 2024-05-09T09:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:03.629+0000 7fe53daa5640 1 -- 172.21.15.64:0/1197274027 wait complete. 2024-05-09T09:24:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:05 smithi139 ceph-mon[33599]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 MiB/s wr, 728 op/s 2024-05-09T09:24:05.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:05 smithi064 ceph-mon[30149]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 MiB/s wr, 728 op/s 2024-05-09T09:24:05.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:05 smithi179 ceph-mon[34086]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 MiB/s wr, 728 op/s 2024-05-09T09:24:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 825.0 s, 736.2 tps, lat 134.099 ms stddev 74.394 2024-05-09T09:24:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:07 smithi139 ceph-mon[33599]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 MiB/s wr, 680 op/s 2024-05-09T09:24:07.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:07 smithi064 ceph-mon[30149]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 MiB/s wr, 680 op/s 2024-05-09T09:24:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:07 smithi179 ceph-mon[34086]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 MiB/s wr, 680 op/s 2024-05-09T09:24:09.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:09 smithi139 ceph-mon[33599]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-05-09T09:24:09.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:09 smithi064 ceph-mon[30149]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-05-09T09:24:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:09 smithi179 ceph-mon[34086]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-05-09T09:24:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:11 smithi139 ceph-mon[33599]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 842 op/s 2024-05-09T09:24:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 830.0 s, 772.5 tps, lat 131.173 ms stddev 95.131 2024-05-09T09:24:11.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:11 smithi064 ceph-mon[30149]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 842 op/s 2024-05-09T09:24:11.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:11 smithi179 ceph-mon[34086]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 842 op/s 2024-05-09T09:24:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:13 smithi139 ceph-mon[33599]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 647 op/s 2024-05-09T09:24:13.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:13 smithi064 ceph-mon[30149]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 647 op/s 2024-05-09T09:24:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:13 smithi179 ceph-mon[34086]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 647 op/s 2024-05-09T09:24:13.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:24:13.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:24:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.900+0000 7f1f6d185640 1 -- 172.21.15.64:0/3433219860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f68102930 msgr2=0x7f1f68102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:13.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.900+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3433219860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f68102930 0x7f1f68102db0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009860 tx=0x7f1f5c02a360 comp rx=0 tx=0).stop 2024-05-09T09:24:13.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.902+0000 7f1f6d185640 1 -- 172.21.15.64:0/3433219860 shutdown_connections 2024-05-09T09:24:13.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.902+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3433219860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f681032f0 0x7f1f68109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.902+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3433219860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f68102930 0x7f1f68102db0 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009860 tx=0x7f1f5c02a360 comp rx=0 tx=0).stop 2024-05-09T09:24:13.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.902+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3433219860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 0x7f1f68101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.902+0000 7f1f6d185640 1 -- 172.21.15.64:0/3433219860 >> 172.21.15.64:0/3433219860 conn(0x7f1f680fccd0 msgr2=0x7f1f680ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.904+0000 7f1f6d185640 1 -- 172.21.15.64:0/3433219860 shutdown_connections 2024-05-09T09:24:13.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.904+0000 7f1f6d185640 1 -- 172.21.15.64:0/3433219860 wait complete. 2024-05-09T09:24:13.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.905+0000 7f1f6d185640 1 Processor -- start 2024-05-09T09:24:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 -- start start 2024-05-09T09:24:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 0x7f1f68075080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f681032f0 0x7f1f680755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f680736c0 0x7f1f68073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806dd00 con 0x7f1f68101740 2024-05-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806db80 con 0x7f1f681032f0 2024-05-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.910+0000 7f1f6d185640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806de80 con 0x7f1f680736c0 2024-05-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.911+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 0x7f1f68075080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.911+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 0x7f1f68075080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53298/0 (socket says 172.21.15.64:53298) 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.911+0000 7f1f67fff640 1 -- 172.21.15.64:0/3245757716 learned_addr learned my addr 172.21.15.64:0/3245757716 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.911+0000 7f1f677fe640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f681032f0 0x7f1f680755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f677fe640 1 -- 172.21.15.64:0/3245757716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f680736c0 msgr2=0x7f1f68073b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f677fe640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f680736c0 0x7f1f68073b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f677fe640 1 -- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 msgr2=0x7f1f68075080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f677fe640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 0x7f1f68075080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f677fe640 1 -- 172.21.15.64:0/3245757716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f68074310 con 0x7f1f681032f0 2024-05-09T09:24:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f677fe640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f681032f0 0x7f1f680755c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c008000 tx=0x7f1f5c032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.912+0000 7f1f657fa640 1 -- 172.21.15.64:0/3245757716 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c040020 con 0x7f1f681032f0 2024-05-09T09:24:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.913+0000 7f1f657fa640 1 -- 172.21.15.64:0/3245757716 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f5c0042c0 con 0x7f1f681032f0 2024-05-09T09:24:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.913+0000 7f1f657fa640 1 -- 172.21.15.64:0/3245757716 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037040 con 0x7f1f681032f0 2024-05-09T09:24:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.913+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f68195510 con 0x7f1f681032f0 2024-05-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.913+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f68195980 con 0x7f1f681032f0 2024-05-09T09:24:13.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.917+0000 7f1f711d4740 1 Processor -- start 2024-05-09T09:24:13.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.917+0000 7f1f711d4740 1 -- start start 2024-05-09T09:24:13.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c273500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 0x56468c2785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c278c40 0x56468c27b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f711d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56468c1d7d00 con 0x56468c273ad0 2024-05-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f711d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56468c17f4c0 con 0x56468c2730e0 2024-05-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f711d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56468c1d3cc0 con 0x56468c278c40 2024-05-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 0x56468c2785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f6c984640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c273500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 0x56468c2785d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53314/0 (socket says 172.21.15.64:53314) 2024-05-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 -- 172.21.15.64:0/729284251 learned_addr learned my addr 172.21.15.64:0/729284251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 -- 172.21.15.64:0/729284251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c278c40 msgr2=0x56468c27b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c278c40 0x56468c27b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 -- 172.21.15.64:0/729284251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 msgr2=0x56468c273500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.918+0000 7f1f67fff640 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c273500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.919+0000 7f1f67fff640 1 -- 172.21.15.64:0/729284251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468bfb57d0 con 0x56468c273ad0 2024-05-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.919+0000 7f1f6c984640 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c273500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.919+0000 7f1f67fff640 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 0x56468c2785d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1f5801ad10 tx=0x7f1f580227c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb45520e9b33bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.920+0000 7f1f657fa640 1 -- 172.21.15.64:0/3245757716 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f5c043020 con 0x7f1f681032f0 2024-05-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.921+0000 7f1f657fa640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1f500a2d60 0x7f1f500a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.921+0000 7f1f657fa640 1 -- 172.21.15.64:0/3245757716 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1f5c0ef1b0 con 0x7f1f681032f0 2024-05-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.922+0000 7f1f4e7fc640 1 -- 172.21.15.64:0/729284251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5802f070 con 0x56468c273ad0 2024-05-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.922+0000 7f1f4e7fc640 1 -- 172.21.15.64:0/729284251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f5801ae00 con 0x56468c273ad0 2024-05-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.922+0000 7f1f4e7fc640 1 -- 172.21.15.64:0/729284251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5802a980 con 0x56468c273ad0 2024-05-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.923+0000 7f1f6c984640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1f500a2d60 0x7f1f500a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.923+0000 7f1f711d4740 1 -- 172.21.15.64:0/729284251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 msgr2=0x56468c2785d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.923+0000 7f1f711d4740 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 0x56468c2785d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1f5801ad10 tx=0x7f1f580227c0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.924+0000 7f1f711d4740 1 -- 172.21.15.64:0/729284251 shutdown_connections 2024-05-09T09:24:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.924+0000 7f1f711d4740 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c278c40 0x56468c27b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.924+0000 7f1f711d4740 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c273ad0 0x56468c2785d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.924+0000 7f1f711d4740 1 --2- 172.21.15.64:0/729284251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c273500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.924+0000 7f1f711d4740 1 -- 172.21.15.64:0/729284251 >> 172.21.15.64:0/729284251 conn(0x56468c26e750 msgr2=0x56468c26eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.925+0000 7f1f6c984640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1f500a2d60 0x7f1f500a5220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x56468c274330 tx=0x7f1f60009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:13.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.927+0000 7f1f711d4740 1 -- 172.21.15.64:0/729284251 shutdown_connections 2024-05-09T09:24:13.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.927+0000 7f1f711d4740 1 -- 172.21.15.64:0/729284251 wait complete. 2024-05-09T09:24:13.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.927+0000 7f1f711d4740 1 -- start start 2024-05-09T09:24:13.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 Processor -- start 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c27d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c27db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c284ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56468c27bbf0 con 0x56468c278c40 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56468c192b90 con 0x56468c2730e0 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56468c27bd70 con 0x56468c273ad0 2024-05-09T09:24:13.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.928+0000 7f1f711d4740 10 client.0 _ll_register_callbacks cb 0x56468c1ceda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c27d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c27d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57466/0 (socket says 172.21.15.64:57466) 2024-05-09T09:24:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 -- 172.21.15.64:0/2492596678 learned_addr learned my addr 172.21.15.64:0/2492596678 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f677fe640 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c27db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 -- 172.21.15.64:0/2492596678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 msgr2=0x56468c27db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:13.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c27db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 -- 172.21.15.64:0/2492596678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 msgr2=0x56468c284ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c284ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 -- 172.21.15.64:0/2492596678 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468beeb3f0 con 0x56468c2730e0 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.929+0000 7f1f67fff640 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c27d620 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1f5802a450 tx=0x7f1f5802bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65daab74d6f3dde5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.930+0000 7f1f477fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.930+0000 7f1f477fe640 1 -- 172.21.15.64:0/2492596678 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58038060 con 0x56468c2730e0 2024-05-09T09:24:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.930+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5801ae00 con 0x56468c2730e0 2024-05-09T09:24:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.930+0000 7f1f477fe640 1 -- 172.21.15.64:0/2492596678 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1f58019040 con 0x56468c2730e0 2024-05-09T09:24:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.930+0000 7f1f477fe640 1 -- 172.21.15.64:0/2492596678 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1f5803f050 con 0x56468c2730e0 2024-05-09T09:24:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.931+0000 7f1f711d4740 10 client.28511 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.931+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56468beab0f0 con 0x56468c2730e0 2024-05-09T09:24:13.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.931+0000 7f1f477fe640 1 -- 172.21.15.64:0/2492596678 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f5801ae00 con 0x56468c2730e0 2024-05-09T09:24:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f477fe640 1 -- 172.21.15.64:0/2492596678 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f1f58032030 con 0x56468c2730e0 2024-05-09T09:24:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f711d4740 10 client.28511 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f711d4740 10 client.28511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f711d4740 20 client.28511 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f711d4740 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c289760 0x56468c28bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f711d4740 4 client.28511 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:13.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.932+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468c1e5430 con 0x56468c289760 2024-05-09T09:24:13.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.933+0000 7f1f677fe640 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c289760 0x56468c28bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.933+0000 7f1f677fe640 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c289760 0x56468c28bc20 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecd5a471c8f20d4 server_cookie=47d7772a86448213 in_seq=0 out_seq=0 2024-05-09T09:24:13.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.934+0000 7f1f477fe640 10 client.28511 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:13.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.937+0000 7f1f477fe640 1 -- 172.21.15.64:0/2492596678 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56468c1e5430 con 0x56468c289760 2024-05-09T09:24:13.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.938+0000 7f1f477fe640 10 client.28511 handle_command_reply: tid=0 2024-05-09T09:24:13.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.938+0000 7f1f711d4740 1 client.28511 shutdown 2024-05-09T09:24:13.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.938+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 msgr2=0x56468c27d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:13.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.938+0000 7f1f711d4740 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c27d620 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1f5802a450 tx=0x7f1f5802bb40 comp rx=0 tx=0).stop 2024-05-09T09:24:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 shutdown_connections 2024-05-09T09:24:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f711d4740 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c289760 0x56468c28bc20 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f711d4740 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c284ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f711d4740 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c27db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f711d4740 1 --2- 172.21.15.64:0/2492596678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c27d620 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f477fe640 0 client.28511 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.939+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 >> 172.21.15.64:0/2492596678 conn(0x56468c26e750 msgr2=0x56468c26da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.940+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 shutdown_connections 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.940+0000 7f1f711d4740 1 -- 172.21.15.64:0/2492596678 wait complete. 2024-05-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.940+0000 7f1f711d4740 20 client.28511 trim_cache size 0 max 16384 2024-05-09T09:24:13.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 Processor -- start 2024-05-09T09:24:13.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 -- start start 2024-05-09T09:24:13.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c2730e0 0x56468c27c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c273ad0 0x56468c284b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c27cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b3550 con 0x56468c278c40 2024-05-09T09:24:13.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b0dc0 con 0x56468c273ad0 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f711d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681aecc0 con 0x56468c2730e0 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f6c984640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c27cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f6c984640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c27cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53332/0 (socket says 172.21.15.64:53332) 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.982+0000 7f1f6c984640 1 -- 172.21.15.64:0/1917218288 learned_addr learned my addr 172.21.15.64:0/1917218288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.983+0000 7f1f6c984640 1 -- 172.21.15.64:0/1917218288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c2730e0 msgr2=0x56468c27c110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.983+0000 7f1f6c984640 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c2730e0 0x56468c27c110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.983+0000 7f1f6c984640 1 -- 172.21.15.64:0/1917218288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c273ad0 msgr2=0x56468c284b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.983+0000 7f1f6c984640 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c273ad0 0x56468c284b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.983+0000 7f1f6c984640 1 -- 172.21.15.64:0/1917218288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468c275680 con 0x56468c278c40 2024-05-09T09:24:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.983+0000 7f1f6c984640 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c27cc40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1f60003b00 tx=0x7f1f60013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a00958c70b937c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.984+0000 7f1f4dffb640 1 -- 172.21.15.64:0/1917218288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60004780 con 0x56468c278c40 2024-05-09T09:24:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.984+0000 7f1f4dffb640 1 -- 172.21.15.64:0/1917218288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f60022070 con 0x56468c278c40 2024-05-09T09:24:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.984+0000 7f1f4dffb640 1 -- 172.21.15.64:0/1917218288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6001d430 con 0x56468c278c40 2024-05-09T09:24:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.985+0000 7f1f711d4740 1 -- 172.21.15.64:0/1917218288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 msgr2=0x56468c27cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.985+0000 7f1f711d4740 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c27cc40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1f60003b00 tx=0x7f1f60013040 comp rx=0 tx=0).stop 2024-05-09T09:24:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 -- 172.21.15.64:0/1917218288 shutdown_connections 2024-05-09T09:24:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c27cc40 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c273ad0 0x56468c284b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 --2- 172.21.15.64:0/1917218288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c2730e0 0x56468c27c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 -- 172.21.15.64:0/1917218288 >> 172.21.15.64:0/1917218288 conn(0x56468c26e750 msgr2=0x56468c26da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 -- 172.21.15.64:0/1917218288 shutdown_connections 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.986+0000 7f1f711d4740 1 -- 172.21.15.64:0/1917218288 wait complete. 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.987+0000 7f1f711d4740 1 -- start start 2024-05-09T09:24:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.987+0000 7f1f711d4740 1 Processor -- start 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c281f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c282470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f711d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c2833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c281f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c281f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57478/0 (socket says 172.21.15.64:57478) 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f67fff640 1 -- 172.21.15.64:0/4199692135 learned_addr learned my addr 172.21.15.64:0/4199692135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681af280 con 0x56468c278c40 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b06f0 con 0x56468c2730e0 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.988+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681af950 con 0x56468c273ad0 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f711d4740 10 client.0 _ll_register_callbacks cb 0x7f1f5802d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f6c984640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c2833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f677fe640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c282470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f67fff640 1 -- 172.21.15.64:0/4199692135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 msgr2=0x56468c282470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:13.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f67fff640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c282470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f67fff640 1 -- 172.21.15.64:0/4199692135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 msgr2=0x56468c2833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f67fff640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c2833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.989+0000 7f1f67fff640 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468c2839b0 con 0x56468c2730e0 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.991+0000 7f1f67fff640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c281f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1f58022800 tx=0x7f1f58022e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f094caa7b26215 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.991+0000 7f1f477fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.991+0000 7f1f477fe640 1 -- 172.21.15.64:0/4199692135 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58038040 con 0x56468c2730e0 2024-05-09T09:24:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.991+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f60022070 con 0x56468c2730e0 2024-05-09T09:24:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.993+0000 7f1f477fe640 1 -- 172.21.15.64:0/4199692135 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1f5802bdc0 con 0x56468c2730e0 2024-05-09T09:24:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.993+0000 7f1f477fe640 1 -- 172.21.15.64:0/4199692135 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1f58049050 con 0x56468c2730e0 2024-05-09T09:24:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.993+0000 7f1f711d4740 10 client.28514 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.993+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56468c284ca0 con 0x56468c2730e0 2024-05-09T09:24:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.993+0000 7f1f477fe640 1 -- 172.21.15.64:0/4199692135 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f60022070 con 0x56468c2730e0 2024-05-09T09:24:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f477fe640 1 -- 172.21.15.64:0/4199692135 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f1f58032040 con 0x56468c2730e0 2024-05-09T09:24:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f711d4740 10 client.28514 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f711d4740 10 client.28514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f711d4740 20 client.28514 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f711d4740 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c2f03f0 0x56468c2f2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f711d4740 4 client.28514 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56468c1df780 con 0x56468c2f03f0 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.994+0000 7f1f677fe640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c2f03f0 0x56468c2f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.996+0000 7f1f677fe640 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c2f03f0 0x56468c2f2810 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518f1385ca5a90fe server_cookie=80ce1b25b37a5901 in_seq=0 out_seq=0 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.996+0000 7f1f477fe640 10 client.28514 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.997+0000 7f1f477fe640 1 -- 172.21.15.64:0/4199692135 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56468c1df780 con 0x56468c2f03f0 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.997+0000 7f1f477fe640 10 client.28514 handle_command_reply: tid=0 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.997+0000 7f1f711d4740 1 client.28514 shutdown 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.997+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 msgr2=0x56468c281f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.997+0000 7f1f711d4740 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c281f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1f58022800 tx=0x7f1f58022e60 comp rx=0 tx=0).stop 2024-05-09T09:24:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 shutdown_connections 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x56468c2f03f0 0x56468c2f2810 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56468c278c40 0x56468c2833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56468c273ad0 0x56468c282470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 --2- 172.21.15.64:0/4199692135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56468c2730e0 0x56468c281f30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 >> 172.21.15.64:0/4199692135 conn(0x56468c26e750 msgr2=0x56468c2716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f477fe640 0 client.28514 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 shutdown_connections 2024-05-09T09:24:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 1 -- 172.21.15.64:0/4199692135 wait complete. 2024-05-09T09:24:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.998+0000 7f1f711d4740 20 client.28514 trim_cache size 0 max 16384 2024-05-09T09:24:14.005 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:24:14.005 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:24:14.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:24:14.006 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:24:14.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1f500a2d60 msgr2=0x7f1f500a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1f500a2d60 0x7f1f500a5220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x56468c274330 tx=0x7f1f60009500 comp rx=0 tx=0).stop 2024-05-09T09:24:14.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f681032f0 msgr2=0x7f1f680755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f681032f0 0x7f1f680755c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c008000 tx=0x7f1f5c032dc0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 shutdown_connections 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f1f500a2d60 0x7f1f500a5220 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f680736c0 0x7f1f68073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:13.999+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f681032f0 0x7f1f680755c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.000+0000 7f1f6d185640 1 --2- 172.21.15.64:0/3245757716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f68101740 0x7f1f68075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.000+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 >> 172.21.15.64:0/3245757716 conn(0x7f1f680fccd0 msgr2=0x7f1f68105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.000+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 shutdown_connections 2024-05-09T09:24:14.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.000+0000 7f1f6d185640 1 -- 172.21.15.64:0/3245757716 wait complete. 2024-05-09T09:24:14.027 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:24:14.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:24:14.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 -- 172.21.15.64:0/1116807530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd0108830 msgr2=0x7fdbd010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 --2- 172.21.15.64:0/1116807530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd0108830 0x7fdbd010ac60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009ca0 tx=0x7fdbc802a8b0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 -- 172.21.15.64:0/1116807530 shutdown_connections 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 --2- 172.21.15.64:0/1116807530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd010db70 0x7fdbd010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 --2- 172.21.15.64:0/1116807530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd010b1a0 0x7fdbd010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 --2- 172.21.15.64:0/1116807530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd0108830 0x7fdbd010ac60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 -- 172.21.15.64:0/1116807530 >> 172.21.15.64:0/1116807530 conn(0x7fdbd006e6d0 msgr2=0x7fdbd0070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 -- 172.21.15.64:0/1116807530 shutdown_connections 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.231+0000 7fdbd5309640 1 -- 172.21.15.64:0/1116807530 wait complete. 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.232+0000 7fdbd5309640 1 Processor -- start 2024-05-09T09:24:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.232+0000 7fdbd5309640 1 -- start start 2024-05-09T09:24:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.232+0000 7fdbd5309640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 0x7fdbd01a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.232+0000 7fdbceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 0x7fdbd01a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.232+0000 7fdbceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 0x7fdbd01a6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:57498/0 (socket says 172.21.15.64:57498) 2024-05-09T09:24:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbd5309640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd010b1a0 0x7fdbd01a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbd5309640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd010db70 0x7fdbd01adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbd5309640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd011c300 con 0x7fdbd010db70 2024-05-09T09:24:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbd5309640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd011c180 con 0x7fdbd0108830 2024-05-09T09:24:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbd5309640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd011c480 con 0x7fdbd010b1a0 2024-05-09T09:24:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbceffd640 1 -- 172.21.15.64:0/2754373002 learned_addr learned my addr 172.21.15.64:0/2754373002 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbce7fc640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd010b1a0 0x7fdbd01a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbcf7fe640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd010db70 0x7fdbd01adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbcf7fe640 1 -- 172.21.15.64:0/2754373002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd010b1a0 msgr2=0x7fdbd01a6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbcf7fe640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd010b1a0 0x7fdbd01a6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbcf7fe640 1 -- 172.21.15.64:0/2754373002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 msgr2=0x7fdbd01a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbcf7fe640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 0x7fdbd01a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.234+0000 7fdbcf7fe640 1 -- 172.21.15.64:0/2754373002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd01ae640 con 0x7fdbd010db70 2024-05-09T09:24:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.235+0000 7fdbceffd640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 0x7fdbd01a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:14.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.235+0000 7fdbcf7fe640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd010db70 0x7fdbd01adfd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b3e0 tx=0x7fdbc400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.235+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4015020 con 0x7fdbd010db70 2024-05-09T09:24:14.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.235+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbc4011ce0 con 0x7fdbd010db70 2024-05-09T09:24:14.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.235+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4010dc0 con 0x7fdbd010db70 2024-05-09T09:24:14.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.236+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd01ae930 con 0x7fdbd010db70 2024-05-09T09:24:14.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.236+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd01aeeb0 con 0x7fdbd010db70 2024-05-09T09:24:14.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.237+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd00711d0 con 0x7fdbd010db70 2024-05-09T09:24:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.239+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbc4018020 con 0x7fdbd010db70 2024-05-09T09:24:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.240+0000 7fdbb7fff640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdbb80a2ce0 0x7fdbb80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.241+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdbc40cea00 con 0x7fdbd010db70 2024-05-09T09:24:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.241+0000 7fdbceffd640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdbb80a2ce0 0x7fdbb80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.242+0000 7fdbceffd640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdbb80a2ce0 0x7fdbb80a51a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009a60 tx=0x7fdbc8030860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.249+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdbc401f020 con 0x7fdbd010db70 2024-05-09T09:24:14.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.708+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd0062430 con 0x7fdbd010db70 2024-05-09T09:24:14.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.709+0000 7fdbb7fff640 1 -- 172.21.15.64:0/2754373002 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11885 (secure 0 0 0) 0x7fdbc409b5d0 con 0x7fdbd010db70 2024-05-09T09:24:14.710 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:24:14.710 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:22:55.565953+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:24:14.710 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 49 2024-05-09T09:24:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.713+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdbb80a2ce0 msgr2=0x7fdbb80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.714+0000 7fdbd5309640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdbb80a2ce0 0x7fdbb80a51a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009a60 tx=0x7fdbc8030860 comp rx=0 tx=0).stop 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.714+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd010db70 msgr2=0x7fdbd01adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.714+0000 7fdbd5309640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd010db70 0x7fdbd01adfd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b3e0 tx=0x7fdbc400b7e0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.715+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 shutdown_connections 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.715+0000 7fdbd5309640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdbb80a2ce0 0x7fdbb80a51a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.715+0000 7fdbd5309640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbd010db70 0x7fdbd01adfd0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.715+0000 7fdbd5309640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd010b1a0 0x7fdbd01a6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.715+0000 7fdbd5309640 1 --2- 172.21.15.64:0/2754373002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbd0108830 0x7fdbd01a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.715+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 >> 172.21.15.64:0/2754373002 conn(0x7fdbd006e6d0 msgr2=0x7fdbd010c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.716+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 shutdown_connections 2024-05-09T09:24:14.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.716+0000 7fdbd5309640 1 -- 172.21.15.64:0/2754373002 wait complete. 2024-05-09T09:24:14.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:24:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.951+0000 7fcf2f59e640 1 -- 172.21.15.64:0/485702008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf30072c70 msgr2=0x7fcf30073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.951+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/485702008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf30072c70 0x7fcf30073090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009860 tx=0x7fcf1802a340 comp rx=0 tx=0).stop 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 -- 172.21.15.64:0/485702008 shutdown_connections 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/485702008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf301098b0 0x7fcf3010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/485702008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf30108f10 0x7fcf30109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/485702008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf30072c70 0x7fcf30073090 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 -- 172.21.15.64:0/485702008 >> 172.21.15.64:0/485702008 conn(0x7fcf3006e740 msgr2=0x7fcf30070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 -- 172.21.15.64:0/485702008 shutdown_connections 2024-05-09T09:24:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.952+0000 7fcf2f59e640 1 -- 172.21.15.64:0/485702008 wait complete. 2024-05-09T09:24:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.953+0000 7fcf2f59e640 1 Processor -- start 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 -- start start 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf30072c70 0x7fcf3019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf30108f10 0x7fcf3019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 0x7fcf301a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30111970 con 0x7fcf30072c70 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301117f0 con 0x7fcf301098b0 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2f59e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30111af0 con 0x7fcf30108f10 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.956+0000 7fcf2ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 0x7fcf301a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 0x7fcf301a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39582/0 (socket says 172.21.15.64:39582) 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 -- 172.21.15.64:0/1178886900 learned_addr learned my addr 172.21.15.64:0/1178886900 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 -- 172.21.15.64:0/1178886900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf30108f10 msgr2=0x7fcf3019e4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf30108f10 0x7fcf3019e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 -- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf30072c70 msgr2=0x7fcf3019df60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf30072c70 0x7fcf3019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 -- 172.21.15.64:0/1178886900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf301a5b80 con 0x7fcf301098b0 2024-05-09T09:24:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.957+0000 7fcf2ed9d640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 0x7fcf301a5510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcf2801fd90 tx=0x7fcf2801d750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.958+0000 7fcf277fe640 1 -- 172.21.15.64:0/1178886900 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2801de60 con 0x7fcf301098b0 2024-05-09T09:24:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.958+0000 7fcf277fe640 1 -- 172.21.15.64:0/1178886900 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf28003970 con 0x7fcf301098b0 2024-05-09T09:24:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.958+0000 7fcf277fe640 1 -- 172.21.15.64:0/1178886900 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28004330 con 0x7fcf301098b0 2024-05-09T09:24:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.958+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf301a5e10 con 0x7fcf301098b0 2024-05-09T09:24:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.958+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301a6280 con 0x7fcf301098b0 2024-05-09T09:24:14.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.962+0000 7fcf277fe640 1 -- 172.21.15.64:0/1178886900 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf2802e020 con 0x7fcf301098b0 2024-05-09T09:24:14.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.964+0000 7fcf37b82740 1 Processor -- start 2024-05-09T09:24:14.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 -- start start 2024-05-09T09:24:14.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32aea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32aef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aefc60 0x55cd32af20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd32a4ec40 con 0x55cd32aea100 2024-05-09T09:24:14.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cd329f6580 con 0x55cd32aefc60 2024-05-09T09:24:14.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf37b82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cd32a4ac00 con 0x55cd32aeaaf0 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32aea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32aef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32aea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56838/0 (socket says 172.21.15.64:56838) 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf27fff640 1 -- 172.21.15.64:0/2044733188 learned_addr learned my addr 172.21.15.64:0/2044733188 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.965+0000 7fcf27fff640 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aefc60 0x55cd32af20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf27fff640 1 -- 172.21.15.64:0/2044733188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 msgr2=0x55cd32aef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf27fff640 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32aef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf27fff640 1 -- 172.21.15.64:0/2044733188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 msgr2=0x55cd32aea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf27fff640 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32aea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf27fff640 1 -- 172.21.15.64:0/2044733188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd327623f0 con 0x55cd32aefc60 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf277fe640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf1c0a2ce0 0x7fcf1c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.966+0000 7fcf27fff640 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aefc60 0x55cd32af20f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009460 tx=0x7fcf20005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abb8a10d1de0ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.967+0000 7fcf24ff9640 1 -- 172.21.15.64:0/2044733188 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200132d0 con 0x55cd32aefc60 2024-05-09T09:24:14.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.967+0000 7fcf24ff9640 1 -- 172.21.15.64:0/2044733188 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf200054d0 con 0x55cd32aefc60 2024-05-09T09:24:14.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.967+0000 7fcf2ed9d640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf1c0a2ce0 0x7fcf1c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.967+0000 7fcf277fe640 1 -- 172.21.15.64:0/1178886900 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcf280dfe20 con 0x7fcf301098b0 2024-05-09T09:24:14.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 -- 172.21.15.64:0/2044733188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aefc60 msgr2=0x55cd32af20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aefc60 0x55cd32af20f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009460 tx=0x7fcf20005f90 comp rx=0 tx=0).stop 2024-05-09T09:24:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 -- 172.21.15.64:0/2044733188 shutdown_connections 2024-05-09T09:24:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aefc60 0x55cd32af20f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32aef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2044733188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32aea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 -- 172.21.15.64:0/2044733188 >> 172.21.15.64:0/2044733188 conn(0x55cd32ae5770 msgr2=0x55cd32ae5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf2ed9d640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf1c0a2ce0 0x7fcf1c0a51a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcf280a8cc0 tx=0x7fcf280a7ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 -- 172.21.15.64:0/2044733188 shutdown_connections 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.968+0000 7fcf37b82740 1 -- 172.21.15.64:0/2044733188 wait complete. 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.969+0000 7fcf37b82740 1 -- start start 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.969+0000 7fcf37b82740 1 Processor -- start 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.969+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aea100 0x55cd32af44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.969+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32af4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.969+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.970+0000 7fcf37b82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd32af2c50 con 0x55cd32aefc60 2024-05-09T09:24:14.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.970+0000 7fcf37b82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cd32a09c40 con 0x55cd32aeaaf0 2024-05-09T09:24:14.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.970+0000 7fcf37b82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cd32af2dd0 con 0x55cd32aea100 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.970+0000 7fcf37b82740 10 client.0 _ll_register_callbacks cb 0x55cd32761be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.970+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32af4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32af4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39602/0 (socket says 172.21.15.64:39602) 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 -- 172.21.15.64:0/1611049036 learned_addr learned my addr 172.21.15.64:0/1611049036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aea100 0x55cd32af44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 -- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aea100 msgr2=0x55cd32af44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aea100 0x55cd32af44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 -- 172.21.15.64:0/1611049036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 msgr2=0x55cd32afba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.971+0000 7fcf2e59c640 1 -- 172.21.15.64:0/1611049036 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd3282c7d0 con 0x55cd32aeaaf0 2024-05-09T09:24:14.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32af4a10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009860 tx=0x7fcf18005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16efd3db5af032e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1611049036 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18040070 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1611049036 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcf18030780 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf200054d0 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1611049036 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcf1804c050 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.972+0000 7fcf37b82740 10 client.28529 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cd32afee60 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1611049036 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf200054d0 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1611049036 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fcf18004750 con 0x55cd32aeaaf0 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 10 client.28529 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 10 client.28529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 20 client.28529 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b005d0 0x55cd32b02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:14.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 4 client.28529 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:14.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd32a5c370 con 0x55cd32b005d0 2024-05-09T09:24:14.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.973+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b005d0 0x55cd32b02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:14.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.974+0000 7fcf15ffb640 10 client.28529 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:14.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.975+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b005d0 0x55cd32b02a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9df79de55f62826b server_cookie=d6d356fd3e9e4aa0 in_seq=0 out_seq=0 2024-05-09T09:24:14.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.976+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1611049036 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd32a5c370 con 0x55cd32b005d0 2024-05-09T09:24:14.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.976+0000 7fcf15ffb640 10 client.28529 handle_command_reply: tid=0 2024-05-09T09:24:14.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.976+0000 7fcf37b82740 1 client.28529 shutdown 2024-05-09T09:24:14.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.976+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 msgr2=0x55cd32af4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:14.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.976+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32af4a10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009860 tx=0x7fcf18005270 comp rx=0 tx=0).stop 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 shutdown_connections 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b005d0 0x55cd32b02a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32af4a10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1611049036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aea100 0x55cd32af44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 >> 172.21.15.64:0/1611049036 conn(0x55cd32ae5770 msgr2=0x55cd32ae4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf15ffb640 0 client.28529 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 shutdown_connections 2024-05-09T09:24:14.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 1 -- 172.21.15.64:0/1611049036 wait complete. 2024-05-09T09:24:14.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:14.979+0000 7fcf37b82740 20 client.28529 trim_cache size 0 max 16384 2024-05-09T09:24:15.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 Processor -- start 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 -- start start 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32ae5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32afba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aefc60 0x55cd32af37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10012500 con 0x55cd32aea100 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1000fd70 con 0x55cd32aeaaf0 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.007+0000 7fcf37b82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1000dbc0 con 0x55cd32aefc60 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.008+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32afba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.008+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32afba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39618/0 (socket says 172.21.15.64:39618) 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.008+0000 7fcf2e59c640 1 -- 172.21.15.64:0/2974577865 learned_addr learned my addr 172.21.15.64:0/2974577865 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.008+0000 7fcf27fff640 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aefc60 0x55cd32af37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.008+0000 7fcf27fff640 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32ae5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.009+0000 7fcf2e59c640 1 -- 172.21.15.64:0/2974577865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aefc60 msgr2=0x55cd32af37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.009+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aefc60 0x55cd32af37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.009+0000 7fcf2e59c640 1 -- 172.21.15.64:0/2974577865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 msgr2=0x55cd32ae5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.009+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32ae5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.009+0000 7fcf2e59c640 1 -- 172.21.15.64:0/2974577865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd32aeaed0 con 0x55cd32aeaaf0 2024-05-09T09:24:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.010+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32afba10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcf1802a860 tx=0x7fcf18005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d3d04b9ad2df2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.010+0000 7fcf17fff640 1 -- 172.21.15.64:0/2974577865 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18040050 con 0x55cd32aeaaf0 2024-05-09T09:24:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.010+0000 7fcf17fff640 1 -- 172.21.15.64:0/2974577865 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf18009450 con 0x55cd32aeaaf0 2024-05-09T09:24:15.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.010+0000 7fcf17fff640 1 -- 172.21.15.64:0/2974577865 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18037070 con 0x55cd32aeaaf0 2024-05-09T09:24:15.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 -- 172.21.15.64:0/2974577865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 msgr2=0x55cd32afba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32afba10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcf1802a860 tx=0x7fcf18005d50 comp rx=0 tx=0).stop 2024-05-09T09:24:15.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 -- 172.21.15.64:0/2974577865 shutdown_connections 2024-05-09T09:24:15.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aefc60 0x55cd32af37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aeaaf0 0x55cd32afba10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 --2- 172.21.15.64:0/2974577865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aea100 0x55cd32ae5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.011+0000 7fcf37b82740 1 -- 172.21.15.64:0/2974577865 >> 172.21.15.64:0/2974577865 conn(0x55cd32ae5770 msgr2=0x55cd32aef3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.012+0000 7fcf37b82740 1 -- 172.21.15.64:0/2974577865 shutdown_connections 2024-05-09T09:24:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.012+0000 7fcf37b82740 1 -- 172.21.15.64:0/2974577865 wait complete. 2024-05-09T09:24:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.012+0000 7fcf37b82740 1 -- start start 2024-05-09T09:24:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.012+0000 7fcf37b82740 1 Processor -- start 2024-05-09T09:24:15.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 0x55cd32af8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32af9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1000e230 con 0x55cd32aefc60 2024-05-09T09:24:15.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1000f6a0 con 0x55cd32aea100 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1000e900 con 0x55cd32aeaaf0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.013+0000 7fcf37b82740 10 client.0 _ll_register_callbacks cb 0x7fcf18035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.015+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32af9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.015+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 0x55cd32af8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.015+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 0x55cd32af8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39624/0 (socket says 172.21.15.64:39624) 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.015+0000 7fcf27fff640 1 -- 172.21.15.64:0/1314109433 learned_addr learned my addr 172.21.15.64:0/1314109433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.015+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.016+0000 7fcf27fff640 1 -- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 msgr2=0x55cd32af9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.016+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32af9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.016+0000 7fcf27fff640 1 -- 172.21.15.64:0/1314109433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 msgr2=0x55cd32afa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.016+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.016+0000 7fcf27fff640 1 -- 172.21.15.64:0/1314109433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd32afa940 con 0x55cd32aea100 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.017+0000 7fcf27fff640 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 0x55cd32af8e00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000c00 tx=0x7fcf20021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a76da8f9032cfbe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.017+0000 7fcf15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:15.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.018+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1314109433 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20027dc0 con 0x55cd32aea100 2024-05-09T09:24:15.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.018+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1314109433 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcf200107a0 con 0x55cd32aea100 2024-05-09T09:24:15.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.018+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf18009450 con 0x55cd32aea100 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.020+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1314109433 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcf200109a0 con 0x55cd32aea100 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.020+0000 7fcf37b82740 10 client.28535 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.020+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55cd32afbc40 con 0x55cd32aea100 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1314109433 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf18009450 con 0x55cd32aea100 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1314109433 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fcf20027340 con 0x55cd32aea100 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf37b82740 10 client.28535 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf37b82740 10 client.28535 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:24:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf37b82740 20 client.28535 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b67190 0x55cd32b695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf37b82740 4 client.28535 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:24:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.021+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd32a566c0 con 0x55cd32b67190 2024-05-09T09:24:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.024+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b67190 0x55cd32b695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.028+0000 7fcf2e59c640 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b67190 0x55cd32b695b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f61720d6f88423e server_cookie=9cb6b49b4b3e33be in_seq=0 out_seq=0 2024-05-09T09:24:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.028+0000 7fcf15ffb640 10 client.28535 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.028+0000 7fcf15ffb640 1 -- 172.21.15.64:0/1314109433 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd32a566c0 con 0x55cd32b67190 2024-05-09T09:24:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.028+0000 7fcf15ffb640 10 client.28535 handle_command_reply: tid=0 2024-05-09T09:24:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 client.28535 shutdown 2024-05-09T09:24:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 msgr2=0x55cd32af8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 0x55cd32af8e00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000c00 tx=0x7fcf20021040 comp rx=0 tx=0).stop 2024-05-09T09:24:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 shutdown_connections 2024-05-09T09:24:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55cd32b67190 0x55cd32b695b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd32aefc60 0x55cd32afa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd32aeaaf0 0x55cd32af9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 --2- 172.21.15.64:0/1314109433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd32aea100 0x55cd32af8e00 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000c00 tx=0x7fcf20021040 comp rx=0 tx=0).stop 2024-05-09T09:24:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 >> 172.21.15.64:0/1314109433 conn(0x55cd32ae5770 msgr2=0x55cd32aeefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf15ffb640 0 client.28535 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:15.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.029+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 shutdown_connections 2024-05-09T09:24:15.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.030+0000 7fcf37b82740 1 -- 172.21.15.64:0/1314109433 wait complete. 2024-05-09T09:24:15.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.030+0000 7fcf37b82740 20 client.28535 trim_cache size 0 max 16384 2024-05-09T09:24:15.034 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:24:15.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf1c0a2ce0 msgr2=0x7fcf1c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf1c0a2ce0 0x7fcf1c0a51a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcf280a8cc0 tx=0x7fcf280a7ef0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 msgr2=0x7fcf301a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 0x7fcf301a5510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcf2801fd90 tx=0x7fcf2801d750 comp rx=0 tx=0).stop 2024-05-09T09:24:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 shutdown_connections 2024-05-09T09:24:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcf1c0a2ce0 0x7fcf1c0a51a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf301098b0 0x7fcf301a5510 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.031+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf30108f10 0x7fcf3019e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.032+0000 7fcf2f59e640 1 --2- 172.21.15.64:0/1178886900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf30072c70 0x7fcf3019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.032+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 >> 172.21.15.64:0/1178886900 conn(0x7fcf3006e740 msgr2=0x7fcf3010c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.032+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 shutdown_connections 2024-05-09T09:24:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.032+0000 7fcf2f59e640 1 -- 172.21.15.64:0/1178886900 wait complete. 2024-05-09T09:24:15.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:24:15.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.251+0000 7f1661272640 1 -- 172.21.15.64:0/4249454572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10db70 msgr2=0x7f165c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.251+0000 7f1661272640 1 --2- 172.21.15.64:0/4249454572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10db70 0x7f165c10e020 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1654009be0 tx=0x7f165402a6d0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 -- 172.21.15.64:0/4249454572 shutdown_connections 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 --2- 172.21.15.64:0/4249454572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10db70 0x7f165c10e020 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 --2- 172.21.15.64:0/4249454572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f165c10b1a0 0x7f165c10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 --2- 172.21.15.64:0/4249454572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c108830 0x7f165c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 -- 172.21.15.64:0/4249454572 >> 172.21.15.64:0/4249454572 conn(0x7f165c06e6c0 msgr2=0x7f165c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 -- 172.21.15.64:0/4249454572 shutdown_connections 2024-05-09T09:24:15.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.259+0000 7f1661272640 1 -- 172.21.15.64:0/4249454572 wait complete. 2024-05-09T09:24:15.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 Processor -- start 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 -- start start 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f165c108830 0x7f165c1a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10b1a0 0x7f165c1a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 0x7f165c1ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f165c11c1e0 con 0x7f165c108830 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f165c11c060 con 0x7f165c10b1a0 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.260+0000 7f1661272640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f165c11c360 con 0x7f165c10db70 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.261+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 0x7f165c1ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.261+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 0x7f165c1ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45730/0 (socket says 172.21.15.64:45730) 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.261+0000 7f165b7fe640 1 -- 172.21.15.64:0/4235937978 learned_addr learned my addr 172.21.15.64:0/4235937978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.261+0000 7f165a7fc640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10b1a0 0x7f165c1a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.261+0000 7f165affd640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f165c108830 0x7f165c1a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.262+0000 7f165b7fe640 1 -- 172.21.15.64:0/4235937978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10b1a0 msgr2=0x7f165c1a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.262+0000 7f165b7fe640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10b1a0 0x7f165c1a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.262+0000 7f165b7fe640 1 -- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f165c108830 msgr2=0x7f165c1a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.262+0000 7f165b7fe640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f165c108830 0x7f165c1a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.262+0000 7f165b7fe640 1 -- 172.21.15.64:0/4235937978 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c1ae470 con 0x7f165c10db70 2024-05-09T09:24:15.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.263+0000 7f165b7fe640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 0x7f165c1ade00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f165402abf0 tx=0x7f1654030f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.263+0000 7f1643fff640 1 -- 172.21.15.64:0/4235937978 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654041020 con 0x7f165c10db70 2024-05-09T09:24:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.264+0000 7f1643fff640 1 -- 172.21.15.64:0/4235937978 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1654037040 con 0x7f165c10db70 2024-05-09T09:24:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.264+0000 7f1643fff640 1 -- 172.21.15.64:0/4235937978 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654036070 con 0x7f165c10db70 2024-05-09T09:24:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.264+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c1ae700 con 0x7f165c10db70 2024-05-09T09:24:15.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.264+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c1aeba0 con 0x7f165c10db70 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.268+0000 7f1663f7f740 1 Processor -- start 2024-05-09T09:24:15.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.268+0000 7f1643fff640 1 -- 172.21.15.64:0/4235937978 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1654044020 con 0x7f165c10db70 2024-05-09T09:24:15.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 -- start start 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1643fff640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16440a2d60 0x7f16440a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345773360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f345778430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f34577af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f3456d7c40 con 0x55f345772f40 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f34567f580 con 0x55f345773930 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1663f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f3456d3c00 con 0x55f345778aa0 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f1643fff640 1 -- 172.21.15.64:0/4235937978 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f16540efd50 con 0x7f165c10db70 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f165affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345773360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f165affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345773360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56892/0 (socket says 172.21.15.64:56892) 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f165affd640 1 -- 172.21.15.64:0/887154986 learned_addr learned my addr 172.21.15.64:0/887154986 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.270+0000 7f165b7fe640 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f34577af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165a7fc640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16440a2d60 0x7f16440a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165affd640 1 -- 172.21.15.64:0/887154986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 msgr2=0x55f34577af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165affd640 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f34577af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165affd640 1 -- 172.21.15.64:0/887154986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 msgr2=0x55f345778430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165affd640 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f345778430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165affd640 1 -- 172.21.15.64:0/887154986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3453eb3f0 con 0x55f345772f40 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165b7fe640 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f34577af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f165affd640 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345773360 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f164c009d20 tx=0x7f164c0227c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517429b4fced0778 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.271+0000 7f16417fa640 1 -- 172.21.15.64:0/887154986 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c02f070 con 0x55f345772f40 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.272+0000 7f165a7fc640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16440a2d60 0x7f16440a5220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55f345773d30 tx=0x7f1650005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.272+0000 7f16417fa640 1 -- 172.21.15.64:0/887154986 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f164c009e10 con 0x55f345772f40 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.272+0000 7f16417fa640 1 -- 172.21.15.64:0/887154986 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c00f840 con 0x55f345772f40 2024-05-09T09:24:15.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.273+0000 7f1663f7f740 1 -- 172.21.15.64:0/887154986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 msgr2=0x55f345773360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.273+0000 7f1663f7f740 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345773360 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f164c009d20 tx=0x7f164c0227c0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.276+0000 7f1663f7f740 1 -- 172.21.15.64:0/887154986 shutdown_connections 2024-05-09T09:24:15.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.276+0000 7f1663f7f740 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f34577af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.276+0000 7f1663f7f740 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f345778430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.276+0000 7f1663f7f740 1 --2- 172.21.15.64:0/887154986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345773360 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.276+0000 7f1663f7f740 1 -- 172.21.15.64:0/887154986 >> 172.21.15.64:0/887154986 conn(0x55f34576e5b0 msgr2=0x55f34576e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.278+0000 7f1663f7f740 1 -- 172.21.15.64:0/887154986 shutdown_connections 2024-05-09T09:24:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.278+0000 7f1663f7f740 1 -- 172.21.15.64:0/887154986 wait complete. 2024-05-09T09:24:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 -- start start 2024-05-09T09:24:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 Processor -- start 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f34577d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f34577d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f3457848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f34577ba50 con 0x55f345772f40 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f345692c40 con 0x55f345773930 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f34577bbd0 con 0x55f345778aa0 2024-05-09T09:24:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.279+0000 7f1663f7f740 10 client.0 _ll_register_callbacks cb 0x55f3453eabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f34577d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f34577d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56902/0 (socket says 172.21.15.64:56902) 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165affd640 1 -- 172.21.15.64:0/261300612 learned_addr learned my addr 172.21.15.64:0/261300612 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165a7fc640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f34577d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165b7fe640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f3457848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165a7fc640 1 -- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 msgr2=0x55f3457848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165a7fc640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f3457848e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165a7fc640 1 -- 172.21.15.64:0/261300612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 msgr2=0x55f34577d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165a7fc640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f34577d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.280+0000 7f165a7fc640 1 -- 172.21.15.64:0/261300612 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3454b57d0 con 0x55f345773930 2024-05-09T09:24:15.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.281+0000 7f165a7fc640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f34577d8a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55f3457743a0 tx=0x7f1650004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b206832d4b04f774 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.281+0000 7f163eff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.281+0000 7f163eff5640 1 -- 172.21.15.64:0/261300612 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165000fa20 con 0x55f345773930 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.281+0000 7f163eff5640 1 -- 172.21.15.64:0/261300612 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f165000fbc0 con 0x55f345773930 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.281+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f164c009e10 con 0x55f345773930 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f163eff5640 1 -- 172.21.15.64:0/261300612 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f165000fe00 con 0x55f345773930 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f1663f7f740 10 client.28544 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f345787c60 con 0x55f345773930 2024-05-09T09:24:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f163eff5640 1 -- 172.21.15.64:0/261300612 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f164c009e10 con 0x55f345773930 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f163eff5640 1 -- 172.21.15.64:0/261300612 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f165001f540 con 0x55f345773930 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f1663f7f740 10 client.28544 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f1663f7f740 10 client.28544 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f1663f7f740 20 client.28544 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.282+0000 7f1663f7f740 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f345789340 0x55f34578b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.283+0000 7f1663f7f740 4 client.28544 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.283+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3456e5370 con 0x55f345789340 2024-05-09T09:24:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.283+0000 7f165affd640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f345789340 0x55f34578b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.283+0000 7f163eff5640 10 client.28544 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.283+0000 7f165affd640 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f345789340 0x55f34578b780 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de6632bc1584949b server_cookie=9d0e63be4335bd36 in_seq=0 out_seq=0 2024-05-09T09:24:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.284+0000 7f163eff5640 1 -- 172.21.15.64:0/261300612 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3456e5370 con 0x55f345789340 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.284+0000 7f163eff5640 10 client.28544 handle_command_reply: tid=0 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.284+0000 7f1663f7f740 1 client.28544 shutdown 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.284+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 msgr2=0x55f34577d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.284+0000 7f1663f7f740 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f34577d8a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55f3457743a0 tx=0x7f1650004730 comp rx=0 tx=0).stop 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 shutdown_connections 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f3457848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f34577d8a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f345789340 0x55f34578b780 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 --2- 172.21.15.64:0/261300612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f34577d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 >> 172.21.15.64:0/261300612 conn(0x55f34576e5b0 msgr2=0x55f34576d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f163eff5640 0 client.28544 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.285+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 shutdown_connections 2024-05-09T09:24:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.286+0000 7f1663f7f740 1 -- 172.21.15.64:0/261300612 wait complete. 2024-05-09T09:24:15.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.286+0000 7f1663f7f740 20 client.28544 trim_cache size 0 max 16384 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 Processor -- start 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 -- start start 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345771530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f3457848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f345771a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16480124e0 con 0x55f345772f40 2024-05-09T09:24:15.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f164800fd50 con 0x55f345773930 2024-05-09T09:24:15.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.314+0000 7f1663f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f164800dba0 con 0x55f345778aa0 2024-05-09T09:24:15.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345771530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345771530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56918/0 (socket says 172.21.15.64:56918) 2024-05-09T09:24:15.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 -- 172.21.15.64:0/1211108085 learned_addr learned my addr 172.21.15.64:0/1211108085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 -- 172.21.15.64:0/1211108085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 msgr2=0x55f345771a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f345771a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 -- 172.21.15.64:0/1211108085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 msgr2=0x55f3457848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f3457848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 -- 172.21.15.64:0/1211108085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34578be00 con 0x55f345772f40 2024-05-09T09:24:15.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.315+0000 7f165affd640 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345771530 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f164c022cc0 tx=0x7f164c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524bb23006116ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.316+0000 7f1640ff9640 1 -- 172.21.15.64:0/1211108085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c040070 con 0x55f345772f40 2024-05-09T09:24:15.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.316+0000 7f1640ff9640 1 -- 172.21.15.64:0/1211108085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f164c030040 con 0x55f345772f40 2024-05-09T09:24:15.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.317+0000 7f1663f7f740 1 -- 172.21.15.64:0/1211108085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 msgr2=0x55f345771530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.317+0000 7f1663f7f740 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345771530 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f164c022cc0 tx=0x7f164c0098f0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 -- 172.21.15.64:0/1211108085 shutdown_connections 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345778aa0 0x55f345771a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345773930 0x55f3457848a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 --2- 172.21.15.64:0/1211108085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345772f40 0x55f345771530 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 -- 172.21.15.64:0/1211108085 >> 172.21.15.64:0/1211108085 conn(0x55f34576e5b0 msgr2=0x55f345774cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 -- 172.21.15.64:0/1211108085 shutdown_connections 2024-05-09T09:24:15.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.321+0000 7f1663f7f740 1 -- 172.21.15.64:0/1211108085 wait complete. 2024-05-09T09:24:15.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 -- start start 2024-05-09T09:24:15.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 Processor -- start 2024-05-09T09:24:15.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:15 smithi139 ceph-mon[33599]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 805 op/s 2024-05-09T09:24:15.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:15 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2754373002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:15.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345772f40 0x55f345782010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345773930 0x55f345782550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 0x55f345783510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f164800e210 con 0x55f345778aa0 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f164800f680 con 0x55f345772f40 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f164800e8e0 con 0x55f345773930 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.322+0000 7f1663f7f740 10 client.0 _ll_register_callbacks cb 0x7f1650009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.323+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 0x55f345783510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.323+0000 7f165b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 0x55f345783510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56920/0 (socket says 172.21.15.64:56920) 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.323+0000 7f165b7fe640 1 -- 172.21.15.64:0/431558708 learned_addr learned my addr 172.21.15.64:0/431558708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.323+0000 7f165b7fe640 1 -- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345773930 msgr2=0x55f345782550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.324+0000 7f165b7fe640 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345773930 0x55f345782550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.324+0000 7f165b7fe640 1 -- 172.21.15.64:0/431558708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345772f40 msgr2=0x55f345782010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.324+0000 7f165b7fe640 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345772f40 0x55f345782010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.324+0000 7f165b7fe640 1 -- 172.21.15.64:0/431558708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f345783b00 con 0x55f345778aa0 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.324+0000 7f165b7fe640 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 0x55f345783510 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f16540b86a0 tx=0x7f16540ba750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc842c07e0c044ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.324+0000 7f163eff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.325+0000 7f163eff5640 1 -- 172.21.15.64:0/431558708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16540bc940 con 0x55f345778aa0 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.325+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f164c030040 con 0x55f345778aa0 2024-05-09T09:24:15.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.325+0000 7f163eff5640 1 -- 172.21.15.64:0/431558708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f16540b88b0 con 0x55f345778aa0 2024-05-09T09:24:15.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.329+0000 7f163eff5640 1 -- 172.21.15.64:0/431558708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f165404d580 con 0x55f345778aa0 2024-05-09T09:24:15.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.329+0000 7f1663f7f740 10 client.19077 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:15.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.329+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f3457ef9d0 con 0x55f345778aa0 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.331+0000 7f163eff5640 1 -- 172.21.15.64:0/431558708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f164c030040 con 0x55f345778aa0 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.331+0000 7f163eff5640 1 -- 172.21.15.64:0/431558708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f165404e060 con 0x55f345778aa0 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.332+0000 7f1663f7f740 10 client.19077 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.332+0000 7f1663f7f740 10 client.19077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.332+0000 7f1663f7f740 20 client.19077 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.332+0000 7f1663f7f740 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f3457f03d0 0x55f3457f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.332+0000 7f1663f7f740 4 client.19077 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:24:15.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.332+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3456df6c0 con 0x55f3457f03d0 2024-05-09T09:24:15.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.337+0000 7f165affd640 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f3457f03d0 0x55f3457f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.341+0000 7f165affd640 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f3457f03d0 0x55f3457f27f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dd08160107a8a1a server_cookie=85a7462dbb45e64a in_seq=0 out_seq=0 2024-05-09T09:24:15.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.342+0000 7f163eff5640 10 client.19077 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:15.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.342+0000 7f163eff5640 1 -- 172.21.15.64:0/431558708 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3456df6c0 con 0x55f3457f03d0 2024-05-09T09:24:15.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.342+0000 7f163eff5640 10 client.19077 handle_command_reply: tid=0 2024-05-09T09:24:15.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.342+0000 7f1663f7f740 1 client.19077 shutdown 2024-05-09T09:24:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.342+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 msgr2=0x55f345783510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.342+0000 7f1663f7f740 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 0x55f345783510 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f16540b86a0 tx=0x7f16540ba750 comp rx=0 tx=0).stop 2024-05-09T09:24:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 shutdown_connections 2024-05-09T09:24:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f1663f7f740 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55f3457f03d0 0x55f3457f27f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f1663f7f740 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f345778aa0 0x55f345783510 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f1663f7f740 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f345773930 0x55f345782550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f1663f7f740 1 --2- 172.21.15.64:0/431558708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f345772f40 0x55f345782010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 >> 172.21.15.64:0/431558708 conn(0x55f34576e5b0 msgr2=0x55f34576cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.344+0000 7f163eff5640 0 client.19077 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:15.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.345+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 shutdown_connections 2024-05-09T09:24:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.349+0000 7f1663f7f740 1 -- 172.21.15.64:0/431558708 wait complete. 2024-05-09T09:24:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.349+0000 7f1663f7f740 20 client.19077 trim_cache size 0 max 16384 2024-05-09T09:24:15.352 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:24:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16440a2d60 msgr2=0x7f16440a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16440a2d60 0x7f16440a5220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55f345773d30 tx=0x7f1650005f90 comp rx=0 tx=0).stop 2024-05-09T09:24:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 msgr2=0x7f165c1ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 0x7f165c1ade00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f165402abf0 tx=0x7f1654030f10 comp rx=0 tx=0).stop 2024-05-09T09:24:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 shutdown_connections 2024-05-09T09:24:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16440a2d60 0x7f16440a5220 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f165c10db70 0x7f165c1ade00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f165c10b1a0 0x7f165c1a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 --2- 172.21.15.64:0/4235937978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f165c108830 0x7f165c1a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.353+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 >> 172.21.15.64:0/4235937978 conn(0x7f165c06e6c0 msgr2=0x7f165c109e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.356+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 shutdown_connections 2024-05-09T09:24:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.357+0000 7f1661272640 1 -- 172.21.15.64:0/4235937978 wait complete. 2024-05-09T09:24:15.378 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:15 smithi064 ceph-mon[30149]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 805 op/s 2024-05-09T09:24:15.378 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:15 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2754373002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:15.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:24:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:15 smithi179 ceph-mon[34086]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 805 op/s 2024-05-09T09:24:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:15 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2754373002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.586+0000 7f61387a2640 1 -- 172.21.15.64:0/1845247008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130100840 msgr2=0x7f6130100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.586+0000 7f61387a2640 1 --2- 172.21.15.64:0/1845247008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130100840 0x7f6130100ca0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6120009860 tx=0x7f612002a380 comp rx=0 tx=0).stop 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.586+0000 7f61387a2640 1 -- 172.21.15.64:0/1845247008 shutdown_connections 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.586+0000 7f61387a2640 1 --2- 172.21.15.64:0/1845247008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61301011e0 0x7f6130105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.586+0000 7f61387a2640 1 --2- 172.21.15.64:0/1845247008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130100840 0x7f6130100ca0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.586+0000 7f61387a2640 1 --2- 172.21.15.64:0/1845247008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300ff710 0x7f61300ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.587+0000 7f61387a2640 1 -- 172.21.15.64:0/1845247008 >> 172.21.15.64:0/1845247008 conn(0x7f61300fb030 msgr2=0x7f61300fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.587+0000 7f61387a2640 1 -- 172.21.15.64:0/1845247008 shutdown_connections 2024-05-09T09:24:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.587+0000 7f61387a2640 1 -- 172.21.15.64:0/1845247008 wait complete. 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.588+0000 7f61387a2640 1 Processor -- start 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.588+0000 7f61387a2640 1 -- start start 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f61387a2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61300ff710 0x7f6130195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f61387a2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6130100840 0x7f6130195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f61387a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 0x7f613019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f61387a2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61301092f0 con 0x7f61300ff710 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f61387a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6130109170 con 0x7f61301011e0 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f61387a2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6130109470 con 0x7f6130100840 2024-05-09T09:24:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f613653e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61300ff710 0x7f6130195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f6136d3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 0x7f613019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f6136d3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 0x7f613019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39680/0 (socket says 172.21.15.64:39680) 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.589+0000 7f6136d3f640 1 -- 172.21.15.64:0/3597424287 learned_addr learned my addr 172.21.15.64:0/3597424287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f6135d3d640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6130100840 0x7f6130195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f6136d3f640 1 -- 172.21.15.64:0/3597424287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6130100840 msgr2=0x7f6130195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6130100840 0x7f6130195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f6136d3f640 1 -- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61300ff710 msgr2=0x7f6130195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61300ff710 0x7f6130195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f6136d3f640 1 -- 172.21.15.64:0/3597424287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613019d550 con 0x7f61301011e0 2024-05-09T09:24:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.590+0000 7f613653e640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61300ff710 0x7f6130195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:15.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.592+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 0x7f613019cee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f612801fd00 tx=0x7f612801fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.592+0000 7f61277fe640 1 -- 172.21.15.64:0/3597424287 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612801dbd0 con 0x7f61301011e0 2024-05-09T09:24:15.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.592+0000 7f61277fe640 1 -- 172.21.15.64:0/3597424287 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6128003e20 con 0x7f61301011e0 2024-05-09T09:24:15.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.592+0000 7f61277fe640 1 -- 172.21.15.64:0/3597424287 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128028440 con 0x7f61301011e0 2024-05-09T09:24:15.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.593+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613019d840 con 0x7f61301011e0 2024-05-09T09:24:15.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.593+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613019dd10 con 0x7f61301011e0 2024-05-09T09:24:15.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.597+0000 7f613b590740 1 Processor -- start 2024-05-09T09:24:15.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.600+0000 7f61277fe640 1 -- 172.21.15.64:0/3597424287 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6128003970 con 0x7f61301011e0 2024-05-09T09:24:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.601+0000 7f613b590740 1 -- start start 2024-05-09T09:24:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.601+0000 7f61277fe640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f611c0a2c60 0x7f611c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.601+0000 7f61277fe640 1 -- 172.21.15.64:0/3597424287 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f61280e0190 con 0x7f61301011e0 2024-05-09T09:24:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.601+0000 7f613b590740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c7a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.601+0000 7f613b590740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7a930 0x558de6c7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.601+0000 7f613b590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c81f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f613b590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558de6bdec40 con 0x558de6c7faa0 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f613b590740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558de6b86580 con 0x558de6c7a930 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f613b590740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558de6bdac00 con 0x558de6c79f40 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f613653e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c7a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f6136d3f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c81f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f613653e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c7a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45782/0 (socket says 172.21.15.64:45782) 2024-05-09T09:24:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.602+0000 7f613653e640 1 -- 172.21.15.64:0/1353832319 learned_addr learned my addr 172.21.15.64:0/1353832319 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.603+0000 7f6136d3f640 1 -- 172.21.15.64:0/1353832319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 msgr2=0x558de6c7a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.603+0000 7f6136d3f640 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c7a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.603+0000 7f6136d3f640 1 -- 172.21.15.64:0/1353832319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7a930 msgr2=0x558de6c7f430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.603+0000 7f6136d3f640 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7a930 0x558de6c7f430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.603+0000 7f6136d3f640 1 -- 172.21.15.64:0/1353832319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de68f23f0 con 0x558de6c7faa0 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.603+0000 7f6136d3f640 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c81f30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f612801fcd0 tx=0x7f612801a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6421338feefd04ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.604+0000 7f6124ff9640 1 -- 172.21.15.64:0/1353832319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280ac660 con 0x558de6c7faa0 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.604+0000 7f6124ff9640 1 -- 172.21.15.64:0/1353832319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61280b0a10 con 0x558de6c7faa0 2024-05-09T09:24:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.604+0000 7f6124ff9640 1 -- 172.21.15.64:0/1353832319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280a8cd0 con 0x558de6c7faa0 2024-05-09T09:24:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 -- 172.21.15.64:0/1353832319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 msgr2=0x558de6c81f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c81f30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f612801fcd0 tx=0x7f612801a490 comp rx=0 tx=0).stop 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f6135d3d640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f611c0a2c60 0x7f611c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 -- 172.21.15.64:0/1353832319 shutdown_connections 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c81f30 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7a930 0x558de6c7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 --2- 172.21.15.64:0/1353832319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c7a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.605+0000 7f613b590740 1 -- 172.21.15.64:0/1353832319 >> 172.21.15.64:0/1353832319 conn(0x558de6c755b0 msgr2=0x558de6c759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.607+0000 7f613b590740 1 -- 172.21.15.64:0/1353832319 shutdown_connections 2024-05-09T09:24:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.607+0000 7f6135d3d640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f611c0a2c60 0x7f611c0a5120 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f612002a970 tx=0x7f6120009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.607+0000 7f613b590740 1 -- 172.21.15.64:0/1353832319 wait complete. 2024-05-09T09:24:15.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.607+0000 7f613b590740 1 -- start start 2024-05-09T09:24:15.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.607+0000 7f613b590740 1 Processor -- start 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c84370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558de6c829f0 con 0x558de6c7a930 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558de6b99c40 con 0x558de6c7faa0 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558de6c82b70 con 0x558de6c79f40 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613b590740 10 client.0 _ll_register_callbacks cb 0x558de68f1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613653e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c84370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613653e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c84370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45784/0 (socket says 172.21.15.64:45784) 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f613653e640 1 -- 172.21.15.64:0/3108281030 learned_addr learned my addr 172.21.15.64:0/3108281030 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.608+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6135d3d640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6136d3f640 1 -- 172.21.15.64:0/3108281030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 msgr2=0x558de6c84370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c84370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6136d3f640 1 -- 172.21.15.64:0/3108281030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 msgr2=0x558de6c848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c848b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6136d3f640 1 -- 172.21.15.64:0/3108281030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de69bc7d0 con 0x558de6c7faa0 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6136d3f640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c8b8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f612801d6f0 tx=0x7f61280970d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3333e05258fd529 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.609+0000 7f6119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.610+0000 7f6119ffb640 1 -- 172.21.15.64:0/3108281030 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280c4070 con 0x558de6c7faa0 2024-05-09T09:24:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.610+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61280b0a10 con 0x558de6c7faa0 2024-05-09T09:24:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.610+0000 7f6119ffb640 1 -- 172.21.15.64:0/3108281030 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f61280b5050 con 0x558de6c7faa0 2024-05-09T09:24:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.610+0000 7f6119ffb640 1 -- 172.21.15.64:0/3108281030 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f61280ac660 con 0x558de6c7faa0 2024-05-09T09:24:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.610+0000 7f613b590740 10 client.28553 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.610+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558de6c9fc70 con 0x558de6c7faa0 2024-05-09T09:24:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f6119ffb640 1 -- 172.21.15.64:0/3108281030 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61280b0a10 con 0x558de6c7faa0 2024-05-09T09:24:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f6119ffb640 1 -- 172.21.15.64:0/3108281030 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f61280ab520 con 0x558de6c7faa0 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f613b590740 10 client.28553 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f613b590740 10 client.28553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f613b590740 20 client.28553 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f613b590740 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6ca1530 0x558de6ca3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f613b590740 4 client.28553 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.611+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558de6bec370 con 0x558de6ca1530 2024-05-09T09:24:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.613+0000 7f613653e640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6ca1530 0x558de6ca3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.614+0000 7f613653e640 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6ca1530 0x558de6ca3970 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8ea989c1fefed6 server_cookie=1496fab9b5f1fe8a in_seq=0 out_seq=0 2024-05-09T09:24:15.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.615+0000 7f6119ffb640 10 client.28553 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.615+0000 7f6119ffb640 1 -- 172.21.15.64:0/3108281030 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558de6bec370 con 0x558de6ca1530 2024-05-09T09:24:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.616+0000 7f6119ffb640 10 client.28553 handle_command_reply: tid=0 2024-05-09T09:24:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.616+0000 7f613b590740 1 client.28553 shutdown 2024-05-09T09:24:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.616+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 msgr2=0x558de6c8b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.616+0000 7f613b590740 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c8b8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f612801d6f0 tx=0x7f61280970d0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 shutdown_connections 2024-05-09T09:24:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f613b590740 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6ca1530 0x558de6ca3970 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f613b590740 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c8b8f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f613b590740 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c848b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f613b590740 1 --2- 172.21.15.64:0/3108281030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c84370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 >> 172.21.15.64:0/3108281030 conn(0x558de6c755b0 msgr2=0x558de6c747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.617+0000 7f6119ffb640 0 client.28553 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.619+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 shutdown_connections 2024-05-09T09:24:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.619+0000 7f613b590740 1 -- 172.21.15.64:0/3108281030 wait complete. 2024-05-09T09:24:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.621+0000 7f613b590740 20 client.28553 trim_cache size 0 max 16384 2024-05-09T09:24:15.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.654+0000 7f613b590740 1 Processor -- start 2024-05-09T09:24:15.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.654+0000 7f613b590740 1 -- start start 2024-05-09T09:24:15.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.654+0000 7f613b590740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c78530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.654+0000 7f613b590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c8b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.655+0000 7f6135d3d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c8b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.655+0000 7f613b590740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.655+0000 7f613b590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61301a2350 con 0x558de6c7a930 2024-05-09T09:24:15.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.655+0000 7f6136d3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.655+0000 7f6136d3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c78a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39698/0 (socket says 172.21.15.64:39698) 2024-05-09T09:24:15.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f6136d3f640 1 -- 172.21.15.64:0/1218450505 learned_addr learned my addr 172.21.15.64:0/1218450505 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f613019fbc0 con 0x558de6c7faa0 2024-05-09T09:24:15.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f61300734e0 con 0x558de6c79f40 2024-05-09T09:24:15.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f6136d3f640 1 -- 172.21.15.64:0/1218450505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 msgr2=0x558de6c78530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f6136d3f640 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c78530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f6136d3f640 1 -- 172.21.15.64:0/1218450505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 msgr2=0x558de6c8b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f6136d3f640 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c8b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.656+0000 7f6136d3f640 1 -- 172.21.15.64:0/1218450505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de6c7ad10 con 0x558de6c7faa0 2024-05-09T09:24:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.657+0000 7f6136d3f640 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c78a70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f612801fcd0 tx=0x7f612801a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997b9e809fda3d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.658+0000 7f611bfff640 1 -- 172.21.15.64:0/1218450505 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280c4070 con 0x558de6c7faa0 2024-05-09T09:24:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.658+0000 7f611bfff640 1 -- 172.21.15.64:0/1218450505 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61280a8690 con 0x558de6c7faa0 2024-05-09T09:24:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.658+0000 7f611bfff640 1 -- 172.21.15.64:0/1218450505 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280b52d0 con 0x558de6c7faa0 2024-05-09T09:24:15.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.659+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 msgr2=0x558de6c78a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.659+0000 7f613b590740 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c78a70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f612801fcd0 tx=0x7f612801a490 comp rx=0 tx=0).stop 2024-05-09T09:24:15.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 shutdown_connections 2024-05-09T09:24:15.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c7faa0 0x558de6c78a70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7a930 0x558de6c8b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 --2- 172.21.15.64:0/1218450505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c79f40 0x558de6c78530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 >> 172.21.15.64:0/1218450505 conn(0x558de6c755b0 msgr2=0x558de6c7bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 shutdown_connections 2024-05-09T09:24:15.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.660+0000 7f613b590740 1 -- 172.21.15.64:0/1218450505 wait complete. 2024-05-09T09:24:15.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.661+0000 7f613b590740 1 -- start start 2024-05-09T09:24:15.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.661+0000 7f613b590740 1 Processor -- start 2024-05-09T09:24:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 0x558de6c88e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c7a930 0x558de6c89380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c8a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f613019e0a0 con 0x558de6c7faa0 2024-05-09T09:24:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f613019f4f0 con 0x558de6c79f40 2024-05-09T09:24:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f613019e750 con 0x558de6c7a930 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613b590740 10 client.0 _ll_register_callbacks cb 0x7f612c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613653e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 0x558de6c88e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613653e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 0x558de6c88e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39708/0 (socket says 172.21.15.64:39708) 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.663+0000 7f613653e640 1 -- 172.21.15.64:0/1876923834 learned_addr learned my addr 172.21.15.64:0/1876923834 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f613653e640 1 -- 172.21.15.64:0/1876923834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c7a930 msgr2=0x558de6c89380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f613653e640 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c7a930 0x558de6c89380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f613653e640 1 -- 172.21.15.64:0/1876923834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 msgr2=0x558de6c8a340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f613653e640 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c8a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f613653e640 1 -- 172.21.15.64:0/1876923834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de6c8a930 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f613653e640 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 0x558de6c88e40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f612c004440 tx=0x7f612c009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315421c84f9c135b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.664+0000 7f6119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.665+0000 7f6119ffb640 1 -- 172.21.15.64:0/1876923834 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c01e070 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.665+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558de6c84160 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.665+0000 7f6119ffb640 1 -- 172.21.15.64:0/1876923834 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f612c01e210 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.666+0000 7f6119ffb640 1 -- 172.21.15.64:0/1876923834 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f612c02a050 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.666+0000 7f613b590740 10 client.28559 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.666+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558de6d07ab0 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f6119ffb640 1 -- 172.21.15.64:0/1876923834 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558de6c84160 con 0x558de6c79f40 2024-05-09T09:24:15.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f6119ffb640 1 -- 172.21.15.64:0/1876923834 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f612c012960 con 0x558de6c79f40 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f613b590740 10 client.28559 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f613b590740 10 client.28559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f613b590740 20 client.28559 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f613b590740 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6d08370 0x558de6d0a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f613b590740 4 client.28559 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.667+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558de6be66c0 con 0x558de6d08370 2024-05-09T09:24:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.668+0000 7f6135d3d640 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6d08370 0x558de6d0a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.673+0000 7f6135d3d640 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6d08370 0x558de6d0a790 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74dabf3db719c208 server_cookie=5f501dc78c675c2e in_seq=0 out_seq=0 2024-05-09T09:24:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.673+0000 7f6119ffb640 10 client.28559 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.675+0000 7f6119ffb640 1 -- 172.21.15.64:0/1876923834 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558de6be66c0 con 0x558de6d08370 2024-05-09T09:24:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.675+0000 7f6119ffb640 10 client.28559 handle_command_reply: tid=0 2024-05-09T09:24:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.675+0000 7f613b590740 1 client.28559 shutdown 2024-05-09T09:24:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.679+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 msgr2=0x558de6c88e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.679+0000 7f613b590740 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 0x558de6c88e40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f612c004440 tx=0x7f612c009d50 comp rx=0 tx=0).stop 2024-05-09T09:24:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.680+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 shutdown_connections 2024-05-09T09:24:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.680+0000 7f613b590740 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558de6d08370 0x558de6d0a790 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.680+0000 7f613b590740 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de6c7faa0 0x558de6c8a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.680+0000 7f613b590740 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558de6c7a930 0x558de6c89380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.680+0000 7f613b590740 1 --2- 172.21.15.64:0/1876923834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558de6c79f40 0x558de6c88e40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.680+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 >> 172.21.15.64:0/1876923834 conn(0x558de6c755b0 msgr2=0x558de6c73f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.681+0000 7f6119ffb640 0 client.28559 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.681+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 shutdown_connections 2024-05-09T09:24:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.681+0000 7f613b590740 1 -- 172.21.15.64:0/1876923834 wait complete. 2024-05-09T09:24:15.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.682+0000 7f613b590740 20 client.28559 trim_cache size 0 max 16384 2024-05-09T09:24:15.682 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:24:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.685+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f611c0a2c60 msgr2=0x7f611c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.685+0000 7f61387a2640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f611c0a2c60 0x7f611c0a5120 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f612002a970 tx=0x7f6120009400 comp rx=0 tx=0).stop 2024-05-09T09:24:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.685+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 msgr2=0x7f613019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.685+0000 7f61387a2640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 0x7f613019cee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f612801fd00 tx=0x7f612801fdf0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.685+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 shutdown_connections 2024-05-09T09:24:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.685+0000 7f61387a2640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f611c0a2c60 0x7f611c0a5120 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.686+0000 7f61387a2640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61301011e0 0x7f613019cee0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.686+0000 7f61387a2640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6130100840 0x7f6130195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.686+0000 7f61387a2640 1 --2- 172.21.15.64:0/3597424287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61300ff710 0x7f6130195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.686+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 >> 172.21.15.64:0/3597424287 conn(0x7f61300fb030 msgr2=0x7f6130103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.687+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 shutdown_connections 2024-05-09T09:24:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:15.687+0000 7f61387a2640 1 -- 172.21.15.64:0/3597424287 wait complete. 2024-05-09T09:24:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 835.0 s, 661.1 tps, lat 150.831 ms stddev 100.603 2024-05-09T09:24:16.716 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:24:16.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:24:16.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:24:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 -- 172.21.15.64:0/4070202677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0107d50 msgr2=0x7fcbb0108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/4070202677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0107d50 0x7fcbb0108170 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009900 tx=0x7fcba002a410 comp rx=0 tx=0).stop 2024-05-09T09:24:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 -- 172.21.15.64:0/4070202677 shutdown_connections 2024-05-09T09:24:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/4070202677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb01098b0 0x7fcbb0110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/4070202677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0108ef0 0x7fcbb0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/4070202677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0107d50 0x7fcbb0108170 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.814+0000 7fcbb6a02640 1 -- 172.21.15.64:0/4070202677 >> 172.21.15.64:0/4070202677 conn(0x7fcbb0076cc0 msgr2=0x7fcbb00790e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.815+0000 7fcbb6a02640 1 -- 172.21.15.64:0/4070202677 shutdown_connections 2024-05-09T09:24:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.815+0000 7fcbb6a02640 1 -- 172.21.15.64:0/4070202677 wait complete. 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.816+0000 7fcbb6a02640 1 Processor -- start 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.816+0000 7fcbb6a02640 1 -- start start 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6a02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0107d50 0x7fcbb019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6a02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0108ef0 0x7fcbb019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6a02640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 0x7fcbb01a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6a02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113800 con 0x7fcbb0107d50 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6a02640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113680 con 0x7fcbb0108ef0 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6a02640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113980 con 0x7fcbb01098b0 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6201640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 0x7fcbb01a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6201640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 0x7fcbb01a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45820/0 (socket says 172.21.15.64:45820) 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb6201640 1 -- 172.21.15.64:0/2333043191 learned_addr learned my addr 172.21.15.64:0/2333043191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0107d50 0x7fcbb019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.817+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0108ef0 0x7fcbb019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.818+0000 7fcbb6201640 1 -- 172.21.15.64:0/2333043191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0108ef0 msgr2=0x7fcbb019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.818+0000 7fcbb6201640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0108ef0 0x7fcbb019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.818+0000 7fcbb6201640 1 -- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0107d50 msgr2=0x7fcbb019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.818+0000 7fcbb6201640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0107d50 0x7fcbb019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.818+0000 7fcbb6201640 1 -- 172.21.15.64:0/2333043191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb007a1b0 con 0x7fcbb01098b0 2024-05-09T09:24:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.818+0000 7fcbb6201640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 0x7fcbb01a54f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00d950 tx=0x7fcbac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.819+0000 7fcba6ffd640 1 -- 172.21.15.64:0/2333043191 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac017030 con 0x7fcbb01098b0 2024-05-09T09:24:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.819+0000 7fcba6ffd640 1 -- 172.21.15.64:0/2333043191 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbac004020 con 0x7fcbb01098b0 2024-05-09T09:24:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.819+0000 7fcba6ffd640 1 -- 172.21.15.64:0/2333043191 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac003b20 con 0x7fcbb01098b0 2024-05-09T09:24:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.819+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb007a4a0 con 0x7fcbb01098b0 2024-05-09T09:24:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.819+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb007a9a0 con 0x7fcbb01098b0 2024-05-09T09:24:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.822+0000 7fcba6ffd640 1 -- 172.21.15.64:0/2333043191 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbac01a070 con 0x7fcbb01098b0 2024-05-09T09:24:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.823+0000 7fcbba974740 1 Processor -- start 2024-05-09T09:24:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.824+0000 7fcba6ffd640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcb9c0a2d50 0x7fcb9c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.824+0000 7fcba6ffd640 1 -- 172.21.15.64:0/2333043191 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcbac0cec80 con 0x7fcbb01098b0 2024-05-09T09:24:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 -- start start 2024-05-09T09:24:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e682820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682df0 0x55e64e6878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e68a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e64e5e6340 con 0x55e64e682df0 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e64e58e290 con 0x55e64e682400 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbba974740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e64e5e2300 con 0x55e64e687f60 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.825+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcb9c0a2d50 0x7fcb9c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e682820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e682820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39738/0 (socket says 172.21.15.64:39738) 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 -- 172.21.15.64:0/4200397934 learned_addr learned my addr 172.21.15.64:0/4200397934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb6201640 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e68a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 -- 172.21.15.64:0/4200397934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 msgr2=0x55e64e68a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e68a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 -- 172.21.15.64:0/4200397934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682df0 msgr2=0x55e64e6878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682df0 0x55e64e6878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.826+0000 7fcbb5a00640 1 -- 172.21.15.64:0/4200397934 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64e5b28e0 con 0x55e64e682400 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.827+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e682820 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcba0031520 tx=0x7fcba0031c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0962697e15cfc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.827+0000 7fcb9bfff640 1 -- 172.21.15.64:0/4200397934 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0040030 con 0x55e64e682400 2024-05-09T09:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.827+0000 7fcb9bfff640 1 -- 172.21.15.64:0/4200397934 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcba0031e30 con 0x55e64e682400 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 -- 172.21.15.64:0/4200397934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 msgr2=0x55e64e682820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e682820 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcba0031520 tx=0x7fcba0031c40 comp rx=0 tx=0).stop 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 -- 172.21.15.64:0/4200397934 shutdown_connections 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e68a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682df0 0x55e64e6878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 --2- 172.21.15.64:0/4200397934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e682820 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.828+0000 7fcbba974740 1 -- 172.21.15.64:0/4200397934 >> 172.21.15.64:0/4200397934 conn(0x55e64e67da70 msgr2=0x55e64e67de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.829+0000 7fcbba974740 1 -- 172.21.15.64:0/4200397934 shutdown_connections 2024-05-09T09:24:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.829+0000 7fcbba974740 1 -- 172.21.15.64:0/4200397934 wait complete. 2024-05-09T09:24:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 -- start start 2024-05-09T09:24:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 Processor -- start 2024-05-09T09:24:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e69d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e69dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e6a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e64e69bf60 con 0x55e64e6a2cc0 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e64e5a1790 con 0x55e64e682400 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e64e69c0e0 con 0x55e64e687f60 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.830+0000 7fcbba974740 10 client.0 _ll_register_callbacks cb 0x55e64e5dca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.831+0000 7fcbb51ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e69dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.831+0000 7fcbb5a00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e69d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.831+0000 7fcbb51ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e69dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45846/0 (socket says 172.21.15.64:45846) 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.831+0000 7fcbb51ff640 1 -- 172.21.15.64:0/2529170437 learned_addr learned my addr 172.21.15.64:0/2529170437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.831+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcb9c0a2d50 0x7fcb9c0a5210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55e64e683280 tx=0x7fcba8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcbb5a00640 1 -- 172.21.15.64:0/2529170437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 msgr2=0x55e64e69dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e69dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcbb5a00640 1 -- 172.21.15.64:0/2529170437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 msgr2=0x55e64e6a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e6a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcbb5a00640 1 -- 172.21.15.64:0/2529170437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64e2fa210 con 0x55e64e682400 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e69d750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcba0034040 tx=0x7fcba0005530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee32ec4ee9a8824e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.832+0000 7fcb997fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.833+0000 7fcb997fa640 1 -- 172.21.15.64:0/2529170437 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba004c020 con 0x55e64e682400 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.833+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba0031e30 con 0x55e64e682400 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.833+0000 7fcb997fa640 1 -- 172.21.15.64:0/2529170437 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcba0004000 con 0x55e64e682400 2024-05-09T09:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.833+0000 7fcb997fa640 1 -- 172.21.15.64:0/2529170437 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcba0057050 con 0x55e64e682400 2024-05-09T09:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.833+0000 7fcbba974740 10 client.28571 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.834+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e64e3c6620 con 0x55e64e682400 2024-05-09T09:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.834+0000 7fcb997fa640 1 -- 172.21.15.64:0/2529170437 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcba0031e30 con 0x55e64e682400 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcb997fa640 1 -- 172.21.15.64:0/2529170437 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fcba00035f0 con 0x55e64e682400 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbba974740 10 client.28571 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbba974740 10 client.28571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbba974740 20 client.28571 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbba974740 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e6a9770 0x55e64e6abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbba974740 4 client.28571 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e64e5f3a70 con 0x55e64e6a9770 2024-05-09T09:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.835+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e6a9770 0x55e64e6abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.836+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e6a9770 0x55e64e6abbb0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506b59302dacb0f6 server_cookie=ba6c1bb2b56f6728 in_seq=0 out_seq=0 2024-05-09T09:24:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.836+0000 7fcb997fa640 10 client.28571 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcb997fa640 1 -- 172.21.15.64:0/2529170437 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e64e5f3a70 con 0x55e64e6a9770 2024-05-09T09:24:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcb997fa640 10 client.28571 handle_command_reply: tid=0 2024-05-09T09:24:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 client.28571 shutdown 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 msgr2=0x55e64e69d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e69d750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcba0034040 tx=0x7fcba0005530 comp rx=0 tx=0).stop 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 shutdown_connections 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e6a9770 0x55e64e6abbb0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e6a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e69dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 --2- 172.21.15.64:0/2529170437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e69d750 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 >> 172.21.15.64:0/2529170437 conn(0x55e64e67da70 msgr2=0x55e64e67cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.837+0000 7fcb997fa640 0 client.28571 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.838+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 shutdown_connections 2024-05-09T09:24:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.838+0000 7fcbba974740 1 -- 172.21.15.64:0/2529170437 wait complete. 2024-05-09T09:24:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.838+0000 7fcbba974740 20 client.28571 trim_cache size 0 max 16384 2024-05-09T09:24:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.870+0000 7fcbba974740 1 Processor -- start 2024-05-09T09:24:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 -- start start 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682400 0x55e64e68a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e687f60 0x55e64e680b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 0x55e64e681040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0070260 con 0x55e64e682400 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb006dad0 con 0x55e64e687f60 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.871+0000 7fcbba974740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0082650 con 0x55e64e6a2cc0 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 0x55e64e681040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 0x55e64e681040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45860/0 (socket says 172.21.15.64:45860) 2024-05-09T09:24:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 -- 172.21.15.64:0/814937327 learned_addr learned my addr 172.21.15.64:0/814937327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e687f60 0x55e64e680b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 -- 172.21.15.64:0/814937327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e687f60 msgr2=0x55e64e680b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e687f60 0x55e64e680b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 -- 172.21.15.64:0/814937327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682400 msgr2=0x55e64e68a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682400 0x55e64e68a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 -- 172.21.15.64:0/814937327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64e687360 con 0x55e64e6a2cc0 2024-05-09T09:24:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.872+0000 7fcbb6201640 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 0x55e64e681040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcbac097500 tx=0x7fcbac005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbafd1f4f94681a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.873+0000 7fcb9b7fe640 1 -- 172.21.15.64:0/814937327 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac09f4a0 con 0x55e64e6a2cc0 2024-05-09T09:24:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.873+0000 7fcb9b7fe640 1 -- 172.21.15.64:0/814937327 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbac0b4070 con 0x55e64e6a2cc0 2024-05-09T09:24:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.873+0000 7fcb9b7fe640 1 -- 172.21.15.64:0/814937327 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac097760 con 0x55e64e6a2cc0 2024-05-09T09:24:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 -- 172.21.15.64:0/814937327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 msgr2=0x55e64e681040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 0x55e64e681040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcbac097500 tx=0x7fcbac005b20 comp rx=0 tx=0).stop 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 -- 172.21.15.64:0/814937327 shutdown_connections 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e6a2cc0 0x55e64e681040 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e687f60 0x55e64e680b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 --2- 172.21.15.64:0/814937327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e682400 0x55e64e68a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.874+0000 7fcbba974740 1 -- 172.21.15.64:0/814937327 >> 172.21.15.64:0/814937327 conn(0x55e64e67da70 msgr2=0x55e64e6876e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.875+0000 7fcbba974740 1 -- 172.21.15.64:0/814937327 shutdown_connections 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.875+0000 7fcbba974740 1 -- 172.21.15.64:0/814937327 wait complete. 2024-05-09T09:24:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.875+0000 7fcbba974740 1 -- start start 2024-05-09T09:24:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.876+0000 7fcbba974740 1 Processor -- start 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e6b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e6ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e70d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb006bf90 con 0x55e64e6a2cc0 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb006d400 con 0x55e64e682400 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb006c660 con 0x55e64e687f60 2024-05-09T09:24:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.877+0000 7fcbba974740 10 client.0 _ll_register_callbacks cb 0x7fcbac096c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb51ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e6ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb6201640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e70d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb6201640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e70d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57014/0 (socket says 172.21.15.64:57014) 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb6201640 1 -- 172.21.15.64:0/1677160748 learned_addr learned my addr 172.21.15.64:0/1677160748 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e6b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb51ff640 1 -- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 msgr2=0x55e64e6b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e6b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb51ff640 1 -- 172.21.15.64:0/1677160748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 msgr2=0x55e64e70d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e70d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.878+0000 7fcbb51ff640 1 -- 172.21.15.64:0/1677160748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64e70dd30 con 0x55e64e687f60 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.879+0000 7fcbb51ff640 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e6ba4e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55e64e6ac3c0 tx=0x7fcba8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e9a0d0cad36d465 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.879+0000 7fcb997fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.880+0000 7fcb997fa640 1 -- 172.21.15.64:0/1677160748 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba802c020 con 0x55e64e687f60 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.880+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbac0b4070 con 0x55e64e687f60 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.883+0000 7fcb997fa640 1 -- 172.21.15.64:0/1677160748 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcba8028d50 con 0x55e64e687f60 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.885+0000 7fcb997fa640 1 -- 172.21.15.64:0/1677160748 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcba8037050 con 0x55e64e687f60 2024-05-09T09:24:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.885+0000 7fcbba974740 10 client.28741 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.885+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e64e70f0f0 con 0x55e64e687f60 2024-05-09T09:24:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.886+0000 7fcb997fa640 1 -- 172.21.15.64:0/1677160748 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbac0b4070 con 0x55e64e687f60 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcb997fa640 1 -- 172.21.15.64:0/1677160748 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fcba80283d0 con 0x55e64e687f60 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbba974740 10 client.28741 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbba974740 10 client.28741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbba974740 20 client.28741 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbba974740 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e70fad0 0x55e64e711ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbba974740 4 client.28741 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:24:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e64e5eddc0 con 0x55e64e70fad0 2024-05-09T09:24:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.887+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e70fad0 0x55e64e711ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.888+0000 7fcbb5a00640 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e70fad0 0x55e64e711ef0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e07cefbad504b3 server_cookie=e4f0092f0219718e in_seq=0 out_seq=0 2024-05-09T09:24:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.888+0000 7fcb997fa640 10 client.28741 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.904+0000 7fcb997fa640 1 -- 172.21.15.64:0/1677160748 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e64e5eddc0 con 0x55e64e70fad0 2024-05-09T09:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.904+0000 7fcb997fa640 10 client.28741 handle_command_reply: tid=0 2024-05-09T09:24:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.905+0000 7fcbba974740 1 client.28741 shutdown 2024-05-09T09:24:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.908+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 msgr2=0x55e64e6ba4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.908+0000 7fcbba974740 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e6ba4e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55e64e6ac3c0 tx=0x7fcba8004a40 comp rx=0 tx=0).stop 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 shutdown_connections 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcbba974740 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e64e70fad0 0x55e64e711ef0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcbba974740 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e64e6a2cc0 0x55e64e70d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcbba974740 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e64e687f60 0x55e64e6ba4e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcbba974740 1 --2- 172.21.15.64:0/1677160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e64e682400 0x55e64e6b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 >> 172.21.15.64:0/1677160748 conn(0x55e64e67da70 msgr2=0x55e64e680b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.910+0000 7fcb997fa640 0 client.28741 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.915+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 shutdown_connections 2024-05-09T09:24:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.915+0000 7fcbba974740 1 -- 172.21.15.64:0/1677160748 wait complete. 2024-05-09T09:24:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.919+0000 7fcbba974740 20 client.28741 trim_cache size 0 max 16384 2024-05-09T09:24:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:24:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:24:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:24:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:24:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.921+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcb9c0a2d50 msgr2=0x7fcb9c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.921+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcb9c0a2d50 0x7fcb9c0a5210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55e64e683280 tx=0x7fcba8009500 comp rx=0 tx=0).stop 2024-05-09T09:24:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.921+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 msgr2=0x7fcbb01a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.921+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 0x7fcbb01a54f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00d950 tx=0x7fcbac00de10 comp rx=0 tx=0).stop 2024-05-09T09:24:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.922+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 shutdown_connections 2024-05-09T09:24:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.922+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fcb9c0a2d50 0x7fcb9c0a5210 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.922+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb01098b0 0x7fcbb01a54f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.922+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0108ef0 0x7fcbb019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.922+0000 7fcbb6a02640 1 --2- 172.21.15.64:0/2333043191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0107d50 0x7fcbb019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.922+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 >> 172.21.15.64:0/2333043191 conn(0x7fcbb0076cc0 msgr2=0x7fcbb010c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.924+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 shutdown_connections 2024-05-09T09:24:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:16.927+0000 7fcbb6a02640 1 -- 172.21.15.64:0/2333043191 wait complete. 2024-05-09T09:24:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.955+0000 7faa0f9ce640 1 -- 172.21.15.64:0/452896244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 msgr2=0x7faa0810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.956+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/452896244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 0x7faa0810fb70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7faa04009900 tx=0x7faa0402a440 comp rx=0 tx=0).stop 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.957+0000 7faa0f9ce640 1 -- 172.21.15.64:0/452896244 shutdown_connections 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.957+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/452896244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.957+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/452896244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 0x7faa0810fb70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.957+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/452896244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa08108830 0x7faa08108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.957+0000 7faa0f9ce640 1 -- 172.21.15.64:0/452896244 >> 172.21.15.64:0/452896244 conn(0x7faa0806e6c0 msgr2=0x7faa08070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.957+0000 7faa0f9ce640 1 -- 172.21.15.64:0/452896244 shutdown_connections 2024-05-09T09:24:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.958+0000 7faa0f9ce640 1 -- 172.21.15.64:0/452896244 wait complete. 2024-05-09T09:24:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.959+0000 7faa0f9ce640 1 Processor -- start 2024-05-09T09:24:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0f9ce640 1 -- start start 2024-05-09T09:24:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0f9ce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa08108830 0x7faa081a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0f9ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 0x7faa081a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0f9ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0f9ce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa08117dd0 con 0x7faa08109190 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0f9ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa08117c50 con 0x7faa081100b0 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.960+0000 7faa0d76a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa08108830 0x7faa081a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0f9ce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa08117f50 con 0x7faa08108830 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081a9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39790/0 (socket says 172.21.15.64:39790) 2024-05-09T09:24:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 -- 172.21.15.64:0/3250251084 learned_addr learned my addr 172.21.15.64:0/3250251084 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 -- 172.21.15.64:0/3250251084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa08108830 msgr2=0x7faa081a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa08108830 0x7faa081a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 -- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 msgr2=0x7faa081a29e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:24:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 0x7faa081a29e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.961+0000 7faa0df6b640 1 -- 172.21.15.64:0/3250251084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa081aa0c0 con 0x7faa081100b0 2024-05-09T09:24:16.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.963+0000 7faa0df6b640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081a9a50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faa0000eb40 tx=0x7faa0000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.963+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3250251084 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0000ce40 con 0x7faa081100b0 2024-05-09T09:24:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.963+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3250251084 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa00005020 con 0x7faa081100b0 2024-05-09T09:24:16.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.964+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3250251084 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00010650 con 0x7faa081100b0 2024-05-09T09:24:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.966+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa081aa3b0 con 0x7faa081100b0 2024-05-09T09:24:16.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.966+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa081aa880 con 0x7faa081100b0 2024-05-09T09:24:16.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.970+0000 7faa127bc740 1 Processor -- start 2024-05-09T09:24:16.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.972+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3250251084 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa000051d0 con 0x7faa081100b0 2024-05-09T09:24:16.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.973+0000 7fa9fe7fc640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9f40a2c80 0x7fa9f40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.973+0000 7faa127bc740 1 -- start start 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.973+0000 7fa9fe7fc640 1 -- 172.21.15.64:0/3250251084 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa000cf290 con 0x7faa081100b0 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa127bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562adecfa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa127bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562adecff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa0cf69640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9f40a2c80 0x7fa9f40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa127bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562aded01cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa127bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562adec55990 con 0x562adecf9cd0 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa127bc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562adebfd5b0 con 0x562adecff830 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa127bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562adec51950 con 0x562adecfa6c0 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa0cf69640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562adecff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa0cf69640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562adecff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45906/0 (socket says 172.21.15.64:45906) 2024-05-09T09:24:16.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa0cf69640 1 -- 172.21.15.64:0/1407324886 learned_addr learned my addr 172.21.15.64:0/1407324886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa0df6b640 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562aded01cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.974+0000 7faa0d76a640 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562adecfa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 -- 172.21.15.64:0/1407324886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 msgr2=0x562aded01cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562aded01cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 -- 172.21.15.64:0/1407324886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 msgr2=0x562adecfa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562adecfa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 -- 172.21.15.64:0/1407324886 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adea317e0 con 0x562adecfa6c0 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9f40a2c80 0x7fa9f40a5140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faa040096c0 tx=0x7faa04000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.975+0000 7faa0cf69640 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562adecff1c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faa04005c20 tx=0x7faa04043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87cdf42425795540 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.976+0000 7fa9f37fe640 1 -- 172.21.15.64:0/1407324886 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04005030 con 0x562adecfa6c0 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.976+0000 7fa9f37fe640 1 -- 172.21.15.64:0/1407324886 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0403e7d0 con 0x562adecfa6c0 2024-05-09T09:24:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.976+0000 7fa9f37fe640 1 -- 172.21.15.64:0/1407324886 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04004920 con 0x562adecfa6c0 2024-05-09T09:24:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.977+0000 7faa127bc740 1 -- 172.21.15.64:0/1407324886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 msgr2=0x562adecff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.977+0000 7faa127bc740 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562adecff1c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faa04005c20 tx=0x7faa04043000 comp rx=0 tx=0).stop 2024-05-09T09:24:16.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.979+0000 7faa127bc740 1 -- 172.21.15.64:0/1407324886 shutdown_connections 2024-05-09T09:24:16.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.979+0000 7faa127bc740 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562aded01cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.979+0000 7faa127bc740 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562adecff1c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.979+0000 7faa127bc740 1 --2- 172.21.15.64:0/1407324886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562adecfa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.980+0000 7faa127bc740 1 -- 172.21.15.64:0/1407324886 >> 172.21.15.64:0/1407324886 conn(0x562adecb90a0 msgr2=0x562adecf5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.980+0000 7faa127bc740 1 -- 172.21.15.64:0/1407324886 shutdown_connections 2024-05-09T09:24:16.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.980+0000 7faa127bc740 1 -- 172.21.15.64:0/1407324886 wait complete. 2024-05-09T09:24:16.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.980+0000 7faa127bc740 1 -- start start 2024-05-09T09:24:16.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 Processor -- start 2024-05-09T09:24:16.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562aded04140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecfa6c0 0x562aded04680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 0x562aded0b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562aded02780 con 0x562adecf9cd0 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562adec10b40 con 0x562adecfa6c0 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562aded02900 con 0x562adecff830 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.981+0000 7faa127bc740 10 client.0 _ll_register_callbacks cb 0x562adec050f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 0x562aded0b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 0x562aded0b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45910/0 (socket says 172.21.15.64:45910) 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 -- 172.21.15.64:0/2975309559 learned_addr learned my addr 172.21.15.64:0/2975309559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:16.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0cf69640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecfa6c0 0x562aded04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0d76a640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562aded04140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 -- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecfa6c0 msgr2=0x562aded04680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecfa6c0 0x562aded04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 -- 172.21.15.64:0/2975309559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 msgr2=0x562aded04140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562aded04140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 -- 172.21.15.64:0/2975309559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adea91f70 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0d76a640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562aded04140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.982+0000 7faa0df6b640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 0x562aded0b6c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c760 tx=0x7faa0009b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c8c20945550973 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.983+0000 7fa9f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.983+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/2975309559 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000cce60 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.983+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/2975309559 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa00015db0 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.983+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0403e7d0 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.983+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/2975309559 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faa000054c0 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.983+0000 7faa127bc740 10 client.28750 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562ade929170 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/2975309559 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0403e7d0 con 0x562adecff830 2024-05-09T09:24:16.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/2975309559 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7faa00097450 con 0x562adecff830 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 10 client.28750 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 10 client.28750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 20 client.28750 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562adecf08b0 0x562adecf2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 4 client.28750 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.984+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562adec630c0 con 0x562adecf08b0 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.985+0000 7faa0d76a640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562adecf08b0 0x562adecf2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:16.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.986+0000 7faa0d76a640 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562adecf08b0 0x562adecf2d70 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830d748dc489e82 server_cookie=72b1ebca331a0fe in_seq=0 out_seq=0 2024-05-09T09:24:16.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.986+0000 7fa9f0ff9640 10 client.28750 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.986+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/2975309559 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562adec630c0 con 0x562adecf08b0 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.986+0000 7fa9f0ff9640 10 client.28750 handle_command_reply: tid=0 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 client.28750 shutdown 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 msgr2=0x562aded0b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 0x562aded0b6c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c760 tx=0x7faa0009b7c0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 shutdown_connections 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562adecf08b0 0x562adecf2d70 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecff830 0x562aded0b6c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecfa6c0 0x562aded04680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 --2- 172.21.15.64:0/2975309559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecf9cd0 0x562aded04140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 >> 172.21.15.64:0/2975309559 conn(0x562adecb90a0 msgr2=0x562adecec080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7fa9f0ff9640 0 client.28750 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:16.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 shutdown_connections 2024-05-09T09:24:16.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 1 -- 172.21.15.64:0/2975309559 wait complete. 2024-05-09T09:24:16.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:16.987+0000 7faa127bc740 20 client.28750 trim_cache size 0 max 16384 2024-05-09T09:24:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.022+0000 7faa127bc740 1 Processor -- start 2024-05-09T09:24:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.023+0000 7faa127bc740 1 -- start start 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.023+0000 7faa127bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 0x562adecf8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.023+0000 7faa127bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecfa6c0 0x562adecf8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa0d76a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 0x562adecf8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa0d76a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 0x562adecf8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45918/0 (socket says 172.21.15.64:45918) 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa0cf69640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecfa6c0 0x562adecf8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa0d76a640 1 -- 172.21.15.64:0/1340196005 learned_addr learned my addr 172.21.15.64:0/1340196005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa127bc740 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562adecf8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa081b31c0 con 0x562adecfa6c0 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa081b0a30 con 0x562adecff830 2024-05-09T09:24:17.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa081ae880 con 0x562adecf9cd0 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.024+0000 7faa0df6b640 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562adecf8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.025+0000 7faa0d76a640 1 -- 172.21.15.64:0/1340196005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 msgr2=0x562adecf8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.025+0000 7faa0d76a640 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562adecf8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.025+0000 7faa0d76a640 1 -- 172.21.15.64:0/1340196005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecfa6c0 msgr2=0x562adecf8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.025+0000 7faa0d76a640 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecfa6c0 0x562adecf8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.025+0000 7faa0d76a640 1 -- 172.21.15.64:0/1340196005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adecf93d0 con 0x562adecf9cd0 2024-05-09T09:24:17.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.026+0000 7faa0d76a640 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 0x562adecf8410 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8000c00 tx=0x7fa9f8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fd7a63bae9a7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.027+0000 7fa9f2ffd640 1 -- 172.21.15.64:0/1340196005 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8021030 con 0x562adecf9cd0 2024-05-09T09:24:17.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.027+0000 7fa9f2ffd640 1 -- 172.21.15.64:0/1340196005 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f801d8d0 con 0x562adecf9cd0 2024-05-09T09:24:17.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.027+0000 7fa9f2ffd640 1 -- 172.21.15.64:0/1340196005 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004a00 con 0x562adecf9cd0 2024-05-09T09:24:17.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.030+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 msgr2=0x562adecf8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.030+0000 7faa127bc740 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 0x562adecf8410 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8000c00 tx=0x7fa9f8010800 comp rx=0 tx=0).stop 2024-05-09T09:24:17.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.032+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 shutdown_connections 2024-05-09T09:24:17.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.032+0000 7faa127bc740 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecff830 0x562adecf8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.032+0000 7faa127bc740 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecfa6c0 0x562adecf8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.032+0000 7faa127bc740 1 --2- 172.21.15.64:0/1340196005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecf9cd0 0x562adecf8410 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.032+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 >> 172.21.15.64:0/1340196005 conn(0x562adecb90a0 msgr2=0x562adecf51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.034+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 shutdown_connections 2024-05-09T09:24:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.036+0000 7faa127bc740 1 -- 172.21.15.64:0/1340196005 wait complete. 2024-05-09T09:24:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.036+0000 7faa127bc740 1 -- start start 2024-05-09T09:24:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.037+0000 7faa127bc740 1 Processor -- start 2024-05-09T09:24:17.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.037+0000 7faa127bc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecf9cd0 0x562aded0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.037+0000 7faa127bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562aded07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.037+0000 7faa127bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecff830 0x562aded076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa0cf69640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562aded07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa0cf69640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562aded07190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45922/0 (socket says 172.21.15.64:45922) 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa0cf69640 1 -- 172.21.15.64:0/1779064638 learned_addr learned my addr 172.21.15.64:0/1779064638 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa081aeef0 con 0x562adecff830 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa081b0360 con 0x562adecf9cd0 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa081af5c0 con 0x562adecfa6c0 2024-05-09T09:24:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.038+0000 7faa127bc740 10 client.0 _ll_register_callbacks cb 0x7faa0009b060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0df6b640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecff830 0x562aded076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0df6b640 1 -- 172.21.15.64:0/1779064638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 msgr2=0x562aded07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0df6b640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562aded07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0df6b640 1 -- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecf9cd0 msgr2=0x562aded0a6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0df6b640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecf9cd0 0x562aded0a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0df6b640 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aded0ad20 con 0x562adecff830 2024-05-09T09:24:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.039+0000 7faa0cf69640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562aded07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:17.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.040+0000 7faa0df6b640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecff830 0x562aded076d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7faa0009b640 tx=0x7faa00015fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f072a3402ab0b946 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.040+0000 7fa9f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:17.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.040+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/1779064638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000cce60 con 0x562adecff830 2024-05-09T09:24:17.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.040+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/1779064638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7faa0009ab50 con 0x562adecff830 2024-05-09T09:24:17.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.041+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f801d8d0 con 0x562adecff830 2024-05-09T09:24:17.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.043+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/1779064638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faa0009b240 con 0x562adecff830 2024-05-09T09:24:17.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.044+0000 7faa127bc740 10 client.19113 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.044+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562aded7f3d0 con 0x562adecff830 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/1779064638 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0009a5c0 con 0x562adecff830 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/1779064638 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7faa0009ca30 con 0x562adecff830 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7faa127bc740 10 client.19113 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7faa127bc740 10 client.19113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7faa127bc740 20 client.19113 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7faa127bc740 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aded7fd90 0x562aded821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7faa127bc740 4 client.19113 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:24:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.046+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa9f801d8d0 con 0x562aded7fd90 2024-05-09T09:24:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.049+0000 7faa0d76a640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aded7fd90 0x562aded821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.054+0000 7faa0d76a640 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aded7fd90 0x562aded821b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993d4fe5ab9e3337 server_cookie=2ab9aa955bf67ed7 in_seq=0 out_seq=0 2024-05-09T09:24:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.054+0000 7fa9f0ff9640 10 client.19113 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.055+0000 7fa9f0ff9640 1 -- 172.21.15.64:0/1779064638 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa9f801d8d0 con 0x562aded7fd90 2024-05-09T09:24:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.055+0000 7fa9f0ff9640 10 client.19113 handle_command_reply: tid=0 2024-05-09T09:24:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.055+0000 7faa127bc740 1 client.19113 shutdown 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecff830 msgr2=0x562aded076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecff830 0x562aded076d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7faa0009b640 tx=0x7faa00015fe0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 shutdown_connections 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562aded7fd90 0x562aded821b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562adecff830 0x562aded076d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7fa9f0ff9640 0 client.19113 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562adecfa6c0 0x562aded07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 --2- 172.21.15.64:0/1779064638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562adecf9cd0 0x562aded0a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.056+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 >> 172.21.15.64:0/1779064638 conn(0x562adecb90a0 msgr2=0x562adecf8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.058+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 shutdown_connections 2024-05-09T09:24:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.058+0000 7faa127bc740 1 -- 172.21.15.64:0/1779064638 wait complete. 2024-05-09T09:24:17.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.059+0000 7faa127bc740 20 client.19113 trim_cache size 0 max 16384 2024-05-09T09:24:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:24:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:24:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "50ff9fa7-6139-4b6e-b4e0-111d1066f5d0", 2024-05-09T09:24:17.059 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:24:17.059 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:24:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.061+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9f40a2c80 msgr2=0x7fa9f40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.061+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9f40a2c80 0x7fa9f40a5140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faa040096c0 tx=0x7faa04000bc0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.061+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 msgr2=0x7faa081a9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.061+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081a9a50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faa0000eb40 tx=0x7faa0000ef40 comp rx=0 tx=0).stop 2024-05-09T09:24:17.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.063+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 shutdown_connections 2024-05-09T09:24:17.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.063+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9f40a2c80 0x7fa9f40a5140 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.063+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa081100b0 0x7faa081a9a50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.063+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08109190 0x7faa081a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.063+0000 7faa0f9ce640 1 --2- 172.21.15.64:0/3250251084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa08108830 0x7faa081a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.063+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 >> 172.21.15.64:0/3250251084 conn(0x7faa0806e6c0 msgr2=0x7faa080704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.068+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 shutdown_connections 2024-05-09T09:24:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.069+0000 7faa0f9ce640 1 -- 172.21.15.64:0/3250251084 wait complete. 2024-05-09T09:24:17.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:24:17.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:24:17.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:17 smithi139 ceph-mon[33599]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 743 op/s 2024-05-09T09:24:17.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.336+0000 7f08a4316640 1 -- 172.21.15.64:0/511252660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c108fe0 msgr2=0x7f089c109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.336+0000 7f08a4316640 1 --2- 172.21.15.64:0/511252660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c108fe0 0x7f089c109460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f088c009860 tx=0x7f088c02a360 comp rx=0 tx=0).stop 2024-05-09T09:24:17.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.337+0000 7f08a4316640 1 -- 172.21.15.64:0/511252660 shutdown_connections 2024-05-09T09:24:17.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.337+0000 7f08a4316640 1 --2- 172.21.15.64:0/511252660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c1099a0 0x7f089c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.337+0000 7f08a4316640 1 --2- 172.21.15.64:0/511252660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c108fe0 0x7f089c109460 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.337+0000 7f08a4316640 1 --2- 172.21.15.64:0/511252660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f089c107e90 0x7f089c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.337+0000 7f08a4316640 1 -- 172.21.15.64:0/511252660 >> 172.21.15.64:0/511252660 conn(0x7f089c076e30 msgr2=0x7f089c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.338+0000 7f08a4316640 1 -- 172.21.15.64:0/511252660 shutdown_connections 2024-05-09T09:24:17.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.338+0000 7f08a4316640 1 -- 172.21.15.64:0/511252660 wait complete. 2024-05-09T09:24:17.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.339+0000 7f08a4316640 1 Processor -- start 2024-05-09T09:24:17.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.339+0000 7f08a4316640 1 -- start start 2024-05-09T09:24:17.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.339+0000 7f08a4316640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 0x7f089c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a4316640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f089c108fe0 0x7f089c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a4316640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c1099a0 0x7f089c1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a4316640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113a20 con 0x7f089c107e90 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a4316640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1138a0 con 0x7f089c108fe0 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a4316640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113ba0 con 0x7f089c1099a0 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a20b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 0x7f089c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a20b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 0x7f089c19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57072/0 (socket says 172.21.15.64:57072) 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a20b2640 1 -- 172.21.15.64:0/127772258 learned_addr learned my addr 172.21.15.64:0/127772258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a18b1640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f089c108fe0 0x7f089c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.340+0000 7f08a28b3640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c1099a0 0x7f089c1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a20b2640 1 -- 172.21.15.64:0/127772258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c1099a0 msgr2=0x7f089c1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a20b2640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c1099a0 0x7f089c1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a20b2640 1 -- 172.21.15.64:0/127772258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f089c108fe0 msgr2=0x7f089c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a20b2640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f089c108fe0 0x7f089c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a20b2640 1 -- 172.21.15.64:0/127772258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c1a5d70 con 0x7f089c107e90 2024-05-09T09:24:17.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a20b2640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 0x7f089c19e1a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f089800e9c0 tx=0x7f089800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08937fe640 1 -- 172.21.15.64:0/127772258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089800ccc0 con 0x7f089c107e90 2024-05-09T09:24:17.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.341+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c06b2b0 con 0x7f089c107e90 2024-05-09T09:24:17.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.342+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c06b7d0 con 0x7f089c107e90 2024-05-09T09:24:17.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.342+0000 7f08937fe640 1 -- 172.21.15.64:0/127772258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08980054d0 con 0x7f089c107e90 2024-05-09T09:24:17.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.342+0000 7f08937fe640 1 -- 172.21.15.64:0/127772258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898010430 con 0x7f089c107e90 2024-05-09T09:24:17.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.346+0000 7f08a701a740 1 Processor -- start 2024-05-09T09:24:17.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.350+0000 7f08a701a740 1 -- start start 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.350+0000 7f08937fe640 1 -- 172.21.15.64:0/127772258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08980046c0 con 0x7f089c107e90 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.350+0000 7f08a701a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 0x559cf0058350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.350+0000 7f08a701a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0058920 0x559cf005d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.350+0000 7f08a701a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf005ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a28b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 0x559cf0058350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a28b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 0x559cf0058350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39844/0 (socket says 172.21.15.64:39844) 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a20b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0058920 0x559cf005d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a701a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559ceffbcd00 con 0x559cf005da90 2024-05-09T09:24:17.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a701a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559ceff644c0 con 0x559cf0057f30 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a701a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559ceffb8cc0 con 0x559cf0058920 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a20b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0058920 0x559cf005d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45966/0 (socket says 172.21.15.64:45966) 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a20b2640 1 -- 172.21.15.64:0/731745094 learned_addr learned my addr 172.21.15.64:0/731745094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a18b1640 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf005ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.351+0000 7f08a28b3640 1 -- 172.21.15.64:0/731745094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0058920 msgr2=0x559cf005d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.352+0000 7f08a28b3640 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0058920 0x559cf005d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.352+0000 7f08a28b3640 1 -- 172.21.15.64:0/731745094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 msgr2=0x559cf005ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.352+0000 7f08a28b3640 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf005ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.352+0000 7f08a28b3640 1 -- 172.21.15.64:0/731745094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cefd9a7d0 con 0x559cf0057f30 2024-05-09T09:24:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.353+0000 7f08937fe640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08880a2d10 0x7f08880a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.354+0000 7f08a28b3640 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 0x559cf0058350 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0894009f00 tx=0x7f0894011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcbe46ca6d3ce4d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.354+0000 7f08937fe640 1 -- 172.21.15.64:0/127772258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f08980cf5d0 con 0x7f089c107e90 2024-05-09T09:24:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.354+0000 7f0890bfe640 1 -- 172.21.15.64:0/731745094 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894004d20 con 0x559cf0057f30 2024-05-09T09:24:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.354+0000 7f0890bfe640 1 -- 172.21.15.64:0/731745094 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0894004030 con 0x559cf0057f30 2024-05-09T09:24:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.354+0000 7f0890bfe640 1 -- 172.21.15.64:0/731745094 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894005280 con 0x559cf0057f30 2024-05-09T09:24:17.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.355+0000 7f08a20b2640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08880a2d10 0x7f08880a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.355+0000 7f08a701a740 1 -- 172.21.15.64:0/731745094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 msgr2=0x559cf0058350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.355+0000 7f08a701a740 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 0x559cf0058350 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0894009f00 tx=0x7f0894011810 comp rx=0 tx=0).stop 2024-05-09T09:24:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.356+0000 7f08a20b2640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08880a2d10 0x7f08880a51d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0898098410 tx=0x7f08980d2f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.356+0000 7f08a701a740 1 -- 172.21.15.64:0/731745094 shutdown_connections 2024-05-09T09:24:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.356+0000 7f08a701a740 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf005ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.356+0000 7f08a701a740 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0058920 0x559cf005d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.356+0000 7f08a701a740 1 --2- 172.21.15.64:0/731745094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0057f30 0x559cf0058350 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.356+0000 7f08a701a740 1 -- 172.21.15.64:0/731745094 >> 172.21.15.64:0/731745094 conn(0x559cf00535a0 msgr2=0x559cf00539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.357+0000 7f08a701a740 1 -- 172.21.15.64:0/731745094 shutdown_connections 2024-05-09T09:24:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.357+0000 7f08a701a740 1 -- 172.21.15.64:0/731745094 wait complete. 2024-05-09T09:24:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.358+0000 7f08a701a740 1 -- start start 2024-05-09T09:24:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.358+0000 7f08a701a740 1 Processor -- start 2024-05-09T09:24:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.359+0000 7f08a701a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf00623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.359+0000 7f08a701a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf0058920 0x559cf00628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.359+0000 7f08a28b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf00623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.359+0000 7f08a28b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf00623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45974/0 (socket says 172.21.15.64:45974) 2024-05-09T09:24:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.359+0000 7f08a28b3640 1 -- 172.21.15.64:0/2062554225 learned_addr learned my addr 172.21.15.64:0/2062554225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf005da90 0x559cf0069930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf0058920 0x559cf00628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559cf0060a40 con 0x559cf0058920 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559ceff77b90 con 0x559cf005da90 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559cf0060bc0 con 0x559cf0057f30 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf005da90 0x559cf0069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a28b3640 1 -- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf005da90 msgr2=0x559cf0069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf005da90 0x559cf0069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a28b3640 1 -- 172.21.15.64:0/2062554225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf0058920 msgr2=0x559cf00628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf0058920 0x559cf00628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.360+0000 7f08a28b3640 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cefcd03f0 con 0x559cf0057f30 2024-05-09T09:24:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.361+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf00623b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0894009270 tx=0x7f0894004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b21961a278667a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.361+0000 7f0885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.361+0000 7f0885ffb640 1 -- 172.21.15.64:0/2062554225 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089402d060 con 0x559cf0057f30 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.361+0000 7f0885ffb640 1 -- 172.21.15.64:0/2062554225 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0894003c40 con 0x559cf0057f30 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.361+0000 7f0885ffb640 10 client.0 inactive, discarding osd_map(94..94 src has 1..94) 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.362+0000 7f08a701a740 10 client.0 _ll_register_callbacks cb 0x559ceffb3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.362+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0894004030 con 0x559cf0057f30 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.362+0000 7f0885ffb640 1 -- 172.21.15.64:0/2062554225 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0894003e40 con 0x559cf0057f30 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.362+0000 7f08a701a740 10 client.28765 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=49}) -- 0x559cefc900f0 con 0x559cf0057f30 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f0885ffb640 1 -- 172.21.15.64:0/2062554225 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0894021070 con 0x559cf0057f30 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 10 client.28765 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 10 client.28765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 20 client.28765 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf0070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 4 client.28765 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:17.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.363+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ceffca430 con 0x559cf006dbf0 2024-05-09T09:24:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.364+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.364+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf0070080 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4088872e84d101a server_cookie=77dcc6a41f791a99 in_seq=0 out_seq=0 2024-05-09T09:24:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.364+0000 7f0885ffb640 10 client.28765 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.365+0000 7f0885ffb640 1 -- 172.21.15.64:0/2062554225 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ceffca430 con 0x559cf006dbf0 2024-05-09T09:24:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.365+0000 7f0885ffb640 10 client.28765 handle_command_reply: tid=0 2024-05-09T09:24:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.365+0000 7f08a701a740 1 client.28765 shutdown 2024-05-09T09:24:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.371+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 msgr2=0x559cf00623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.371+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf00623b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0894009270 tx=0x7f0894004260 comp rx=0 tx=0).stop 2024-05-09T09:24:17.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.373+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 shutdown_connections 2024-05-09T09:24:17.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.373+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf0070080 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.373+0000 7f0885ffb640 0 client.28765 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.373+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf005da90 0x559cf0069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.374+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf0058920 0x559cf00628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.374+0000 7f08a701a740 1 --2- 172.21.15.64:0/2062554225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf00623b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.374+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 >> 172.21.15.64:0/2062554225 conn(0x559cf00535a0 msgr2=0x559cf00527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.375+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 shutdown_connections 2024-05-09T09:24:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.375+0000 7f08a701a740 1 -- 172.21.15.64:0/2062554225 wait complete. 2024-05-09T09:24:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.376+0000 7f08a701a740 20 client.28765 trim_cache size 0 max 16384 2024-05-09T09:24:17.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:17 smithi064 ceph-mon[30149]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 743 op/s 2024-05-09T09:24:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.427+0000 7f08a701a740 1 Processor -- start 2024-05-09T09:24:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.427+0000 7f08a701a740 1 -- start start 2024-05-09T09:24:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.427+0000 7f08a701a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0061fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.427+0000 7f08a701a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.427+0000 7f08a701a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf0062500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a701a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0880012530 con 0x559cf005da90 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a701a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f088000fda0 con 0x559cf0058920 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a701a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f088000dbf0 con 0x559cf0057f30 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00698c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:39870/0 (socket says 172.21.15.64:39870) 2024-05-09T09:24:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 -- 172.21.15.64:0/2560273223 learned_addr learned my addr 172.21.15.64:0/2560273223 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 -- 172.21.15.64:0/2560273223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 msgr2=0x559cf0061fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0061fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0061fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 -- 172.21.15.64:0/2560273223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 msgr2=0x559cf0062500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf0062500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.428+0000 7f08a18b1640 1 -- 172.21.15.64:0/2560273223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf006a9b0 con 0x559cf0058920 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.429+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0061fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.429+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00698c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f088c009860 tx=0x7f088c005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=891574ec1b0ae860 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.429+0000 7f0887fff640 1 -- 172.21.15.64:0/2560273223 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c058030 con 0x559cf0058920 2024-05-09T09:24:17.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.429+0000 7f0887fff640 1 -- 172.21.15.64:0/2560273223 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f088c048040 con 0x559cf0058920 2024-05-09T09:24:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.429+0000 7f0887fff640 1 -- 172.21.15.64:0/2560273223 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0052a0 con 0x559cf0058920 2024-05-09T09:24:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.430+0000 7f08a701a740 1 -- 172.21.15.64:0/2560273223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 msgr2=0x559cf00698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.434+0000 7f08a701a740 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00698c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f088c009860 tx=0x7f088c005900 comp rx=0 tx=0).stop 2024-05-09T09:24:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.434+0000 7f08a701a740 1 -- 172.21.15.64:0/2560273223 shutdown_connections 2024-05-09T09:24:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.434+0000 7f08a701a740 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf0062500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.434+0000 7f08a701a740 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00698c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.434+0000 7f08a701a740 1 --2- 172.21.15.64:0/2560273223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0061fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.435+0000 7f08a701a740 1 -- 172.21.15.64:0/2560273223 >> 172.21.15.64:0/2560273223 conn(0x559cf00535a0 msgr2=0x559cf005fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.435+0000 7f08a701a740 1 -- 172.21.15.64:0/2560273223 shutdown_connections 2024-05-09T09:24:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.435+0000 7f08a701a740 1 -- 172.21.15.64:0/2560273223 wait complete. 2024-05-09T09:24:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.435+0000 7f08a701a740 1 -- start start 2024-05-09T09:24:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.436+0000 7f08a701a740 1 Processor -- start 2024-05-09T09:24:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.436+0000 7f08a701a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0066e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.436+0000 7f08a701a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.436+0000 7f08a701a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf00683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.436+0000 7f08a701a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f088000e260 con 0x559cf005da90 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a701a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f088000f6d0 con 0x559cf0058920 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a701a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f088000e930 con 0x559cf0057f30 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0066e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:45992/0 (socket says 172.21.15.64:45992) 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 -- 172.21.15.64:0/2130479857 learned_addr learned my addr 172.21.15.64:0/2130479857 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a701a740 10 client.0 _ll_register_callbacks cb 0x7f088c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf00683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 -- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 msgr2=0x559cf00673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 -- 172.21.15.64:0/2130479857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 msgr2=0x559cf00683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf00683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.437+0000 7f08a28b3640 1 -- 172.21.15.64:0/2130479857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf00689b0 con 0x559cf0057f30 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.438+0000 7f08a28b3640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0066e70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08940046b0 tx=0x7f0894012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c631a0c3af9fdecc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:17.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.438+0000 7f0885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:17.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.438+0000 7f0885ffb640 1 -- 172.21.15.64:0/2130479857 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089402d040 con 0x559cf0057f30 2024-05-09T09:24:17.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.439+0000 7f0885ffb640 1 -- 172.21.15.64:0/2130479857 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f089401e040 con 0x559cf0057f30 2024-05-09T09:24:17.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.439+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f088c048040 con 0x559cf0057f30 2024-05-09T09:24:17.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.439+0000 7f0885ffb640 1 -- 172.21.15.64:0/2130479857 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0894043050 con 0x559cf0057f30 2024-05-09T09:24:17.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.439+0000 7f08a701a740 10 client.28768 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:17.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.439+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559cf0069c30 con 0x559cf0057f30 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.439+0000 7f0885ffb640 1 -- 172.21.15.64:0/2130479857 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f088c048040 con 0x559cf0057f30 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f0885ffb640 1 -- 172.21.15.64:0/2130479857 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0894021070 con 0x559cf0057f30 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a701a740 10 client.28768 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a701a740 10 client.28768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a701a740 20 client.28768 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a701a740 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf00d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a701a740 4 client.28768 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ceffc4780 con 0x559cf006dbf0 2024-05-09T09:24:17.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.440+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf00d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:17.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.441+0000 7f08a18b1640 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf00d6a40 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5784d5b84804ca server_cookie=bb33559a61a79b5f in_seq=0 out_seq=0 2024-05-09T09:24:17.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.441+0000 7f0885ffb640 10 client.28768 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:17.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.442+0000 7f0885ffb640 1 -- 172.21.15.64:0/2130479857 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559ceffc4780 con 0x559cf006dbf0 2024-05-09T09:24:17.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.442+0000 7f0885ffb640 10 client.28768 handle_command_reply: tid=0 2024-05-09T09:24:17.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.442+0000 7f08a701a740 1 client.28768 shutdown 2024-05-09T09:24:17.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.445+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 msgr2=0x559cf0066e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.445+0000 7f08a701a740 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0066e70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08940046b0 tx=0x7f0894012f90 comp rx=0 tx=0).stop 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.445+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 shutdown_connections 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x559cf006dbf0 0x559cf00d6a40 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559cf005da90 0x559cf00683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f0885ffb640 0 client.28768 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559cf0058920 0x559cf00673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 --2- 172.21.15.64:0/2130479857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559cf0057f30 0x559cf0066e70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 >> 172.21.15.64:0/2130479857 conn(0x559cf00535a0 msgr2=0x559cf0061fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 shutdown_connections 2024-05-09T09:24:17.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.446+0000 7f08a701a740 1 -- 172.21.15.64:0/2130479857 wait complete. 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.447+0000 7f08a701a740 20 client.28768 trim_cache size 0 max 16384 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stdout: "50ff9fa7-6139-4b6e-b4e0-111d1066f5d0": { 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:24:17.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "50ff9fa7-6139-4b6e-b4e0-111d1066f5d0", 2024-05-09T09:24:17.448 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:24:17.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:24:17.448 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:24:17.448 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:24:17.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.449+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08880a2d10 msgr2=0x7f08880a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.449+0000 7f08a4316640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08880a2d10 0x7f08880a51d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0898098410 tx=0x7f08980d2f90 comp rx=0 tx=0).stop 2024-05-09T09:24:17.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.449+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 msgr2=0x7f089c19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:17.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.449+0000 7f08a4316640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 0x7f089c19e1a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f089800e9c0 tx=0x7f089800ee80 comp rx=0 tx=0).stop 2024-05-09T09:24:17.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.450+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 shutdown_connections 2024-05-09T09:24:17.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.450+0000 7f08a4316640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f08880a2d10 0x7f08880a51d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.450+0000 7f08a4316640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f089c1099a0 0x7f089c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.450+0000 7f08a4316640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f089c108fe0 0x7f089c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.450+0000 7f08a4316640 1 --2- 172.21.15.64:0/127772258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f089c107e90 0x7f089c19e1a0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:17.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.450+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 >> 172.21.15.64:0/127772258 conn(0x7f089c076e30 msgr2=0x7f089c10c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:17.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.452+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 shutdown_connections 2024-05-09T09:24:17.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:17.453+0000 7f08a4316640 1 -- 172.21.15.64:0/127772258 wait complete. 2024-05-09T09:24:17.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:17 smithi179 ceph-mon[34086]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 743 op/s 2024-05-09T09:24:17.481 INFO:tasks.cephfs.filesystem:scrub status for tag:50ff9fa7-6139-4b6e-b4e0-111d1066f5d0 - {'path': '/', 'tag': '50ff9fa7-6139-4b6e-b4e0-111d1066f5d0', 'options': 'recursive,force'} 2024-05-09T09:24:18.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:24:18.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:24:18.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:24:18.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:24:18.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:18 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:18 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:18.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:18 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:19.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:19 smithi064 ceph-mon[30149]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 655 op/s 2024-05-09T09:24:19.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:19 smithi179 ceph-mon[34086]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 655 op/s 2024-05-09T09:24:19.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:19 smithi139 ceph-mon[33599]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 655 op/s 2024-05-09T09:24:21.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 840.0 s, 635.4 tps, lat 156.272 ms stddev 79.760 2024-05-09T09:24:21.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:21 smithi139 ceph-mon[33599]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 902 op/s 2024-05-09T09:24:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:21 smithi064 ceph-mon[30149]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 902 op/s 2024-05-09T09:24:21.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:21 smithi179 ceph-mon[34086]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 902 op/s 2024-05-09T09:24:23.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:23 smithi064 ceph-mon[30149]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.0 MiB/s wr, 666 op/s 2024-05-09T09:24:23.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:23 smithi179 ceph-mon[34086]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.0 MiB/s wr, 666 op/s 2024-05-09T09:24:23.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:23 smithi139 ceph-mon[33599]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.0 MiB/s wr, 666 op/s 2024-05-09T09:24:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:25 smithi064 ceph-mon[30149]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 794 op/s 2024-05-09T09:24:25.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:25 smithi179 ceph-mon[34086]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 794 op/s 2024-05-09T09:24:25.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:25 smithi139 ceph-mon[33599]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 794 op/s 2024-05-09T09:24:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 845.0 s, 709.7 tps, lat 140.399 ms stddev 99.633 2024-05-09T09:24:27.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:27 smithi064 ceph-mon[30149]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.5 MiB/s wr, 727 op/s 2024-05-09T09:24:27.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:27 smithi179 ceph-mon[34086]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.5 MiB/s wr, 727 op/s 2024-05-09T09:24:27.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:27 smithi139 ceph-mon[33599]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.5 MiB/s wr, 727 op/s 2024-05-09T09:24:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:29 smithi064 ceph-mon[30149]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.9 MiB/s wr, 648 op/s 2024-05-09T09:24:29.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:29 smithi179 ceph-mon[34086]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.9 MiB/s wr, 648 op/s 2024-05-09T09:24:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:29 smithi139 ceph-mon[33599]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.9 MiB/s wr, 648 op/s 2024-05-09T09:24:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 -- 172.21.15.64:0/4058220282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc109800 msgr2=0x7f99fc10e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 --2- 172.21.15.64:0/4058220282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc109800 0x7f99fc10e170 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009900 tx=0x7f99ec02a420 comp rx=0 tx=0).stop 2024-05-09T09:24:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 -- 172.21.15.64:0/4058220282 shutdown_connections 2024-05-09T09:24:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 --2- 172.21.15.64:0/4058220282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc109800 0x7f99fc10e170 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 --2- 172.21.15.64:0/4058220282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc108e40 0x7f99fc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 --2- 172.21.15.64:0/4058220282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc072b60 0x7f99fc072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.105+0000 7f99fbfff640 1 -- 172.21.15.64:0/4058220282 >> 172.21.15.64:0/4058220282 conn(0x7f99fc06e6d0 msgr2=0x7f99fc070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.106+0000 7f99fbfff640 1 -- 172.21.15.64:0/4058220282 shutdown_connections 2024-05-09T09:24:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.106+0000 7f99fbfff640 1 -- 172.21.15.64:0/4058220282 wait complete. 2024-05-09T09:24:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.107+0000 7f99fbfff640 1 Processor -- start 2024-05-09T09:24:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.107+0000 7f99fbfff640 1 -- start start 2024-05-09T09:24:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 0x7f99fc1199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc108e40 0x7f99fc119ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc109800 0x7f99fc117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fbfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc110820 con 0x7f99fc072b60 2024-05-09T09:24:30.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fbfff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1106a0 con 0x7f99fc108e40 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fbfff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1109a0 con 0x7f99fc109800 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99faffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 0x7f99fc1199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99faffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 0x7f99fc1199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58256/0 (socket says 172.21.15.64:58256) 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99faffd640 1 -- 172.21.15.64:0/3547137774 learned_addr learned my addr 172.21.15.64:0/3547137774 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc108e40 0x7f99fc119ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.108+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc109800 0x7f99fc117ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fa7fc640 1 -- 172.21.15.64:0/3547137774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc109800 msgr2=0x7f99fc117ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc109800 0x7f99fc117ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fa7fc640 1 -- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 msgr2=0x7f99fc1199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 0x7f99fc1199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fa7fc640 1 -- 172.21.15.64:0/3547137774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc118690 con 0x7f99fc108e40 2024-05-09T09:24:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99faffd640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 0x7f99fc1199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc109800 0x7f99fc117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.109+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc108e40 0x7f99fc119ef0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f99f400ccb0 tx=0x7f99f4007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.110+0000 7f99e3fff640 1 -- 172.21.15.64:0/3547137774 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4003990 con 0x7f99fc108e40 2024-05-09T09:24:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.111+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc118980 con 0x7f99fc108e40 2024-05-09T09:24:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.111+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc1b7460 con 0x7f99fc108e40 2024-05-09T09:24:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.113+0000 7f99e3fff640 1 -- 172.21.15.64:0/3547137774 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f4004020 con 0x7f99fc108e40 2024-05-09T09:24:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.113+0000 7f99e3fff640 1 -- 172.21.15.64:0/3547137774 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400f6b0 con 0x7f99fc108e40 2024-05-09T09:24:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.115+0000 7f99e3fff640 1 -- 172.21.15.64:0/3547137774 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99f400f910 con 0x7f99fc108e40 2024-05-09T09:24:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.115+0000 7f9a043ac740 1 Processor -- start 2024-05-09T09:24:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.116+0000 7f9a043ac740 1 -- start start 2024-05-09T09:24:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.116+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f90625820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.116+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.116+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f9062d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.116+0000 7f9a043ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f90589340 con 0x562f90625df0 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f9a043ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f90531290 con 0x562f90625400 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f9a043ac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562f90585300 con 0x562f9062af60 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f9062d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f90625820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f90625820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37486/0 (socket says 172.21.15.64:37486) 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f9062d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:47504/0 (socket says 172.21.15.64:47504) 2024-05-09T09:24:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fb7fe640 1 -- 172.21.15.64:0/2939253020 learned_addr learned my addr 172.21.15.64:0/2939253020 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 -- 172.21.15.64:0/2939253020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 msgr2=0x562f90625820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f90625820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 -- 172.21.15.64:0/2939253020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 msgr2=0x562f9062a8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.117+0000 7f99fa7fc640 1 -- 172.21.15.64:0/2939253020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f905558e0 con 0x562f9062af60 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.118+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f9062d3f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99f40cd1a0 tx=0x7f99f40d59c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c0b15dc67923b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.118+0000 7f99e17fa640 1 -- 172.21.15.64:0/2939253020 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40e3070 con 0x562f9062af60 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.118+0000 7f99e17fa640 1 -- 172.21.15.64:0/2939253020 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40db9b0 con 0x562f9062af60 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.119+0000 7f99e3fff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e40a2d10 0x7f99e40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.119+0000 7f99e3fff640 1 -- 172.21.15.64:0/3547137774 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f40cfaf0 con 0x7f99fc108e40 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.119+0000 7f9a043ac740 1 -- 172.21.15.64:0/2939253020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 msgr2=0x562f9062d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.119+0000 7f9a043ac740 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f9062d3f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99f40cd1a0 tx=0x7f99f40d59c0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e40a2d10 0x7f99e40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 -- 172.21.15.64:0/2939253020 shutdown_connections 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f9062d3f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 --2- 172.21.15.64:0/2939253020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f90625820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 -- 172.21.15.64:0/2939253020 >> 172.21.15.64:0/2939253020 conn(0x562f90620a70 msgr2=0x562f90620e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 -- 172.21.15.64:0/2939253020 shutdown_connections 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.120+0000 7f9a043ac740 1 -- 172.21.15.64:0/2939253020 wait complete. 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 -- start start 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 Processor -- start 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e40a2d10 0x7f99e40a51d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99ec02a990 tx=0x7f99ec02a920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f90636bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f9062df60 con 0x562f9062af60 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f90544790 con 0x562f90625400 2024-05-09T09:24:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562f9062e0e0 con 0x562f90625df0 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.121+0000 7f9a043ac740 10 client.0 _ll_register_callbacks cb 0x562f9057fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f90636bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f90636bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58262/0 (socket says 172.21.15.64:58262) 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 -- 172.21.15.64:0/1409162691 learned_addr learned my addr 172.21.15.64:0/1409162691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99faffd640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 -- 172.21.15.64:0/1409162691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625df0 msgr2=0x562f9062fb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 -- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 msgr2=0x562f9062f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99fb7fe640 1 -- 172.21.15.64:0/1409162691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9029d210 con 0x562f9062af60 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.122+0000 7f99faffd640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f90636bd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009230 tx=0x7f99ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c9cdf6d2e147c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f99deff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f99deff5640 1 -- 172.21.15.64:0/1409162691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec005450 con 0x562f9062af60 2024-05-09T09:24:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f99deff5640 1 -- 172.21.15.64:0/1409162691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99ec0055f0 con 0x562f9062af60 2024-05-09T09:24:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f40db9b0 con 0x562f9062af60 2024-05-09T09:24:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f99deff5640 1 -- 172.21.15.64:0/1409162691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f99ec009460 con 0x562f9062af60 2024-05-09T09:24:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.123+0000 7f9a043ac740 10 client.19137 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:30.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.124+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562f90369620 con 0x562f9062af60 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.127+0000 7f99deff5640 1 -- 172.21.15.64:0/1409162691 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40db9b0 con 0x562f9062af60 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.127+0000 7f99deff5640 1 -- 172.21.15.64:0/1409162691 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f99ec044030 con 0x562f9062af60 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.128+0000 7f9a043ac740 10 client.19137 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.128+0000 7f9a043ac740 10 client.19137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.128+0000 7f9a043ac740 20 client.19137 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.128+0000 7f9a043ac740 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f9063b670 0x562f9063dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.128+0000 7f9a043ac740 4 client.19137 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.128+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f90596a70 con 0x562f9063b670 2024-05-09T09:24:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.130+0000 7f99faffd640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f9063b670 0x562f9063dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.135+0000 7f99faffd640 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f9063b670 0x562f9063dad0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97734d1dc802a42 server_cookie=ddcab1ba558d4367 in_seq=0 out_seq=0 2024-05-09T09:24:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.135+0000 7f99deff5640 10 client.19137 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.136+0000 7f99deff5640 1 -- 172.21.15.64:0/1409162691 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f90596a70 con 0x562f9063b670 2024-05-09T09:24:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.136+0000 7f99deff5640 10 client.19137 handle_command_reply: tid=0 2024-05-09T09:24:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.136+0000 7f9a043ac740 1 client.19137 shutdown 2024-05-09T09:24:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 msgr2=0x562f90636bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f90636bd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009230 tx=0x7f99ec009260 comp rx=0 tx=0).stop 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 shutdown_connections 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f9063b670 0x562f9063dad0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f90636bd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 --2- 172.21.15.64:0/1409162691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.137+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 >> 172.21.15.64:0/1409162691 conn(0x562f90620a70 msgr2=0x562f9061fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.138+0000 7f99deff5640 0 client.19137 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.138+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 shutdown_connections 2024-05-09T09:24:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.140+0000 7f9a043ac740 1 -- 172.21.15.64:0/1409162691 wait complete. 2024-05-09T09:24:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.140+0000 7f9a043ac740 20 client.19137 trim_cache size 0 max 16384 2024-05-09T09:24:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.173+0000 7f9a043ac740 1 Processor -- start 2024-05-09T09:24:30.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.173+0000 7f9a043ac740 1 -- start start 2024-05-09T09:24:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.174+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.174+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.174+0000 7f99faffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.174+0000 7f99faffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:37512/0 (socket says 172.21.15.64:37512) 2024-05-09T09:24:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.174+0000 7f99faffd640 1 -- 172.21.15.64:0/768427958 learned_addr learned my addr 172.21.15.64:0/768427958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.174+0000 7f9a043ac740 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f90636b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8012650 con 0x562f90625df0 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99fb7fe640 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f90636b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99e800fec0 con 0x562f90625400 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99e800dd10 con 0x562f9062af60 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99faffd640 1 -- 172.21.15.64:0/768427958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 msgr2=0x562f90636b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99faffd640 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f90636b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99faffd640 1 -- 172.21.15.64:0/768427958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 msgr2=0x562f9062fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99faffd640 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.175+0000 7f99faffd640 1 -- 172.21.15.64:0/768427958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f90630410 con 0x562f90625400 2024-05-09T09:24:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.176+0000 7f99faffd640 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x562f906261f0 tx=0x7f99f0013a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1badbbd5e7572be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.176+0000 7f99e0ff9640 1 -- 172.21.15.64:0/768427958 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0025070 con 0x562f90625400 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.176+0000 7f99e0ff9640 1 -- 172.21.15.64:0/768427958 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f001f890 con 0x562f90625400 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.176+0000 7f99e0ff9640 1 -- 172.21.15.64:0/768427958 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f00055f0 con 0x562f90625400 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.177+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 msgr2=0x562f9062f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.177+0000 7f9a043ac740 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x562f906261f0 tx=0x7f99f0013a50 comp rx=0 tx=0).stop 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.177+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 shutdown_connections 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.177+0000 7f9a043ac740 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9062af60 0x562f90636b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.178+0000 7f9a043ac740 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f90625df0 0x562f9062fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.178+0000 7f9a043ac740 1 --2- 172.21.15.64:0/768427958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625400 0x562f9062f650 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.178+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 >> 172.21.15.64:0/768427958 conn(0x562f90620a70 msgr2=0x562f90626f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.178+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 shutdown_connections 2024-05-09T09:24:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.178+0000 7f9a043ac740 1 -- 172.21.15.64:0/768427958 wait complete. 2024-05-09T09:24:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.179+0000 7f9a043ac740 1 -- start start 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.179+0000 7f9a043ac740 1 Processor -- start 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 0x562f906336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625df0 0x562f90633c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f9069fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99e800e380 con 0x562f9062af60 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99e800f7f0 con 0x562f90625df0 2024-05-09T09:24:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99e800ea50 con 0x562f90625400 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f9a043ac740 10 client.0 _ll_register_callbacks cb 0x7f99f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99fa7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625df0 0x562f90633c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 0x562f906336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 0x562f906336d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:47536/0 (socket says 172.21.15.64:47536) 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 -- 172.21.15.64:0/546812760 learned_addr learned my addr 172.21.15.64:0/546812760 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 -- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625df0 msgr2=0x562f90633c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625df0 0x562f90633c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 -- 172.21.15.64:0/546812760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 msgr2=0x562f9069fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f9069fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.180+0000 7f99faffd640 1 -- 172.21.15.64:0/546812760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f90636ce0 con 0x562f90625400 2024-05-09T09:24:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.181+0000 7f99faffd640 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 0x562f906336d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99f0012530 tx=0x7f99f0009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa5aa7aba240bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.181+0000 7f99deff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.182+0000 7f99deff5640 1 -- 172.21.15.64:0/546812760 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f002e060 con 0x562f90625400 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.182+0000 7f99deff5640 1 -- 172.21.15.64:0/546812760 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f99f001f3e0 con 0x562f90625400 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.182+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f001f890 con 0x562f90625400 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.183+0000 7f99deff5640 1 -- 172.21.15.64:0/546812760 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f99f0025040 con 0x562f90625400 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.183+0000 7f9a043ac740 10 client.28780 fetch_fsmap learned FSMap version 49 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.183+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562f906a10a0 con 0x562f90625400 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f99deff5640 1 -- 172.21.15.64:0/546812760 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f001f890 con 0x562f90625400 2024-05-09T09:24:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f99deff5640 1 -- 172.21.15.64:0/546812760 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f99f00046c0 con 0x562f90625400 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f9a043ac740 10 client.28780 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f9a043ac740 10 client.28780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f9a043ac740 20 client.28780 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f9a043ac740 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f906a1a10 0x562f906a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f9a043ac740 4 client.28780 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.184+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f90590dc0 con 0x562f906a1a10 2024-05-09T09:24:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.187+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f906a1a10 0x562f906a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.188+0000 7f99fa7fc640 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f906a1a10 0x562f906a3e30 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a339f555dac7f7 server_cookie=b7b81de95289eba3 in_seq=0 out_seq=0 2024-05-09T09:24:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.188+0000 7f99deff5640 10 client.28780 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f99deff5640 1 -- 172.21.15.64:0/546812760 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f90590dc0 con 0x562f906a1a10 2024-05-09T09:24:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f99deff5640 10 client.28780 handle_command_reply: tid=0 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 client.28780 shutdown 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 msgr2=0x562f906336d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 0x562f906336d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99f0012530 tx=0x7f99f0009ec0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 shutdown_connections 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x562f906a1a10 0x562f906a3e30 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f9062af60 0x562f9069fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f90625df0 0x562f90633c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 --2- 172.21.15.64:0/546812760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f90625400 0x562f906336d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f99deff5640 0 client.28780 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.197+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 >> 172.21.15.64:0/546812760 conn(0x562f90620a70 msgr2=0x562f90626f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.198+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 shutdown_connections 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.198+0000 7f9a043ac740 1 -- 172.21.15.64:0/546812760 wait complete. 2024-05-09T09:24:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.198+0000 7f9a043ac740 20 client.28780 trim_cache size 0 max 16384 2024-05-09T09:24:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:24:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:24:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:24:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:24:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.204+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e40a2d10 msgr2=0x7f99e40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.204+0000 7f99fbfff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e40a2d10 0x7f99e40a51d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99ec02a990 tx=0x7f99ec02a920 comp rx=0 tx=0).stop 2024-05-09T09:24:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.204+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc108e40 msgr2=0x7f99fc119ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.204+0000 7f99fbfff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc108e40 0x7f99fc119ef0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f99f400ccb0 tx=0x7f99f4007610 comp rx=0 tx=0).stop 2024-05-09T09:24:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.205+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 shutdown_connections 2024-05-09T09:24:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.205+0000 7f99fbfff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f99e40a2d10 0x7f99e40a51d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.205+0000 7f99fbfff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99fc109800 0x7f99fc117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.205+0000 7f99fbfff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc108e40 0x7f99fc119ef0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.206+0000 7f99fbfff640 1 --2- 172.21.15.64:0/3547137774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99fc072b60 0x7f99fc1199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.206+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 >> 172.21.15.64:0/3547137774 conn(0x7f99fc06e6d0 msgr2=0x7f99fc10c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.208+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 shutdown_connections 2024-05-09T09:24:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:30.208+0000 7f99fbfff640 1 -- 172.21.15.64:0/3547137774 wait complete. 2024-05-09T09:24:31.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 850.0 s, 753.7 tps, lat 134.384 ms stddev 87.554 2024-05-09T09:24:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:31 smithi064 ceph-mon[30149]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.8 MiB/s wr, 870 op/s 2024-05-09T09:24:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:31 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:24:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:31 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:24:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:31 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:24:31.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:31 smithi179 ceph-mon[34086]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.8 MiB/s wr, 870 op/s 2024-05-09T09:24:31.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:31 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:24:31.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:31 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:24:31.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:31 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:24:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:31 smithi139 ceph-mon[33599]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.8 MiB/s wr, 870 op/s 2024-05-09T09:24:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:31 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:24:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:31 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:24:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:31 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:24:33.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:33 smithi064 ceph-mon[30149]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.4 MiB/s wr, 615 op/s 2024-05-09T09:24:33.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:33 smithi179 ceph-mon[34086]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.4 MiB/s wr, 615 op/s 2024-05-09T09:24:33.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:33 smithi139 ceph-mon[33599]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.4 MiB/s wr, 615 op/s 2024-05-09T09:24:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:35 smithi064 ceph-mon[30149]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2024-05-09T09:24:35.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:35 smithi179 ceph-mon[34086]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2024-05-09T09:24:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:35 smithi139 ceph-mon[33599]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2024-05-09T09:24:36.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 855.0 s, 780.6 tps, lat 128.519 ms stddev 92.028 2024-05-09T09:24:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:37 smithi064 ceph-mon[30149]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 745 op/s 2024-05-09T09:24:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:37 smithi179 ceph-mon[34086]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 745 op/s 2024-05-09T09:24:37.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:37 smithi139 ceph-mon[33599]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 745 op/s 2024-05-09T09:24:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:38 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:38 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:38 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:39 smithi064 ceph-mon[30149]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 666 op/s 2024-05-09T09:24:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:39 smithi179 ceph-mon[34086]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 666 op/s 2024-05-09T09:24:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:39 smithi139 ceph-mon[33599]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 666 op/s 2024-05-09T09:24:41.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 860.0 s, 767.4 tps, lat 129.959 ms stddev 121.294 2024-05-09T09:24:41.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:41 smithi064 ceph-mon[30149]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 921 op/s 2024-05-09T09:24:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:41 smithi179 ceph-mon[34086]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 921 op/s 2024-05-09T09:24:41.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:41 smithi139 ceph-mon[33599]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 921 op/s 2024-05-09T09:24:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.305+0000 7f92125a9640 1 -- 172.21.15.64:0/4006458810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102340 msgr2=0x7f920c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.305+0000 7f92125a9640 1 --2- 172.21.15.64:0/4006458810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102340 0x7f920c108bd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9204009ca0 tx=0x7f920402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- 172.21.15.64:0/4006458810 shutdown_connections 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 --2- 172.21.15.64:0/4006458810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102340 0x7f920c108bd0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 --2- 172.21.15.64:0/4006458810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c101980 0x7f920c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 --2- 172.21.15.64:0/4006458810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f920c100790 0x7f920c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- 172.21.15.64:0/4006458810 >> 172.21.15.64:0/4006458810 conn(0x7f920c0fbd00 msgr2=0x7f920c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- 172.21.15.64:0/4006458810 shutdown_connections 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- 172.21.15.64:0/4006458810 wait complete. 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 Processor -- start 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- start start 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c100790 0x7f920c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 0x7f920c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f920c102340 0x7f920c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10d440 con 0x7f920c101980 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10d2c0 con 0x7f920c102340 2024-05-09T09:24:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f92125a9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10d5c0 con 0x7f920c100790 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f920b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 0x7f920c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f920b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 0x7f920c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48952/0 (socket says 172.21.15.64:48952) 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.306+0000 7f920b7fe640 1 -- 172.21.15.64:0/2080392232 learned_addr learned my addr 172.21.15.64:0/2080392232 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920bfff640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c100790 0x7f920c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f9210b46640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f920c102340 0x7f920c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920b7fe640 1 -- 172.21.15.64:0/2080392232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c100790 msgr2=0x7f920c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920b7fe640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c100790 0x7f920c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920b7fe640 1 -- 172.21.15.64:0/2080392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f920c102340 msgr2=0x7f920c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920b7fe640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f920c102340 0x7f920c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920b7fe640 1 -- 172.21.15.64:0/2080392232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c19f5f0 con 0x7f920c101980 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f920b7fe640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 0x7f920c197f60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f920000d950 tx=0x7f920000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f92097fa640 1 -- 172.21.15.64:0/2080392232 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200017070 con 0x7f920c101980 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f92097fa640 1 -- 172.21.15.64:0/2080392232 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9200009c30 con 0x7f920c101980 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f92097fa640 1 -- 172.21.15.64:0/2080392232 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920000f620 con 0x7f920c101980 2024-05-09T09:24:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c19f8e0 con 0x7f920c101980 2024-05-09T09:24:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.307+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c19fe80 con 0x7f920c101980 2024-05-09T09:24:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.309+0000 7f92097fa640 1 -- 172.21.15.64:0/2080392232 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92000109a0 con 0x7f920c101980 2024-05-09T09:24:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.309+0000 7f9215398740 1 Processor -- start 2024-05-09T09:24:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 -- start start 2024-05-09T09:24:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadbb7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563aadb13340 con 0x563aadbb4db0 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563aadabb290 con 0x563aadbaf250 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9215398740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563aadb0f300 con 0x563aadbafc40 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadbb7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9210b46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadbb7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48964/0 (socket says 172.21.15.64:48964) 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 -- 172.21.15.64:0/1991179138 learned_addr learned my addr 172.21.15.64:0/1991179138 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920bfff640 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 -- 172.21.15.64:0/1991179138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 msgr2=0x563aadbb4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 -- 172.21.15.64:0/1991179138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 msgr2=0x563aadbaf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f92097fa640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91f40a2cb0 0x7f91f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f920b7fe640 1 -- 172.21.15.64:0/1991179138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aadadf8e0 con 0x563aadbb4db0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9210b46640 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f9210b46640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91f40a2cb0 0x7f91f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.310+0000 7f92097fa640 1 -- 172.21.15.64:0/2080392232 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f92000ceef0 con 0x7f920c101980 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadbb7240 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f92000a41f0 tx=0x7f92000977d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e50fd60f513aada server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f91f27fc640 1 -- 172.21.15.64:0/1991179138 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200020070 con 0x563aadbb4db0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f91f27fc640 1 -- 172.21.15.64:0/1991179138 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92000a4030 con 0x563aadbb4db0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f91f27fc640 1 -- 172.21.15.64:0/1991179138 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920009afb0 con 0x563aadbb4db0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 -- 172.21.15.64:0/1991179138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 msgr2=0x563aadbb7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9210b46640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91f40a2cb0 0x7f91f40a5170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f920402f3a0 tx=0x7f9204009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadbb7240 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f92000a41f0 tx=0x7f92000977d0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 -- 172.21.15.64:0/1991179138 shutdown_connections 2024-05-09T09:24:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadbb7240 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 --2- 172.21.15.64:0/1991179138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 -- 172.21.15.64:0/1991179138 >> 172.21.15.64:0/1991179138 conn(0x563aadbaa8c0 msgr2=0x563aadbaacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.311+0000 7f9215398740 1 -- 172.21.15.64:0/1991179138 shutdown_connections 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 -- 172.21.15.64:0/1991179138 wait complete. 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 -- start start 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 Processor -- start 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 0x563aadbb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbb4db0 0x563aadbc09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563aadbb7d60 con 0x563aadbaf250 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563aadace790 con 0x563aadbb4db0 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563aadbb7ee0 con 0x563aadbafc40 2024-05-09T09:24:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9215398740 10 client.0 _ll_register_callbacks cb 0x563aadb09a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f920bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 0x563aadbb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f920bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 0x563aadbb9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48968/0 (socket says 172.21.15.64:48968) 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f920bfff640 1 -- 172.21.15.64:0/1251739444 learned_addr learned my addr 172.21.15.64:0/1251739444 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f9210b46640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbb4db0 0x563aadbc09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.312+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f920bfff640 1 -- 172.21.15.64:0/1251739444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 msgr2=0x563aadbb99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f920bfff640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f920bfff640 1 -- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbb4db0 msgr2=0x563aadbc09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f920bfff640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbb4db0 0x563aadbc09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f920bfff640 1 -- 172.21.15.64:0/1251739444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aad827210 con 0x563aadbaf250 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f920bfff640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 0x563aadbb9460 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00bdf0 tx=0x7f91fc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa27b92fa342ea9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f91eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1251739444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc004220 con 0x563aadbaf250 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1251739444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f91fc0043c0 con 0x563aadbaf250 2024-05-09T09:24:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563aadb20a70 con 0x563aadbaf250 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1251739444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f91fc005020 con 0x563aadbaf250 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 10 client.19152 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x563aad8f3620 con 0x563aadbaf250 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1251739444 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563aadb20a70 con 0x563aadbaf250 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1251739444 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f91fc019030 con 0x563aadbaf250 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 10 client.19152 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 10 client.19152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 20 client.19152 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadbc5000 0x563aadbc7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 4 client.19152 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.313+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aadb1adc0 con 0x563aadbc5000 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.314+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadbc5000 0x563aadbc7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.314+0000 7f91eb7fe640 10 client.19152 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.314+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadbc5000 0x563aadbc7470 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7823bba905a461 server_cookie=aec52810e8810860 in_seq=0 out_seq=0 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1251739444 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563aadb1adc0 con 0x563aadbc5000 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f91eb7fe640 10 client.19152 handle_command_reply: tid=0 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 client.19152 shutdown 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 msgr2=0x563aadbb9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 0x563aadbb9460 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00bdf0 tx=0x7f91fc012820 comp rx=0 tx=0).stop 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 shutdown_connections 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadbc5000 0x563aadbc7470 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbb4db0 0x563aadbc09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbb99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f91eb7fe640 0 client.19152 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 --2- 172.21.15.64:0/1251739444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbaf250 0x563aadbb9460 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 >> 172.21.15.64:0/1251739444 conn(0x563aadbaa8c0 msgr2=0x563aadba9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 shutdown_connections 2024-05-09T09:24:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 1 -- 172.21.15.64:0/1251739444 wait complete. 2024-05-09T09:24:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.315+0000 7f9215398740 20 client.19152 trim_cache size 0 max 16384 2024-05-09T09:24:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.330+0000 7f9215398740 1 Processor -- start 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.330+0000 7f9215398740 1 -- start start 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f9215398740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f9215398740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbc09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f9215398740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadba9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f9215398740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1acdc0 con 0x563aadbb4db0 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f9215398740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1aa5d0 con 0x563aadbaf250 2024-05-09T09:24:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f9215398740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1a8420 con 0x563aadbafc40 2024-05-09T09:24:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.331+0000 7f920bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.335+0000 7f920bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34788/0 (socket says 172.21.15.64:34788) 2024-05-09T09:24:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.335+0000 7f920bfff640 1 -- 172.21.15.64:0/1458753619 learned_addr learned my addr 172.21.15.64:0/1458753619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.335+0000 7f9210b46640 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadba9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.335+0000 7f920b7fe640 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbc09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f9210b46640 1 -- 172.21.15.64:0/1458753619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 msgr2=0x563aadbc09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f9210b46640 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbc09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f9210b46640 1 -- 172.21.15.64:0/1458753619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 msgr2=0x563aadbaaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f9210b46640 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f9210b46640 1 -- 172.21.15.64:0/1458753619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aadbab630 con 0x563aadbb4db0 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f920bfff640 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f9210b46640 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadba9170 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f920402ade0 tx=0x7f9204041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5444c26f8e25ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f91f1ffb640 1 -- 172.21.15.64:0/1458753619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204031d00 con 0x563aadbb4db0 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f91f1ffb640 1 -- 172.21.15.64:0/1458753619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9204055070 con 0x563aadbb4db0 2024-05-09T09:24:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.336+0000 7f91f1ffb640 1 -- 172.21.15.64:0/1458753619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204030920 con 0x563aadbb4db0 2024-05-09T09:24:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.337+0000 7f9215398740 1 -- 172.21.15.64:0/1458753619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 msgr2=0x563aadba9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.337+0000 7f9215398740 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadba9170 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f920402ade0 tx=0x7f9204041000 comp rx=0 tx=0).stop 2024-05-09T09:24:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.342+0000 7f9215398740 1 -- 172.21.15.64:0/1458753619 shutdown_connections 2024-05-09T09:24:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.342+0000 7f9215398740 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbb4db0 0x563aadba9170 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.342+0000 7f9215398740 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbafc40 0x563aadbc09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.342+0000 7f9215398740 1 --2- 172.21.15.64:0/1458753619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbaaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.342+0000 7f9215398740 1 -- 172.21.15.64:0/1458753619 >> 172.21.15.64:0/1458753619 conn(0x563aadbaa8c0 msgr2=0x563aadbb4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.349+0000 7f9215398740 1 -- 172.21.15.64:0/1458753619 shutdown_connections 2024-05-09T09:24:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.349+0000 7f9215398740 1 -- 172.21.15.64:0/1458753619 wait complete. 2024-05-09T09:24:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.350+0000 7f9215398740 1 -- start start 2024-05-09T09:24:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.350+0000 7f9215398740 1 Processor -- start 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f9215398740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbbdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f9215398740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbafc40 0x563aadbbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f920bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbbdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f920bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbbdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:34798/0 (socket says 172.21.15.64:34798) 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f920bfff640 1 -- 172.21.15.64:0/3814908279 learned_addr learned my addr 172.21.15.64:0/3814908279 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f920b7fe640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbafc40 0x563aadbbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbb4db0 0x563aadbbf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.351+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1a8a90 con 0x563aadbafc40 2024-05-09T09:24:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.352+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1a9f00 con 0x563aadbaf250 2024-05-09T09:24:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.352+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1a9160 con 0x563aadbb4db0 2024-05-09T09:24:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.352+0000 7f9215398740 10 client.0 _ll_register_callbacks cb 0x7f91fc005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.356+0000 7f9210b46640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbb4db0 0x563aadbbf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.358+0000 7f9210b46640 1 -- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 msgr2=0x563aadbbdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.358+0000 7f9210b46640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbbdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.358+0000 7f9210b46640 1 -- 172.21.15.64:0/3814908279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbafc40 msgr2=0x563aadbbe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.358+0000 7f9210b46640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbafc40 0x563aadbbe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.358+0000 7f9210b46640 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aadbbf870 con 0x563aadbb4db0 2024-05-09T09:24:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.359+0000 7f9210b46640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbb4db0 0x563aadbbf280 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9204031aa0 tx=0x7f9204004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db94b1d990012718 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.360+0000 7f91eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.360+0000 7f91eb7fe640 1 -- 172.21.15.64:0/3814908279 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204061030 con 0x563aadbb4db0 2024-05-09T09:24:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.360+0000 7f91eb7fe640 1 -- 172.21.15.64:0/3814908279 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9204042070 con 0x563aadbb4db0 2024-05-09T09:24:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.363+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9204055070 con 0x563aadbb4db0 2024-05-09T09:24:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.364+0000 7f91eb7fe640 1 -- 172.21.15.64:0/3814908279 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9204004500 con 0x563aadbb4db0 2024-05-09T09:24:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.364+0000 7f9215398740 10 client.28798 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.364+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x563aadbc0b80 con 0x563aadbb4db0 2024-05-09T09:24:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.365+0000 7f91eb7fe640 1 -- 172.21.15.64:0/3814908279 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9204036050 con 0x563aadbb4db0 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f91eb7fe640 1 -- 172.21.15.64:0/3814908279 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f92040516b0 con 0x563aadbb4db0 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f9215398740 10 client.28798 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f9215398740 10 client.28798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f9215398740 20 client.28798 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadc2baf0 0x563aadc2df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f9215398740 4 client.28798 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:24:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.366+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9204055070 con 0x563aadc2baf0 2024-05-09T09:24:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.367+0000 7f920bfff640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadc2baf0 0x563aadc2df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.368+0000 7f920bfff640 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadc2baf0 0x563aadc2df10 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41cbfb1466b0a58 server_cookie=a71ebdb909fb3b17 in_seq=0 out_seq=0 2024-05-09T09:24:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.368+0000 7f91eb7fe640 10 client.28798 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f91eb7fe640 1 -- 172.21.15.64:0/3814908279 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9204055070 con 0x563aadc2baf0 2024-05-09T09:24:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f91eb7fe640 10 client.28798 handle_command_reply: tid=0 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 client.28798 shutdown 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbb4db0 msgr2=0x563aadbbf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbb4db0 0x563aadbbf280 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9204031aa0 tx=0x7f9204004040 comp rx=0 tx=0).stop 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 shutdown_connections 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x563aadc2baf0 0x563aadc2df10 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563aadbb4db0 0x563aadbbf280 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aadbafc40 0x563aadbbe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 --2- 172.21.15.64:0/3814908279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563aadbaf250 0x563aadbbdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 >> 172.21.15.64:0/3814908279 conn(0x563aadbaa8c0 msgr2=0x563aadbad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.412+0000 7f91eb7fe640 0 client.28798 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 shutdown_connections 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f9215398740 1 -- 172.21.15.64:0/3814908279 wait complete. 2024-05-09T09:24:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f9215398740 20 client.28798 trim_cache size 0 max 16384 2024-05-09T09:24:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:24:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:24:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:24:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91f40a2cb0 msgr2=0x7f91f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91f40a2cb0 0x7f91f40a5170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f920402f3a0 tx=0x7f9204009360 comp rx=0 tx=0).stop 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 msgr2=0x7f920c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 0x7f920c197f60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f920000d950 tx=0x7f920000de10 comp rx=0 tx=0).stop 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 shutdown_connections 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91f40a2cb0 0x7f91f40a5170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f920c102340 0x7f920c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c101980 0x7f920c197f60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 --2- 172.21.15.64:0/2080392232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c100790 0x7f920c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 >> 172.21.15.64:0/2080392232 conn(0x7f920c0fbd00 msgr2=0x7f920c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 shutdown_connections 2024-05-09T09:24:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:43.413+0000 7f92125a9640 1 -- 172.21.15.64:0/2080392232 wait complete. 2024-05-09T09:24:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:43 smithi179 ceph-mon[34086]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 711 op/s 2024-05-09T09:24:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:43 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:43 smithi139 ceph-mon[33599]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 711 op/s 2024-05-09T09:24:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:43 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:43.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:43 smithi064 ceph-mon[30149]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 711 op/s 2024-05-09T09:24:43.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:43 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:45 smithi179 ceph-mon[34086]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 13 MiB/s wr, 855 op/s 2024-05-09T09:24:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:45 smithi139 ceph-mon[33599]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 13 MiB/s wr, 855 op/s 2024-05-09T09:24:45.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:45 smithi064 ceph-mon[30149]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 13 MiB/s wr, 855 op/s 2024-05-09T09:24:46.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 865.0 s, 149.2 tps, lat 371.729 ms stddev 993.816 2024-05-09T09:24:47.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:47 smithi179 ceph-mon[34086]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-09T09:24:47.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:24:47.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:24:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:47 smithi064 ceph-mon[30149]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-09T09:24:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:47 smithi139 ceph-mon[33599]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 785 op/s 2024-05-09T09:24:47.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.663+0000 7f8819658640 1 -- 172.21.15.64:0/235427629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88141011e0 msgr2=0x7f8814105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.663+0000 7f8819658640 1 --2- 172.21.15.64:0/235427629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88141011e0 0x7f8814105aa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f880c009ca0 tx=0x7f880c02a8f0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.664+0000 7f8819658640 1 -- 172.21.15.64:0/235427629 shutdown_connections 2024-05-09T09:24:47.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.664+0000 7f8819658640 1 --2- 172.21.15.64:0/235427629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88141011e0 0x7f8814105aa0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.664+0000 7f8819658640 1 --2- 172.21.15.64:0/235427629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8814100840 0x7f8814100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.664+0000 7f8819658640 1 --2- 172.21.15.64:0/235427629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88140ff710 0x7f88140ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.664+0000 7f8819658640 1 -- 172.21.15.64:0/235427629 >> 172.21.15.64:0/235427629 conn(0x7f88140fb030 msgr2=0x7f88140fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:47.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.669+0000 7f8819658640 1 -- 172.21.15.64:0/235427629 shutdown_connections 2024-05-09T09:24:47.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.671+0000 7f8819658640 1 -- 172.21.15.64:0/235427629 wait complete. 2024-05-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 Processor -- start 2024-05-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 -- start start 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88140ff710 0x7f8814110d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8814100840 0x7f8814111290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 0x7f881410f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8814107c30 con 0x7f88140ff710 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8814107ab0 con 0x7f88141011e0 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.672+0000 7f8819658640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8814107db0 con 0x7f8814100840 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 0x7f881410f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f8812ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88140ff710 0x7f8814110d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 0x7f881410f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33762/0 (socket says 172.21.15.64:33762) 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 -- 172.21.15.64:0/1425306051 learned_addr learned my addr 172.21.15.64:0/1425306051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 -- 172.21.15.64:0/1425306051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8814100840 msgr2=0x7f8814111290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8814100840 0x7f8814111290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 -- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88140ff710 msgr2=0x7f8814110d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.673+0000 7f88137fe640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88140ff710 0x7f8814110d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.674+0000 7f88137fe640 1 -- 172.21.15.64:0/1425306051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881410fb50 con 0x7f88141011e0 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.674+0000 7f88137fe640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 0x7f881410f420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f880c005e30 tx=0x7f880c030820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.674+0000 7f87fbfff640 1 -- 172.21.15.64:0/1425306051 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c030b10 con 0x7f88141011e0 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.674+0000 7f87fbfff640 1 -- 172.21.15.64:0/1425306051 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f880c02d070 con 0x7f88141011e0 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.674+0000 7f87fbfff640 1 -- 172.21.15.64:0/1425306051 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c03a480 con 0x7f88141011e0 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.674+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881410fde0 con 0x7f88141011e0 2024-05-09T09:24:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.675+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8814119110 con 0x7f88141011e0 2024-05-09T09:24:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.681+0000 7f87fbfff640 1 -- 172.21.15.64:0/1425306051 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f880c03a620 con 0x7f88141011e0 2024-05-09T09:24:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.683+0000 7f87fbfff640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f87fc0a2d90 0x7f87fc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.683+0000 7f87fbfff640 1 -- 172.21.15.64:0/1425306051 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f880c0f1730 con 0x7f88141011e0 2024-05-09T09:24:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.685+0000 7f8812ffd640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f87fc0a2d90 0x7f87fc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.687+0000 7f881c363740 1 Processor -- start 2024-05-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.688+0000 7f8812ffd640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f87fc0a2d90 0x7f87fc0a5250 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8814110b00 tx=0x7f8804009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 -- start start 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da21f30 0x55744da22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da27420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55744d986d00 con 0x55744da21f30 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55744d92e4c0 con 0x55744da22920 2024-05-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.689+0000 7f881c363740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55744d982cc0 con 0x55744da27a90 2024-05-09T09:24:47.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.693+0000 7f88137fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.693+0000 7f88137fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da29f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33074/0 (socket says 172.21.15.64:33074) 2024-05-09T09:24:47.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.693+0000 7f88137fe640 1 -- 172.21.15.64:0/3597575929 learned_addr learned my addr 172.21.15.64:0/3597575929 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:47.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.693+0000 7f8812ffd640 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da21f30 0x55744da22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.695+0000 7f8812ffd640 1 -- 172.21.15.64:0/3597575929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 msgr2=0x55744da29f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.695+0000 7f8812ffd640 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da29f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.695+0000 7f8812ffd640 1 -- 172.21.15.64:0/3597575929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 msgr2=0x55744da27420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.695+0000 7f88127fc640 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da27420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.695+0000 7f8812ffd640 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da27420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.695+0000 7f8812ffd640 1 -- 172.21.15.64:0/3597575929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55744d7647d0 con 0x55744da21f30 2024-05-09T09:24:47.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.696+0000 7f88127fc640 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da27420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:24:47.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.697+0000 7f8812ffd640 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da21f30 0x55744da22350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f880400e5a0 tx=0x7f8804011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95cb61b060046b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:47.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.697+0000 7f87f97fa640 1 -- 172.21.15.64:0/3597575929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880401b040 con 0x55744da21f30 2024-05-09T09:24:47.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.697+0000 7f87f97fa640 1 -- 172.21.15.64:0/3597575929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8804024070 con 0x55744da21f30 2024-05-09T09:24:47.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.697+0000 7f87f97fa640 1 -- 172.21.15.64:0/3597575929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880401f3f0 con 0x55744da21f30 2024-05-09T09:24:47.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 -- 172.21.15.64:0/3597575929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da21f30 msgr2=0x55744da22350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da21f30 0x55744da22350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f880400e5a0 tx=0x7f8804011000 comp rx=0 tx=0).stop 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 -- 172.21.15.64:0/3597575929 shutdown_connections 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da27420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 --2- 172.21.15.64:0/3597575929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da21f30 0x55744da22350 secure :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f880400e5a0 tx=0x7f8804011000 comp rx=0 tx=0).stop 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 -- 172.21.15.64:0/3597575929 >> 172.21.15.64:0/3597575929 conn(0x55744da1d5a0 msgr2=0x55744da1d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 -- 172.21.15.64:0/3597575929 shutdown_connections 2024-05-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.699+0000 7f881c363740 1 -- 172.21.15.64:0/3597575929 wait complete. 2024-05-09T09:24:47.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 -- start start 2024-05-09T09:24:47.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 Processor -- start 2024-05-09T09:24:47.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da2c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da2cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da31be0 0x55744da33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55744da2aa40 con 0x55744da31be0 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55744d941b90 con 0x55744da22920 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55744da2abc0 con 0x55744da27a90 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.700+0000 7f881c363740 10 client.0 _ll_register_callbacks cb 0x55744d97dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f8812ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da2c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f8812ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da2c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33786/0 (socket says 172.21.15.64:33786) 2024-05-09T09:24:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f8812ffd640 1 -- 172.21.15.64:0/949038183 learned_addr learned my addr 172.21.15.64:0/949038183 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f88127fc640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da2cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f88137fe640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da31be0 0x55744da33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f8812ffd640 1 -- 172.21.15.64:0/949038183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 msgr2=0x55744da2cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f8812ffd640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da2cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.701+0000 7f8812ffd640 1 -- 172.21.15.64:0/949038183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da31be0 msgr2=0x55744da33fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.702+0000 7f8812ffd640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da31be0 0x55744da33fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.702+0000 7f8812ffd640 1 -- 172.21.15.64:0/949038183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55744d69a3f0 con 0x55744da22920 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.702+0000 7f8812ffd640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da2c670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55744da23180 tx=0x7f8804013950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce298c52db05a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.703+0000 7f87f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.703+0000 7f87f6ff5640 1 -- 172.21.15.64:0/949038183 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804015c00 con 0x55744da22920 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.703+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8804024070 con 0x55744da22920 2024-05-09T09:24:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.703+0000 7f87f6ff5640 1 -- 172.21.15.64:0/949038183 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8804015da0 con 0x55744da22920 2024-05-09T09:24:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.704+0000 7f87f6ff5640 1 -- 172.21.15.64:0/949038183 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8804031400 con 0x55744da22920 2024-05-09T09:24:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.704+0000 7f881c363740 10 client.28649 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.704+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55744d65a0f0 con 0x55744da22920 2024-05-09T09:24:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f87f6ff5640 1 -- 172.21.15.64:0/949038183 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8804024070 con 0x55744da22920 2024-05-09T09:24:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f87f6ff5640 1 -- 172.21.15.64:0/949038183 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f880401e030 con 0x55744da22920 2024-05-09T09:24:47.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f881c363740 10 client.28649 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f881c363740 10 client.28649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f881c363740 20 client.28649 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f881c363740 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da38490 0x55744da3a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f881c363740 4 client.28649 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.706+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55744d994430 con 0x55744da38490 2024-05-09T09:24:47.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.707+0000 7f88127fc640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da38490 0x55744da3a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.707+0000 7f88127fc640 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da38490 0x55744da3a880 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f077c2b3cb442cd4 server_cookie=50ebaf20febfdc24 in_seq=0 out_seq=0 2024-05-09T09:24:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.707+0000 7f87f6ff5640 10 client.28649 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.708+0000 7f87f6ff5640 1 -- 172.21.15.64:0/949038183 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55744d994430 con 0x55744da38490 2024-05-09T09:24:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.708+0000 7f87f6ff5640 10 client.28649 handle_command_reply: tid=0 2024-05-09T09:24:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.708+0000 7f881c363740 1 client.28649 shutdown 2024-05-09T09:24:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 msgr2=0x55744da2c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da2c670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55744da23180 tx=0x7f8804013950 comp rx=0 tx=0).stop 2024-05-09T09:24:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 shutdown_connections 2024-05-09T09:24:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da38490 0x55744da3a880 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da31be0 0x55744da33fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da2cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 --2- 172.21.15.64:0/949038183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da22920 0x55744da2c670 secure :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55744da23180 tx=0x7f8804013950 comp rx=0 tx=0).stop 2024-05-09T09:24:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 >> 172.21.15.64:0/949038183 conn(0x55744da1d5a0 msgr2=0x55744da1c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.709+0000 7f87f6ff5640 0 client.28649 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:47.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.710+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 shutdown_connections 2024-05-09T09:24:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.711+0000 7f881c363740 1 -- 172.21.15.64:0/949038183 wait complete. 2024-05-09T09:24:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.711+0000 7f881c363740 20 client.28649 trim_cache size 0 max 16384 2024-05-09T09:24:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.750+0000 7f881c363740 1 Processor -- start 2024-05-09T09:24:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.750+0000 7f881c363740 1 -- start start 2024-05-09T09:24:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f881c363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da27a90 0x55744da29e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f881c363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 0x55744da2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f881c363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da2c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f881c363740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f88141aad50 con 0x55744da2b4f0 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f881c363740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8814123850 con 0x55744da27a90 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f881c363740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f88141216f0 con 0x55744da31be0 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f88127fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 0x55744da2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f88127fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 0x55744da2afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33092/0 (socket says 172.21.15.64:33092) 2024-05-09T09:24:47.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f88127fc640 1 -- 172.21.15.64:0/3713955640 learned_addr learned my addr 172.21.15.64:0/3713955640 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.751+0000 7f88137fe640 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da2c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f8812ffd640 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da27a90 0x55744da29e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f88127fc640 1 -- 172.21.15.64:0/3713955640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da27a90 msgr2=0x55744da29e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f88127fc640 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da27a90 0x55744da29e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f88127fc640 1 -- 172.21.15.64:0/3713955640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 msgr2=0x55744da2c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f88127fc640 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da2c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f88127fc640 1 -- 172.21.15.64:0/3713955640 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55744da21b70 con 0x55744da31be0 2024-05-09T09:24:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.752+0000 7f88127fc640 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 0x55744da2afb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55744da22d20 tx=0x7f8808009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28622b246874d2c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.753+0000 7f87f8ff9640 1 -- 172.21.15.64:0/3713955640 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808013bf0 con 0x55744da31be0 2024-05-09T09:24:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.753+0000 7f87f8ff9640 1 -- 172.21.15.64:0/3713955640 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8808013d90 con 0x55744da31be0 2024-05-09T09:24:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.753+0000 7f87f8ff9640 1 -- 172.21.15.64:0/3713955640 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808004a00 con 0x55744da31be0 2024-05-09T09:24:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.754+0000 7f881c363740 1 -- 172.21.15.64:0/3713955640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 msgr2=0x55744da2afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.754+0000 7f881c363740 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 0x55744da2afb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55744da22d20 tx=0x7f8808009350 comp rx=0 tx=0).stop 2024-05-09T09:24:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.755+0000 7f881c363740 1 -- 172.21.15.64:0/3713955640 shutdown_connections 2024-05-09T09:24:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.755+0000 7f881c363740 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da2c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.755+0000 7f881c363740 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da31be0 0x55744da2afb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.755+0000 7f881c363740 1 --2- 172.21.15.64:0/3713955640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da27a90 0x55744da29e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.755+0000 7f881c363740 1 -- 172.21.15.64:0/3713955640 >> 172.21.15.64:0/3713955640 conn(0x55744da1d5a0 msgr2=0x55744da23420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.756+0000 7f881c363740 1 -- 172.21.15.64:0/3713955640 shutdown_connections 2024-05-09T09:24:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.756+0000 7f881c363740 1 -- 172.21.15.64:0/3713955640 wait complete. 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.756+0000 7f881c363740 1 -- start start 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.757+0000 7f881c363740 1 Processor -- start 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da49110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da49650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da31be0 0x55744da4a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8814121d10 con 0x55744da2b4f0 2024-05-09T09:24:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8814123180 con 0x55744da31be0 2024-05-09T09:24:47.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f88141223e0 con 0x55744da27a90 2024-05-09T09:24:47.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.762+0000 7f881c363740 10 client.0 _ll_register_callbacks cb 0x7f880400ea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:47.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.763+0000 7f8812ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da49110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.763+0000 7f8812ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da49110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33094/0 (socket says 172.21.15.64:33094) 2024-05-09T09:24:47.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.763+0000 7f8812ffd640 1 -- 172.21.15.64:0/4156279059 learned_addr learned my addr 172.21.15.64:0/4156279059 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:47.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.766+0000 7f88137fe640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da31be0 0x55744da4a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.767+0000 7f88127fc640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da49650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.769+0000 7f8812ffd640 1 -- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da31be0 msgr2=0x55744da4a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.769+0000 7f8812ffd640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da31be0 0x55744da4a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.769+0000 7f8812ffd640 1 -- 172.21.15.64:0/4156279059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 msgr2=0x55744da49650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.769+0000 7f8812ffd640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da49650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.769+0000 7f8812ffd640 1 -- 172.21.15.64:0/4156279059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55744da4ac50 con 0x55744da27a90 2024-05-09T09:24:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.770+0000 7f8812ffd640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da49110 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8804029860 tx=0x7f880400e620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0bbec08a01d4605 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.770+0000 7f87f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.770+0000 7f87f6ff5640 1 -- 172.21.15.64:0/4156279059 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804013760 con 0x55744da27a90 2024-05-09T09:24:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.770+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8808013d90 con 0x55744da27a90 2024-05-09T09:24:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.771+0000 7f87f6ff5640 1 -- 172.21.15.64:0/4156279059 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8804029400 con 0x55744da27a90 2024-05-09T09:24:47.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.771+0000 7f87f6ff5640 1 -- 172.21.15.64:0/4156279059 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f880404b050 con 0x55744da27a90 2024-05-09T09:24:47.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.774+0000 7f881c363740 10 client.28810 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:47.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.774+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55744da4bf60 con 0x55744da27a90 2024-05-09T09:24:47.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f87f6ff5640 1 -- 172.21.15.64:0/4156279059 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8808013d90 con 0x55744da27a90 2024-05-09T09:24:47.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f87f6ff5640 1 -- 172.21.15.64:0/4156279059 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f8804046030 con 0x55744da27a90 2024-05-09T09:24:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f881c363740 10 client.28810 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f881c363740 10 client.28810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f881c363740 20 client.28810 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f881c363740 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da9ebd0 0x55744daa0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f881c363740 4 client.28810 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:24:47.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.777+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55744d98e780 con 0x55744da9ebd0 2024-05-09T09:24:47.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.781+0000 7f88127fc640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da9ebd0 0x55744daa0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.782+0000 7f88127fc640 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da9ebd0 0x55744daa0ff0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e0d879b9c55542 server_cookie=c37ebf3a5e814dc8 in_seq=0 out_seq=0 2024-05-09T09:24:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.782+0000 7f87f6ff5640 10 client.28810 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:47.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.782+0000 7f87f6ff5640 1 -- 172.21.15.64:0/4156279059 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55744d98e780 con 0x55744da9ebd0 2024-05-09T09:24:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.782+0000 7f87f6ff5640 10 client.28810 handle_command_reply: tid=0 2024-05-09T09:24:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.782+0000 7f881c363740 1 client.28810 shutdown 2024-05-09T09:24:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 msgr2=0x55744da49110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da49110 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8804029860 tx=0x7f880400e620 comp rx=0 tx=0).stop 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 shutdown_connections 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55744da9ebd0 0x55744daa0ff0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55744da31be0 0x55744da4a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55744da2b4f0 0x55744da49650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 --2- 172.21.15.64:0/4156279059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55744da27a90 0x55744da49110 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 >> 172.21.15.64:0/4156279059 conn(0x55744da1d5a0 msgr2=0x55744da2c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f87f6ff5640 0 client.28810 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 shutdown_connections 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 1 -- 172.21.15.64:0/4156279059 wait complete. 2024-05-09T09:24:47.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.783+0000 7f881c363740 20 client.28810 trim_cache size 0 max 16384 2024-05-09T09:24:47.784 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:24:47.784 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:24:47.784 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:24:47.784 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:24:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.784+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f87fc0a2d90 msgr2=0x7f87fc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.784+0000 7f8819658640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f87fc0a2d90 0x7f87fc0a5250 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8814110b00 tx=0x7f8804009110 comp rx=0 tx=0).stop 2024-05-09T09:24:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.784+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 msgr2=0x7f881410f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.784+0000 7f8819658640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 0x7f881410f420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f880c005e30 tx=0x7f880c030820 comp rx=0 tx=0).stop 2024-05-09T09:24:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.785+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 shutdown_connections 2024-05-09T09:24:47.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.785+0000 7f8819658640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f87fc0a2d90 0x7f87fc0a5250 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.785+0000 7f8819658640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88141011e0 0x7f881410f420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.785+0000 7f8819658640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8814100840 0x7f8814111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.785+0000 7f8819658640 1 --2- 172.21.15.64:0/1425306051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88140ff710 0x7f8814110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.785+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 >> 172.21.15.64:0/1425306051 conn(0x7f88140fb030 msgr2=0x7f8814103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.786+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 shutdown_connections 2024-05-09T09:24:47.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:47.786+0000 7f8819658640 1 -- 172.21.15.64:0/1425306051 wait complete. 2024-05-09T09:24:47.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:24:47.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:24:48.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.023+0000 7f4eb0961640 1 -- 172.21.15.64:0/3479381707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac101040 msgr2=0x7f4eac103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.023+0000 7f4eb0961640 1 --2- 172.21.15.64:0/3479381707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac101040 0x7f4eac103430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009ca0 tx=0x7f4ea402c7d0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.035+0000 7f4eb0961640 1 -- 172.21.15.64:0/3479381707 shutdown_connections 2024-05-09T09:24:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.035+0000 7f4eb0961640 1 --2- 172.21.15.64:0/3479381707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac106380 0x7f4eac108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.035+0000 7f4eb0961640 1 --2- 172.21.15.64:0/3479381707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 0x7f4eac105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.035+0000 7f4eb0961640 1 --2- 172.21.15.64:0/3479381707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac101040 0x7f4eac103430 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.035+0000 7f4eb0961640 1 -- 172.21.15.64:0/3479381707 >> 172.21.15.64:0/3479381707 conn(0x7f4eac0fb020 msgr2=0x7f4eac0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.039+0000 7f4eb0961640 1 -- 172.21.15.64:0/3479381707 shutdown_connections 2024-05-09T09:24:48.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.042+0000 7f4eb0961640 1 -- 172.21.15.64:0/3479381707 wait complete. 2024-05-09T09:24:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.043+0000 7f4eb0961640 1 Processor -- start 2024-05-09T09:24:48.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.043+0000 7f4eb0961640 1 -- start start 2024-05-09T09:24:48.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.051+0000 7f4eb0961640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac101040 0x7f4eac19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.051+0000 7f4eb0961640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 0x7f4eac19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.051+0000 7f4eb0961640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 0x7f4eac1a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.051+0000 7f4eb0961640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10f6f0 con 0x7f4eac106380 2024-05-09T09:24:48.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.051+0000 7f4eb0961640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10f570 con 0x7f4eac101040 2024-05-09T09:24:48.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.051+0000 7f4eb0961640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10f870 con 0x7f4eac103970 2024-05-09T09:24:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.052+0000 7f4eaad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 0x7f4eac1a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.052+0000 7f4eaad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 0x7f4eac1a5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59210/0 (socket says 172.21.15.64:59210) 2024-05-09T09:24:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.052+0000 7f4eaad9d640 1 -- 172.21.15.64:0/3056212547 learned_addr learned my addr 172.21.15.64:0/3056212547 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.053+0000 7f4ea9d9b640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 0x7f4eac19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.053+0000 7f4eaa59c640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac101040 0x7f4eac19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaad9d640 1 -- 172.21.15.64:0/3056212547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 msgr2=0x7f4eac19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaad9d640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 0x7f4eac19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaad9d640 1 -- 172.21.15.64:0/3056212547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac101040 msgr2=0x7f4eac19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaad9d640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac101040 0x7f4eac19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaad9d640 1 -- 172.21.15.64:0/3056212547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac1a5e70 con 0x7f4eac106380 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaa59c640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac101040 0x7f4eac19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:48.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.058+0000 7f4eaad9d640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 0x7f4eac1a5800 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009fd0 tx=0x7f4ea000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.061+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0011830 con 0x7f4eac106380 2024-05-09T09:24:48.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.061+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ea0016070 con 0x7f4eac106380 2024-05-09T09:24:48.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.061+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0010e50 con 0x7f4eac106380 2024-05-09T09:24:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.062+0000 7f4ea9d9b640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 0x7f4eac19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.063+0000 7f4eb0961640 1 -- 172.21.15.64:0/3056212547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac194e30 con 0x7f4eac106380 2024-05-09T09:24:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.063+0000 7f4eb0961640 1 -- 172.21.15.64:0/3056212547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1953b0 con 0x7f4eac106380 2024-05-09T09:24:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.064+0000 7f4eb0961640 1 -- 172.21.15.64:0/3056212547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac0fcb10 con 0x7f4eac106380 2024-05-09T09:24:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.075+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ea00119d0 con 0x7f4eac106380 2024-05-09T09:24:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.076+0000 7f4e9b7fe640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4e940a2f30 0x7f4e940a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.076+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4ea00cf560 con 0x7f4eac106380 2024-05-09T09:24:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.078+0000 7f4eaa59c640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4e940a2f30 0x7f4e940a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.080+0000 7f4eaa59c640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4e940a2f30 0x7f4e940a53f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4035720 tx=0x7f4ea400d380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.082+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ea009c150 con 0x7f4eac106380 2024-05-09T09:24:48.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.506+0000 7f4eb0961640 1 -- 172.21.15.64:0/3056212547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eac02d380 con 0x7f4eac106380 2024-05-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.508+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/3056212547 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11888 (secure 0 0 0) 0x7f4ea009ba50 con 0x7f4eac106380 2024-05-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:24:41.258489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:24:48.509 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 51 2024-05-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.512+0000 7f4e997fa640 1 -- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4e940a2f30 msgr2=0x7f4e940a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.512+0000 7f4e997fa640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4e940a2f30 0x7f4e940a53f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4035720 tx=0x7f4ea400d380 comp rx=0 tx=0).stop 2024-05-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.512+0000 7f4e997fa640 1 -- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 msgr2=0x7f4eac1a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.512+0000 7f4e997fa640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 0x7f4eac1a5800 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009fd0 tx=0x7f4ea000b8e0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.513+0000 7f4e997fa640 1 -- 172.21.15.64:0/3056212547 shutdown_connections 2024-05-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.513+0000 7f4e997fa640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4e940a2f30 0x7f4e940a53f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.514+0000 7f4e997fa640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eac106380 0x7f4eac1a5800 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.514+0000 7f4e997fa640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eac103970 0x7f4eac19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.514+0000 7f4e997fa640 1 --2- 172.21.15.64:0/3056212547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac101040 0x7f4eac19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.514+0000 7f4e997fa640 1 -- 172.21.15.64:0/3056212547 >> 172.21.15.64:0/3056212547 conn(0x7f4eac0fb020 msgr2=0x7f4eac101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.516+0000 7f4e997fa640 1 -- 172.21.15.64:0/3056212547 shutdown_connections 2024-05-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.517+0000 7f4e997fa640 1 -- 172.21.15.64:0/3056212547 wait complete. 2024-05-09T09:24:48.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:24:48.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/1424081598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 msgr2=0x7f3ff8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/1424081598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff8108830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009ca0 tx=0x7f3ff002a940 comp rx=0 tx=0).stop 2024-05-09T09:24:48.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/1424081598 shutdown_connections 2024-05-09T09:24:48.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/1424081598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff810fbc0 0x7f3ff8112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/1424081598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 0x7f3ff8109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/1424081598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff8108830 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.736+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/1424081598 >> 172.21.15.64:0/1424081598 conn(0x7f3ff806e740 msgr2=0x7f3ff8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.740+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/1424081598 shutdown_connections 2024-05-09T09:24:48.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.742+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/1424081598 wait complete. 2024-05-09T09:24:48.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 Processor -- start 2024-05-09T09:24:48.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 -- start start 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 0x7f3ff81a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff810fbc0 0x7f3ff8079a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8115b10 con 0x7f3ff8108e00 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8115990 con 0x7f3ff810fbc0 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.743+0000 7f3ffe3f6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8115c90 con 0x7f3ff8073600 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.744+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.745+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33130/0 (socket says 172.21.15.64:33130) 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.745+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/858071186 learned_addr learned my addr 172.21.15.64:0/858071186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:48.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.746+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 0x7f3ff81a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.747+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff810fbc0 0x7f3ff8079a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.748+0000 7f3ffcbf3640 1 -- 172.21.15.64:0/858071186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 msgr2=0x7f3ff81a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.748+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff81a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.748+0000 7f3ffcbf3640 1 -- 172.21.15.64:0/858071186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff810fbc0 msgr2=0x7f3ff8079a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.748+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff810fbc0 0x7f3ff8079a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.748+0000 7f3ffcbf3640 1 -- 172.21.15.64:0/858071186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff807a130 con 0x7f3ff8108e00 2024-05-09T09:24:48.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.749+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 0x7f3ff81a06d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800b3f0 tx=0x7f3fe800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.750+0000 7f3fee7fc640 1 -- 172.21.15.64:0/858071186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8015020 con 0x7f3ff8108e00 2024-05-09T09:24:48.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.750+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff807a420 con 0x7f3ff8108e00 2024-05-09T09:24:48.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.750+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff807aa40 con 0x7f3ff8108e00 2024-05-09T09:24:48.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.750+0000 7f3fee7fc640 1 -- 172.21.15.64:0/858071186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fe8011ce0 con 0x7f3ff8108e00 2024-05-09T09:24:48.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.750+0000 7f3fee7fc640 1 -- 172.21.15.64:0/858071186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8010d80 con 0x7f3ff8108e00 2024-05-09T09:24:48.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.754+0000 7f4002366740 1 Processor -- start 2024-05-09T09:24:48.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 -- start start 2024-05-09T09:24:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 0x55e097a58450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a5f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0979b3c40 con 0x55e097a5d4f0 2024-05-09T09:24:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e09795b580 con 0x55e097a58a20 2024-05-09T09:24:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.755+0000 7f4002366740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e0979afc00 con 0x55e097a58030 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 0x55e097a58450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 0x55e097a58450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33134/0 (socket says 172.21.15.64:33134) 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 -- 172.21.15.64:0/3353670107 learned_addr learned my addr 172.21.15.64:0/3353670107 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 -- 172.21.15.64:0/3353670107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 msgr2=0x55e097a5ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a5ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 -- 172.21.15.64:0/3353670107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 msgr2=0x55e097a5f980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a5f980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.756+0000 7f3ffdbf5640 1 -- 172.21.15.64:0/3353670107 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0976c73f0 con 0x55e097a58030 2024-05-09T09:24:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.757+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 0x55e097a58450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400bc70 tx=0x7f3ff400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a68f12555f3a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.757+0000 7f3fee7fc640 1 -- 172.21.15.64:0/858071186 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3fe8018020 con 0x7f3ff8108e00 2024-05-09T09:24:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.757+0000 7f3fe37fe640 1 -- 172.21.15.64:0/3353670107 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4015070 con 0x55e097a58030 2024-05-09T09:24:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.757+0000 7f3fe37fe640 1 -- 172.21.15.64:0/3353670107 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ff401a910 con 0x55e097a58030 2024-05-09T09:24:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.757+0000 7f3fe37fe640 1 -- 172.21.15.64:0/3353670107 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4011830 con 0x55e097a58030 2024-05-09T09:24:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f3fee7fc640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3fe40a2ce0 0x7f3fe40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f3fee7fc640 1 -- 172.21.15.64:0/858071186 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3fe80cea80 con 0x7f3ff8108e00 2024-05-09T09:24:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3fe40a2ce0 0x7f3fe40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 -- 172.21.15.64:0/3353670107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 msgr2=0x55e097a58450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 0x55e097a58450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400bc70 tx=0x7f3ff400f730 comp rx=0 tx=0).stop 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 -- 172.21.15.64:0/3353670107 shutdown_connections 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a5f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 --2- 172.21.15.64:0/3353670107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a58030 0x55e097a58450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.758+0000 7f4002366740 1 -- 172.21.15.64:0/3353670107 >> 172.21.15.64:0/3353670107 conn(0x55e097a4a720 msgr2=0x55e097a4ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.759+0000 7f4002366740 1 -- 172.21.15.64:0/3353670107 shutdown_connections 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.759+0000 7f4002366740 1 -- 172.21.15.64:0/3353670107 wait complete. 2024-05-09T09:24:48.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.759+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3fe40a2ce0 0x7f3fe40a51a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55e097a58e20 tx=0x7f3ff0009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.759+0000 7f4002366740 1 -- start start 2024-05-09T09:24:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.759+0000 7f4002366740 1 Processor -- start 2024-05-09T09:24:48.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 0x55e097a61de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a58a20 0x55e097a62320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a5d4f0 0x55e097a69360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e097a604a0 con 0x55e097a58a20 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e09796ec40 con 0x55e097a58030 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e097a60620 con 0x55e097a5d4f0 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f4002366740 10 client.0 _ll_register_callbacks cb 0x55e0976c6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 0x55e097a61de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 0x55e097a61de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33854/0 (socket says 172.21.15.64:33854) 2024-05-09T09:24:48.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/862229128 learned_addr learned my addr 172.21.15.64:0/862229128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a58a20 0x55e097a62320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a5d4f0 0x55e097a69360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a5d4f0 msgr2=0x55e097a69360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a5d4f0 0x55e097a69360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/862229128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a58a20 msgr2=0x55e097a62320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a58a20 0x55e097a62320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/862229128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0977917d0 con 0x55e097a58030 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a5d4f0 0x55e097a69360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.760+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a58a20 0x55e097a62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.761+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 0x55e097a61de0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006010 tx=0x7f3ff00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb24a5fe59901ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.761+0000 7f3fe0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.761+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/862229128 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0034040 con 0x55e097a58030 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.761+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/862229128 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ff0003f70 con 0x55e097a58030 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.761+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff401a910 con 0x55e097a58030 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/862229128 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0004170 con 0x55e097a58030 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 10 client.28667 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e097a6c740 con 0x55e097a58030 2024-05-09T09:24:48.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/862229128 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ff401a910 con 0x55e097a58030 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/862229128 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f3ff0050070 con 0x55e097a58030 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 10 client.28667 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 10 client.28667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 20 client.28667 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097a6ddf0 0x55e097a70230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 4 client.28667 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0979c1370 con 0x55e097a6ddf0 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.762+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097a6ddf0 0x55e097a70230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.763+0000 7f3fe0ff9640 10 client.28667 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.763+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097a6ddf0 0x55e097a70230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=215e79f0fdb63831 server_cookie=cc4cd70de045b619 in_seq=0 out_seq=0 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.764+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/862229128 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0979c1370 con 0x55e097a6ddf0 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.764+0000 7f3fe0ff9640 10 client.28667 handle_command_reply: tid=0 2024-05-09T09:24:48.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.764+0000 7f4002366740 1 client.28667 shutdown 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.764+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 msgr2=0x55e097a61de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.764+0000 7f4002366740 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 0x55e097a61de0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006010 tx=0x7f3ff00095c0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 shutdown_connections 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097a6ddf0 0x55e097a70230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a5d4f0 0x55e097a69360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a58a20 0x55e097a62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 --2- 172.21.15.64:0/862229128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58030 0x55e097a61de0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 >> 172.21.15.64:0/862229128 conn(0x55e097a4a720 msgr2=0x55e097a576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f3fe0ff9640 0 client.28667 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 shutdown_connections 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 1 -- 172.21.15.64:0/862229128 wait complete. 2024-05-09T09:24:48.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.765+0000 7f4002366740 20 client.28667 trim_cache size 0 max 16384 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.789+0000 7f4002366740 1 Processor -- start 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.789+0000 7f4002366740 1 -- start start 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f4002366740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f4002366740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a58e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33870/0 (socket says 172.21.15.64:33870) 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/3344001204 learned_addr learned my addr 172.21.15.64:0/3344001204 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f4002366740 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a59ee0 0x55e097a619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc012550 con 0x55e097a5d4f0 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00fd60 con 0x55e097a58a20 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.790+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00dbb0 con 0x55e097a59ee0 2024-05-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.791+0000 7f3ffdbf5640 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a59ee0 0x55e097a619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.791+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/3344001204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a59ee0 msgr2=0x55e097a619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.791+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a59ee0 0x55e097a619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.791+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/3344001204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 msgr2=0x55e097a69320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.791+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a69320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.791+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/3344001204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e097a4b490 con 0x55e097a58a20 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.792+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a58e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0052c20 tx=0x7f3ff002f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805a31a6417c9b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.792+0000 7f3fe2ffd640 1 -- 172.21.15.64:0/3344001204 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0034030 con 0x55e097a58a20 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.792+0000 7f3fe2ffd640 1 -- 172.21.15.64:0/3344001204 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ff0030a10 con 0x55e097a58a20 2024-05-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.792+0000 7f3fe2ffd640 1 -- 172.21.15.64:0/3344001204 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0048690 con 0x55e097a58a20 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.793+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 msgr2=0x55e097a58e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.793+0000 7f4002366740 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a58e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0052c20 tx=0x7f3ff002f500 comp rx=0 tx=0).stop 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.793+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 shutdown_connections 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.793+0000 7f4002366740 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a59ee0 0x55e097a619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.793+0000 7f4002366740 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a69320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.793+0000 7f4002366740 1 --2- 172.21.15.64:0/3344001204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a58e50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.794+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 >> 172.21.15.64:0/3344001204 conn(0x55e097a4a720 msgr2=0x55e097a597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.794+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 shutdown_connections 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.794+0000 7f4002366740 1 -- 172.21.15.64:0/3344001204 wait complete. 2024-05-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.794+0000 7f4002366740 1 -- start start 2024-05-09T09:24:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.795+0000 7f4002366740 1 Processor -- start 2024-05-09T09:24:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.795+0000 7f4002366740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.795+0000 7f4002366740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a66ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.795+0000 7f4002366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a67af0 0x55e097a67f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffcbf3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a66ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffcbf3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a66ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59266/0 (socket says 172.21.15.64:59266) 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffd3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33882/0 (socket says 172.21.15.64:33882) 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffdbf5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a67af0 0x55e097a67f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffdbf5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a67af0 0x55e097a67f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33168/0 (socket says 172.21.15.64:33168) 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffcbf3640 1 -- 172.21.15.64:0/1841124693 learned_addr learned my addr 172.21.15.64:0/1841124693 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00e220 con 0x55e097a5d4f0 2024-05-09T09:24:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00f690 con 0x55e097a58a20 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00e8f0 con 0x55e097a67af0 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f4002366740 10 client.0 _ll_register_callbacks cb 0x7f3ff002f390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a67af0 msgr2=0x55e097a67f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a67af0 0x55e097a67f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/1841124693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 msgr2=0x55e097a66ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.796+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a66ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f3ffd3f4640 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e097a684b0 con 0x55e097a58a20 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f3ffd3f4640 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a665a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000f80 tx=0x7f3ff0004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d4bdde82ab7aad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f3fe0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/1841124693 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0034040 con 0x55e097a58a20 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/1841124693 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3ff0052980 con 0x55e097a58a20 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff0030a10 con 0x55e097a58a20 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/1841124693 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0048430 con 0x55e097a58a20 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f4002366740 10 client.28673 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.797+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e097ad43b0 con 0x55e097a58a20 2024-05-09T09:24:48.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/1841124693 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ff0048b40 con 0x55e097a58a20 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/1841124693 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f3ff0050030 con 0x55e097a58a20 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f4002366740 10 client.28673 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f4002366740 10 client.28673 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f4002366740 20 client.28673 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f4002366740 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097ad4c70 0x55e097ad7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f4002366740 4 client.28673 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.798+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3ff0030a10 con 0x55e097ad4c70 2024-05-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.799+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097ad4c70 0x55e097ad7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.799+0000 7f3fe0ff9640 10 client.28673 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.799+0000 7f3ffcbf3640 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097ad4c70 0x55e097ad7090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b77e37ed211d3aea server_cookie=4696de251b4a74b3 in_seq=0 out_seq=0 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f3fe0ff9640 1 -- 172.21.15.64:0/1841124693 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3ff0030a10 con 0x55e097ad4c70 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f3fe0ff9640 10 client.28673 handle_command_reply: tid=0 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 client.28673 shutdown 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 msgr2=0x55e097a665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a665a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000f80 tx=0x7f3ff0004540 comp rx=0 tx=0).stop 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 shutdown_connections 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x55e097ad4c70 0x55e097ad7090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f3fe0ff9640 0 client.28673 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e097a67af0 0x55e097a67f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e097a5d4f0 0x55e097a66ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 --2- 172.21.15.64:0/1841124693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e097a58a20 0x55e097a665a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.800+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 >> 172.21.15.64:0/1841124693 conn(0x55e097a4a720 msgr2=0x55e097a619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.801+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 shutdown_connections 2024-05-09T09:24:48.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.801+0000 7f4002366740 1 -- 172.21.15.64:0/1841124693 wait complete. 2024-05-09T09:24:48.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.801+0000 7f4002366740 20 client.28673 trim_cache size 0 max 16384 2024-05-09T09:24:48.803 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.802+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3fe40a2ce0 msgr2=0x7f3fe40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.802+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3fe40a2ce0 0x7f3fe40a51a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55e097a58e20 tx=0x7f3ff0009800 comp rx=0 tx=0).stop 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.802+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 msgr2=0x7f3ff81a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.802+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 0x7f3ff81a06d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800b3f0 tx=0x7f3fe800b8b0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 shutdown_connections 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3fe40a2ce0 0x7f3fe40a51a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff810fbc0 0x7f3ff8079a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff8108e00 0x7f3ff81a06d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 --2- 172.21.15.64:0/858071186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff8073600 0x7f3ff81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 >> 172.21.15.64:0/858071186 conn(0x7f3ff806e740 msgr2=0x7f3ff810bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 shutdown_connections 2024-05-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:48.803+0000 7f3ffe3f6640 1 -- 172.21.15.64:0/858071186 wait complete. 2024-05-09T09:24:48.821 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:24:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 -- 172.21.15.64:0/3362001772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 msgr2=0x7f6d14111fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 --2- 172.21.15.64:0/3362001772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 0x7f6d14111fe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009ca0 tx=0x7f6d0c02a960 comp rx=0 tx=0).stop 2024-05-09T09:24:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 -- 172.21.15.64:0/3362001772 shutdown_connections 2024-05-09T09:24:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 --2- 172.21.15.64:0/3362001772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 0x7f6d14111fe0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 --2- 172.21.15.64:0/3362001772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 --2- 172.21.15.64:0/3362001772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d140735e0 0x7f6d14108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.026+0000 7f6d1359e640 1 -- 172.21.15.64:0/3362001772 >> 172.21.15.64:0/3362001772 conn(0x7f6d1406e930 msgr2=0x7f6d14070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.028+0000 7f6d1359e640 1 -- 172.21.15.64:0/3362001772 shutdown_connections 2024-05-09T09:24:49.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.028+0000 7f6d1359e640 1 -- 172.21.15.64:0/3362001772 wait complete. 2024-05-09T09:24:49.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.029+0000 7f6d1359e640 1 Processor -- start 2024-05-09T09:24:49.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d1359e640 1 -- start start 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d1359e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d140735e0 0x7f6d141a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d1359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d1359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 0x7f6d141a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d11d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d1259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d140735e0 0x7f6d141a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d11d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33894/0 (socket says 172.21.15.64:33894) 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d11d9b640 1 -- 172.21.15.64:0/1232350948 learned_addr learned my addr 172.21.15.64:0/1232350948 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.030+0000 7f6d1359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14115b70 con 0x7f6d1410fb70 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.031+0000 7f6d12d9d640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 0x7f6d141a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.031+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d141159f0 con 0x7f6d14108d70 2024-05-09T09:24:49.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.031+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14115cf0 con 0x7f6d140735e0 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.032+0000 7f6d11d9b640 1 -- 172.21.15.64:0/1232350948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d140735e0 msgr2=0x7f6d141a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.032+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d140735e0 0x7f6d141a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.032+0000 7f6d11d9b640 1 -- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 msgr2=0x7f6d141a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.032+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 0x7f6d141a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.032+0000 7f6d11d9b640 1 -- 172.21.15.64:0/1232350948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d141a7ee0 con 0x7f6d14108d70 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.032+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141a0800 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00b470 tx=0x7f6cfc00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.033+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/1232350948 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc01d070 con 0x7f6d14108d70 2024-05-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.033+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d141099a0 con 0x7f6d14108d70 2024-05-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.033+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d14109fc0 con 0x7f6d14108d70 2024-05-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.033+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/1232350948 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cfc018440 con 0x7f6d14108d70 2024-05-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.033+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/1232350948 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc016580 con 0x7f6d14108d70 2024-05-09T09:24:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.039+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/1232350948 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cfc020070 con 0x7f6d14108d70 2024-05-09T09:24:49.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.040+0000 7f6d1b78a740 1 Processor -- start 2024-05-09T09:24:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 -- start start 2024-05-09T09:24:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d0b7fe640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d000ab4f0 0x7f6d000ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fb4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fb95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 0x562f28fbc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f28f18c40 con 0x562f28fb9c60 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f28ec0580 con 0x562f28fb4af0 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562f28f14c00 con 0x562f28fb4100 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/1232350948 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6cfc0ce850 con 0x7f6d14108d70 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.041+0000 7f6d11d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fb95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.042+0000 7f6d11d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fb95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33906/0 (socket says 172.21.15.64:33906) 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.042+0000 7f6d11d9b640 1 -- 172.21.15.64:0/2405652820 learned_addr learned my addr 172.21.15.64:0/2405652820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.043+0000 7f6d11d9b640 1 -- 172.21.15.64:0/2405652820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 msgr2=0x562f28fb4520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.043+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fb4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.043+0000 7f6d11d9b640 1 -- 172.21.15.64:0/2405652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 msgr2=0x562f28fbc0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.043+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 0x562f28fbc0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.043+0000 7f6d11d9b640 1 -- 172.21.15.64:0/2405652820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f28c2c3f0 con 0x562f28fb4af0 2024-05-09T09:24:49.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.043+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fb95f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc09ba60 tx=0x7f6cfc012070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed668cb299a7c8bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.044+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2405652820 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc09fe80 con 0x562f28fb4af0 2024-05-09T09:24:49.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.044+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2405652820 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cfc0a4740 con 0x562f28fb4af0 2024-05-09T09:24:49.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.044+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2405652820 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc098140 con 0x562f28fb4af0 2024-05-09T09:24:49.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.045+0000 7f6d12d9d640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d000ab4f0 0x7f6d000ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.046+0000 7f6d1b78a740 1 -- 172.21.15.64:0/2405652820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 msgr2=0x562f28fb95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.046+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fb95f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc09ba60 tx=0x7f6cfc012070 comp rx=0 tx=0).stop 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.046+0000 7f6d12d9d640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d000ab4f0 0x7f6d000ad9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c02f9b0 tx=0x7f6d0c009910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.047+0000 7f6d1b78a740 1 -- 172.21.15.64:0/2405652820 shutdown_connections 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.047+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 0x562f28fbc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.047+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fb95f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.047+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/2405652820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fb4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.047+0000 7f6d1b78a740 1 -- 172.21.15.64:0/2405652820 >> 172.21.15.64:0/2405652820 conn(0x562f28faf770 msgr2=0x562f28fafb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.047+0000 7f6d1b78a740 1 -- 172.21.15.64:0/2405652820 shutdown_connections 2024-05-09T09:24:49.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 -- 172.21.15.64:0/2405652820 wait complete. 2024-05-09T09:24:49.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 -- start start 2024-05-09T09:24:49.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 Processor -- start 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb9c60 0x562f28fc5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.048+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f28fbc9f0 con 0x562f28fb4af0 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562f28ed3c40 con 0x562f28fb9c60 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562f28fbcb70 con 0x562f28fb4100 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1b78a740 10 client.0 _ll_register_callbacks cb 0x562f28c2bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:49.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbe410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33210/0 (socket says 172.21.15.64:33210) 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 -- 172.21.15.64:0/1906087094 learned_addr learned my addr 172.21.15.64:0/1906087094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 -- 172.21.15.64:0/1906087094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb9c60 msgr2=0x562f28fc5990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb9c60 0x562f28fc5990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 -- 172.21.15.64:0/1906087094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 msgr2=0x562f28fbe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fbe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.049+0000 7f6d1259c640 1 -- 172.21.15.64:0/1906087094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f28cf67d0 con 0x562f28fb4100 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6d1259c640 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbe410 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400e480 tx=0x7f6d0400e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3d46703ac5e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1906087094 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04017070 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1906087094 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d04013680 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc0a4740 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1906087094 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6d04023050 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6d1b78a740 10 client.28837 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.050+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562f28fc8dd0 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1906087094 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cfc0a4740 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1906087094 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f6d040128e0 con 0x562f28fb4100 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6d1b78a740 10 client.28837 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6d1b78a740 10 client.28837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6d1b78a740 20 client.28837 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f28fca490 0x562f28fcc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6d1b78a740 4 client.28837 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:49.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.051+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f28f26370 con 0x562f28fca490 2024-05-09T09:24:49.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.052+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f28fca490 0x562f28fcc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.052+0000 7f6cf9ffb640 10 client.28837 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:49.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.053+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f28fca490 0x562f28fcc930 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3873c91b9161f1a7 server_cookie=1ff7a9cf797ca6ec in_seq=0 out_seq=0 2024-05-09T09:24:49.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.056+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1906087094 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f28f26370 con 0x562f28fca490 2024-05-09T09:24:49.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.056+0000 7f6cf9ffb640 10 client.28837 handle_command_reply: tid=0 2024-05-09T09:24:49.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.056+0000 7f6d1b78a740 1 client.28837 shutdown 2024-05-09T09:24:49.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 msgr2=0x562f28fbe410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbe410 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400e480 tx=0x7f6d0400e940 comp rx=0 tx=0).stop 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 shutdown_connections 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb9c60 0x562f28fc5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f28fca490 0x562f28fcc930 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fbe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1906087094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbe410 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 >> 172.21.15.64:0/1906087094 conn(0x562f28faf770 msgr2=0x562f28fae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.057+0000 7f6cf9ffb640 0 client.28837 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.058+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 shutdown_connections 2024-05-09T09:24:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.058+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1906087094 wait complete. 2024-05-09T09:24:49.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.058+0000 7f6d1b78a740 20 client.28837 trim_cache size 0 max 16384 2024-05-09T09:24:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 Processor -- start 2024-05-09T09:24:49.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 -- start start 2024-05-09T09:24:49.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fbdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 0x562f28fbe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4034550 con 0x562f28fb9c60 2024-05-09T09:24:49.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4031d60 con 0x562f28fb4af0 2024-05-09T09:24:49.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.094+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf402fbb0 con 0x562f28fb4100 2024-05-09T09:24:49.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.095+0000 7f6d11d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fbdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.095+0000 7f6d11d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fbdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33924/0 (socket says 172.21.15.64:33924) 2024-05-09T09:24:49.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.095+0000 7f6d11d9b640 1 -- 172.21.15.64:0/3134434390 learned_addr learned my addr 172.21.15.64:0/3134434390 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.096+0000 7f6d11d9b640 1 -- 172.21.15.64:0/3134434390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 msgr2=0x562f28fbd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.096+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.096+0000 7f6d11d9b640 1 -- 172.21.15.64:0/3134434390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 msgr2=0x562f28fbe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.096+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 0x562f28fbe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.096+0000 7f6d11d9b640 1 -- 172.21.15.64:0/3134434390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f28fb04e0 con 0x562f28fb4af0 2024-05-09T09:24:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.097+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fbdc20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0984a0 tx=0x7f6cfc00bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94bebd2899f0cb53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.097+0000 7f6cfbfff640 1 -- 172.21.15.64:0/3134434390 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc09fe80 con 0x562f28fb4af0 2024-05-09T09:24:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.097+0000 7f6cfbfff640 1 -- 172.21.15.64:0/3134434390 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cfc0a4740 con 0x562f28fb4af0 2024-05-09T09:24:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 -- 172.21.15.64:0/3134434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 msgr2=0x562f28fbdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fbdc20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0984a0 tx=0x7f6cfc00bf20 comp rx=0 tx=0).stop 2024-05-09T09:24:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 -- 172.21.15.64:0/3134434390 shutdown_connections 2024-05-09T09:24:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb9c60 0x562f28fbe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4af0 0x562f28fbdc20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/3134434390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb4100 0x562f28fbd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.098+0000 7f6d1b78a740 1 -- 172.21.15.64:0/3134434390 >> 172.21.15.64:0/3134434390 conn(0x562f28faf770 msgr2=0x562f28fae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.101+0000 7f6d1b78a740 1 -- 172.21.15.64:0/3134434390 shutdown_connections 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.101+0000 7f6d1b78a740 1 -- 172.21.15.64:0/3134434390 wait complete. 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.101+0000 7f6d1b78a740 1 -- start start 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.101+0000 7f6d1b78a740 1 Processor -- start 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4100 0x562f28fc2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fc3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb9c60 0x562f28fc4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4030220 con 0x562f28fb4af0 2024-05-09T09:24:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4031690 con 0x562f28fb4100 2024-05-09T09:24:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40308f0 con 0x562f28fb9c60 2024-05-09T09:24:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1b78a740 10 client.0 _ll_register_callbacks cb 0x7f6d04009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d12d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb9c60 0x562f28fc4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d1259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4100 0x562f28fc2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d12d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb9c60 0x562f28fc4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33220/0 (socket says 172.21.15.64:33220) 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d12d9d640 1 -- 172.21.15.64:0/1907666566 learned_addr learned my addr 172.21.15.64:0/1907666566 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.102+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fc3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6d1259c640 1 -- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb9c60 msgr2=0x562f28fc4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6d1259c640 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb9c60 0x562f28fc4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6d1259c640 1 -- 172.21.15.64:0/1907666566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 msgr2=0x562f28fc3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6d1259c640 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fc3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6d1259c640 1 -- 172.21.15.64:0/1907666566 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f28fc4940 con 0x562f28fb4100 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6d1259c640 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4100 0x562f28fc2e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d04012f70 tx=0x7f6d0401b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442a81a6f4a33bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.103+0000 7f6cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.104+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1907666566 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04017070 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.104+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1907666566 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d040127e0 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.104+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc0a4740 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.104+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1907666566 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6d04012a20 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.104+0000 7f6d1b78a740 10 client.28685 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.104+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562f29031280 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1907666566 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cfc0a4740 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1907666566 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f6d0401c440 con 0x562f28fb4100 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d1b78a740 10 client.28685 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d1b78a740 10 client.28685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d1b78a740 20 client.28685 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:49.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f29031be0 0x562f29034000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d1b78a740 4 client.28685 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:24:49.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f28f206c0 con 0x562f29031be0 2024-05-09T09:24:49.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.105+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f29031be0 0x562f29034000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6cf9ffb640 10 client.28685 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6d11d9b640 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f29031be0 0x562f29034000 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56c19f1928636976 server_cookie=fade4fe44728d904 in_seq=0 out_seq=0 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6cf9ffb640 1 -- 172.21.15.64:0/1907666566 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f28f206c0 con 0x562f29031be0 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6cf9ffb640 10 client.28685 handle_command_reply: tid=0 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6d1b78a740 1 client.28685 shutdown 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4100 msgr2=0x562f28fc2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.106+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4100 0x562f28fc2e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d04012f70 tx=0x7f6d0401b660 comp rx=0 tx=0).stop 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 shutdown_connections 2024-05-09T09:24:49.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f28fb9c60 0x562f28fc4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x562f29031be0 0x562f29034000 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f28fb4af0 0x562f28fc3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 --2- 172.21.15.64:0/1907666566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562f28fb4100 0x562f28fc2e50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 >> 172.21.15.64:0/1907666566 conn(0x562f28faf770 msgr2=0x562f28fb26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6cf9ffb640 0 client.28685 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 shutdown_connections 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 1 -- 172.21.15.64:0/1907666566 wait complete. 2024-05-09T09:24:49.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.107+0000 7f6d1b78a740 20 client.28685 trim_cache size 0 max 16384 2024-05-09T09:24:49.109 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.108+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d000ab4f0 msgr2=0x7f6d000ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.108+0000 7f6d1359e640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d000ab4f0 0x7f6d000ad9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c02f9b0 tx=0x7f6d0c009910 comp rx=0 tx=0).stop 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.108+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 msgr2=0x7f6d141a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.108+0000 7f6d1359e640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141a0800 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00b470 tx=0x7f6cfc00b870 comp rx=0 tx=0).stop 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.109+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 shutdown_connections 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.109+0000 7f6d1359e640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6d000ab4f0 0x7f6d000ad9b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.109+0000 7f6d1359e640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1410fb70 0x7f6d141a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.109+0000 7f6d1359e640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d14108d70 0x7f6d141a0800 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.109+0000 7f6d1359e640 1 --2- 172.21.15.64:0/1232350948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d140735e0 0x7f6d141a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.109+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 >> 172.21.15.64:0/1232350948 conn(0x7f6d1406e930 msgr2=0x7f6d1410bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.113+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 shutdown_connections 2024-05-09T09:24:49.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.113+0000 7f6d1359e640 1 -- 172.21.15.64:0/1232350948 wait complete. 2024-05-09T09:24:49.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:24:49.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 -- 172.21.15.64:0/294730030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 msgr2=0x7f6118109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 --2- 172.21.15.64:0/294730030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118109460 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6110009860 tx=0x7f611002a340 comp rx=0 tx=0).stop 2024-05-09T09:24:49.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 -- 172.21.15.64:0/294730030 shutdown_connections 2024-05-09T09:24:49.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 --2- 172.21.15.64:0/294730030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61181099a0 0x7f61181102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 --2- 172.21.15.64:0/294730030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118109460 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 --2- 172.21.15.64:0/294730030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6118107e90 0x7f61181082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.331+0000 7f611cb70640 1 -- 172.21.15.64:0/294730030 >> 172.21.15.64:0/294730030 conn(0x7f6118076e30 msgr2=0x7f6118079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.332+0000 7f611cb70640 1 -- 172.21.15.64:0/294730030 shutdown_connections 2024-05-09T09:24:49.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.332+0000 7f611cb70640 1 -- 172.21.15.64:0/294730030 wait complete. 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.332+0000 7f611cb70640 1 Processor -- start 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.332+0000 7f611cb70640 1 -- start start 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611cb70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 0x7f61180712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611cb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611cb70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61181099a0 0x7f6118072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 0x7f61180712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 0x7f61180712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59336/0 (socket says 172.21.15.64:59336) 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611cb70640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f611806bb50 con 0x7f6118107e90 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 -- 172.21.15.64:0/1600025381 learned_addr learned my addr 172.21.15.64:0/1600025381 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f611806b9d0 con 0x7f6118108fe0 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f611806bcd0 con 0x7f61181099a0 2024-05-09T09:24:49.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f6115d9b640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f6116d9d640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61181099a0 0x7f6118072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 -- 172.21.15.64:0/1600025381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61181099a0 msgr2=0x7f6118072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61181099a0 0x7f6118072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 -- 172.21.15.64:0/1600025381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 msgr2=0x7f6118071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f611659c640 1 -- 172.21.15.64:0/1600025381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6118072ed0 con 0x7f6118107e90 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.333+0000 7f6115d9b640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.334+0000 7f611659c640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 0x7f61180712e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c920 tx=0x7f610c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.334+0000 7f61077fe640 1 -- 172.21.15.64:0/1600025381 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c007c70 con 0x7f6118107e90 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.334+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61180731c0 con 0x7f6118107e90 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.334+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6118073730 con 0x7f6118107e90 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.334+0000 7f61077fe640 1 -- 172.21.15.64:0/1600025381 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f610c007e10 con 0x7f6118107e90 2024-05-09T09:24:49.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.334+0000 7f61077fe640 1 -- 172.21.15.64:0/1600025381 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c004e10 con 0x7f6118107e90 2024-05-09T09:24:49.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 Processor -- start 2024-05-09T09:24:49.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 -- start start 2024-05-09T09:24:49.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc57e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc583430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc583aa0 0x5629dc585f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5629dc4e2c40 con 0x5629dc57df40 2024-05-09T09:24:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5629dc48a580 con 0x5629dc583aa0 2024-05-09T09:24:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f611f883740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629dc4dec00 con 0x5629dc57e930 2024-05-09T09:24:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.338+0000 7f6116d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc57e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.339+0000 7f6116d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc57e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59340/0 (socket says 172.21.15.64:59340) 2024-05-09T09:24:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.339+0000 7f6116d9d640 1 -- 172.21.15.64:0/4136051590 learned_addr learned my addr 172.21.15.64:0/4136051590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.339+0000 7f61077fe640 1 -- 172.21.15.64:0/1600025381 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f610c004fb0 con 0x7f6118107e90 2024-05-09T09:24:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.340+0000 7f6116d9d640 1 -- 172.21.15.64:0/4136051590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 msgr2=0x5629dc583430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.340+0000 7f6116d9d640 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc583430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.340+0000 7f6116d9d640 1 -- 172.21.15.64:0/4136051590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc583aa0 msgr2=0x5629dc585f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:24:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.340+0000 7f6116d9d640 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc583aa0 0x5629dc585f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.340+0000 7f6116d9d640 1 -- 172.21.15.64:0/4136051590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dc1f63f0 con 0x5629dc57df40 2024-05-09T09:24:49.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.341+0000 7f61077fe640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61000a2be0 0x7f61000a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.341+0000 7f61077fe640 1 -- 172.21.15.64:0/1600025381 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f610c0cf440 con 0x7f6118107e90 2024-05-09T09:24:49.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.341+0000 7f611659c640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61000a2be0 0x7f61000a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.341+0000 7f6116d9d640 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc57e360 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f610800f480 tx=0x7f610800f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47e5d897667c04a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.342+0000 7f6104ff9640 1 -- 172.21.15.64:0/4136051590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610801d070 con 0x5629dc57df40 2024-05-09T09:24:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.343+0000 7f6104ff9640 1 -- 172.21.15.64:0/4136051590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f610800cb50 con 0x5629dc57df40 2024-05-09T09:24:49.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.343+0000 7f6104ff9640 1 -- 172.21.15.64:0/4136051590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108018c30 con 0x5629dc57df40 2024-05-09T09:24:49.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.343+0000 7f611f883740 1 -- 172.21.15.64:0/4136051590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 msgr2=0x5629dc57e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.343+0000 7f611f883740 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc57e360 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f610800f480 tx=0x7f610800f940 comp rx=0 tx=0).stop 2024-05-09T09:24:49.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611659c640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61000a2be0 0x7f61000a50a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f610c09b610 tx=0x7f610c0cf740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 -- 172.21.15.64:0/4136051590 shutdown_connections 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc583aa0 0x5629dc585f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc583430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 --2- 172.21.15.64:0/4136051590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc57e360 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 -- 172.21.15.64:0/4136051590 >> 172.21.15.64:0/4136051590 conn(0x5629dc5795b0 msgr2=0x5629dc5799b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 -- 172.21.15.64:0/4136051590 shutdown_connections 2024-05-09T09:24:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.344+0000 7f611f883740 1 -- 172.21.15.64:0/4136051590 wait complete. 2024-05-09T09:24:49.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 -- start start 2024-05-09T09:24:49.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 Processor -- start 2024-05-09T09:24:49.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc5882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc588820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc583aa0 0x5629dc58f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5629dc586a50 con 0x5629dc57df40 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5629dc49dc40 con 0x5629dc57e930 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629dc586bd0 con 0x5629dc583aa0 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f611f883740 10 client.0 _ll_register_callbacks cb 0x5629dc1f5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f6116d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc5882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f6116d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc5882e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59354/0 (socket says 172.21.15.64:59354) 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.345+0000 7f6116d9d640 1 -- 172.21.15.64:0/2121697129 learned_addr learned my addr 172.21.15.64:0/2121697129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6115d9b640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc588820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc583aa0 0x5629dc58f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 -- 172.21.15.64:0/2121697129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc583aa0 msgr2=0x5629dc58f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc583aa0 0x5629dc58f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 -- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 msgr2=0x5629dc588820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc588820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 -- 172.21.15.64:0/2121697129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dc2c07d0 con 0x5629dc57df40 2024-05-09T09:24:49.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f6116d9d640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc5882e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6108006010 tx=0x7f6108005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c19a916a0f7566 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.346+0000 7f60fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.347+0000 7f60fdffb640 1 -- 172.21.15.64:0/2121697129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610801d070 con 0x5629dc57df40 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.347+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610800cb50 con 0x5629dc57df40 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.347+0000 7f60fdffb640 1 -- 172.21.15.64:0/2121697129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f61080189b0 con 0x5629dc57df40 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.347+0000 7f60fdffb640 1 -- 172.21.15.64:0/2121697129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6108018bb0 con 0x5629dc57df40 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.347+0000 7f611f883740 10 client.19218 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.347+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5629dc592c90 con 0x5629dc57df40 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f60fdffb640 1 -- 172.21.15.64:0/2121697129 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f610800cb50 con 0x5629dc57df40 2024-05-09T09:24:49.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f60fdffb640 1 -- 172.21.15.64:0/2121697129 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f610800b040 con 0x5629dc57df40 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f611f883740 10 client.19218 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f611f883740 10 client.19218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f611f883740 20 client.19218 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f611f883740 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc593fe0 0x5629dc5963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f611f883740 4 client.19218 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629dc4f0370 con 0x5629dc593fe0 2024-05-09T09:24:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.348+0000 7f6115d9b640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc593fe0 0x5629dc5963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.349+0000 7f6115d9b640 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc593fe0 0x5629dc5963d0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ffd6fc5cfa7367 server_cookie=c8af07041332717f in_seq=0 out_seq=0 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.349+0000 7f60fdffb640 10 client.19218 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f60fdffb640 1 -- 172.21.15.64:0/2121697129 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629dc4f0370 con 0x5629dc593fe0 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f60fdffb640 10 client.19218 handle_command_reply: tid=0 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 client.19218 shutdown 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 msgr2=0x5629dc5882e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc5882e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6108006010 tx=0x7f6108005ad0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 shutdown_connections 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc593fe0 0x5629dc5963d0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc583aa0 0x5629dc58f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc588820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 --2- 172.21.15.64:0/2121697129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc5882e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 >> 172.21.15.64:0/2121697129 conn(0x5629dc5795b0 msgr2=0x5629dc5787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.350+0000 7f60fdffb640 0 client.19218 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.351+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 shutdown_connections 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.351+0000 7f611f883740 1 -- 172.21.15.64:0/2121697129 wait complete. 2024-05-09T09:24:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.351+0000 7f611f883740 20 client.19218 trim_cache size 0 max 16384 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.375+0000 7f611f883740 1 Processor -- start 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.375+0000 7f611f883740 1 -- start start 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f611f883740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc5795b0 0x5629dc57ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f611f883740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc58f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f611f883740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc57cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc58f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc58f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59364/0 (socket says 172.21.15.64:59364) 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 -- 172.21.15.64:0/482518206 learned_addr learned my addr 172.21.15.64:0/482518206 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f611f883740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8012540 con 0x5629dc57df40 2024-05-09T09:24:49.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60f800fd50 con 0x5629dc57e930 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60f800dba0 con 0x5629dc5795b0 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6116d9d640 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc5795b0 0x5629dc57ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6116d9d640 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc57cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 -- 172.21.15.64:0/482518206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc5795b0 msgr2=0x5629dc57ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc5795b0 0x5629dc57ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 -- 172.21.15.64:0/482518206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 msgr2=0x5629dc57cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc57cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6115d9b640 1 -- 172.21.15.64:0/482518206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dc57a020 con 0x5629dc57df40 2024-05-09T09:24:49.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.376+0000 7f6116d9d640 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc57cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:24:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.377+0000 7f6115d9b640 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc58f820 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f611002a900 tx=0x7f6110009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161f56856e55ab6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.378+0000 7f60fffff640 1 -- 172.21.15.64:0/482518206 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110037040 con 0x5629dc57df40 2024-05-09T09:24:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.378+0000 7f60fffff640 1 -- 172.21.15.64:0/482518206 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611004b070 con 0x5629dc57df40 2024-05-09T09:24:49.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.378+0000 7f60fffff640 1 -- 172.21.15.64:0/482518206 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110032d10 con 0x5629dc57df40 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 msgr2=0x5629dc58f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc58f820 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f611002a900 tx=0x7f6110009490 comp rx=0 tx=0).stop 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 shutdown_connections 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57e930 0x5629dc57cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc57df40 0x5629dc58f820 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 --2- 172.21.15.64:0/482518206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc5795b0 0x5629dc57ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.379+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 >> 172.21.15.64:0/482518206 conn(0x5629dc5967c0 msgr2=0x5629dc57c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.380+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 shutdown_connections 2024-05-09T09:24:49.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.380+0000 7f611f883740 1 -- 172.21.15.64:0/482518206 wait complete. 2024-05-09T09:24:49.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.381+0000 7f611f883740 1 -- start start 2024-05-09T09:24:49.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.381+0000 7f611f883740 1 Processor -- start 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 0x5629dc58d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57df40 0x5629dc58d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc58e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60f800e210 con 0x5629dc5795b0 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f6116d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 0x5629dc58d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f6116d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 0x5629dc58d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59372/0 (socket says 172.21.15.64:59372) 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f6116d9d640 1 -- 172.21.15.64:0/3620060196 learned_addr learned my addr 172.21.15.64:0/3620060196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f6115d9b640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57df40 0x5629dc58d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60f800f680 con 0x5629dc57df40 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60f800e8e0 con 0x5629dc57e930 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.382+0000 7f611f883740 10 client.0 _ll_register_callbacks cb 0x7f610801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc58e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 -- 172.21.15.64:0/3620060196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 msgr2=0x5629dc58e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc58e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 -- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57df40 msgr2=0x5629dc58d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57df40 0x5629dc58d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 -- 172.21.15.64:0/3620060196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dc58eec0 con 0x5629dc5795b0 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f6116d9d640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 0x5629dc58d3d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f610800fe40 tx=0x7f6108005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e35057e3d59ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f60fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f60fdffb640 1 -- 172.21.15.64:0/3620060196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610801d070 con 0x5629dc5795b0 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f60fdffb640 1 -- 172.21.15.64:0/3620060196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f610800cb00 con 0x5629dc5795b0 2024-05-09T09:24:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.383+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611004b070 con 0x5629dc5795b0 2024-05-09T09:24:49.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.384+0000 7f60fdffb640 1 -- 172.21.15.64:0/3620060196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6108038040 con 0x5629dc5795b0 2024-05-09T09:24:49.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.384+0000 7f611f883740 10 client.19224 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:49.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.384+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5629dc5fa840 con 0x5629dc5795b0 2024-05-09T09:24:49.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.384+0000 7f60fdffb640 1 -- 172.21.15.64:0/3620060196 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611004b070 con 0x5629dc5795b0 2024-05-09T09:24:49.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f60fdffb640 1 -- 172.21.15.64:0/3620060196 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f610800b040 con 0x5629dc5795b0 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f611f883740 10 client.19224 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f611f883740 10 client.19224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f611f883740 20 client.19224 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f611f883740 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc583aa0 0x5629dc5fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f611f883740 4 client.19224 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629dc4ea6c0 con 0x5629dc583aa0 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.385+0000 7f6115d9b640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc583aa0 0x5629dc5fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.386+0000 7f60fdffb640 10 client.19224 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:49.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.386+0000 7f6115d9b640 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc583aa0 0x5629dc5fd070 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c25bee213988a81 server_cookie=b429411eb04275f4 in_seq=0 out_seq=0 2024-05-09T09:24:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.386+0000 7f60fdffb640 1 -- 172.21.15.64:0/3620060196 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629dc4ea6c0 con 0x5629dc583aa0 2024-05-09T09:24:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.386+0000 7f60fdffb640 10 client.19224 handle_command_reply: tid=0 2024-05-09T09:24:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.387+0000 7f611f883740 1 client.19224 shutdown 2024-05-09T09:24:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.387+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 msgr2=0x5629dc58d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.387+0000 7f611f883740 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 0x5629dc58d3d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f610800fe40 tx=0x7f6108005ad0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 shutdown_connections 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5629dc583aa0 0x5629dc5fd070 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f60fdffb640 0 client.19224 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629dc57e930 0x5629dc58e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5629dc57df40 0x5629dc58d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 --2- 172.21.15.64:0/3620060196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629dc5795b0 0x5629dc58d3d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 >> 172.21.15.64:0/3620060196 conn(0x5629dc5967c0 msgr2=0x5629dc57cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 shutdown_connections 2024-05-09T09:24:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 1 -- 172.21.15.64:0/3620060196 wait complete. 2024-05-09T09:24:49.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.388+0000 7f611f883740 20 client.19224 trim_cache size 0 max 16384 2024-05-09T09:24:49.389 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:24:49.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61000a2be0 msgr2=0x7f61000a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61000a2be0 0x7f61000a50a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f610c09b610 tx=0x7f610c0cf740 comp rx=0 tx=0).stop 2024-05-09T09:24:49.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 msgr2=0x7f61180712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 0x7f61180712e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c920 tx=0x7f610c00cde0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 shutdown_connections 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f61000a2be0 0x7f61000a50a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61181099a0 0x7f6118072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6118108fe0 0x7f6118071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 --2- 172.21.15.64:0/1600025381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6118107e90 0x7f61180712e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.389+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 >> 172.21.15.64:0/1600025381 conn(0x7f6118076e30 msgr2=0x7f611810c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.390+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 shutdown_connections 2024-05-09T09:24:49.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:49.390+0000 7f611cb70640 1 -- 172.21.15.64:0/1600025381 wait complete. 2024-05-09T09:24:49.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:49 smithi064 ceph-mon[30149]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.9 MiB/s wr, 643 op/s 2024-05-09T09:24:49.404 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:49 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3056212547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:49.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:49 smithi179 ceph-mon[34086]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.9 MiB/s wr, 643 op/s 2024-05-09T09:24:49.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:49 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3056212547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:49 smithi139 ceph-mon[33599]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.9 MiB/s wr, 643 op/s 2024-05-09T09:24:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:49 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3056212547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:24:50.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:24:50.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:24:50.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:24:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.609+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/2684164725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d8108830 msgr2=0x7fe3d810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.609+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/2684164725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d8108830 0x7fe3d810ac60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009900 tx=0x7fe3cc02a440 comp rx=0 tx=0).stop 2024-05-09T09:24:50.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.611+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/2684164725 shutdown_connections 2024-05-09T09:24:50.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.611+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/2684164725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d810db70 0x7fe3d810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.611+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/2684164725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d810b1a0 0x7fe3d810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.611+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/2684164725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d8108830 0x7fe3d810ac60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.611+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/2684164725 >> 172.21.15.64:0/2684164725 conn(0x7fe3d806e6c0 msgr2=0x7fe3d8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.615+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/2684164725 shutdown_connections 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.615+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/2684164725 wait complete. 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 Processor -- start 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 -- start start 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 0x7fe3d8115000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d810b1a0 0x7fe3d8115540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d810db70 0x7fe3d81a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d811c320 con 0x7fe3d810b1a0 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d811c1a0 con 0x7fe3d8108830 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3dd0f5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d811c4a0 con 0x7fe3d810db70 2024-05-09T09:24:50.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 0x7fe3d8115000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3d6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 0x7fe3d8115000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:33990/0 (socket says 172.21.15.64:33990) 2024-05-09T09:24:50.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3d6d9d640 1 -- 172.21.15.64:0/3369252611 learned_addr learned my addr 172.21.15.64:0/3369252611 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.616+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d810db70 0x7fe3d81a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.617+0000 7fe3d6d9d640 1 -- 172.21.15.64:0/3369252611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d810db70 msgr2=0x7fe3d81a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.617+0000 7fe3d6d9d640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d810db70 0x7fe3d81a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.617+0000 7fe3d6d9d640 1 -- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d810b1a0 msgr2=0x7fe3d8115540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.617+0000 7fe3d6d9d640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d810b1a0 0x7fe3d8115540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.617+0000 7fe3d6d9d640 1 -- 172.21.15.64:0/3369252611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d81a5f60 con 0x7fe3d8108830 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.618+0000 7fe3d6d9d640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 0x7fe3d8115000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009850 tx=0x7fe3cc003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.618+0000 7fe3bffff640 1 -- 172.21.15.64:0/3369252611 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc03f070 con 0x7fe3d8108830 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.618+0000 7fe3bffff640 1 -- 172.21.15.64:0/3369252611 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3cc03b460 con 0x7fe3d8108830 2024-05-09T09:24:50.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.618+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d81a6190 con 0x7fe3d8108830 2024-05-09T09:24:50.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.618+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a6750 con 0x7fe3d8108830 2024-05-09T09:24:50.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.619+0000 7fe3bffff640 1 -- 172.21.15.64:0/3369252611 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc035030 con 0x7fe3d8108830 2024-05-09T09:24:50.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.622+0000 7fe3dfe03740 1 Processor -- start 2024-05-09T09:24:50.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.622+0000 7fe3bffff640 1 -- 172.21.15.64:0/3369252611 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3cc042070 con 0x7fe3d8108830 2024-05-09T09:24:50.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.622+0000 7fe3dfe03740 1 -- start start 2024-05-09T09:24:50.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.622+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.622+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd328e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33306/0 (socket says 172.21.15.64:33306) 2024-05-09T09:24:50.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.622+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3d759e640 1 -- 172.21.15.64:0/3172492060 learned_addr learned my addr 172.21.15.64:0/3172492060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c0dc96990 con 0x555c0dd38020 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3d6d9d640 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555c0dc3e5b0 con 0x555c0dd32eb0 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555c0dc92950 con 0x555c0dd324c0 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.623+0000 7fe3d659c640 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.624+0000 7fe3d759e640 1 -- 172.21.15.64:0/3172492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 msgr2=0x555c0dd379b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.624+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd379b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.624+0000 7fe3d759e640 1 -- 172.21.15.64:0/3172492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 msgr2=0x555c0dd3a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.624+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd3a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.624+0000 7fe3d759e640 1 -- 172.21.15.64:0/3172492060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0da727e0 con 0x555c0dd324c0 2024-05-09T09:24:50.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.624+0000 7fe3bffff640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3c00a2c60 0x7fe3c00a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.625+0000 7fe3d6d9d640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3c00a2c60 0x7fe3c00a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.625+0000 7fe3bffff640 1 -- 172.21.15.64:0/3369252611 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe3cc0efec0 con 0x7fe3d8108830 2024-05-09T09:24:50.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.626+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd328e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009f30 tx=0x7fe3d000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f3f69703b75687 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.626+0000 7fe3bd7fa640 1 -- 172.21.15.64:0/3172492060 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d001a070 con 0x555c0dd324c0 2024-05-09T09:24:50.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.626+0000 7fe3bd7fa640 1 -- 172.21.15.64:0/3172492060 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3d00158c0 con 0x555c0dd324c0 2024-05-09T09:24:50.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.626+0000 7fe3bd7fa640 1 -- 172.21.15.64:0/3172492060 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000a7a0 con 0x555c0dd324c0 2024-05-09T09:24:50.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.627+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 msgr2=0x555c0dd328e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.627+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd328e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009f30 tx=0x7fe3d000c8b0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.627+0000 7fe3d6d9d640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3c00a2c60 0x7fe3c00a5120 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0c5ae0 tx=0x7fe3cc0f3900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.627+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 shutdown_connections 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.628+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd3a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.628+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.628+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3172492060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd328e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.628+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 >> 172.21.15.64:0/3172492060 conn(0x555c0dcfa0a0 msgr2=0x555c0dd2df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.628+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 shutdown_connections 2024-05-09T09:24:50.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.628+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3172492060 wait complete. 2024-05-09T09:24:50.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.629+0000 7fe3dfe03740 1 -- start start 2024-05-09T09:24:50.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.629+0000 7fe3dfe03740 1 Processor -- start 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd3cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c0dd3afd0 con 0x555c0dd38020 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555c0dc51b40 con 0x555c0dd32eb0 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555c0dd3b150 con 0x555c0dd324c0 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33322/0 (socket says 172.21.15.64:33322) 2024-05-09T09:24:50.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3d759e640 1 -- 172.21.15.64:0/1768248457 learned_addr learned my addr 172.21.15.64:0/1768248457 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3dfe03740 10 client.0 _ll_register_callbacks cb 0x555c0dc460f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:50.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.630+0000 7fe3d659c640 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd3cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.631+0000 7fe3d659c640 1 -- 172.21.15.64:0/1768248457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 msgr2=0x555c0dd3c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.631+0000 7fe3d659c640 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.631+0000 7fe3d659c640 1 -- 172.21.15.64:0/1768248457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 msgr2=0x555c0dd43f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.631+0000 7fe3d659c640 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.631+0000 7fe3d659c640 1 -- 172.21.15.64:0/1768248457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0dad2f70 con 0x555c0dd32eb0 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.632+0000 7fe3d659c640 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd3cee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8116120 tx=0x7fe3c800d5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756d27c0cce9506 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.632+0000 7fe3baff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.632+0000 7fe3baff5640 1 -- 172.21.15.64:0/1768248457 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8014430 con 0x555c0dd32eb0 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.632+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d00158c0 con 0x555c0dd32eb0 2024-05-09T09:24:50.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.632+0000 7fe3baff5640 1 -- 172.21.15.64:0/1768248457 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe3c80145d0 con 0x555c0dd32eb0 2024-05-09T09:24:50.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3baff5640 1 -- 172.21.15.64:0/1768248457 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe3c8022050 con 0x555c0dd32eb0 2024-05-09T09:24:50.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3dfe03740 10 client.28709 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555c0d96a170 con 0x555c0dd32eb0 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3baff5640 1 -- 172.21.15.64:0/1768248457 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3d00158c0 con 0x555c0dd32eb0 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3baff5640 1 -- 172.21.15.64:0/1768248457 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7fe3c8013070 con 0x555c0dd32eb0 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3dfe03740 10 client.28709 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3dfe03740 10 client.28709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3dfe03740 20 client.28709 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.633+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0dd489b0 0x555c0dd4ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.634+0000 7fe3dfe03740 4 client.28709 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.634+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0dca40c0 con 0x555c0dd489b0 2024-05-09T09:24:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.634+0000 7fe3d759e640 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0dd489b0 0x555c0dd4ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.634+0000 7fe3baff5640 10 client.28709 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.634+0000 7fe3d759e640 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0dd489b0 0x555c0dd4ae10 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a81dee19e9976c4 server_cookie=e6e231c2fcdcea55 in_seq=0 out_seq=0 2024-05-09T09:24:50.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.635+0000 7fe3baff5640 1 -- 172.21.15.64:0/1768248457 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c0dca40c0 con 0x555c0dd489b0 2024-05-09T09:24:50.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.635+0000 7fe3baff5640 10 client.28709 handle_command_reply: tid=0 2024-05-09T09:24:50.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.635+0000 7fe3dfe03740 1 client.28709 shutdown 2024-05-09T09:24:50.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 msgr2=0x555c0dd3cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd3cee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8116120 tx=0x7fe3c800d5f0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 shutdown_connections 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0dd489b0 0x555c0dd4ae10 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd32eb0 0x555c0dd3cee0 secure :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8116120 tx=0x7fe3c800d5f0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/1768248457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 >> 172.21.15.64:0/1768248457 conn(0x555c0dcfa0a0 msgr2=0x555c0dd2d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3baff5640 0 client.28709 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 shutdown_connections 2024-05-09T09:24:50.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 1 -- 172.21.15.64:0/1768248457 wait complete. 2024-05-09T09:24:50.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.636+0000 7fe3dfe03740 20 client.28709 trim_cache size 0 max 16384 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.669+0000 7fe3dfe03740 1 Processor -- start 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.669+0000 7fe3dfe03740 1 -- start start 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd30c00 0x555c0dd310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3dfe03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4012550 con 0x555c0dd38020 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3dfe03740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400fd60 con 0x555c0dd30c00 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3dfe03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400dbb0 con 0x555c0dd324c0 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59418/0 (socket says 172.21.15.64:59418) 2024-05-09T09:24:50.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd30c00 0x555c0dd310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3d659c640 1 -- 172.21.15.64:0/3877366319 learned_addr learned my addr 172.21.15.64:0/3877366319 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.670+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.671+0000 7fe3d759e640 1 -- 172.21.15.64:0/3877366319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 msgr2=0x555c0dd3c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.671+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.671+0000 7fe3d759e640 1 -- 172.21.15.64:0/3877366319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 msgr2=0x555c0dd43ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.671+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.671+0000 7fe3d759e640 1 -- 172.21.15.64:0/3877366319 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0dd3d840 con 0x555c0dd30c00 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.672+0000 7fe3d759e640 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd30c00 0x555c0dd310b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x555c0dd4b650 tx=0x7fe3d0009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87c168e73698de91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.672+0000 7fe3bcff9640 1 -- 172.21.15.64:0/3877366319 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d001a040 con 0x555c0dd30c00 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.672+0000 7fe3bcff9640 1 -- 172.21.15.64:0/3877366319 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3d0029030 con 0x555c0dd30c00 2024-05-09T09:24:50.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.672+0000 7fe3bcff9640 1 -- 172.21.15.64:0/3877366319 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0024590 con 0x555c0dd30c00 2024-05-09T09:24:50.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3877366319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd30c00 msgr2=0x555c0dd310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd30c00 0x555c0dd310b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x555c0dd4b650 tx=0x7fe3d0009a60 comp rx=0 tx=0).stop 2024-05-09T09:24:50.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3877366319 shutdown_connections 2024-05-09T09:24:50.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd30c00 0x555c0dd310b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd38020 0x555c0dd43ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/3877366319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd324c0 0x555c0dd3c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.674+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3877366319 >> 172.21.15.64:0/3877366319 conn(0x555c0dcfa0a0 msgr2=0x555c0dd3bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.675+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3877366319 shutdown_connections 2024-05-09T09:24:50.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.675+0000 7fe3dfe03740 1 -- 172.21.15.64:0/3877366319 wait complete. 2024-05-09T09:24:50.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.675+0000 7fe3dfe03740 1 -- start start 2024-05-09T09:24:50.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.675+0000 7fe3dfe03740 1 Processor -- start 2024-05-09T09:24:50.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.676+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 0x555c0dd41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.676+0000 7fe3dfe03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd324c0 0x555c0dd42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.676+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 0x555c0dd41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.676+0000 7fe3d759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 0x555c0dd41f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33340/0 (socket says 172.21.15.64:33340) 2024-05-09T09:24:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.676+0000 7fe3d759e640 1 -- 172.21.15.64:0/166543176 learned_addr learned my addr 172.21.15.64:0/166543176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.677+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd38020 0x555c0dd43450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.677+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400e220 con 0x555c0dd324c0 2024-05-09T09:24:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.677+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400f690 con 0x555c0dd38020 2024-05-09T09:24:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.677+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400e8f0 con 0x555c0dd30c00 2024-05-09T09:24:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.677+0000 7fe3d659c640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd324c0 0x555c0dd42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd38020 0x555c0dd43450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 -- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd38020 msgr2=0x555c0dd43450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd38020 0x555c0dd43450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 -- 172.21.15.64:0/166543176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd324c0 msgr2=0x555c0dd42440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd324c0 0x555c0dd42440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0dd43a40 con 0x555c0dd30c00 2024-05-09T09:24:50.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3d759e640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 0x555c0dd41f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00240d0 tx=0x7fe3d00155b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7baa3c3e5c0b2ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3baff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3baff5640 1 -- 172.21.15.64:0/166543176 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d001a040 con 0x555c0dd30c00 2024-05-09T09:24:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.678+0000 7fe3dfe03740 10 client.0 _ll_register_callbacks cb 0x7fe3c800b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.679+0000 7fe3baff5640 1 -- 172.21.15.64:0/166543176 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe3d0003a10 con 0x555c0dd30c00 2024-05-09T09:24:50.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.679+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d0029030 con 0x555c0dd30c00 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.679+0000 7fe3baff5640 1 -- 172.21.15.64:0/166543176 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe3d0003c10 con 0x555c0dd30c00 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.679+0000 7fe3dfe03740 10 client.28867 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.679+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=51}) -- 0x555c0ddaf3f0 con 0x555c0dd30c00 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3baff5640 1 -- 172.21.15.64:0/166543176 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7fe3d0024490 con 0x555c0dd30c00 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3dfe03740 10 client.28867 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3dfe03740 10 client.28867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3dfe03740 20 client.28867 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0ddafb10 0x555c0ddb1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3dfe03740 4 client.28867 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:24:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555c0dc9e410 con 0x555c0ddafb10 2024-05-09T09:24:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.680+0000 7fe3d659c640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0ddafb10 0x555c0ddb1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.681+0000 7fe3d659c640 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0ddafb10 0x555c0ddb1fd0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f998d8e4dd2fb35b server_cookie=a3f75d717a54ccb0 in_seq=0 out_seq=0 2024-05-09T09:24:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.681+0000 7fe3baff5640 10 client.28867 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.682+0000 7fe3baff5640 1 -- 172.21.15.64:0/166543176 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c0dc9e410 con 0x555c0ddafb10 2024-05-09T09:24:50.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.682+0000 7fe3baff5640 10 client.28867 handle_command_reply: tid=0 2024-05-09T09:24:50.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.682+0000 7fe3dfe03740 1 client.28867 shutdown 2024-05-09T09:24:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.684+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 msgr2=0x555c0dd41f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.684+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 0x555c0dd41f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00240d0 tx=0x7fe3d00155b0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 shutdown_connections 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x555c0ddafb10 0x555c0ddb1fd0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3baff5640 0 client.28867 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c0dd38020 0x555c0dd43450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0dd324c0 0x555c0dd42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3dfe03740 1 --2- 172.21.15.64:0/166543176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c0dd30c00 0x555c0dd41f00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.685+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 >> 172.21.15.64:0/166543176 conn(0x555c0dcfa0a0 msgr2=0x555c0dd31050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.686+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 shutdown_connections 2024-05-09T09:24:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.686+0000 7fe3dfe03740 1 -- 172.21.15.64:0/166543176 wait complete. 2024-05-09T09:24:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.686+0000 7fe3dfe03740 20 client.28867 trim_cache size 0 max 16384 2024-05-09T09:24:50.688 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:24:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:24:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "89041a75-55d7-4110-82d5-af49ba7c90fd", 2024-05-09T09:24:50.688 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:24:50.688 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:24:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3c00a2c60 msgr2=0x7fe3c00a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3c00a2c60 0x7fe3c00a5120 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0c5ae0 tx=0x7fe3cc0f3900 comp rx=0 tx=0).stop 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 msgr2=0x7fe3d8115000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 0x7fe3d8115000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009850 tx=0x7fe3cc003e10 comp rx=0 tx=0).stop 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 shutdown_connections 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fe3c00a2c60 0x7fe3c00a5120 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d810db70 0x7fe3d81a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d810b1a0 0x7fe3d8115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 --2- 172.21.15.64:0/3369252611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d8108830 0x7fe3d8115000 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.689+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 >> 172.21.15.64:0/3369252611 conn(0x7fe3d806e6c0 msgr2=0x7fe3d810c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.690+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 shutdown_connections 2024-05-09T09:24:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.690+0000 7fe3dd0f5640 1 -- 172.21.15.64:0/3369252611 wait complete. 2024-05-09T09:24:50.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:24:50.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:24:50.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.913+0000 7fa7f1088640 1 -- 172.21.15.64:0/4076155223 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8030850 con 0x7fa7e40a4c40 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 -- 172.21.15.64:0/4076155223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a4c40 msgr2=0x7fa7e40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 --2- 172.21.15.64:0/4076155223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a4c40 0x7fa7e40a5060 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009860 tx=0x7fa7e802a340 comp rx=0 tx=0).stop 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 -- 172.21.15.64:0/4076155223 shutdown_connections 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 --2- 172.21.15.64:0/4076155223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e40a6710 0x7fa7e40aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 --2- 172.21.15.64:0/4076155223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a5d70 0x7fa7e40a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 --2- 172.21.15.64:0/4076155223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a4c40 0x7fa7e40a5060 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 -- 172.21.15.64:0/4076155223 >> 172.21.15.64:0/4076155223 conn(0x7fa7e40a00d0 msgr2=0x7fa7e40a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 -- 172.21.15.64:0/4076155223 shutdown_connections 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.914+0000 7fa7f308c640 1 -- 172.21.15.64:0/4076155223 wait complete. 2024-05-09T09:24:50.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.915+0000 7fa7f308c640 1 Processor -- start 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.915+0000 7fa7f308c640 1 -- start start 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f308c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e40a4c40 0x7fa7e4143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f308c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a5d70 0x7fa7e4143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f308c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 0x7fa7e414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f308c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40b8f70 con 0x7fa7e40a6710 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f308c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40b8df0 con 0x7fa7e40a4c40 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f308c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40b90f0 con 0x7fa7e40a5d70 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f1889640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a5d70 0x7fa7e4143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f288b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 0x7fa7e414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f288b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 0x7fa7e414abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59442/0 (socket says 172.21.15.64:59442) 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f288b640 1 -- 172.21.15.64:0/553627047 learned_addr learned my addr 172.21.15.64:0/553627047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f1889640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a5d70 0x7fa7e4143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33360/0 (socket says 172.21.15.64:33360) 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.916+0000 7fa7f208a640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e40a4c40 0x7fa7e4143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7f288b640 1 -- 172.21.15.64:0/553627047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a5d70 msgr2=0x7fa7e4143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7f288b640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a5d70 0x7fa7e4143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7f288b640 1 -- 172.21.15.64:0/553627047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e40a4c40 msgr2=0x7fa7e4143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7f288b640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e40a4c40 0x7fa7e4143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7f288b640 1 -- 172.21.15.64:0/553627047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e414b220 con 0x7fa7e40a6710 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7f288b640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 0x7fa7e414abb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec050860 tx=0x7fa7ec06d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.917+0000 7fa7e37fe640 1 -- 172.21.15.64:0/553627047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec079020 con 0x7fa7e40a6710 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.918+0000 7fa7e37fe640 1 -- 172.21.15.64:0/553627047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec0758d0 con 0x7fa7e40a6710 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.918+0000 7fa7e37fe640 1 -- 172.21.15.64:0/553627047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0746b0 con 0x7fa7e40a6710 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.918+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e414b510 con 0x7fa7e40a6710 2024-05-09T09:24:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.918+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e414ba10 con 0x7fa7e40a6710 2024-05-09T09:24:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.922+0000 7fa7f6ffc740 1 Processor -- start 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.922+0000 7fa7e37fe640 1 -- 172.21.15.64:0/553627047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7ec07c020 con 0x7fa7e40a6710 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 -- start start 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842120 0x5650ce842540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 0x5650ce847610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce84a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5650ce7a6d00 con 0x5650ce842b10 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5650ce74e4c0 con 0x5650ce847c80 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5650ce7a2cc0 con 0x5650ce842120 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f208a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 0x5650ce847610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f208a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 0x5650ce847610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59454/0 (socket says 172.21.15.64:59454) 2024-05-09T09:24:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f208a640 1 -- 172.21.15.64:0/1373484497 learned_addr learned my addr 172.21.15.64:0/1373484497 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f1889640 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842120 0x5650ce842540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.923+0000 7fa7f1889640 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce84a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f208a640 1 -- 172.21.15.64:0/1373484497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842120 msgr2=0x5650ce842540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f208a640 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842120 0x5650ce842540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f208a640 1 -- 172.21.15.64:0/1373484497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 msgr2=0x5650ce84a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f208a640 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce84a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f208a640 1 -- 172.21.15.64:0/1373484497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ce5847d0 con 0x5650ce842b10 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7e37fe640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7d80a2d60 0x7fa7d80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f208a640 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 0x5650ce847610 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80057c0 tx=0x7fa7e8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707f3c89a0d60ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7e37fe640 1 -- 172.21.15.64:0/553627047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa7ec132280 con 0x7fa7e40a6710 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.924+0000 7fa7f1889640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7d80a2d60 0x7fa7d80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.925+0000 7fa7e0bfe640 1 -- 172.21.15.64:0/1373484497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8043070 con 0x5650ce842b10 2024-05-09T09:24:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.925+0000 7fa7e0bfe640 1 -- 172.21.15.64:0/1373484497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e8038040 con 0x5650ce842b10 2024-05-09T09:24:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.925+0000 7fa7e0bfe640 1 -- 172.21.15.64:0/1373484497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8037040 con 0x5650ce842b10 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/1373484497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 msgr2=0x5650ce847610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 0x5650ce847610 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80057c0 tx=0x7fa7e8009510 comp rx=0 tx=0).stop 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f1889640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7d80a2d60 0x7fa7d80a5220 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5650ce843370 tx=0x7fa7dc015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/1373484497 shutdown_connections 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce84a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842b10 0x5650ce847610 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/1373484497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842120 0x5650ce842540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/1373484497 >> 172.21.15.64:0/1373484497 conn(0x5650ce83d790 msgr2=0x5650ce83db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/1373484497 shutdown_connections 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.926+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/1373484497 wait complete. 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.927+0000 7fa7f6ffc740 1 -- start start 2024-05-09T09:24:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.927+0000 7fa7f6ffc740 1 Processor -- start 2024-05-09T09:24:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.927+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce84c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce84cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f1889640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce84cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f208a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce84c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f208a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce84c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59462/0 (socket says 172.21.15.64:59462) 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f1889640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce84cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:33370/0 (socket says 172.21.15.64:33370) 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce853b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5650ce84abd0 con 0x5650ce842120 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5650ce761b90 con 0x5650ce847c80 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f208a640 1 -- 172.21.15.64:0/2426576697 learned_addr learned my addr 172.21.15.64:0/2426576697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5650ce84ad50 con 0x5650ce842b10 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f6ffc740 10 client.0 _ll_register_callbacks cb 0x5650ce79dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.928+0000 7fa7f288b640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce853b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f208a640 1 -- 172.21.15.64:0/2426576697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 msgr2=0x5650ce84cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f208a640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce84cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f208a640 1 -- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 msgr2=0x5650ce853b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f208a640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce853b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f208a640 1 -- 172.21.15.64:0/2426576697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ce4ba3f0 con 0x5650ce842120 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f288b640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce853b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7f208a640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce84c5a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802a860 tx=0x7fa7e8030e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01d724afa022c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.929+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/2426576697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e804e070 con 0x5650ce842120 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.930+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/2426576697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa7e8030550 con 0x5650ce842120 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.930+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e8038040 con 0x5650ce842120 2024-05-09T09:24:50.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.930+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/2426576697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa7e805a050 con 0x5650ce842120 2024-05-09T09:24:50.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.930+0000 7fa7f6ffc740 10 client.19248 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:50.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.930+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5650ce47a0f0 con 0x5650ce842120 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/2426576697 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e8038040 con 0x5650ce842120 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/2426576697 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7fa7e8032480 con 0x5650ce842120 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f6ffc740 10 client.19248 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f6ffc740 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f6ffc740 20 client.19248 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce858010 0x5650ce85a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f6ffc740 4 client.19248 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650ce7b4430 con 0x5650ce858010 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.931+0000 7fa7f1889640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce858010 0x5650ce85a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.932+0000 7fa7d5ffb640 10 client.19248 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.932+0000 7fa7f1889640 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce858010 0x5650ce85a400 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c63c21c414df0c server_cookie=ea903f231e04ebdb in_seq=0 out_seq=0 2024-05-09T09:24:50.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.933+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/2426576697 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650ce7b4430 con 0x5650ce858010 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.933+0000 7fa7d5ffb640 10 client.19248 handle_command_reply: tid=0 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.933+0000 7fa7f6ffc740 1 client.19248 shutdown 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.933+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 msgr2=0x5650ce84c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.933+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce84c5a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802a860 tx=0x7fa7e8030e80 comp rx=0 tx=0).stop 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.933+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 shutdown_connections 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce858010 0x5650ce85a400 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce847c80 0x5650ce853b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7d5ffb640 0 client.19248 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce84cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/2426576697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce84c5a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 >> 172.21.15.64:0/2426576697 conn(0x5650ce83d790 msgr2=0x5650ce83ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 shutdown_connections 2024-05-09T09:24:50.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.934+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/2426576697 wait complete. 2024-05-09T09:24:50.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.935+0000 7fa7f6ffc740 20 client.19248 trim_cache size 0 max 16384 2024-05-09T09:24:50.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.969+0000 7fa7f6ffc740 1 Processor -- start 2024-05-09T09:24:50.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f6ffc740 1 -- start start 2024-05-09T09:24:50.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce83d790 0x5650ce840670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce853ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842b10 0x5650ce840bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f1889640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce853ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f1889640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce853ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59464/0 (socket says 172.21.15.64:59464) 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.970+0000 7fa7f1889640 1 -- 172.21.15.64:0/3604352157 learned_addr learned my addr 172.21.15.64:0/3604352157 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e41587e0 con 0x5650ce842120 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4155ff0 con 0x5650ce842b10 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f288b640 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842b10 0x5650ce840bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4153e40 con 0x5650ce83d790 2024-05-09T09:24:50.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce83d790 0x5650ce840670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f208a640 1 -- 172.21.15.64:0/3604352157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842b10 msgr2=0x5650ce840bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842b10 0x5650ce840bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f208a640 1 -- 172.21.15.64:0/3604352157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 msgr2=0x5650ce853ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce853ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.971+0000 7fa7f208a640 1 -- 172.21.15.64:0/3604352157 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ce83d5c0 con 0x5650ce83d790 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.972+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce83d790 0x5650ce840670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802a860 tx=0x7fa7e8032f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8761c9d62c9285c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.972+0000 7fa7d7fff640 1 -- 172.21.15.64:0/3604352157 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e804e040 con 0x5650ce83d790 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.972+0000 7fa7d7fff640 1 -- 172.21.15.64:0/3604352157 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e8004250 con 0x5650ce83d790 2024-05-09T09:24:50.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.972+0000 7fa7d7fff640 1 -- 172.21.15.64:0/3604352157 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8037040 con 0x5650ce83d790 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.973+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce83d790 msgr2=0x5650ce840670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.973+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce83d790 0x5650ce840670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802a860 tx=0x7fa7e8032f10 comp rx=0 tx=0).stop 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.973+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 shutdown_connections 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.973+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842b10 0x5650ce840bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.973+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce842120 0x5650ce853ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.973+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3604352157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce83d790 0x5650ce840670 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.974+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 >> 172.21.15.64:0/3604352157 conn(0x5650ce838d20 msgr2=0x5650ce83db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.974+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 shutdown_connections 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.974+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3604352157 wait complete. 2024-05-09T09:24:50.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.974+0000 7fa7f6ffc740 1 -- start start 2024-05-09T09:24:50.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.975+0000 7fa7f6ffc740 1 Processor -- start 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.975+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 0x5650ce850df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.975+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842120 0x5650ce851330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.975+0000 7fa7f6ffc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce8522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.975+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e41544b0 con 0x5650ce83d790 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.975+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4155920 con 0x5650ce842120 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f6ffc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4154b80 con 0x5650ce842b10 2024-05-09T09:24:50.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f6ffc740 10 client.0 _ll_register_callbacks cb 0x7fa7dc009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f208a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 0x5650ce850df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f208a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 0x5650ce850df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59478/0 (socket says 172.21.15.64:59478) 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f208a640 1 -- 172.21.15.64:0/3446949051 learned_addr learned my addr 172.21.15.64:0/3446949051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f1889640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842120 0x5650ce851330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f288b640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce8522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f288b640 1 -- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842120 msgr2=0x5650ce851330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f288b640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842120 0x5650ce851330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f288b640 1 -- 172.21.15.64:0/3446949051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 msgr2=0x5650ce850df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f288b640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 0x5650ce850df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f288b640 1 -- 172.21.15.64:0/3446949051 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ce82e930 con 0x5650ce842b10 2024-05-09T09:24:50.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.976+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 0x5650ce850df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:50.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7f288b640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce8522d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec12f990 tx=0x7fa7ec067460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cce05f5a09aa840 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:50.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:24:50.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/3446949051 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec102aa0 con 0x5650ce842b10 2024-05-09T09:24:50.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/3446949051 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa7ec12faa0 con 0x5650ce842b10 2024-05-09T09:24:50.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e8004250 con 0x5650ce842b10 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/3446949051 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec10c050 con 0x5650ce842b10 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7f6ffc740 10 client.28885 fetch_fsmap learned FSMap version 51 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.977+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5650ce853a70 con 0x5650ce842b10 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/3446949051 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e8004250 con 0x5650ce842b10 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/3446949051 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7fa7ec10f020 con 0x5650ce842b10 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f6ffc740 10 client.28885 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f6ffc740 10 client.28885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f6ffc740 20 client.28885 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce847c80 0x5650ce8c09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f6ffc740 4 client.28885 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:24:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650ce7ae780 con 0x5650ce847c80 2024-05-09T09:24:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.978+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce847c80 0x5650ce8c09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.979+0000 7fa7f208a640 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce847c80 0x5650ce8c09e0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63331ab3d3a9ab1 server_cookie=c5e505a7e61bb465 in_seq=0 out_seq=0 2024-05-09T09:24:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.979+0000 7fa7d5ffb640 10 client.28885 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.980+0000 7fa7d5ffb640 1 -- 172.21.15.64:0/3446949051 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650ce7ae780 con 0x5650ce847c80 2024-05-09T09:24:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.980+0000 7fa7d5ffb640 10 client.28885 handle_command_reply: tid=0 2024-05-09T09:24:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.980+0000 7fa7f6ffc740 1 client.28885 shutdown 2024-05-09T09:24:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 msgr2=0x5650ce8522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce8522d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec12f990 tx=0x7fa7ec067460 comp rx=0 tx=0).stop 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 shutdown_connections 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5650ce847c80 0x5650ce8c09e0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5650ce842b10 0x5650ce8522d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7d5ffb640 0 client.28885 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5650ce842120 0x5650ce851330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 --2- 172.21.15.64:0/3446949051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650ce83d790 0x5650ce850df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.981+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 >> 172.21.15.64:0/3446949051 conn(0x5650ce838d20 msgr2=0x5650ce83c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.982+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 shutdown_connections 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.982+0000 7fa7f6ffc740 1 -- 172.21.15.64:0/3446949051 wait complete. 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.982+0000 7fa7f6ffc740 20 client.28885 trim_cache size 0 max 16384 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stdout: "89041a75-55d7-4110-82d5-af49ba7c90fd": { 2024-05-09T09:24:50.984 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:24:50.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "89041a75-55d7-4110-82d5-af49ba7c90fd", 2024-05-09T09:24:50.985 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:24:50.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:24:50.985 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:24:50.985 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:24:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7d80a2d60 msgr2=0x7fa7d80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7d80a2d60 0x7fa7d80a5220 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5650ce843370 tx=0x7fa7dc015040 comp rx=0 tx=0).stop 2024-05-09T09:24:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 msgr2=0x7fa7e414abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 0x7fa7e414abb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec050860 tx=0x7fa7ec06d430 comp rx=0 tx=0).stop 2024-05-09T09:24:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 shutdown_connections 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa7d80a2d60 0x7fa7d80a5220 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7e40a6710 0x7fa7e414abb0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7e40a5d70 0x7fa7e4143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 --2- 172.21.15.64:0/553627047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e40a4c40 0x7fa7e4143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.983+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 >> 172.21.15.64:0/553627047 conn(0x7fa7e40a00d0 msgr2=0x7fa7e40a0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.984+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 shutdown_connections 2024-05-09T09:24:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:24:50.984+0000 7fa7f308c640 1 -- 172.21.15.64:0/553627047 wait complete. 2024-05-09T09:24:51.007 INFO:tasks.cephfs.filesystem:scrub status for tag:89041a75-55d7-4110-82d5-af49ba7c90fd - {'path': '/', 'tag': '89041a75-55d7-4110-82d5-af49ba7c90fd', 'options': 'recursive,force'} 2024-05-09T09:24:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 870.0 s, 913.6 tps, lat 157.446 ms stddev 473.546 2024-05-09T09:24:51.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:51 smithi064 ceph-mon[30149]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 703 op/s 2024-05-09T09:24:51.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:51 smithi179 ceph-mon[34086]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 703 op/s 2024-05-09T09:24:51.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:51 smithi139 ceph-mon[33599]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 703 op/s 2024-05-09T09:24:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:24:52.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:52 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:24:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:24:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:24:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:24:52.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:52 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:52.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:24:52.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:52 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:24:53.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:53 smithi179 ceph-mon[34086]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.7 MiB/s wr, 455 op/s 2024-05-09T09:24:53.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:53 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:24:53.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:53 smithi139 ceph-mon[33599]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.7 MiB/s wr, 455 op/s 2024-05-09T09:24:53.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:53 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:24:53.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:53 smithi064 ceph-mon[30149]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.7 MiB/s wr, 455 op/s 2024-05-09T09:24:53.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:53 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:24:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:24:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:24:54.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:54 smithi179 ceph-mon[34086]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.4 MiB/s wr, 615 op/s 2024-05-09T09:24:54.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:24:54.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:24:54.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:24:54.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:54 smithi139 ceph-mon[33599]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.4 MiB/s wr, 615 op/s 2024-05-09T09:24:54.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:24:54.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:24:54.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:24:54.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:54 smithi064 ceph-mon[30149]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.4 MiB/s wr, 615 op/s 2024-05-09T09:24:54.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:24:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 875.0 s, 826.2 tps, lat 122.011 ms stddev 74.825 2024-05-09T09:24:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.559+0000 7facf568c640 1 -- 172.21.15.64:0/4147293745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0100f00 msgr2=0x7facf01013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.559+0000 7facf568c640 1 --2- 172.21.15.64:0/4147293745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0100f00 0x7facf01013b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7face8009ca0 tx=0x7face802a8d0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.560+0000 7facf568c640 1 -- 172.21.15.64:0/4147293745 shutdown_connections 2024-05-09T09:24:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.560+0000 7facf568c640 1 --2- 172.21.15.64:0/4147293745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0100f00 0x7facf01013b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.560+0000 7facf568c640 1 --2- 172.21.15.64:0/4147293745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf00ff740 0x7facf0100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.560+0000 7facf568c640 1 --2- 172.21.15.64:0/4147293745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 0x7facf00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.560+0000 7facf568c640 1 -- 172.21.15.64:0/4147293745 >> 172.21.15.64:0/4147293745 conn(0x7facf00fae90 msgr2=0x7facf00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.561+0000 7facf568c640 1 -- 172.21.15.64:0/4147293745 shutdown_connections 2024-05-09T09:24:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.561+0000 7facf568c640 1 -- 172.21.15.64:0/4147293745 wait complete. 2024-05-09T09:24:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.562+0000 7facf568c640 1 Processor -- start 2024-05-09T09:24:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.562+0000 7facf568c640 1 -- start start 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf568c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 0x7facf019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf568c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00ff740 0x7facf019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf568c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0100f00 0x7facf01a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf568c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7facf0102730 con 0x7facf00ff290 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf568c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facf01025b0 con 0x7facf00ff740 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf568c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facf01028b0 con 0x7facf0100f00 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7faceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 0x7facf019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7faceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 0x7facf019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51572/0 (socket says 172.21.15.64:51572) 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7faceffff640 1 -- 172.21.15.64:0/298357888 learned_addr learned my addr 172.21.15.64:0/298357888 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facf4e8b640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0100f00 0x7facf01a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.563+0000 7facef7fe640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00ff740 0x7facf019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.564+0000 7facf4e8b640 1 -- 172.21.15.64:0/298357888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00ff740 msgr2=0x7facf019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.564+0000 7facf4e8b640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00ff740 0x7facf019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.564+0000 7facf4e8b640 1 -- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 msgr2=0x7facf019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.564+0000 7facf4e8b640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 0x7facf019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.564+0000 7facf4e8b640 1 -- 172.21.15.64:0/298357888 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf01a5b40 con 0x7facf0100f00 2024-05-09T09:24:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.564+0000 7facf4e8b640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0100f00 0x7facf01a54d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7face8009bf0 tx=0x7face8003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.565+0000 7faced7fa640 1 -- 172.21.15.64:0/298357888 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8004040 con 0x7facf0100f00 2024-05-09T09:24:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.565+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf01a5dd0 con 0x7facf0100f00 2024-05-09T09:24:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.565+0000 7faced7fa640 1 -- 172.21.15.64:0/298357888 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face8004690 con 0x7facf0100f00 2024-05-09T09:24:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.565+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01a6360 con 0x7facf0100f00 2024-05-09T09:24:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.565+0000 7faced7fa640 1 -- 172.21.15.64:0/298357888 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8004040 con 0x7facf0100f00 2024-05-09T09:24:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.568+0000 7facf95f9740 1 Processor -- start 2024-05-09T09:24:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.569+0000 7faced7fa640 1 -- 172.21.15.64:0/298357888 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7face80041e0 con 0x7facf0100f00 2024-05-09T09:24:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.569+0000 7facf95f9740 1 -- start start 2024-05-09T09:24:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.569+0000 7facf95f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x565542443820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facf95f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x5655424488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facf95f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542448f60 0x56554244b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facf95f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5655423a7340 con 0x565542443400 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facf95f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56554234f290 con 0x565542448f60 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facf95f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5655423a3300 con 0x565542443df0 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x5655424488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x5655424488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:52990/0 (socket says 172.21.15.64:52990) 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 -- 172.21.15.64:0/253877590 learned_addr learned my addr 172.21.15.64:0/253877590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facef7fe640 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x565542443820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7facef7fe640 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542448f60 0x56554244b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 -- 172.21.15.64:0/253877590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542448f60 msgr2=0x56554244b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542448f60 0x56554244b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 -- 172.21.15.64:0/253877590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 msgr2=0x565542443820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x565542443820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.570+0000 7faceffff640 1 -- 172.21.15.64:0/253877590 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655423738e0 con 0x565542443df0 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.571+0000 7faceffff640 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x5655424488f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7face4009af0 tx=0x7face400e1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6bd39dec752f229 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.571+0000 7faced7fa640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facd80a2dd0 0x7facd80a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.572+0000 7faced7fa640 1 -- 172.21.15.64:0/298357888 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7face80f08d0 con 0x7facf0100f00 2024-05-09T09:24:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.572+0000 7facd67fc640 1 -- 172.21.15.64:0/253877590 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4017070 con 0x565542443df0 2024-05-09T09:24:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.572+0000 7facd67fc640 1 -- 172.21.15.64:0/253877590 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face400ec10 con 0x565542443df0 2024-05-09T09:24:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.572+0000 7facef7fe640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facd80a2dd0 0x7facd80a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.572+0000 7facd67fc640 1 -- 172.21.15.64:0/253877590 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4012630 con 0x565542443df0 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 -- 172.21.15.64:0/253877590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 msgr2=0x5655424488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x5655424488f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7face4009af0 tx=0x7face400e1d0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facef7fe640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facd80a2dd0 0x7facd80a5290 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x565542444650 tx=0x7face000a690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 -- 172.21.15.64:0/253877590 shutdown_connections 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542448f60 0x56554244b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x5655424488f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 --2- 172.21.15.64:0/253877590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x565542443820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 -- 172.21.15.64:0/253877590 >> 172.21.15.64:0/253877590 conn(0x56554243ea70 msgr2=0x56554243ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 -- 172.21.15.64:0/253877590 shutdown_connections 2024-05-09T09:24:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.573+0000 7facf95f9740 1 -- 172.21.15.64:0/253877590 wait complete. 2024-05-09T09:24:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 -- start start 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 Processor -- start 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x56554244d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x56554244dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542454be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56554244bfb0 con 0x565542443400 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565542362790 con 0x565542443df0 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.574+0000 7facf95f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56554244c130 con 0x565542448f60 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.575+0000 7facf95f9740 10 client.0 _ll_register_callbacks cb 0x56554239da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.575+0000 7faceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x56554244d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.575+0000 7facef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x56554244dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.575+0000 7faceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x56554244d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51596/0 (socket says 172.21.15.64:51596) 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.575+0000 7faceffff640 1 -- 172.21.15.64:0/3032294021 learned_addr learned my addr 172.21.15.64:0/3032294021 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faceffff640 1 -- 172.21.15.64:0/3032294021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 msgr2=0x565542454be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faceffff640 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542454be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faceffff640 1 -- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 msgr2=0x56554244dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faceffff640 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x56554244dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faceffff640 1 -- 172.21.15.64:0/3032294021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655420bb210 con 0x565542443400 2024-05-09T09:24:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faceffff640 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x56554244d660 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7face4000c00 tx=0x7face40136a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff9fdf10b3819e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.576+0000 7faccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:24:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.577+0000 7faccf7fe640 1 -- 172.21.15.64:0/3032294021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4029060 con 0x565542443400 2024-05-09T09:24:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.577+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face400ec10 con 0x565542443400 2024-05-09T09:24:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.578+0000 7faccf7fe640 1 -- 172.21.15.64:0/3032294021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7face401c450 con 0x565542443400 2024-05-09T09:24:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.578+0000 7faccf7fe640 1 -- 172.21.15.64:0/3032294021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7face4017040 con 0x565542443400 2024-05-09T09:24:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.578+0000 7facf95f9740 10 client.19263 fetch_fsmap learned FSMap version 52 2024-05-09T09:24:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.578+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x565542187620 con 0x565542443400 2024-05-09T09:24:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7faccf7fe640 1 -- 172.21.15.64:0/3032294021 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face400ec10 con 0x565542443400 2024-05-09T09:24:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7faccf7fe640 1 -- 172.21.15.64:0/3032294021 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 52) ==== 8366+0+0 (secure 0 0 0) 0x7face4024830 con 0x565542443400 2024-05-09T09:24:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7facf95f9740 10 client.19263 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T09:24:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7facf95f9740 10 client.19263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7facf95f9740 20 client.19263 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7facf95f9740 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565542458e00 0x56554245b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7facf95f9740 4 client.19263 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:24:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.579+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655423b4a70 con 0x565542458e00 2024-05-09T09:24:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.580+0000 7facef7fe640 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565542458e00 0x56554245b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.580+0000 7faccf7fe640 10 client.19263 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.580+0000 7facef7fe640 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565542458e00 0x56554245b250 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc0ffd0f0b55b6b server_cookie=72c8f22101fe9e86 in_seq=0 out_seq=0 2024-05-09T09:24:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.583+0000 7faccf7fe640 1 -- 172.21.15.64:0/3032294021 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5655423b4a70 con 0x565542458e00 2024-05-09T09:24:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.583+0000 7faccf7fe640 10 client.19263 handle_command_reply: tid=0 2024-05-09T09:24:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.583+0000 7facf95f9740 1 client.19263 shutdown 2024-05-09T09:24:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 msgr2=0x56554244d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x56554244d660 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7face4000c00 tx=0x7face40136a0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 shutdown_connections 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x565542458e00 0x56554245b250 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542454be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x56554244dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 --2- 172.21.15.64:0/3032294021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x56554244d660 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 >> 172.21.15.64:0/3032294021 conn(0x56554243ea70 msgr2=0x56554243dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7faccf7fe640 0 client.19263 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 shutdown_connections 2024-05-09T09:24:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.586+0000 7facf95f9740 1 -- 172.21.15.64:0/3032294021 wait complete. 2024-05-09T09:24:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.587+0000 7facf95f9740 20 client.19263 trim_cache size 0 max 16384 2024-05-09T09:24:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7facf95f9740 1 Processor -- start 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7facf95f9740 1 -- start start 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7facf95f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443400 0x56554243da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7facf95f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x565542454b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7facf95f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 0x56554245fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7faceffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443400 0x56554243da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.622+0000 7faceffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443400 0x56554243da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:38966/0 (socket says 172.21.15.64:38966) 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facef7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x565542454b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facef7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x565542454b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:53000/0 (socket says 172.21.15.64:53000) 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facf4e8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 0x56554245fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facf4e8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 0x56554245fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51598/0 (socket says 172.21.15.64:51598) 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7faceffff640 1 -- 172.21.15.64:0/3089242410 learned_addr learned my addr 172.21.15.64:0/3089242410 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b32b0 con 0x565542448f60 2024-05-09T09:24:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b0af0 con 0x565542443400 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.623+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ae940 con 0x565542443df0 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.624+0000 7facf4e8b640 1 -- 172.21.15.64:0/3089242410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 msgr2=0x565542454b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.624+0000 7facf4e8b640 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x565542454b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.624+0000 7facf4e8b640 1 -- 172.21.15.64:0/3089242410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443400 msgr2=0x56554243da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.624+0000 7facf4e8b640 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443400 0x56554243da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.624+0000 7facf4e8b640 1 -- 172.21.15.64:0/3089242410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554245b8d0 con 0x565542448f60 2024-05-09T09:24:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.624+0000 7facf4e8b640 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 0x56554245fb50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7face80edfe0 tx=0x7face8031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6c4fab1815e610 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.626+0000 7facd5ffb640 1 -- 172.21.15.64:0/3089242410 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80044d0 con 0x565542448f60 2024-05-09T09:24:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.626+0000 7facd5ffb640 1 -- 172.21.15.64:0/3089242410 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face8036030 con 0x565542448f60 2024-05-09T09:24:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.626+0000 7facd5ffb640 1 -- 172.21.15.64:0/3089242410 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80c59b0 con 0x565542448f60 2024-05-09T09:24:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.629+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 msgr2=0x56554245fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.629+0000 7facf95f9740 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 0x56554245fb50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7face80edfe0 tx=0x7face8031a90 comp rx=0 tx=0).stop 2024-05-09T09:24:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.631+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 shutdown_connections 2024-05-09T09:24:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.631+0000 7facf95f9740 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542448f60 0x56554245fb50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.631+0000 7facf95f9740 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542443df0 0x565542454b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.631+0000 7facf95f9740 1 --2- 172.21.15.64:0/3089242410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443400 0x56554243da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.631+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 >> 172.21.15.64:0/3089242410 conn(0x56554243ea70 msgr2=0x56554243d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.632+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 shutdown_connections 2024-05-09T09:24:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.632+0000 7facf95f9740 1 -- 172.21.15.64:0/3089242410 wait complete. 2024-05-09T09:24:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.632+0000 7facf95f9740 1 -- start start 2024-05-09T09:24:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.633+0000 7facf95f9740 1 Processor -- start 2024-05-09T09:24:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.633+0000 7facf95f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x5655424525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.633+0000 7facf95f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x565542452b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7faceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x5655424525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7faceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x5655424525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51614/0 (socket says 172.21.15.64:51614) 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7faceffff640 1 -- 172.21.15.64:0/2175355852 learned_addr learned my addr 172.21.15.64:0/2175355852 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542453b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facef7fe640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x565542452b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7facf01aefb0 con 0x565542443400 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b0420 con 0x565542443df0 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facf01af680 con 0x565542448f60 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facf95f9740 10 client.0 _ll_register_callbacks cb 0x7face4009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:24:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.634+0000 7facf4e8b640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542453b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facef7fe640 1 -- 172.21.15.64:0/2175355852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 msgr2=0x565542453b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facef7fe640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542453b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facef7fe640 1 -- 172.21.15.64:0/2175355852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 msgr2=0x5655424525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facef7fe640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x5655424525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facef7fe640 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554243f7e0 con 0x565542443df0 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7faceffff640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x5655424525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facf4e8b640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542453b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:24:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7facef7fe640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x565542452b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x56554245bad0 tx=0x7face00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d068f321f667fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:24:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.635+0000 7faccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:24:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.636+0000 7faccf7fe640 1 -- 172.21.15.64:0/2175355852 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0030070 con 0x565542443df0 2024-05-09T09:24:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.636+0000 7faccf7fe640 1 -- 172.21.15.64:0/2175355852 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7face0035460 con 0x565542443df0 2024-05-09T09:24:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.636+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face8036030 con 0x565542443df0 2024-05-09T09:24:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.636+0000 7faccf7fe640 1 -- 172.21.15.64:0/2175355852 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7face0035660 con 0x565542443df0 2024-05-09T09:24:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.639+0000 7facf95f9740 10 client.28748 fetch_fsmap learned FSMap version 52 2024-05-09T09:24:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.639+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5655424bf590 con 0x565542443df0 2024-05-09T09:24:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.640+0000 7faccf7fe640 1 -- 172.21.15.64:0/2175355852 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5655423b4a70 con 0x565542443df0 2024-05-09T09:24:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.641+0000 7faccf7fe640 1 -- 172.21.15.64:0/2175355852 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 52) ==== 8366+0+0 (secure 0 0 0) 0x7face003f730 con 0x565542443df0 2024-05-09T09:24:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.641+0000 7facf95f9740 10 client.28748 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T09:24:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.641+0000 7facf95f9740 10 client.28748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:24:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.642+0000 7facf95f9740 20 client.28748 populate_metadata read hostname 'smithi064' 2024-05-09T09:24:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.642+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5655424bff20 0x5655424c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:24:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.642+0000 7facf95f9740 4 client.28748 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:24:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.642+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7face8036030 con 0x5655424bff20 2024-05-09T09:24:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.642+0000 7faceffff640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5655424bff20 0x5655424c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:24:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.648+0000 7faccf7fe640 10 client.28748 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.648+0000 7faceffff640 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5655424bff20 0x5655424c2340 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed20158ff68b5d5 server_cookie=d2105fd319c8140 in_seq=0 out_seq=0 2024-05-09T09:24:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.668+0000 7faccf7fe640 1 -- 172.21.15.64:0/2175355852 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7face8036030 con 0x5655424bff20 2024-05-09T09:24:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.668+0000 7faccf7fe640 10 client.28748 handle_command_reply: tid=0 2024-05-09T09:24:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.668+0000 7facf95f9740 1 client.28748 shutdown 2024-05-09T09:24:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 msgr2=0x565542452b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x565542452b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x56554245bad0 tx=0x7face00046c0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 shutdown_connections 2024-05-09T09:24:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5655424bff20 0x5655424c2340 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7faccf7fe640 0 client.28748 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565542448f60 0x565542453b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565542443df0 0x565542452b00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 --2- 172.21.15.64:0/2175355852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565542443400 0x5655424525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 >> 172.21.15.64:0/2175355852 conn(0x56554243ea70 msgr2=0x56554243dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 shutdown_connections 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.669+0000 7facf95f9740 1 -- 172.21.15.64:0/2175355852 wait complete. 2024-05-09T09:24:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.670+0000 7facf95f9740 20 client.28748 trim_cache size 0 max 16384 2024-05-09T09:24:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:24:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:24:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:24:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:24:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facd80a2dd0 msgr2=0x7facd80a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facd80a2dd0 0x7facd80a5290 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x565542444650 tx=0x7face000a690 comp rx=0 tx=0).stop 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0100f00 msgr2=0x7facf01a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0100f00 0x7facf01a54d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7face8009bf0 tx=0x7face8003ba0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 shutdown_connections 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7facd80a2dd0 0x7facd80a5290 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facf0100f00 0x7facf01a54d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00ff740 0x7facf019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 --2- 172.21.15.64:0/298357888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facf00ff290 0x7facf019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:24:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 >> 172.21.15.64:0/298357888 conn(0x7facf00fae90 msgr2=0x7facf00fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 shutdown_connections 2024-05-09T09:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:24:56.671+0000 7facf568c640 1 -- 172.21.15.64:0/298357888 wait complete. 2024-05-09T09:24:56.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:56 smithi064 ceph-mon[30149]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.7 MiB/s wr, 606 op/s 2024-05-09T09:24:56.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:56 smithi179 ceph-mon[34086]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.7 MiB/s wr, 606 op/s 2024-05-09T09:24:57.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:56 smithi139 ceph-mon[33599]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.7 MiB/s wr, 606 op/s 2024-05-09T09:24:58.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:24:58 smithi064 ceph-mon[30149]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.2 MiB/s wr, 544 op/s 2024-05-09T09:24:58.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:24:58 smithi179 ceph-mon[34086]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.2 MiB/s wr, 544 op/s 2024-05-09T09:24:59.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:24:58 smithi139 ceph-mon[33599]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.2 MiB/s wr, 544 op/s 2024-05-09T09:25:00.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:00 smithi064 ceph-mon[30149]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 12 MiB/s wr, 817 op/s 2024-05-09T09:25:00.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:00 smithi179 ceph-mon[34086]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 12 MiB/s wr, 817 op/s 2024-05-09T09:25:01.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:00 smithi139 ceph-mon[33599]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 12 MiB/s wr, 817 op/s 2024-05-09T09:25:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 880.0 s, 820.8 tps, lat 120.973 ms stddev 78.720 2024-05-09T09:25:01.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:01 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:25:01.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:01 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:25:02.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:01 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:25:02.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:02 smithi064 ceph-mon[30149]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 10 MiB/s wr, 758 op/s 2024-05-09T09:25:02.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:02 smithi179 ceph-mon[34086]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 10 MiB/s wr, 758 op/s 2024-05-09T09:25:03.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:02 smithi139 ceph-mon[33599]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 10 MiB/s wr, 758 op/s 2024-05-09T09:25:04.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:04 smithi064 ceph-mon[30149]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 12 MiB/s wr, 895 op/s 2024-05-09T09:25:04.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:04 smithi179 ceph-mon[34086]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 12 MiB/s wr, 895 op/s 2024-05-09T09:25:05.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:04 smithi139 ceph-mon[33599]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 12 MiB/s wr, 895 op/s 2024-05-09T09:25:05.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:25:05.904 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:05 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:25:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:25:05.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:05 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:25:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:25:06.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:05 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:25:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 885.0 s, 817.2 tps, lat 122.543 ms stddev 69.376 2024-05-09T09:25:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:06 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:25:06.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:06 smithi064 ceph-mon[30149]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 11 MiB/s wr, 826 op/s 2024-05-09T09:25:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:06 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:25:06.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:06 smithi179 ceph-mon[34086]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 11 MiB/s wr, 826 op/s 2024-05-09T09:25:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:06 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:25:07.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:06 smithi139 ceph-mon[33599]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 11 MiB/s wr, 826 op/s 2024-05-09T09:25:08.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:08 smithi064 ceph-mon[30149]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 701 op/s 2024-05-09T09:25:08.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:08 smithi179 ceph-mon[34086]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 701 op/s 2024-05-09T09:25:09.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:08 smithi139 ceph-mon[33599]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 701 op/s 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.845+0000 7f4318e38640 1 -- 172.21.15.64:0/3066057357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314108830 msgr2=0x7f4314108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.845+0000 7f4318e38640 1 --2- 172.21.15.64:0/3066057357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314108830 0x7f4314108c50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4308009900 tx=0x7f430802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.846+0000 7f4318e38640 1 -- 172.21.15.64:0/3066057357 shutdown_connections 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.846+0000 7f4318e38640 1 --2- 172.21.15.64:0/3066057357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43141100b0 0x7f43141124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.846+0000 7f4318e38640 1 --2- 172.21.15.64:0/3066057357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4314109190 0x7f431410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.846+0000 7f4318e38640 1 --2- 172.21.15.64:0/3066057357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314108830 0x7f4314108c50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.846+0000 7f4318e38640 1 -- 172.21.15.64:0/3066057357 >> 172.21.15.64:0/3066057357 conn(0x7f431406e8a0 msgr2=0x7f4314070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.847+0000 7f4318e38640 1 -- 172.21.15.64:0/3066057357 shutdown_connections 2024-05-09T09:25:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.847+0000 7f4318e38640 1 -- 172.21.15.64:0/3066057357 wait complete. 2024-05-09T09:25:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.848+0000 7f4318e38640 1 Processor -- start 2024-05-09T09:25:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.848+0000 7f4318e38640 1 -- start start 2024-05-09T09:25:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.849+0000 7f4318e38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4314108830 0x7f43141a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.849+0000 7f4318e38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4314109190 0x7f43141a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.849+0000 7f4318e38640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43141100b0 0x7f43141a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.849+0000 7f4318e38640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4314115be0 con 0x7f4314109190 2024-05-09T09:25:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.850+0000 7f4318e38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4314115a60 con 0x7f4314108830 2024-05-09T09:25:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.850+0000 7f4312d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43141100b0 0x7f43141a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.850+0000 7f4312d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43141100b0 0x7f43141a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57296/0 (socket says 172.21.15.64:57296) 2024-05-09T09:25:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.850+0000 7f4312d9d640 1 -- 172.21.15.64:0/3849962461 learned_addr learned my addr 172.21.15.64:0/3849962461 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.850+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4314115d60 con 0x7f43141100b0 2024-05-09T09:25:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.851+0000 7f431259c640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4314108830 0x7f43141a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.851+0000 7f431259c640 1 -- 172.21.15.64:0/3849962461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43141100b0 msgr2=0x7f43141a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.851+0000 7f431259c640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43141100b0 0x7f43141a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.851+0000 7f431259c640 1 -- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4314109190 msgr2=0x7f43141a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.851+0000 7f431259c640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4314109190 0x7f43141a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.851+0000 7f431259c640 1 -- 172.21.15.64:0/3849962461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43141a7df0 con 0x7f4314108830 2024-05-09T09:25:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.852+0000 7f431259c640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4314108830 0x7f43141a01d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f430802a930 tx=0x7f4308003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.852+0000 7f43037fe640 1 -- 172.21.15.64:0/3849962461 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308035030 con 0x7f4314108830 2024-05-09T09:25:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.852+0000 7f43037fe640 1 -- 172.21.15.64:0/3849962461 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4308041070 con 0x7f4314108830 2024-05-09T09:25:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.852+0000 7f43037fe640 1 -- 172.21.15.64:0/3849962461 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080303a0 con 0x7f4314108830 2024-05-09T09:25:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.854+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4314109940 con 0x7f4314108830 2024-05-09T09:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.854+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4314109cf0 con 0x7f4314108830 2024-05-09T09:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.858+0000 7f43037fe640 1 -- 172.21.15.64:0/3849962461 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4308031710 con 0x7f4314108830 2024-05-09T09:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.859+0000 7f43037fe640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42fc0a2cb0 0x7f42fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.859+0000 7f43037fe640 1 -- 172.21.15.64:0/3849962461 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43080f3f90 con 0x7f4314108830 2024-05-09T09:25:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.862+0000 7f4311d9b640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42fc0a2cb0 0x7f42fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.863+0000 7f4311d9b640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42fc0a2cb0 0x7f42fc0a5170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f430c005c40 tx=0x7f430c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.864+0000 7f431bc21740 1 Processor -- start 2024-05-09T09:25:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.864+0000 7f431bc21740 1 -- start start 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.864+0000 7f431bc21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058250 0x55e13b058670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.864+0000 7f431bc21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b05d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.864+0000 7f431bc21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b05ddb0 0x55e13b060240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f431bc21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e13afbc340 con 0x55e13b05ddb0 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f431bc21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e13af64290 con 0x55e13b058c40 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f431bc21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e13afb8300 con 0x55e13b058250 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b05d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b05d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:45922/0 (socket says 172.21.15.64:45922) 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 -- 172.21.15.64:0/837624498 learned_addr learned my addr 172.21.15.64:0/837624498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f431259c640 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058250 0x55e13b058670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 -- 172.21.15.64:0/837624498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058250 msgr2=0x55e13b058670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058250 0x55e13b058670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 -- 172.21.15.64:0/837624498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b05ddb0 msgr2=0x55e13b060240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b05ddb0 0x55e13b060240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 -- 172.21.15.64:0/837624498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13af888e0 con 0x55e13b058c40 2024-05-09T09:25:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.865+0000 7f4311d9b640 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b05d740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f430c010530 tx=0x7f430c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a39f1dd1bbd3b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.866+0000 7f4300ff9640 1 -- 172.21.15.64:0/837624498 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c020070 con 0x55e13b058c40 2024-05-09T09:25:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.866+0000 7f4300ff9640 1 -- 172.21.15.64:0/837624498 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f430c01cb90 con 0x55e13b058c40 2024-05-09T09:25:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.866+0000 7f4300ff9640 1 -- 172.21.15.64:0/837624498 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c025ba0 con 0x55e13b058c40 2024-05-09T09:25:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 -- 172.21.15.64:0/837624498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 msgr2=0x55e13b05d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b05d740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f430c010530 tx=0x7f430c011000 comp rx=0 tx=0).stop 2024-05-09T09:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 -- 172.21.15.64:0/837624498 shutdown_connections 2024-05-09T09:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b05ddb0 0x55e13b060240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b05d740 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 --2- 172.21.15.64:0/837624498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058250 0x55e13b058670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.867+0000 7f431bc21740 1 -- 172.21.15.64:0/837624498 >> 172.21.15.64:0/837624498 conn(0x55e13b0538c0 msgr2=0x55e13b053cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.868+0000 7f431bc21740 1 -- 172.21.15.64:0/837624498 shutdown_connections 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.868+0000 7f431bc21740 1 -- 172.21.15.64:0/837624498 wait complete. 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.868+0000 7f431bc21740 1 -- start start 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.868+0000 7f431bc21740 1 Processor -- start 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b062550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b062a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e13b060d60 con 0x55e13b058250 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e13af77790 con 0x55e13b058c40 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e13b060ee0 con 0x55e13b05ddb0 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.869+0000 7f431bc21740 10 client.0 _ll_register_callbacks cb 0x55e13afb2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b062550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b062550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52512/0 (socket says 172.21.15.64:52512) 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 -- 172.21.15.64:0/4105379623 learned_addr learned my addr 172.21.15.64:0/4105379623 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f4311d9b640 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b062a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 -- 172.21.15.64:0/4105379623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 msgr2=0x55e13b069ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b069ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 -- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 msgr2=0x55e13b062a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b062a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 -- 172.21.15.64:0/4105379623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13acd0210 con 0x55e13b058250 2024-05-09T09:25:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f431259c640 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b062550 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f43080c0390 tx=0x7f43080c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba40b79cf4acb36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.872+0000 7f42f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.873+0000 7f42f9ffb640 1 -- 172.21.15.64:0/4105379623 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080d8070 con 0x55e13b058250 2024-05-09T09:25:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.873+0000 7f42f9ffb640 1 -- 172.21.15.64:0/4105379623 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43080d9260 con 0x55e13b058250 2024-05-09T09:25:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.874+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c01cb90 con 0x55e13b058250 2024-05-09T09:25:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.874+0000 7f42f9ffb640 1 -- 172.21.15.64:0/4105379623 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f43080c0400 con 0x55e13b058250 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.874+0000 7f431bc21740 10 client.19275 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.874+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55e13ad9c620 con 0x55e13b058250 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f42f9ffb640 1 -- 172.21.15.64:0/4105379623 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43080c47b0 con 0x55e13b058250 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f42f9ffb640 1 -- 172.21.15.64:0/4105379623 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f430804e020 con 0x55e13b058250 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f431bc21740 10 client.19275 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f431bc21740 10 client.19275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f431bc21740 20 client.19275 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f431bc21740 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b0922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f431bc21740 4 client.19275 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f430c01cb90 con 0x55e13b08fe40 2024-05-09T09:25:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.875+0000 7f4311d9b640 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b0922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.876+0000 7f4311d9b640 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b0922c0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6536aee03fb724 server_cookie=f32681497d4dc74b in_seq=0 out_seq=0 2024-05-09T09:25:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.876+0000 7f42f9ffb640 10 client.19275 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.878+0000 7f42f9ffb640 1 -- 172.21.15.64:0/4105379623 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f430c01cb90 con 0x55e13b08fe40 2024-05-09T09:25:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.878+0000 7f42f9ffb640 10 client.19275 handle_command_reply: tid=0 2024-05-09T09:25:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.878+0000 7f431bc21740 1 client.19275 shutdown 2024-05-09T09:25:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 msgr2=0x55e13b062550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b062550 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f43080c0390 tx=0x7f43080c0000 comp rx=0 tx=0).stop 2024-05-09T09:25:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 shutdown_connections 2024-05-09T09:25:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b0922c0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f42f9ffb640 0 client.19275 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b062a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 --2- 172.21.15.64:0/4105379623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b062550 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.880+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 >> 172.21.15.64:0/4105379623 conn(0x55e13b0538c0 msgr2=0x55e13b052a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.881+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 shutdown_connections 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.881+0000 7f431bc21740 1 -- 172.21.15.64:0/4105379623 wait complete. 2024-05-09T09:25:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.882+0000 7f431bc21740 20 client.19275 trim_cache size 0 max 16384 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.913+0000 7f431bc21740 1 Processor -- start 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.913+0000 7f431bc21740 1 -- start start 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.913+0000 7f431bc21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b0629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.913+0000 7f431bc21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 0x55e13b069a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.913+0000 7f431bc21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b05ddb0 0x55e13b056f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f431bc21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f431407ecd0 con 0x55e13b058250 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f431bc21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f431407c4e0 con 0x55e13b05ddb0 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f431bc21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f431407a330 con 0x55e13b058c40 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 0x55e13b069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4312d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b05ddb0 0x55e13b056f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 0x55e13b069a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:57332/0 (socket says 172.21.15.64:57332) 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4311d9b640 1 -- 172.21.15.64:0/3083269427 learned_addr learned my addr 172.21.15.64:0/3083269427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f431259c640 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b0629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4312d9d640 1 -- 172.21.15.64:0/3083269427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 msgr2=0x55e13b069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4312d9d640 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 0x55e13b069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4312d9d640 1 -- 172.21.15.64:0/3083269427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 msgr2=0x55e13b0629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4312d9d640 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b0629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4312d9d640 1 -- 172.21.15.64:0/3083269427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13b054630 con 0x55e13b05ddb0 2024-05-09T09:25:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.914+0000 7f4311d9b640 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 0x55e13b069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.915+0000 7f4312d9d640 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b05ddb0 0x55e13b056f80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55e13b054830 tx=0x7f430400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0943fa827724ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.915+0000 7f42fbfff640 1 -- 172.21.15.64:0/3083269427 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304010c90 con 0x55e13b05ddb0 2024-05-09T09:25:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.915+0000 7f42fbfff640 1 -- 172.21.15.64:0/3083269427 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4304010e30 con 0x55e13b05ddb0 2024-05-09T09:25:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.915+0000 7f42fbfff640 1 -- 172.21.15.64:0/3083269427 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304011440 con 0x55e13b05ddb0 2024-05-09T09:25:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.916+0000 7f431bc21740 1 -- 172.21.15.64:0/3083269427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b05ddb0 msgr2=0x55e13b056f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.916+0000 7f431bc21740 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b05ddb0 0x55e13b056f80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55e13b054830 tx=0x7f430400bf90 comp rx=0 tx=0).stop 2024-05-09T09:25:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 -- 172.21.15.64:0/3083269427 shutdown_connections 2024-05-09T09:25:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b05ddb0 0x55e13b056f80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b058c40 0x55e13b069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 --2- 172.21.15.64:0/3083269427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b0629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 -- 172.21.15.64:0/3083269427 >> 172.21.15.64:0/3083269427 conn(0x55e13b0538c0 msgr2=0x55e13b056870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 -- 172.21.15.64:0/3083269427 shutdown_connections 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.917+0000 7f431bc21740 1 -- 172.21.15.64:0/3083269427 wait complete. 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.918+0000 7f431bc21740 1 -- start start 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.918+0000 7f431bc21740 1 Processor -- start 2024-05-09T09:25:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.919+0000 7f431bc21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b067240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.919+0000 7f431259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b067240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.920+0000 7f431bc21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.920+0000 7f431bc21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b068720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.920+0000 7f431bc21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f431407a9a0 con 0x55e13b058250 2024-05-09T09:25:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.920+0000 7f431bc21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f431407be10 con 0x55e13b058c40 2024-05-09T09:25:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.920+0000 7f431bc21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f431407b070 con 0x55e13b05ddb0 2024-05-09T09:25:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.920+0000 7f431bc21740 10 client.0 _ll_register_callbacks cb 0x7f43080095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.927+0000 7f431259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b067240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52528/0 (socket says 172.21.15.64:52528) 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.927+0000 7f431259c640 1 -- 172.21.15.64:0/3297707072 learned_addr learned my addr 172.21.15.64:0/3297707072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f4311d9b640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f4312d9d640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b068720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f431259c640 1 -- 172.21.15.64:0/3297707072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 msgr2=0x55e13b068720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f431259c640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b068720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f431259c640 1 -- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 msgr2=0x55e13b067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f431259c640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f431259c640 1 -- 172.21.15.64:0/3297707072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13b068d10 con 0x55e13b058250 2024-05-09T09:25:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.928+0000 7f4312d9d640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b068720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.929+0000 7f431259c640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b067240 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f43080be6b0 tx=0x7f4308005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c0bc5b8c1aa06e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.929+0000 7f42f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.929+0000 7f42f9ffb640 1 -- 172.21.15.64:0/3297707072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080a7070 con 0x55e13b058250 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.929+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4304010e30 con 0x55e13b058250 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.929+0000 7f42f9ffb640 1 -- 172.21.15.64:0/3297707072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f43080f1bb0 con 0x55e13b058250 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.930+0000 7f42f9ffb640 1 -- 172.21.15.64:0/3297707072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f43080d8070 con 0x55e13b058250 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.930+0000 7f431bc21740 10 client.19281 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.930+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55e13b0fecb0 con 0x55e13b058250 2024-05-09T09:25:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f42f9ffb640 1 -- 172.21.15.64:0/3297707072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43080c47b0 con 0x55e13b058250 2024-05-09T09:25:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f42f9ffb640 1 -- 172.21.15.64:0/3297707072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f430804e070 con 0x55e13b058250 2024-05-09T09:25:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f431bc21740 10 client.19281 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f431bc21740 10 client.19281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f431bc21740 20 client.19281 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f431bc21740 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b101250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f431bc21740 4 client.19281 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:25:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.931+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4304010e30 con 0x55e13b08fe40 2024-05-09T09:25:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.932+0000 7f4311d9b640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.933+0000 7f42f9ffb640 10 client.19281 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.933+0000 7f4311d9b640 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b101250 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c93651f0f5b6d1b server_cookie=1b77a5862a4a8d9 in_seq=0 out_seq=0 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.946+0000 7f42f9ffb640 1 -- 172.21.15.64:0/3297707072 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4304010e30 con 0x55e13b08fe40 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.946+0000 7f42f9ffb640 10 client.19281 handle_command_reply: tid=0 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.946+0000 7f431bc21740 1 client.19281 shutdown 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.946+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 msgr2=0x55e13b067240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.946+0000 7f431bc21740 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b067240 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f43080be6b0 tx=0x7f4308005fd0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 shutdown_connections 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55e13b08fe40 0x55e13b101250 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e13b05ddb0 0x55e13b068720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e13b058c40 0x55e13b067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f42f9ffb640 0 client.19281 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 --2- 172.21.15.64:0/3297707072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e13b058250 0x55e13b067240 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 >> 172.21.15.64:0/3297707072 conn(0x55e13b0538c0 msgr2=0x55e13b052170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 shutdown_connections 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.947+0000 7f431bc21740 1 -- 172.21.15.64:0/3297707072 wait complete. 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.948+0000 7f431bc21740 20 client.19281 trim_cache size 0 max 16384 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:25:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:25:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42fc0a2cb0 msgr2=0x7f42fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42fc0a2cb0 0x7f42fc0a5170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f430c005c40 tx=0x7f430c005bd0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4314108830 msgr2=0x7f43141a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4314108830 0x7f43141a01d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f430802a930 tx=0x7f4308003c10 comp rx=0 tx=0).stop 2024-05-09T09:25:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 shutdown_connections 2024-05-09T09:25:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42fc0a2cb0 0x7f42fc0a5170 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43141100b0 0x7f43141a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4314109190 0x7f43141a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 --2- 172.21.15.64:0/3849962461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4314108830 0x7f43141a01d0 secure :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f430802a930 tx=0x7f4308003c10 comp rx=0 tx=0).stop 2024-05-09T09:25:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.952+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 >> 172.21.15.64:0/3849962461 conn(0x7f431406e8a0 msgr2=0x7f4314070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.953+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 shutdown_connections 2024-05-09T09:25:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:09.953+0000 7f4318e38640 1 -- 172.21.15.64:0/3849962461 wait complete. 2024-05-09T09:25:10.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:10 smithi064 ceph-mon[30149]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-05-09T09:25:10.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:10 smithi179 ceph-mon[34086]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-05-09T09:25:11.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:10 smithi139 ceph-mon[33599]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-05-09T09:25:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 890.0 s, 895.6 tps, lat 112.416 ms stddev 97.663 2024-05-09T09:25:12.903 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:12 smithi064 ceph-mon[30149]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.7 MiB/s wr, 713 op/s 2024-05-09T09:25:12.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:12 smithi179 ceph-mon[34086]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.7 MiB/s wr, 713 op/s 2024-05-09T09:25:13.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:12 smithi139 ceph-mon[33599]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.7 MiB/s wr, 713 op/s 2024-05-09T09:25:14.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:14 smithi179 ceph-mon[34086]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 864 op/s 2024-05-09T09:25:15.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:14 smithi139 ceph-mon[33599]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 864 op/s 2024-05-09T09:25:15.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:14 smithi064 ceph-mon[30149]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 864 op/s 2024-05-09T09:25:16.352 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 895.0 s, 811.0 tps, lat 123.007 ms stddev 87.251 2024-05-09T09:25:16.982 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:16 smithi179 ceph-mon[34086]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 838 op/s 2024-05-09T09:25:17.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:16 smithi139 ceph-mon[33599]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 838 op/s 2024-05-09T09:25:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:16 smithi064 ceph-mon[30149]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 838 op/s 2024-05-09T09:25:18.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:18 smithi179 ceph-mon[34086]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-05-09T09:25:19.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:18 smithi139 ceph-mon[33599]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-05-09T09:25:19.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:18 smithi064 ceph-mon[30149]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-05-09T09:25:20.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:20 smithi179 ceph-mon[34086]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 13 MiB/s wr, 963 op/s 2024-05-09T09:25:21.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:25:21.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:25:21.046 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:20 smithi064 ceph-mon[30149]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 13 MiB/s wr, 963 op/s 2024-05-09T09:25:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:20 smithi139 ceph-mon[33599]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 13 MiB/s wr, 963 op/s 2024-05-09T09:25:21.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 -- 172.21.15.64:0/4166545804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4410 msgr2=0x7f34280a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 --2- 172.21.15.64:0/4166545804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4410 0x7f34280a61c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3424009900 tx=0x7f342402a440 comp rx=0 tx=0).stop 2024-05-09T09:25:21.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 -- 172.21.15.64:0/4166545804 shutdown_connections 2024-05-09T09:25:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 --2- 172.21.15.64:0/4166545804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34280a5450 0x7f34280a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 --2- 172.21.15.64:0/4166545804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4fd0 0x7f34280a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 --2- 172.21.15.64:0/4166545804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4410 0x7f34280a61c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.209+0000 7f34353dd640 1 -- 172.21.15.64:0/4166545804 >> 172.21.15.64:0/4166545804 conn(0x7f34280a00d0 msgr2=0x7f34280a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.210+0000 7f34353dd640 1 -- 172.21.15.64:0/4166545804 shutdown_connections 2024-05-09T09:25:21.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.210+0000 7f34353dd640 1 -- 172.21.15.64:0/4166545804 wait complete. 2024-05-09T09:25:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.211+0000 7f34353dd640 1 Processor -- start 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.211+0000 7f34353dd640 1 -- start start 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.211+0000 7f34353dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 0x7f34280b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f34353dd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4fd0 0x7f34280b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f34353dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34280a5450 0x7f34280b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f342ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 0x7f34280b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f342ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 0x7f34280b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47224/0 (socket says 172.21.15.64:47224) 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f342f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4fd0 0x7f34280b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f342ffff640 1 -- 172.21.15.64:0/1758735049 learned_addr learned my addr 172.21.15.64:0/1758735049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34280b4cc0 con 0x7f34280a5450 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34280b4b40 con 0x7f34280a4410 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.212+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34280b4e40 con 0x7f34280a4fd0 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342ffff640 1 -- 172.21.15.64:0/1758735049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4fd0 msgr2=0x7f34280b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342ffff640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4fd0 0x7f34280b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342ffff640 1 -- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34280a5450 msgr2=0x7f34280b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342ffff640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34280a5450 0x7f34280b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342ffff640 1 -- 172.21.15.64:0/1758735049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34280b1480 con 0x7f34280a4410 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342ffff640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 0x7f34280b4050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3424005ab0 tx=0x7f3424003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342d7fa640 1 -- 172.21.15.64:0/1758735049 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342403f070 con 0x7f34280a4410 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f342d7fa640 1 -- 172.21.15.64:0/1758735049 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3424031c30 con 0x7f34280a4410 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34280b1710 con 0x7f34280a4410 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.213+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342814bb00 con 0x7f34280a4410 2024-05-09T09:25:21.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.214+0000 7f342d7fa640 1 -- 172.21.15.64:0/1758735049 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424035030 con 0x7f34280a4410 2024-05-09T09:25:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.218+0000 7f343942b740 1 Processor -- start 2024-05-09T09:25:21.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.220+0000 7f342d7fa640 1 -- 172.21.15.64:0/1758735049 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3424042070 con 0x7f34280a4410 2024-05-09T09:25:21.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.223+0000 7f343942b740 1 -- start start 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.225+0000 7f342d7fa640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34200a2c60 0x7f34200a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f342d7fa640 1 -- 172.21.15.64:0/1758735049 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f34240efbb0 con 0x7f34280a4410 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f343942b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edf120 0x55ae34edf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f343942b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34ee4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f343942b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ee7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f343942b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ae34e43d00 con 0x55ae34edf120 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f343942b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ae34deb4c0 con 0x55ae34ee4c80 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f343942b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae34e3fcc0 con 0x55ae34edfb10 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f3434bdc640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34200a2c60 0x7f34200a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f342f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34ee4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.226+0000 7f3434bdc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ee7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ee7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47238/0 (socket says 172.21.15.64:47238) 2024-05-09T09:25:21.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 -- 172.21.15.64:0/2109981126 learned_addr learned my addr 172.21.15.64:0/2109981126 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 -- 172.21.15.64:0/2109981126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 msgr2=0x55ae34ee4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34ee4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 -- 172.21.15.64:0/2109981126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edf120 msgr2=0x55ae34edf540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T09:25:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edf120 0x55ae34edf540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 -- 172.21.15.64:0/2109981126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae34c217d0 con 0x55ae34ee4c80 2024-05-09T09:25:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.227+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ee7110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f34300673a0 tx=0x7f343007b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=162081b887c7412e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.229+0000 7f3434bdc640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34200a2c60 0x7f34200a5120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3430067120 tx=0x7f3430071cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.231+0000 7f341a7fc640 1 -- 172.21.15.64:0/2109981126 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430075be0 con 0x55ae34ee4c80 2024-05-09T09:25:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.231+0000 7f341a7fc640 1 -- 172.21.15.64:0/2109981126 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3430075d80 con 0x55ae34ee4c80 2024-05-09T09:25:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.231+0000 7f341a7fc640 1 -- 172.21.15.64:0/2109981126 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430080640 con 0x55ae34ee4c80 2024-05-09T09:25:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.232+0000 7f343942b740 1 -- 172.21.15.64:0/2109981126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 msgr2=0x55ae34ee7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.232+0000 7f343942b740 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ee7110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f34300673a0 tx=0x7f343007b000 comp rx=0 tx=0).stop 2024-05-09T09:25:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.233+0000 7f343942b740 1 -- 172.21.15.64:0/2109981126 shutdown_connections 2024-05-09T09:25:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.233+0000 7f343942b740 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ee7110 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.233+0000 7f343942b740 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34ee4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.233+0000 7f343942b740 1 --2- 172.21.15.64:0/2109981126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edf120 0x55ae34edf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.233+0000 7f343942b740 1 -- 172.21.15.64:0/2109981126 >> 172.21.15.64:0/2109981126 conn(0x55ae34eda790 msgr2=0x55ae34edab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.238+0000 7f343942b740 1 -- 172.21.15.64:0/2109981126 shutdown_connections 2024-05-09T09:25:21.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.238+0000 7f343942b740 1 -- 172.21.15.64:0/2109981126 wait complete. 2024-05-09T09:25:21.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.239+0000 7f343942b740 1 -- start start 2024-05-09T09:25:21.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.239+0000 7f343942b740 1 Processor -- start 2024-05-09T09:25:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34ee94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ee99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34ee4c80 0x55ae34ef0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ae34ee7bd0 con 0x55ae34edfb10 2024-05-09T09:25:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ae34dfeb90 con 0x55ae34edf120 2024-05-09T09:25:21.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae34ee7d50 con 0x55ae34ee4c80 2024-05-09T09:25:21.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f343942b740 10 client.0 _ll_register_callbacks cb 0x55ae34e3ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:21.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f342ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34ee94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f342ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34ee94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47244/0 (socket says 172.21.15.64:47244) 2024-05-09T09:25:21.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.241+0000 7f342ffff640 1 -- 172.21.15.64:0/2630689924 learned_addr learned my addr 172.21.15.64:0/2630689924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:21.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.242+0000 7f342ffff640 1 -- 172.21.15.64:0/2630689924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34ee4c80 msgr2=0x55ae34ef0a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.242+0000 7f342ffff640 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34ee4c80 0x55ae34ef0a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.242+0000 7f342ffff640 1 -- 172.21.15.64:0/2630689924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 msgr2=0x55ae34ee99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.242+0000 7f342ffff640 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ee99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.242+0000 7f342ffff640 1 -- 172.21.15.64:0/2630689924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae34b573f0 con 0x55ae34edf120 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.243+0000 7f342ffff640 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34ee94b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3424005dc0 tx=0x7f3424031de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5947eac71bef5659 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.243+0000 7f340f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:25:21.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.243+0000 7f340f7fe640 1 -- 172.21.15.64:0/2630689924 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424049070 con 0x55ae34edf120 2024-05-09T09:25:21.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.243+0000 7f340f7fe640 1 -- 172.21.15.64:0/2630689924 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f34240b86f0 con 0x55ae34edf120 2024-05-09T09:25:21.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.244+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3430075d80 con 0x55ae34edf120 2024-05-09T09:25:21.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.244+0000 7f340f7fe640 1 -- 172.21.15.64:0/2630689924 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f34240c0430 con 0x55ae34edf120 2024-05-09T09:25:21.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.244+0000 7f343942b740 10 client.28775 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:21.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.244+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ae34b170f0 con 0x55ae34edf120 2024-05-09T09:25:21.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.245+0000 7f340f7fe640 1 -- 172.21.15.64:0/2630689924 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3430075d80 con 0x55ae34edf120 2024-05-09T09:25:21.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f340f7fe640 1 -- 172.21.15.64:0/2630689924 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f342403b580 con 0x55ae34edf120 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f343942b740 10 client.28775 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f343942b740 10 client.28775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f343942b740 20 client.28775 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f343942b740 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34ef52e0 0x55ae34ef77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f343942b740 4 client.28775 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae34e51430 con 0x55ae34ef52e0 2024-05-09T09:25:21.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.246+0000 7f342f7fe640 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34ef52e0 0x55ae34ef77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.251+0000 7f342f7fe640 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34ef52e0 0x55ae34ef77a0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442625b038514160 server_cookie=f57c081b6e043864 in_seq=0 out_seq=0 2024-05-09T09:25:21.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.251+0000 7f340f7fe640 10 client.28775 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:21.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f340f7fe640 1 -- 172.21.15.64:0/2630689924 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae34e51430 con 0x55ae34ef52e0 2024-05-09T09:25:21.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f340f7fe640 10 client.28775 handle_command_reply: tid=0 2024-05-09T09:25:21.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 client.28775 shutdown 2024-05-09T09:25:21.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 msgr2=0x55ae34ee94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34ee94b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3424005dc0 tx=0x7f3424031de0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 shutdown_connections 2024-05-09T09:25:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34ef52e0 0x55ae34ef77a0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34ee4c80 0x55ae34ef0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ee99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 --2- 172.21.15.64:0/2630689924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34ee94b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 >> 172.21.15.64:0/2630689924 conn(0x55ae34eda790 msgr2=0x55ae34ed9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.254+0000 7f340f7fe640 0 client.28775 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:21.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.255+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 shutdown_connections 2024-05-09T09:25:21.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.255+0000 7f343942b740 1 -- 172.21.15.64:0/2630689924 wait complete. 2024-05-09T09:25:21.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.255+0000 7f343942b740 20 client.28775 trim_cache size 0 max 16384 2024-05-09T09:25:21.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.292+0000 7f343942b740 1 Processor -- start 2024-05-09T09:25:21.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.293+0000 7f343942b740 1 -- start start 2024-05-09T09:25:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.293+0000 7f343942b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edf120 0x55ae34ee97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.294+0000 7f343942b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ef09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.296+0000 7f342ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edf120 0x55ae34ee97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.298+0000 7f343942b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ed91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.298+0000 7f343942b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34281589e0 con 0x55ae34edfb10 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.298+0000 7f343942b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34281561f0 con 0x55ae34ee4c80 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.298+0000 7f343942b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3428154040 con 0x55ae34edf120 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.299+0000 7f342f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ef09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.299+0000 7f3434bdc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ed91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.299+0000 7f3434bdc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ed91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47248/0 (socket says 172.21.15.64:47248) 2024-05-09T09:25:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.299+0000 7f3434bdc640 1 -- 172.21.15.64:0/2379490977 learned_addr learned my addr 172.21.15.64:0/2379490977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.300+0000 7f3434bdc640 1 -- 172.21.15.64:0/2379490977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edf120 msgr2=0x55ae34ee97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.300+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edf120 0x55ae34ee97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.300+0000 7f3434bdc640 1 -- 172.21.15.64:0/2379490977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 msgr2=0x55ae34ef09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.300+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ef09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.300+0000 7f3434bdc640 1 -- 172.21.15.64:0/2379490977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae34ecb930 con 0x55ae34ee4c80 2024-05-09T09:25:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.301+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ed91a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3430000b60 tx=0x7f3430078000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5852a595d2a1008e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.302+0000 7f3419ffb640 1 -- 172.21.15.64:0/2379490977 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430087040 con 0x55ae34ee4c80 2024-05-09T09:25:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.302+0000 7f3419ffb640 1 -- 172.21.15.64:0/2379490977 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3430078070 con 0x55ae34ee4c80 2024-05-09T09:25:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.302+0000 7f3419ffb640 1 -- 172.21.15.64:0/2379490977 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430080bb0 con 0x55ae34ee4c80 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 -- 172.21.15.64:0/2379490977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 msgr2=0x55ae34ed91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ed91a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3430000b60 tx=0x7f3430078000 comp rx=0 tx=0).stop 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 -- 172.21.15.64:0/2379490977 shutdown_connections 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34ee4c80 0x55ae34ed91a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34edfb10 0x55ae34ef09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 --2- 172.21.15.64:0/2379490977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edf120 0x55ae34ee97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.303+0000 7f343942b740 1 -- 172.21.15.64:0/2379490977 >> 172.21.15.64:0/2379490977 conn(0x55ae34eda790 msgr2=0x55ae34ee90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.306+0000 7f343942b740 1 -- 172.21.15.64:0/2379490977 shutdown_connections 2024-05-09T09:25:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.306+0000 7f343942b740 1 -- 172.21.15.64:0/2379490977 wait complete. 2024-05-09T09:25:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.307+0000 7f343942b740 1 -- start start 2024-05-09T09:25:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.307+0000 7f343942b740 1 Processor -- start 2024-05-09T09:25:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f343942b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34eed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f343942b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34eede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f343942b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34ee4c80 0x55ae34f598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f342ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34eed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f342ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34eed8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47256/0 (socket says 172.21.15.64:47256) 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f342f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34eede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f342f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34eede20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55028/0 (socket says 172.21.15.64:55028) 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f342ffff640 1 -- 172.21.15.64:0/2836762287 learned_addr learned my addr 172.21.15.64:0/2836762287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34281546b0 con 0x55ae34ee4c80 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.308+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3428155b20 con 0x55ae34edf120 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.309+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3428154d80 con 0x55ae34edfb10 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.309+0000 7f343942b740 10 client.0 _ll_register_callbacks cb 0x7f34240bbbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.309+0000 7f3434bdc640 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34ee4c80 0x55ae34f598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f342f7fe640 1 -- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 msgr2=0x55ae34eed8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:21.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f342f7fe640 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34eed8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f342f7fe640 1 -- 172.21.15.64:0/2836762287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34ee4c80 msgr2=0x55ae34f598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:21.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f342f7fe640 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34ee4c80 0x55ae34f598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f342f7fe640 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae34f59e10 con 0x55ae34edfb10 2024-05-09T09:25:21.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f342f7fe640 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34eede20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55ae34edb660 tx=0x7f341c009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67aa3b7e13a85f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f340f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:25:21.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.310+0000 7f340f7fe640 1 -- 172.21.15.64:0/2836762287 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c016ae0 con 0x55ae34edfb10 2024-05-09T09:25:21.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.311+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3430078070 con 0x55ae34edfb10 2024-05-09T09:25:21.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.311+0000 7f340f7fe640 1 -- 172.21.15.64:0/2836762287 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f341c00f040 con 0x55ae34edfb10 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.311+0000 7f340f7fe640 1 -- 172.21.15.64:0/2836762287 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f341c02c050 con 0x55ae34edfb10 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.311+0000 7f343942b740 10 client.28924 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.311+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ae34f5b1a0 con 0x55ae34edfb10 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.311+0000 7f340f7fe640 1 -- 172.21.15.64:0/2836762287 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341c016c80 con 0x55ae34edfb10 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f340f7fe640 1 -- 172.21.15.64:0/2836762287 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f341c014410 con 0x55ae34edfb10 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f343942b740 10 client.28924 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f343942b740 10 client.28924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f343942b740 20 client.28924 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:21.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f343942b740 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34f5bb60 0x55ae34f5df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f343942b740 4 client.28924 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:25:21.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3430078070 con 0x55ae34f5bb60 2024-05-09T09:25:21.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.312+0000 7f342ffff640 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34f5bb60 0x55ae34f5df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.319+0000 7f342ffff640 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34f5bb60 0x55ae34f5df80 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2935f4458598da29 server_cookie=d4ad58deae99381b in_seq=0 out_seq=0 2024-05-09T09:25:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.319+0000 7f340f7fe640 10 client.28924 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.320+0000 7f340f7fe640 1 -- 172.21.15.64:0/2836762287 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f3430078070 con 0x55ae34f5bb60 2024-05-09T09:25:21.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.320+0000 7f340f7fe640 10 client.28924 handle_command_reply: tid=0 2024-05-09T09:25:21.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.320+0000 7f343942b740 1 client.28924 shutdown 2024-05-09T09:25:21.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.320+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 msgr2=0x55ae34eede20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.320+0000 7f343942b740 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34eede20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55ae34edb660 tx=0x7f341c009ea0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 shutdown_connections 2024-05-09T09:25:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f343942b740 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x55ae34f5bb60 0x55ae34f5df80 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f343942b740 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae34ee4c80 0x55ae34f598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f340f7fe640 0 client.28924 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f343942b740 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae34edfb10 0x55ae34eede20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f343942b740 1 --2- 172.21.15.64:0/2836762287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ae34edf120 0x55ae34eed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.321+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 >> 172.21.15.64:0/2836762287 conn(0x55ae34eda790 msgr2=0x55ae34ed91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.322+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 shutdown_connections 2024-05-09T09:25:21.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.322+0000 7f343942b740 1 -- 172.21.15.64:0/2836762287 wait complete. 2024-05-09T09:25:21.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.322+0000 7f343942b740 20 client.28924 trim_cache size 0 max 16384 2024-05-09T09:25:21.324 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:25:21.324 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:25:21.324 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:25:21.324 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:25:21.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.325+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34200a2c60 msgr2=0x7f34200a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.325+0000 7f34353dd640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34200a2c60 0x7f34200a5120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3430067120 tx=0x7f3430071cf0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.325+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 msgr2=0x7f34280b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.325+0000 7f34353dd640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 0x7f34280b4050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3424005ab0 tx=0x7f3424003c20 comp rx=0 tx=0).stop 2024-05-09T09:25:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.326+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 shutdown_connections 2024-05-09T09:25:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.326+0000 7f34353dd640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f34200a2c60 0x7f34200a5120 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.326+0000 7f34353dd640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34280a5450 0x7f34280b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.326+0000 7f34353dd640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34280a4fd0 0x7f34280b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.326+0000 7f34353dd640 1 --2- 172.21.15.64:0/1758735049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34280a4410 0x7f34280b4050 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.326+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 >> 172.21.15.64:0/1758735049 conn(0x7f34280a00d0 msgr2=0x7f34280aa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.327+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 shutdown_connections 2024-05-09T09:25:21.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.327+0000 7f34353dd640 1 -- 172.21.15.64:0/1758735049 wait complete. 2024-05-09T09:25:21.346 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:25:21.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:25:21.353 INFO:teuthology.orchestra.run.smithi064.stderr:progress: 900.0 s, 835.0 tps, lat 119.991 ms stddev 54.525 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/308432685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 msgr2=0x7f5f3410dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/308432685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 0x7f5f3410dfd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c01acb0 tx=0x7f5f2c03b900 comp rx=0 tx=0).stop 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/308432685 shutdown_connections 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/308432685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 0x7f5f3410dfd0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/308432685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f3410b1a0 0x7f5f3410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/308432685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34108830 0x7f5f3410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/308432685 >> 172.21.15.64:0/308432685 conn(0x7f5f3406e5c0 msgr2=0x7f5f34070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/308432685 shutdown_connections 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.575+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/308432685 wait complete. 2024-05-09T09:25:21.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 Processor -- start 2024-05-09T09:25:21.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 -- start start 2024-05-09T09:25:21.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 0x7f5f3419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f3410b1a0 0x7f5f3419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 0x7f5f341a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341117d0 con 0x7f5f3410db50 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34111650 con 0x7f5f3410b1a0 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.576+0000 7f5f3bdd5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34111950 con 0x7f5f34108830 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 0x7f5f3419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 0x7f5f3419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55046/0 (socket says 172.21.15.64:55046) 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 -- 172.21.15.64:0/1126261317 learned_addr learned my addr 172.21.15.64:0/1126261317 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f3a372640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 0x7f5f341a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 -- 172.21.15.64:0/1126261317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f3410b1a0 msgr2=0x7f5f3419e3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f3410b1a0 0x7f5f3419e3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 -- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 msgr2=0x7f5f341a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 0x7f5f341a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.577+0000 7f5f39b71640 1 -- 172.21.15.64:0/1126261317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341a5a80 con 0x7f5f34108830 2024-05-09T09:25:21.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.578+0000 7f5f39b71640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 0x7f5f3419de60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000e9f0 tx=0x7f5f3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.578+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000cd80 con 0x7f5f34108830 2024-05-09T09:25:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.578+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f300108e0 con 0x7f5f34108830 2024-05-09T09:25:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.579+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300188c0 con 0x7f5f34108830 2024-05-09T09:25:21.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.580+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341a5d70 con 0x7f5f34108830 2024-05-09T09:25:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.580+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a6180 con 0x7f5f34108830 2024-05-09T09:25:21.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.581+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f341a6470 con 0x7f5f34108830 2024-05-09T09:25:21.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.583+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f3001f020 con 0x7f5f34108830 2024-05-09T09:25:21.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.584+0000 7f5f2affd640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f200a2ce0 0x7f5f200a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.584+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5f300cf6d0 con 0x7f5f34108830 2024-05-09T09:25:21.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.585+0000 7f5f39370640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f200a2ce0 0x7f5f200a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:21.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.585+0000 7f5f39370640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f200a2ce0 0x7f5f200a51a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f3419f370 tx=0x7f5f2400b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:21.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.586+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5f3009c2c0 con 0x7f5f34108830 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:transaction type: 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:scaling factor: 32 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:query mode: simple 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:number of clients: 100 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:number of threads: 4 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:duration: 900 s 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:number of transactions actually processed: 765352 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:latency average = 117.559 ms 2024-05-09T09:25:21.592 INFO:teuthology.orchestra.run.smithi064.stdout:latency stddev = 158.511 ms 2024-05-09T09:25:21.593 INFO:teuthology.orchestra.run.smithi064.stdout:tps = 850.164575 (including connections establishing) 2024-05-09T09:25:21.593 INFO:teuthology.orchestra.run.smithi064.stdout:tps = 850.176022 (excluding connections establishing) 2024-05-09T09:25:21.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-09T09:25:21.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.978+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f341a6820 con 0x7f5f34108830 2024-05-09T09:25:21.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.981+0000 7f5f2affd640 1 -- 172.21.15.64:0/1126261317 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11888 (secure 0 0 0) 0x7f5f3009bbc0 con 0x7f5f34108830 2024-05-09T09:25:21.981 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:21.981 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:25:00.568509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:25:21.982 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 53 2024-05-09T09:25:21.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.989+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f200a2ce0 msgr2=0x7f5f200a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.989+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f200a2ce0 0x7f5f200a51a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f3419f370 tx=0x7f5f2400b370 comp rx=0 tx=0).stop 2024-05-09T09:25:21.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.989+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 msgr2=0x7f5f3419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:21.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.989+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 0x7f5f3419de60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000e9f0 tx=0x7f5f3000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.990+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 shutdown_connections 2024-05-09T09:25:21.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.990+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f5f200a2ce0 0x7f5f200a51a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.991+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f3410db50 0x7f5f341a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.991+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f3410b1a0 0x7f5f3419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.991+0000 7f5f3bdd5640 1 --2- 172.21.15.64:0/1126261317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f34108830 0x7f5f3419de60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:21.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.991+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 >> 172.21.15.64:0/1126261317 conn(0x7f5f3406e5c0 msgr2=0x7f5f3410b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:21.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.993+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 shutdown_connections 2024-05-09T09:25:21.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:21.993+0000 7f5f3bdd5640 1 -- 172.21.15.64:0/1126261317 wait complete. 2024-05-09T09:25:22.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:25:22.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 -- 172.21.15.64:0/669238843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010db70 msgr2=0x7fc85010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 --2- 172.21.15.64:0/669238843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010db70 0x7fc85010e020 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc84801acb0 tx=0x7fc84803b920 comp rx=0 tx=0).stop 2024-05-09T09:25:22.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 -- 172.21.15.64:0/669238843 shutdown_connections 2024-05-09T09:25:22.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 --2- 172.21.15.64:0/669238843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010db70 0x7fc85010e020 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 --2- 172.21.15.64:0/669238843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010b1a0 0x7fc85010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 --2- 172.21.15.64:0/669238843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 0x7fc85010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.228+0000 7fc85574f640 1 -- 172.21.15.64:0/669238843 >> 172.21.15.64:0/669238843 conn(0x7fc85006e6c0 msgr2=0x7fc850070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.234+0000 7fc85574f640 1 -- 172.21.15.64:0/669238843 shutdown_connections 2024-05-09T09:25:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.236+0000 7fc85574f640 1 -- 172.21.15.64:0/669238843 wait complete. 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.237+0000 7fc85574f640 1 Processor -- start 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 -- start start 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 0x7fc85019e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010b1a0 0x7fc85019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010db70 0x7fc8501a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111a80 con 0x7fc850108830 2024-05-09T09:25:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111900 con 0x7fc85010b1a0 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.238+0000 7fc85574f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111c00 con 0x7fc85010db70 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010db70 0x7fc8501a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010b1a0 0x7fc85019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010db70 0x7fc8501a5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55072/0 (socket says 172.21.15.64:55072) 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84f7fe640 1 -- 172.21.15.64:0/3196512175 learned_addr learned my addr 172.21.15.64:0/3196512175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84effd640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 0x7fc85019e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84e7fc640 1 -- 172.21.15.64:0/3196512175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010db70 msgr2=0x7fc8501a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010db70 0x7fc8501a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84e7fc640 1 -- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 msgr2=0x7fc85019e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 0x7fc85019e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84e7fc640 1 -- 172.21.15.64:0/3196512175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8501a5e00 con 0x7fc85010b1a0 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.239+0000 7fc84effd640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 0x7fc85019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.240+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010b1a0 0x7fc85019e720 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc84000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.243+0000 7fc837fff640 1 -- 172.21.15.64:0/3196512175 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840015020 con 0x7fc85010b1a0 2024-05-09T09:25:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.243+0000 7fc837fff640 1 -- 172.21.15.64:0/3196512175 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc840011ce0 con 0x7fc85010b1a0 2024-05-09T09:25:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.243+0000 7fc837fff640 1 -- 172.21.15.64:0/3196512175 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840010d50 con 0x7fc85010b1a0 2024-05-09T09:25:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.243+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8501a60f0 con 0x7fc85010b1a0 2024-05-09T09:25:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.244+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8501a6670 con 0x7fc85010b1a0 2024-05-09T09:25:22.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.249+0000 7fc858459740 1 Processor -- start 2024-05-09T09:25:22.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.251+0000 7fc837fff640 1 -- 172.21.15.64:0/3196512175 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc840018020 con 0x7fc85010b1a0 2024-05-09T09:25:22.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.252+0000 7fc837fff640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8380a2d90 0x7fc8380a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.253+0000 7fc837fff640 1 -- 172.21.15.64:0/3196512175 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8400ce990 con 0x7fc85010b1a0 2024-05-09T09:25:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.253+0000 7fc858459740 1 -- start start 2024-05-09T09:25:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc858459740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dcdf40 0x560f52dce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc858459740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dce930 0x560f52dd3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc858459740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8380a2d90 0x7fc8380a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54384/0 (socket says 172.21.15.64:54384) 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc858459740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f52d32c40 con 0x560f52dd3aa0 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc858459740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560f52cda580 con 0x560f52dce930 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc858459740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560f52d2ec00 con 0x560f52dcdf40 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc84f7fe640 1 -- 172.21.15.64:0/573528145 learned_addr learned my addr 172.21.15.64:0/573528145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.254+0000 7fc84effd640 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dcdf40 0x560f52dce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.255+0000 7fc84effd640 1 -- 172.21.15.64:0/573528145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dce930 msgr2=0x560f52dd3430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.255+0000 7fc84effd640 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dce930 0x560f52dd3430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.255+0000 7fc84effd640 1 -- 172.21.15.64:0/573528145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 msgr2=0x560f52dd5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.255+0000 7fc84effd640 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.255+0000 7fc84effd640 1 -- 172.21.15.64:0/573528145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f52a463f0 con 0x560f52dcdf40 2024-05-09T09:25:22.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.255+0000 7fc84effd640 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dcdf40 0x560f52dce360 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc84400a940 tx=0x7fc84400ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f742eb2cbaf26e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.258+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8380a2d90 0x7fc8380a5250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc8480406c0 tx=0x7fc848040500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.258+0000 7fc8357fa640 1 -- 172.21.15.64:0/573528145 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844015070 con 0x560f52dcdf40 2024-05-09T09:25:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.258+0000 7fc8357fa640 1 -- 172.21.15.64:0/573528145 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc84400cc00 con 0x560f52dcdf40 2024-05-09T09:25:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.258+0000 7fc8357fa640 1 -- 172.21.15.64:0/573528145 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440105b0 con 0x560f52dcdf40 2024-05-09T09:25:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.261+0000 7fc858459740 1 -- 172.21.15.64:0/573528145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dcdf40 msgr2=0x560f52dce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.261+0000 7fc858459740 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dcdf40 0x560f52dce360 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc84400a940 tx=0x7fc84400ae00 comp rx=0 tx=0).stop 2024-05-09T09:25:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.267+0000 7fc858459740 1 -- 172.21.15.64:0/573528145 shutdown_connections 2024-05-09T09:25:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.267+0000 7fc858459740 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.267+0000 7fc858459740 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dce930 0x560f52dd3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.267+0000 7fc858459740 1 --2- 172.21.15.64:0/573528145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dcdf40 0x560f52dce360 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.267+0000 7fc858459740 1 -- 172.21.15.64:0/573528145 >> 172.21.15.64:0/573528145 conn(0x560f52dc95b0 msgr2=0x560f52dc99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.273+0000 7fc858459740 1 -- 172.21.15.64:0/573528145 shutdown_connections 2024-05-09T09:25:22.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.274+0000 7fc858459740 1 -- 172.21.15.64:0/573528145 wait complete. 2024-05-09T09:25:22.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.274+0000 7fc858459740 1 -- start start 2024-05-09T09:25:22.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.274+0000 7fc858459740 1 Processor -- start 2024-05-09T09:25:22.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dd8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52dd89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52ddf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f52dd6a50 con 0x560f52dd3aa0 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560f52cedc40 con 0x560f52dcdf40 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560f52dd6bd0 con 0x560f52dce930 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc858459740 10 client.0 _ll_register_callbacks cb 0x560f52a45be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52ddf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52ddf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54388/0 (socket says 172.21.15.64:54388) 2024-05-09T09:25:22.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.275+0000 7fc84f7fe640 1 -- 172.21.15.64:0/2747563795 learned_addr learned my addr 172.21.15.64:0/2747563795 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.276+0000 7fc84effd640 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dd8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.276+0000 7fc84f7fe640 1 -- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 msgr2=0x560f52dd89d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:25:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.276+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52dd89d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.276+0000 7fc84f7fe640 1 -- 172.21.15.64:0/2747563795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 msgr2=0x560f52dd8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.276+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dd8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.276+0000 7fc84f7fe640 1 -- 172.21.15.64:0/2747563795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f52b107d0 con 0x560f52dd3aa0 2024-05-09T09:25:22.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.277+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52ddf9a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc848006010 tx=0x7fc848005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1725a514f65cf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.279+0000 7fc832ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.279+0000 7fc832ff5640 1 -- 172.21.15.64:0/2747563795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84805f070 con 0x560f52dd3aa0 2024-05-09T09:25:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.279+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84400cc00 con 0x560f52dd3aa0 2024-05-09T09:25:22.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.279+0000 7fc832ff5640 1 -- 172.21.15.64:0/2747563795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc848019040 con 0x560f52dd3aa0 2024-05-09T09:25:22.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.282+0000 7fc832ff5640 1 -- 172.21.15.64:0/2747563795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc848045040 con 0x560f52dd3aa0 2024-05-09T09:25:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.284+0000 7fc858459740 10 client.19299 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:22.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.284+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560f52de2db0 con 0x560f52dd3aa0 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc832ff5640 1 -- 172.21.15.64:0/2747563795 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc84400cc00 con 0x560f52dd3aa0 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc832ff5640 1 -- 172.21.15.64:0/2747563795 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7fc848064780 con 0x560f52dd3aa0 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc858459740 10 client.19299 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc858459740 10 client.19299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc858459740 20 client.19299 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc858459740 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52de4970 0x560f52de6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc858459740 4 client.19299 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.288+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f52d40370 con 0x560f52de4970 2024-05-09T09:25:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.289+0000 7fc84effd640 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52de4970 0x560f52de6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.294+0000 7fc84effd640 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52de4970 0x560f52de6e30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b33a3c3e9bc58dd server_cookie=565bf4ae04cb237a in_seq=0 out_seq=0 2024-05-09T09:25:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.294+0000 7fc832ff5640 10 client.19299 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.295+0000 7fc832ff5640 1 -- 172.21.15.64:0/2747563795 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f52d40370 con 0x560f52de4970 2024-05-09T09:25:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.295+0000 7fc832ff5640 10 client.19299 handle_command_reply: tid=0 2024-05-09T09:25:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.295+0000 7fc858459740 1 client.19299 shutdown 2024-05-09T09:25:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.295+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 msgr2=0x560f52ddf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.295+0000 7fc858459740 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52ddf9a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc848006010 tx=0x7fc848005ef0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.300+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 shutdown_connections 2024-05-09T09:25:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.300+0000 7fc858459740 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52de4970 0x560f52de6e30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.300+0000 7fc858459740 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52ddf9a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.300+0000 7fc858459740 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52dd89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.300+0000 7fc858459740 1 --2- 172.21.15.64:0/2747563795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dd8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.300+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 >> 172.21.15.64:0/2747563795 conn(0x560f52dc95b0 msgr2=0x560f52dc87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.302+0000 7fc832ff5640 0 client.19299 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:22.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.302+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 shutdown_connections 2024-05-09T09:25:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.303+0000 7fc858459740 1 -- 172.21.15.64:0/2747563795 wait complete. 2024-05-09T09:25:22.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.304+0000 7fc858459740 20 client.19299 trim_cache size 0 max 16384 2024-05-09T09:25:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.331+0000 7fc858459740 1 Processor -- start 2024-05-09T09:25:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.331+0000 7fc858459740 1 -- start start 2024-05-09T09:25:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.331+0000 7fc858459740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dcfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc858459740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc858459740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc84effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dcfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc84effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dcfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47306/0 (socket says 172.21.15.64:47306) 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc84f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54390/0 (socket says 172.21.15.64:54390) 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc84effd640 1 -- 172.21.15.64:0/1132884926 learned_addr learned my addr 172.21.15.64:0/1132884926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c012660 con 0x560f52dd3aa0 2024-05-09T09:25:22.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c00fe70 con 0x560f52dcdf40 2024-05-09T09:25:22.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.332+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c00dcc0 con 0x560f52dce930 2024-05-09T09:25:22.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.333+0000 7fc84f7fe640 1 -- 172.21.15.64:0/1132884926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 msgr2=0x560f52ddf960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T09:25:22.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.333+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddf960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.333+0000 7fc84f7fe640 1 -- 172.21.15.64:0/1132884926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 msgr2=0x560f52dcfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.333+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dcfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.333+0000 7fc84f7fe640 1 -- 172.21.15.64:0/1132884926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f52dbad20 con 0x560f52dd3aa0 2024-05-09T09:25:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.333+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd0370 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc848064ba0 tx=0x7fc848005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddea59c591b1b95b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.334+0000 7fc834ff9640 1 -- 172.21.15.64:0/1132884926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84805f040 con 0x560f52dd3aa0 2024-05-09T09:25:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.334+0000 7fc834ff9640 1 -- 172.21.15.64:0/1132884926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc848064910 con 0x560f52dd3aa0 2024-05-09T09:25:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.334+0000 7fc834ff9640 1 -- 172.21.15.64:0/1132884926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8480041b0 con 0x560f52dd3aa0 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 msgr2=0x560f52dd0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd0370 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc848064ba0 tx=0x7fc848005c50 comp rx=0 tx=0).stop 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 shutdown_connections 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dd0370 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 --2- 172.21.15.64:0/1132884926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52dcfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 >> 172.21.15.64:0/1132884926 conn(0x560f52dc95b0 msgr2=0x560f52dd8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 shutdown_connections 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 -- 172.21.15.64:0/1132884926 wait complete. 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 -- start start 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.335+0000 7fc858459740 1 Processor -- start 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52ddce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dde370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c00e330 con 0x560f52dd3aa0 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c00f7a0 con 0x560f52dcdf40 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c00ea00 con 0x560f52dce930 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52ddce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84e7fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52ddce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47318/0 (socket says 172.21.15.64:47318) 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 -- 172.21.15.64:0/728858531 learned_addr learned my addr 172.21.15.64:0/728858531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc858459740 10 client.0 _ll_register_callbacks cb 0x7fc84801a8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dde370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 -- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 msgr2=0x560f52ddd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 -- 172.21.15.64:0/728858531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 msgr2=0x560f52dde370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dde370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 -- 172.21.15.64:0/728858531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f52dde960 con 0x560f52dcdf40 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84f7fe640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dde370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc84effd640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52ddce20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc84400a940 tx=0x7fc84400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312c945806cc9f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.336+0000 7fc832ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:25:22.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc832ff5640 1 -- 172.21.15.64:0/728858531 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844015040 con 0x560f52dcdf40 2024-05-09T09:25:22.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc832ff5640 1 -- 172.21.15.64:0/728858531 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8440115f0 con 0x560f52dcdf40 2024-05-09T09:25:22.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc848064910 con 0x560f52dcdf40 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc832ff5640 1 -- 172.21.15.64:0/728858531 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc8440117f0 con 0x560f52dcdf40 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 10 client.28799 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560f52ddfc40 con 0x560f52dcdf40 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc832ff5640 1 -- 172.21.15.64:0/728858531 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc848064910 con 0x560f52dcdf40 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc832ff5640 1 -- 172.21.15.64:0/728858531 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7fc844027790 con 0x560f52dcdf40 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 10 client.28799 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:22.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 10 client.28799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 20 client.28799 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.337+0000 7fc858459740 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52e4b4b0 0x560f52e4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc858459740 4 client.28799 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f52d3a6c0 con 0x560f52e4b4b0 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52e4b4b0 0x560f52e4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc84e7fc640 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52e4b4b0 0x560f52e4d8d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9b6bf17b29f50a0 server_cookie=1a4e765fc141b463 in_seq=0 out_seq=0 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc832ff5640 10 client.28799 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc832ff5640 1 -- 172.21.15.64:0/728858531 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f52d3a6c0 con 0x560f52e4b4b0 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc832ff5640 10 client.28799 handle_command_reply: tid=0 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.338+0000 7fc858459740 1 client.28799 shutdown 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 msgr2=0x560f52ddce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52ddce20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc84400a940 tx=0x7fc84400cf60 comp rx=0 tx=0).stop 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 shutdown_connections 2024-05-09T09:25:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x560f52e4b4b0 0x560f52e4d8d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc832ff5640 0 client.28799 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f52dd3aa0 0x560f52dde370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f52dce930 0x560f52ddd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 --2- 172.21.15.64:0/728858531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f52dcdf40 0x560f52ddce20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 >> 172.21.15.64:0/728858531 conn(0x560f52dc95b0 msgr2=0x560f52dd83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 shutdown_connections 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.339+0000 7fc858459740 1 -- 172.21.15.64:0/728858531 wait complete. 2024-05-09T09:25:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.340+0000 7fc858459740 20 client.28799 trim_cache size 0 max 16384 2024-05-09T09:25:22.344 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8380a2d90 msgr2=0x7fc8380a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8380a2d90 0x7fc8380a5250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc8480406c0 tx=0x7fc848040500 comp rx=0 tx=0).stop 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010b1a0 msgr2=0x7fc85019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010b1a0 0x7fc85019e720 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc84000b870 comp rx=0 tx=0).stop 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 shutdown_connections 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc8380a2d90 0x7fc8380a5250 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc85010db70 0x7fc8501a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85010b1a0 0x7fc85019e720 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 --2- 172.21.15.64:0/3196512175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc850108830 0x7fc85019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 >> 172.21.15.64:0/3196512175 conn(0x7fc85006e6c0 msgr2=0x7fc850109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.341+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 shutdown_connections 2024-05-09T09:25:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.342+0000 7fc85574f640 1 -- 172.21.15.64:0/3196512175 wait complete. 2024-05-09T09:25:22.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:25:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2781236477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 msgr2=0x7f1a0c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2781236477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c1070b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009900 tx=0x7f19fc02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2781236477 shutdown_connections 2024-05-09T09:25:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2781236477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a0c1075f0 0x7f1a0c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2781236477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c1070b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2781236477 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 0x7f1a0c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2781236477 >> 172.21.15.64:0/2781236477 conn(0x7f1a0c0fbce0 msgr2=0x7f1a0c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2781236477 shutdown_connections 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.478+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2781236477 wait complete. 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 Processor -- start 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 -- start start 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 0x7f1a0c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a0c1075f0 0x7f1a0c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c10d3a0 con 0x7f1a0c104cc0 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c10d220 con 0x7f1a0c102390 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a13fa5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c10d520 con 0x7f1a0c1075f0 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54430/0 (socket says 172.21.15.64:54430) 2024-05-09T09:25:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 -- 172.21.15.64:0/2862342644 learned_addr learned my addr 172.21.15.64:0/2862342644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11d41640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 0x7f1a0c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a12542640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a0c1075f0 0x7f1a0c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 -- 172.21.15.64:0/2862342644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a0c1075f0 msgr2=0x7f1a0c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a0c1075f0 0x7f1a0c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 -- 172.21.15.64:0/2862342644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 msgr2=0x7f1a0c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 0x7f1a0c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.479+0000 7f1a11540640 1 -- 172.21.15.64:0/2862342644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c19f710 con 0x7f1a0c104cc0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a11d41640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 0x7f1a0c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a11540640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c198080 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0314e0 tx=0x7f19fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a02ffd640 1 -- 172.21.15.64:0/2862342644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc03f030 con 0x7f1a0c104cc0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a02ffd640 1 -- 172.21.15.64:0/2862342644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19fc003aa0 con 0x7f1a0c104cc0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c19f9a0 con 0x7f1a0c104cc0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a02ffd640 1 -- 172.21.15.64:0/2862342644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc035070 con 0x7f1a0c104cc0 2024-05-09T09:25:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.480+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c19fe90 con 0x7f1a0c104cc0 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.482+0000 7f1a02ffd640 1 -- 172.21.15.64:0/2862342644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19fc042030 con 0x7f1a0c104cc0 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 Processor -- start 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 -- start start 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabde1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabde6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabde8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fabd45c40 con 0x555fabde0f40 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555fabced580 con 0x555fabde1930 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555fabd41c00 con 0x555fabde6aa0 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a12542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabde1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a11540640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabde8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a12542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabde1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54446/0 (socket says 172.21.15.64:54446) 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a12542640 1 -- 172.21.15.64:0/2170881508 learned_addr learned my addr 172.21.15.64:0/2170881508 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a02ffd640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f19f80a2b90 0x7f19f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.483+0000 7f1a11d41640 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabde6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 -- 172.21.15.64:0/2170881508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 msgr2=0x555fabde8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabde8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 -- 172.21.15.64:0/2170881508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 msgr2=0x555fabde6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabde6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 -- 172.21.15.64:0/2170881508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555faba593f0 con 0x555fabde0f40 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a02ffd640 1 -- 172.21.15.64:0/2862342644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f19fc0efac0 con 0x7f1a0c104cc0 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f19f80a2b90 0x7f19f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabde1360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1a04011590 tx=0x7f1a04011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155972b2f18f673 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f19f7fff640 1 -- 172.21.15.64:0/2170881508 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04017050 con 0x555fabde0f40 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f1a12542640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f19f80a2b90 0x7f19f80a5050 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1a04022000 tx=0x7f1a04023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f19f7fff640 1 -- 172.21.15.64:0/2170881508 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0400fdb0 con 0x555fabde0f40 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.484+0000 7f19f7fff640 1 -- 172.21.15.64:0/2170881508 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0401bb40 con 0x555fabde0f40 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2170881508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 msgr2=0x555fabde1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabde1360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1a04011590 tx=0x7f1a04011a50 comp rx=0 tx=0).stop 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2170881508 shutdown_connections 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabde8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabde6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2170881508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabde1360 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2170881508 >> 172.21.15.64:0/2170881508 conn(0x555fabddc5b0 msgr2=0x555fabddc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2170881508 shutdown_connections 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2170881508 wait complete. 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 -- start start 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.485+0000 7f1a16d8e740 1 Processor -- start 2024-05-09T09:25:22.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabdeb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabdeb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabdf2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fabde9a50 con 0x555fabde0f40 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555fabd00c40 con 0x555fabde1930 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555fabde9bd0 con 0x555fabde6aa0 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a16d8e740 10 client.0 _ll_register_callbacks cb 0x555faba58be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabdeb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabdeb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54460/0 (socket says 172.21.15.64:54460) 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 -- 172.21.15.64:0/2121544320 learned_addr learned my addr 172.21.15.64:0/2121544320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11540640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabdeb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a12542640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabdf2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 -- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 msgr2=0x555fabdf2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabdf2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 -- 172.21.15.64:0/2121544320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 msgr2=0x555fabdeb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabdeb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11d41640 1 -- 172.21.15.64:0/2121544320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fabb237d0 con 0x555fabde0f40 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.486+0000 7f1a11540640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabdeb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f1a11d41640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabdeb3c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f1a08012d20 tx=0x7f1a0800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ee059aafad5ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f19f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f19f57fa640 1 -- 172.21.15.64:0/2121544320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08012110 con 0x555fabde0f40 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f19f57fa640 1 -- 172.21.15.64:0/2121544320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1a08004050 con 0x555fabde0f40 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0400fdb0 con 0x555fabde0f40 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f19f57fa640 1 -- 172.21.15.64:0/2121544320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1a08025050 con 0x555fabde0f40 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f1a16d8e740 10 client.19317 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:22.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x555fabdf5d80 con 0x555fabde0f40 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f19f57fa640 1 -- 172.21.15.64:0/2121544320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0400fdb0 con 0x555fabde0f40 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f19f57fa640 1 -- 172.21.15.64:0/2121544320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f1a08003c30 con 0x555fabde0f40 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f1a16d8e740 10 client.19317 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.487+0000 7f1a16d8e740 10 client.19317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f1a16d8e740 20 client.19317 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabdf74b0 0x555fabdf9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f1a16d8e740 4 client.19317 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fabd53370 con 0x555fabdf74b0 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f1a11540640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabdf74b0 0x555fabdf9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f1a11540640 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabdf74b0 0x555fabdf9950 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e117ffda58e03c0a server_cookie=61f7ade20f949610 in_seq=0 out_seq=0 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.488+0000 7f19f57fa640 10 client.19317 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f19f57fa640 1 -- 172.21.15.64:0/2121544320 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fabd53370 con 0x555fabdf74b0 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f19f57fa640 10 client.19317 handle_command_reply: tid=0 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 client.19317 shutdown 2024-05-09T09:25:22.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 msgr2=0x555fabdeb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabdeb3c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f1a08012d20 tx=0x7f1a0800f790 comp rx=0 tx=0).stop 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 shutdown_connections 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabdf2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde1930 0x555fabdeb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabdf74b0 0x555fabdf9950 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/2121544320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde0f40 0x555fabdeb3c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 >> 172.21.15.64:0/2121544320 conn(0x555fabddc5b0 msgr2=0x555fabddb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f19f57fa640 0 client.19317 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 shutdown_connections 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 1 -- 172.21.15.64:0/2121544320 wait complete. 2024-05-09T09:25:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.489+0000 7f1a16d8e740 20 client.19317 trim_cache size 0 max 16384 2024-05-09T09:25:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.504+0000 7f1a16d8e740 1 Processor -- start 2024-05-09T09:25:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.504+0000 7f1a16d8e740 1 -- start start 2024-05-09T09:25:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde0f40 0x555fabddfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabddff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde6aa0 0x555fabdf2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1acd80 con 0x555fabde1930 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1aa590 con 0x555fabde6aa0 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1a83e0 con 0x555fabde0f40 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabddff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11d41640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde0f40 0x555fabddfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabddff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54470/0 (socket says 172.21.15.64:54470) 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 -- 172.21.15.64:0/3898922776 learned_addr learned my addr 172.21.15.64:0/3898922776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a12542640 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde6aa0 0x555fabdf2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 -- 172.21.15.64:0/3898922776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde0f40 msgr2=0x555fabddfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde0f40 0x555fabddfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 -- 172.21.15.64:0/3898922776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde6aa0 msgr2=0x555fabdf2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde6aa0 0x555fabdf2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 -- 172.21.15.64:0/3898922776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fabdcdd20 con 0x555fabde1930 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11d41640 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde0f40 0x555fabddfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f1a11540640 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabddff80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ed230 tx=0x7f19fc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2a8770519a69a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f19f77fe640 1 -- 172.21.15.64:0/3898922776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc049040 con 0x555fabde1930 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f19f77fe640 1 -- 172.21.15.64:0/3898922776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19fc0bc040 con 0x555fabde1930 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.505+0000 7f19f77fe640 1 -- 172.21.15.64:0/3898922776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0b86b0 con 0x555fabde1930 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 -- 172.21.15.64:0/3898922776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 msgr2=0x555fabddff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabddff80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ed230 tx=0x7f19fc0dc040 comp rx=0 tx=0).stop 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 -- 172.21.15.64:0/3898922776 shutdown_connections 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde6aa0 0x555fabdf2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabddff80 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/3898922776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde0f40 0x555fabddfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 -- 172.21.15.64:0/3898922776 >> 172.21.15.64:0/3898922776 conn(0x555fabddc5b0 msgr2=0x555fabddf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 -- 172.21.15.64:0/3898922776 shutdown_connections 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 -- 172.21.15.64:0/3898922776 wait complete. 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.506+0000 7f1a16d8e740 1 -- start start 2024-05-09T09:25:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 Processor -- start 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde0f40 0x555fabdef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabdefc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabe5bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1a8a50 con 0x555fabde1930 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1a9ec0 con 0x555fabde0f40 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c1a9120 con 0x555fabde6aa0 2024-05-09T09:25:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a16d8e740 10 client.0 _ll_register_callbacks cb 0x7f1a08009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabdefc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabdefc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54482/0 (socket says 172.21.15.64:54482) 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11540640 1 -- 172.21.15.64:0/179950778 learned_addr learned my addr 172.21.15.64:0/179950778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11d41640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde0f40 0x555fabdef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a12542640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabe5bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11d41640 1 -- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 msgr2=0x555fabe5bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11d41640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabe5bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11d41640 1 -- 172.21.15.64:0/179950778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 msgr2=0x555fabdefc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11d41640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabdefc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11d41640 1 -- 172.21.15.64:0/179950778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fabdf02c0 con 0x555fabde0f40 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.507+0000 7f1a11540640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabdefc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a11d41640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde0f40 0x555fabdef710 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1a08010500 tx=0x7f1a0800fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=303c11a02e62ad06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f19f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f19f57fa640 1 -- 172.21.15.64:0/179950778 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08022040 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19fc0bc040 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f19f57fa640 1 -- 172.21.15.64:0/179950778 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1a080050b0 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f19f57fa640 1 -- 172.21.15.64:0/179950778 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1a08038050 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a16d8e740 10 client.28817 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x555fabe5d170 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f19f57fa640 1 -- 172.21.15.64:0/179950778 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19fc0bc040 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f19f57fa640 1 -- 172.21.15.64:0/179950778 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f1a080035f0 con 0x555fabde0f40 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a16d8e740 10 client.28817 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a16d8e740 10 client.28817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.508+0000 7f1a16d8e740 20 client.28817 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabe5dbd0 0x555fabe5fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a16d8e740 4 client.28817 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fabde2860 con 0x555fabe5dbd0 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a11540640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabe5dbd0 0x555fabe5fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a11540640 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabe5dbd0 0x555fabe5fff0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f155ed09bb266c94 server_cookie=3c25c38035f605a4 in_seq=0 out_seq=0 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f19f57fa640 10 client.28817 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f19f57fa640 1 -- 172.21.15.64:0/179950778 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fabde2860 con 0x555fabe5dbd0 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f19f57fa640 10 client.28817 handle_command_reply: tid=0 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a16d8e740 1 client.28817 shutdown 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde0f40 msgr2=0x555fabdef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.509+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde0f40 0x555fabdef710 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1a08010500 tx=0x7f1a0800fc90 comp rx=0 tx=0).stop 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 shutdown_connections 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fabde6aa0 0x555fabe5bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x555fabe5dbd0 0x555fabe5fff0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fabde1930 0x555fabdefc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 --2- 172.21.15.64:0/179950778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555fabde0f40 0x555fabdef710 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 >> 172.21.15.64:0/179950778 conn(0x555fabddc5b0 msgr2=0x555fabddb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f19f57fa640 0 client.28817 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:22.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 shutdown_connections 2024-05-09T09:25:22.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 1 -- 172.21.15.64:0/179950778 wait complete. 2024-05-09T09:25:22.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a16d8e740 20 client.28817 trim_cache size 0 max 16384 2024-05-09T09:25:22.516 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f19f80a2b90 msgr2=0x7f19f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f19f80a2b90 0x7f19f80a5050 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1a04022000 tx=0x7f1a04023000 comp rx=0 tx=0).stop 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 msgr2=0x7f1a0c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c198080 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0314e0 tx=0x7f19fc004270 comp rx=0 tx=0).stop 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 shutdown_connections 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f19f80a2b90 0x7f19f80a5050 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a0c1075f0 0x7f1a0c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a0c104cc0 0x7f1a0c198080 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.510+0000 7f1a13fa5640 1 --2- 172.21.15.64:0/2862342644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a0c102390 0x7f1a0c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.511+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 >> 172.21.15.64:0/2862342644 conn(0x7f1a0c0fbce0 msgr2=0x7f1a0c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.511+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 shutdown_connections 2024-05-09T09:25:22.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.511+0000 7f1a13fa5640 1 -- 172.21.15.64:0/2862342644 wait complete. 2024-05-09T09:25:22.523 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:25:22.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.642+0000 7f3b983b7640 1 -- 172.21.15.64:0/2021494738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90102a70 msgr2=0x7f3b90102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.642+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2021494738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90102a70 0x7f3b90102ef0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b8002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- 172.21.15.64:0/2021494738 shutdown_connections 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2021494738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 0x7f3b90109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2021494738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90102a70 0x7f3b90102ef0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2021494738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b90101880 0x7f3b90101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- 172.21.15.64:0/2021494738 >> 172.21.15.64:0/2021494738 conn(0x7f3b900fce10 msgr2=0x7f3b900ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- 172.21.15.64:0/2021494738 shutdown_connections 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- 172.21.15.64:0/2021494738 wait complete. 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 Processor -- start 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- start start 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 0x7f3b90197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b90102a70 0x7f3b90198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 0x7f3b9019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9010d3b0 con 0x7f3b90101880 2024-05-09T09:25:22.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9010d230 con 0x7f3b90103430 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.643+0000 7f3b983b7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9010d530 con 0x7f3b90102a70 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 0x7f3b90197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 0x7f3b90197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54510/0 (socket says 172.21.15.64:54510) 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 -- 172.21.15.64:0/2766865929 learned_addr learned my addr 172.21.15.64:0/2766865929 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b95952640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b90102a70 0x7f3b90198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96954640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 0x7f3b9019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 -- 172.21.15.64:0/2766865929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b90102a70 msgr2=0x7f3b90198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b90102a70 0x7f3b90198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 -- 172.21.15.64:0/2766865929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 msgr2=0x7f3b9019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 0x7f3b9019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 -- 172.21.15.64:0/2766865929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9019f700 con 0x7f3b90101880 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96954640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 0x7f3b9019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b96153640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 0x7f3b90197ae0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e9c0 tx=0x7f3b8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b877fe640 1 -- 172.21.15.64:0/2766865929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00cd80 con 0x7f3b90101880 2024-05-09T09:25:22.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b877fe640 1 -- 172.21.15.64:0/2766865929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b8c0035f0 con 0x7f3b90101880 2024-05-09T09:25:22.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9019f9f0 con 0x7f3b90101880 2024-05-09T09:25:22.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b877fe640 1 -- 172.21.15.64:0/2766865929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c010690 con 0x7f3b90101880 2024-05-09T09:25:22.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.644+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9019ff10 con 0x7f3b90101880 2024-05-09T09:25:22.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.647+0000 7f3b877fe640 1 -- 172.21.15.64:0/2766865929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b8c0037a0 con 0x7f3b90101880 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.647+0000 7f3b9b1a4740 1 Processor -- start 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 -- start start 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef4f40 0x5645b0ef5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0efa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0efcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5645b0e59c40 con 0x5645b0efaaa0 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5645b0e01580 con 0x5645b0ef4f40 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5645b0e55c00 con 0x5645b0ef5930 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0efcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b96954640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef4f40 0x5645b0ef5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0efcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54516/0 (socket says 172.21.15.64:54516) 2024-05-09T09:25:22.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 -- 172.21.15.64:0/1515323553 learned_addr learned my addr 172.21.15.64:0/1515323553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b96153640 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0efa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 -- 172.21.15.64:0/1515323553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 msgr2=0x5645b0efa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0efa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 -- 172.21.15.64:0/1515323553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef4f40 msgr2=0x5645b0ef5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef4f40 0x5645b0ef5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b95952640 1 -- 172.21.15.64:0/1515323553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0b6d3f0 con 0x5645b0efaaa0 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.648+0000 7f3b96954640 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef4f40 0x5645b0ef5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b877fe640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b7c0a2c10 0x7f3b7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b95952640 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0efcf30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b80005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95745dfbc0d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b96954640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b7c0a2c10 0x7f3b7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b877fe640 1 -- 172.21.15.64:0/2766865929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3b8c0cf920 con 0x7f3b90101880 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b84bfe640 1 -- 172.21.15.64:0/1515323553 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80035030 con 0x5645b0efaaa0 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b84bfe640 1 -- 172.21.15.64:0/1515323553 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80040070 con 0x5645b0efaaa0 2024-05-09T09:25:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.649+0000 7f3b84bfe640 1 -- 172.21.15.64:0/1515323553 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80003a10 con 0x5645b0efaaa0 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/1515323553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 msgr2=0x5645b0efcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b96954640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b7c0a2c10 0x7f3b7c0a50d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x5645b0ef6190 tx=0x7f3b88009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0efcf30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b80005c00 comp rx=0 tx=0).stop 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/1515323553 shutdown_connections 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0efcf30 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0efa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/1515323553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef4f40 0x5645b0ef5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/1515323553 >> 172.21.15.64:0/1515323553 conn(0x5645b0ef05b0 msgr2=0x5645b0ef09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/1515323553 shutdown_connections 2024-05-09T09:25:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/1515323553 wait complete. 2024-05-09T09:25:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.650+0000 7f3b9b1a4740 1 -- start start 2024-05-09T09:25:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 Processor -- start 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0eff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef5930 0x5645b0eff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0f06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5645b0efd9f0 con 0x5645b0ef5930 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5645b0e14c40 con 0x5645b0efaaa0 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5645b0efdb70 con 0x5645b0ef4f40 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b9b1a4740 10 client.0 _ll_register_callbacks cb 0x5645b0b6cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b95952640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef5930 0x5645b0eff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96153640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0eff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b95952640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef5930 0x5645b0eff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54518/0 (socket says 172.21.15.64:54518) 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b95952640 1 -- 172.21.15.64:0/4162326446 learned_addr learned my addr 172.21.15.64:0/4162326446 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96954640 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0f06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96153640 1 -- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 msgr2=0x5645b0f06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96153640 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0f06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96153640 1 -- 172.21.15.64:0/4162326446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef5930 msgr2=0x5645b0eff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96153640 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef5930 0x5645b0eff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.651+0000 7f3b96153640 1 -- 172.21.15.64:0/4162326446 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b0c377d0 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b96153640 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0eff2e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0cd0e0 tx=0x7f3b8c097970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ddca8e2ec6e7a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b79ffb640 1 -- 172.21.15.64:0/4162326446 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c09f060 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b80040070 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b79ffb640 1 -- 172.21.15.64:0/4162326446 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3b8c09b440 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b79ffb640 1 -- 172.21.15.64:0/4162326446 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c09b930 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b9b1a4740 10 client.28969 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5645b0f09c90 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b79ffb640 1 -- 172.21.15.64:0/4162326446 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80040070 con 0x5645b0ef4f40 2024-05-09T09:25:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b79ffb640 1 -- 172.21.15.64:0/4162326446 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f3b8c09acb0 con 0x5645b0ef4f40 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b9b1a4740 10 client.28969 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.652+0000 7f3b9b1a4740 10 client.28969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b9b1a4740 20 client.28969 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f0b2f0 0x5645b0f0d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b9b1a4740 4 client.28969 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b0e67370 con 0x5645b0f0b2f0 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b95952640 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f0b2f0 0x5645b0f0d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b95952640 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f0b2f0 0x5645b0f0d7b0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41d66abfc0c6740 server_cookie=ea58baefa5525a3f in_seq=0 out_seq=0 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.653+0000 7f3b79ffb640 10 client.28969 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b79ffb640 1 -- 172.21.15.64:0/4162326446 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645b0e67370 con 0x5645b0f0b2f0 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b79ffb640 10 client.28969 handle_command_reply: tid=0 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 client.28969 shutdown 2024-05-09T09:25:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 msgr2=0x5645b0eff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0eff2e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0cd0e0 tx=0x7f3b8c097970 comp rx=0 tx=0).stop 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 shutdown_connections 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f0b2f0 0x5645b0f0d7b0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0f06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef5930 0x5645b0eff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/4162326446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0eff2e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b79ffb640 0 client.28969 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.654+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 >> 172.21.15.64:0/4162326446 conn(0x5645b0ef05b0 msgr2=0x5645b0eef7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.655+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 shutdown_connections 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.655+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/4162326446 wait complete. 2024-05-09T09:25:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.655+0000 7f3b9b1a4740 20 client.28969 trim_cache size 0 max 16384 2024-05-09T09:25:22.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 Processor -- start 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 -- start start 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef4f40 0x5645b0eff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0f067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0eeef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901acd80 con 0x5645b0ef4f40 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901aa590 con 0x5645b0efaaa0 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901a83e0 con 0x5645b0ef5930 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b96153640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef4f40 0x5645b0eff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b95952640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0f067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b96153640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef4f40 0x5645b0eff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54526/0 (socket says 172.21.15.64:54526) 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b96153640 1 -- 172.21.15.64:0/2145505876 learned_addr learned my addr 172.21.15.64:0/2145505876 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b96954640 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0eeef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b95952640 1 -- 172.21.15.64:0/2145505876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 msgr2=0x5645b0eeef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b95952640 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0eeef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.671+0000 7f3b95952640 1 -- 172.21.15.64:0/2145505876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef4f40 msgr2=0x5645b0eff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.672+0000 7f3b95952640 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef4f40 0x5645b0eff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.672+0000 7f3b95952640 1 -- 172.21.15.64:0/2145505876 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0ef0150 con 0x5645b0ef5930 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.672+0000 7f3b95952640 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0f067f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5645b0f0dff0 tx=0x7f3b80031730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c7951d5aff443d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.672+0000 7f3b7bfff640 1 -- 172.21.15.64:0/2145505876 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80059070 con 0x5645b0ef5930 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.672+0000 7f3b7bfff640 1 -- 172.21.15.64:0/2145505876 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80040070 con 0x5645b0ef5930 2024-05-09T09:25:22.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.672+0000 7f3b7bfff640 1 -- 172.21.15.64:0/2145505876 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80035040 con 0x5645b0ef5930 2024-05-09T09:25:22.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/2145505876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 msgr2=0x5645b0f067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0f067f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5645b0f0dff0 tx=0x7f3b80031730 comp rx=0 tx=0).stop 2024-05-09T09:25:22.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/2145505876 shutdown_connections 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0efaaa0 0x5645b0eeef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef5930 0x5645b0f067f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/2145505876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0ef4f40 0x5645b0eff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/2145505876 >> 172.21.15.64:0/2145505876 conn(0x5645b0ef05b0 msgr2=0x5645b0efeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/2145505876 shutdown_connections 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/2145505876 wait complete. 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 -- start start 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.673+0000 7f3b9b1a4740 1 Processor -- start 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0f6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef5930 0x5645b0f6d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0f74190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901a8a50 con 0x5645b0efaaa0 2024-05-09T09:25:22.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901a9ec0 con 0x5645b0ef5930 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901a9120 con 0x5645b0ef4f40 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b9b1a4740 10 client.0 _ll_register_callbacks cb 0x7f3b8c0d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96153640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0f6cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0f74190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96153640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0f6cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55198/0 (socket says 172.21.15.64:55198) 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96153640 1 -- 172.21.15.64:0/3091304956 learned_addr learned my addr 172.21.15.64:0/3091304956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0f74190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54540/0 (socket says 172.21.15.64:54540) 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b95952640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef5930 0x5645b0f6d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 -- 172.21.15.64:0/3091304956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 msgr2=0x5645b0f6cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0f6cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 -- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef5930 msgr2=0x5645b0f6d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef5930 0x5645b0f6d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96954640 1 -- 172.21.15.64:0/3091304956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b0efa7f0 con 0x5645b0efaaa0 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b95952640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef5930 0x5645b0f6d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.674+0000 7f3b96153640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0f6cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:22.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.675+0000 7f3b96954640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0f74190 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005fd0 tx=0x7f3b88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987f38c22ed457b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:22.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.675+0000 7f3b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.676+0000 7f3b79ffb640 1 -- 172.21.15.64:0/3091304956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8802e060 con 0x5645b0efaaa0 2024-05-09T09:25:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.676+0000 7f3b79ffb640 1 -- 172.21.15.64:0/3091304956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3b88010cb0 con 0x5645b0efaaa0 2024-05-09T09:25:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.676+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b80040070 con 0x5645b0efaaa0 2024-05-09T09:25:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b79ffb640 1 -- 172.21.15.64:0/3091304956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3b88035050 con 0x5645b0efaaa0 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 10 client.19341 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5645b0f759b0 con 0x5645b0efaaa0 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b79ffb640 1 -- 172.21.15.64:0/3091304956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80040070 con 0x5645b0efaaa0 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b79ffb640 1 -- 172.21.15.64:0/3091304956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f3b88028070 con 0x5645b0efaaa0 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 10 client.19341 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 20 client.19341 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f76470 0x5645b0f78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 4 client.19341 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645b0e616c0 con 0x5645b0f76470 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.677+0000 7f3b96153640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f76470 0x5645b0f78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b96153640 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f76470 0x5645b0f78890 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6490a25514e21af8 server_cookie=f908e92e8bea0486 in_seq=0 out_seq=0 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b79ffb640 10 client.19341 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b79ffb640 1 -- 172.21.15.64:0/3091304956 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645b0e616c0 con 0x5645b0f76470 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b79ffb640 10 client.19341 handle_command_reply: tid=0 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 client.19341 shutdown 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 msgr2=0x5645b0f74190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0f74190 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005fd0 tx=0x7f3b88004ab0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 shutdown_connections 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5645b0f76470 0x5645b0f78890 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645b0efaaa0 0x5645b0f74190 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645b0ef5930 0x5645b0f6d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b79ffb640 0 client.19341 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 --2- 172.21.15.64:0/3091304956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645b0ef4f40 0x5645b0f6cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 >> 172.21.15.64:0/3091304956 conn(0x5645b0ef05b0 msgr2=0x5645b0eff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 shutdown_connections 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 1 -- 172.21.15.64:0/3091304956 wait complete. 2024-05-09T09:25:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.678+0000 7f3b9b1a4740 20 client.19341 trim_cache size 0 max 16384 2024-05-09T09:25:22.683 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:25:22.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b7c0a2c10 msgr2=0x7f3b7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b7c0a2c10 0x7f3b7c0a50d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x5645b0ef6190 tx=0x7f3b88009500 comp rx=0 tx=0).stop 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 msgr2=0x7f3b90197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 0x7f3b90197ae0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e9c0 tx=0x7f3b8c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 shutdown_connections 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3b7c0a2c10 0x7f3b7c0a50d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b90103430 0x7f3b9019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b90102a70 0x7f3b90198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 --2- 172.21.15.64:0/2766865929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b90101880 0x7f3b90197ae0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 >> 172.21.15.64:0/2766865929 conn(0x7f3b900fce10 msgr2=0x7f3b90105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 shutdown_connections 2024-05-09T09:25:22.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:22.679+0000 7f3b983b7640 1 -- 172.21.15.64:0/2766865929 wait complete. 2024-05-09T09:25:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:22 smithi179 ceph-mon[34086]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.1 MiB/s wr, 700 op/s 2024-05-09T09:25:22.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:22 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1126261317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 -- 172.21.15.64:0/2995043591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d901018a0 msgr2=0x7f9d90101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 --2- 172.21.15.64:0/2995043591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d901018a0 0x7f9d90101cc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009900 tx=0x7f9d8c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 -- 172.21.15.64:0/2995043591 shutdown_connections 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 --2- 172.21.15.64:0/2995043591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d90103450 0x7f9d90109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 --2- 172.21.15.64:0/2995043591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d90102a90 0x7f9d90102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 --2- 172.21.15.64:0/2995043591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d901018a0 0x7f9d90101cc0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 -- 172.21.15.64:0/2995043591 >> 172.21.15.64:0/2995043591 conn(0x7f9d900fcdf0 msgr2=0x7f9d900ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 -- 172.21.15.64:0/2995043591 shutdown_connections 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.066+0000 7f9d98577640 1 -- 172.21.15.64:0/2995043591 wait complete. 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 Processor -- start 2024-05-09T09:25:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 -- start start 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d901018a0 0x7f9d900755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 0x7f9d90073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d90103450 0x7f9d90074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9010d3a0 con 0x7f9d90102a90 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9010d220 con 0x7f9d901018a0 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d98577640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9010d520 con 0x7f9d90103450 2024-05-09T09:25:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 0x7f9d90073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 0x7f9d90073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54556/0 (socket says 172.21.15.64:54556) 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d95b12640 1 -- 172.21.15.64:0/2216401527 learned_addr learned my addr 172.21.15.64:0/2216401527 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.067+0000 7f9d96313640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d901018a0 0x7f9d900755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d90103450 0x7f9d90074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96313640 1 -- 172.21.15.64:0/2216401527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d90103450 msgr2=0x7f9d90074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96313640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d90103450 0x7f9d90074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96313640 1 -- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 msgr2=0x7f9d90073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96313640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 0x7f9d90073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96313640 1 -- 172.21.15.64:0/2216401527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d900749f0 con 0x7f9d901018a0 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d95b12640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 0x7f9d90073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d96313640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d901018a0 0x7f9d900755e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009230 tx=0x7f9d8c031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d877fe640 1 -- 172.21.15.64:0/2216401527 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c03f030 con 0x7f9d901018a0 2024-05-09T09:25:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d901a1b00 con 0x7f9d901018a0 2024-05-09T09:25:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.068+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d901a2070 con 0x7f9d901018a0 2024-05-09T09:25:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.069+0000 7f9d877fe640 1 -- 172.21.15.64:0/2216401527 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c003aa0 con 0x7f9d901018a0 2024-05-09T09:25:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.069+0000 7f9d877fe640 1 -- 172.21.15.64:0/2216401527 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c049060 con 0x7f9d901018a0 2024-05-09T09:25:23.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:22 smithi139 ceph-mon[33599]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.1 MiB/s wr, 700 op/s 2024-05-09T09:25:23.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:22 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1126261317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.071+0000 7f9d877fe640 1 -- 172.21.15.64:0/2216401527 <== mon.2 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d8c042030 con 0x7f9d901018a0 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.071+0000 7f9d9b367740 1 Processor -- start 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.071+0000 7f9d9b367740 1 -- start start 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef72f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef72fd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7300240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d9b367740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ef725c340 con 0x558ef72f8250 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d9b367740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558ef7204290 con 0x558ef72fddb0 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d9b367740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558ef7258300 con 0x558ef72f8c40 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d96313640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef72fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d96b14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef72f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7300240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7300240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47472/0 (socket says 172.21.15.64:47472) 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d96313640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef72fd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55230/0 (socket says 172.21.15.64:55230) 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 -- 172.21.15.64:0/3383752226 learned_addr learned my addr 172.21.15.64:0/3383752226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d877fe640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2c30 0x7f9d7c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2c30 0x7f9d7c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 -- 172.21.15.64:0/3383752226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 msgr2=0x558ef72fd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef72fd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 -- 172.21.15.64:0/3383752226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 msgr2=0x558ef72f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef72f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 -- 172.21.15.64:0/3383752226 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef72288e0 con 0x558ef72fddb0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d96b14640 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef72f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d877fe640 1 -- 172.21.15.64:0/2216401527 <== mon.2 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c0ef550 con 0x7f9d901018a0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.072+0000 7f9d95b12640 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7300240 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9d80010610 tx=0x7f9d80010ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4a666b95ed18e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.073+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2c30 0x7f9d7c0a50f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x558ef72f94a0 tx=0x7f9d88004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.073+0000 7f9d84bfe640 1 -- 172.21.15.64:0/3383752226 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000e850 con 0x558ef72fddb0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.073+0000 7f9d84bfe640 1 -- 172.21.15.64:0/3383752226 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d80011440 con 0x558ef72fddb0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.073+0000 7f9d84bfe640 1 -- 172.21.15.64:0/3383752226 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000eea0 con 0x558ef72fddb0 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.073+0000 7f9d9b367740 1 -- 172.21.15.64:0/3383752226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 msgr2=0x558ef7300240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.073+0000 7f9d9b367740 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7300240 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9d80010610 tx=0x7f9d80010ad0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- 172.21.15.64:0/3383752226 shutdown_connections 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7300240 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef72fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 --2- 172.21.15.64:0/3383752226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef72f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- 172.21.15.64:0/3383752226 >> 172.21.15.64:0/3383752226 conn(0x558ef72f38c0 msgr2=0x558ef72f3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- 172.21.15.64:0/3383752226 shutdown_connections 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- 172.21.15.64:0/3383752226 wait complete. 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- start start 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 Processor -- start 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8250 0x558ef7302410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 0x558ef7302950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7309990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ef7300d60 con 0x558ef72f8c40 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558ef7217790 con 0x558ef72fddb0 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558ef7300ee0 con 0x558ef72f8250 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.074+0000 7f9d9b367740 10 client.0 _ll_register_callbacks cb 0x558ef7252a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 0x558ef7302950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 0x558ef7302950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54580/0 (socket says 172.21.15.64:54580) 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8250 0x558ef7302410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d95b12640 1 -- 172.21.15.64:0/2589061495 learned_addr learned my addr 172.21.15.64:0/2589061495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7309990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 -- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 msgr2=0x558ef7309990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7309990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 -- 172.21.15.64:0/2589061495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 msgr2=0x558ef7302950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 0x558ef7302950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 -- 172.21.15.64:0/2589061495 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef6f70210 con 0x558ef72f8250 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d95b12640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 0x558ef7302950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d96313640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8250 0x558ef7302410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0ecd10 tx=0x7f9d8c0ef850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c9f2e699adea2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d79ffb640 1 -- 172.21.15.64:0/2589061495 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c035040 con 0x558ef72f8250 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d79ffb640 1 -- 172.21.15.64:0/2589061495 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c0ef2c0 con 0x558ef72f8250 2024-05-09T09:25:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.075+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d80011440 con 0x558ef72f8250 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d79ffb640 1 -- 172.21.15.64:0/2589061495 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c0bbad0 con 0x558ef72f8250 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 10 client.28987 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x558ef703c620 con 0x558ef72f8250 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d79ffb640 1 -- 172.21.15.64:0/2589061495 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d80011440 con 0x558ef72f8250 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d79ffb640 1 -- 172.21.15.64:0/2589061495 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f9d8c04c060 con 0x558ef72f8250 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 10 client.28987 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 10 client.28987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 20 client.28987 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7310310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 4 client.28987 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ef7269a70 con 0x558ef730de80 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.076+0000 7f9d95b12640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7310310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.077+0000 7f9d79ffb640 10 client.28987 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.077+0000 7f9d95b12640 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7310310 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0ff05f62a031df server_cookie=594a703ae6f58e3 in_seq=0 out_seq=0 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d79ffb640 1 -- 172.21.15.64:0/2589061495 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ef7269a70 con 0x558ef730de80 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d79ffb640 10 client.28987 handle_command_reply: tid=0 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 client.28987 shutdown 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8250 msgr2=0x558ef7302410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8250 0x558ef7302410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0ecd10 tx=0x7f9d8c0ef850 comp rx=0 tx=0).stop 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 shutdown_connections 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7310310 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef7309990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8c40 0x558ef7302950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d79ffb640 0 client.28987 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2589061495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8250 0x558ef7302410 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 >> 172.21.15.64:0/2589061495 conn(0x558ef72f38c0 msgr2=0x558ef72f2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 shutdown_connections 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 1 -- 172.21.15.64:0/2589061495 wait complete. 2024-05-09T09:25:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.078+0000 7f9d9b367740 20 client.28987 trim_cache size 0 max 16384 2024-05-09T09:25:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 Processor -- start 2024-05-09T09:25:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 -- start start 2024-05-09T09:25:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef7302710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72f8c40 0x558ef72f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 0x558ef72f2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901aedb0 con 0x558ef72f8250 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901ac5c0 con 0x558ef72f8c40 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d9b367740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901aa410 con 0x558ef72fddb0 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96313640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef7302710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 0x558ef72f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 0x558ef72f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55244/0 (socket says 172.21.15.64:55244) 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 -- 172.21.15.64:0/2292195767 learned_addr learned my addr 172.21.15.64:0/2292195767 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d95b12640 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72f8c40 0x558ef72f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 -- 172.21.15.64:0/2292195767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72f8c40 msgr2=0x558ef72f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72f8c40 0x558ef72f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 -- 172.21.15.64:0/2292195767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 msgr2=0x558ef7302710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef7302710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.096+0000 7f9d96b14640 1 -- 172.21.15.64:0/2292195767 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef72f1f00 con 0x558ef72fddb0 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.097+0000 7f9d96b14640 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 0x558ef72f2170 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9d88013400 tx=0x7f9d88009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cce5fdcceb5ea6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.097+0000 7f9d7bfff640 1 -- 172.21.15.64:0/2292195767 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88022bd0 con 0x558ef72fddb0 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.097+0000 7f9d7bfff640 1 -- 172.21.15.64:0/2292195767 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d88022d70 con 0x558ef72fddb0 2024-05-09T09:25:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.097+0000 7f9d7bfff640 1 -- 172.21.15.64:0/2292195767 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8802a640 con 0x558ef72fddb0 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- 172.21.15.64:0/2292195767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 msgr2=0x558ef72f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 0x558ef72f2170 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9d88013400 tx=0x7f9d88009c80 comp rx=0 tx=0).stop 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- 172.21.15.64:0/2292195767 shutdown_connections 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72fddb0 0x558ef72f2170 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72f8c40 0x558ef72f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- 172.21.15.64:0/2292195767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef7302710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- 172.21.15.64:0/2292195767 >> 172.21.15.64:0/2292195767 conn(0x558ef72f38c0 msgr2=0x558ef72f6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- 172.21.15.64:0/2292195767 shutdown_connections 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- 172.21.15.64:0/2292195767 wait complete. 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- start start 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 Processor -- start 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef73062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef7306810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef73077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901aaa80 con 0x558ef72f8250 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901abef0 con 0x558ef72fddb0 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901ab150 con 0x558ef72f8c40 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d9b367740 10 client.0 _ll_register_callbacks cb 0x7f9d8c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.098+0000 7f9d96313640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef73062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d95b12640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef7306810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef73062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54588/0 (socket says 172.21.15.64:54588) 2024-05-09T09:25:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 -- 172.21.15.64:0/1174492458 learned_addr learned my addr 172.21.15.64:0/1174492458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96b14640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef73077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 -- 172.21.15.64:0/1174492458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 msgr2=0x558ef7306810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef7306810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 -- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 msgr2=0x558ef73077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef73077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 -- 172.21.15.64:0/1174492458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef7307dc0 con 0x558ef72f8250 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d95b12640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef7306810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d96313640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef73062d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0bb7d0 tx=0x7f9d8c0b9fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf3ebae101e866c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1174492458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c035040 con 0x558ef72f8250 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1174492458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c0c4400 con 0x558ef72f8250 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d88022d70 con 0x558ef72f8250 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1174492458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c003ca0 con 0x558ef72f8250 2024-05-09T09:25:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d9b367740 10 client.19359 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.099+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x558ef7309140 con 0x558ef72f8250 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1174492458 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d88022d70 con 0x558ef72f8250 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1174492458 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f9d8c0baf20 con 0x558ef72f8250 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d9b367740 10 client.19359 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d9b367740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d9b367740 20 client.19359 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d9b367740 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7376ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d9b367740 4 client.19359 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ef7263dc0 con 0x558ef730de80 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d95b12640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7376ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d95b12640 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7376ac0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3970c1624d8e0c31 server_cookie=434bc1b4b806a74d in_seq=0 out_seq=0 2024-05-09T09:25:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.100+0000 7f9d79ffb640 10 client.19359 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d79ffb640 1 -- 172.21.15.64:0/1174492458 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ef7263dc0 con 0x558ef730de80 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d79ffb640 10 client.19359 handle_command_reply: tid=0 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 client.19359 shutdown 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 msgr2=0x558ef73062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef73062d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0bb7d0 tx=0x7f9d8c0b9fe0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 shutdown_connections 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558ef730de80 0x558ef7376ac0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558ef72fddb0 0x558ef73077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d79ffb640 0 client.19359 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ef72f8c40 0x558ef7306810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 --2- 172.21.15.64:0/1174492458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ef72f8250 0x558ef73062d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 >> 172.21.15.64:0/1174492458 conn(0x558ef72f38c0 msgr2=0x558ef72f2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 shutdown_connections 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 1 -- 172.21.15.64:0/1174492458 wait complete. 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.112+0000 7f9d9b367740 20 client.19359 trim_cache size 0 max 16384 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-05-09T09:25:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-05-09T09:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2c30 msgr2=0x7f9d7c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2c30 0x7f9d7c0a50f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x558ef72f94a0 tx=0x7f9d88004ab0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d901018a0 msgr2=0x7f9d900755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d901018a0 0x7f9d900755e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009230 tx=0x7f9d8c031a30 comp rx=0 tx=0).stop 2024-05-09T09:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 shutdown_connections 2024-05-09T09:25:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f9d7c0a2c30 0x7f9d7c0a50f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d90103450 0x7f9d90074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d90102a90 0x7f9d90073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 --2- 172.21.15.64:0/2216401527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d901018a0 0x7f9d900755e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 >> 172.21.15.64:0/2216401527 conn(0x7f9d900fcdf0 msgr2=0x7f9d90105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 shutdown_connections 2024-05-09T09:25:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-05-09T09:25:23.113+0000 7f9d98577640 1 -- 172.21.15.64:0/2216401527 wait complete. 2024-05-09T09:25:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:22 smithi064 ceph-mon[30149]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.1 MiB/s wr, 700 op/s 2024-05-09T09:25:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:22 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1126261317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:23.691 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T09:25:23.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:25:23.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T09:25:23.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 -- 172.21.15.64:0/2536819911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc102320 msgr2=0x7f16cc108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/2536819911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc102320 0x7f16cc108bb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009ca0 tx=0x7f16c402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 -- 172.21.15.64:0/2536819911 shutdown_connections 2024-05-09T09:25:23.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/2536819911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc102320 0x7f16cc108bb0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/2536819911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc101960 0x7f16cc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/2536819911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16cc100770 0x7f16cc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 -- 172.21.15.64:0/2536819911 >> 172.21.15.64:0/2536819911 conn(0x7f16cc0fbce0 msgr2=0x7f16cc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 -- 172.21.15.64:0/2536819911 shutdown_connections 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.809+0000 7f16d3b9b640 1 -- 172.21.15.64:0/2536819911 wait complete. 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 Processor -- start 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 -- start start 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc100770 0x7f16cc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 0x7f16cc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16cc102320 0x7f16cc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10c2d0 con 0x7f16cc101960 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10c150 con 0x7f16cc100770 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d3b9b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10c450 con 0x7f16cc102320 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d1136640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 0x7f16cc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d1136640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 0x7f16cc19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54606/0 (socket says 172.21.15.64:54606) 2024-05-09T09:25:23.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d1136640 1 -- 172.21.15.64:0/423812956 learned_addr learned my addr 172.21.15.64:0/423812956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.810+0000 7f16d1937640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc100770 0x7f16cc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d1136640 1 -- 172.21.15.64:0/423812956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16cc102320 msgr2=0x7f16cc19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d1136640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16cc102320 0x7f16cc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d1136640 1 -- 172.21.15.64:0/423812956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc100770 msgr2=0x7f16cc069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d1136640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc100770 0x7f16cc069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d1136640 1 -- 172.21.15.64:0/423812956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc19f690 con 0x7f16cc101960 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d1136640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 0x7f16cc19c760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00d950 tx=0x7f16bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16c2ffd640 1 -- 172.21.15.64:0/423812956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc017070 con 0x7f16cc101960 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16c2ffd640 1 -- 172.21.15.64:0/423812956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16bc009c30 con 0x7f16cc101960 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc19f980 con 0x7f16cc101960 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16c2ffd640 1 -- 172.21.15.64:0/423812956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00f620 con 0x7f16cc101960 2024-05-09T09:25:23.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.811+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc19ff20 con 0x7f16cc101960 2024-05-09T09:25:23.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 Processor -- start 2024-05-09T09:25:23.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 -- start start 2024-05-09T09:25:23.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16c2ffd640 1 -- 172.21.15.64:0/423812956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f16bc0109a0 con 0x7f16cc101960 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19798cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e1979ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e196f8990 con 0x558e197988d0 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e196a05b0 con 0x558e1979e430 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d6983740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e196f4950 con 0x558e197992c0 2024-05-09T09:25:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d2138640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19798cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d1937640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e1979ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d2138640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19798cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54612/0 (socket says 172.21.15.64:54612) 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d1937640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e1979ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55280/0 (socket says 172.21.15.64:55280) 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.815+0000 7f16d2138640 1 -- 172.21.15.64:0/1051384989 learned_addr learned my addr 172.21.15.64:0/1051384989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1136640 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1937640 1 -- 172.21.15.64:0/1051384989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 msgr2=0x558e197a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1937640 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1937640 1 -- 172.21.15.64:0/1051384989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 msgr2=0x558e19798cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1937640 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19798cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1937640 1 -- 172.21.15.64:0/1051384989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e194d47e0 con 0x558e197992c0 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d2138640 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19798cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1136640 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16c2ffd640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16b80a2c10 0x7f16b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d1937640 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e1979ddc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009bf0 tx=0x7f16c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d64d8aafa2ef65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16d2138640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16b80a2c10 0x7f16b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16b7fff640 1 -- 172.21.15.64:0/1051384989 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8009da0 con 0x558e197992c0 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16c2ffd640 1 -- 172.21.15.64:0/423812956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f16bc0cefa0 con 0x7f16cc101960 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16b7fff640 1 -- 172.21.15.64:0/1051384989 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8005020 con 0x558e197992c0 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.816+0000 7f16b7fff640 1 -- 172.21.15.64:0/1051384989 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004ad0 con 0x558e197992c0 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 -- 172.21.15.64:0/1051384989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 msgr2=0x558e1979ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d2138640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16b80a2c10 0x7f16b80a50d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f16c4004af0 tx=0x7f16c4004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e1979ddc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009bf0 tx=0x7f16c8011810 comp rx=0 tx=0).stop 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 -- 172.21.15.64:0/1051384989 shutdown_connections 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e1979ddc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 --2- 172.21.15.64:0/1051384989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19798cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 -- 172.21.15.64:0/1051384989 >> 172.21.15.64:0/1051384989 conn(0x558e1975c0a0 msgr2=0x558e1978ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 -- 172.21.15.64:0/1051384989 shutdown_connections 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.817+0000 7f16d6983740 1 -- 172.21.15.64:0/1051384989 wait complete. 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 -- start start 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 Processor -- start 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 0x558e197a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197992c0 0x558e197a3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e197a13e0 con 0x558e197992c0 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e196b3b40 con 0x558e1979e430 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e197a1560 con 0x558e197988d0 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d6983740 10 client.0 _ll_register_callbacks cb 0x558e196a80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d1937640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 0x558e197a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d1136640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197992c0 0x558e197a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d1937640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 0x558e197a2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55284/0 (socket says 172.21.15.64:55284) 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d1937640 1 -- 172.21.15.64:0/3454788249 learned_addr learned my addr 172.21.15.64:0/3454788249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d2138640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d1937640 1 -- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 msgr2=0x558e197aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.818+0000 7f16d1937640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d1937640 1 -- 172.21.15.64:0/3454788249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197992c0 msgr2=0x558e197a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d1937640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197992c0 0x558e197a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d1937640 1 -- 172.21.15.64:0/3454788249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e19534f70 con 0x558e197988d0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d1136640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197992c0 0x558e197a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d1937640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 0x558e197a2d20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f16c8005790 tx=0x7f16c80056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ddc501f5ec5b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16b57fa640 1 -- 172.21.15.64:0/3454788249 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8029020 con 0x558e197988d0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c8005020 con 0x558e197988d0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16b57fa640 1 -- 172.21.15.64:0/3454788249 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f16c8004dd0 con 0x558e197988d0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16b57fa640 1 -- 172.21.15.64:0/3454788249 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f16c8034050 con 0x558e197988d0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d6983740 10 client.29002 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.819+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x558e193cc170 con 0x558e197988d0 2024-05-09T09:25:23.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16b57fa640 1 -- 172.21.15.64:0/3454788249 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8005020 con 0x558e197988d0 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16b57fa640 1 -- 172.21.15.64:0/3454788249 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f16c8019450 con 0x558e197988d0 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d6983740 10 client.29002 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d6983740 10 client.29002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d6983740 20 client.29002 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d6983740 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e197ae540 0x558e197b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d6983740 4 client.29002 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e197060c0 con 0x558e197ae540 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d1136640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e197ae540 0x558e197b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16d1136640 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e197ae540 0x558e197b09b0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e56a709bcc214e server_cookie=8dacb425d41de1da in_seq=0 out_seq=0 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.820+0000 7f16b57fa640 10 client.29002 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16b57fa640 1 -- 172.21.15.64:0/3454788249 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e197060c0 con 0x558e197ae540 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16b57fa640 10 client.29002 handle_command_reply: tid=0 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 client.29002 shutdown 2024-05-09T09:25:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 msgr2=0x558e197a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 0x558e197a2d20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f16c8005790 tx=0x7f16c80056b0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 shutdown_connections 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e197ae540 0x558e197b09b0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197992c0 0x558e197a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 --2- 172.21.15.64:0/3454788249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197988d0 0x558e197a2d20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16b57fa640 0 client.29002 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 >> 172.21.15.64:0/3454788249 conn(0x558e1975c0a0 msgr2=0x558e1978f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 shutdown_connections 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.821+0000 7f16d6983740 1 -- 172.21.15.64:0/3454788249 wait complete. 2024-05-09T09:25:23.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.822+0000 7f16d6983740 20 client.29002 trim_cache size 0 max 16384 2024-05-09T09:25:23.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 Processor -- start 2024-05-09T09:25:23.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 -- start start 2024-05-09T09:25:23.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19793510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e197aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1aac00 con 0x558e197988d0 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1a8410 con 0x558e1979e430 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.840+0000 7f16d6983740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1a6260 con 0x558e197992c0 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19793510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1136640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e197aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19793510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54620/0 (socket says 172.21.15.64:54620) 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1136640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e197aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55290/0 (socket says 172.21.15.64:55290) 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 -- 172.21.15.64:0/2573159837 learned_addr learned my addr 172.21.15.64:0/2573159837 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d2138640 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 -- 172.21.15.64:0/2573159837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 msgr2=0x558e197aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e197aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 -- 172.21.15.64:0/2573159837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 msgr2=0x558e197a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 -- 172.21.15.64:0/2573159837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e19790bf0 con 0x558e197988d0 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1136640 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e197aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16d1937640 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19793510 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f16c801b010 tx=0x7f16c8005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cca632d436e649 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16b77fe640 1 -- 172.21.15.64:0/2573159837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8029040 con 0x558e197988d0 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16b77fe640 1 -- 172.21.15.64:0/2573159837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8016030 con 0x558e197988d0 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.841+0000 7f16b77fe640 1 -- 172.21.15.64:0/2573159837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004dd0 con 0x558e197988d0 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 -- 172.21.15.64:0/2573159837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 msgr2=0x558e19793510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19793510 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f16c801b010 tx=0x7f16c8005550 comp rx=0 tx=0).stop 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 -- 172.21.15.64:0/2573159837 shutdown_connections 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e1979e430 0x558e197a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e197992c0 0x558e197aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 --2- 172.21.15.64:0/2573159837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e19793510 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 -- 172.21.15.64:0/2573159837 >> 172.21.15.64:0/2573159837 conn(0x558e1975c0a0 msgr2=0x558e19792e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 -- 172.21.15.64:0/2573159837 shutdown_connections 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 -- 172.21.15.64:0/2573159837 wait complete. 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.842+0000 7f16d6983740 1 -- start start 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 Processor -- start 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e197a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e197992c0 0x558e197a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1979e430 0x558e197a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1a68d0 con 0x558e197988d0 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1a7d40 con 0x558e197992c0 2024-05-09T09:25:23.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1a6fa0 con 0x558e1979e430 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d6983740 10 client.0 _ll_register_callbacks cb 0x7f16c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e197a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e197a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54630/0 (socket says 172.21.15.64:54630) 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 -- 172.21.15.64:0/2447381808 learned_addr learned my addr 172.21.15.64:0/2447381808 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d2138640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1979e430 0x558e197a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1136640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e197992c0 0x558e197a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 -- 172.21.15.64:0/2447381808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1979e430 msgr2=0x558e197a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1979e430 0x558e197a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 -- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e197992c0 msgr2=0x558e197a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e197992c0 0x558e197a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.843+0000 7f16d1937640 1 -- 172.21.15.64:0/2447381808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e197a8f10 con 0x558e197988d0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16d1937640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e197a73d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f16c8011850 tx=0x7f16c8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56ced172ed25695 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16b57fa640 1 -- 172.21.15.64:0/2447381808 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c803d070 con 0x558e197988d0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16b57fa640 1 -- 172.21.15.64:0/2447381808 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f16c8004dd0 con 0x558e197988d0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c8016030 con 0x558e197988d0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16b57fa640 1 -- 172.21.15.64:0/2447381808 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f16c8046050 con 0x558e197988d0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16d6983740 10 client.19377 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x558e197aa250 con 0x558e197988d0 2024-05-09T09:25:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16b57fa640 1 -- 172.21.15.64:0/2447381808 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8016030 con 0x558e197988d0 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16b57fa640 1 -- 172.21.15.64:0/2447381808 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f16c8040020 con 0x558e197988d0 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16d6983740 10 client.19377 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.844+0000 7f16d6983740 10 client.19377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16d6983740 20 client.19377 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16d6983740 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e19814f70 0x558e19817390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16d6983740 4 client.19377 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e19700410 con 0x558e19814f70 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16d1136640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e19814f70 0x558e19817390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16d1136640 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e19814f70 0x558e19817390 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd6d961bd59c39a server_cookie=2527ec862b939a25 in_seq=0 out_seq=0 2024-05-09T09:25:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.845+0000 7f16b57fa640 10 client.19377 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16b57fa640 1 -- 172.21.15.64:0/2447381808 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558e19700410 con 0x558e19814f70 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16b57fa640 10 client.19377 handle_command_reply: tid=0 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 client.19377 shutdown 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 msgr2=0x558e197a73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e197a73d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f16c8011850 tx=0x7f16c8009810 comp rx=0 tx=0).stop 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 shutdown_connections 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x558e19814f70 0x558e19817390 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e1979e430 0x558e197a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e197992c0 0x558e197a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16b57fa640 0 client.19377 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:23.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 --2- 172.21.15.64:0/2447381808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e197988d0 0x558e197a73d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 >> 172.21.15.64:0/2447381808 conn(0x558e1975c0a0 msgr2=0x558e1978e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 shutdown_connections 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 1 -- 172.21.15.64:0/2447381808 wait complete. 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.846+0000 7f16d6983740 20 client.19377 trim_cache size 0 max 16384 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "928bf444-f535-4f17-b17a-7a19780727c8", 2024-05-09T09:25:23.855 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-05-09T09:25:23.856 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16b80a2c10 msgr2=0x7f16b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16b80a2c10 0x7f16b80a50d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f16c4004af0 tx=0x7f16c4004a80 comp rx=0 tx=0).stop 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 msgr2=0x7f16cc19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 0x7f16cc19c760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00d950 tx=0x7f16bc00de10 comp rx=0 tx=0).stop 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 shutdown_connections 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f16b80a2c10 0x7f16b80a50d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16cc102320 0x7f16cc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16cc101960 0x7f16cc19c760 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 --2- 172.21.15.64:0/423812956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc100770 0x7f16cc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 >> 172.21.15.64:0/423812956 conn(0x7f16cc0fbce0 msgr2=0x7f16cc104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 shutdown_connections 2024-05-09T09:25:23.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.847+0000 7f16d3b9b640 1 -- 172.21.15.64:0/423812956 wait complete. 2024-05-09T09:25:23.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:25:23.860 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:25:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 -- 172.21.15.64:0/1259490234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0102320 msgr2=0x7f97e0108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1259490234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0102320 0x7f97e0108bb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009ca0 tx=0x7f97d802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 -- 172.21.15.64:0/1259490234 shutdown_connections 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1259490234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0102320 0x7f97e0108bb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1259490234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e0101960 0x7f97e0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1259490234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97e0100770 0x7f97e0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 -- 172.21.15.64:0/1259490234 >> 172.21.15.64:0/1259490234 conn(0x7f97e00fbce0 msgr2=0x7f97e00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.992+0000 7f97e7db7640 1 -- 172.21.15.64:0/1259490234 shutdown_connections 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 -- 172.21.15.64:0/1259490234 wait complete. 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 Processor -- start 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 -- start start 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 0x7f97e0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e0101960 0x7f97e019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97e0102320 0x7f97e019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97e010c2d0 con 0x7f97e0100770 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97e010c150 con 0x7f97e0101960 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.993+0000 7f97e7db7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97e010c450 con 0x7f97e0102320 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 0x7f97e0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 0x7f97e0069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54648/0 (socket says 172.21.15.64:54648) 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 -- 172.21.15.64:0/1040656159 learned_addr learned my addr 172.21.15.64:0/1040656159 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5352640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e0101960 0x7f97e019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e6354640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97e0102320 0x7f97e019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 -- 172.21.15.64:0/1040656159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97e0102320 msgr2=0x7f97e019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97e0102320 0x7f97e019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 -- 172.21.15.64:0/1040656159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e0101960 msgr2=0x7f97e019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e0101960 0x7f97e019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 -- 172.21.15.64:0/1040656159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e019f6f0 con 0x7f97e0100770 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e5b53640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 0x7f97e0069530 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9c0 tx=0x7f97dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97d6ffd640 1 -- 172.21.15.64:0/1040656159 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00cd80 con 0x7f97e0100770 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e019f980 con 0x7f97e0100770 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97d6ffd640 1 -- 172.21.15.64:0/1040656159 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97dc0035f0 con 0x7f97e0100770 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97d6ffd640 1 -- 172.21.15.64:0/1040656159 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc010690 con 0x7f97e0100770 2024-05-09T09:25:23.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.994+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e019fed0 con 0x7f97e0100770 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.997+0000 7f97d6ffd640 1 -- 172.21.15.64:0/1040656159 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97dc0037a0 con 0x7f97e0100770 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.997+0000 7f97eaba4740 1 Processor -- start 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 -- start start 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a9301420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a9303f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5605a9260d00 con 0x5605a92fc920 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5605a92084c0 con 0x5605a9301a90 2024-05-09T09:25:24.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97eaba4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5605a925ccc0 con 0x5605a92fbf30 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a9301420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e6354640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a9301420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54652/0 (socket says 172.21.15.64:54652) 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5352640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a9303f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5b53640 1 -- 172.21.15.64:0/4034230568 learned_addr learned my addr 172.21.15.64:0/4034230568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e6354640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55328/0 (socket says 172.21.15.64:55328) 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97d6ffd640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97cc0a2b90 0x7f97cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5352640 1 -- 172.21.15.64:0/4034230568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 msgr2=0x5605a92fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5352640 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5352640 1 -- 172.21.15.64:0/4034230568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 msgr2=0x5605a9301420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5352640 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a9301420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5352640 1 -- 172.21.15.64:0/4034230568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a903e7d0 con 0x5605a9301a90 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e6354640 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e5b53640 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a9301420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97e6354640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97cc0a2b90 0x7f97cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.998+0000 7f97d6ffd640 1 -- 172.21.15.64:0/1040656159 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f97dc0cf770 con 0x7f97e0100770 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.999+0000 7f97e5352640 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a9303f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97d000bbc0 tx=0x7f97d00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09e1f4777ba6a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.999+0000 7f97cbfff640 1 -- 172.21.15.64:0/4034230568 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0004030 con 0x5605a9301a90 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.999+0000 7f97cbfff640 1 -- 172.21.15.64:0/4034230568 <== mon.2 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d00035f0 con 0x5605a9301a90 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.999+0000 7f97cbfff640 1 -- 172.21.15.64:0/4034230568 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011680 con 0x5605a9301a90 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.999+0000 7f97eaba4740 1 -- 172.21.15.64:0/4034230568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 msgr2=0x5605a9303f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:23.999+0000 7f97e6354640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97cc0a2b90 0x7f97cc0a5050 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f97d802f3a0 tx=0x7f97d80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a9303f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97d000bbc0 tx=0x7f97d00107a0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- 172.21.15.64:0/4034230568 shutdown_connections 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a9303f20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a9301420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- 172.21.15.64:0/4034230568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- 172.21.15.64:0/4034230568 >> 172.21.15.64:0/4034230568 conn(0x5605a92f75a0 msgr2=0x5605a92f79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- 172.21.15.64:0/4034230568 shutdown_connections 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- 172.21.15.64:0/4034230568 wait complete. 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- start start 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 Processor -- start 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a9306330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 0x5605a9306870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5605a9304a40 con 0x5605a9301a90 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5605a921bb90 con 0x5605a92fc920 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.000+0000 7f97eaba4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5605a9304bc0 con 0x5605a92fbf30 2024-05-09T09:25:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97eaba4740 10 client.0 _ll_register_callbacks cb 0x5605a9257da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a9306330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 0x5605a9306870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 0x5605a9306870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.64:47558/0 (socket says 172.21.15.64:47558) 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 -- 172.21.15.64:0/113831134 learned_addr learned my addr 172.21.15.64:0/113831134 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a9306330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:55336/0 (socket says 172.21.15.64:55336) 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e6354640 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 -- 172.21.15.64:0/113831134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 msgr2=0x5605a9306330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a9306330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 -- 172.21.15.64:0/113831134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 msgr2=0x5605a930d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 -- 172.21.15.64:0/113831134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a8f743f0 con 0x5605a92fc920 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97e5352640 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 0x5605a9306870 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000c00 tx=0x7f97d00039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fead7fa8667df794 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.001+0000 7f97c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:25:24.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97c97fa640 1 -- 172.21.15.64:0/113831134 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d002a060 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d00035f0 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97c97fa640 1 -- 172.21.15.64:0/113831134 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f97d00041f0 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97c97fa640 1 -- 172.21.15.64:0/113831134 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f97d0031050 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 10 client.28874 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5605a8f340f0 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97c97fa640 1 -- 172.21.15.64:0/113831134 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d00035f0 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97c97fa640 1 -- 172.21.15.64:0/113831134 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f97d0004880 con 0x5605a92fc920 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 10 client.28874 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 10 client.28874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 20 client.28874 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a9311da0 0x5605a93141f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 4 client.28874 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.002+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605a926e430 con 0x5605a9311da0 2024-05-09T09:25:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.003+0000 7f97e5b53640 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a9311da0 0x5605a93141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.003+0000 7f97c97fa640 10 client.28874 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.003+0000 7f97e5b53640 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a9311da0 0x5605a93141f0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b18d5e0eaa25043 server_cookie=9dfd41be43dbea2b in_seq=0 out_seq=0 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97c97fa640 1 -- 172.21.15.64:0/113831134 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605a926e430 con 0x5605a9311da0 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97c97fa640 10 client.28874 handle_command_reply: tid=0 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 client.28874 shutdown 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 msgr2=0x5605a9306870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 0x5605a9306870 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000c00 tx=0x7f97d00039d0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 shutdown_connections 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a9311da0 0x5605a93141f0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fc920 0x5605a9306870 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 --2- 172.21.15.64:0/113831134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a9306330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97c97fa640 0 client.28874 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 >> 172.21.15.64:0/113831134 conn(0x5605a92f75a0 msgr2=0x5605a92f67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 shutdown_connections 2024-05-09T09:25:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 1 -- 172.21.15.64:0/113831134 wait complete. 2024-05-09T09:25:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.004+0000 7f97eaba4740 20 client.28874 trim_cache size 0 max 16384 2024-05-09T09:25:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.022+0000 7f97eaba4740 1 Processor -- start 2024-05-09T09:25:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.022+0000 7f97eaba4740 1 -- start start 2024-05-09T09:25:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a930d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a92fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97eaba4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01acdf0 con 0x5605a92fc920 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97eaba4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01aa600 con 0x5605a9301a90 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97eaba4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01a8450 con 0x5605a92fbf30 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a930d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a930d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54660/0 (socket says 172.21.15.64:54660) 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 -- 172.21.15.64:0/2417728960 learned_addr learned my addr 172.21.15.64:0/2417728960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e6354640 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a92fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 -- 172.21.15.64:0/2417728960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 msgr2=0x5605a92fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 -- 172.21.15.64:0/2417728960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 msgr2=0x5605a92fa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a92fa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 -- 172.21.15.64:0/2417728960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a92fba90 con 0x5605a92fc920 2024-05-09T09:25:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e6354640 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a92fa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97e5352640 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a930d870 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f97d000b3d0 tx=0x7f97d000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341d834caab8d25e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97cb7fe640 1 -- 172.21.15.64:0/2417728960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d002a040 con 0x5605a92fc920 2024-05-09T09:25:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97cb7fe640 1 -- 172.21.15.64:0/2417728960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d000bd70 con 0x5605a92fc920 2024-05-09T09:25:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.023+0000 7f97cb7fe640 1 -- 172.21.15.64:0/2417728960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0004e40 con 0x5605a92fc920 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 -- 172.21.15.64:0/2417728960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 msgr2=0x5605a930d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a930d870 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f97d000b3d0 tx=0x7f97d000bac0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 -- 172.21.15.64:0/2417728960 shutdown_connections 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a9301a90 0x5605a92fa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a92fc920 0x5605a930d870 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 --2- 172.21.15.64:0/2417728960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fbf30 0x5605a92fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 -- 172.21.15.64:0/2417728960 >> 172.21.15.64:0/2417728960 conn(0x5605a92f75a0 msgr2=0x5605a92f7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 -- 172.21.15.64:0/2417728960 shutdown_connections 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.024+0000 7f97eaba4740 1 -- 172.21.15.64:0/2417728960 wait complete. 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 -- start start 2024-05-09T09:25:24.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 Processor -- start 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fbf30 0x5605a930b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fc920 0x5605a930bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01a8ac0 con 0x5605a9301a90 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01a9f30 con 0x5605a92fbf30 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01a9190 con 0x5605a92fc920 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97eaba4740 10 client.0 _ll_register_callbacks cb 0x7f97dc0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97e6354640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97e5352640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fc920 0x5605a930bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97e5b53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fbf30 0x5605a930b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97e6354640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54672/0 (socket says 172.21.15.64:54672) 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.025+0000 7f97e6354640 1 -- 172.21.15.64:0/622452348 learned_addr learned my addr 172.21.15.64:0/622452348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5352640 1 -- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fbf30 msgr2=0x5605a930b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5352640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fbf30 0x5605a930b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5352640 1 -- 172.21.15.64:0/622452348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 msgr2=0x5605a930cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5352640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5352640 1 -- 172.21.15.64:0/622452348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a930d130 con 0x5605a92fc920 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e6354640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5b53640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fbf30 0x5605a930b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97e5352640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fc920 0x5605a930bb30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x5605a9307000 tx=0x7f97d000bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b218d2b5af028cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97c97fa640 1 -- 172.21.15.64:0/622452348 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0043060 con 0x5605a92fc920 2024-05-09T09:25:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d000bd70 con 0x5605a92fc920 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97c97fa640 1 -- 172.21.15.64:0/622452348 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f97d0011910 con 0x5605a92fc920 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97c97fa640 1 -- 172.21.15.64:0/622452348 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f97d004a050 con 0x5605a92fc920 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97eaba4740 10 client.29026 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.026+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5605a93789c0 con 0x5605a92fc920 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97c97fa640 1 -- 172.21.15.64:0/622452348 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d000bd70 con 0x5605a92fc920 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97c97fa640 1 -- 172.21.15.64:0/622452348 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f97d00052c0 con 0x5605a92fc920 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97eaba4740 10 client.29026 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97eaba4740 10 client.29026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97eaba4740 20 client.29026 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97eaba4740 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a93793d0 0x5605a937b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97eaba4740 4 client.29026 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605a9268780 con 0x5605a93793d0 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97e5b53640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a93793d0 0x5605a937b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97c97fa640 10 client.29026 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.027+0000 7f97e5b53640 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a93793d0 0x5605a937b7f0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55373f48675bc277 server_cookie=ce9bc2fd6dd44c6 in_seq=0 out_seq=0 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97c97fa640 1 -- 172.21.15.64:0/622452348 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5605a9268780 con 0x5605a93793d0 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97c97fa640 10 client.29026 handle_command_reply: tid=0 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 client.29026 shutdown 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fc920 msgr2=0x5605a930bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fc920 0x5605a930bb30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x5605a9307000 tx=0x7f97d000bf20 comp rx=0 tx=0).stop 2024-05-09T09:25:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 shutdown_connections 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5605a93793d0 0x5605a937b7f0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97c97fa640 0 client.29026 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605a9301a90 0x5605a930cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605a92fc920 0x5605a930bb30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 --2- 172.21.15.64:0/622452348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5605a92fbf30 0x5605a930b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 >> 172.21.15.64:0/622452348 conn(0x5605a92f75a0 msgr2=0x5605a92f7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 shutdown_connections 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 1 -- 172.21.15.64:0/622452348 wait complete. 2024-05-09T09:25:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.028+0000 7f97eaba4740 20 client.29026 trim_cache size 0 max 16384 2024-05-09T09:25:24.036 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:25:24.036 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T09:25:24.036 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-05-09T09:25:24.036 INFO:teuthology.orchestra.run.smithi064.stdout: "928bf444-f535-4f17-b17a-7a19780727c8": { 2024-05-09T09:25:24.036 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-05-09T09:25:24.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "928bf444-f535-4f17-b17a-7a19780727c8", 2024-05-09T09:25:24.037 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-05-09T09:25:24.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:24.037 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:24.037 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97cc0a2b90 msgr2=0x7f97cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97cc0a2b90 0x7f97cc0a5050 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f97d802f3a0 tx=0x7f97d80095c0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 msgr2=0x7f97e0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 0x7f97e0069530 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9c0 tx=0x7f97dc00ee80 comp rx=0 tx=0).stop 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 shutdown_connections 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f97cc0a2b90 0x7f97cc0a5050 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97e0102320 0x7f97e019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e0101960 0x7f97e019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 --2- 172.21.15.64:0/1040656159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97e0100770 0x7f97e0069530 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:24.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 >> 172.21.15.64:0/1040656159 conn(0x7f97e00fbce0 msgr2=0x7f97e0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:24.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 shutdown_connections 2024-05-09T09:25:24.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:24.029+0000 7f97e7db7640 1 -- 172.21.15.64:0/1040656159 wait complete. 2024-05-09T09:25:24.041 INFO:tasks.cephfs.filesystem:scrub status for tag:928bf444-f535-4f17-b17a-7a19780727c8 - {'path': '/', 'tag': '928bf444-f535-4f17-b17a-7a19780727c8', 'options': 'recursive,force'} 2024-05-09T09:25:24.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-05-09T09:25:24.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: scrub queued for path: / 2024-05-09T09:25:24.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:24.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:25:24.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:25:24.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:24.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:24 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-05-09T09:25:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: scrub queued for path: / 2024-05-09T09:25:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:25:25.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:25:25.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:24 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: scrub queued for path: / 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:24 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:25 smithi179 ceph-mon[34086]: scrub summary: active paths [/] 2024-05-09T09:25:25.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:25 smithi179 ceph-mon[34086]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:26.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:25 smithi139 ceph-mon[33599]: scrub summary: active paths [/] 2024-05-09T09:25:26.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:25 smithi139 ceph-mon[33599]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:25 smithi064 ceph-mon[30149]: scrub summary: active paths [/] 2024-05-09T09:25:26.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:25 smithi064 ceph-mon[30149]: scrub summary: idle+waiting paths [/] 2024-05-09T09:25:26.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:26 smithi179 ceph-mon[34086]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.2 MiB/s wr, 812 op/s 2024-05-09T09:25:27.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:26 smithi139 ceph-mon[33599]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.2 MiB/s wr, 812 op/s 2024-05-09T09:25:27.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:26 smithi064 ceph-mon[30149]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.2 MiB/s wr, 812 op/s 2024-05-09T09:25:28.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:28 smithi179 ceph-mon[34086]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.8 MiB/s wr, 727 op/s 2024-05-09T09:25:29.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:28 smithi139 ceph-mon[33599]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.8 MiB/s wr, 727 op/s 2024-05-09T09:25:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:28 smithi064 ceph-mon[30149]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.8 MiB/s wr, 727 op/s 2024-05-09T09:25:30.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:30 smithi179 ceph-mon[34086]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:25:31.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:30 smithi139 ceph-mon[33599]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:25:31.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:30 smithi064 ceph-mon[30149]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-09T09:25:31.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/: 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:total 2 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287631 May 9 09:09 . 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1572287631 May 9 09:09 .. 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 data 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data: 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:total 46 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 . 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287631 May 9 09:09 .. 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 598193717 May 9 09:10 base 2024-05-09T09:25:32.228 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 9 09:09 current_logfiles 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 9 09:09 global 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 9 09:09 log 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_commit_ts 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_dynshmem 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 9 09:09 pg_hba.conf 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 9 09:09 pg_ident.conf 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:25 pg_logical 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 pg_multixact 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_notify 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_replslot 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_serial 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_snapshots 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 9 09:25 pg_stat 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:25 pg_stat_tmp 2024-05-09T09:25:32.229 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 9 09:25 pg_subtrans 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_tblspc 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 pg_twophase 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 973078528 May 9 09:25 pg_wal 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 pg_xact 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 9 09:09 postgresql.auto.conf 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 9 09:09 postgresql.conf 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 9 09:09 postmaster.opts 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base: 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:total 3 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 598193717 May 9 09:10 . 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.230 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 1 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 13493 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 582267439 May 9 09:15 13494 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:10 pgsql_tmp 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:total 7778 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 . 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 598193717 May 9 09:10 .. 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 112 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 113 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 1247 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1247_fsm 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1247_vm 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 09:09 1249 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1249_fsm 2024-05-09T09:25:32.231 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1249_vm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 09:09 1255 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1255_fsm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1255_vm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 09:09 1259 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1259_fsm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1259_vm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 09:09 13344 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13344_fsm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13344_vm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13346 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13348 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13349_fsm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349_vm 2024-05-09T09:25:32.232 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13351 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13353 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13354_fsm 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354_vm 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13356 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13358 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13359_fsm 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359_vm 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13361 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13363 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1417 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1418 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 174 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 175 2024-05-09T09:25:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2187 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2224 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2328 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2336 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2337 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2579 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2600 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2600_fsm 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2600_vm 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2601_fsm 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601_vm 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2602 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2602_fsm 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2602_vm 2024-05-09T09:25:32.234 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2603 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2603_fsm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2603_vm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2604 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2605 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2605_fsm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2605_vm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2606_fsm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606_vm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2607 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2607_fsm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2607_vm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 09:09 2608 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2608_fsm 2024-05-09T09:25:32.235 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2608_vm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 09:09 2609 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2609_fsm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2609_vm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2610 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2610_fsm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2610_vm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2611 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2612_fsm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612_vm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2613 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2615_fsm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615_vm 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2616 2024-05-09T09:25:32.236 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2616_fsm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2616_vm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 09:09 2617 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2617_fsm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2617_vm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 09:09 2618 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2618_fsm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2618_vm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 09:09 2619 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2619_fsm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2619_vm 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2620 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2650 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2651 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2652 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2653 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2654 2024-05-09T09:25:32.237 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2655 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2656 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2657 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 09:09 2658 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 2659 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2660 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2661 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2662 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2663 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2664 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2665 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2666 2024-05-09T09:25:32.238 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2667 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2668 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2669 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2670 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 09:09 2673 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 09:09 2674 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 2675 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2678 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2679 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2680 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2681 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2682 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2683 2024-05-09T09:25:32.239 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2684 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2685 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2686 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2687 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2688 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2689 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:09 2690 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 09:09 2691 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2692 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2693 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2696 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2699 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2701 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2702 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2703 2024-05-09T09:25:32.240 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2704 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2753 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2753_fsm 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2753_vm 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2754 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2755 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2756 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2757 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2830 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2831 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2832 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2833 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2834 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2835 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2836 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2837 2024-05-09T09:25:32.241 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 09:09 2838 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2838_fsm 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2838_vm 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2839 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840_fsm 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2840_vm 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2841 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2995 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2996 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3079_fsm 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079_vm 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3080 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3081 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3085 2024-05-09T09:25:32.242 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3118 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3119 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 09:09 3164 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3256 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3257 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3258 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3350 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3351 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3379 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3380 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3381 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3394 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3394_fsm 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3394_vm 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3395 2024-05-09T09:25:32.243 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3429 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3430 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3431 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3433 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3439 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3440 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3455 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 09:09 3456 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3456_fsm 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3456_vm 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3466 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3467 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3468 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3501 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3502 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3503 2024-05-09T09:25:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3534 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3541_fsm 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541_vm 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3542 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3574 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3575 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3576 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3596 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3597 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3598 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3599 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3600_fsm 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600_vm 2024-05-09T09:25:32.245 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3601_fsm 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601_vm 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3602_fsm 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602_vm 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603_fsm 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3603_vm 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3604 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3605 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3606 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3607 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3608 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 3609 2024-05-09T09:25:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3712 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3764_fsm 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764_vm 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3766 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3767 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3997 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4143 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4144 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4145 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4146 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4147 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4148 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4149 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4150 2024-05-09T09:25:32.247 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4151 2024-05-09T09:25:32.484 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4152 2024-05-09T09:25:32.484 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4153 2024-05-09T09:25:32.484 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4154 2024-05-09T09:25:32.484 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4155 2024-05-09T09:25:32.484 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4156 2024-05-09T09:25:32.485 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4157 2024-05-09T09:25:32.485 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4158 2024-05-09T09:25:32.485 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4159 2024-05-09T09:25:32.485 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4160 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4161 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4162 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4163 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4164 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4165 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4166 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4167 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4168 2024-05-09T09:25:32.486 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4169 2024-05-09T09:25:32.487 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4170 2024-05-09T09:25:32.487 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4171 2024-05-09T09:25:32.487 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4172 2024-05-09T09:25:32.487 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4173 2024-05-09T09:25:32.487 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4174 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 5002 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 548 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 549 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6102 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6104 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6106 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6110 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6111 2024-05-09T09:25:32.491 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6112 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6113 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6117 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 826 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 827 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 828 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.492 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:total 7778 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 9 09:09 . 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 598193717 May 9 09:10 .. 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 112 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 113 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 1247 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1247_fsm 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1247_vm 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 09:09 1249 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1249_fsm 2024-05-09T09:25:32.493 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1249_vm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 09:09 1255 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1255_fsm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1255_vm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 09:09 1259 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1259_fsm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1259_vm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 09:09 13344 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13344_fsm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13344_vm 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13346 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13348 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349 2024-05-09T09:25:32.494 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13349_fsm 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349_vm 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13351 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13353 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13354_fsm 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354_vm 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13356 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13358 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13359_fsm 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359_vm 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13361 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13363 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1417 2024-05-09T09:25:32.495 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1418 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 174 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 175 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2187 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2224 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2328 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2336 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2337 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2579 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2600 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2600_fsm 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2600_vm 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2601_fsm 2024-05-09T09:25:32.496 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601_vm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2602 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2602_fsm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2602_vm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2603 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2603_fsm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2603_vm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2604 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2605 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2605_fsm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2605_vm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2606_fsm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2606_vm 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2607 2024-05-09T09:25:32.497 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2607_fsm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2607_vm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 09:09 2608 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2608_fsm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2608_vm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 09:09 2609 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2609_fsm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2609_vm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2610 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2610_fsm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2610_vm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2611 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2612_fsm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612_vm 2024-05-09T09:25:32.498 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2613 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2615_fsm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615_vm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2616 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2616_fsm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2616_vm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 09:09 2617 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2617_fsm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2617_vm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 09:09 2618 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2618_fsm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2618_vm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 09:09 2619 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2619_fsm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2619_vm 2024-05-09T09:25:32.499 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2620 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2650 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2651 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2652 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2653 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2654 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2655 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2656 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2657 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 09:09 2658 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:09 2659 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2660 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2661 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2662 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2663 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2664 2024-05-09T09:25:32.500 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2665 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2666 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2667 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2668 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2669 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2670 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 09:09 2673 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 09:09 2674 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 2675 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2678 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2679 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2680 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2681 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2682 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2683 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2684 2024-05-09T09:25:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2685 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2686 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2687 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2688 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2689 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:09 2690 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 09:09 2691 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2692 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2693 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2696 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2699 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2701 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2702 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2703 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2704 2024-05-09T09:25:32.502 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2753 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2753_fsm 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2753_vm 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2754 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2755 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2756 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2757 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2830 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2831 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2832 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2833 2024-05-09T09:25:32.503 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2834 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2835 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2836 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2837 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 09:09 2838 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2838_fsm 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2838_vm 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2839 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840_fsm 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2840_vm 2024-05-09T09:25:32.504 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2841 2024-05-09T09:25:32.505 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2995 2024-05-09T09:25:32.505 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2996 2024-05-09T09:25:32.505 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3079_fsm 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079_vm 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3080 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3081 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3085 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3118 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3119 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 09:09 3164 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3256 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3257 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3258 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3350 2024-05-09T09:25:32.506 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3351 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3379 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3380 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3381 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3394 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3394_fsm 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3394_vm 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3395 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3429 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3430 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3431 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3433 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3439 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3440 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3455 2024-05-09T09:25:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 09:09 3456 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3456_fsm 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3456_vm 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3466 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3467 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3468 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3501 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3502 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3503 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3534 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3541_fsm 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541_vm 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3542 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3574 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3575 2024-05-09T09:25:32.508 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3576 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3596 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3597 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3598 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3599 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3600_fsm 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600_vm 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3601_fsm 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601_vm 2024-05-09T09:25:32.509 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3602_fsm 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602_vm 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603_fsm 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3603_vm 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3604 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3605 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3606 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3607 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3608 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 3609 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3712 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3764_fsm 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764_vm 2024-05-09T09:25:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3766 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3767 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3997 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4143 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4144 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4145 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4146 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4147 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4148 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4149 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4150 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4151 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4152 2024-05-09T09:25:32.511 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4153 2024-05-09T09:25:32.768 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4154 2024-05-09T09:25:32.769 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4155 2024-05-09T09:25:32.769 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4156 2024-05-09T09:25:32.770 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4157 2024-05-09T09:25:32.770 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4158 2024-05-09T09:25:32.770 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4159 2024-05-09T09:25:32.770 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4160 2024-05-09T09:25:32.770 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4161 2024-05-09T09:25:32.771 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4162 2024-05-09T09:25:32.771 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4163 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4164 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4165 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4166 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4167 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4168 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4169 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4170 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4171 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4172 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4173 2024-05-09T09:25:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4174 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 5002 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 548 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 549 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6102 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6104 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6106 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6110 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6111 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6112 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6113 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6117 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 826 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 827 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 828 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:25:32.773 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout:total 568623 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 582267439 May 9 09:15 . 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 598193717 May 9 09:10 .. 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 112 2024-05-09T09:25:32.774 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 113 2024-05-09T09:25:32.775 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:10 1247 2024-05-09T09:25:32.775 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1247_fsm 2024-05-09T09:25:32.775 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 1247_vm 2024-05-09T09:25:32.775 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 09:10 1249 2024-05-09T09:25:32.775 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:10 1249_fsm 2024-05-09T09:25:32.775 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 1249_vm 2024-05-09T09:25:32.776 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 09:09 1255 2024-05-09T09:25:32.776 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1255_fsm 2024-05-09T09:25:32.776 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1255_vm 2024-05-09T09:25:32.776 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 09:25 1259 2024-05-09T09:25:32.776 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1259_fsm 2024-05-09T09:25:32.776 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 1259_vm 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 09:09 13344 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13344_fsm 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13344_vm 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13346 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13348 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13349_fsm 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13349_vm 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13351 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13353 2024-05-09T09:25:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13354_fsm 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13354_vm 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13356 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13358 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 13359_fsm 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13359_vm 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 13361 2024-05-09T09:25:32.778 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 13363 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1417 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 1418 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 448151552 May 9 09:25 16396 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 9 09:21 16396_fsm 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:19 16396_vm 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 6275072 May 9 09:25 16397 2024-05-09T09:25:32.779 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:25 16397_fsm 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:25 16397_vm 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 5734400 May 9 09:25 16399 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:25 16399_fsm 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:25 16399_vm 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:25 16400 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:25 16402 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 9 09:25 16404 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 41689088 May 9 09:25 16406 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:25 16406_fsm 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:23 16406_vm 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 174 2024-05-09T09:25:32.780 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 175 2024-05-09T09:25:32.781 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2187 2024-05-09T09:25:32.781 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2224 2024-05-09T09:25:32.781 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2328 2024-05-09T09:25:32.781 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2336 2024-05-09T09:25:32.781 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2337 2024-05-09T09:25:32.781 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2579 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2600 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2600_fsm 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2600_vm 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2601_fsm 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2601_vm 2024-05-09T09:25:32.782 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2602 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2602_fsm 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2602_vm 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2603 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2603_fsm 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2603_vm 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2604 2024-05-09T09:25:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2605 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2605_fsm 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2605_vm 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 2606 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2606_fsm 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 2606_vm 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2607 2024-05-09T09:25:32.784 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2607_fsm 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2607_vm 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 09:10 2608 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2608_fsm 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 2608_vm 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 09:09 2609 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2609_fsm 2024-05-09T09:25:32.785 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2609_vm 2024-05-09T09:25:32.786 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:10 2610 2024-05-09T09:25:32.786 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2610_fsm 2024-05-09T09:25:32.786 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 2610_vm 2024-05-09T09:25:32.786 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2611 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2612_fsm 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2612_vm 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2613 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2615_fsm 2024-05-09T09:25:32.787 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2615_vm 2024-05-09T09:25:32.788 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2616 2024-05-09T09:25:32.788 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2616_fsm 2024-05-09T09:25:32.788 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2616_vm 2024-05-09T09:25:32.788 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 09:09 2617 2024-05-09T09:25:32.788 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2617_fsm 2024-05-09T09:25:32.788 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2617_vm 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 09:09 2618 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2618_fsm 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2618_vm 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 May 9 09:25 2619 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:19 2619_fsm 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:10 2619_vm 2024-05-09T09:25:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2620 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2650 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2651 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2652 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2653 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2654 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2655 2024-05-09T09:25:32.790 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2656 2024-05-09T09:25:32.791 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2657 2024-05-09T09:25:32.791 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 09:10 2658 2024-05-09T09:25:32.791 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 09:10 2659 2024-05-09T09:25:32.791 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2660 2024-05-09T09:25:32.791 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2661 2024-05-09T09:25:32.791 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:10 2662 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:10 2663 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2664 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2665 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2666 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2667 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2668 2024-05-09T09:25:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2669 2024-05-09T09:25:32.793 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2670 2024-05-09T09:25:32.793 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 09:10 2673 2024-05-09T09:25:32.793 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 09:10 2674 2024-05-09T09:25:32.793 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 2675 2024-05-09T09:25:32.793 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2678 2024-05-09T09:25:32.793 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:10 2679 2024-05-09T09:25:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2680 2024-05-09T09:25:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2681 2024-05-09T09:25:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2682 2024-05-09T09:25:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2683 2024-05-09T09:25:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2684 2024-05-09T09:25:32.794 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2685 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2686 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2687 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 2688 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 09:09 2689 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 09:09 2690 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 09:09 2691 2024-05-09T09:25:32.795 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2692 2024-05-09T09:25:32.796 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2693 2024-05-09T09:25:32.796 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:21 2696 2024-05-09T09:25:32.796 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2699 2024-05-09T09:25:32.796 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2701 2024-05-09T09:25:32.796 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2702 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:10 2703 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:10 2704 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2753 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2753_fsm 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2753_vm 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2754 2024-05-09T09:25:32.797 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2755 2024-05-09T09:25:32.798 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2756 2024-05-09T09:25:32.798 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 2757 2024-05-09T09:25:32.798 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2830 2024-05-09T09:25:32.798 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2831 2024-05-09T09:25:32.798 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2832 2024-05-09T09:25:32.798 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2833 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2834 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2835 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2836 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2837 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 09:09 2838 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2838_fsm 2024-05-09T09:25:32.799 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2838_vm 2024-05-09T09:25:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2839 2024-05-09T09:25:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840 2024-05-09T09:25:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2840_fsm 2024-05-09T09:25:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2840_vm 2024-05-09T09:25:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2841 2024-05-09T09:25:32.800 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2995 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2996 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3079_fsm 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3079_vm 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3080 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3081 2024-05-09T09:25:32.801 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:09 3085 2024-05-09T09:25:32.802 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3118 2024-05-09T09:25:32.802 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3119 2024-05-09T09:25:32.802 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 09:09 3164 2024-05-09T09:25:32.802 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3256 2024-05-09T09:25:32.802 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3257 2024-05-09T09:25:32.802 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3258 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3350 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3351 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3379 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3380 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3381 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3394 2024-05-09T09:25:32.803 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3394_fsm 2024-05-09T09:25:32.804 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3394_vm 2024-05-09T09:25:32.804 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3395 2024-05-09T09:25:32.804 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3429 2024-05-09T09:25:32.804 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3430 2024-05-09T09:25:32.804 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3431 2024-05-09T09:25:32.804 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3433 2024-05-09T09:25:32.805 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3439 2024-05-09T09:25:32.805 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3440 2024-05-09T09:25:32.805 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 09:10 3455 2024-05-09T09:25:32.805 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 09:09 3456 2024-05-09T09:25:32.805 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3456_fsm 2024-05-09T09:25:32.805 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3456_vm 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3466 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3467 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3468 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3501 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3502 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3503 2024-05-09T09:25:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3534 2024-05-09T09:25:32.807 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541 2024-05-09T09:25:32.807 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3541_fsm 2024-05-09T09:25:32.807 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3541_vm 2024-05-09T09:25:32.807 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3542 2024-05-09T09:25:32.807 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3574 2024-05-09T09:25:32.807 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3575 2024-05-09T09:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3576 2024-05-09T09:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3596 2024-05-09T09:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3597 2024-05-09T09:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3598 2024-05-09T09:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3599 2024-05-09T09:25:32.808 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3600_fsm 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3600_vm 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3601_fsm 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3601_vm 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602 2024-05-09T09:25:32.809 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3602_fsm 2024-05-09T09:25:32.810 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3602_vm 2024-05-09T09:25:32.810 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603 2024-05-09T09:25:32.810 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3603_fsm 2024-05-09T09:25:32.810 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3603_vm 2024-05-09T09:25:32.810 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3604 2024-05-09T09:25:32.810 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3605 2024-05-09T09:25:32.811 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3606 2024-05-09T09:25:32.811 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3607 2024-05-09T09:25:32.811 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3608 2024-05-09T09:25:32.811 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 09:09 3609 2024-05-09T09:25:32.811 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3712 2024-05-09T09:25:32.811 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764 2024-05-09T09:25:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 3764_fsm 2024-05-09T09:25:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3764_vm 2024-05-09T09:25:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3766 2024-05-09T09:25:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 3767 2024-05-09T09:25:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3997 2024-05-09T09:25:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4143 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4144 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4145 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4146 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4147 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4148 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4149 2024-05-09T09:25:32.813 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4150 2024-05-09T09:25:32.814 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4151 2024-05-09T09:25:32.814 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4152 2024-05-09T09:25:32.814 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4153 2024-05-09T09:25:32.814 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4154 2024-05-09T09:25:32.814 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4155 2024-05-09T09:25:32.814 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4156 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4157 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4158 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4159 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4160 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4161 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4162 2024-05-09T09:25:32.815 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4163 2024-05-09T09:25:32.816 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4164 2024-05-09T09:25:32.816 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4165 2024-05-09T09:25:32.816 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4166 2024-05-09T09:25:32.816 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4167 2024-05-09T09:25:32.816 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4168 2024-05-09T09:25:32.816 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4169 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4170 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4171 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4172 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4173 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4174 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 5002 2024-05-09T09:25:32.817 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 548 2024-05-09T09:25:32.818 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 549 2024-05-09T09:25:32.818 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6102 2024-05-09T09:25:32.818 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6104 2024-05-09T09:25:32.818 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6106 2024-05-09T09:25:32.818 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6110 2024-05-09T09:25:32.819 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6111 2024-05-09T09:25:32.819 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6112 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6113 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6117 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 826 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 827 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 828 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 9 09:09 pg_internal.init 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 09:09 PG_VERSION 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:10 . 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 598193717 May 9 09:10 .. 2024-05-09T09:25:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/global: 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:total 561 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 9 09:09 . 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1213 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1213_fsm 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1213_vm 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1214 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1214_fsm 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1214_vm 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 1232 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 1233 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1260 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1260_fsm 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1260_vm 2024-05-09T09:25:32.821 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1261 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1261_fsm 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1261_vm 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1262 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 1262_fsm 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 1262_vm 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2396 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 09:09 2396_fsm 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2396_vm 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2397 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2671 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2672 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2676 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2677 2024-05-09T09:25:32.822 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2694 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2695 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2697 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 2698 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2846 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2847 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2964 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2965 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 2966 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 2967 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 3592 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 3593 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4060 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4061 2024-05-09T09:25:32.828 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4175 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4176 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4177 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4178 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4181 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4182 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4183 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4184 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 4185 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 4186 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6000 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6001 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6002 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 6100 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6114 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 6115 2024-05-09T09:25:32.829 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:25 pg_control 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 09:09 pg_filenode.map 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 9 09:09 pg_internal.init 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/log: 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:total 3 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 9 09:09 . 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 May 9 09:25 postgresql-Thu.log 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-09T09:25:32.830 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.831 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.831 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.831 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-09T09:25:32.831 INFO:teuthology.orchestra.run.smithi064.stdout:total 3 2024-05-09T09:25:32.875 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:25 . 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 mappings 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:25 replorigin_checkpoint 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 snapshots 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:25 .. 2024-05-09T09:25:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 09:25 .. 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:total 2 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 . 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 members 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 offsets 2024-05-09T09:25:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:total 9 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 . 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 .. 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 0000 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:total 9 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:09 . 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 09:09 .. 2024-05-09T09:25:32.878 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 09:14 0000 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.879 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:total 16 2024-05-09T09:25:32.880 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 9 09:25 . 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 9 09:25 db_0.stat 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 9 09:25 db_13494.stat 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 9 09:25 global.stat 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:25 . 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:total 177 2024-05-09T09:25:32.881 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 9 09:25 . 2024-05-09T09:25:32.882 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 9 09:25 000B 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-09T09:25:32.926 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout:total 950274 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 973078528 May 9 09:25 . 2024-05-09T09:25:32.927 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:25 000000010000000000000073 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:15 000000010000000000000074 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:15 000000010000000000000075 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 000000010000000000000076 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:15 000000010000000000000077 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:15 000000010000000000000078 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:16 000000010000000000000079 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:14 00000001000000000000007A 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:17 00000001000000000000007B 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:16 00000001000000000000007C 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:15 00000001000000000000007D 2024-05-09T09:25:32.928 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:15 00000001000000000000007E 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 00000001000000000000007F 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000080 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 000000010000000000000081 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000082 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000083 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:21 000000010000000000000084 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:23 000000010000000000000085 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 000000010000000000000086 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000087 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:22 000000010000000000000088 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:21 000000010000000000000089 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 00000001000000000000008A 2024-05-09T09:25:32.929 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:22 00000001000000000000008B 2024-05-09T09:25:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 00000001000000000000008C 2024-05-09T09:25:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:21 00000001000000000000008D 2024-05-09T09:25:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 00000001000000000000008E 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 00000001000000000000008F 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 000000010000000000000090 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 000000010000000000000091 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000092 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:23 000000010000000000000093 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:21 000000010000000000000094 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 000000010000000000000095 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 000000010000000000000096 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000097 2024-05-09T09:25:32.933 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 000000010000000000000098 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:22 000000010000000000000099 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:20 00000001000000000000009A 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 00000001000000000000009B 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 00000001000000000000009C 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:19 00000001000000000000009D 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:21 00000001000000000000009E 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 00000001000000000000009F 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A0 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A1 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A2 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:25 0000000100000000000000A3 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A4 2024-05-09T09:25:32.934 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A5 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:25 0000000100000000000000A6 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:25 0000000100000000000000A7 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A8 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000A9 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:25 0000000100000000000000AA 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:24 0000000100000000000000AB 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 09:25 0000000100000000000000AC 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 archive_status 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:total 1 2024-05-09T09:25:32.935 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 09:09 . 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 973078528 May 9 09:25 .. 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout:total 193 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:09 . 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1572287524 May 9 09:25 .. 2024-05-09T09:25:32.936 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 09:25 0000 2024-05-09T09:25:32.937 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T09:25:32.956 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:25:32.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:32 smithi064 ceph-mon[30149]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 8.6 MiB/s wr, 987 op/s 2024-05-09T09:25:32.966 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T09:25:32.984 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T09:25:32.984 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:32 smithi179 ceph-mon[34086]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 8.6 MiB/s wr, 987 op/s 2024-05-09T09:25:33.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:32 smithi139 ceph-mon[33599]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 8.6 MiB/s wr, 987 op/s 2024-05-09T09:25:34.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:34 smithi179 ceph-mon[34086]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 9.6 MiB/s wr, 1.08k op/s 2024-05-09T09:25:35.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:34 smithi139 ceph-mon[33599]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 9.6 MiB/s wr, 1.08k op/s 2024-05-09T09:25:35.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:34 smithi064 ceph-mon[30149]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 9.6 MiB/s wr, 1.08k op/s 2024-05-09T09:25:35.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:35 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:25:35.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:35 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:25:35.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:35 smithi179 ceph-mon[34086]: scrub summary: idle 2024-05-09T09:25:36.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:35 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:25:36.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:35 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:25:36.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:35 smithi139 ceph-mon[33599]: scrub summary: idle 2024-05-09T09:25:36.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:35 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:25:36.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:35 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:25:36.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:35 smithi064 ceph-mon[30149]: scrub summary: idle 2024-05-09T09:25:36.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:36 smithi179 ceph-mon[34086]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.9 MiB/s wr, 955 op/s 2024-05-09T09:25:37.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:36 smithi139 ceph-mon[33599]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.9 MiB/s wr, 955 op/s 2024-05-09T09:25:37.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:36 smithi064 ceph-mon[30149]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.9 MiB/s wr, 955 op/s 2024-05-09T09:25:38.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:38 smithi179 ceph-mon[34086]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.8 MiB/s wr, 809 op/s 2024-05-09T09:25:39.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:38 smithi139 ceph-mon[33599]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.8 MiB/s wr, 809 op/s 2024-05-09T09:25:39.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:38 smithi064 ceph-mon[30149]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.8 MiB/s wr, 809 op/s 2024-05-09T09:25:40.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:40 smithi179 ceph-mon[34086]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 6.4 MiB/s wr, 753 op/s 2024-05-09T09:25:41.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:40 smithi139 ceph-mon[33599]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 6.4 MiB/s wr, 753 op/s 2024-05-09T09:25:41.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:40 smithi064 ceph-mon[30149]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 6.4 MiB/s wr, 753 op/s 2024-05-09T09:25:42.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:42 smithi179 ceph-mon[34086]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-09T09:25:43.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:42 smithi139 ceph-mon[33599]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-09T09:25:43.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:42 smithi064 ceph-mon[30149]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-09T09:25:44.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:44 smithi179 ceph-mon[34086]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-05-09T09:25:45.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:44 smithi139 ceph-mon[33599]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-05-09T09:25:45.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:44 smithi064 ceph-mon[30149]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-05-09T09:25:46.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:46 smithi179 ceph-mon[34086]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 631 KiB/s wr, 78 op/s 2024-05-09T09:25:47.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:46 smithi139 ceph-mon[33599]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 631 KiB/s wr, 78 op/s 2024-05-09T09:25:47.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:46 smithi064 ceph-mon[30149]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 631 KiB/s wr, 78 op/s 2024-05-09T09:25:49.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:48 smithi139 ceph-mon[33599]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-09T09:25:49.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:48 smithi064 ceph-mon[30149]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-09T09:25:49.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:48 smithi179 ceph-mon[34086]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-09T09:25:51.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:50 smithi139 ceph-mon[33599]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-05-09T09:25:51.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:50 smithi064 ceph-mon[30149]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-05-09T09:25:51.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:50 smithi179 ceph-mon[34086]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-05-09T09:25:53.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:52 smithi139 ceph-mon[33599]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-05-09T09:25:53.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:52 smithi064 ceph-mon[30149]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-05-09T09:25:53.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:52 smithi179 ceph-mon[34086]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-05-09T09:25:54.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-09T09:25:54.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-09T09:25:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 -- 172.21.15.64:0/24558276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 msgr2=0x7fdc78107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/24558276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78107050 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009900 tx=0x7fdc6c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 -- 172.21.15.64:0/24558276 shutdown_connections 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/24558276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78107590 0x7fdc781099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/24558276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78107050 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/24558276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc78102330 0x7fdc78104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 -- 172.21.15.64:0/24558276 >> 172.21.15.64:0/24558276 conn(0x7fdc780fbce0 msgr2=0x7fdc780fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 -- 172.21.15.64:0/24558276 shutdown_connections 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.150+0000 7fdc7d05d640 1 -- 172.21.15.64:0/24558276 wait complete. 2024-05-09T09:25:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 Processor -- start 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 -- start start 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78102330 0x7fdc78197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc78107590 0x7fdc7819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7810d3a0 con 0x7fdc78104c60 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7810d220 con 0x7fdc78107590 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.151+0000 7fdc7d05d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7810d520 con 0x7fdc78102330 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51074/0 (socket says 172.21.15.64:51074) 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 -- 172.21.15.64:0/3842901074 learned_addr learned my addr 172.21.15.64:0/3842901074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78102330 0x7fdc78197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7759e640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc78107590 0x7fdc7819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 -- 172.21.15.64:0/3842901074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78102330 msgr2=0x7fdc78197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78102330 0x7fdc78197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 -- 172.21.15.64:0/3842901074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc78107590 msgr2=0x7fdc7819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc78107590 0x7fdc7819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 -- 172.21.15.64:0/3842901074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7819f750 con 0x7fdc78104c60 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78102330 0x7fdc78197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc7659c640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78198070 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0314e0 tx=0x7fdc6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.152+0000 7fdc5ffff640 1 -- 172.21.15.64:0/3842901074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c03f030 con 0x7fdc78104c60 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.153+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7819f9e0 con 0x7fdc78104c60 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.153+0000 7fdc5ffff640 1 -- 172.21.15.64:0/3842901074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc6c003aa0 con 0x7fdc78104c60 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.153+0000 7fdc5ffff640 1 -- 172.21.15.64:0/3842901074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c035070 con 0x7fdc78104c60 2024-05-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.153+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7819fed0 con 0x7fdc78104c60 2024-05-09T09:25:54.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc5ffff640 1 -- 172.21.15.64:0/3842901074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc6c042030 con 0x7fdc78104c60 2024-05-09T09:25:54.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 Processor -- start 2024-05-09T09:25:54.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 -- start start 2024-05-09T09:25:54.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5652e5251d00 con 0x5652e52ecf30 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5652e51f94c0 con 0x5652e52f2a90 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5652e524dcc0 con 0x5652e52ed920 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.155+0000 7fdc7759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc7759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52ed350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51090/0 (socket says 172.21.15.64:51090) 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc7759e640 1 -- 172.21.15.64:0/3571784759 learned_addr learned my addr 172.21.15.64:0/3571784759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc7659c640 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc5ffff640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc600a2b90 0x7fdc600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 -- 172.21.15.64:0/3571784759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 msgr2=0x5652e52f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 -- 172.21.15.64:0/3571784759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 msgr2=0x5652e52ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 -- 172.21.15.64:0/3571784759 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e502f7d0 con 0x5652e52ed920 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc7759e640 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc7759e640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc600a2b90 0x7fdc600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc5ffff640 1 -- 172.21.15.64:0/3842901074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc6c0efa90 con 0x7fdc78104c60 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f2420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fd0 tx=0x7fdc680118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=416fb59c943ade75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc5d7fa640 1 -- 172.21.15.64:0/3571784759 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004030 con 0x5652e52ed920 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc5d7fa640 1 -- 172.21.15.64:0/3571784759 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc680054d0 con 0x5652e52ed920 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc7759e640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc600a2b90 0x7fdc600a5050 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x5652e52ee180 tx=0x7fdc70004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.156+0000 7fdc5d7fa640 1 -- 172.21.15.64:0/3571784759 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004c40 con 0x5652e52ed920 2024-05-09T09:25:54.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/3571784759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 msgr2=0x5652e52f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f2420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fd0 tx=0x7fdc680118d0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/3571784759 shutdown_connections 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f2420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/3571784759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/3571784759 >> 172.21.15.64:0/3571784759 conn(0x5652e52e85a0 msgr2=0x5652e52e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/3571784759 shutdown_connections 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/3571784759 wait complete. 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.157+0000 7fdc7fd6a740 1 -- start start 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 Processor -- start 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52f7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5652e52f5a40 con 0x5652e52ecf30 2024-05-09T09:25:54.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5652e520cb90 con 0x5652e52f2a90 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5652e52f5bc0 con 0x5652e52ed920 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7fd6a740 10 client.0 _ll_register_callbacks cb 0x5652e5248da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52f7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51100/0 (socket says 172.21.15.64:51100) 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 -- 172.21.15.64:0/2247925692 learned_addr learned my addr 172.21.15.64:0/2247925692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7759e640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 -- 172.21.15.64:0/2247925692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 msgr2=0x5652e52f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 -- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 msgr2=0x5652e52fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc76d9d640 1 -- 172.21.15.64:0/2247925692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e4f653f0 con 0x5652e52ecf30 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.158+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52f7330 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fa0 tx=0x7fdc6800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f9cda0ce7042bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:54.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2247925692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68029020 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2247925692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc68025680 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc680054d0 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2247925692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdc68034050 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 10 client.19407 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5652e4f250f0 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2247925692 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc680054d0 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2247925692 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7fdc680049b0 con 0x5652e52ecf30 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 10 client.19407 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 10 client.19407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 20 client.19407 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e5302fc0 0x5652e5305460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 4 client.19407 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.159+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652e525f430 con 0x5652e5302fc0 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.160+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e5302fc0 0x5652e5305460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.160+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e5302fc0 0x5652e5305460 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=840a12afd27b8836 server_cookie=9261ec136c1e99d5 in_seq=0 out_seq=0 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.160+0000 7fdc5aff5640 10 client.19407 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2247925692 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652e525f430 con 0x5652e5302fc0 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc5aff5640 10 client.19407 handle_command_reply: tid=0 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 client.19407 shutdown 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 msgr2=0x5652e52f7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52f7330 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fa0 tx=0x7fdc6800fc80 comp rx=0 tx=0).stop 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 shutdown_connections 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e5302fc0 0x5652e5305460 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ed920 0x5652e52f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc5aff5640 0 client.19407 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2247925692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ecf30 0x5652e52f7330 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 >> 172.21.15.64:0/2247925692 conn(0x5652e52e85a0 msgr2=0x5652e52e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 shutdown_connections 2024-05-09T09:25:54.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2247925692 wait complete. 2024-05-09T09:25:54.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.161+0000 7fdc7fd6a740 20 client.19407 trim_cache size 0 max 16384 2024-05-09T09:25:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 Processor -- start 2024-05-09T09:25:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 -- start start 2024-05-09T09:25:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ecf30 0x5652e52f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781ace20 con 0x5652e52ed920 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781aa630 con 0x5652e52f2a90 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781a8480 con 0x5652e52ecf30 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51108/0 (socket says 172.21.15.64:51108) 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 -- 172.21.15.64:0/2712148122 learned_addr learned my addr 172.21.15.64:0/2712148122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ecf30 0x5652e52f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7759e640 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 -- 172.21.15.64:0/2712148122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ecf30 msgr2=0x5652e52f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ecf30 0x5652e52f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 -- 172.21.15.64:0/2712148122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 msgr2=0x5652e52f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 -- 172.21.15.64:0/2712148122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e52f5920 con 0x5652e52ed920 2024-05-09T09:25:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ecf30 0x5652e52f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.179+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fe870 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0b8420 tx=0x7fdc6c0ed0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5268e17c7db53e9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc5cff9640 1 -- 172.21.15.64:0/2712148122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c049040 con 0x5652e52ed920 2024-05-09T09:25:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc5cff9640 1 -- 172.21.15.64:0/2712148122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc6c0bc010 con 0x5652e52ed920 2024-05-09T09:25:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc5cff9640 1 -- 172.21.15.64:0/2712148122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0b8680 con 0x5652e52ed920 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2712148122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 msgr2=0x5652e52fe870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fe870 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0b8420 tx=0x7fdc6c0ed0d0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2712148122 shutdown_connections 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52f2a90 0x5652e52f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fe870 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2712148122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52ecf30 0x5652e52f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.180+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2712148122 >> 172.21.15.64:0/2712148122 conn(0x5652e52e85a0 msgr2=0x5652e52e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2712148122 shutdown_connections 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2712148122 wait complete. 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 -- start start 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 Processor -- start 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52ecf30 0x5652e52fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 0x5652e53675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781a8af0 con 0x5652e52ed920 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781a9f60 con 0x5652e52ecf30 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781a91c0 con 0x5652e52f2a90 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7fd6a740 10 client.0 _ll_register_callbacks cb 0x7fdc70009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 0x5652e53675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51112/0 (socket says 172.21.15.64:51112) 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 0x5652e53675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.64:50364/0 (socket says 172.21.15.64:50364) 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc76d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52ecf30 0x5652e52fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.181+0000 7fdc7659c640 1 -- 172.21.15.64:0/2325446731 learned_addr learned my addr 172.21.15.64:0/2325446731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7759e640 1 -- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52ecf30 msgr2=0x5652e52fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7759e640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52ecf30 0x5652e52fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7759e640 1 -- 172.21.15.64:0/2325446731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 msgr2=0x5652e52fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7759e640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7759e640 1 -- 172.21.15.64:0/2325446731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e5367bc0 con 0x5652e52f2a90 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7659c640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52ecf30 0x5652e52fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7759e640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 0x5652e53675d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc70004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40444d358741fa7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2325446731 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70004b20 con 0x5652e52f2a90 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6c0bc010 con 0x5652e52f2a90 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2325446731 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdc70004cc0 con 0x5652e52f2a90 2024-05-09T09:25:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2325446731 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdc70037050 con 0x5652e52f2a90 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7fd6a740 10 client.29044 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.182+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5652e5368f30 con 0x5652e52f2a90 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2325446731 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc6c0bc010 con 0x5652e52f2a90 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2325446731 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7fdc700053d0 con 0x5652e52f2a90 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc7fd6a740 10 client.29044 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc7fd6a740 10 client.29044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc7fd6a740 20 client.29044 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e53699f0 0x5652e536be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc7fd6a740 4 client.29044 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652e5259780 con 0x5652e53699f0 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e53699f0 0x5652e536be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc76d9d640 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e53699f0 0x5652e536be10 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374e6da71ec8e34d server_cookie=b2ad1236b65c2d5e in_seq=0 out_seq=0 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc5aff5640 10 client.29044 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.183+0000 7fdc5aff5640 1 -- 172.21.15.64:0/2325446731 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652e5259780 con 0x5652e53699f0 2024-05-09T09:25:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc5aff5640 10 client.29044 handle_command_reply: tid=0 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 client.29044 shutdown 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 msgr2=0x5652e53675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 0x5652e53675d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc70004ab0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 shutdown_connections 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5652e53699f0 0x5652e536be10 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652e52f2a90 0x5652e53675d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e52ed920 0x5652e52fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc5aff5640 0 client.29044 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 --2- 172.21.15.64:0/2325446731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5652e52ecf30 0x5652e52fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 >> 172.21.15.64:0/2325446731 conn(0x5652e52e85a0 msgr2=0x5652e52eb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 shutdown_connections 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 1 -- 172.21.15.64:0/2325446731 wait complete. 2024-05-09T09:25:54.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7fd6a740 20 client.29044 trim_cache size 0 max 16384 2024-05-09T09:25:54.190 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:25:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-05-09T09:25:54.190 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-05-09T09:25:54.190 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc600a2b90 msgr2=0x7fdc600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc600a2b90 0x7fdc600a5050 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x5652e52ee180 tx=0x7fdc70004a40 comp rx=0 tx=0).stop 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 msgr2=0x7fdc78198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.184+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78198070 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0314e0 tx=0x7fdc6c004270 comp rx=0 tx=0).stop 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 shutdown_connections 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fdc600a2b90 0x7fdc600a5050 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc78107590 0x7fdc7819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc78104c60 0x7fdc78198070 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 --2- 172.21.15.64:0/3842901074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc78102330 0x7fdc78197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 >> 172.21.15.64:0/3842901074 conn(0x7fdc780fbce0 msgr2=0x7fdc78104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 shutdown_connections 2024-05-09T09:25:54.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.185+0000 7fdc7d05d640 1 -- 172.21.15.64:0/3842901074 wait complete. 2024-05-09T09:25:54.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T09:25:54.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 -- 172.21.15.64:0/3123606889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4100770 msgr2=0x7f14d4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/3123606889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4100770 0x7f14d4100b90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 -- 172.21.15.64:0/3123606889 shutdown_connections 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/3123606889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d4102320 0x7f14d4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/3123606889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d4101960 0x7f14d4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/3123606889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4100770 0x7f14d4100b90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 -- 172.21.15.64:0/3123606889 >> 172.21.15.64:0/3123606889 conn(0x7f14d40fbce0 msgr2=0x7f14d40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 -- 172.21.15.64:0/3123606889 shutdown_connections 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.303+0000 7f14d8f5a640 1 -- 172.21.15.64:0/3123606889 wait complete. 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 Processor -- start 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 -- start start 2024-05-09T09:25:54.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d4100770 0x7f14d4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 0x7f14d4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d4102320 0x7f14d419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f14d410c2a0 con 0x7f14d4101960 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14d410c120 con 0x7f14d4102320 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d8f5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14d410c420 con 0x7f14d4100770 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 0x7f14d4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 0x7f14d4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51130/0 (socket says 172.21.15.64:51130) 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 -- 172.21.15.64:0/563198524 learned_addr learned my addr 172.21.15.64:0/563198524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d259c640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d4100770 0x7f14d4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d2d9d640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d4102320 0x7f14d419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 -- 172.21.15.64:0/563198524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d4100770 msgr2=0x7f14d4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d4100770 0x7f14d4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 -- 172.21.15.64:0/563198524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d4102320 msgr2=0x7f14d419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d4102320 0x7f14d419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 -- 172.21.15.64:0/563198524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d419f6e0 con 0x7f14d4101960 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.304+0000 7f14d1d9b640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 0x7f14d4198000 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00e9f0 tx=0x7f14bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.305+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00cd80 con 0x7f14d4101960 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.305+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d419f9d0 con 0x7f14d4101960 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.305+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14bc0108e0 con 0x7f14d4101960 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.305+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc018920 con 0x7f14d4101960 2024-05-09T09:25:54.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.305+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d419ff70 con 0x7f14d4101960 2024-05-09T09:25:54.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.306+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d41a01a0 con 0x7f14d4101960 2024-05-09T09:25:54.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.307+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14bc010a90 con 0x7f14d4101960 2024-05-09T09:25:54.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.307+0000 7f14cb7fe640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f14c00a2b90 0x7f14c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.308+0000 7f14d259c640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f14c00a2b90 0x7f14c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.308+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f14bc0cf6a0 con 0x7f14d4101960 2024-05-09T09:25:54.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.308+0000 7f14d259c640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f14c00a2b90 0x7f14c00a5050 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f14c4004770 tx=0x7f14c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.312+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14bc09bc20 con 0x7f14d4101960 2024-05-09T09:25:54.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.566+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14d41a0370 con 0x7f14d4101960 2024-05-09T09:25:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.567+0000 7f14cb7fe640 1 -- 172.21.15.64:0/563198524 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11888 (secure 0 0 0) 0x7f14bc086090 con 0x7f14d4101960 2024-05-09T09:25:54.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:54.567 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:25:00.568509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:25:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 53 2024-05-09T09:25:54.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f14c00a2b90 msgr2=0x7f14c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f14c00a2b90 0x7f14c00a5050 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f14c4004770 tx=0x7f14c4000bc0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 msgr2=0x7f14d4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 0x7f14d4198000 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00e9f0 tx=0x7f14bc00eeb0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 shutdown_connections 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f14c00a2b90 0x7f14c00a5050 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d4102320 0x7f14d419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d4101960 0x7f14d4198000 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 --2- 172.21.15.64:0/563198524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d4100770 0x7f14d4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 >> 172.21.15.64:0/563198524 conn(0x7f14d40fbce0 msgr2=0x7f14d40689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 shutdown_connections 2024-05-09T09:25:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.570+0000 7f14d8f5a640 1 -- 172.21.15.64:0/563198524 wait complete. 2024-05-09T09:25:54.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 -- 172.21.15.64:0/857180865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c104c40 msgr2=0x7fa45c107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 --2- 172.21.15.64:0/857180865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c104c40 0x7fa45c107030 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa450009900 tx=0x7fa45002a410 comp rx=0 tx=0).stop 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 -- 172.21.15.64:0/857180865 shutdown_connections 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 --2- 172.21.15.64:0/857180865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa45c107570 0x7fa45c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 --2- 172.21.15.64:0/857180865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c104c40 0x7fa45c107030 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 --2- 172.21.15.64:0/857180865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c102310 0x7fa45c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 -- 172.21.15.64:0/857180865 >> 172.21.15.64:0/857180865 conn(0x7fa45c0fbd20 msgr2=0x7fa45c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 -- 172.21.15.64:0/857180865 shutdown_connections 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.688+0000 7fa461d09640 1 -- 172.21.15.64:0/857180865 wait complete. 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 Processor -- start 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 -- start start 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa45c102310 0x7fa45c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c104c40 0x7fa45c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 0x7fa45c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c10d3d0 con 0x7fa45c107570 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c10d250 con 0x7fa45c102310 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa461d09640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c10d550 con 0x7fa45c104c40 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 0x7fa45c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 0x7fa45c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48852/0 (socket says 172.21.15.64:48852) 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 -- 172.21.15.64:0/2602793453 learned_addr learned my addr 172.21.15.64:0/2602793453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45affd640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c104c40 0x7fa45c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 -- 172.21.15.64:0/2602793453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c104c40 msgr2=0x7fa45c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c104c40 0x7fa45c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 -- 172.21.15.64:0/2602793453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa45c102310 msgr2=0x7fa45c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa45c102310 0x7fa45c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45bfff640 1 -- 172.21.15.64:0/2602793453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c19f740 con 0x7fa45c107570 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.689+0000 7fa45affd640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c104c40 0x7fa45c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.690+0000 7fa45bfff640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 0x7fa45c19f0d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa45400eb10 tx=0x7fa45400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.690+0000 7fa458ff9640 1 -- 172.21.15.64:0/2602793453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454004b20 con 0x7fa45c107570 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.690+0000 7fa458ff9640 1 -- 172.21.15.64:0/2602793453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa454004cc0 con 0x7fa45c107570 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.690+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c19fa30 con 0x7fa45c107570 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.690+0000 7fa458ff9640 1 -- 172.21.15.64:0/2602793453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454010690 con 0x7fa45c107570 2024-05-09T09:25:54.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.690+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c19ff50 con 0x7fa45c107570 2024-05-09T09:25:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 Processor -- start 2024-05-09T09:25:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa458ff9640 1 -- 172.21.15.64:0/2602793453 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa454005020 con 0x7fa45c107570 2024-05-09T09:25:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 -- start start 2024-05-09T09:25:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374350 0x561b37374770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 0x561b37379840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b3737c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561b372d4c40 con 0x561b37374d40 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561b3727c580 con 0x561b37379eb0 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.693+0000 7fa464a10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561b372d0c00 con 0x561b37374350 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 0x561b37379840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 0x561b37379840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48860/0 (socket says 172.21.15.64:48860) 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 -- 172.21.15.64:0/2345622325 learned_addr learned my addr 172.21.15.64:0/2345622325 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45affd640 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374350 0x561b37374770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 -- 172.21.15.64:0/2345622325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374350 msgr2=0x561b37374770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374350 0x561b37374770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45affd640 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b3737c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 -- 172.21.15.64:0/2345622325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 msgr2=0x561b3737c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b3737c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 -- 172.21.15.64:0/2345622325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b36fe83f0 con 0x561b37374d40 2024-05-09T09:25:54.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45affd640 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b3737c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa458ff9640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4440a2c10 0x7fa4440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 0x561b37379840 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009c40 tx=0x7fa44c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf503727faad5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa45affd640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4440a2c10 0x7fa4440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa458ff9640 1 -- 172.21.15.64:0/2602793453 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4540cfa50 con 0x7fa45c107570 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa441ffb640 1 -- 172.21.15.64:0/2345622325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c005020 con 0x561b37374d40 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa441ffb640 1 -- 172.21.15.64:0/2345622325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa44c0047a0 con 0x561b37374d40 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.694+0000 7fa441ffb640 1 -- 172.21.15.64:0/2345622325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c010690 con 0x561b37374d40 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 -- 172.21.15.64:0/2345622325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 msgr2=0x561b37379840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa45affd640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4440a2c10 0x7fa4440a50d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa450000c00 tx=0x7fa4500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 0x561b37379840 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009c40 tx=0x7fa44c00e720 comp rx=0 tx=0).stop 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 -- 172.21.15.64:0/2345622325 shutdown_connections 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b3737c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374d40 0x561b37379840 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 --2- 172.21.15.64:0/2345622325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374350 0x561b37374770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 -- 172.21.15.64:0/2345622325 >> 172.21.15.64:0/2345622325 conn(0x561b3736b5b0 msgr2=0x561b3736b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 -- 172.21.15.64:0/2345622325 shutdown_connections 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 -- 172.21.15.64:0/2345622325 wait complete. 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.695+0000 7fa464a10740 1 -- start start 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 Processor -- start 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37374350 0x561b3737e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3737ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 0x561b37385c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561b3737ce60 con 0x561b37379eb0 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561b3728fc40 con 0x561b37374350 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561b3737cfe0 con 0x561b37374d40 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa464a10740 10 client.0 _ll_register_callbacks cb 0x561b36fe7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 0x561b37385c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 0x561b37385c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48866/0 (socket says 172.21.15.64:48866) 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 -- 172.21.15.64:0/2226993964 learned_addr learned my addr 172.21.15.64:0/2226993964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37374350 0x561b3737e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45affd640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3737ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 -- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 msgr2=0x561b3737ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3737ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 -- 172.21.15.64:0/2226993964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37374350 msgr2=0x561b3737e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37374350 0x561b3737e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 -- 172.21.15.64:0/2226993964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b370b27d0 con 0x561b37379eb0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa45bfff640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 0x561b37385c90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa45409beb0 tx=0x7fa45409b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665308fc919b7fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.696+0000 7fa43affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa43affd640 1 -- 172.21.15.64:0/2226993964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4540a02d0 con 0x561b37379eb0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa43affd640 1 -- 172.21.15.64:0/2226993964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4540a4700 con 0x561b37379eb0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa44c0047a0 con 0x561b37379eb0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa43affd640 1 -- 172.21.15.64:0/2226993964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa4540a4b90 con 0x561b37379eb0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 10 client.19431 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x561b37389030 con 0x561b37379eb0 2024-05-09T09:25:54.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa43affd640 1 -- 172.21.15.64:0/2226993964 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa44c0047a0 con 0x561b37379eb0 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa43affd640 1 -- 172.21.15.64:0/2226993964 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7fa45409f070 con 0x561b37379eb0 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 10 client.19431 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 10 client.19431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 20 client.19431 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b3738a760 0x561b3738cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 4 client.19431 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.697+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b372e2370 con 0x561b3738a760 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.698+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b3738a760 0x561b3738cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.698+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b3738a760 0x561b3738cbc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9316d6313437a13b server_cookie=26e7464717d150ba in_seq=0 out_seq=0 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.698+0000 7fa43affd640 10 client.19431 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa43affd640 1 -- 172.21.15.64:0/2226993964 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b372e2370 con 0x561b3738a760 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa43affd640 10 client.19431 handle_command_reply: tid=0 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 client.19431 shutdown 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 msgr2=0x561b37385c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 0x561b37385c90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa45409beb0 tx=0x7fa45409b940 comp rx=0 tx=0).stop 2024-05-09T09:25:54.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 shutdown_connections 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b3738a760 0x561b3738cbc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37379eb0 0x561b37385c90 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa43affd640 0 client.19431 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3737ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 --2- 172.21.15.64:0/2226993964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37374350 0x561b3737e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 >> 172.21.15.64:0/2226993964 conn(0x561b3736b5b0 msgr2=0x561b3736a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 shutdown_connections 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 1 -- 172.21.15.64:0/2226993964 wait complete. 2024-05-09T09:25:54.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.699+0000 7fa464a10740 20 client.19431 trim_cache size 0 max 16384 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.712+0000 7fa464a10740 1 Processor -- start 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.712+0000 7fa464a10740 1 -- start start 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa464a10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b3736a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa464a10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3736ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa464a10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37385c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa464a10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1aaa80 con 0x561b37374350 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa464a10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1a8290 con 0x561b37379eb0 2024-05-09T09:25:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa464a10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1a60e0 con 0x561b37374d40 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b3736a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b3736a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48874/0 (socket says 172.21.15.64:48874) 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 -- 172.21.15.64:0/393856202 learned_addr learned my addr 172.21.15.64:0/393856202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45affd640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3736ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45bfff640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37385c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 -- 172.21.15.64:0/393856202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 msgr2=0x561b3736ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3736ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 -- 172.21.15.64:0/393856202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 msgr2=0x561b37385c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37385c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 -- 172.21.15.64:0/393856202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3736b0f0 con 0x561b37374350 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45bfff640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37385c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45affd640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3736ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b3736a630 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009c40 tx=0x7fa44c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb35ab83d47c4ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa4417fa640 1 -- 172.21.15.64:0/393856202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c01d040 con 0x561b37374350 2024-05-09T09:25:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa4417fa640 1 -- 172.21.15.64:0/393856202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa44c02e070 con 0x561b37374350 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.713+0000 7fa4417fa640 1 -- 172.21.15.64:0/393856202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004d10 con 0x561b37374350 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 -- 172.21.15.64:0/393856202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 msgr2=0x561b3736a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b3736a630 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009c40 tx=0x7fa44c00ecd0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 -- 172.21.15.64:0/393856202 shutdown_connections 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37385c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b3736ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 --2- 172.21.15.64:0/393856202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b3736a630 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 -- 172.21.15.64:0/393856202 >> 172.21.15.64:0/393856202 conn(0x561b3736b5b0 msgr2=0x561b37369f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 -- 172.21.15.64:0/393856202 shutdown_connections 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 -- 172.21.15.64:0/393856202 wait complete. 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.714+0000 7fa464a10740 1 -- start start 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 Processor -- start 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b37383400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b37383940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37384900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1a6750 con 0x561b37374350 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1a7bc0 con 0x561b37379eb0 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1a6e20 con 0x561b37374d40 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 10 client.0 _ll_register_callbacks cb 0x7fa454004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b37383400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b37383400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48884/0 (socket says 172.21.15.64:48884) 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 -- 172.21.15.64:0/1883536780 learned_addr learned my addr 172.21.15.64:0/1883536780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45affd640 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b37383940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 -- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 msgr2=0x561b37383940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b37383940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 -- 172.21.15.64:0/1883536780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 msgr2=0x561b37384900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37384900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 -- 172.21.15.64:0/1883536780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b37384ef0 con 0x561b37374350 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa45b7fe640 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b37383400 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009670 tx=0x7fa44c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dc15796e9f229d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa43affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa43affd640 1 -- 172.21.15.64:0/1883536780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004a90 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa43affd640 1 -- 172.21.15.64:0/1883536780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa44c004c30 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.715+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa44c02e070 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa43affd640 1 -- 172.21.15.64:0/1883536780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa44c004040 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 10 client.19437 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x561b373f0d60 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa43affd640 1 -- 172.21.15.64:0/1883536780 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa44c02e070 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa43affd640 1 -- 172.21.15.64:0/1883536780 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7fa44c004fd0 con 0x561b37374350 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 10 client.19437 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 10 client.19437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 20 client.19437 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b373f15d0 0x561b373f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 4 client.19437 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:25:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 --> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b372dc6c0 con 0x561b373f15d0 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.716+0000 7fa45affd640 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b373f15d0 0x561b373f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa45affd640 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b373f15d0 0x561b373f39f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28977df1c13da93e server_cookie=38203aebb7788917 in_seq=0 out_seq=0 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa43affd640 10 client.19437 ms_handle_connect on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa43affd640 1 -- 172.21.15.64:0/1883536780 <== mds.2 v2:172.21.15.179:6836/2858892786 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b372dc6c0 con 0x561b373f15d0 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa43affd640 10 client.19437 handle_command_reply: tid=0 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 client.19437 shutdown 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 msgr2=0x561b37383400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b37383400 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009670 tx=0x7fa44c0096a0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 shutdown_connections 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:6836/2858892786,v1:172.21.15.179:6837/2858892786] conn(0x561b373f15d0 0x561b373f39f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b37379eb0 0x561b37384900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b37374d40 0x561b37383940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 --2- 172.21.15.64:0/1883536780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b37374350 0x561b37383400 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa43affd640 0 client.19437 ms_handle_reset on v2:172.21.15.179:6836/2858892786 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 >> 172.21.15.64:0/1883536780 conn(0x561b3736b5b0 msgr2=0x561b3736a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 shutdown_connections 2024-05-09T09:25:54.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 1 -- 172.21.15.64:0/1883536780 wait complete. 2024-05-09T09:25:54.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.717+0000 7fa464a10740 20 client.19437 trim_cache size 0 max 16384 2024-05-09T09:25:54.724 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4440a2c10 msgr2=0x7fa4440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4440a2c10 0x7fa4440a50d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa450000c00 tx=0x7fa4500314a0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 msgr2=0x7fa45c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 0x7fa45c19f0d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa45400eb10 tx=0x7fa45400c2d0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 shutdown_connections 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa4440a2c10 0x7fa4440a50d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa45c107570 0x7fa45c19f0d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c104c40 0x7fa45c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 --2- 172.21.15.64:0/2602793453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa45c102310 0x7fa45c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 >> 172.21.15.64:0/2602793453 conn(0x7fa45c0fbd20 msgr2=0x7fa45c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 shutdown_connections 2024-05-09T09:25:54.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.718+0000 7fa461d09640 1 -- 172.21.15.64:0/2602793453 wait complete. 2024-05-09T09:25:54.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 -- 172.21.15.64:0/1733269867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff920104c20 msgr2=0x7ff920107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 --2- 172.21.15.64:0/1733269867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff920104c20 0x7ff920107010 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 -- 172.21.15.64:0/1733269867 shutdown_connections 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 --2- 172.21.15.64:0/1733269867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920107550 0x7ff9201099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 --2- 172.21.15.64:0/1733269867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff920104c20 0x7ff920107010 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 --2- 172.21.15.64:0/1733269867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9201022f0 0x7ff9201046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.835+0000 7ff92617e640 1 -- 172.21.15.64:0/1733269867 >> 172.21.15.64:0/1733269867 conn(0x7ff9200fbce0 msgr2=0x7ff9200fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 -- 172.21.15.64:0/1733269867 shutdown_connections 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 -- 172.21.15.64:0/1733269867 wait complete. 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 Processor -- start 2024-05-09T09:25:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 -- start start 2024-05-09T09:25:54.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 0x7ff920197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920104c20 0x7ff920198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff920107550 0x7ff92019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.836+0000 7ff92617e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff92010d390 con 0x7ff9201022f0 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff92617e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff92010d210 con 0x7ff920107550 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff92617e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff92010d510 con 0x7ff920104c20 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 0x7ff920197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 0x7ff920197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48916/0 (socket says 172.21.15.64:48916) 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 -- 172.21.15.64:0/83024783 learned_addr learned my addr 172.21.15.64:0/83024783 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91ffff640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff920107550 0x7ff92019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91effd640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920104c20 0x7ff920198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 -- 172.21.15.64:0/83024783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920104c20 msgr2=0x7ff920198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920104c20 0x7ff920198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 -- 172.21.15.64:0/83024783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff920107550 msgr2=0x7ff92019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff920107550 0x7ff92019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 -- 172.21.15.64:0/83024783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92019f770 con 0x7ff9201022f0 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91effd640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920104c20 0x7ff920198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 0x7ff920197b50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff91000eab0 tx=0x7ff91000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91cff9640 1 -- 172.21.15.64:0/83024783 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91000ce40 con 0x7ff9201022f0 2024-05-09T09:25:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91cff9640 1 -- 172.21.15.64:0/83024783 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff910011920 con 0x7ff9201022f0 2024-05-09T09:25:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92019fa60 con 0x7ff9201022f0 2024-05-09T09:25:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff91cff9640 1 -- 172.21.15.64:0/83024783 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910010a00 con 0x7ff9201022f0 2024-05-09T09:25:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.837+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92019ffd0 con 0x7ff9201022f0 2024-05-09T09:25:54.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91cff9640 1 -- 172.21.15.64:0/83024783 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff910011ad0 con 0x7ff9201022f0 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 Processor -- start 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 -- start start 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7f3eb0 0x55621e7f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55621e74ec40 con 0x55621e7eed40 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55621e6f6580 con 0x55621e7f3eb0 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff928f6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55621e74ac00 con 0x55621e7ee350 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91cff9640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2c10 0x7ff9080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48924/0 (socket says 172.21.15.64:48924) 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91f7fe640 1 -- 172.21.15.64:0/1346151620 learned_addr learned my addr 172.21.15.64:0/1346151620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91ffff640 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.840+0000 7ff91ffff640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2c10 0x7ff9080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91effd640 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7f3eb0 0x55621e7f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91cff9640 1 -- 172.21.15.64:0/83024783 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff9100cf9f0 con 0x7ff9201022f0 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91f7fe640 1 -- 172.21.15.64:0/1346151620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 msgr2=0x55621e7ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91f7fe640 1 -- 172.21.15.64:0/1346151620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7f3eb0 msgr2=0x55621e7f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7f3eb0 0x55621e7f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91f7fe640 1 -- 172.21.15.64:0/1346151620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621e4623f0 con 0x55621e7eed40 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91ffff640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2c10 0x7ff9080a50d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55621e7ef5a0 tx=0x7ff918005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f3840 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff9100cd0a0 tx=0x7ff9100094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2270c6712a25d64a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff905ffb640 1 -- 172.21.15.64:0/1346151620 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91009c5c0 con 0x55621e7eed40 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff905ffb640 1 -- 172.21.15.64:0/1346151620 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9100a0040 con 0x55621e7eed40 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.841+0000 7ff905ffb640 1 -- 172.21.15.64:0/1346151620 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a46a0 con 0x55621e7eed40 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 -- 172.21.15.64:0/1346151620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 msgr2=0x55621e7f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f3840 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff9100cd0a0 tx=0x7ff9100094a0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 -- 172.21.15.64:0/1346151620 shutdown_connections 2024-05-09T09:25:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7f3eb0 0x55621e7f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f3840 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 --2- 172.21.15.64:0/1346151620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 -- 172.21.15.64:0/1346151620 >> 172.21.15.64:0/1346151620 conn(0x55621e7e99c0 msgr2=0x55621e7e9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 -- 172.21.15.64:0/1346151620 shutdown_connections 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 -- 172.21.15.64:0/1346151620 wait complete. 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 -- start start 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.842+0000 7ff928f6c740 1 Processor -- start 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55621e7f6da0 con 0x55621e7f3eb0 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55621e709c40 con 0x55621e7eed40 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55621e7f6f20 con 0x55621e7ee350 2024-05-09T09:25:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff928f6c740 10 client.0 _ll_register_callbacks cb 0x55621e461be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48936/0 (socket says 172.21.15.64:48936) 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 -- 172.21.15.64:0/567426770 learned_addr learned my addr 172.21.15.64:0/567426770 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 -- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 msgr2=0x55621e7f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 -- 172.21.15.64:0/567426770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 msgr2=0x55621e7f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 -- 172.21.15.64:0/567426770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621e52c7d0 con 0x55621e7f3eb0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91effd640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff91ffff640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff91800f6a0 tx=0x7ff9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa66a8f3ca45419 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff8feffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.843+0000 7ff8feffd640 1 -- 172.21.15.64:0/567426770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918025070 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9100a0040 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff8feffd640 1 -- 172.21.15.64:0/567426770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff91801e8d0 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff8feffd640 1 -- 172.21.15.64:0/567426770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff91801ead0 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 10 client.19449 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55621e8030f0 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff8feffd640 1 -- 172.21.15.64:0/567426770 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9100a0040 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff8feffd640 1 -- 172.21.15.64:0/567426770 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7ff91801f3f0 con 0x55621e7f3eb0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 10 client.19449 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 10 client.19449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 20 client.19449 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e8047b0 0x55621e806be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 4 client.19449 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55621e75c370 con 0x55621e8047b0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.844+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e8047b0 0x55621e806be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.845+0000 7ff8feffd640 10 client.19449 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.845+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e8047b0 0x55621e806be0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d74ea09978ec6636 server_cookie=7bc3c6cc7553e3cf in_seq=0 out_seq=0 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff8feffd640 1 -- 172.21.15.64:0/567426770 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55621e75c370 con 0x55621e8047b0 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff8feffd640 10 client.19449 handle_command_reply: tid=0 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 client.19449 shutdown 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 msgr2=0x55621e7ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff91800f6a0 tx=0x7ff9180046c0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 shutdown_connections 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e8047b0 0x55621e806be0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc60 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 --2- 172.21.15.64:0/567426770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff8feffd640 0 client.19449 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 >> 172.21.15.64:0/567426770 conn(0x55621e7e99c0 msgr2=0x55621e7e47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 shutdown_connections 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 1 -- 172.21.15.64:0/567426770 wait complete. 2024-05-09T09:25:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.846+0000 7ff928f6c740 20 client.19449 trim_cache size 0 max 16384 2024-05-09T09:25:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 Processor -- start 2024-05-09T09:25:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 -- start start 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201aab80 con 0x55621e7eed40 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201a8390 con 0x55621e7ee350 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.859+0000 7ff928f6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201a61e0 con 0x55621e7f3eb0 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48942/0 (socket says 172.21.15.64:48942) 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 -- 172.21.15.64:0/3987289531 learned_addr learned my addr 172.21.15.64:0/3987289531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91ffff640 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 -- 172.21.15.64:0/3987289531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7f3eb0 msgr2=0x55621e7ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 -- 172.21.15.64:0/3987289531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7ee350 msgr2=0x55621e7f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 -- 172.21.15.64:0/3987289531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621e7d6d20 con 0x55621e7eed40 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff91effd640 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff920069520 tx=0x7ff914031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971531db84a62277 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff9057fa640 1 -- 172.21.15.64:0/3987289531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914035040 con 0x55621e7eed40 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff9057fa640 1 -- 172.21.15.64:0/3987289531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff914005040 con 0x55621e7eed40 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.860+0000 7ff9057fa640 1 -- 172.21.15.64:0/3987289531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914004ae0 con 0x55621e7eed40 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 -- 172.21.15.64:0/3987289531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 msgr2=0x55621e7f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff920069520 tx=0x7ff914031730 comp rx=0 tx=0).stop 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 -- 172.21.15.64:0/3987289531 shutdown_connections 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7f3eb0 0x55621e7ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7eed40 0x55621e7f8c20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 --2- 172.21.15.64:0/3987289531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7ee350 0x55621e7f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 -- 172.21.15.64:0/3987289531 >> 172.21.15.64:0/3987289531 conn(0x55621e7e99c0 msgr2=0x55621e7f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 -- 172.21.15.64:0/3987289531 shutdown_connections 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 -- 172.21.15.64:0/3987289531 wait complete. 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 -- start start 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.861+0000 7ff928f6c740 1 Processor -- start 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201a6850 con 0x55621e7f3eb0 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201a7cc0 con 0x55621e7eed40 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201a6f20 con 0x55621e7ee350 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff928f6c740 10 client.0 _ll_register_callbacks cb 0x7ff91800be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48954/0 (socket says 172.21.15.64:48954) 2024-05-09T09:25:54.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91ffff640 1 -- 172.21.15.64:0/2580900198 learned_addr learned my addr 172.21.15.64:0/2580900198 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 -- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 msgr2=0x55621e7ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 -- 172.21.15.64:0/2580900198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 msgr2=0x55621e7fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 -- 172.21.15.64:0/2580900198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621e868e00 con 0x55621e7eed40 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91ffff640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff91effd640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7fc6c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff914004730 tx=0x7ff914004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3ac8a9bbae77d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.862+0000 7ff8feffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff8feffd640 1 -- 172.21.15.64:0/2580900198 <== mon.2 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914035050 con 0x55621e7eed40 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff914005040 con 0x55621e7eed40 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff8feffd640 1 -- 172.21.15.64:0/2580900198 <== mon.2 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff914045480 con 0x55621e7eed40 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff8feffd640 1 -- 172.21.15.64:0/2580900198 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff914056050 con 0x55621e7eed40 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 10 client.28931 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55621e86a130 con 0x55621e7eed40 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff8feffd640 1 -- 172.21.15.64:0/2580900198 <== mon.2 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff914005040 con 0x55621e7eed40 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff8feffd640 1 -- 172.21.15.64:0/2580900198 <== mon.2 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7ff91403b430 con 0x55621e7eed40 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 10 client.28931 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 10 client.28931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 20 client.28931 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e86aae0 0x55621e86cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 4 client.28931 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 --> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55621e7566c0 con 0x55621e86aae0 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.863+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e86aae0 0x55621e86cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff8feffd640 10 client.28931 ms_handle_connect on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff91f7fe640 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e86aae0 0x55621e86cf00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73102fa58b11349c server_cookie=4e0407979d0e0eea in_seq=0 out_seq=0 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff8feffd640 1 -- 172.21.15.64:0/2580900198 <== mds.1 v2:172.21.15.179:6838/1577800782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55621e7566c0 con 0x55621e86aae0 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff8feffd640 10 client.28931 handle_command_reply: tid=0 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 client.28931 shutdown 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 msgr2=0x55621e7fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7fc6c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff914004730 tx=0x7ff914004a90 comp rx=0 tx=0).stop 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 shutdown_connections 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:6838/1577800782,v1:172.21.15.179:6839/1577800782] conn(0x55621e86aae0 0x55621e86cf00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55621e7f3eb0 0x55621e7fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff8feffd640 0 client.28931 ms_handle_reset on v2:172.21.15.179:6838/1577800782 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55621e7eed40 0x55621e7fc6c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 --2- 172.21.15.64:0/2580900198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55621e7ee350 0x55621e7ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 >> 172.21.15.64:0/2580900198 conn(0x55621e7e99c0 msgr2=0x55621e7e3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 shutdown_connections 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 1 -- 172.21.15.64:0/2580900198 wait complete. 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.864+0000 7ff928f6c740 20 client.28931 trim_cache size 0 max 16384 2024-05-09T09:25:54.870 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:25:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2c10 msgr2=0x7ff9080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2c10 0x7ff9080a50d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55621e7ef5a0 tx=0x7ff918005cc0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 msgr2=0x7ff920197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 0x7ff920197b50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff91000eab0 tx=0x7ff91000ef70 comp rx=0 tx=0).stop 2024-05-09T09:25:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 shutdown_connections 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7ff9080a2c10 0x7ff9080a50d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff920107550 0x7ff92019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff920104c20 0x7ff920198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 --2- 172.21.15.64:0/83024783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9201022f0 0x7ff920197b50 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 >> 172.21.15.64:0/83024783 conn(0x7ff9200fbce0 msgr2=0x7ff920102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 shutdown_connections 2024-05-09T09:25:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.865+0000 7ff92617e640 1 -- 172.21.15.64:0/83024783 wait complete. 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:25:54.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:54 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/563198524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:54.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 -- 172.21.15.64:0/1170023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90104ca0 msgr2=0x7f4a90107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/1170023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90104ca0 0x7f4a90107090 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009900 tx=0x7f4a7802a410 comp rx=0 tx=0).stop 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 -- 172.21.15.64:0/1170023823 shutdown_connections 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/1170023823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a901075d0 0x7f4a901099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/1170023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90104ca0 0x7f4a90107090 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/1170023823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a90102370 0x7f4a90104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 -- 172.21.15.64:0/1170023823 >> 172.21.15.64:0/1170023823 conn(0x7f4a900fbd20 msgr2=0x7f4a900fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 -- 172.21.15.64:0/1170023823 shutdown_connections 2024-05-09T09:25:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.984+0000 7f4a94ba2640 1 -- 172.21.15.64:0/1170023823 wait complete. 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 Processor -- start 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 -- start start 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 0x7f4a90101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a90104ca0 0x7f4a90100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a901075d0 0x7f4a90100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9010d3c0 con 0x7f4a90102370 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9010d240 con 0x7f4a901075d0 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a94ba2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9010d540 con 0x7f4a90104ca0 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 0x7f4a90101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 0x7f4a90101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48966/0 (socket says 172.21.15.64:48966) 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 -- 172.21.15.64:0/376464669 learned_addr learned my addr 172.21.15.64:0/376464669 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a901075d0 0x7f4a90100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a87fff640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a90104ca0 0x7f4a90100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 -- 172.21.15.64:0/376464669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a90104ca0 msgr2=0x7f4a90100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a90104ca0 0x7f4a90100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 -- 172.21.15.64:0/376464669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a901075d0 msgr2=0x7f4a90100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a901075d0 0x7f4a90100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8e59c640 1 -- 172.21.15.64:0/376464669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a901010f0 con 0x7f4a90102370 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.985+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a901075d0 0x7f4a90100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.986+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 0x7f4a90101ce0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000e9f0 tx=0x7f4a8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.986+0000 7f4a877fe640 1 -- 172.21.15.64:0/376464669 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8000cd80 con 0x7f4a90102370 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.986+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a901a1c20 con 0x7f4a90102370 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.986+0000 7f4a877fe640 1 -- 172.21.15.64:0/376464669 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a800108e0 con 0x7f4a90102370 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.986+0000 7f4a877fe640 1 -- 172.21.15.64:0/376464669 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80018920 con 0x7f4a90102370 2024-05-09T09:25:54.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.986+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a901a2190 con 0x7f4a90102370 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a877fe640 1 -- 172.21.15.64:0/376464669 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a80010a90 con 0x7f4a90102370 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 Processor -- start 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 -- start start 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 0x5647b01c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c2930 0x5647b01c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c7aa0 0x5647b01c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647b0126c40 con 0x5647b01c1f40 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5647b00ce580 con 0x5647b01c2930 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a97991740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5647b0122c00 con 0x5647b01c7aa0 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.989+0000 7f4a8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 0x5647b01c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 0x5647b01c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48974/0 (socket says 172.21.15.64:48974) 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/1402377304 learned_addr learned my addr 172.21.15.64:0/1402377304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c2930 0x5647b01c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a87fff640 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c7aa0 0x5647b01c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a877fe640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4a7c0a2b90 0x7f4a7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/1402377304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c7aa0 msgr2=0x5647b01c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c7aa0 0x5647b01c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/1402377304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c2930 msgr2=0x5647b01c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c2930 0x5647b01c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/1402377304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647afe3a3f0 con 0x5647b01c1f40 2024-05-09T09:25:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a877fe640 1 -- 172.21.15.64:0/376464669 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4a800cf800 con 0x7f4a90102370 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 0x5647b01c2360 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009fb0 tx=0x7f4a88010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8532211b27a8e00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4a7c0a2b90 0x7f4a7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a84ff9640 1 -- 172.21.15.64:0/1402377304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88022070 con 0x5647b01c1f40 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a84ff9640 1 -- 172.21.15.64:0/1402377304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a880113e0 con 0x5647b01c1f40 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.990+0000 7f4a84ff9640 1 -- 172.21.15.64:0/1402377304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88016070 con 0x5647b01c1f40 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 -- 172.21.15.64:0/1402377304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 msgr2=0x5647b01c2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4a7c0a2b90 0x7f4a7c0a5050 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4a8801f000 tx=0x7f4a88022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 0x5647b01c2360 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009fb0 tx=0x7f4a88010800 comp rx=0 tx=0).stop 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 -- 172.21.15.64:0/1402377304 shutdown_connections 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c7aa0 0x5647b01c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c2930 0x5647b01c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 --2- 172.21.15.64:0/1402377304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c1f40 0x5647b01c2360 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 -- 172.21.15.64:0/1402377304 >> 172.21.15.64:0/1402377304 conn(0x5647b01bd5b0 msgr2=0x5647b01bd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 -- 172.21.15.64:0/1402377304 shutdown_connections 2024-05-09T09:25:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.991+0000 7f4a97991740 1 -- 172.21.15.64:0/1402377304 wait complete. 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 -- start start 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 Processor -- start 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647b01caa50 con 0x5647b01c7aa0 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5647b00e1c40 con 0x5647b01c1f40 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5647b01cabd0 con 0x5647b01c2930 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a97991740 10 client.0 _ll_register_callbacks cb 0x5647afe39be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01d3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48990/0 (socket says 172.21.15.64:48990) 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/2253671459 learned_addr learned my addr 172.21.15.64:0/2253671459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a87fff640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/2253671459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 msgr2=0x5647b01cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 msgr2=0x5647b01cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/2253671459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647aff047d0 con 0x5647b01c7aa0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a87fff640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.992+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01d3850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4a88006010 tx=0x7f4a88011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62d86b0751a15d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a75ffb640 1 -- 172.21.15.64:0/2253671459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88034060 con 0x5647b01c7aa0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a75ffb640 1 -- 172.21.15.64:0/2253671459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4a88011790 con 0x5647b01c7aa0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a880113e0 con 0x5647b01c7aa0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a75ffb640 1 -- 172.21.15.64:0/2253671459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f4a8803b050 con 0x5647b01c7aa0 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 10 client.19467 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:54.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5647b01d6bc0 con 0x5647b01c7aa0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a75ffb640 1 -- 172.21.15.64:0/2253671459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a880113e0 con 0x5647b01c7aa0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a75ffb640 1 -- 172.21.15.64:0/2253671459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f4a8802e030 con 0x5647b01c7aa0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 10 client.19467 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 20 client.19467 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b01d80a0 0x5647b01da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 4 client.19467 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647b0134370 con 0x5647b01d80a0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.993+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b01d80a0 0x5647b01da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.994+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b01d80a0 0x5647b01da540 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f735e00c0752835 server_cookie=ec234a605146a5c0 in_seq=0 out_seq=0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.994+0000 7f4a75ffb640 10 client.19467 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.994+0000 7f4a75ffb640 1 -- 172.21.15.64:0/2253671459 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647b0134370 con 0x5647b01d80a0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.994+0000 7f4a75ffb640 10 client.19467 handle_command_reply: tid=0 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.994+0000 7f4a97991740 1 client.19467 shutdown 2024-05-09T09:25:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 msgr2=0x5647b01d3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01d3850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4a88006010 tx=0x7f4a88011a90 comp rx=0 tx=0).stop 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 shutdown_connections 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b01d80a0 0x5647b01da540 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01d3850 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a75ffb640 0 client.19467 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 --2- 172.21.15.64:0/2253671459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 >> 172.21.15.64:0/2253671459 conn(0x5647b01bd5b0 msgr2=0x5647b01bc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 shutdown_connections 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 1 -- 172.21.15.64:0/2253671459 wait complete. 2024-05-09T09:25:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:54.995+0000 7f4a97991740 20 client.19467 trim_cache size 0 max 16384 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 Processor -- start 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 -- start start 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901af080 con 0x5647b01c7aa0 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901ac890 con 0x5647b01c1f40 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a97991740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901aa6e0 con 0x5647b01c2930 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48998/0 (socket says 172.21.15.64:48998) 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/737429555 learned_addr learned my addr 172.21.15.64:0/737429555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/737429555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 msgr2=0x5647b01d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a87fff640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.008+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/737429555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 msgr2=0x5647b01c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01c0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a8ed9d640 1 -- 172.21.15.64:0/737429555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b01aed20 con 0x5647b01c7aa0 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a87fff640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01c0870 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4a88011010 tx=0x7f4a88012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d49f3980f093c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a77fff640 1 -- 172.21.15.64:0/737429555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88034040 con 0x5647b01c7aa0 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a77fff640 1 -- 172.21.15.64:0/737429555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a88005340 con 0x5647b01c7aa0 2024-05-09T09:25:55.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a77fff640 1 -- 172.21.15.64:0/737429555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88016050 con 0x5647b01c7aa0 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 -- 172.21.15.64:0/737429555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 msgr2=0x5647b01c0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01c0870 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4a88011010 tx=0x7f4a88012b80 comp rx=0 tx=0).stop 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 -- 172.21.15.64:0/737429555 shutdown_connections 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c7aa0 0x5647b01c0870 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c2930 0x5647b01d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 --2- 172.21.15.64:0/737429555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c1f40 0x5647b01c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 -- 172.21.15.64:0/737429555 >> 172.21.15.64:0/737429555 conn(0x5647b01bd5b0 msgr2=0x5647b01c3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 -- 172.21.15.64:0/737429555 shutdown_connections 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.009+0000 7f4a97991740 1 -- 172.21.15.64:0/737429555 wait complete. 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 -- start start 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 Processor -- start 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c1f40 0x5647b01d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 0x5647b01d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c7aa0 0x5647b01d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901aad50 con 0x5647b01c2930 2024-05-09T09:25:55.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901ac1c0 con 0x5647b01c7aa0 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901ab420 con 0x5647b01c1f40 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a97991740 10 client.0 _ll_register_callbacks cb 0x7f4a88020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 0x5647b01d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 0x5647b01d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49010/0 (socket says 172.21.15.64:49010) 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 -- 172.21.15.64:0/299786818 learned_addr learned my addr 172.21.15.64:0/299786818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c7aa0 0x5647b01d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c1f40 0x5647b01d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 -- 172.21.15.64:0/299786818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c1f40 msgr2=0x5647b01d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c1f40 0x5647b01d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 -- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c7aa0 msgr2=0x5647b01d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c7aa0 0x5647b01d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a87fff640 1 -- 172.21.15.64:0/299786818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b023c670 con 0x5647b01c2930 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c1f40 0x5647b01d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.010+0000 7f4a8ed9d640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c7aa0 0x5647b01d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:25:55.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a87fff640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 0x5647b01d0200 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4a78005940 tx=0x7f4a78031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1b8a07239a264a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a75ffb640 1 -- 172.21.15.64:0/299786818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78035050 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a75ffb640 1 -- 172.21.15.64:0/299786818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4a78004450 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a88005340 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a75ffb640 1 -- 172.21.15.64:0/299786818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f4a7804c050 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 10 client.19473 fetch_fsmap learned FSMap version 53 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5647b023d900 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a75ffb640 1 -- 172.21.15.64:0/299786818 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a88005340 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a75ffb640 1 -- 172.21.15.64:0/299786818 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) ==== 8370+0+0 (secure 0 0 0) 0x7f4a78004c30 con 0x5647b01c2930 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 10 client.19473 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 10 client.19473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 20 client.19473 populate_metadata read hostname 'smithi064' 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b023e310 0x5647b0240730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 4 client.19473 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T09:25:55.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.011+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 --> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647b012e6c0 con 0x5647b023e310 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b023e310 0x5647b0240730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a75ffb640 10 client.19473 ms_handle_connect on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a8e59c640 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b023e310 0x5647b0240730 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a934cca6efa46b server_cookie=376f295e684e0cfa in_seq=0 out_seq=0 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a75ffb640 1 -- 172.21.15.64:0/299786818 <== mds.0 v2:172.21.15.139:6838/1823707244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647b012e6c0 con 0x5647b023e310 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a75ffb640 10 client.19473 handle_command_reply: tid=0 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 client.19473 shutdown 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 msgr2=0x5647b01d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 0x5647b01d0200 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4a78005940 tx=0x7f4a78031820 comp rx=0 tx=0).stop 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 shutdown_connections 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:6838/1823707244,v1:172.21.15.139:6839/1823707244] conn(0x5647b023e310 0x5647b0240730 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5647b01c7aa0 0x5647b01d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b01c2930 0x5647b01d0200 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 --2- 172.21.15.64:0/299786818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647b01c1f40 0x5647b01d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a75ffb640 0 client.19473 ms_handle_reset on v2:172.21.15.139:6838/1823707244 2024-05-09T09:25:55.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 >> 172.21.15.64:0/299786818 conn(0x5647b01bd5b0 msgr2=0x5647b01bbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:55.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 shutdown_connections 2024-05-09T09:25:55.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 1 -- 172.21.15.64:0/299786818 wait complete. 2024-05-09T09:25:55.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.012+0000 7f4a97991740 20 client.19473 trim_cache size 0 max 16384 2024-05-09T09:25:55.019 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4a7c0a2b90 msgr2=0x7f4a7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4a7c0a2b90 0x7f4a7c0a5050 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4a8801f000 tx=0x7f4a88022000 comp rx=0 tx=0).stop 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 msgr2=0x7f4a90101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 0x7f4a90101ce0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000e9f0 tx=0x7f4a8000eeb0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 shutdown_connections 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4a7c0a2b90 0x7f4a7c0a5050 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a901075d0 0x7f4a90100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a90104ca0 0x7f4a90100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 --2- 172.21.15.64:0/376464669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a90102370 0x7f4a90101ce0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 >> 172.21.15.64:0/376464669 conn(0x7f4a900fbd20 msgr2=0x7f4a90102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 shutdown_connections 2024-05-09T09:25:55.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:55.013+0000 7f4a94ba2640 1 -- 172.21.15.64:0/376464669 wait complete. 2024-05-09T09:25:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-09T09:25:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:25:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:25:55.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:54 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/563198524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-09T09:25:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' 2024-05-09T09:25:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:54 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/563198524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:25:56.024 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-09T09:25:56.025 INFO:tasks.fwd_scrub:done joining 2024-05-09T09:25:56.025 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T09:25:56.035 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:25:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 -- 172.21.15.64:0/525328700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c104c40 msgr2=0x7f762c107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 --2- 172.21.15.64:0/525328700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c104c40 0x7f762c107030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7614009900 tx=0x7f761402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 -- 172.21.15.64:0/525328700 shutdown_connections 2024-05-09T09:25:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 --2- 172.21.15.64:0/525328700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c107570 0x7f762c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 --2- 172.21.15.64:0/525328700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c104c40 0x7f762c107030 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 --2- 172.21.15.64:0/525328700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c102310 0x7f762c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 -- 172.21.15.64:0/525328700 >> 172.21.15.64:0/525328700 conn(0x7f762c0fbd20 msgr2=0x7f762c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 -- 172.21.15.64:0/525328700 shutdown_connections 2024-05-09T09:25:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.155+0000 7f7631fc8640 1 -- 172.21.15.64:0/525328700 wait complete. 2024-05-09T09:25:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 Processor -- start 2024-05-09T09:25:56.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 -- start start 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 0x7f762c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c104c40 0x7f762c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c107570 0x7f762c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10d3b0 con 0x7f762c102310 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10d230 con 0x7f762c107570 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7631fc8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10d530 con 0x7f762c104c40 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 0x7f762c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 0x7f762c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49026/0 (socket says 172.21.15.64:49026) 2024-05-09T09:25:56.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 -- 172.21.15.64:0/141185742 learned_addr learned my addr 172.21.15.64:0/141185742 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f7622ffd640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c104c40 0x7f762c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762bfff640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c107570 0x7f762c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 -- 172.21.15.64:0/141185742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c104c40 msgr2=0x7f762c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c104c40 0x7f762c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 -- 172.21.15.64:0/141185742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c107570 msgr2=0x7f762c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c107570 0x7f762c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762b7fe640 1 -- 172.21.15.64:0/141185742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c19f720 con 0x7f762c102310 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.156+0000 7f762bfff640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c107570 0x7f762c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.157+0000 7f762b7fe640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 0x7f762c197b50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f761c00d950 tx=0x7f761c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.157+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c017070 con 0x7f762c102310 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.157+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f761c009c30 con 0x7f762c102310 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.157+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c00f620 con 0x7f762c102310 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.157+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c19fa10 con 0x7f762c102310 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.157+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c19ff30 con 0x7f762c102310 2024-05-09T09:25:56.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.158+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762c0fd810 con 0x7f762c102310 2024-05-09T09:25:56.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.159+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f761c0109a0 con 0x7f762c102310 2024-05-09T09:25:56.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.160+0000 7f76297fa640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76180a2b90 0x7f76180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.160+0000 7f7622ffd640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76180a2b90 0x7f76180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.160+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f761c0cf0d0 con 0x7f762c102310 2024-05-09T09:25:56.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.161+0000 7f7622ffd640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76180a2b90 0x7f76180a5050 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f76140314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.164+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f761c009de0 con 0x7f762c102310 2024-05-09T09:25:56.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:56 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-09T09:25:56.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:56 smithi139 ceph-mon[33599]: Adjusting osd_memory_target on smithi139 to 176.1M 2024-05-09T09:25:56.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:56 smithi139 ceph-mon[33599]: Unable to set osd_memory_target on smithi064 to 184775987: error parsing value: Value '184775987' is below minimum 939524096 2024-05-09T09:25:56.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:56 smithi139 ceph-mon[33599]: Unable to set osd_memory_target on smithi139 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-09T09:25:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.353+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f762c02d1c0 con 0x7f762c102310 2024-05-09T09:25:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.355+0000 7f76297fa640 1 -- 172.21.15.64:0/141185742 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f761c09bca0 con 0x7f762c102310 2024-05-09T09:25:56.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76180a2b90 msgr2=0x7f76180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76180a2b90 0x7f76180a5050 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f76140314a0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 msgr2=0x7f762c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 0x7f762c197b50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f761c00d950 tx=0x7f761c00de10 comp rx=0 tx=0).stop 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 shutdown_connections 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76180a2b90 0x7f76180a5050 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762c107570 0x7f762c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c104c40 0x7f762c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 --2- 172.21.15.64:0/141185742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f762c102310 0x7f762c197b50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 >> 172.21.15.64:0/141185742 conn(0x7f762c0fbd20 msgr2=0x7f762c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 shutdown_connections 2024-05-09T09:25:56.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.357+0000 7f7631fc8640 1 -- 172.21.15.64:0/141185742 wait complete. 2024-05-09T09:25:56.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:56 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-09T09:25:56.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:56 smithi064 ceph-mon[30149]: Adjusting osd_memory_target on smithi139 to 176.1M 2024-05-09T09:25:56.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:56 smithi064 ceph-mon[30149]: Unable to set osd_memory_target on smithi064 to 184775987: error parsing value: Value '184775987' is below minimum 939524096 2024-05-09T09:25:56.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:56 smithi064 ceph-mon[30149]: Unable to set osd_memory_target on smithi139 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-09T09:25:56.369 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:25:56.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 -- 172.21.15.64:0/306954026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 msgr2=0x7f82c8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/306954026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c8101c90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 -- 172.21.15.64:0/306954026 shutdown_connections 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/306954026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82c8103420 0x7f82c8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/306954026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8102a60 0x7f82c8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/306954026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c8101c90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 -- 172.21.15.64:0/306954026 >> 172.21.15.64:0/306954026 conn(0x7f82c80fcde0 msgr2=0x7f82c80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:56.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 -- 172.21.15.64:0/306954026 shutdown_connections 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.478+0000 7f82cf3f9640 1 -- 172.21.15.64:0/306954026 wait complete. 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 Processor -- start 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 -- start start 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c80755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82c8102a60 0x7f82c8073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8103420 0x7f82c8074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810d390 con 0x7f82c8101870 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810d210 con 0x7f82c8102a60 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cf3f9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810d510 con 0x7f82c8103420 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cd195640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c80755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cd195640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c80755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49042/0 (socket says 172.21.15.64:49042) 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cd195640 1 -- 172.21.15.64:0/913873167 learned_addr learned my addr 172.21.15.64:0/913873167 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cc994640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82c8102a60 0x7f82c8073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.479+0000 7f82cd996640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8103420 0x7f82c8074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd195640 1 -- 172.21.15.64:0/913873167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8103420 msgr2=0x7f82c8074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd195640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8103420 0x7f82c8074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd195640 1 -- 172.21.15.64:0/913873167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82c8102a60 msgr2=0x7f82c8073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd195640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82c8102a60 0x7f82c8073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd195640 1 -- 172.21.15.64:0/913873167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c80749f0 con 0x7f82c8101870 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd996640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8103420 0x7f82c8074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cd195640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c80755e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c403f030 con 0x7f82c8101870 2024-05-09T09:25:56.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82c4003cb0 con 0x7f82c8101870 2024-05-09T09:25:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c81a1b80 con 0x7f82c8101870 2024-05-09T09:25:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4035070 con 0x7f82c8101870 2024-05-09T09:25:56.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.480+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c81a20d0 con 0x7f82c8101870 2024-05-09T09:25:56.483 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:56 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-09T09:25:56.483 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:56 smithi179 ceph-mon[34086]: Adjusting osd_memory_target on smithi139 to 176.1M 2024-05-09T09:25:56.483 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:56 smithi179 ceph-mon[34086]: Unable to set osd_memory_target on smithi064 to 184775987: error parsing value: Value '184775987' is below minimum 939524096 2024-05-09T09:25:56.484 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:56 smithi179 ceph-mon[34086]: Unable to set osd_memory_target on smithi139 to 184752332: error parsing value: Value '184752332' is below minimum 939524096 2024-05-09T09:25:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.481+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c8108d00 con 0x7f82c8101870 2024-05-09T09:25:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.483+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82c4042030 con 0x7f82c8101870 2024-05-09T09:25:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.484+0000 7f82be7fc640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82b40a2b90 0x7f82b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.484+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f82c40efee0 con 0x7f82c8101870 2024-05-09T09:25:56.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.484+0000 7f82cc994640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82b40a2b90 0x7f82b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.484+0000 7f82cc994640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82b40a2b90 0x7f82b40a5050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f82b8005fd0 tx=0x7f82b8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:56.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.487+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82c40bcab0 con 0x7f82c8101870 2024-05-09T09:25:56.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.674+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82c802d410 con 0x7f82c8101870 2024-05-09T09:25:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.675+0000 7f82be7fc640 1 -- 172.21.15.64:0/913873167 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f82c40bc3b0 con 0x7f82c8101870 2024-05-09T09:25:56.675 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:25:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.676+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82b40a2b90 msgr2=0x7f82b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.676+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82b40a2b90 0x7f82b40a5050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f82b8005fd0 tx=0x7f82b8005e20 comp rx=0 tx=0).stop 2024-05-09T09:25:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.676+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 msgr2=0x7f82c80755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.676+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c80755e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c4004340 comp rx=0 tx=0).stop 2024-05-09T09:25:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 shutdown_connections 2024-05-09T09:25:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f82b40a2b90 0x7f82b40a5050 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82c8103420 0x7f82c8074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82c8102a60 0x7f82c8073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 --2- 172.21.15.64:0/913873167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82c8101870 0x7f82c80755e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 >> 172.21.15.64:0/913873167 conn(0x7f82c80fcde0 msgr2=0x7f82c8105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 shutdown_connections 2024-05-09T09:25:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.677+0000 7f82cf3f9640 1 -- 172.21.15.64:0/913873167 wait complete. 2024-05-09T09:25:56.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:25:56.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 -- 172.21.15.64:0/2397755083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 msgr2=0x7fc61c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 --2- 172.21.15.64:0/2397755083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c108bd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc614009ca0 tx=0x7fc61402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 -- 172.21.15.64:0/2397755083 shutdown_connections 2024-05-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 --2- 172.21.15.64:0/2397755083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c108bd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 --2- 172.21.15.64:0/2397755083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 0x7fc61c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 --2- 172.21.15.64:0/2397755083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc61c100790 0x7fc61c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 -- 172.21.15.64:0/2397755083 >> 172.21.15.64:0/2397755083 conn(0x7fc61c0fbd00 msgr2=0x7fc61c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 -- 172.21.15.64:0/2397755083 shutdown_connections 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.846+0000 7fc623f42640 1 -- 172.21.15.64:0/2397755083 wait complete. 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 Processor -- start 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 -- start start 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc61c100790 0x7fc61c104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 0x7fc61c102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c1032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c10c2c0 con 0x7fc61c102340 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c10c140 con 0x7fc61c100790 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc623f42640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c10c440 con 0x7fc61c101980 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc6224df640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c1032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc6224df640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c1032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49066/0 (socket says 172.21.15.64:49066) 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc6224df640 1 -- 172.21.15.64:0/1601499643 learned_addr learned my addr 172.21.15.64:0/1601499643 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.847+0000 7fc621cde640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc61c100790 0x7fc61c104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6214dd640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 0x7fc61c102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6224df640 1 -- 172.21.15.64:0/1601499643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 msgr2=0x7fc61c102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6224df640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 0x7fc61c102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6224df640 1 -- 172.21.15.64:0/1601499643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc61c100790 msgr2=0x7fc61c104720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6224df640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc61c100790 0x7fc61c104720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6224df640 1 -- 172.21.15.64:0/1601499643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c103b30 con 0x7fc61c102340 2024-05-09T09:25:56.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6214dd640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 0x7fc61c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:56.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc6224df640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c1032a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc614006010 tx=0x7fc614003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:56.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614004040 con 0x7fc61c102340 2024-05-09T09:25:56.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c1a1c30 con 0x7fc61c102340 2024-05-09T09:25:56.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc614004690 con 0x7fc61c102340 2024-05-09T09:25:56.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614030bf0 con 0x7fc61c102340 2024-05-09T09:25:56.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.848+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c1a2180 con 0x7fc61c102340 2024-05-09T09:25:56.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.849+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c1a2350 con 0x7fc61c102340 2024-05-09T09:25:56.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.850+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc6140041e0 con 0x7fc61c102340 2024-05-09T09:25:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.851+0000 7fc612ffd640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6080a2b90 0x7fc6080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.851+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc6140f08e0 con 0x7fc61c102340 2024-05-09T09:25:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.851+0000 7fc621cde640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6080a2b90 0x7fc6080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.851+0000 7fc621cde640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6080a2b90 0x7fc6080a5050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc61c1017e0 tx=0x7fc61800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:56.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:56.855+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6140bd4b0 con 0x7fc61c102340 2024-05-09T09:25:57.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.042+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc61c1a2520 con 0x7fc61c102340 2024-05-09T09:25:57.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.043+0000 7fc612ffd640 1 -- 172.21.15.64:0/1601499643 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fc6140bcdb0 con 0x7fc61c102340 2024-05-09T09:25:57.044 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6080a2b90 msgr2=0x7fc6080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6080a2b90 0x7fc6080a5050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc61c1017e0 tx=0x7fc61800a2d0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 msgr2=0x7fc61c1032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c1032a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc614006010 tx=0x7fc614003ba0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 shutdown_connections 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6080a2b90 0x7fc6080a5050 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc61c102340 0x7fc61c1032a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc61c101980 0x7fc61c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 --2- 172.21.15.64:0/1601499643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc61c100790 0x7fc61c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 >> 172.21.15.64:0/1601499643 conn(0x7fc61c0fbd00 msgr2=0x7fc61c0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:57.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 shutdown_connections 2024-05-09T09:25:57.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.045+0000 7fc623f42640 1 -- 172.21.15.64:0/1601499643 wait complete. 2024-05-09T09:25:57.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:25:57.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.165+0000 7f767afb7640 1 -- 172.21.15.64:0/1549576372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674100790 msgr2=0x7f7674100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.165+0000 7f767afb7640 1 --2- 172.21.15.64:0/1549576372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674100790 0x7f7674100bb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f766c009900 tx=0x7f766c02a410 comp rx=0 tx=0).stop 2024-05-09T09:25:57.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- 172.21.15.64:0/1549576372 shutdown_connections 2024-05-09T09:25:57.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 --2- 172.21.15.64:0/1549576372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7674102340 0x7f7674108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 --2- 172.21.15.64:0/1549576372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674101980 0x7f7674101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 --2- 172.21.15.64:0/1549576372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674100790 0x7f7674100bb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- 172.21.15.64:0/1549576372 >> 172.21.15.64:0/1549576372 conn(0x7f76740fbd20 msgr2=0x7f76740fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- 172.21.15.64:0/1549576372 shutdown_connections 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- 172.21.15.64:0/1549576372 wait complete. 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 Processor -- start 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- start start 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674100790 0x7f767406a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 0x7f767419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7674102340 0x7f767419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f767410c290 con 0x7f7674101980 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f767410c110 con 0x7f7674102340 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f767afb7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f767410c410 con 0x7f7674100790 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f7673fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 0x7f767419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.166+0000 7f7673fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 0x7f767419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49094/0 (socket says 172.21.15.64:49094) 2024-05-09T09:25:57.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 -- 172.21.15.64:0/3029821392 learned_addr learned my addr 172.21.15.64:0/3029821392 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7678d53640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674100790 0x7f767406a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 -- 172.21.15.64:0/3029821392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674100790 msgr2=0x7f767406a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674100790 0x7f767406a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 -- 172.21.15.64:0/3029821392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7674102340 msgr2=0x7f767419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7674102340 0x7f767419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 -- 172.21.15.64:0/3029821392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767419f6c0 con 0x7f7674101980 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7678d53640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674100790 0x7f767406a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:25:57.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7673fff640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 0x7f767419c7c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f765c00c920 tx=0x7f765c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c007c70 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f765c007e10 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c018820 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767419f9b0 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.167+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767419ff50 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.168+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7674107bd0 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.169+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f765c0189c0 con 0x7f7674101980 2024-05-09T09:25:57.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.170+0000 7f7671ffb640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76600a2c10 0x7f76600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.170+0000 7f7678d53640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76600a2c10 0x7f76600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.170+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f765c0cf570 con 0x7f7674101980 2024-05-09T09:25:57.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.171+0000 7f7678d53640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76600a2c10 0x7f76600a50d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:57.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.174+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f765c014030 con 0x7f7674101980 2024-05-09T09:25:57.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:57 smithi139 ceph-mon[33599]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-09T09:25:57.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/141185742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:57.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/913873167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:57.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:57 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1601499643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:57.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.361+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7674062460 con 0x7f7674101980 2024-05-09T09:25:57.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.362+0000 7f7671ffb640 1 -- 172.21.15.64:0/3029821392 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f765c09c140 con 0x7f7674101980 2024-05-09T09:25:57.362 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:25:57.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76600a2c10 msgr2=0x7f76600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76600a2c10 0x7f76600a50d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 msgr2=0x7f767419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 0x7f767419c7c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f765c00c920 tx=0x7f765c00cde0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 shutdown_connections 2024-05-09T09:25:57.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f76600a2c10 0x7f76600a50d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7674102340 0x7f767419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7674101980 0x7f767419c7c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 --2- 172.21.15.64:0/3029821392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7674100790 0x7f767406a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 >> 172.21.15.64:0/3029821392 conn(0x7f76740fbd20 msgr2=0x7f7674104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:57.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 shutdown_connections 2024-05-09T09:25:57.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.364+0000 7f767afb7640 1 -- 172.21.15.64:0/3029821392 wait complete. 2024-05-09T09:25:57.375 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:57 smithi064 ceph-mon[30149]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-09T09:25:57.375 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/141185742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:57.376 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/913873167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:57.376 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:57 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1601499643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:57.378 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:25:57.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:57 smithi179 ceph-mon[34086]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-09T09:25:57.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/141185742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:25:57.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/913873167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:25:57.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:57 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1601499643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:25:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 -- 172.21.15.64:0/1858036901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0107570 msgr2=0x7f91a01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1858036901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0107570 0x7f91a01099c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9198009ca0 tx=0x7f919802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 -- 172.21.15.64:0/1858036901 shutdown_connections 2024-05-09T09:25:57.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1858036901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0107570 0x7f91a01099c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1858036901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a0104c40 0x7f91a0107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1858036901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91a0102310 0x7f91a0104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 -- 172.21.15.64:0/1858036901 >> 172.21.15.64:0/1858036901 conn(0x7f91a00fbd20 msgr2=0x7f91a00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 -- 172.21.15.64:0/1858036901 shutdown_connections 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 -- 172.21.15.64:0/1858036901 wait complete. 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 Processor -- start 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.496+0000 7f91a6d53640 1 -- start start 2024-05-09T09:25:57.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91a0102310 0x7f91a0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 0x7f91a0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a0107570 0x7f91a019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91a010d3c0 con 0x7f91a0104c40 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91a010d240 con 0x7f91a0107570 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91a010d540 con 0x7f91a0102310 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 0x7f91a0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 0x7f91a0197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49116/0 (socket says 172.21.15.64:49116) 2024-05-09T09:25:57.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 -- 172.21.15.64:0/1972909367 learned_addr learned my addr 172.21.15.64:0/1972909367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a4aef640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91a0102310 0x7f91a0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a52f0640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a0107570 0x7f91a019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 -- 172.21.15.64:0/1972909367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91a0102310 msgr2=0x7f91a0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91a0102310 0x7f91a0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 -- 172.21.15.64:0/1972909367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a0107570 msgr2=0x7f91a019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a0107570 0x7f91a019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 -- 172.21.15.64:0/1972909367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a019f6b0 con 0x7f91a0104c40 2024-05-09T09:25:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919ffff640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 0x7f91a0197fd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9c0 tx=0x7f918c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cd80 con 0x7f91a0104c40 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a019f9a0 con 0x7f91a0104c40 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f918c0054d0 con 0x7f91a0104c40 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.497+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c010430 con 0x7f91a0104c40 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.498+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a019fef0 con 0x7f91a0104c40 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.499+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a00fd810 con 0x7f91a0104c40 2024-05-09T09:25:57.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.500+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f918c004fd0 con 0x7f91a0104c40 2024-05-09T09:25:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.500+0000 7f919dffb640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91900a2b90 0x7f91900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.500+0000 7f91a4aef640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91900a2b90 0x7f91900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.500+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f918c0cf8c0 con 0x7f91a0104c40 2024-05-09T09:25:57.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.501+0000 7f91a4aef640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91900a2b90 0x7f91900a5050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f919400bbe0 tx=0x7f919400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:57.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.505+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f918c09c490 con 0x7f91a0104c40 2024-05-09T09:25:57.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.690+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f91a0062430 con 0x7f91a0104c40 2024-05-09T09:25:57.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.690+0000 7f919dffb640 1 -- 172.21.15.64:0/1972909367 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f918c09bd90 con 0x7f91a0104c40 2024-05-09T09:25:57.691 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:25:57.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91900a2b90 msgr2=0x7f91900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91900a2b90 0x7f91900a5050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f919400bbe0 tx=0x7f919400b390 comp rx=0 tx=0).stop 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 msgr2=0x7f91a0197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 0x7f91a0197fd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9c0 tx=0x7f918c00ee80 comp rx=0 tx=0).stop 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 shutdown_connections 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f91900a2b90 0x7f91900a5050 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a0107570 0x7f91a019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91a0104c40 0x7f91a0197fd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 --2- 172.21.15.64:0/1972909367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91a0102310 0x7f91a0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 >> 172.21.15.64:0/1972909367 conn(0x7f91a00fbd20 msgr2=0x7f91a0102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:57.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 shutdown_connections 2024-05-09T09:25:57.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.692+0000 7f91a6d53640 1 -- 172.21.15.64:0/1972909367 wait complete. 2024-05-09T09:25:57.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T09:25:57.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.821+0000 7f3a494a5640 1 -- 172.21.15.64:0/1235221517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44100770 msgr2=0x7f3a44100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.821+0000 7f3a494a5640 1 --2- 172.21.15.64:0/1235221517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44100770 0x7f3a44100b90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009900 tx=0x7f3a3402a410 comp rx=0 tx=0).stop 2024-05-09T09:25:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- 172.21.15.64:0/1235221517 shutdown_connections 2024-05-09T09:25:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 --2- 172.21.15.64:0/1235221517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a44102320 0x7f3a44108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 --2- 172.21.15.64:0/1235221517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a44101960 0x7f3a44101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 --2- 172.21.15.64:0/1235221517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44100770 0x7f3a44100b90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- 172.21.15.64:0/1235221517 >> 172.21.15.64:0/1235221517 conn(0x7f3a440fbce0 msgr2=0x7f3a440fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- 172.21.15.64:0/1235221517 shutdown_connections 2024-05-09T09:25:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- 172.21.15.64:0/1235221517 wait complete. 2024-05-09T09:25:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 Processor -- start 2024-05-09T09:25:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- start start 2024-05-09T09:25:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a44100770 0x7f3a44197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a44101960 0x7f3a44197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 0x7f3a4419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4410c270 con 0x7f3a44102320 2024-05-09T09:25:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4410c0f0 con 0x7f3a44101960 2024-05-09T09:25:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.822+0000 7f3a494a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4410c3f0 con 0x7f3a44100770 2024-05-09T09:25:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 0x7f3a4419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 0x7f3a4419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49130/0 (socket says 172.21.15.64:49130) 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 -- 172.21.15.64:0/146472875 learned_addr learned my addr 172.21.15.64:0/146472875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a3bfff640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a44101960 0x7f3a44197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a42ffd640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a44100770 0x7f3a44197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 -- 172.21.15.64:0/146472875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a44100770 msgr2=0x7f3a44197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a44100770 0x7f3a44197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 -- 172.21.15.64:0/146472875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a44101960 msgr2=0x7f3a44197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:57.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a44101960 0x7f3a44197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 -- 172.21.15.64:0/146472875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4419f680 con 0x7f3a44102320 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a437fe640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 0x7f3a4419f010 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00cce0 tx=0x7f3a3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00f450 con 0x7f3a44102320 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a3c00faa0 con 0x7f3a44102320 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4419f970 con 0x7f3a44102320 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c018b70 con 0x7f3a44102320 2024-05-09T09:25:57.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.823+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4419fe70 con 0x7f3a44102320 2024-05-09T09:25:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.824+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a441a0100 con 0x7f3a44102320 2024-05-09T09:25:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.825+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a3c018d10 con 0x7f3a44102320 2024-05-09T09:25:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.826+0000 7f3a40ff9640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3a300a2b90 0x7f3a300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:25:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.826+0000 7f3a42ffd640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3a300a2b90 0x7f3a300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:25:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.826+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3a3c0cf770 con 0x7f3a44102320 2024-05-09T09:25:57.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.827+0000 7f3a42ffd640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3a300a2b90 0x7f3a300a5050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3a34004770 tx=0x7f3a34000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:25:57.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:57.830+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a3c014030 con 0x7f3a44102320 2024-05-09T09:25:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.066+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3a441a0370 con 0x7f3a44102320 2024-05-09T09:25:58.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.074+0000 7f3a40ff9640 1 -- 172.21.15.64:0/146472875 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740368 (secure 0 0 0) 0x7f3a3c09c340 con 0x7f3a44102320 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 18, 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 14152, 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "x", 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2024-05-09T09:25:58.076 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6800", 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2084062196 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6801", 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2084062196 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.077 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6801/2084062196", 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2024-05-09T08:59:35.011264+0000", 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24109, 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "z", 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540701547738038271, 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.078 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.080 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.083 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.087 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.090 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.091 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-05-09T09:25:58.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-05-09T09:25:58.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-05-09T09:25:58.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-05-09T09:25:58.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-05-09T09:25:58.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-05-09T09:25:58.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.102 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.103 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.104 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.106 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.108 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.110 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-05-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T09:25:58.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.125 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-05-09T09:25:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-05-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-05-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-05-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T09:25:58.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-05-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T09:25:58.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.138 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-05-09T09:25:58.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-05-09T09:25:58.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-05-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T09:25:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-05-09T09:25:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-05-09T09:25:58.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-05-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.169 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.171 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.177 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-05-09T09:25:58.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.181 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T09:25:58.185 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-05-09T09:25:58.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-05-09T09:25:58.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T09:25:58.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.201 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-05-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.229 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.230 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.233 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T09:25:58.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-05-09T09:25:58.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-05-09T09:25:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.245 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:58.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.254 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-05-09T09:25:58.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-05-09T09:25:58.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-05-09T09:25:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.258 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.261 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.262 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-05-09T09:25:58.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.266 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-05-09T09:25:58.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-09T09:25:58.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-09T09:25:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-05-09T09:25:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.271 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-05-09T09:25:58.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.273 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.274 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-09T09:25:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.287 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.288 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-05-09T09:25:58.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.296 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.297 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-05-09T09:25:58.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-05-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.303 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.305 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.307 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.309 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.313 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.314 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-05-09T09:25:58.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.321 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.322 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-05-09T09:25:58.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.325 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.327 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.328 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-05-09T09:25:58.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-05-09T09:25:58.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.333 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.336 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.339 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.340 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.341 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.342 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.345 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.347 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.348 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-05-09T09:25:58.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.353 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.354 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.355 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.356 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.358 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.359 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.361 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.362 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.364 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.366 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.367 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.369 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.378 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.379 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.381 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.382 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:58 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3029821392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:58.382 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:58 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1972909367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:58.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-05-09T09:25:58.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.388 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.394 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-05-09T09:25:58.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-09T09:25:58.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.397 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.398 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.399 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-05-09T09:25:58.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-05-09T09:25:58.406 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.409 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.413 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:58.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.416 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.421 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.424 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-05-09T09:25:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.426 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-05-09T09:25:58.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.442 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-05-09T09:25:58.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.450 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.454 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.455 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.462 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.464 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.466 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-05-09T09:25:58.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-05-09T09:25:58.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.479 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.482 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-05-09T09:25:58.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-05-09T09:25:58.486 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.487 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.488 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.490 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.492 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.495 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-05-09T09:25:58.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.500 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-05-09T09:25:58.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.505 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.506 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24110, 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "y", 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540701547738038271, 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.515 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-05-09T09:25:58.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.521 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-05-09T09:25:58.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-05-09T09:25:58.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-05-09T09:25:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.530 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.533 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.536 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:58 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3029821392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:58.536 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:58 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1972909367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.538 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.543 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-05-09T09:25:58.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-05-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-05-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-05-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.553 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.554 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-05-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T09:25:58.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-05-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-05-09T09:25:58.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-05-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-05-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-05-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-05-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-05-09T09:25:58.592 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T09:25:58.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.599 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-05-09T09:25:58.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-05-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-05-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.633 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T09:25:58.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T09:25:58.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.643 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T09:25:58.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T09:25:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T09:25:58.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:58.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.655 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-05-09T09:25:58.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-05-09T09:25:58.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-09T09:25:58.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.662 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-05-09T09:25:58.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.670 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-05-09T09:25:58.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-05-09T09:25:58.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.674 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-05-09T09:25:58.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-05-09T09:25:58.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.687 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.693 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.694 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T09:25:58.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.697 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.700 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-05-09T09:25:58.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-05-09T09:25:58.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.708 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.710 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.712 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.715 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.717 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.718 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.722 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.725 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.726 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.728 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-05-09T09:25:58.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.738 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.744 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.754 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.757 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.758 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.760 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-05-09T09:25:58.762 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.763 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.764 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-05-09T09:25:58.767 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.768 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.774 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.776 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.777 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.779 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-05-09T09:25:58.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.788 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.792 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.794 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-09T09:25:58.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.798 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-05-09T09:25:58.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-05-09T09:25:58.805 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.807 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.808 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3a300a2b90 msgr2=0x7f3a300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3a300a2b90 0x7f3a300a5050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3a34004770 tx=0x7f3a34000bc0 comp rx=0 tx=0).stop 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 msgr2=0x7f3a4419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 0x7f3a4419f010 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00cce0 tx=0x7f3a3c00a620 comp rx=0 tx=0).stop 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 shutdown_connections 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f3a300a2b90 0x7f3a300a5050 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a44102320 0x7f3a4419f010 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a44101960 0x7f3a44197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 --2- 172.21.15.64:0/146472875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a44100770 0x7f3a44197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 >> 172.21.15.64:0/146472875 conn(0x7f3a440fbce0 msgr2=0x7f3a440fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 shutdown_connections 2024-05-09T09:25:58.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:25:58.080+0000 7f3a494a5640 1 -- 172.21.15.64:0/146472875 wait complete. 2024-05-09T09:25:58.810 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.810 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.810 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.810 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.810 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.810 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.812 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.819 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.821 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.822 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.823 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.824 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.828 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.830 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.839 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.840 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-05-09T09:25:58.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.847 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.850 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.852 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.854 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.861 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.864 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T09:25:58.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.873 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-05-09T09:25:58.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-05-09T09:25:58.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-05-09T09:25:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.881 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.888 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.891 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.893 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.894 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.896 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-05-09T09:25:58.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.906 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-05-09T09:25:58.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.912 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard", 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "restful" 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.913 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:58.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:58.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:58 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3029821392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:25:58.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:58 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1972909367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T09:25:58.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.918 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-05-09T09:25:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.927 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-05-09T09:25:58.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-05-09T09:25:58.930 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.931 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.935 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-05-09T09:25:58.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-05-09T09:25:58.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.941 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.942 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-05-09T09:25:58.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.947 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.948 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:58.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.950 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T09:25:58.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T09:25:58.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T09:25:58.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.959 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-05-09T09:25:58.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-05-09T09:25:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.966 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T09:25:58.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-05-09T09:25:58.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-05-09T09:25:58.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-05-09T09:25:58.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.985 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.990 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T09:25:58.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-05-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-05-09T09:25:58.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T09:25:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.020 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.027 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T09:25:59.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T09:25:59.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:59.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.036 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.037 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.050 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.052 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T09:25:59.053 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T09:25:59.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-05-09T09:25:59.062 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.065 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.069 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.070 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:59.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:59.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-05-09T09:25:59.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.082 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-05-09T09:25:59.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.093 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.094 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-05-09T09:25:59.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.101 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-05-09T09:25:59.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.109 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-05-09T09:25:59.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-05-09T09:25:59.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.112 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.113 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.114 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.119 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.120 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.122 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T09:25:59.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.126 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T09:25:59.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.132 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.134 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.135 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-05-09T09:25:59.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.142 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.144 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.146 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.147 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.150 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-05-09T09:25:59.151 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.152 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.154 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.156 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.158 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.160 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.171 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.177 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-05-09T09:25:59.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.186 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.192 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.195 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.196 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T09:25:59.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-05-09T09:25:59.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-05-09T09:25:59.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-05-09T09:25:59.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.203 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.204 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.207 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-05-09T09:25:59.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.217 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.218 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.222 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.224 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.229 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.235 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-05-09T09:25:59.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-05-09T09:25:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.245 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-05-09T09:25:59.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.252 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.259 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.260 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.265 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:59.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.270 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.272 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.274 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-05-09T09:25:59.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-05-09T09:25:59.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T09:25:59.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.284 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.287 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.288 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-05-09T09:25:59.291 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-05-09T09:25:59.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.293 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-05-09T09:25:59.294 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.295 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.300 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.303 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-05-09T09:25:59.304 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-05-09T09:25:59.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-05-09T09:25:59.308 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.310 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-05-09T09:25:59.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.313 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-05-09T09:25:59.314 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-05-09T09:25:59.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.316 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-05-09T09:25:59.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "services": { 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard": "https://172.21.15.64:8443/" 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-09T09:25:59.319 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-09T09:25:59.320 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-09T09:25:59.321 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-05-09T09:25:59.322 INFO:teuthology.orchestra.run.smithi064.stdout: "squid": [ 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 3, 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 608771884 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-09T09:25:59.324 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2225685543 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1191000631 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 31296508 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-09T09:25:59.326 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:25:59.332 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T09:25:59.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T09:25:59.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:59 smithi139 ceph-mon[33599]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-05-09T09:25:59.333 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:25:59 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/146472875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T09:25:59.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:59 smithi064 ceph-mon[30149]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-05-09T09:25:59.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:25:59 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/146472875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T09:25:59.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:59 smithi179 ceph-mon[34086]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-05-09T09:25:59.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:25:59 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/146472875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T09:25:59.604 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:26:01.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:01 smithi139 ceph-mon[33599]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-09T09:26:01.377 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:01 smithi064 ceph-mon[30149]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-09T09:26:01.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:01 smithi179 ceph-mon[34086]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-09T09:26:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001618164, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000391471, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:01.999 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 238, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 197376, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.018867190, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.013590867, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001791092, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:02.001 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 236, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.015349353, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000065039 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 197376 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.002 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 319, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 314, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 508656, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 193008, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.048343912, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.019901356, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.018684316, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.008903013, 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:02.003 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 314, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.029717380, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000094641 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 508656, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 193008 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.004 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:02.005 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:02.006 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-09T09:26:02.007 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148744, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:02.008 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:02.009 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 317, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 461198, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 317, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 317, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 461198, 2024-05-09T09:26:02.010 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-09T09:26:02.011 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:02.011 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:02.011 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:02.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:02.011 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:02.011 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:26:02.765 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-09T09:26:02.765 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-09T09:26:02.765 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T09:26:02.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T09:26:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:26:03.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:03 smithi139 ceph-mon[33599]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T09:26:03.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:03 smithi064 ceph-mon[30149]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T09:26:03.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:03 smithi179 ceph-mon[34086]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T09:26:04.894 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:26:04.894 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:04.894 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-05-09T09:26:04.894 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001373277, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000339942, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:04.895 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:04.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 325, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 320, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 510288, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 196848, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.049064902, 2024-05-09T09:26:04.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.020785662, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.018975378, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.008005119, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 320, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.017100734, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000053439 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:04.898 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:04.899 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:04.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 510288, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 196848 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 243, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 198256, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.019542093, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.016332081, 2024-05-09T09:26:04.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001438708, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 241, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.011895007, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000049356 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 198256 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-09T09:26:04.901 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:04.902 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:04.903 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148784, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-09T09:26:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:04.905 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 323, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 461978, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 323, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 323, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 461978, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:04.906 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:04.907 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:04.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:04.907 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:04.907 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:26:05.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:05 smithi064 ceph-mon[30149]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T09:26:05.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:05 smithi139 ceph-mon[33599]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T09:26:05.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:05 smithi179 ceph-mon[34086]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T09:26:05.588 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T09:26:05.588 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T09:26:05.588 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T09:26:05.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T09:26:05.844 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:26:07.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:07 smithi139 ceph-mon[33599]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:26:07.331 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:07 smithi064 ceph-mon[30149]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:26:07.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:07 smithi179 ceph-mon[34086]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:26:07.851 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:26:07.851 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:07.851 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 321, 2024-05-09T09:26:07.851 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 316, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 509200, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 194288, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.049619825, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.019600360, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.019783198, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.008490627, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 316, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.021565424, 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000068245 2024-05-09T09:26:07.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 509200, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 194288 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 240, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 197728, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:07.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.021603755, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.015324536, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002146869, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 238, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.018786544, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000078935 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:07.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 197728 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000417004, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000336553, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:07.855 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-09T09:26:07.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:07.857 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:07.858 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148704, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 29, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:07.859 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-09T09:26:07.860 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 319, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 461458, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 319, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-09T09:26:07.861 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 319, 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 461458, 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:07.862 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:26:08.603 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-09T09:26:08.603 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-09T09:26:08.603 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T09:26:08.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T09:26:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T09:26:09.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:09 smithi064 ceph-mon[30149]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:26:09.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:09 smithi139 ceph-mon[33599]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:26:09.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:09 smithi179 ceph-mon[34086]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T09:26:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-09T09:26:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 327, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 322, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 510832, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 198128, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.054471220, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.021541218, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.022014741, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.008479524, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 322, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.019420414, 2024-05-09T09:26:10.487 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000060311 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.488 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 510832, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 198128 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 245, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 198608, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.020005268, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.015608024, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001501011, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:10.489 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 243, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.012880691, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000053006 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:10.490 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 198608 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000327414, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T09:26:10.491 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000250029, 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:10.492 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:10.493 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:10.494 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:10.495 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:10.496 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148721, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-09T09:26:10.498 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-09T09:26:10.498 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:10.498 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:10.498 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-09T09:26:10.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 325, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 462238, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 325, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-09T09:26:10.500 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 325, 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 462238, 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-09T09:26:10.501 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-09T09:26:11.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:11 smithi064 ceph-mon[30149]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T09:26:11.296 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T09:26:11.297 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T09:26:11.297 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T09:26:11.297 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T09:26:11.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:11 smithi139 ceph-mon[33599]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T09:26:11.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:11 smithi179 ceph-mon[34086]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T09:26:11.634 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:26:13.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:13 smithi139 ceph-mon[33599]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:13.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:13 smithi064 ceph-mon[30149]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:13.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:13 smithi179 ceph-mon[34086]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:14.087 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 11281, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 12219, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 4008620, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 19249123, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 394, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.815868700, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.244641310, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.407970172, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.135121736, 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 12220, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.429975113, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000117019 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 12624, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002555620, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000202 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:14.089 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 17104, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 19250, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 6571021, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 14082053, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 12, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 1.070097865, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.376127759, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.629330184, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.161462489, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 19250, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.737075636, 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000090237 2024-05-09T09:26:14.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 24484, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.004208707, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000171 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 741728, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 268416 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 9075, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 9926, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 4170553, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 9859968, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 14, 2024-05-09T09:26:14.091 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 7, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.575573787, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.195414161, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.211604145, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.141421444, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 9921, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.248980148, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000125892 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 7246, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001397628, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000192 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.092 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 513984 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 12200, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 5.215653550, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000427512 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.093 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 915, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.063706852, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000069624 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 5522, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 4042, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 4042, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 14.577312543, 2024-05-09T09:26:14.094 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003606460 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 1480, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 11, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 4, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 975, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 3, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 275, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 3, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 261, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 11, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 270, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 1526, 2024-05-09T09:26:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 253, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 253, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 10, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 7285, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 5694, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 1464, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 5, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 778, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 37, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 1544, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 25, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 809, 2024-05-09T09:26:14.096 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 7178, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 1019, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 1572287829, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 7178, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 848, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 30, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 818, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 6330, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 5, 2024-05-09T09:26:14.097 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 1297, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 1839, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 5378, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 1863, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 66, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 2102 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 7, 2024-05-09T09:26:14.098 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 24, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 224, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 223, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 1, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:26:14.099 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 30, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 42, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 3, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 8056, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 7976, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 7976, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 80, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 21, 2024-05-09T09:26:14.100 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 81, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 80, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 80, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 1, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 1, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 1, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 22581430, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 22689683, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2791, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 30.828761888, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.011045776 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-05-09T09:26:14.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 275, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 2019, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 1744, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 31, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 84, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 53, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 275, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 2047, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 1772, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 253, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 2003, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 1750, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 72752, 2024-05-09T09:26:14.102 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 231804 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 6655, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 481, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 5522, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 63, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 1134, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1176, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 6.406223642, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.005447469 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 403, 2024-05-09T09:26:14.103 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.140073225, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000347576 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002752910, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002752910 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1051, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 6.182448609, 2024-05-09T09:26:14.104 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.005882443 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.105 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 37, 2024-05-09T09:26:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.042898776, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001159426 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1063, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.701130304, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000659576 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 66, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.345785634, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.005239176 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 215, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.642782707, 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002989687 2024-05-09T09:26:14.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.004463896, 2024-05-09T09:26:14.110 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002231948 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2, 2024-05-09T09:26:14.111 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002079325, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001039662 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.112 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 8, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.020990297, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002623787 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-05-09T09:26:14.113 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 18, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.085683218, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.004760178 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-05-09T09:26:14.114 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 3, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 2, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 244, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 244, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 1025, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:14.115 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:14.116 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:14.117 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 299826, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 119, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8888, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 170, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:14.118 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 922148, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 4819, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 15104, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21665553, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 19, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 15104, 2024-05-09T09:26:14.119 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 15104, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 368.840160388, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.024420031 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 15104, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 20498 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 15104, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 7373, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 7731, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-05-09T09:26:14.120 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 904, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 2148, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 150, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 2911, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 348, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 3, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 994, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 7223, 2024-05-09T09:26:14.121 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 3258, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-05-09T09:26:14.122 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 94, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 9, 2024-05-09T09:26:14.123 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 12, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 12, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 2355, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 30, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 174 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-05-09T09:26:14.124 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 22, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 8, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 474, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 223, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:14.125 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 37456, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 11347735, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 37456, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 37456, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 11347735, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-05-09T09:26:14.126 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 15104, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 21689652, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 15104, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 21689652, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.127 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 15104, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 15104, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 15104, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 15104, 2024-05-09T09:26:14.128 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 223, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 24291, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 223, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:14.129 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 120, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 24291, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55f5a798e500": { 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 8056, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 18495379, 2024-05-09T09:26:14.130 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 8056, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2791, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 18495379, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:14.131 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-05-09T09:26:15.073 INFO:tasks.check_counter:Daemon mds.b mds.exported=37 2024-05-09T09:26:15.073 INFO:tasks.check_counter:Daemon mds.b mds.imported=25 2024-05-09T09:26:15.073 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T09:26:15.073 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T09:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:15 smithi139 ceph-mon[33599]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:15.362 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:26:15.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:15 smithi064 ceph-mon[30149]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:15.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:15 smithi179 ceph-mon[34086]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:17.206 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:17 smithi139 ceph-mon[33599]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:17.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:17 smithi064 ceph-mon[30149]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:17.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:17 smithi179 ceph-mon[34086]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 362, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 330, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 651250, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 199540, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.045429872, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.017077111, 2024-05-09T09:26:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.019719132, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.007337150, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 329, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.013222726, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000040190 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 650416, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 199216 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1043, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1286, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1089367, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 832833, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.090076863, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.029475277, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.018397455, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.033207725, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1284, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.038380381, 2024-05-09T09:26:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000029891 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 501472 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 2562, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2562, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 4947105, 2024-05-09T09:26:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 813950, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.201794259, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.065242215, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.053619235, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.067221903, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2561, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.076410249, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000029836 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-05-09T09:26:17.497 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6128, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.145640779, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000023766 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-05-09T09:26:17.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 267, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 267, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 267, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 3, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-05-09T09:26:17.500 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:26:17.501 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:26:17.502 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:26:17.503 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:26:17.504 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 3005, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-05-09T09:26:17.505 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 5, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 2, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 9220501, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 9219927, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 3005 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 72, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 1503, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 1431, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 13, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 54, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 41, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 267, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 1719, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 1452, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-05-09T09:26:17.507 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 57016, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215420 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.508 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-05-09T09:26:17.509 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.511 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.512 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-05-09T09:26:17.513 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.513 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.513 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.513 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.516 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.517 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.518 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:26:17.519 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 1, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 2142, 2024-05-09T09:26:17.520 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:17.521 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:17.522 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 198327, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 58, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:17.523 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8888, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 170, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 367281, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 11512, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.524 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 3601, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 3601, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3601, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 4.071915720, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.001130773 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-05-09T09:26:17.525 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3601, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3602 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 3601, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 3601, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 2174, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1425, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-05-09T09:26:17.526 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:26:17.527 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-05-09T09:26:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 94, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 4, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 6, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 7, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 1, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-05-09T09:26:17.529 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-05-09T09:26:17.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 3963, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 6309643, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 3963, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3963, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 6309643, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.531 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 3601, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 5026202, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3601, 2024-05-09T09:26:17.532 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 5026202, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:17.533 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 3601, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 3601, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3601, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 3601, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:17.534 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:17.535 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x56264d7246e0": { 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:17.536 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:17.537 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.537 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-05-09T09:26:18.429 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T09:26:18.430 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T09:26:18.430 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T09:26:18.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T09:26:18.674 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:26:19.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:19 smithi139 ceph-mon[33599]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:19.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:19 smithi064 ceph-mon[30149]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:19.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:19 smithi179 ceph-mon[34086]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:20.424 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-05-09T09:26:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1509, 2024-05-09T09:26:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1477, 2024-05-09T09:26:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1036208, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 563958, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.149902526, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.055137610, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.044104396, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.040927130, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1476, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.063048395, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000042715 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 650144, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 198576 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 447, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 689, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 2304717, 2024-05-09T09:26:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 644802, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.043954548, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.016885911, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.010776754, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.012013097, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 687, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.026242808, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000038199 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 502912 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 2099, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2099, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 13126035, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 666665, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.128417308, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.029377933, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.044461314, 2024-05-09T09:26:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.046465069, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2098, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.055318745, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000026367 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6278, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.134387462, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000021406 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-05-09T09:26:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 896, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 431, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 465, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 280, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-05-09T09:26:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 11, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-05-09T09:26:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:26:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-05-09T09:26:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-05-09T09:26:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 10676, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 14, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 2, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-05-09T09:26:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 18969465, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 18968891, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 10676 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 753, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 1334, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 581, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 61, 2024-05-09T09:26:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 76, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 15, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 896, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 1568, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 672, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 55276, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215420 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-05-09T09:26:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-05-09T09:26:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-05-09T09:26:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-05-09T09:26:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-05-09T09:26:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-05-09T09:26:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 1, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-05-09T09:26:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 2145, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 190991, 2024-05-09T09:26:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 55, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 8888, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 170, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 2274531, 2024-05-09T09:26:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 5204, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 3690, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 3690, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3690, 2024-05-09T09:26:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3.547926310, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000961497 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3690, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3691 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 3690, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 3690, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 2182, 2024-05-09T09:26:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1506, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-05-09T09:26:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-05-09T09:26:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 94, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 4, 2024-05-09T09:26:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 5, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 6, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 1, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 4051, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 16081013, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 4051, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 4051, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 16081013, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-05-09T09:26:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 3690, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 14782882, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3690, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 14782882, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 3690, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 3690, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3690, 2024-05-09T09:26:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 3690, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55fd26f7c6e0": { 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-05-09T09:26:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:20.469 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-05-09T09:26:21.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:21 smithi139 ceph-mon[33599]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:21.247 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T09:26:21.247 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T09:26:21.247 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T09:26:21.248 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T09:26:21.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:21 smithi064 ceph-mon[30149]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:21.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:21 smithi179 ceph-mon[34086]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T09:26:22.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:22 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:26:22.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:22 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:26:22.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:22 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:26:23.411 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:23 smithi064 ceph-mon[30149]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:23.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:23 smithi139 ceph-mon[33599]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:23.471 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 0, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 0, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 0, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 0, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.001072038, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T09:26:23.472 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.000304730, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.473 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 2, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 248, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 320, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 199136, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.012520652, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.008947151, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.001456260, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:23.474 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 246, 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.011887194, 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000048321 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:23.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 199136 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 331, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 326, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 511920, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 200688, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 2, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.041977277, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.016438965, 2024-05-09T09:26:23.476 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.016801251, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.007172831, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 326, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.013307929, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000040821 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 511920, 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 200688 2024-05-09T09:26:23.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:23.478 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:23.479 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 148704, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 29, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:23.480 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 928, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 2, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:23.481 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 329, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 462758, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 329, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-05-09T09:26:23.482 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 329, 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 462758, 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-05-09T09:26:23.483 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-05-09T09:26:23.488 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:23 smithi179 ceph-mon[34086]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:24.150 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-09T09:26:24.150 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-09T09:26:24.150 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T09:26:24.151 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T09:26:24.434 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:26:25.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:25 smithi064 ceph-mon[30149]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:25.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:25 smithi179 ceph-mon[34086]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:25.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:25 smithi139 ceph-mon[33599]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:26.354 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 15081, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 14473, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 8389086, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 10232884, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 69, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.792667627, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.256714952, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.322767144, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.193669541, 2024-05-09T09:26:26.355 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 14475, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.615831982, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000042544 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7385, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001576871, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000213 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 8036, 2024-05-09T09:26:26.356 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 8280, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 4527934, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 4136160, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.479142245, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.185665339, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.107770455, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.139399339, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8277, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.349770179, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000042258 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:26.357 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 594784 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 17893, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 23994, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 6499994, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 11937433, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 12, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 1.089762049, 2024-05-09T09:26:26.358 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.474413041, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.691703713, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.040503654, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 23991, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.762725319, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000031792 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 33639, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.006382940, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000189 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 674512, 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 231968 2024-05-09T09:26:26.359 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 18179, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.430781365, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000188722 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-09T09:26:26.360 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 46, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.002036938, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000044281 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 6812, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 6703, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6703, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.824366827, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001018106 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-09T09:26:26.361 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 109, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 269, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 152, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 117, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 268, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 1456, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 62, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 62, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 6, 2024-05-09T09:26:26.362 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 11845, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 11085, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 109, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 11, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 87, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 22, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 1440, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 34, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 1591, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 19667, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-09T09:26:26.363 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 1023, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 1572918318, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 19667, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 86, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 86, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 19581, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 9, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 1524, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 1638, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:26:26.364 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 4090, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 1623, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 3, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 168 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 15, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 7, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:26:26.365 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 55, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 9, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 46, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 23, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-09T09:26:26.366 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 3005, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 3005, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 5, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-09T09:26:26.367 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 5, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 9220501, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 802, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.052938197, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002559773 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-09T09:26:26.368 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 83, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 1506, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 1423, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 26, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 54, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 28, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 269, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 1734, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 1465, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 62, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1486, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 1424, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 64020, 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223612 2024-05-09T09:26:26.369 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 9096, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 323, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 6812, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 33, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 825, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 12, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.028583419, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002381951 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1310, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.318565179, 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001769897 2024-05-09T09:26:26.370 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 46, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.124289242, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002701940 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3040, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.601393989, 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000855721 2024-05-09T09:26:26.371 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.372 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.373 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.374 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2184, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.250703002, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000572666 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 10, 2024-05-09T09:26:26.375 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.008432103, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000843210 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 46, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.351359825, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.007638257 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.376 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 55, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.141040068, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002564364 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.380 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 1, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 0, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 1979, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1979, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1041, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:26.381 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:26.382 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:26.383 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 282166, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 108, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8888, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 170, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:26.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 447257, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 13550, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 20736, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 5196471, 2024-05-09T09:26:26.385 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 28, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 20736, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 20736, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 22.730547416, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001096187 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 20736, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 20906 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 20736, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 19672, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 1064, 2024-05-09T09:26:26.386 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 5, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 807, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 70, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 31, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-09T09:26:26.387 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 19667, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-09T09:26:26.388 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-05-09T09:26:26.389 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 237, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 59 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-09T09:26:26.390 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 5, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 18, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 9, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 41006, 2024-05-09T09:26:26.391 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 15700995, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 41006, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 41006, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 15700995, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:26.392 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 20736, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 5197452, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 20736, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 5197452, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.393 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 20736, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 20736, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 20736, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 20736, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 9, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 981, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 9, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 5, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 981, 2024-05-09T09:26:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55c66fbb2500": { 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3005, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 5026197, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3005, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:26.396 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 802, 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 5026197, 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:26.397 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-09T09:26:27.171 INFO:tasks.check_counter:Daemon mds.c mds.exported=22 2024-05-09T09:26:27.171 INFO:tasks.check_counter:Daemon mds.c mds.imported=34 2024-05-09T09:26:27.171 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T09:26:27.172 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T09:26:27.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:27 smithi064 ceph-mon[30149]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:27.431 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:26:27.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:27 smithi179 ceph-mon[34086]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:27.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:27 smithi139 ceph-mon[33599]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:29.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:29 smithi179 ceph-mon[34086]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:29.257 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2123, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2125, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 6547718, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 675124, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 6, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.174812541, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.049971462, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.049514264, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.060611764, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2123, 2024-05-09T09:26:29.258 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.080425252, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000037882 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1399, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1642, 2024-05-09T09:26:29.259 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 8754870, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 948862, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.104450718, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.034413348, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.031021294, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.030616048, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1641, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.059120109, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000036026 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.260 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 505504 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 797, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 764, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 4956327, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 338033, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-05-09T09:26:29.261 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.081802691, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.027911794, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.031184293, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.020409860, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 763, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.029205438, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000038277 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.262 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 655488, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 200656 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6686, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.178936827, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000026762 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.263 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.264 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 512, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 340, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 172, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 48, 2024-05-09T09:26:29.265 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 10, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-05-09T09:26:29.266 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 1019, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 1572287829, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:26:29.267 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-05-09T09:26:29.268 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-09T09:26:29.269 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-09T09:26:29.270 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 8056, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 75, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 22581430, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 22689683, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 22684531, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.271 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 8056 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 272, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 2010, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 1738, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 27, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 87, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 60, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 512, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 2062, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 1550, 2024-05-09T09:26:29.272 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 54080, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215420 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.273 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.274 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.277 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.278 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.279 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.280 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.283 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.284 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 2160, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.285 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:29.286 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:29.287 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 198281, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 58, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:29.288 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8888, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 170, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 964220, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 2957, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-09T09:26:29.289 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3949, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 26, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 4, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3949, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3949, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.528929787, 2024-05-09T09:26:29.290 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000893626 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3949, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3950 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3949, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3949, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 2324, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1622, 2024-05-09T09:26:29.291 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 2, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:26:29.292 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-09T09:26:29.293 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 6, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 6, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-09T09:26:29.294 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-09T09:26:29.295 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 4315, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 19848965, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 4315, 2024-05-09T09:26:29.296 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4315, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 19848965, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:29.297 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3949, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 18504345, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3949, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 18504345, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.298 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3949, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3949, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3949, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3949, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.299 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:29.300 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55877249a6e0": { 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:29.301 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:29.302 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-09T09:26:29.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:29 smithi064 ceph-mon[30149]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:29.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:29 smithi139 ceph-mon[33599]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:30.047 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T09:26:30.047 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T09:26:30.047 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T09:26:30.047 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T09:26:30.293 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:26:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:31 smithi064 ceph-mon[30149]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:31.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:31 smithi064 ceph-mon[30149]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:26:31.428 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:31 smithi179 ceph-mon[34086]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:31.428 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:31 smithi179 ceph-mon[34086]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:26:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:31 smithi139 ceph-mon[33599]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:31.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:31 smithi139 ceph-mon[33599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 16389, 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 12997, 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 6397312, 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 17493112, 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 66, 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-05-09T09:26:31.993 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.910681565, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.366366334, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.571846151, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.073090716, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 12997, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.340436607, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000026193 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 24016, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.003756232, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000156 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.994 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 685808, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 234016 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 15803, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 19509, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 5286733, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 8806925, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 17, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 1.013855599, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.390972330, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.772054891, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.015444572, 2024-05-09T09:26:31.995 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 19507, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.332213072, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000068294 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 28569, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.005755070, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000201 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 597248 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:31.996 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2857, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2859, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1026215, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 3168353, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 8, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.193774322, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.060375973, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.045822439, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.063249563, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2857, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.174503625, 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000061079 2024-05-09T09:26:31.997 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-09T09:26:31.998 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5995, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.772084703, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000462399 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 90, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.003567946, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000039643 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:31.999 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 4056, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 3126, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3126, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 9.649431130, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003086830 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 930, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-09T09:26:32.000 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 881, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 1, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 746, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 134, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 879, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 674, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 744, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 744, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 12, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 8551, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 5569, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 915, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 10, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-09T09:26:32.001 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 8, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 59, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 14, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 667, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 14, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 1251, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 3514, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 1023, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 1572918318, 2024-05-09T09:26:32.002 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 3514, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 118, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 118, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 3396, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 8, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 1709, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 2265, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T09:26:32.003 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 10919, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 8283, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 9, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 484 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 18, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 9, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-09T09:26:32.004 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 145, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 20, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 125, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 15, 2024-05-09T09:26:32.005 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 10676, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 10676, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 14, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-09T09:26:32.006 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 14, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 18969465, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1665, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.234255230, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003744297 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-05-09T09:26:32.007 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 760, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 1330, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 570, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 67, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 71, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 4, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 881, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 1576, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 695, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 744, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1318, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 574, 2024-05-09T09:26:32.008 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 65648, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223612 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 6674, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 289, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 4056, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 31, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 1317, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 81, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.197585585, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002439328 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-09T09:26:32.009 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 91, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.022774131, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000250265 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 125, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.394556376, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003156451 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.010 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2032, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.992371567, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003441127 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.011 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-09T09:26:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-09T09:26:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.014 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 483, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.375239535, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000776893 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-09T09:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.036138261, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000903456 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 145, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.305860705, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.009005935 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-09T09:26:32.016 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.017 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 129, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.324904970, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002518643 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-09T09:26:32.020 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 1, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 0, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 211, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 211, 2024-05-09T09:26:32.021 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1047, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:32.022 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:32.023 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 300846, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 122, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:32.024 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8888, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 170, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 2612995, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 12996, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:32.025 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 5634, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 14867354, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 5634, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5634, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 11.508770646, 2024-05-09T09:26:32.026 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002042735 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5634, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 5973 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 5634, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3525, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 2109, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 11, 2024-05-09T09:26:32.027 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 1676, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 78, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 55, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 3514, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-09T09:26:32.028 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-09T09:26:32.029 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-05-09T09:26:32.030 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 476, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 133 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-05-09T09:26:32.031 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 5, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 40, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 20, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 35045, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 9530576, 2024-05-09T09:26:32.032 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 35045, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 35045, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 9530576, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:32.033 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 5634, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 14869534, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 5634, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 14869534, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.034 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 5634, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 5634, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 5634, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 5634, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.035 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 20, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 2180, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 20, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 11, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2180, 2024-05-09T09:26:32.036 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55d9154fc500": { 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 10676, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 14775161, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 10676, 2024-05-09T09:26:32.037 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 1665, 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 14775161, 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:32.038 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-09T09:26:32.796 INFO:tasks.check_counter:Daemon mds.i mds.exported=14 2024-05-09T09:26:32.797 INFO:tasks.check_counter:Daemon mds.i mds.imported=14 2024-05-09T09:26:32.797 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T09:26:32.797 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 2877b902-0de2-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T09:26:33.040 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T09:26:33.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:33 smithi064 ceph-mon[30149]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:33.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:33 smithi179 ceph-mon[34086]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:33.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:33 smithi139 ceph-mon[33599]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:34.729 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-09T09:26:34.729 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T09:26:34.729 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 0, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 0, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 0, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 0, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.001087949, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.000306190, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:34.730 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 251, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 320, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 199664, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-05-09T09:26:34.731 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.012242281, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.008958180, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.001066305, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 249, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.009974804, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040059 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:34.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 199664 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 334, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 329, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 512736, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 202608, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 2, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.041976772, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.015785077, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.017705862, 2024-05-09T09:26:34.733 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.007308066, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 329, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.013222714, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040190 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 512736, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 202608 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.734 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T09:26:34.735 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-09T09:26:34.736 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 148784, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 31, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-09T09:26:34.737 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 928, 2024-05-09T09:26:34.738 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 2, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 332, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 463148, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 332, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-09T09:26:34.739 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 332, 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 463148, 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-09T09:26:34.740 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-09T09:26:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:35 smithi179 ceph-mon[34086]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:35.384 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-09T09:26:35.384 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-09T09:26:35.385 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-09T09:26:35.395 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T09:26:35.395 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:26:35.395 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:26:35.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:35 smithi064 ceph-mon[30149]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:35.428 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:26:35.428 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:26:35.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:26:35.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:35 smithi139 ceph-mon[33599]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:35.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 -- 172.21.15.64:0/685473384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288101880 msgr2=0x7f1288101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 --2- 172.21.15.64:0/685473384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288101880 0x7f1288101ca0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1284009900 tx=0x7f128402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 -- 172.21.15.64:0/685473384 shutdown_connections 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 --2- 172.21.15.64:0/685473384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1288103430 0x7f1288109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 --2- 172.21.15.64:0/685473384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1288102a70 0x7f1288102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 --2- 172.21.15.64:0/685473384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288101880 0x7f1288101ca0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 -- 172.21.15.64:0/685473384 >> 172.21.15.64:0/685473384 conn(0x7f12880fce30 msgr2=0x7f12880ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 -- 172.21.15.64:0/685473384 shutdown_connections 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 -- 172.21.15.64:0/685473384 wait complete. 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.646+0000 7f12906fc640 1 Processor -- start 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 -- start start 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1288101880 0x7f12880755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1288102a70 0x7f1288073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 0x7f1288074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f128810d3b0 con 0x7f1288103430 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f128810d230 con 0x7f1288101880 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f12906fc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f128810d530 con 0x7f1288102a70 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 0x7f1288074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 0x7f1288074160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38212/0 (socket says 172.21.15.64:38212) 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 -- 172.21.15.64:0/752844523 learned_addr learned my addr 172.21.15.64:0/752844523 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128e498640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1288101880 0x7f12880755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128dc97640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1288102a70 0x7f1288073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 -- 172.21.15.64:0/752844523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1288102a70 msgr2=0x7f1288073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1288102a70 0x7f1288073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 -- 172.21.15.64:0/752844523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1288101880 msgr2=0x7f12880755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1288101880 0x7f12880755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 -- 172.21.15.64:0/752844523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12880749f0 con 0x7f1288103430 2024-05-09T09:26:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.647+0000 7f128ec99640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 0x7f1288074160 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f128000eb10 tx=0x7f128000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.648+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280004b20 con 0x7f1288103430 2024-05-09T09:26:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.648+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12881a1c00 con 0x7f1288103430 2024-05-09T09:26:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.648+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1280004cc0 con 0x7f1288103430 2024-05-09T09:26:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.648+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280010690 con 0x7f1288103430 2024-05-09T09:26:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.648+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12881a2150 con 0x7f1288103430 2024-05-09T09:26:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.649+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1288108c20 con 0x7f1288103430 2024-05-09T09:26:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.650+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1280005000 con 0x7f1288103430 2024-05-09T09:26:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.650+0000 7f127f7fe640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12740a2b90 0x7f12740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.651+0000 7f128e498640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12740a2b90 0x7f12740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.651+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f12800cfdb0 con 0x7f1288103430 2024-05-09T09:26:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.651+0000 7f128e498640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12740a2b90 0x7f12740a5050 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1284004770 tx=0x7f1284000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.654+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f128009c980 con 0x7f1288103430 2024-05-09T09:26:35.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.839+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f128802d410 con 0x7f1288103430 2024-05-09T09:26:35.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.841+0000 7f127f7fe640 1 -- 172.21.15.64:0/752844523 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f128009c280 con 0x7f1288103430 2024-05-09T09:26:35.841 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:26:35.841 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.842+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12740a2b90 msgr2=0x7f12740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.842+0000 7f12906fc640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12740a2b90 0x7f12740a5050 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1284004770 tx=0x7f1284000bc0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.842+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 msgr2=0x7f1288074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.842+0000 7f12906fc640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 0x7f1288074160 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f128000eb10 tx=0x7f128000c2d0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 shutdown_connections 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f12740a2b90 0x7f12740a5050 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1288103430 0x7f1288074160 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1288102a70 0x7f1288073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 --2- 172.21.15.64:0/752844523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1288101880 0x7f12880755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 >> 172.21.15.64:0/752844523 conn(0x7f12880fce30 msgr2=0x7f1288105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:35.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 shutdown_connections 2024-05-09T09:26:35.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.843+0000 7f12906fc640 1 -- 172.21.15.64:0/752844523 wait complete. 2024-05-09T09:26:35.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:26:35.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 -- 172.21.15.64:0/3841010401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 msgr2=0x7f4244102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 --2- 172.21.15.64:0/3841010401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244102ef0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4234009900 tx=0x7f423402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 -- 172.21.15.64:0/3841010401 shutdown_connections 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 --2- 172.21.15.64:0/3841010401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4244103430 0x7f4244109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 --2- 172.21.15.64:0/3841010401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244102ef0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 --2- 172.21.15.64:0/3841010401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 0x7f4244101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 -- 172.21.15.64:0/3841010401 >> 172.21.15.64:0/3841010401 conn(0x7f42440fce10 msgr2=0x7f42440ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 -- 172.21.15.64:0/3841010401 shutdown_connections 2024-05-09T09:26:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 -- 172.21.15.64:0/3841010401 wait complete. 2024-05-09T09:26:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.963+0000 7f4249053640 1 Processor -- start 2024-05-09T09:26:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 -- start start 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 0x7f42440755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4244103430 0x7f4244074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d390 con 0x7f4244102a70 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d210 con 0x7f4244103430 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4249053640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d510 con 0x7f4244101880 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38236/0 (socket says 172.21.15.64:38236) 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 -- 172.21.15.64:0/575523350 learned_addr learned my addr 172.21.15.64:0/575523350 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424359e640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4244103430 0x7f4244074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4242d9d640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 0x7f42440755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 -- 172.21.15.64:0/575523350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 msgr2=0x7f42440755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 0x7f42440755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 -- 172.21.15.64:0/575523350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4244103430 msgr2=0x7f4244074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4244103430 0x7f4244074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f424259c640 1 -- 172.21.15.64:0/575523350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440749f0 con 0x7f4244102a70 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.964+0000 7f4242d9d640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 0x7f42440755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.965+0000 7f424259c640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244073c20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4234004770 tx=0x7f42340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.965+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423403f030 con 0x7f4244102a70 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.965+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4234003cb0 con 0x7f4244102a70 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.965+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42441a1b40 con 0x7f4244102a70 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.965+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234035070 con 0x7f4244102a70 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.965+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441a2090 con 0x7f4244102a70 2024-05-09T09:26:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.966+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244108c90 con 0x7f4244102a70 2024-05-09T09:26:35.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.967+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4234042030 con 0x7f4244102a70 2024-05-09T09:26:35.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.968+0000 7f423bfff640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42300a2b90 0x7f42300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:35.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.968+0000 7f4242d9d640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42300a2b90 0x7f42300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:35.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.968+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f42340efd60 con 0x7f4244102a70 2024-05-09T09:26:35.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.968+0000 7f4242d9d640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42300a2b90 0x7f42300a5050 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f422c005d00 tx=0x7f422c005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:35.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:35.972+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f42340bc930 con 0x7f4244102a70 2024-05-09T09:26:36.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.156+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4244062430 con 0x7f4244102a70 2024-05-09T09:26:36.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.156+0000 7f423bfff640 1 -- 172.21.15.64:0/575523350 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f42340bc230 con 0x7f4244102a70 2024-05-09T09:26:36.156 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42300a2b90 msgr2=0x7f42300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42300a2b90 0x7f42300a5050 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f422c005d00 tx=0x7f422c005c90 comp rx=0 tx=0).stop 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 msgr2=0x7f4244073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244073c20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4234004770 tx=0x7f42340042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 shutdown_connections 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f42300a2b90 0x7f42300a5050 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4244103430 0x7f4244074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4244102a70 0x7f4244073c20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 --2- 172.21.15.64:0/575523350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4244101880 0x7f42440755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 >> 172.21.15.64:0/575523350 conn(0x7f42440fce10 msgr2=0x7f4244105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:36.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 shutdown_connections 2024-05-09T09:26:36.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.158+0000 7f4249053640 1 -- 172.21.15.64:0/575523350 wait complete. 2024-05-09T09:26:36.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:26:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 -- 172.21.15.64:0/1961292054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00100770 msgr2=0x7fac00100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 --2- 172.21.15.64:0/1961292054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00100770 0x7fac00100b90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009900 tx=0x7fabf002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 -- 172.21.15.64:0/1961292054 shutdown_connections 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 --2- 172.21.15.64:0/1961292054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac00102320 0x7fac00108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 --2- 172.21.15.64:0/1961292054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac00101960 0x7fac00101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 --2- 172.21.15.64:0/1961292054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00100770 0x7fac00100b90 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 -- 172.21.15.64:0/1961292054 >> 172.21.15.64:0/1961292054 conn(0x7fac000fbce0 msgr2=0x7fac000fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 -- 172.21.15.64:0/1961292054 shutdown_connections 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 -- 172.21.15.64:0/1961292054 wait complete. 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 Processor -- start 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.278+0000 7fac04cb3640 1 -- start start 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac00100770 0x7fac00197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac00101960 0x7fac00198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 0x7fac0019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fac0010c2c0 con 0x7fac00102320 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fac0010c140 con 0x7fac00100770 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fac0010c440 con 0x7fac00101960 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 0x7fac0019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 0x7fac0019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38254/0 (socket says 172.21.15.64:38254) 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 -- 172.21.15.64:0/245699073 learned_addr learned my addr 172.21.15.64:0/245699073 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfdd9b640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac00101960 0x7fac00198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfe59c640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac00100770 0x7fac00197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 -- 172.21.15.64:0/245699073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac00101960 msgr2=0x7fac00198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac00101960 0x7fac00198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 -- 172.21.15.64:0/245699073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac00100770 msgr2=0x7fac00197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac00100770 0x7fac00197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 -- 172.21.15.64:0/245699073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0019f6e0 con 0x7fac00102320 2024-05-09T09:26:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabfed9d640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 0x7fac0019f070 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fabf800cce0 tx=0x7fabf800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf800f450 con 0x7fac00102320 2024-05-09T09:26:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabf800faa0 con 0x7fac00102320 2024-05-09T09:26:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0019f9d0 con 0x7fac00102320 2024-05-09T09:26:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.279+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8018b70 con 0x7fac00102320 2024-05-09T09:26:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.280+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0019ff20 con 0x7fac00102320 2024-05-09T09:26:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.281+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac001a01b0 con 0x7fac00102320 2024-05-09T09:26:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.282+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fabf8018d10 con 0x7fac00102320 2024-05-09T09:26:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.283+0000 7fabef7fe640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fabe80a2c10 0x7fabe80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.283+0000 7fabfe59c640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fabe80a2c10 0x7fabe80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.283+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fabf80cf670 con 0x7fac00102320 2024-05-09T09:26:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.283+0000 7fabfe59c640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fabe80a2c10 0x7fabe80a50d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005ab0 tx=0x7fabf0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.287+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fabf8014030 con 0x7fac00102320 2024-05-09T09:26:36.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:36 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/752844523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:36.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.473+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac001a0380 con 0x7fac00102320 2024-05-09T09:26:36.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.474+0000 7fabef7fe640 1 -- 172.21.15.64:0/245699073 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fabf809c240 con 0x7fac00102320 2024-05-09T09:26:36.474 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fabe80a2c10 msgr2=0x7fabe80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fabe80a2c10 0x7fabe80a50d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005ab0 tx=0x7fabf0005a20 comp rx=0 tx=0).stop 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 msgr2=0x7fac0019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 0x7fac0019f070 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fabf800cce0 tx=0x7fabf800a620 comp rx=0 tx=0).stop 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 shutdown_connections 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fabe80a2c10 0x7fabe80a50d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac00102320 0x7fac0019f070 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac00101960 0x7fac00198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 --2- 172.21.15.64:0/245699073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac00100770 0x7fac00197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 >> 172.21.15.64:0/245699073 conn(0x7fac000fbce0 msgr2=0x7fac000fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 shutdown_connections 2024-05-09T09:26:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.476+0000 7fac04cb3640 1 -- 172.21.15.64:0/245699073 wait complete. 2024-05-09T09:26:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:36 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/752844523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:36.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:26:36.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:36 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/752844523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:36.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 -- 172.21.15.64:0/3449391657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 msgr2=0x7fa1bc073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/3449391657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc073c40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009900 tx=0x7fa1ac02a480 comp rx=0 tx=0).stop 2024-05-09T09:26:36.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 -- 172.21.15.64:0/3449391657 shutdown_connections 2024-05-09T09:26:36.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/3449391657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1bc109740 0x7fa1bc10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/3449391657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1bc074210 0x7fa1bc074670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/3449391657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc073c40 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 -- 172.21.15.64:0/3449391657 >> 172.21.15.64:0/3449391657 conn(0x7fa1bc0fce00 msgr2=0x7fa1bc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 -- 172.21.15.64:0/3449391657 shutdown_connections 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.595+0000 7fa1c15b7640 1 -- 172.21.15.64:0/3449391657 wait complete. 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 Processor -- start 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 -- start start 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1bc074210 0x7fa1bc199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1bc109740 0x7fa1bc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10f5c0 con 0x7fa1bc0757f0 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10f440 con 0x7fa1bc074210 2024-05-09T09:26:36.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1c15b7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10f740 con 0x7fa1bc109740 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1ba7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1ba7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38262/0 (socket says 172.21.15.64:38262) 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1ba7fc640 1 -- 172.21.15.64:0/2260625969 learned_addr learned my addr 172.21.15.64:0/2260625969 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.596+0000 7fa1bb7fe640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1bc109740 0x7fa1bc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1baffd640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1bc074210 0x7fa1bc199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1ba7fc640 1 -- 172.21.15.64:0/2260625969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1bc109740 msgr2=0x7fa1bc1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1ba7fc640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1bc109740 0x7fa1bc1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1ba7fc640 1 -- 172.21.15.64:0/2260625969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1bc074210 msgr2=0x7fa1bc199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1ba7fc640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1bc074210 0x7fa1bc199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1ba7fc640 1 -- 172.21.15.64:0/2260625969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc1a1940 con 0x7fa1bc0757f0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1ba7fc640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc19a270 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000e9f0 tx=0x7fa1b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b000cd80 con 0x7fa1bc0757f0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc1a1c30 con 0x7fa1bc0757f0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1b00054d0 con 0x7fa1bc0757f0 2024-05-09T09:26:36.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0010490 con 0x7fa1bc0757f0 2024-05-09T09:26:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.597+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1a2180 con 0x7fa1bc0757f0 2024-05-09T09:26:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.598+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc108500 con 0x7fa1bc0757f0 2024-05-09T09:26:36.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.599+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa1b0005020 con 0x7fa1bc0757f0 2024-05-09T09:26:36.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.600+0000 7fa1a3fff640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa1a40a2b90 0x7fa1a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.600+0000 7fa1baffd640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa1a40a2b90 0x7fa1a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.600+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa1b00cf870 con 0x7fa1bc0757f0 2024-05-09T09:26:36.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.600+0000 7fa1baffd640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa1a40a2b90 0x7fa1a40a5050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac006010 tx=0x7fa1ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:36.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.604+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa1b009c440 con 0x7fa1bc0757f0 2024-05-09T09:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.805+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1bc062430 con 0x7fa1bc0757f0 2024-05-09T09:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.806+0000 7fa1a3fff640 1 -- 172.21.15.64:0/2260625969 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fa1b009bd40 con 0x7fa1bc0757f0 2024-05-09T09:26:36.805 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:26:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.807+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa1a40a2b90 msgr2=0x7fa1a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.807+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa1a40a2b90 0x7fa1a40a5050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac006010 tx=0x7fa1ac000bc0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.807+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 msgr2=0x7fa1bc19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.807+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc19a270 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000e9f0 tx=0x7fa1b000edf0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 shutdown_connections 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa1a40a2b90 0x7fa1a40a5050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1bc109740 0x7fa1bc1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0757f0 0x7fa1bc19a270 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 --2- 172.21.15.64:0/2260625969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1bc074210 0x7fa1bc199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 >> 172.21.15.64:0/2260625969 conn(0x7fa1bc0fce00 msgr2=0x7fa1bc1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 shutdown_connections 2024-05-09T09:26:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.808+0000 7fa1c15b7640 1 -- 172.21.15.64:0/2260625969 wait complete. 2024-05-09T09:26:36.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:26:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/1670574878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8100790 msgr2=0x7fd1e8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/1670574878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8100790 0x7fd1e8100bb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/1670574878 shutdown_connections 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/1670574878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8102340 0x7fd1e8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/1670574878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8101980 0x7fd1e8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/1670574878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8100790 0x7fd1e8100bb0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/1670574878 >> 172.21.15.64:0/1670574878 conn(0x7fd1e80fbd20 msgr2=0x7fd1e80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/1670574878 shutdown_connections 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/1670574878 wait complete. 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.927+0000 7fd1ef0ee640 1 Processor -- start 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 -- start start 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8100790 0x7fd1e8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 0x7fd1e8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8102340 0x7fd1e819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e810d3e0 con 0x7fd1e8101980 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e810d260 con 0x7fd1e8100790 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ef0ee640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e810d560 con 0x7fd1e8102340 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 0x7fd1e8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 0x7fd1e8197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38286/0 (socket says 172.21.15.64:38286) 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 -- 172.21.15.64:0/2499867928 learned_addr learned my addr 172.21.15.64:0/2499867928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ece8a640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8100790 0x7fd1e8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ed68b640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8102340 0x7fd1e819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 -- 172.21.15.64:0/2499867928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8102340 msgr2=0x7fd1e819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8102340 0x7fd1e819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 -- 172.21.15.64:0/2499867928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8100790 msgr2=0x7fd1e8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8100790 0x7fd1e8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 -- 172.21.15.64:0/2499867928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e819f690 con 0x7fd1e8101980 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1ed68b640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8102340 0x7fd1e819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.928+0000 7fd1e7fff640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 0x7fd1e8197fb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c920 tx=0x7fd1d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.929+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4007c70 con 0x7fd1e8101980 2024-05-09T09:26:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.929+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d4007e10 con 0x7fd1e8101980 2024-05-09T09:26:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.929+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4018820 con 0x7fd1e8101980 2024-05-09T09:26:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.929+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e819f980 con 0x7fd1e8101980 2024-05-09T09:26:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.929+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e819fed0 con 0x7fd1e8101980 2024-05-09T09:26:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.930+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e8107b50 con 0x7fd1e8101980 2024-05-09T09:26:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.931+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1d40189c0 con 0x7fd1e8101980 2024-05-09T09:26:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.931+0000 7fd1e5ffb640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2b90 0x7fd1d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.932+0000 7fd1ece8a640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2b90 0x7fd1d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.932+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd1d40cf500 con 0x7fd1e8101980 2024-05-09T09:26:36.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.932+0000 7fd1ece8a640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2b90 0x7fd1d80a5050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc004770 tx=0x7fd1dc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:36.936+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd1d4014030 con 0x7fd1e8101980 2024-05-09T09:26:37.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.120+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd1e802d1c0 con 0x7fd1e8101980 2024-05-09T09:26:37.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.121+0000 7fd1e5ffb640 1 -- 172.21.15.64:0/2499867928 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fd1d409c0d0 con 0x7fd1e8101980 2024-05-09T09:26:37.121 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2b90 msgr2=0x7fd1d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2b90 0x7fd1d80a5050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc004770 tx=0x7fd1dc000bc0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 msgr2=0x7fd1e8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 0x7fd1e8197fb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c920 tx=0x7fd1d400cde0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 shutdown_connections 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd1d80a2b90 0x7fd1d80a5050 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e8102340 0x7fd1e819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1e8101980 0x7fd1e8197fb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 --2- 172.21.15.64:0/2499867928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8100790 0x7fd1e8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 >> 172.21.15.64:0/2499867928 conn(0x7fd1e80fbd20 msgr2=0x7fd1e8104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 shutdown_connections 2024-05-09T09:26:37.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.123+0000 7fd1ef0ee640 1 -- 172.21.15.64:0/2499867928 wait complete. 2024-05-09T09:26:37.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 -- 172.21.15.64:0/3512567152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b781035a0 msgr2=0x7f4b78109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 --2- 172.21.15.64:0/3512567152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b781035a0 0x7f4b78109e30 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009cd0 tx=0x7f4b7002a8e0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 -- 172.21.15.64:0/3512567152 shutdown_connections 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 --2- 172.21.15.64:0/3512567152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b781035a0 0x7f4b78109e30 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 --2- 172.21.15.64:0/3512567152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b78102be0 0x7f4b78103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 --2- 172.21.15.64:0/3512567152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b780ffb70 0x7f4b780fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 -- 172.21.15.64:0/3512567152 >> 172.21.15.64:0/3512567152 conn(0x7f4b780fb460 msgr2=0x7f4b780fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 -- 172.21.15.64:0/3512567152 shutdown_connections 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 -- 172.21.15.64:0/3512567152 wait complete. 2024-05-09T09:26:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 Processor -- start 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 -- start start 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 0x7f4b78199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.242+0000 7f4b80458640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78102be0 0x7f4b7819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b80458640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b781035a0 0x7f4b781a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b80458640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d3c0 con 0x7f4b780ffb70 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b80458640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d240 con 0x7f4b781035a0 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b80458640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d540 con 0x7f4b78102be0 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 0x7f4b78199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 0x7f4b78199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38316/0 (socket says 172.21.15.64:38316) 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 -- 172.21.15.64:0/163122387 learned_addr learned my addr 172.21.15.64:0/163122387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7d9f3640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78102be0 0x7f4b7819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e9f5640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b781035a0 0x7f4b781a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 -- 172.21.15.64:0/163122387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78102be0 msgr2=0x7f4b7819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78102be0 0x7f4b7819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 -- 172.21.15.64:0/163122387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b781035a0 msgr2=0x7f4b781a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b781035a0 0x7f4b781a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 -- 172.21.15.64:0/163122387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b781a1910 con 0x7f4b780ffb70 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e9f5640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b781035a0 0x7f4b781a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b7e1f4640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 0x7f4b78199d00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400e9c0 tx=0x7f4b7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400cd80 con 0x7f4b780ffb70 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b740035f0 con 0x7f4b780ffb70 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b781a1c00 con 0x7f4b780ffb70 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74010690 con 0x7f4b780ffb70 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.243+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b781a2100 con 0x7f4b780ffb70 2024-05-09T09:26:37.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.244+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b781a22d0 con 0x7f4b780ffb70 2024-05-09T09:26:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.245+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b740037a0 con 0x7f4b780ffb70 2024-05-09T09:26:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.246+0000 7f4b6f7fe640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b640a2b90 0x7f4b640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.246+0000 7f4b7d9f3640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b640a2b90 0x7f4b640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.246+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4b740cfa40 con 0x7f4b780ffb70 2024-05-09T09:26:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.247+0000 7f4b7d9f3640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b640a2b90 0x7f4b640a5050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009bd0 tx=0x7f4b68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:37.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.250+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b7409c610 con 0x7f4b780ffb70 2024-05-09T09:26:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:37 smithi064 ceph-mon[30149]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:37 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/575523350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:37 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/245699073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:37 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2260625969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:37 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2499867928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.452+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b781a2540 con 0x7f4b780ffb70 2024-05-09T09:26:37.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.453+0000 7f4b6f7fe640 1 -- 172.21.15.64:0/163122387 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+588 (secure 0 0 0) 0x7f4b7409bf10 con 0x7f4b780ffb70 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1152515804 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3043598500 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2011681658 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3326248924 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1070396506 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2526964267 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:37.453 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:37.454 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b640a2b90 msgr2=0x7f4b640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b640a2b90 0x7f4b640a5050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009bd0 tx=0x7f4b68005f90 comp rx=0 tx=0).stop 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 msgr2=0x7f4b78199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 0x7f4b78199d00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400e9c0 tx=0x7f4b7400ee80 comp rx=0 tx=0).stop 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 shutdown_connections 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f4b640a2b90 0x7f4b640a5050 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b781035a0 0x7f4b781a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78102be0 0x7f4b7819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 --2- 172.21.15.64:0/163122387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b780ffb70 0x7f4b78199d00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 >> 172.21.15.64:0/163122387 conn(0x7f4b780fb460 msgr2=0x7f4b781009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 shutdown_connections 2024-05-09T09:26:37.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.455+0000 7f4b80458640 1 -- 172.21.15.64:0/163122387 wait complete. 2024-05-09T09:26:37.466 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-09T09:26:37.466 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T09:26:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:37 smithi179 ceph-mon[34086]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:37.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:37 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/575523350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:37 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/245699073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:37 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2260625969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:37 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2499867928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:26:37.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:37 smithi139 ceph-mon[33599]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T09:26:37.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:37 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/575523350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:37 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/245699073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:37 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2260625969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:37 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2499867928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:26:37.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 -- 172.21.15.64:0/2749364386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8102a90 msgr2=0x7f59b8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 --2- 172.21.15.64:0/2749364386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8102a90 0x7f59b8102f10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009900 tx=0x7f59ac02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 -- 172.21.15.64:0/2749364386 shutdown_connections 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 --2- 172.21.15.64:0/2749364386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b8103450 0x7f59b8109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 --2- 172.21.15.64:0/2749364386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8102a90 0x7f59b8102f10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 --2- 172.21.15.64:0/2749364386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59b81018a0 0x7f59b8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 -- 172.21.15.64:0/2749364386 >> 172.21.15.64:0/2749364386 conn(0x7f59b80fce10 msgr2=0x7f59b80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 -- 172.21.15.64:0/2749364386 shutdown_connections 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 -- 172.21.15.64:0/2749364386 wait complete. 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.629+0000 7f59be25d640 1 Processor -- start 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 -- start start 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b81018a0 0x7f59b8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59b8102a90 0x7f59b8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 0x7f59b819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810d420 con 0x7f59b8103450 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810d2a0 con 0x7f59b81018a0 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59be25d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810d5a0 con 0x7f59b8102a90 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 0x7f59b819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 0x7f59b819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38338/0 (socket says 172.21.15.64:38338) 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 -- 172.21.15.64:0/818190987 learned_addr learned my addr 172.21.15.64:0/818190987 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b77fe640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b81018a0 0x7f59b8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b6ffd640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59b8102a90 0x7f59b8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 -- 172.21.15.64:0/818190987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59b8102a90 msgr2=0x7f59b8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59b8102a90 0x7f59b8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 -- 172.21.15.64:0/818190987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b81018a0 msgr2=0x7f59b8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b81018a0 0x7f59b8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 -- 172.21.15.64:0/818190987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b819f710 con 0x7f59b8103450 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b77fe640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b81018a0 0x7f59b8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.630+0000 7f59b7fff640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 0x7f59b819f0a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f59b000eaa0 tx=0x7f59b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.631+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b000cdb0 con 0x7f59b8103450 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.631+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819fa00 con 0x7f59b8103450 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.631+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b0005020 con 0x7f59b8103450 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.631+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00116d0 con 0x7f59b8103450 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.631+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b819fed0 con 0x7f59b8103450 2024-05-09T09:26:37.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.632+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b8108c40 con 0x7f59b8103450 2024-05-09T09:26:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.633+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f59b00051d0 con 0x7f59b8103450 2024-05-09T09:26:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.634+0000 7f59b4ff9640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59a00a2b90 0x7f59a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.634+0000 7f59b77fe640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59a00a2b90 0x7f59a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.634+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f59b00cf5d0 con 0x7f59b8103450 2024-05-09T09:26:37.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.634+0000 7f59b77fe640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59a00a2b90 0x7f59a00a5050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005fd0 tx=0x7f59a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.638+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59b0016030 con 0x7f59b8103450 2024-05-09T09:26:37.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.826+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59b802d400 con 0x7f59b8103450 2024-05-09T09:26:37.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.828+0000 7f59b4ff9640 1 -- 172.21.15.64:0/818190987 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f59b009c1a0 con 0x7f59b8103450 2024-05-09T09:26:37.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:26:37.828 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59a00a2b90 msgr2=0x7f59a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59a00a2b90 0x7f59a00a5050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005fd0 tx=0x7f59a800b390 comp rx=0 tx=0).stop 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 msgr2=0x7f59b819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 0x7f59b819f0a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f59b000eaa0 tx=0x7f59b000ef60 comp rx=0 tx=0).stop 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 shutdown_connections 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f59a00a2b90 0x7f59a00a5050 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59b8103450 0x7f59b819f0a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59b8102a90 0x7f59b8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 --2- 172.21.15.64:0/818190987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59b81018a0 0x7f59b8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 >> 172.21.15.64:0/818190987 conn(0x7f59b80fce10 msgr2=0x7f59b8105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 shutdown_connections 2024-05-09T09:26:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.830+0000 7f59be25d640 1 -- 172.21.15.64:0/818190987 wait complete. 2024-05-09T09:26:37.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T09:01:59.549448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T09:06:39.033238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T09:06:39.869486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T09:06:43.592276+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T09:26:37.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 -- 172.21.15.64:0/2972994248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4101890 msgr2=0x7fd4e4101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/2972994248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4101890 0x7fd4e4101cb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009ca0 tx=0x7fd4d802a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 -- 172.21.15.64:0/2972994248 shutdown_connections 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/2972994248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e4103440 0x7fd4e4109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/2972994248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4102a80 0x7fd4e4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/2972994248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4101890 0x7fd4e4101cb0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 -- 172.21.15.64:0/2972994248 >> 172.21.15.64:0/2972994248 conn(0x7fd4e40fce00 msgr2=0x7fd4e40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 -- 172.21.15.64:0/2972994248 shutdown_connections 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.950+0000 7fd4ec16a640 1 -- 172.21.15.64:0/2972994248 wait complete. 2024-05-09T09:26:37.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 Processor -- start 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 -- start start 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e4101890 0x7fd4e4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4102a80 0x7fd4e4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 0x7fd4e419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410d390 con 0x7fd4e4103440 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410d210 con 0x7fd4e4101890 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ec16a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410d510 con 0x7fd4e4102a80 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 0x7fd4e419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 0x7fd4e419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38356/0 (socket says 172.21.15.64:38356) 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 -- 172.21.15.64:0/3919058240 learned_addr learned my addr 172.21.15.64:0/3919058240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4e9705640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4102a80 0x7fd4e4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4e9f06640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e4101890 0x7fd4e4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 -- 172.21.15.64:0/3919058240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4102a80 msgr2=0x7fd4e4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4102a80 0x7fd4e4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 -- 172.21.15.64:0/3919058240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e4101890 msgr2=0x7fd4e4197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e4101890 0x7fd4e4197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.951+0000 7fd4ea707640 1 -- 172.21.15.64:0/3919058240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e419f720 con 0x7fd4e4103440 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.952+0000 7fd4ea707640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 0x7fd4e419f0b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c970 tx=0x7fd4dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.952+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc007bb0 con 0x7fd4e4103440 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.952+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e419fa10 con 0x7fd4e4103440 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.952+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4dc007d50 con 0x7fd4e4103440 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.952+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc004e10 con 0x7fd4e4103440 2024-05-09T09:26:37.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.952+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e419ff10 con 0x7fd4e4103440 2024-05-09T09:26:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.953+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e4108d70 con 0x7fd4e4103440 2024-05-09T09:26:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.954+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4dc005020 con 0x7fd4e4103440 2024-05-09T09:26:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.954+0000 7fd4d6ffd640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4d00a2b90 0x7fd4d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.955+0000 7fd4e9f06640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4d00a2b90 0x7fd4d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.955+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd4dc0ceff0 con 0x7fd4e4103440 2024-05-09T09:26:37.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.955+0000 7fd4e9f06640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4d00a2b90 0x7fd4d00a5050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8000f80 tx=0x7fd4d8031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:37.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:37.959+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4dc014030 con 0x7fd4e4103440 2024-05-09T09:26:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.145+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4e4062460 con 0x7fd4e4103440 2024-05-09T09:26:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.145+0000 7fd4d6ffd640 1 -- 172.21.15.64:0/3919058240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd4dc09b570 con 0x7fd4e4103440 2024-05-09T09:26:38.145 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4d00a2b90 msgr2=0x7fd4d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4d00a2b90 0x7fd4d00a5050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8000f80 tx=0x7fd4d8031000 comp rx=0 tx=0).stop 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 msgr2=0x7fd4e419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 0x7fd4e419f0b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c970 tx=0x7fd4dc00ce30 comp rx=0 tx=0).stop 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 shutdown_connections 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fd4d00a2b90 0x7fd4d00a5050 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4e4103440 0x7fd4e419f0b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4102a80 0x7fd4e4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 --2- 172.21.15.64:0/3919058240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e4101890 0x7fd4e4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 >> 172.21.15.64:0/3919058240 conn(0x7fd4e40fce00 msgr2=0x7fd4e4105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 shutdown_connections 2024-05-09T09:26:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.147+0000 7fd4ec16a640 1 -- 172.21.15.64:0/3919058240 wait complete. 2024-05-09T09:26:38.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T09:26:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 -- 172.21.15.64:0/3852833304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 msgr2=0x7fbdac1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 --2- 172.21.15.64:0/3852833304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac1099c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009ca0 tx=0x7fbda402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 -- 172.21.15.64:0/3852833304 shutdown_connections 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 --2- 172.21.15.64:0/3852833304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac1099c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 --2- 172.21.15.64:0/3852833304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdac104c40 0x7fbdac107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 --2- 172.21.15.64:0/3852833304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac102310 0x7fbdac104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 -- 172.21.15.64:0/3852833304 >> 172.21.15.64:0/3852833304 conn(0x7fbdac0fbd20 msgr2=0x7fbdac0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 -- 172.21.15.64:0/3852833304 shutdown_connections 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.267+0000 7fbdb2114640 1 -- 172.21.15.64:0/3852833304 wait complete. 2024-05-09T09:26:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 Processor -- start 2024-05-09T09:26:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 -- start start 2024-05-09T09:26:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdac102310 0x7fbdac197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac104c40 0x7fbdac197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac10d430 con 0x7fbdac107570 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac10d2b0 con 0x7fbdac102310 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdb2114640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac10d5b0 con 0x7fbdac104c40 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38372/0 (socket says 172.21.15.64:38372) 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.268+0000 7fbdabfff640 1 -- 172.21.15.64:0/22018393 learned_addr learned my addr 172.21.15.64:0/22018393 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdaaffd640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac104c40 0x7fbdac197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdab7fe640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdac102310 0x7fbdac197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdabfff640 1 -- 172.21.15.64:0/22018393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac104c40 msgr2=0x7fbdac197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdabfff640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac104c40 0x7fbdac197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdabfff640 1 -- 172.21.15.64:0/22018393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdac102310 msgr2=0x7fbdac197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdabfff640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdac102310 0x7fbdac197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdabfff640 1 -- 172.21.15.64:0/22018393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac19f6d0 con 0x7fbdac107570 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdabfff640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac19f060 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fbda4031040 tx=0x7fbda4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4032ec0 con 0x7fbdac107570 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbda40044f0 con 0x7fbdac107570 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdac19f960 con 0x7fbdac107570 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4030d10 con 0x7fbdac107570 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.269+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdac19feb0 con 0x7fbdac107570 2024-05-09T09:26:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.270+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdac0fd810 con 0x7fbdac107570 2024-05-09T09:26:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.271+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbda4004040 con 0x7fbdac107570 2024-05-09T09:26:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.272+0000 7fbda8ff9640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd980a2d30 0x7fbd980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.272+0000 7fbdab7fe640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd980a2d30 0x7fbd980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.272+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fbda40f0980 con 0x7fbdac107570 2024-05-09T09:26:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.273+0000 7fbdab7fe640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd980a2d30 0x7fbd980a51f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005fd0 tx=0x7fbd9c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.276+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbda40bd550 con 0x7fbdac107570 2024-05-09T09:26:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/163122387' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:26:38.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:38 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/818190987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.462+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbdac02d3a0 con 0x7fbdac107570 2024-05-09T09:26:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.463+0000 7fbda8ff9640 1 -- 172.21.15.64:0/22018393 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fbda40bce50 con 0x7fbdac107570 2024-05-09T09:26:38.463 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd980a2d30 msgr2=0x7fbd980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd980a2d30 0x7fbd980a51f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005fd0 tx=0x7fbd9c00b390 comp rx=0 tx=0).stop 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 msgr2=0x7fbdac19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac19f060 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fbda4031040 tx=0x7fbda4003ba0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 shutdown_connections 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fbd980a2d30 0x7fbd980a51f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdac107570 0x7fbdac19f060 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdac104c40 0x7fbdac197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 --2- 172.21.15.64:0/22018393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdac102310 0x7fbdac197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 >> 172.21.15.64:0/22018393 conn(0x7fbdac0fbd20 msgr2=0x7fbdac102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 shutdown_connections 2024-05-09T09:26:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.465+0000 7fbdb2114640 1 -- 172.21.15.64:0/22018393 wait complete. 2024-05-09T09:26:38.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T09:26:38.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/163122387' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:26:38.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:38 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/818190987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/163122387' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:26:38.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:38 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/818190987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:38.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 -- 172.21.15.64:0/3336949105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050102310 msgr2=0x7f6050104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 --2- 172.21.15.64:0/3336949105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050102310 0x7f6050104700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 -- 172.21.15.64:0/3336949105 shutdown_connections 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 --2- 172.21.15.64:0/3336949105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6050107570 0x7f60501099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 --2- 172.21.15.64:0/3336949105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050104c40 0x7f6050107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 --2- 172.21.15.64:0/3336949105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050102310 0x7f6050104700 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 -- 172.21.15.64:0/3336949105 >> 172.21.15.64:0/3336949105 conn(0x7f60500fbd20 msgr2=0x7f60500fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 -- 172.21.15.64:0/3336949105 shutdown_connections 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 -- 172.21.15.64:0/3336949105 wait complete. 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.584+0000 7f6055109640 1 Processor -- start 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 -- start start 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6050102310 0x7f6050197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 0x7f6050198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107570 0x7f605019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f605010d3f0 con 0x7f6050104c40 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f605010d270 con 0x7f6050102310 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f6055109640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f605010d570 con 0x7f6050107570 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 0x7f6050198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 0x7f6050198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38384/0 (socket says 172.21.15.64:38384) 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 -- 172.21.15.64:0/1077074299 learned_addr learned my addr 172.21.15.64:0/1077074299 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604ed9d640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6050102310 0x7f6050197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604f59e640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107570 0x7f605019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 -- 172.21.15.64:0/1077074299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107570 msgr2=0x7f605019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107570 0x7f605019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 -- 172.21.15.64:0/1077074299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6050102310 msgr2=0x7f6050197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6050102310 0x7f6050197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604e59c640 1 -- 172.21.15.64:0/1077074299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605019f710 con 0x7f6050104c40 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.585+0000 7f604f59e640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107570 0x7f605019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.586+0000 7f604e59c640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 0x7f6050198030 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f604000c970 tx=0x7f604000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.586+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040007bb0 con 0x7f6050104c40 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.586+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605019fa00 con 0x7f6050104c40 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.586+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6040007d50 con 0x7f6050104c40 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.586+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040018730 con 0x7f6050104c40 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.586+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605019ff50 con 0x7f6050104c40 2024-05-09T09:26:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.587+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60500fd810 con 0x7f6050104c40 2024-05-09T09:26:38.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.588+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f604000f450 con 0x7f6050104c40 2024-05-09T09:26:38.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.589+0000 7f6047fff640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f603c0a2b90 0x7f603c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.589+0000 7f604ed9d640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f603c0a2b90 0x7f603c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.589+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f60400cf330 con 0x7f6050104c40 2024-05-09T09:26:38.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.589+0000 7f604ed9d640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f603c0a2b90 0x7f603c0a5050 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6038000c00 tx=0x7f6038005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:38.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.593+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6040014030 con 0x7f6050104c40 2024-05-09T09:26:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.777+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f605002d370 con 0x7f6050104c40 2024-05-09T09:26:38.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.778+0000 7f6047fff640 1 -- 172.21.15.64:0/1077074299 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f604009bfa0 con 0x7f6050104c40 2024-05-09T09:26:38.778 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-05-09T09:26:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f603c0a2b90 msgr2=0x7f603c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f603c0a2b90 0x7f603c0a5050 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6038000c00 tx=0x7f6038005e40 comp rx=0 tx=0).stop 2024-05-09T09:26:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 msgr2=0x7f6050198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 0x7f6050198030 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f604000c970 tx=0x7f604000ce30 comp rx=0 tx=0).stop 2024-05-09T09:26:38.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 shutdown_connections 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f603c0a2b90 0x7f603c0a5050 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107570 0x7f605019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6050104c40 0x7f6050198030 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 --2- 172.21.15.64:0/1077074299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6050102310 0x7f6050197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 >> 172.21.15.64:0/1077074299 conn(0x7f60500fbd20 msgr2=0x7f6050102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 shutdown_connections 2024-05-09T09:26:38.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.780+0000 7f6055109640 1 -- 172.21.15.64:0/1077074299 wait complete. 2024-05-09T09:26:38.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T09:26:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 -- 172.21.15.64:0/3793085260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c101890 msgr2=0x7fac8c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3793085260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c101890 0x7fac8c101cb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fac84009ca0 tx=0x7fac8402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 -- 172.21.15.64:0/3793085260 shutdown_connections 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3793085260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 0x7fac8c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3793085260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c102a80 0x7fac8c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3793085260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c101890 0x7fac8c101cb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 -- 172.21.15.64:0/3793085260 >> 172.21.15.64:0/3793085260 conn(0x7fac8c0fce00 msgr2=0x7fac8c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 -- 172.21.15.64:0/3793085260 shutdown_connections 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.899+0000 7fac93ddc640 1 -- 172.21.15.64:0/3793085260 wait complete. 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 Processor -- start 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 -- start start 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c101890 0x7fac8c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 0x7fac8c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 0x7fac8c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c10d460 con 0x7fac8c102a80 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c10d2e0 con 0x7fac8c101890 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac93ddc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c10d5e0 con 0x7fac8c103440 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 0x7fac8c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 0x7fac8c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38408/0 (socket says 172.21.15.64:38408) 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 -- 172.21.15.64:0/3523414878 learned_addr learned my addr 172.21.15.64:0/3523414878 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91b78640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c101890 0x7fac8c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac92379640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 0x7fac8c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 -- 172.21.15.64:0/3523414878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 msgr2=0x7fac8c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 0x7fac8c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 -- 172.21.15.64:0/3523414878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c101890 msgr2=0x7fac8c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:38.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c101890 0x7fac8c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 -- 172.21.15.64:0/3523414878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c19f720 con 0x7fac8c102a80 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac92379640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 0x7fac8c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.900+0000 7fac91377640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 0x7fac8c198040 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b4a0 tx=0x7fac8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.901+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80014070 con 0x7fac8c102a80 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.901+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac80010ac0 con 0x7fac8c102a80 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.901+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c19fa10 con 0x7fac8c102a80 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.901+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80017c30 con 0x7fac8c102a80 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.901+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c19ff60 con 0x7fac8c102a80 2024-05-09T09:26:38.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.902+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c108c30 con 0x7fac8c102a80 2024-05-09T09:26:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.903+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac80017070 con 0x7fac8c102a80 2024-05-09T09:26:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.903+0000 7fac8affd640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac740a2b90 0x7fac740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.904+0000 7fac91b78640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac740a2b90 0x7fac740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.904+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fac800ceb50 con 0x7fac8c102a80 2024-05-09T09:26:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.904+0000 7fac91b78640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac740a2b90 0x7fac740a5050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fac84006010 tx=0x7fac84000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:38.908+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fac8009b720 con 0x7fac8c102a80 2024-05-09T09:26:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.092+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fac8c062430 con 0x7fac8c102a80 2024-05-09T09:26:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.093+0000 7fac8affd640 1 -- 172.21.15.64:0/3523414878 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fac8009b020 con 0x7fac8c102a80 2024-05-09T09:26:39.092 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-05-09T09:26:39.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac740a2b90 msgr2=0x7fac740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac740a2b90 0x7fac740a5050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fac84006010 tx=0x7fac84000f40 comp rx=0 tx=0).stop 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 msgr2=0x7fac8c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 0x7fac8c198040 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b4a0 tx=0x7fac8000b960 comp rx=0 tx=0).stop 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 shutdown_connections 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fac740a2b90 0x7fac740a5050 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac8c103440 0x7fac8c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac8c102a80 0x7fac8c198040 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.094+0000 7fac93ddc640 1 --2- 172.21.15.64:0/3523414878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c101890 0x7fac8c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.095+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 >> 172.21.15.64:0/3523414878 conn(0x7fac8c0fce00 msgr2=0x7fac8c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.095+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 shutdown_connections 2024-05-09T09:26:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.095+0000 7fac93ddc640 1 -- 172.21.15.64:0/3523414878 wait complete. 2024-05-09T09:26:39.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T09:26:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 -- 172.21.15.64:0/3405456673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 msgr2=0x7fa944101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 --2- 172.21.15.64:0/3405456673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944101e00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa930009900 tx=0x7fa93002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:39.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 -- 172.21.15.64:0/3405456673 shutdown_connections 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 --2- 172.21.15.64:0/3405456673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944102340 0x7fa944108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 --2- 172.21.15.64:0/3405456673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944101e00 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 --2- 172.21.15.64:0/3405456673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa944100790 0x7fa944100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 -- 172.21.15.64:0/3405456673 >> 172.21.15.64:0/3405456673 conn(0x7fa9440fbd20 msgr2=0x7fa9440fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 -- 172.21.15.64:0/3405456673 shutdown_connections 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.213+0000 7fa94aa76640 1 -- 172.21.15.64:0/3405456673 wait complete. 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 Processor -- start 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 -- start start 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa944100790 0x7fa944197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944102340 0x7fa94419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410c2c0 con 0x7fa944101980 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410c140 con 0x7fa944100790 2024-05-09T09:26:39.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa94aa76640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410c440 con 0x7fa944102340 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38426/0 (socket says 172.21.15.64:38426) 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 -- 172.21.15.64:0/2913222511 learned_addr learned my addr 172.21.15.64:0/2913222511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa948812640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa944100790 0x7fa944197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa949013640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944102340 0x7fa94419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 -- 172.21.15.64:0/2913222511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944102340 msgr2=0x7fa94419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944102340 0x7fa94419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 -- 172.21.15.64:0/2913222511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa944100790 msgr2=0x7fa944197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa944100790 0x7fa944197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 -- 172.21.15.64:0/2913222511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94419f730 con 0x7fa944101980 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.214+0000 7fa943fff640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944198050 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa930004770 tx=0x7fa9300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.215+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93003f030 con 0x7fa944101980 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.215+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94419f9c0 con 0x7fa944101980 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.215+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa930003cb0 con 0x7fa944101980 2024-05-09T09:26:39.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.215+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930035070 con 0x7fa944101980 2024-05-09T09:26:39.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.215+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94419feb0 con 0x7fa944101980 2024-05-09T09:26:39.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.216+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9441a0140 con 0x7fa944101980 2024-05-09T09:26:39.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.217+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa930042030 con 0x7fa944101980 2024-05-09T09:26:39.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.218+0000 7fa941ffb640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9340a2b90 0x7fa9340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.218+0000 7fa948812640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9340a2b90 0x7fa9340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.218+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa9300f00d0 con 0x7fa944101980 2024-05-09T09:26:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.218+0000 7fa948812640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9340a2b90 0x7fa9340a5050 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa938005fd0 tx=0x7fa93800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:39.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.221+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa9300bcca0 con 0x7fa944101980 2024-05-09T09:26:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:39 smithi064 ceph-mon[30149]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3919058240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/22018393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1077074299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:39 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/3523414878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.419+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9441a0430 con 0x7fa944101980 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.419+0000 7fa941ffb640 1 -- 172.21.15.64:0/2913222511 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+588 (secure 0 0 0) 0x7fa9300bc5a0 con 0x7fa944101980 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1152515804 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3043598500 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2011681658 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/615130791 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:39.419 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3326248924 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:39.420 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1070396506 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:39.420 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2526964267 2024-05-10T08:59:35.011155+0000 2024-05-09T09:26:39.420 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:39.420 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/421426620 2024-05-10T08:59:08.628153+0000 2024-05-09T09:26:39.420 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9340a2b90 msgr2=0x7fa9340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9340a2b90 0x7fa9340a5050 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa938005fd0 tx=0x7fa93800b410 comp rx=0 tx=0).stop 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 msgr2=0x7fa944198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944198050 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa930004770 tx=0x7fa9300042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 shutdown_connections 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fa9340a2b90 0x7fa9340a5050 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944102340 0x7fa94419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa944101980 0x7fa944198050 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 --2- 172.21.15.64:0/2913222511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa944100790 0x7fa944197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:39.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 >> 172.21.15.64:0/2913222511 conn(0x7fa9440fbd20 msgr2=0x7fa944069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:39.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 shutdown_connections 2024-05-09T09:26:39.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:39.421+0000 7fa94aa76640 1 -- 172.21.15.64:0/2913222511 wait complete. 2024-05-09T09:26:39.432 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-05-09T09:26:39.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:39.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /tmp/cephfs 2024-05-09T09:26:39.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[82154]: fuse finished with error 0 and tester_r 0 2024-05-09T09:26:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:39 smithi179 ceph-mon[34086]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3919058240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/22018393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1077074299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:39 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/3523414878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:39 smithi139 ceph-mon[33599]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3919058240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/22018393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1077074299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:39 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/3523414878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T09:26:39.577 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T09:26:40.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:40 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2913222511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:26:40.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:40 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2913222511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:26:40.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:40 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2913222511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T09:26:41.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:41 smithi064 ceph-mon[30149]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:41.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:41 smithi179 ceph-mon[34086]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:41.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:41 smithi139 ceph-mon[33599]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T09:26:43.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:43 smithi064 ceph-mon[30149]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:43.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:43 smithi179 ceph-mon[34086]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:43.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:43 smithi139 ceph-mon[33599]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T09:26:45.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:45 smithi064 ceph-mon[30149]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-09T09:26:45.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:45 smithi179 ceph-mon[34086]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-09T09:26:45.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:45 smithi139 ceph-mon[33599]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-09T09:26:45.679 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:26:45.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:45.679 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-09T09:26:45.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:45.714 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-05-09T09:26:45.793 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-09T09:26:45.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:45.793 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-09T09:26:45.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T09:26:45.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-05-09T09:26:45.974 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T09:26:45.985 INFO:tasks.cephadm:Teardown begin 2024-05-09T09:26:45.985 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T09:26:45.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 -- 172.21.15.64:0/3084214586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c100770 msgr2=0x7f6c2c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 --2- 172.21.15.64:0/3084214586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c100770 0x7f6c2c100b90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 -- 172.21.15.64:0/3084214586 shutdown_connections 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 --2- 172.21.15.64:0/3084214586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c102320 0x7f6c2c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 --2- 172.21.15.64:0/3084214586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c2c101960 0x7f6c2c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 --2- 172.21.15.64:0/3084214586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c100770 0x7f6c2c100b90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 -- 172.21.15.64:0/3084214586 >> 172.21.15.64:0/3084214586 conn(0x7f6c2c0fbce0 msgr2=0x7f6c2c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 -- 172.21.15.64:0/3084214586 shutdown_connections 2024-05-09T09:26:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 -- 172.21.15.64:0/3084214586 wait complete. 2024-05-09T09:26:46.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 Processor -- start 2024-05-09T09:26:46.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.096+0000 7f6c31293640 1 -- start start 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c31293640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c2c100770 0x7f6c2c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c31293640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 0x7f6c2c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c31293640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c102320 0x7f6c2c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c31293640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c10c2a0 con 0x7f6c2c101960 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c31293640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c10c120 con 0x7f6c2c100770 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c31293640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c10c420 con 0x7f6c2c102320 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 0x7f6c2c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 0x7f6c2c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41886/0 (socket says 172.21.15.64:41886) 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 -- 172.21.15.64:0/4141010850 learned_addr learned my addr 172.21.15.64:0/4141010850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2affd640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c2c100770 0x7f6c2c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2b7fe640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c102320 0x7f6c2c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 -- 172.21.15.64:0/4141010850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c102320 msgr2=0x7f6c2c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c102320 0x7f6c2c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 -- 172.21.15.64:0/4141010850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c2c100770 msgr2=0x7f6c2c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c2c100770 0x7f6c2c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 -- 172.21.15.64:0/4141010850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c19f700 con 0x7f6c2c101960 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c2a7fc640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 0x7f6c2c198020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000d950 tx=0x7f6c2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.097+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20017070 con 0x7f6c2c101960 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.098+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c19f9f0 con 0x7f6c2c101960 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.098+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c20009c30 con 0x7f6c2c101960 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.098+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000f620 con 0x7f6c2c101960 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.098+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c19ff40 con 0x7f6c2c101960 2024-05-09T09:26:46.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.099+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c1a0110 con 0x7f6c2c101960 2024-05-09T09:26:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.100+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c200109a0 con 0x7f6c2c101960 2024-05-09T09:26:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.100+0000 7f6c13fff640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c140a2b90 0x7f6c140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.101+0000 7f6c2affd640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c140a2b90 0x7f6c140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.101+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c200cef70 con 0x7f6c2c101960 2024-05-09T09:26:46.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.101+0000 7f6c2affd640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c140a2b90 0x7f6c140a5050 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c004770 tx=0x7f6c1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.105+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6c20009de0 con 0x7f6c2c101960 2024-05-09T09:26:46.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.330+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c2c1a04a0 con 0x7f6c2c101960 2024-05-09T09:26:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.331+0000 7f6c13fff640 1 -- 172.21.15.64:0/4141010850 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11885 (secure 0 0 0) 0x7f6c2009b4f0 con 0x7f6c2c101960 2024-05-09T09:26:46.331 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:26:46.331 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:26:30.095938+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:26:46.331 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 55 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c140a2b90 msgr2=0x7f6c140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c140a2b90 0x7f6c140a5050 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c004770 tx=0x7f6c1c000bc0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 msgr2=0x7f6c2c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 0x7f6c2c198020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000d950 tx=0x7f6c2000de10 comp rx=0 tx=0).stop 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 shutdown_connections 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f6c140a2b90 0x7f6c140a5050 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c102320 0x7f6c2c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c2c101960 0x7f6c2c198020 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 --2- 172.21.15.64:0/4141010850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c2c100770 0x7f6c2c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 >> 172.21.15.64:0/4141010850 conn(0x7f6c2c0fbce0 msgr2=0x7f6c2c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 shutdown_connections 2024-05-09T09:26:46.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.333+0000 7f6c31293640 1 -- 172.21.15.64:0/4141010850 wait complete. 2024-05-09T09:26:46.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 -- 172.21.15.64:0/1532217881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 msgr2=0x7f7b14101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 --2- 172.21.15.64:0/1532217881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14101cc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009900 tx=0x7f7b1002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:46.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 -- 172.21.15.64:0/1532217881 shutdown_connections 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 --2- 172.21.15.64:0/1532217881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b14103450 0x7f7b14109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 --2- 172.21.15.64:0/1532217881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b14102a90 0x7f7b14102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 --2- 172.21.15.64:0/1532217881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14101cc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 -- 172.21.15.64:0/1532217881 >> 172.21.15.64:0/1532217881 conn(0x7f7b140fce30 msgr2=0x7f7b140ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 -- 172.21.15.64:0/1532217881 shutdown_connections 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.452+0000 7f7b1c342640 1 -- 172.21.15.64:0/1532217881 wait complete. 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 Processor -- start 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 -- start start 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b14102a90 0x7f7b14198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b14103450 0x7f7b1419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1410d3c0 con 0x7f7b141018a0 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1410d240 con 0x7f7b14103450 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1c342640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1410d540 con 0x7f7b14102a90 2024-05-09T09:26:46.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41896/0 (socket says 172.21.15.64:41896) 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 -- 172.21.15.64:0/2957717260 learned_addr learned my addr 172.21.15.64:0/2957717260 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b198dd640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b14102a90 0x7f7b14198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 -- 172.21.15.64:0/2957717260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b14102a90 msgr2=0x7f7b14198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a8df640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b14103450 0x7f7b1419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b14102a90 0x7f7b14198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 -- 172.21.15.64:0/2957717260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b14103450 msgr2=0x7f7b1419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.453+0000 7f7b1a0de640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b14103450 0x7f7b1419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b1a0de640 1 -- 172.21.15.64:0/2957717260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1419f720 con 0x7f7b141018a0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b1a0de640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14197b50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7b100314e0 tx=0x7f7b100042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1003f030 con 0x7f7b141018a0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1419f9b0 con 0x7f7b141018a0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b10003cb0 con 0x7f7b141018a0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10035070 con 0x7f7b141018a0 2024-05-09T09:26:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.454+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1419fea0 con 0x7f7b141018a0 2024-05-09T09:26:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.455+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b14108d30 con 0x7f7b141018a0 2024-05-09T09:26:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.457+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b10042030 con 0x7f7b141018a0 2024-05-09T09:26:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.458+0000 7f7b0b7fe640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b000a2b90 0x7f7b000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.458+0000 7f7b198dd640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b000a2b90 0x7f7b000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.458+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7b100efec0 con 0x7f7b141018a0 2024-05-09T09:26:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.458+0000 7f7b198dd640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b000a2b90 0x7f7b000a5050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7b04005fd0 tx=0x7f7b04005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.461+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7b100bca90 con 0x7f7b141018a0 2024-05-09T09:26:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.684+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b1402d1c0 con 0x7f7b141018a0 2024-05-09T09:26:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.685+0000 7f7b0b7fe640 1 -- 172.21.15.64:0/2957717260 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11885 (secure 0 0 0) 0x7f7b100bc390 con 0x7f7b141018a0 2024-05-09T09:26:46.685 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:26:46.685 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/201840839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":201840839},{"type":"v1","addr":"172.21.15.64:6835","nonce":201840839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4065753229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4065753229},{"type":"v1","addr":"172.21.15.64:6837","nonce":4065753229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/437565752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":437565752},{"type":"v1","addr":"172.21.15.64:6839","nonce":437565752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/4047794572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":4047794572},{"type":"v1","addr":"172.21.15.64:6841","nonce":4047794572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3830149624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3830149624},{"type":"v1","addr":"172.21.15.139:6833","nonce":3830149624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3794352368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3794352368},{"type":"v1","addr":"172.21.15.179:6833","nonce":3794352368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T09:06:40.458278+0000","modified":"2024-05-09T09:26:30.095938+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24496,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/3760508086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3760508086},{"type":"v1","addr":"172.21.15.139:6835","nonce":3760508086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2410946724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2410946724},{"type":"v1","addr":"172.21.15.139:6837","nonce":2410946724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/502414567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":502414567},{"type":"v1","addr":"172.21.15.179:6835","nonce":502414567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6837/2858892786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2858892786},{"type":"v1","addr":"172.21.15.179:6837","nonce":2858892786}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/1577800782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":1577800782},{"type":"v1","addr":"172.21.15.179:6839","nonce":1577800782}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6839/1823707244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1823707244},{"type":"v1","addr":"172.21.15.139:6839","nonce":1823707244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24487,24496,24503]},"id":1}]} 2024-05-09T09:26:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 55 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b000a2b90 msgr2=0x7f7b000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b000a2b90 0x7f7b000a5050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7b04005fd0 tx=0x7f7b04005850 comp rx=0 tx=0).stop 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 msgr2=0x7f7b14197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14197b50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7b100314e0 tx=0x7f7b100042e0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 shutdown_connections 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f7b000a2b90 0x7f7b000a5050 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b14103450 0x7f7b1419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b14102a90 0x7f7b14198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 --2- 172.21.15.64:0/2957717260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b141018a0 0x7f7b14197b50 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 >> 172.21.15.64:0/2957717260 conn(0x7f7b140fce30 msgr2=0x7f7b14105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 shutdown_connections 2024-05-09T09:26:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.687+0000 7f7b1c342640 1 -- 172.21.15.64:0/2957717260 wait complete. 2024-05-09T09:26:46.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.803+0000 7f2d29770640 1 -- 172.21.15.64:0/157743460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24102320 msgr2=0x7f2d24108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.803+0000 7f2d29770640 1 --2- 172.21.15.64:0/157743460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24102320 0x7f2d24108bb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009900 tx=0x7f2d1c02a410 comp rx=0 tx=0).stop 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 -- 172.21.15.64:0/157743460 shutdown_connections 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 --2- 172.21.15.64:0/157743460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24102320 0x7f2d24108bb0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 --2- 172.21.15.64:0/157743460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d24101960 0x7f2d24101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 --2- 172.21.15.64:0/157743460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24100770 0x7f2d24100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 -- 172.21.15.64:0/157743460 >> 172.21.15.64:0/157743460 conn(0x7f2d240fbce0 msgr2=0x7f2d240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 -- 172.21.15.64:0/157743460 shutdown_connections 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 -- 172.21.15.64:0/157743460 wait complete. 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 Processor -- start 2024-05-09T09:26:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.804+0000 7f2d29770640 1 -- start start 2024-05-09T09:26:46.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 0x7f2d2410b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d24101960 0x7f2d24109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24102320 0x7f2d24109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410d3c0 con 0x7f2d24100770 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410d240 con 0x7f2d24101960 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410d540 con 0x7f2d24102320 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 0x7f2d2410b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 0x7f2d2410b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41928/0 (socket says 172.21.15.64:41928) 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 -- 172.21.15.64:0/776113735 learned_addr learned my addr 172.21.15.64:0/776113735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d237fe640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24102320 0x7f2d24109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d227fc640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d24101960 0x7f2d24109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 -- 172.21.15.64:0/776113735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24102320 msgr2=0x7f2d24109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24102320 0x7f2d24109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 -- 172.21.15.64:0/776113735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d24101960 msgr2=0x7f2d24109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d24101960 0x7f2d24109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 -- 172.21.15.64:0/776113735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2410a500 con 0x7f2d24100770 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d237fe640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24102320 0x7f2d24109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d22ffd640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 0x7f2d2410b0f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400eaa0 tx=0x7f2d1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14016070 con 0x7f2d24100770 2024-05-09T09:26:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d14003e20 con 0x7f2d24100770 2024-05-09T09:26:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140116d0 con 0x7f2d24100770 2024-05-09T09:26:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241a1ae0 con 0x7f2d24100770 2024-05-09T09:26:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.805+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a2030 con 0x7f2d24100770 2024-05-09T09:26:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.806+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24107a90 con 0x7f2d24100770 2024-05-09T09:26:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.808+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d14019070 con 0x7f2d24100770 2024-05-09T09:26:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.809+0000 7f2d1bfff640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d100a2c10 0x7f2d100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.809+0000 7f2d227fc640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d100a2c10 0x7f2d100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.809+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2d140cf950 con 0x7f2d24100770 2024-05-09T09:26:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.810+0000 7f2d227fc640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d100a2c10 0x7f2d100a50d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005e10 tx=0x7f2d0c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:46.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:46.812+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d1409c5a0 con 0x7f2d24100770 2024-05-09T09:26:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.000+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d2402d390 con 0x7f2d24100770 2024-05-09T09:26:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.002+0000 7f2d1bfff640 1 -- 172.21.15.64:0/776113735 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24747 (secure 0 0 0) 0x7f2d1409bea0 con 0x7f2d24100770 2024-05-09T09:26:47.002 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-09T09:26:47.002 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":94,"fsid":"2877b902-0de2-11ef-bc98-c7b262605968","created":"2024-05-09T08:58:31.503287+0000","modified":"2024-05-09T09:08:36.699384+0000","last_up_change":"2024-05-09T09:05:47.640315+0000","last_in_change":"2024-05-09T09:05:27.488744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T09:01:59.549448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T09:06:39.033238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T09:06:39.869486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T09:06:43.592276+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f17d19a2-cc4a-48e7-a9ab-39baba537bd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6803","nonce":2595942736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6805","nonce":2595942736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6809","nonce":2595942736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2595942736},{"type":"v1","addr":"172.21.15.64:6807","nonce":2595942736}]},"public_addr":"172.21.15.64:6803/2595942736","cluster_addr":"172.21.15.64:6805/2595942736","heartbeat_back_addr":"172.21.15.64:6809/2595942736","heartbeat_front_addr":"172.21.15.64:6807/2595942736","state":["exists","up"]},{"osd":1,"uuid":"fa7e401d-8122-43c1-9c90-c5ec8f78e1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6801","nonce":1721854761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6803","nonce":1721854761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6807","nonce":1721854761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1721854761},{"type":"v1","addr":"172.21.15.139:6805","nonce":1721854761}]},"public_addr":"172.21.15.139:6801/1721854761","cluster_addr":"172.21.15.139:6803/1721854761","heartbeat_back_addr":"172.21.15.139:6807/1721854761","heartbeat_front_addr":"172.21.15.139:6805/1721854761","state":["exists","up"]},{"osd":2,"uuid":"2b8bc7a9-9231-4f53-ae83-151cc4898360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6801","nonce":3117011354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6803","nonce":3117011354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6807","nonce":3117011354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3117011354},{"type":"v1","addr":"172.21.15.179:6805","nonce":3117011354}]},"public_addr":"172.21.15.179:6801/3117011354","cluster_addr":"172.21.15.179:6803/3117011354","heartbeat_back_addr":"172.21.15.179:6807/3117011354","heartbeat_front_addr":"172.21.15.179:6805/3117011354","state":["exists","up"]},{"osd":3,"uuid":"3cd6b939-e38b-417b-86af-bf815725ee86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6811","nonce":3665517430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6813","nonce":3665517430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6817","nonce":3665517430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3665517430},{"type":"v1","addr":"172.21.15.64:6815","nonce":3665517430}]},"public_addr":"172.21.15.64:6811/3665517430","cluster_addr":"172.21.15.64:6813/3665517430","heartbeat_back_addr":"172.21.15.64:6817/3665517430","heartbeat_front_addr":"172.21.15.64:6815/3665517430","state":["exists","up"]},{"osd":4,"uuid":"01787529-c26f-4e8e-aca7-3254a587b036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6809","nonce":2100308120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6811","nonce":2100308120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6815","nonce":2100308120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2100308120},{"type":"v1","addr":"172.21.15.139:6813","nonce":2100308120}]},"public_addr":"172.21.15.139:6809/2100308120","cluster_addr":"172.21.15.139:6811/2100308120","heartbeat_back_addr":"172.21.15.139:6815/2100308120","heartbeat_front_addr":"172.21.15.139:6813/2100308120","state":["exists","up"]},{"osd":5,"uuid":"6f02944f-d0d9-423c-8747-06be792aa16d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6809","nonce":2037999484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6811","nonce":2037999484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6815","nonce":2037999484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2037999484},{"type":"v1","addr":"172.21.15.179:6813","nonce":2037999484}]},"public_addr":"172.21.15.179:6809/2037999484","cluster_addr":"172.21.15.179:6811/2037999484","heartbeat_back_addr":"172.21.15.179:6815/2037999484","heartbeat_front_addr":"172.21.15.179:6813/2037999484","state":["exists","up"]},{"osd":6,"uuid":"4df6bd66-57d2-4efd-927e-75def737bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6819","nonce":784727785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6821","nonce":784727785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6825","nonce":784727785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":784727785},{"type":"v1","addr":"172.21.15.64:6823","nonce":784727785}]},"public_addr":"172.21.15.64:6819/784727785","cluster_addr":"172.21.15.64:6821/784727785","heartbeat_back_addr":"172.21.15.64:6825/784727785","heartbeat_front_addr":"172.21.15.64:6823/784727785","state":["exists","up"]},{"osd":7,"uuid":"4e0802fb-8046-46e9-9a0a-923e400299d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6817","nonce":3275587574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6819","nonce":3275587574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6823","nonce":3275587574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3275587574},{"type":"v1","addr":"172.21.15.139:6821","nonce":3275587574}]},"public_addr":"172.21.15.139:6817/3275587574","cluster_addr":"172.21.15.139:6819/3275587574","heartbeat_back_addr":"172.21.15.139:6823/3275587574","heartbeat_front_addr":"172.21.15.139:6821/3275587574","state":["exists","up"]},{"osd":8,"uuid":"35bee3f0-b62d-4743-a492-5b91e6d58126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6817","nonce":365956094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6819","nonce":365956094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6823","nonce":365956094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":365956094},{"type":"v1","addr":"172.21.15.179:6821","nonce":365956094}]},"public_addr":"172.21.15.179:6817/365956094","cluster_addr":"172.21.15.179:6819/365956094","heartbeat_back_addr":"172.21.15.179:6823/365956094","heartbeat_front_addr":"172.21.15.179:6821/365956094","state":["exists","up"]},{"osd":9,"uuid":"b9f9fbad-4a3a-4dd6-91c1-b856e1b53018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6827","nonce":2394827163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6829","nonce":2394827163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6833","nonce":2394827163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2394827163},{"type":"v1","addr":"172.21.15.64:6831","nonce":2394827163}]},"public_addr":"172.21.15.64:6827/2394827163","cluster_addr":"172.21.15.64:6829/2394827163","heartbeat_back_addr":"172.21.15.64:6833/2394827163","heartbeat_front_addr":"172.21.15.64:6831/2394827163","state":["exists","up"]},{"osd":10,"uuid":"38196d80-7f0b-4dd2-af5f-5c7341bcd2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6825","nonce":4105758039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6827","nonce":4105758039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6831","nonce":4105758039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":4105758039},{"type":"v1","addr":"172.21.15.139:6829","nonce":4105758039}]},"public_addr":"172.21.15.139:6825/4105758039","cluster_addr":"172.21.15.139:6827/4105758039","heartbeat_back_addr":"172.21.15.139:6831/4105758039","heartbeat_front_addr":"172.21.15.139:6829/4105758039","state":["exists","up"]},{"osd":11,"uuid":"5c6dc3b4-d0c3-4a23-91a2-fa45e2b9b998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6825","nonce":880358389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6827","nonce":880358389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6831","nonce":880358389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":880358389},{"type":"v1","addr":"172.21.15.179:6829","nonce":880358389}]},"public_addr":"172.21.15.179:6825/880358389","cluster_addr":"172.21.15.179:6827/880358389","heartbeat_back_addr":"172.21.15.179:6831/880358389","heartbeat_front_addr":"172.21.15.179:6829/880358389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:08.282444+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:31.743685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:01:55.449273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:21.571839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:02:46.138742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:10.479110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:03:35.974245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:01.771022+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:26.640398+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:04:53.347513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:19.804633+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T09:05:45.423199+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1152515804":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3043598500":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/2011681658":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/615130791":"2024-05-10T08:59:35.011155+0000","172.21.15.64:0/3326248924":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/1070396506":"2024-05-10T08:59:08.628153+0000","172.21.15.64:0/2526964267":"2024-05-10T08:59:35.011155+0000","172.21.15.64:6800/421426620":"2024-05-10T08:59:08.628153+0000","172.21.15.64:6801/421426620":"2024-05-10T08:59:08.628153+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d100a2c10 msgr2=0x7f2d100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d100a2c10 0x7f2d100a50d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005e10 tx=0x7f2d0c005da0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 msgr2=0x7f2d2410b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 0x7f2d2410b0f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400eaa0 tx=0x7f2d1400ef60 comp rx=0 tx=0).stop 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 shutdown_connections 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f2d100a2c10 0x7f2d100a50d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24102320 0x7f2d24109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d24101960 0x7f2d24109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 --2- 172.21.15.64:0/776113735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d24100770 0x7f2d2410b0f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 >> 172.21.15.64:0/776113735 conn(0x7f2d240fbce0 msgr2=0x7f2d24104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 shutdown_connections 2024-05-09T09:26:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.004+0000 7f2d29770640 1 -- 172.21.15.64:0/776113735 wait complete. 2024-05-09T09:26:47.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 -- 172.21.15.64:0/3586944820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 msgr2=0x7f32b0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 --2- 172.21.15.64:0/3586944820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0102f00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3298009900 tx=0x7f329802a410 comp rx=0 tx=0).stop 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 -- 172.21.15.64:0/3586944820 shutdown_connections 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 --2- 172.21.15.64:0/3586944820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b0103440 0x7f32b0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 --2- 172.21.15.64:0/3586944820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0102f00 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 --2- 172.21.15.64:0/3586944820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b0101890 0x7f32b0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 -- 172.21.15.64:0/3586944820 >> 172.21.15.64:0/3586944820 conn(0x7f32b00fce00 msgr2=0x7f32b00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 -- 172.21.15.64:0/3586944820 shutdown_connections 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.122+0000 7f32b49b7640 1 -- 172.21.15.64:0/3586944820 wait complete. 2024-05-09T09:26:47.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 Processor -- start 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 -- start start 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b0101890 0x7f32b0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b0103440 0x7f32b019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f32b010d3c0 con 0x7f32b0102a80 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32b010d240 con 0x7f32b0101890 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32b49b7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f32b010d540 con 0x7f32b0103440 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41940/0 (socket says 172.21.15.64:41940) 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 -- 172.21.15.64:0/1837928709 learned_addr learned my addr 172.21.15.64:0/1837928709 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32aed9d640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b0103440 0x7f32b019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32ae59c640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b0101890 0x7f32b0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 -- 172.21.15.64:0/1837928709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b0103440 msgr2=0x7f32b019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b0103440 0x7f32b019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 -- 172.21.15.64:0/1837928709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b0101890 msgr2=0x7f32b0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b0101890 0x7f32b0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.123+0000 7f32a7fff640 1 -- 172.21.15.64:0/1837928709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b019f710 con 0x7f32b0102a80 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.124+0000 7f32a7fff640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0198030 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3298004770 tx=0x7f32980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.124+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329803f030 con 0x7f32b0102a80 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.124+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3298003cb0 con 0x7f32b0102a80 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.124+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b019f9a0 con 0x7f32b0102a80 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.124+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298035070 con 0x7f32b0102a80 2024-05-09T09:26:47.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.124+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b019fe90 con 0x7f32b0102a80 2024-05-09T09:26:47.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.125+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b0108c20 con 0x7f32b0102a80 2024-05-09T09:26:47.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.126+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3298042030 con 0x7f32b0102a80 2024-05-09T09:26:47.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.127+0000 7f32a77fe640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f329c0a2b90 0x7f329c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.127+0000 7f32ae59c640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f329c0a2b90 0x7f329c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.127+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f32980f0020 con 0x7f32b0102a80 2024-05-09T09:26:47.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.127+0000 7f32ae59c640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f329c0a2b90 0x7f329c0a5050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f32a0005fd0 tx=0x7f32a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.130+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32980bcbf0 con 0x7f32b0102a80 2024-05-09T09:26:47.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.354+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f32b002d410 con 0x7f32b0102a80 2024-05-09T09:26:47.355 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:47 smithi064 ceph-mon[30149]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-05-09T09:26:47.355 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/4141010850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.356 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2957717260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.356 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:47 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/776113735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.358+0000 7f32a77fe640 1 -- 172.21.15.64:0/1837928709 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7f32980bc4f0 con 0x7f32b0102a80 2024-05-09T09:26:47.357 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f329c0a2b90 msgr2=0x7f329c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f329c0a2b90 0x7f329c0a5050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f32a0005fd0 tx=0x7f32a0005850 comp rx=0 tx=0).stop 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 msgr2=0x7f32b0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0198030 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3298004770 tx=0x7f32980042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 shutdown_connections 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f329c0a2b90 0x7f329c0a5050 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b0103440 0x7f32b019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b0102a80 0x7f32b0198030 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 --2- 172.21.15.64:0/1837928709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b0101890 0x7f32b0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 >> 172.21.15.64:0/1837928709 conn(0x7f32b00fce00 msgr2=0x7f32b0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 shutdown_connections 2024-05-09T09:26:47.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.360+0000 7f32b49b7640 1 -- 172.21.15.64:0/1837928709 wait complete. 2024-05-09T09:26:47.373 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T09:26:47.443 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mon[33599]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-05-09T09:26:47.443 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/4141010850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.445 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2957717260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.445 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/776113735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mon[34086]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-05-09T09:26:47.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/4141010850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2957717260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/776113735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T09:26:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 -- 172.21.15.64:0/333920381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 msgr2=0x7feb10100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 --2- 172.21.15.64:0/333920381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb10100ec0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7feb00009900 tx=0x7feb0002a410 comp rx=0 tx=0).stop 2024-05-09T09:26:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 -- 172.21.15.64:0/333920381 shutdown_connections 2024-05-09T09:26:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 --2- 172.21.15.64:0/333920381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10101400 0x7feb10101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 --2- 172.21.15.64:0/333920381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb10100ec0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 --2- 172.21.15.64:0/333920381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb10100100 0x7feb10100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.482+0000 7feb173db640 1 -- 172.21.15.64:0/333920381 >> 172.21.15.64:0/333920381 conn(0x7feb100fbd20 msgr2=0x7feb100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 -- 172.21.15.64:0/333920381 shutdown_connections 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 -- 172.21.15.64:0/333920381 wait complete. 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 Processor -- start 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 -- start start 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10100100 0x7feb10199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb1019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb10101400 0x7feb101a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010f580 con 0x7feb10100a60 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010f400 con 0x7feb10101400 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.483+0000 7feb173db640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010f700 con 0x7feb10100100 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb1019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb1019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41956/0 (socket says 172.21.15.64:41956) 2024-05-09T09:26:47.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 -- 172.21.15.64:0/2732770340 learned_addr learned my addr 172.21.15.64:0/2732770340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb15177640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10100100 0x7feb10199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb15978640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb10101400 0x7feb101a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 -- 172.21.15.64:0/2732770340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10100100 msgr2=0x7feb10199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10100100 0x7feb10199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 -- 172.21.15.64:0/2732770340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb10101400 msgr2=0x7feb101a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb10101400 0x7feb101a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 -- 172.21.15.64:0/2732770340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb101a1890 con 0x7feb10100a60 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb15177640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10100100 0x7feb10199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb14976640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb1019a1c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7feb00004770 tx=0x7feb000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0003f030 con 0x7feb10100a60 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb00003cb0 con 0x7feb10100a60 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb101a1b20 con 0x7feb10100a60 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00035070 con 0x7feb10100a60 2024-05-09T09:26:47.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.484+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a2010 con 0x7feb10100a60 2024-05-09T09:26:47.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.485+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1010ae50 con 0x7feb10100a60 2024-05-09T09:26:47.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.487+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb00042030 con 0x7feb10100a60 2024-05-09T09:26:47.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.487+0000 7feb067fc640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feafc0a2b90 0x7feafc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:47.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.487+0000 7feb15177640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feafc0a2b90 0x7feafc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:47.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.487+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feb000f01b0 con 0x7feb10100a60 2024-05-09T09:26:47.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.488+0000 7feb15177640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feafc0a2b90 0x7feafc0a5050 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feb0c005fd0 tx=0x7feb0c005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:47.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.491+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feb000bcc80 con 0x7feb10100a60 2024-05-09T09:26:47.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:47 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a[30145]: 2024-05-09T09:26:47.355+0000 7f41f5b24700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:26:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:47.713+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feb1002d350 con 0x7feb10100a60 2024-05-09T09:26:47.729 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c[64381]: ignoring --setuser ceph since I am not root 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c[64381]: ignoring --setgroup ceph since I am not root 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mds[64385]: -- 172.21.15.179:0/2474923644 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aeb58b7a40 con 0x55aeb588b000 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c[64381]: starting mds.c at 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f[64064]: ignoring --setuser ceph since I am not root 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f[64064]: ignoring --setgroup ceph since I am not root 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mds[64074]: -- 172.21.15.179:0/184590499 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e4f67ba40 con 0x558e4f64f000 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f[64064]: starting mds.f at 2024-05-09T09:26:47.730 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i[64686]: ignoring --setuser ceph since I am not root 2024-05-09T09:26:47.731 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i[64686]: ignoring --setgroup ceph since I am not root 2024-05-09T09:26:47.731 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:26:47 smithi179 ceph-mds[64690]: -- 172.21.15.179:0/4199114325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d7c0aba40 con 0x564d7c07ec00 2024-05-09T09:26:47.731 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:26:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i[64686]: starting mds.i at 2024-05-09T09:26:47.823 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: ignoring --setuser ceph since I am not root 2024-05-09T09:26:47.823 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: ignoring --setgroup ceph since I am not root 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mds[64284]: -- 172.21.15.139:0/4135325608 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d03789da40 con 0x55d037870c00 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: starting mds.b at 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h[63973]: ignoring --setuser ceph since I am not root 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h[63973]: ignoring --setgroup ceph since I am not root 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mds[63977]: -- 172.21.15.139:0/3942926056 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55925239ba40 con 0x55925236ec00 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h[63973]: starting mds.h at 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e[63663]: ignoring --setuser ceph since I am not root 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e[63663]: ignoring --setgroup ceph since I am not root 2024-05-09T09:26:47.824 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:26:47 smithi139 ceph-mds[63667]: -- 172.21.15.139:0/824601598 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55911b007a40 con 0x55911afda800 2024-05-09T09:26:47.825 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:26:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e[63663]: starting mds.e at 2024-05-09T09:26:48.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.365+0000 7feb067fc640 1 -- 172.21.15.64:0/2732770340 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7feb000bc580 con 0x7feb10100a60 2024-05-09T09:26:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feafc0a2b90 msgr2=0x7feafc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feafc0a2b90 0x7feafc0a5050 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feb0c005fd0 tx=0x7feb0c005f40 comp rx=0 tx=0).stop 2024-05-09T09:26:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 msgr2=0x7feb1019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb1019a1c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7feb00004770 tx=0x7feb000042f0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 shutdown_connections 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7feafc0a2b90 0x7feafc0a5050 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb10101400 0x7feb101a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb10100a60 0x7feb1019a1c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 --2- 172.21.15.64:0/2732770340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb10100100 0x7feb10199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 >> 172.21.15.64:0/2732770340 conn(0x7feb100fbd20 msgr2=0x7feb100fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.368+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 shutdown_connections 2024-05-09T09:26:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.369+0000 7feb173db640 1 -- 172.21.15.64:0/2732770340 wait complete. 2024-05-09T09:26:48.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T09:26:48.408 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1837928709' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T09:26:48.408 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T09:26:48.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:26:48.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1837928709' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T09:26:48.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T09:26:48.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T09:26:48.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:48 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2732770340' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1837928709' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1837928709' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T09:26:48.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:48 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2732770340' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 -- 172.21.15.64:0/299083679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 msgr2=0x7fc6e4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/299083679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4101c90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009900 tx=0x7fc6d402a410 comp rx=0 tx=0).stop 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 -- 172.21.15.64:0/299083679 shutdown_connections 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/299083679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4103420 0x7fc6e4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/299083679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e4102a60 0x7fc6e4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/299083679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4101c90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.499+0000 7fc6eab5a640 1 -- 172.21.15.64:0/299083679 >> 172.21.15.64:0/299083679 conn(0x7fc6e40fcde0 msgr2=0x7fc6e40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 -- 172.21.15.64:0/299083679 shutdown_connections 2024-05-09T09:26:48.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 -- 172.21.15.64:0/299083679 wait complete. 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 Processor -- start 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 -- start start 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4102a60 0x7fc6e4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e4103420 0x7fc6e419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410d370 con 0x7fc6e4101870 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410d1f0 con 0x7fc6e4103420 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6eab5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410d4f0 con 0x7fc6e4102a60 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6e88f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6e88f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41980/0 (socket says 172.21.15.64:41980) 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6e88f6640 1 -- 172.21.15.64:0/1742042398 learned_addr learned my addr 172.21.15.64:0/1742042398 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.500+0000 7fc6e90f7640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e4103420 0x7fc6e419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e3fff640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4102a60 0x7fc6e4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e88f6640 1 -- 172.21.15.64:0/1742042398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4102a60 msgr2=0x7fc6e4198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e88f6640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4102a60 0x7fc6e4198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e88f6640 1 -- 172.21.15.64:0/1742042398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e4103420 msgr2=0x7fc6e419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e88f6640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e4103420 0x7fc6e419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e88f6640 1 -- 172.21.15.64:0/1742042398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e419f700 con 0x7fc6e4101870 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e88f6640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4197b30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40314e0 tx=0x7fc6d40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d403f030 con 0x7fc6e4101870 2024-05-09T09:26:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d4003cb0 con 0x7fc6e4101870 2024-05-09T09:26:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e419f990 con 0x7fc6e4101870 2024-05-09T09:26:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4035070 con 0x7fc6e4101870 2024-05-09T09:26:48.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.501+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e419fed0 con 0x7fc6e4101870 2024-05-09T09:26:48.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.502+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e4108d20 con 0x7fc6e4101870 2024-05-09T09:26:48.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.504+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc6d4042030 con 0x7fc6e4101870 2024-05-09T09:26:48.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.504+0000 7fc6e1ffb640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6cc0a2c10 0x7fc6cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:48.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.505+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6d40efa60 con 0x7fc6e4101870 2024-05-09T09:26:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.505+0000 7fc6e3fff640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6cc0a2c10 0x7fc6cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:48.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.505+0000 7fc6e3fff640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6cc0a2c10 0x7fc6cc0a50d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:48.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.509+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6d40bc530 con 0x7fc6e4101870 2024-05-09T09:26:48.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1837928709' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T09:26:48.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T09:26:48.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T09:26:48.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1837928709' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T09:26:48.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T09:26:48.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T09:26:48.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:48 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2732770340' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T09:26:48.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:48.694+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6e4062430 con 0x7fc6e4101870 2024-05-09T09:26:49.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.367+0000 7fc6e1ffb640 1 -- 172.21.15.64:0/1742042398 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7fc6d40bbe30 con 0x7fc6e4101870 2024-05-09T09:26:49.368 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-05-09T09:26:49.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6cc0a2c10 msgr2=0x7fc6cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:49.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6cc0a2c10 0x7fc6cc0a50d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d8005850 comp rx=0 tx=0).stop 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 msgr2=0x7fc6e4197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4197b30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40314e0 tx=0x7fc6d40042e0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 shutdown_connections 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7fc6cc0a2c10 0x7fc6cc0a50d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e4103420 0x7fc6e419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4102a60 0x7fc6e4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 --2- 172.21.15.64:0/1742042398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e4101870 0x7fc6e4197b30 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 >> 172.21.15.64:0/1742042398 conn(0x7fc6e40fcde0 msgr2=0x7fc6e4105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 shutdown_connections 2024-05-09T09:26:49.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.372+0000 7fc6eab5a640 1 -- 172.21.15.64:0/1742042398 wait complete. 2024-05-09T09:26:49.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T09:26:49.454 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: Cluster is now healthy 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2732770340' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6834/2367714828,v1:172.21.15.139:6835/2367714828] up:boot 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6836/4059235972,v1:172.21.15.139:6838/4059235972] up:boot 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.139:6837/3448208772,v1:172.21.15.139:6839/3448208772] up:boot 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6834/1916763268,v1:172.21.15.179:6837/1916763268] up:boot 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6835/3538357478,v1:172.21.15.179:6839/3538357478] up:boot 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: mds.? [v2:172.21.15.179:6836/259989780,v1:172.21.15.179:6838/259989780] up:boot 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: fsmap 12 up:standby 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:26:49.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:26:49.456 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:26:49.456 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:26:49.456 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:26:49.456 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:26:49.456 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:49 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1742042398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: Cluster is now healthy 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2732770340' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6834/2367714828,v1:172.21.15.139:6835/2367714828] up:boot 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6836/4059235972,v1:172.21.15.139:6838/4059235972] up:boot 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.139:6837/3448208772,v1:172.21.15.139:6839/3448208772] up:boot 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6834/1916763268,v1:172.21.15.179:6837/1916763268] up:boot 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6835/3538357478,v1:172.21.15.179:6839/3538357478] up:boot 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: mds.? [v2:172.21.15.179:6836/259989780,v1:172.21.15.179:6838/259989780] up:boot 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: fsmap 12 up:standby 2024-05-09T09:26:49.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:26:49.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:26:49.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:26:49.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:26:49.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:26:49.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:26:49.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:49 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1742042398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 -- 172.21.15.64:0/999681230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 msgr2=0x7f063c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 --2- 172.21.15.64:0/999681230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c108bd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0634009ca0 tx=0x7f063402a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 -- 172.21.15.64:0/999681230 shutdown_connections 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 --2- 172.21.15.64:0/999681230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c108bd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 --2- 172.21.15.64:0/999681230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 0x7f063c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 --2- 172.21.15.64:0/999681230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f063c100790 0x7f063c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 -- 172.21.15.64:0/999681230 >> 172.21.15.64:0/999681230 conn(0x7f063c0fbd20 msgr2=0x7f063c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 -- 172.21.15.64:0/999681230 shutdown_connections 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.550+0000 7f0642838640 1 -- 172.21.15.64:0/999681230 wait complete. 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 Processor -- start 2024-05-09T09:26:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 -- start start 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f063c100790 0x7f063c104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 0x7f063c102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c1032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10c2a0 con 0x7f063c102340 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10c120 con 0x7f063c101980 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.551+0000 7f0642838640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10c420 con 0x7f063c100790 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c1032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c1032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42002/0 (socket says 172.21.15.64:42002) 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 -- 172.21.15.64:0/2556784204 learned_addr learned my addr 172.21.15.64:0/2556784204 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f063bfff640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f063c100790 0x7f063c104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f063b7fe640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 0x7f063c102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 -- 172.21.15.64:0/2556784204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f063c100790 msgr2=0x7f063c104720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f063c100790 0x7f063c104720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 -- 172.21.15.64:0/2556784204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 msgr2=0x7f063c102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 0x7f063c102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 -- 172.21.15.64:0/2556784204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c103b30 con 0x7f063c102340 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f063b7fe640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 0x7f063c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0640dd5640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c1032a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0634009dc0 tx=0x7f0634003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634004040 con 0x7f063c102340 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0634004690 con 0x7f063c102340 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c1a1bc0 con 0x7f063c102340 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063402f4c0 con 0x7f063c102340 2024-05-09T09:26:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.552+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1a2110 con 0x7f063c102340 2024-05-09T09:26:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.553+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c1a22e0 con 0x7f063c102340 2024-05-09T09:26:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.555+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0634030650 con 0x7f063c102340 2024-05-09T09:26:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.556+0000 7f06397fa640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06240a2b90 0x7f06240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.556+0000 7f063bfff640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06240a2b90 0x7f06240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.556+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8912+0+0 (secure 0 0 0) 0x7f06340f0bf0 con 0x7f063c102340 2024-05-09T09:26:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.557+0000 7f063bfff640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06240a2b90 0x7f06240a5050 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f063c1017e0 tx=0x7f062c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:49.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.560+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f06340bd8b0 con 0x7f063c102340 2024-05-09T09:26:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-05-09T09:26:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T09:26:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T09:26:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: Cluster is now healthy 2024-05-09T09:26:49.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2732770340' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6834/2367714828,v1:172.21.15.139:6835/2367714828] up:boot 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6836/4059235972,v1:172.21.15.139:6838/4059235972] up:boot 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.139:6837/3448208772,v1:172.21.15.139:6839/3448208772] up:boot 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6834/1916763268,v1:172.21.15.179:6837/1916763268] up:boot 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6835/3538357478,v1:172.21.15.179:6839/3538357478] up:boot 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: mds.? [v2:172.21.15.179:6836/259989780,v1:172.21.15.179:6838/259989780] up:boot 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: fsmap 12 up:standby 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='mgr.14152 172.21.15.64:0/1526750650' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T09:26:49.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:49 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1742042398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:49.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:49.751+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f063c1a24b0 con 0x7f063c102340 2024-05-09T09:26:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.375+0000 7f06397fa640 1 -- 172.21.15.64:0/2556784204 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f06340bd1b0 con 0x7f063c102340 2024-05-09T09:26:50.375 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.379+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06240a2b90 msgr2=0x7f06240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.379+0000 7f0642838640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06240a2b90 0x7f06240a5050 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f063c1017e0 tx=0x7f062c00a2d0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.379+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 msgr2=0x7f063c1032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.379+0000 7f0642838640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c1032a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0634009dc0 tx=0x7f0634003ba0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 shutdown_connections 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f06240a2b90 0x7f06240a5050 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f063c102340 0x7f063c1032a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c101980 0x7f063c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 --2- 172.21.15.64:0/2556784204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f063c100790 0x7f063c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 >> 172.21.15.64:0/2556784204 conn(0x7f063c0fbd20 msgr2=0x7f063c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 shutdown_connections 2024-05-09T09:26:50.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.380+0000 7f0642838640 1 -- 172.21.15.64:0/2556784204 wait complete. 2024-05-09T09:26:50.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-09T09:26:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 -- 172.21.15.64:0/702580524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0100790 msgr2=0x7f83c0100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 --2- 172.21.15.64:0/702580524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0100790 0x7f83c0100bb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009ca0 tx=0x7f83b002a8c0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 -- 172.21.15.64:0/702580524 shutdown_connections 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 --2- 172.21.15.64:0/702580524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83c0102340 0x7f83c0108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 --2- 172.21.15.64:0/702580524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101980 0x7f83c0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 --2- 172.21.15.64:0/702580524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0100790 0x7f83c0100bb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 -- 172.21.15.64:0/702580524 >> 172.21.15.64:0/702580524 conn(0x7f83c00fbd20 msgr2=0x7f83c00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 -- 172.21.15.64:0/702580524 shutdown_connections 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.556+0000 7f83c8190640 1 -- 172.21.15.64:0/702580524 wait complete. 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 Processor -- start 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 -- start start 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83c0100790 0x7f83c006a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101980 0x7f83c019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 0x7f83c019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f83c010c330 con 0x7f83c0102340 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83c010c1b0 con 0x7f83c0100790 2024-05-09T09:26:50.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c8190640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f83c010c4b0 con 0x7f83c0101980 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 0x7f83c019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 0x7f83c019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42022/0 (socket says 172.21.15.64:42022) 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 -- 172.21.15.64:0/799578891 learned_addr learned my addr 172.21.15.64:0/799578891 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c572b640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101980 0x7f83c019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c5f2c640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83c0100790 0x7f83c006a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 -- 172.21.15.64:0/799578891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101980 msgr2=0x7f83c019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101980 0x7f83c019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 -- 172.21.15.64:0/799578891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83c0100790 msgr2=0x7f83c006a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83c0100790 0x7f83c006a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.557+0000 7f83c672d640 1 -- 172.21.15.64:0/799578891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c019f7f0 con 0x7f83c0102340 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.558+0000 7f83c672d640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 0x7f83c019eee0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009fd0 tx=0x7f83b800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.558+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8011830 con 0x7f83c0102340 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.558+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83b8016070 con 0x7f83c0102340 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.558+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8010de0 con 0x7f83c0102340 2024-05-09T09:26:50.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.558+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c019fa80 con 0x7f83c0102340 2024-05-09T09:26:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.558+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c019ffd0 con 0x7f83c0102340 2024-05-09T09:26:50.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.559+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c0107bd0 con 0x7f83c0102340 2024-05-09T09:26:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.560+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f83b80119d0 con 0x7f83c0102340 2024-05-09T09:26:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.561+0000 7f83beffd640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83a80a2c10 0x7f83a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T09:26:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.561+0000 7f83c5f2c640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83a80a2c10 0x7f83a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T09:26:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.561+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8324+0+0 (secure 0 0 0) 0x7f83b80ce730 con 0x7f83c0102340 2024-05-09T09:26:50.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.562+0000 7f83c5f2c640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83a80a2c10 0x7f83a80a50d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000f80 tx=0x7f83b0031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T09:26:50.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.566+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83b809b630 con 0x7f83c0102340 2024-05-09T09:26:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:50 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/1742042398' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:26:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:50 smithi064 ceph-mon[30149]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T09:26:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:50 smithi064 ceph-mon[30149]: pgmap v903: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:26:50.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:50 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2556784204' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:50.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:50 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/1742042398' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:26:50.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:50 smithi179 ceph-mon[34086]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T09:26:50.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:50 smithi179 ceph-mon[34086]: pgmap v903: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:26:50.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:50 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2556784204' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:50.775+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f83c0062430 con 0x7f83c0102340 2024-05-09T09:26:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:50 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/1742042398' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:26:50.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:50 smithi139 ceph-mon[33599]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T09:26:50.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:50 smithi139 ceph-mon[33599]: pgmap v903: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T09:26:50.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:50 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2556784204' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:51.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.377+0000 7f83beffd640 1 -- 172.21.15.64:0/799578891 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7f83b809af30 con 0x7f83c0102340 2024-05-09T09:26:51.377 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data_ec' removed 2024-05-09T09:26:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83a80a2c10 msgr2=0x7f83a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83a80a2c10 0x7f83a80a50d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000f80 tx=0x7f83b0031000 comp rx=0 tx=0).stop 2024-05-09T09:26:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 msgr2=0x7f83c019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T09:26:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 0x7f83c019eee0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009fd0 tx=0x7f83b800b8e0 comp rx=0 tx=0).stop 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 shutdown_connections 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:6800/2084062196,v1:172.21.15.64:6801/2084062196] conn(0x7f83a80a2c10 0x7f83a80a50d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83c0102340 0x7f83c019eee0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101980 0x7f83c019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 --2- 172.21.15.64:0/799578891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83c0100790 0x7f83c006a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.380+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 >> 172.21.15.64:0/799578891 conn(0x7f83c00fbd20 msgr2=0x7f83c0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.381+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 shutdown_connections 2024-05-09T09:26:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-09T09:26:51.381+0000 7f83c8190640 1 -- 172.21.15.64:0/799578891 wait complete. 2024-05-09T09:26:51.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:26:51.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:26:51.466 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:26:51.510 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T09:26:51.511 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:26:51.531 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:26:51.548 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T09:26:51.568 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T09:26:51.569 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T09:26:51.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a 2024-05-09T09:26:51.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:51 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/2556784204' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:26:51.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:51 smithi064 ceph-mon[30149]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T09:26:51.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:51 smithi064 ceph-mon[30149]: from='client.? 172.21.15.64:0/799578891' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:51.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:51 smithi064 systemd[1]: Stopping Ceph mon.a for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:26:51.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:51 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/2556784204' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:26:51.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:51 smithi179 ceph-mon[34086]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T09:26:51.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:51 smithi179 ceph-mon[34086]: from='client.? 172.21.15.64:0/799578891' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:51.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:51 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/2556784204' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T09:26:51.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:51 smithi139 ceph-mon[33599]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T09:26:51.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:51 smithi139 ceph-mon[33599]: from='client.? 172.21.15.64:0/799578891' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T09:26:52.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a[30145]: 2024-05-09T09:26:52.079+0000 7f41fb32f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:26:52.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a[30145]: 2024-05-09T09:26:52.079+0000 7f41fb32f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T09:26:52.403 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:52 smithi064 podman[110625]: 2024-05-09 09:26:52.17215385 +0000 UTC m=+0.341153940 container died 8a68b30aba117b1b13482e7c4007555df3b861870b653b3e05457d2913383a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T09:26:52.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:52 smithi064 podman[110625]: 2024-05-09 09:26:52.572205425 +0000 UTC m=+0.741205514 container cleanup 8a68b30aba117b1b13482e7c4007555df3b861870b653b3e05457d2913383a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:26:52.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:May 09 09:26:52 smithi064 bash[110625]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-a 2024-05-09T09:26:52.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.a.service' 2024-05-09T09:26:53.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:26:53.091 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T09:26:53.091 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T09:26:53.091 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.b 2024-05-09T09:26:53.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:53 smithi139 systemd[1]: Stopping Ceph mon.b for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:26:53.799 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-b[33595]: 2024-05-09T09:26:53.531+0000 7f9ffa0ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:26:54.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-b[33595]: 2024-05-09T09:26:53.531+0000 7f9ffa0ed700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T09:26:54.224 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:53 smithi139 podman[70003]: 2024-05-09 09:26:53.583442566 +0000 UTC m=+0.187570712 container died 0a9efa793d1900cc172762dc745c8c27093950670b508ac871f3355eb04f34c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-09T09:26:54.225 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:54 smithi139 podman[70003]: 2024-05-09 09:26:54.000581474 +0000 UTC m=+0.604709620 container cleanup 0a9efa793d1900cc172762dc745c8c27093950670b508ac871f3355eb04f34c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T09:26:54.225 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:54 smithi139 bash[70003]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-b 2024-05-09T09:26:54.426 INFO:journalctl@ceph.mon.b.smithi139.stdout:May 09 09:26:54 smithi139 podman[70013]: 2024-05-09 09:26:54.319379654 +0000 UTC m=+0.711125621 container remove 0a9efa793d1900cc172762dc745c8c27093950670b508ac871f3355eb04f34c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:26:54.442 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.b.service' 2024-05-09T09:26:54.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:26:54.496 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T09:26:54.496 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T09:26:54.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.c 2024-05-09T09:26:54.868 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:54 smithi179 systemd[1]: Stopping Ceph mon.c for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:26:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-c[34082]: 2024-05-09T09:26:54.868+0000 7f7ee4ab8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:26:55.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-c[34082]: 2024-05-09T09:26:54.868+0000 7f7ee4ab8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T09:26:55.230 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:55 smithi179 podman[70238]: 2024-05-09 09:26:55.114001552 +0000 UTC m=+0.366506656 container died 915ca2680a39e78ad55277d5e62d16cf8b5667269e07f1b49f531ad5bf4089b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T09:26:55.729 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:55 smithi179 podman[70238]: 2024-05-09 09:26:55.518765062 +0000 UTC m=+0.771270239 container cleanup 915ca2680a39e78ad55277d5e62d16cf8b5667269e07f1b49f531ad5bf4089b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:26:55.730 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 09 09:26:55 smithi179 bash[70238]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mon-c 2024-05-09T09:26:55.926 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mon.c.service' 2024-05-09T09:26:55.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:26:55.976 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T09:26:55.976 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T09:26:55.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x 2024-05-09T09:26:56.285 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:26:56 smithi064 systemd[1]: Stopping Ceph mgr.x for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:26:56.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:26:56 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x[30380]: 2024-05-09T09:26:56.283+0000 7f4e1bd6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:26:56.653 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:26:56 smithi064 podman[110747]: 2024-05-09 09:26:56.316921739 +0000 UTC m=+0.154006896 container died ef41d033accc09139b9c1592f15cbadfc7d4fad3d46f11575bdbacfcbb5d687b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T09:26:57.112 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:26:56 smithi064 podman[110747]: 2024-05-09 09:26:56.861361621 +0000 UTC m=+0.698446789 container cleanup ef41d033accc09139b9c1592f15cbadfc7d4fad3d46f11575bdbacfcbb5d687b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T09:26:57.112 INFO:journalctl@ceph.mgr.x.smithi064.stdout:May 09 09:26:56 smithi064 bash[110747]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-x 2024-05-09T09:26:57.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-09T09:26:57.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:26:57.368 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T09:26:57.369 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T09:26:57.369 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.y 2024-05-09T09:26:57.709 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:26:57 smithi139 systemd[1]: Stopping Ceph mgr.y for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:26:58.014 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:26:57 smithi139 podman[70132]: 2024-05-09 09:26:57.709828108 +0000 UTC m=+0.140986552 container died 92f936f5be87ace2a0483581b84c6ce49d1cc03ef98b5d20149c254911e63b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T09:26:58.323 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:26:58 smithi139 podman[70132]: 2024-05-09 09:26:58.114841522 +0000 UTC m=+0.545999971 container cleanup 92f936f5be87ace2a0483581b84c6ce49d1cc03ef98b5d20149c254911e63b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T09:26:58.323 INFO:journalctl@ceph.mgr.y.smithi139.stdout:May 09 09:26:58 smithi139 bash[70132]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-y 2024-05-09T09:26:58.487 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-09T09:26:58.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:26:58.525 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T09:26:58.525 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T09:26:58.525 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.z 2024-05-09T09:26:58.919 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:26:58 smithi179 systemd[1]: Stopping Ceph mgr.z for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:26:59.224 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:26:58 smithi179 podman[70375]: 2024-05-09 09:26:58.920116084 +0000 UTC m=+0.205354516 container died 5170f2b17a6448f422403346fed04b4fe1e33e68e7cc39a1170e036951332223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:26:59.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:26:59 smithi179 podman[70375]: 2024-05-09 09:26:59.324253297 +0000 UTC m=+0.609491655 container cleanup 5170f2b17a6448f422403346fed04b4fe1e33e68e7cc39a1170e036951332223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T09:26:59.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 09 09:26:59 smithi179 bash[70375]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mgr-z 2024-05-09T09:27:00.002 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-09T09:27:00.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:00.039 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T09:27:00.039 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T09:27:00.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.0 2024-05-09T09:27:00.374 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:00 smithi064 systemd[1]: Stopping Ceph osd.0 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:00.653 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0[42705]: 2024-05-09T09:27:00.373+0000 7fa3c84af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:00.653 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0[42705]: 2024-05-09T09:27:00.373+0000 7fa3c84af700 -1 osd.0 98 *** Got signal Terminated *** 2024-05-09T09:27:00.653 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0[42705]: 2024-05-09T09:27:00.373+0000 7fa3c84af700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:06.153 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:05 smithi064 podman[110869]: 2024-05-09 09:27:05.883693686 +0000 UTC m=+5.638755148 container died 5240c23a9f5529c7d9cda52ea5723df59020d96ff24de92aca1306ffde05ba3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T09:27:06.451 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:06 smithi064 podman[110869]: 2024-05-09 09:27:06.408497983 +0000 UTC m=+6.163559435 container cleanup 5240c23a9f5529c7d9cda52ea5723df59020d96ff24de92aca1306ffde05ba3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True) 2024-05-09T09:27:06.451 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:06 smithi064 bash[110869]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0 2024-05-09T09:27:06.903 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:06 smithi064 podman[110889]: 2024-05-09 09:27:06.677226144 +0000 UTC m=+0.791796518 container remove 5240c23a9f5529c7d9cda52ea5723df59020d96ff24de92aca1306ffde05ba3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0, RELEASE=squid-96805dc, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T09:27:07.403 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:07 smithi064 podman[110984]: 2024-05-09 09:27:07.007641449 +0000 UTC m=+0.018583809 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:07.403 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:07 smithi064 podman[110984]: 2024-05-09 09:27:07.153441864 +0000 UTC m=+0.164384242 container create 0726976b8e9ed45ef70ad8d39b23d75acabc2cbc9ce7f3ed383449f1ca6bf25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:27:07.903 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:07 smithi064 podman[110984]: 2024-05-09 09:27:07.427171456 +0000 UTC m=+0.438113792 container init 0726976b8e9ed45ef70ad8d39b23d75acabc2cbc9ce7f3ed383449f1ca6bf25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True) 2024-05-09T09:27:07.903 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:07 smithi064 podman[110984]: 2024-05-09 09:27:07.430665321 +0000 UTC m=+0.441607657 container start 0726976b8e9ed45ef70ad8d39b23d75acabc2cbc9ce7f3ed383449f1ca6bf25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T09:27:07.903 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:07 smithi064 podman[110984]: 2024-05-09 09:27:07.531617721 +0000 UTC m=+0.542560068 container attach 0726976b8e9ed45ef70ad8d39b23d75acabc2cbc9ce7f3ed383449f1ca6bf25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:27:08.403 INFO:journalctl@ceph.osd.0.smithi064.stdout:May 09 09:27:08 smithi064 podman[110984]: 2024-05-09 09:27:08.109644273 +0000 UTC m=+1.120586620 container died 0726976b8e9ed45ef70ad8d39b23d75acabc2cbc9ce7f3ed383449f1ca6bf25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T09:27:08.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.0.service' 2024-05-09T09:27:08.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:08.966 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T09:27:08.967 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T09:27:08.967 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.1 2024-05-09T09:27:09.296 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:09 smithi139 systemd[1]: Stopping Ceph osd.1 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:09.573 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1[38199]: 2024-05-09T09:27:09.296+0000 7f2cd2c20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:09.573 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1[38199]: 2024-05-09T09:27:09.296+0000 7f2cd2c20700 -1 osd.1 98 *** Got signal Terminated *** 2024-05-09T09:27:09.573 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1[38199]: 2024-05-09T09:27:09.296+0000 7f2cd2c20700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:15.273 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:14 smithi139 podman[70258]: 2024-05-09 09:27:14.966264343 +0000 UTC m=+5.802333653 container died e23eb75103fec2b0e60cdd0200111086ca36dcf5829c28efcdad27e2d2ecc003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:27:15.573 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:15 smithi139 podman[70258]: 2024-05-09 09:27:15.372135885 +0000 UTC m=+6.208205172 container cleanup e23eb75103fec2b0e60cdd0200111086ca36dcf5829c28efcdad27e2d2ecc003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0) 2024-05-09T09:27:15.573 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:15 smithi139 bash[70258]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1 2024-05-09T09:27:15.892 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:15 smithi139 podman[70275]: 2024-05-09 09:27:15.582432956 +0000 UTC m=+0.616283658 container remove e23eb75103fec2b0e60cdd0200111086ca36dcf5829c28efcdad27e2d2ecc003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T09:27:16.186 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:15 smithi139 podman[70362]: 2024-05-09 09:27:15.79629625 +0000 UTC m=+0.018080534 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:16.186 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:15 smithi139 podman[70362]: 2024-05-09 09:27:15.90911652 +0000 UTC m=+0.130900795 container create e50fae7dfd457edfb98691523fc6c2bfbc7b81bbf84852adef16406d0cd93ef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:27:16.186 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:16 smithi139 podman[70362]: 2024-05-09 09:27:16.187326063 +0000 UTC m=+0.409110338 container init e50fae7dfd457edfb98691523fc6c2bfbc7b81bbf84852adef16406d0cd93ef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T09:27:16.573 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:16 smithi139 podman[70362]: 2024-05-09 09:27:16.190728634 +0000 UTC m=+0.412512913 container start e50fae7dfd457edfb98691523fc6c2bfbc7b81bbf84852adef16406d0cd93ef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:16.574 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:16 smithi139 podman[70362]: 2024-05-09 09:27:16.31237049 +0000 UTC m=+0.534154769 container attach e50fae7dfd457edfb98691523fc6c2bfbc7b81bbf84852adef16406d0cd93ef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:17.073 INFO:journalctl@ceph.osd.1.smithi139.stdout:May 09 09:27:16 smithi139 podman[70362]: 2024-05-09 09:27:16.728365279 +0000 UTC m=+0.950149566 container died e50fae7dfd457edfb98691523fc6c2bfbc7b81bbf84852adef16406d0cd93ef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T09:27:17.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.1.service' 2024-05-09T09:27:17.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:17.486 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T09:27:17.486 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T09:27:17.486 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.2 2024-05-09T09:27:17.920 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:17 smithi179 systemd[1]: Stopping Ceph osd.2 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:18.229 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2[39241]: 2024-05-09T09:27:17.919+0000 7f7625e07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:18.230 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2[39241]: 2024-05-09T09:27:17.919+0000 7f7625e07700 -1 osd.2 98 *** Got signal Terminated *** 2024-05-09T09:27:18.230 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2[39241]: 2024-05-09T09:27:17.919+0000 7f7625e07700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:23.820 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:23 smithi179 podman[70500]: 2024-05-09 09:27:23.517432414 +0000 UTC m=+5.840455676 container died 5a7f4e139547d4bbc8e086305932aa1b7b2090910d55124a0dac97c9f4893e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T09:27:24.154 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:23 smithi179 podman[70500]: 2024-05-09 09:27:23.903880073 +0000 UTC m=+6.226903337 container cleanup 5a7f4e139547d4bbc8e086305932aa1b7b2090910d55124a0dac97c9f4893e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:27:24.154 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:23 smithi179 bash[70500]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2 2024-05-09T09:27:24.479 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:24 smithi179 podman[70518]: 2024-05-09 09:27:24.155312042 +0000 UTC m=+0.635726210 container remove 5a7f4e139547d4bbc8e086305932aa1b7b2090910d55124a0dac97c9f4893e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:24.800 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:24 smithi179 podman[70607]: 2024-05-09 09:27:24.421582519 +0000 UTC m=+0.018227544 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:24.800 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:24 smithi179 podman[70607]: 2024-05-09 09:27:24.556267247 +0000 UTC m=+0.152912270 container create 9cbd06423e2a63c9972dc8985c6530f18f5ca9ad6dcc5431bf3c1176b310cbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:25.230 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:24 smithi179 podman[70607]: 2024-05-09 09:27:24.801009767 +0000 UTC m=+0.397654801 container init 9cbd06423e2a63c9972dc8985c6530f18f5ca9ad6dcc5431bf3c1176b310cbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:27:25.248 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:24 smithi179 podman[70607]: 2024-05-09 09:27:24.804883392 +0000 UTC m=+0.401528417 container start 9cbd06423e2a63c9972dc8985c6530f18f5ca9ad6dcc5431bf3c1176b310cbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:27:25.248 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:24 smithi179 podman[70607]: 2024-05-09 09:27:24.933889048 +0000 UTC m=+0.530534086 container attach 9cbd06423e2a63c9972dc8985c6530f18f5ca9ad6dcc5431bf3c1176b310cbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:25.617 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 09 09:27:25 smithi179 podman[70607]: 2024-05-09 09:27:25.341810081 +0000 UTC m=+0.938455100 container died 9cbd06423e2a63c9972dc8985c6530f18f5ca9ad6dcc5431bf3c1176b310cbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc) 2024-05-09T09:27:25.997 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.2.service' 2024-05-09T09:27:26.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:26.037 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T09:27:26.037 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T09:27:26.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.3 2024-05-09T09:27:26.272 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:26 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:26.016+0000 7f67f2ab1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.610042+0000 front 2024-05-09T09:27:03.610167+0000 (oldest deadline 2024-05-09T09:27:25.910189+0000) 2024-05-09T09:27:26.272 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:26 smithi064 systemd[1]: Stopping Ceph osd.3 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:26.304 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:26.047+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:26.653 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:26 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:26.363+0000 7f67fa4c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:26.653 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:26 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:26.363+0000 7f67fa4c5700 -1 osd.3 98 *** Got signal Terminated *** 2024-05-09T09:27:26.653 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:26 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:26.363+0000 7f67fa4c5700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:27.292 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:26 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:26.989+0000 7f67f2ab1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.610042+0000 front 2024-05-09T09:27:03.610167+0000 (oldest deadline 2024-05-09T09:27:25.910189+0000) 2024-05-09T09:27:27.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:27.011+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:27.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:27.199+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:28.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:27.966+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:28.403 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:27 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:27.946+0000 7f67f2ab1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.610042+0000 front 2024-05-09T09:27:03.610167+0000 (oldest deadline 2024-05-09T09:27:25.910189+0000) 2024-05-09T09:27:28.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:28 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:28.178+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:28.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:28.161+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:29.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:28.930+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:29.403 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:28 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:28.974+0000 7f67f2ab1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.610042+0000 front 2024-05-09T09:27:03.610167+0000 (oldest deadline 2024-05-09T09:27:25.910189+0000) 2024-05-09T09:27:29.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:29 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:29.183+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:29.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:29.149+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:29.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:29.003+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:29.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:29.147+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:30.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:29.918+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:30.403 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:29 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:29.984+0000 7f67f2ab1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.610042+0000 front 2024-05-09T09:27:03.610167+0000 (oldest deadline 2024-05-09T09:27:25.910189+0000) 2024-05-09T09:27:30.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:30 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:30.156+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:30.479 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:30.044+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:30.480 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:30.156+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:30.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:30.146+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:31.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:30 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:30.923+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:31.403 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:31 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3[49614]: 2024-05-09T09:27:31.001+0000 7f67f2ab1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.610042+0000 front 2024-05-09T09:27:03.610167+0000 (oldest deadline 2024-05-09T09:27:25.910189+0000) 2024-05-09T09:27:31.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:31 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:31.189+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:31.479 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:31.032+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:31.480 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:31.114+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:31.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:31.174+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:32.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:31 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:31.964+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:32.403 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:32 smithi064 podman[111090]: 2024-05-09 09:27:32.031474729 +0000 UTC m=+5.786536294 container died e6dca3b6e2dcc74ce2072cddb8c529bc6225b28c3d9c3863964b00ac65fd8030 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T09:27:32.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:32 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:32.142+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:32.479 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:32.005+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:32.480 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:32.083+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:32.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:32.209+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:32.797 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:32 smithi064 podman[111090]: 2024-05-09 09:27:32.563526297 +0000 UTC m=+6.318587851 container cleanup e6dca3b6e2dcc74ce2072cddb8c529bc6225b28c3d9c3863964b00ac65fd8030 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:27:32.797 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:32 smithi064 bash[111090]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3 2024-05-09T09:27:33.126 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:32 smithi064 podman[111119]: 2024-05-09 09:27:32.798094406 +0000 UTC m=+0.760511203 container remove e6dca3b6e2dcc74ce2072cddb8c529bc6225b28c3d9c3863964b00ac65fd8030 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc) 2024-05-09T09:27:33.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:32 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:32.925+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:33.376 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:33 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:33.126+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:33.377 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:33 smithi064 podman[111212]: 2024-05-09 09:27:33.131701366 +0000 UTC m=+0.018656622 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:33.377 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:33 smithi064 podman[111212]: 2024-05-09 09:27:33.249347248 +0000 UTC m=+0.136302524 container create 33e9c299c91ba80a41c8b2e34bfcd28930f982bd656bfd33a1ea94259c5f41ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T09:27:33.479 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:33.013+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:33.480 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:33.036+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:33.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:33.178+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:33.653 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:33 smithi064 podman[111212]: 2024-05-09 09:27:33.485528028 +0000 UTC m=+0.372483264 container init 33e9c299c91ba80a41c8b2e34bfcd28930f982bd656bfd33a1ea94259c5f41ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T09:27:33.654 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:33 smithi064 podman[111212]: 2024-05-09 09:27:33.489066006 +0000 UTC m=+0.376021242 container start 33e9c299c91ba80a41c8b2e34bfcd28930f982bd656bfd33a1ea94259c5f41ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T09:27:33.654 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:33 smithi064 podman[111212]: 2024-05-09 09:27:33.585459512 +0000 UTC m=+0.472414766 container attach 33e9c299c91ba80a41c8b2e34bfcd28930f982bd656bfd33a1ea94259c5f41ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T09:27:34.089 INFO:journalctl@ceph.osd.3.smithi064.stdout:May 09 09:27:33 smithi064 podman[111212]: 2024-05-09 09:27:33.794056975 +0000 UTC m=+0.681012210 container died 33e9c299c91ba80a41c8b2e34bfcd28930f982bd656bfd33a1ea94259c5f41ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:27:34.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:33 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:33.957+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:34.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:34 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:34.089+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:34.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.3.service' 2024-05-09T09:27:34.479 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:34.005+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:34.480 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:34.040+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:34.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:34.215+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:34.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:34.508 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T09:27:34.508 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T09:27:34.508 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.4 2024-05-09T09:27:34.823 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:34 smithi139 systemd[1]: Stopping Ceph osd.4 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:35.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:34 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:34.998+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:35.324 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:34 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:27:34.827+0000 7f5685ea9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:35.324 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:34 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:27:34.827+0000 7f5685ea9700 -1 osd.4 98 *** Got signal Terminated *** 2024-05-09T09:27:35.324 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:34 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:27:34.827+0000 7f5685ea9700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:35.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:35 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:35.083+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:35.479 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:35.003+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:35.480 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:35.017+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:35.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:35.227+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:36.260 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:36.052+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:36.261 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:36.003+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:36.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:35 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:35.987+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:36.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:35 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:35.943+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:36.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:36 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:36.080+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:36.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:36 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:36.080+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:36.632 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:36.260+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:37.279 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:37 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:37.009+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:37.279 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:37 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:37.057+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:37.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:36 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:36.897+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:37.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:36 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:36.974+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:37.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:37 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:37.124+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:37.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:37 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:37.124+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:37.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:37 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:37.279+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:37.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:37 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:37.279+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:38.291 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:37 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:37.991+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:38.292 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:38 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:38.066+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:38.292 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:38 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:38.066+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:38.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:37 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:37.880+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:38.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:37 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:37.936+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:38.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:37 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:37.936+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:38.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:38 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:38.166+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:38.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:38 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:38.166+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:38.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:38 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:38.291+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:38.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:38 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:38.291+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:38.903 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:38 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:38.800+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:38.926 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:38 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:27:38.665+0000 7f568249d700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.727530+0000 front 2024-05-09T09:27:13.727570+0000 (oldest deadline 2024-05-09T09:27:38.427747+0000) 2024-05-09T09:27:39.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:38 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:38.927+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:39.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:38 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:38.946+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:39.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:38 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:38.946+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:39.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:39 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:39.022+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:39.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:39 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:39.076+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:39.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:39 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:39.076+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:39.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:39 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:39.267+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:39.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:39 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:39.267+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:39.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:39 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:39.193+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:39.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:39 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:39.193+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:39.941 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:39 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4[45123]: 2024-05-09T09:27:39.617+0000 7f568249d700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.727530+0000 front 2024-05-09T09:27:13.727570+0000 (oldest deadline 2024-05-09T09:27:38.427747+0000) 2024-05-09T09:27:40.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:39 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:39.824+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:40.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:39 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:39.942+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:40.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:39 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:39.976+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:40.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:39 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:39.976+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:40.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:39 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:39.991+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:40.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:40 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:40.085+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:40.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:40 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:40.085+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:40.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:40 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:40.234+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:40.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:40 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:40.234+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:40.654 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:40 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:40.238+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:40.654 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:40 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:40.238+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:40.701 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:40 smithi139 podman[70479]: 2024-05-09 09:27:40.416159138 +0000 UTC m=+5.712226721 container died 74427bc885621090e594d9816ca315141da5ec389511753df97e41f41ee45975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:27:40.994 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:40 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:40.899+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:40.994 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:40 smithi139 podman[70479]: 2024-05-09 09:27:40.802510597 +0000 UTC m=+6.098578188 container cleanup 74427bc885621090e594d9816ca315141da5ec389511753df97e41f41ee45975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T09:27:40.994 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:40 smithi139 bash[70479]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4 2024-05-09T09:27:41.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:40 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:40.826+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:41.323 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70496]: 2024-05-09 09:27:41.078958385 +0000 UTC m=+0.664203064 container remove 74427bc885621090e594d9816ca315141da5ec389511753df97e41f41ee45975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T09:27:41.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:40 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:40.994+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:41.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:40 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:40.994+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:41.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:41 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:41.034+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:41.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:41 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:41.074+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:41.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:41 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:41.074+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:41.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:41 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:41.201+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:41.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:41 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:41.201+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:41.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:41 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:41.230+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:41.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:41 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:41.230+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:41.690 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70587]: 2024-05-09 09:27:41.314665894 +0000 UTC m=+0.017956291 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:41.690 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70587]: 2024-05-09 09:27:41.438357475 +0000 UTC m=+0.141647915 container create 0c92712949485fd3ee4f248233de1af40674f80aba2d02188ab81a3d7f136896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc) 2024-05-09T09:27:41.690 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70587]: 2024-05-09 09:27:41.690979325 +0000 UTC m=+0.394269732 container init 0c92712949485fd3ee4f248233de1af40674f80aba2d02188ab81a3d7f136896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, ceph=True) 2024-05-09T09:27:41.949 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:41 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:41.908+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:41.949 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70587]: 2024-05-09 09:27:41.695134995 +0000 UTC m=+0.398425402 container start 0c92712949485fd3ee4f248233de1af40674f80aba2d02188ab81a3d7f136896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-09T09:27:41.949 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70587]: 2024-05-09 09:27:41.773911684 +0000 UTC m=+0.477202103 container attach 0c92712949485fd3ee4f248233de1af40674f80aba2d02188ab81a3d7f136896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T09:27:42.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:41 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:41.806+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:42.323 INFO:journalctl@ceph.osd.4.smithi139.stdout:May 09 09:27:41 smithi139 podman[70587]: 2024-05-09 09:27:41.988155644 +0000 UTC m=+0.691446045 container died 0c92712949485fd3ee4f248233de1af40674f80aba2d02188ab81a3d7f136896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T09:27:42.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:41 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:41.949+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:42.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:41 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:41.949+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:42.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:42 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:42.024+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:42.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:42 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:42.113+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:42.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:42 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:42.113+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:42.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:42 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:42.220+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:42.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:42 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:42.220+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:42.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:42 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:42.223+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:42.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:42 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:42.223+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:42.697 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.4.service' 2024-05-09T09:27:42.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:42.737 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T09:27:42.737 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T09:27:42.737 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.5 2024-05-09T09:27:42.932 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:42 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:42.901+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:42.932 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:42 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:42.901+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:43.060 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:42 smithi179 systemd[1]: Stopping Ceph osd.5 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:43.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:42 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:42.853+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:43.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:42 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:42.931+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:43.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:43.060+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:43.197+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:43.197+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:43.082+0000 7f2e329f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:43.082+0000 7f2e329f9700 -1 osd.5 98 *** Got signal Terminated *** 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:43.082+0000 7f2e329f9700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:43.113+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:43.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:43 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:43.113+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:43.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:43 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:43.174+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:43.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:43 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:43.174+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:44.145 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:43 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:43.883+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:44.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:43 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:43.912+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:44.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:43 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:43.912+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:44.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:43 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:43.902+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:44.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:43 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:43.902+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:44.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:44 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:44.146+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:44.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:44 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:44.146+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:44.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:44.012+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:44.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:44.122+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:44.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:44.122+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:44.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:44.152+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:44.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:44.152+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:44.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:44.152+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:45.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:44 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:44.884+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:45.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:44 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:44.884+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:45.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:44 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:44.915+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:45.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:44 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:44.915+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:45.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:44 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:44.931+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:45.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:44 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:44.931+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:45.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:45 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:45.101+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:45.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:45 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:45.101+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:45.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:44 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:44.995+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:45.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:45 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:45.123+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:45.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:45 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:45.123+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:45.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:45 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:45.195+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:45.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:45 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:45.195+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:45.480 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:45 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:45.195+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:46.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:45 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:45.971+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:46.230 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:46.170+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:46.230 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:46.170+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:46.326 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:45 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:45.878+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:46.326 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:45 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:45.878+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:46.326 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:45 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:45.889+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:46.326 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:45 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:45.889+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:46.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:45 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:45.971+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:46.404 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:45 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:45.971+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:46.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:46 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:46.151+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:46.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:46 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:46.151+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:46.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:46 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:46.151+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:46.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:46.243+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:46.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:46.243+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:46.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:46.243+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:47.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:46 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:46.943+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:47.230 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:47.123+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.073728+0000 front 2024-05-09T09:27:03.073830+0000 (oldest deadline 2024-05-09T09:27:28.973723+0000) 2024-05-09T09:27:47.230 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:47.123+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.274956+0000 front 2024-05-09T09:27:11.274881+0000 (oldest deadline 2024-05-09T09:27:37.174771+0000) 2024-05-09T09:27:47.230 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5[45699]: 2024-05-09T09:27:47.123+0000 7f2e2afe5700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.875755+0000 front 2024-05-09T09:27:21.875786+0000 (oldest deadline 2024-05-09T09:27:46.576012+0000) 2024-05-09T09:27:47.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:46.910+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:47.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:46.910+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:47.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:46.843+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:47.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:46 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:46.843+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:47.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:46 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:46.936+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:47.404 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:46 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:46.936+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:47.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:47 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:47.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:47.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:47 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:47.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:47.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:47 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:47.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:47.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:47.290+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:47.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:47.290+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:47.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:47.290+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:48.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:47.940+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:48.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:47 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:47.940+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:48.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:47.942+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:48.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:47.942+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:48.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:47.876+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:48.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:47.876+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:48.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:47 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:47.876+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:48.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:47 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:47.925+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:48.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:47 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:47.925+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:48.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:48.126+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:48.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:48.126+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:48.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:48.126+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:48.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:48.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:48.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:48.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:48.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:48.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:49.082 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:48 smithi179 podman[70727]: 2024-05-09 09:27:48.779112001 +0000 UTC m=+5.837928208 container died 29232c80157f194914b3e199629e24049c4f887c20502a5ba044a279a07ea17a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:49.082 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:48.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:49.082 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:48 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:48.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:49.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:48.972+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:49.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:48.972+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:49.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:48.867+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:49.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:48.867+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:49.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:48 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:48.867+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:49.399 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:49 smithi179 podman[70727]: 2024-05-09 09:27:49.198887526 +0000 UTC m=+6.257703732 container cleanup 29232c80157f194914b3e199629e24049c4f887c20502a5ba044a279a07ea17a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.build-date=20240507) 2024-05-09T09:27:49.399 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:49 smithi179 bash[70727]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5 2024-05-09T09:27:49.399 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:49.266+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:49.400 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:49.266+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:49.400 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:49.266+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:49.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:48.966+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:49.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:48 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:48.966+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:49.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:49.124+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:49.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:49.124+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:49.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:49.124+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:49.729 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:49 smithi179 podman[70747]: 2024-05-09 09:27:49.400431876 +0000 UTC m=+0.623200044 container remove 29232c80157f194914b3e199629e24049c4f887c20502a5ba044a279a07ea17a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T09:27:50.061 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:49.908+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:50.061 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:49 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:49.908+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:50.061 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:49 smithi179 podman[70837]: 2024-05-09 09:27:49.703394325 +0000 UTC m=+0.018433565 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:50.062 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:49 smithi179 podman[70837]: 2024-05-09 09:27:49.809797583 +0000 UTC m=+0.124836851 container create 0c735958d14b4646b03bd0454722fc52967644990e9b1b80943422d97fb38ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T09:27:50.062 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:50 smithi179 podman[70837]: 2024-05-09 09:27:50.062519241 +0000 UTC m=+0.377558473 container init 0c735958d14b4646b03bd0454722fc52967644990e9b1b80943422d97fb38ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T09:27:50.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:49.969+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:50.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:49.969+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:50.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:49.893+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:50.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:49.893+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:50.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:49 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:49.893+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:50.357 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:50 smithi179 podman[70837]: 2024-05-09 09:27:50.065847781 +0000 UTC m=+0.380887013 container start 0c735958d14b4646b03bd0454722fc52967644990e9b1b80943422d97fb38ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:50.357 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:50 smithi179 podman[70837]: 2024-05-09 09:27:50.162573993 +0000 UTC m=+0.477613243 container attach 0c735958d14b4646b03bd0454722fc52967644990e9b1b80943422d97fb38ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T09:27:50.357 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:50.303+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:50.357 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:50.303+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:50.357 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:50.303+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:50.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:49.922+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:50.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:49.922+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:50.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:50.152+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:50.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:50.152+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:50.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:50.152+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:50.656 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 09 09:27:50 smithi179 podman[70837]: 2024-05-09 09:27:50.358440475 +0000 UTC m=+0.673479711 container died 0c735958d14b4646b03bd0454722fc52967644990e9b1b80943422d97fb38ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T09:27:51.007 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.5.service' 2024-05-09T09:27:51.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:51.048 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T09:27:51.048 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T09:27:51.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.6 2024-05-09T09:27:51.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:50.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:51.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:50 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:50.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:51.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:51.002+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:51.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:51.002+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:51.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:50 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:50.872+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:51.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:50 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:50.872+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:51.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:50 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:50.872+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:51.375 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:51.113+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:51.375 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:51.113+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:51.375 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:51.113+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:51.376 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:50.904+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:51.376 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:50 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:50.904+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:51.376 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:51 smithi064 systemd[1]: Stopping Ceph osd.6 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:51.376 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:51.374+0000 7fe897764700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:51.376 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:51.374+0000 7fe897764700 -1 osd.6 98 *** Got signal Terminated *** 2024-05-09T09:27:51.376 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:51.374+0000 7fe897764700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:27:51.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:51 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:51.348+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:51.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:51 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:51.348+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:51.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:51 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:51.348+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:52.073 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:51.966+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:52.073 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:51.966+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:52.073 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:51.823+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:52.074 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:51.823+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:52.074 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:51 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:51.823+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:52.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:51.893+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:52.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:51 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:51.893+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:52.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:52.111+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:52.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:52.111+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:52.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:52.111+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:52.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:51 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:51.908+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:52.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:51 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:51.908+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:52.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:52 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:52.338+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:52.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:52 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:52.338+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:52.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:52 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:52.338+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:53.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:52.902+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:53.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:52 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:52.902+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:53.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:53.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:53.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:53.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:53.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:53.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:53.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:52 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:52.885+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:53.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:52 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:52.885+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:53.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:52.979+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:53.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:52.979+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:53.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:52.827+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:53.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:52.827+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:53.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:52 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:52.827+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:53.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:53 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:53.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:53.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:53 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:53.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:53.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:53 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:53.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:54.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:53 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:53.857+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:54.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:53 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:53.857+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:54.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:53.954+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:54.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:53.954+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:54.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:53.954+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:27:54.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:53.875+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:54.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:53.875+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:54.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:53 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:53.875+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:54.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:53.906+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:54.403 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:53 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:53.906+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:54.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:54.069+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:54.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:54.069+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:54.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:54.069+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:54.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:54.069+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:27:54.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:54.331+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:54.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:54.331+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:54.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:54.331+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:55.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:54.873+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:55.154 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:54.873+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:55.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:55.046+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:55.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:55.046+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:55.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:55.046+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:55.155 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:55.046+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:27:55.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:54.844+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:55.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:54 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:54.844+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:55.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:54.940+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:55.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:54.940+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:55.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:54.940+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:27:55.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:54.882+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:55.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:54.882+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:55.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:54.882+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:55.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:54 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:54.882+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:27:55.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:55 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:55.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:55.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:55 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:55.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:55.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:55 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:55.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:56.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:55.841+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:13.754737+0000 front 2024-05-09T09:27:13.754975+0000 (oldest deadline 2024-05-09T09:27:38.454767+0000) 2024-05-09T09:27:56.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:55 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6[56163]: 2024-05-09T09:27:55.841+0000 7fe893d58700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:18.455461+0000 front 2024-05-09T09:27:18.455376+0000 (oldest deadline 2024-05-09T09:27:44.355437+0000) 2024-05-09T09:27:56.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:56 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:56.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:56.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:56 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:56.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:56.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:56 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:56.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:56.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:56 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:56.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:27:56.225 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:55.901+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:56.338 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:55.901+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:56.338 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:55.901+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:27:56.338 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:55.902+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:56.338 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:55.902+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:56.339 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:55.902+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:56.339 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:55 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:55.902+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:27:56.339 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:55 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:55.800+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:56.339 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:55 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:55.800+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:56.507 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:56.302+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:56.507 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:56.302+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:56.507 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:56.302+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:56.507 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:56.302+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:27:57.153 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:56 smithi064 podman[111319]: 2024-05-09 09:27:56.880781317 +0000 UTC m=+5.631571885 container died e6d27b07303dff36a272a51edfe076457a6a1cd4c777bf3c3805cf33e696eea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T09:27:57.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:57 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:57.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:57.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:57 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:57.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:57.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:57 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:57.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:57.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:57 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:57.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:27:57.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:56.790+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:57.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:56 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:56.790+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:57.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:56.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:57.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:56.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:57.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:56.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:57.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:56.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:27:57.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:56.914+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:57.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:56.914+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:57.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:56 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:56.914+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:27:57.551 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:57 smithi064 podman[111319]: 2024-05-09 09:27:57.292212227 +0000 UTC m=+6.043002788 container cleanup e6d27b07303dff36a272a51edfe076457a6a1cd4c777bf3c3805cf33e696eea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T09:27:57.551 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:57 smithi064 bash[111319]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6 2024-05-09T09:27:57.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:57.310+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:57.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:57.310+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:57.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:57.310+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:57.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:57.310+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:27:57.903 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:57 smithi064 podman[111349]: 2024-05-09 09:27:57.552207715 +0000 UTC m=+0.674244883 container remove e6d27b07303dff36a272a51edfe076457a6a1cd4c777bf3c3805cf33e696eea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T09:27:58.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:57.836+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:58.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:57.836+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:58.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:57 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:57.836+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:27:58.287 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:58 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:58.036+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:58 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:58.036+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:58 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:58.036+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:58 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:58.036+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:57 smithi064 podman[111442]: 2024-05-09 09:27:57.878184633 +0000 UTC m=+0.018629831 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:57 smithi064 podman[111442]: 2024-05-09 09:27:57.986497908 +0000 UTC m=+0.126943128 container create 38d04a7c8a41ccaec83167ce07b49211ee960aa91e82c513bc2a2967df1c5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:58 smithi064 podman[111442]: 2024-05-09 09:27:58.20542488 +0000 UTC m=+0.345870074 container init 38d04a7c8a41ccaec83167ce07b49211ee960aa91e82c513bc2a2967df1c5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:58 smithi064 podman[111442]: 2024-05-09 09:27:58.209849472 +0000 UTC m=+0.350294658 container start 38d04a7c8a41ccaec83167ce07b49211ee960aa91e82c513bc2a2967df1c5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-96805dc) 2024-05-09T09:27:58.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:58 smithi064 podman[111442]: 2024-05-09 09:27:58.288726754 +0000 UTC m=+0.429171933 container attach 38d04a7c8a41ccaec83167ce07b49211ee960aa91e82c513bc2a2967df1c5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:27:58.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:57.879+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:58.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:57.879+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:58.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:57.879+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:58.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:57.879+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:27:58.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:57.911+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:58.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:57.911+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:58.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:57 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:57.911+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:27:58.653 INFO:journalctl@ceph.osd.6.smithi064.stdout:May 09 09:27:58 smithi064 podman[111442]: 2024-05-09 09:27:58.518112407 +0000 UTC m=+0.658557600 container died 38d04a7c8a41ccaec83167ce07b49211ee960aa91e82c513bc2a2967df1c5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T09:27:58.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:58.337+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:58.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:58.337+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:58.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:58.337+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:58.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:58.337+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:27:59.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.6.service' 2024-05-09T09:27:59.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:27:59.156 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T09:27:59.156 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T09:27:59.156 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.7 2024-05-09T09:27:59.185 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:58.900+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:27:59.185 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:58.900+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:27:59.185 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:58.900+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:27:59.186 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:58.910+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:27:59.186 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:58.910+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:27:59.186 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:58.910+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:27:59.186 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:58.910+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:27:59.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:58.885+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:27:59.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:58.885+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:27:59.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:58 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:58.885+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:27:59.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:59 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:59.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:27:59.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:59 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:59.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:27:59.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:59 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:59.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:27:59.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:27:59 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:27:59.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:27:59.470 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 systemd[1]: Stopping Ceph osd.7 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:27:59.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:59.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:27:59.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:59.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:27:59.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:59.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:27:59.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:27:59.293+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:27:59.823 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:59.470+0000 7f19013c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:27:59.823 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:59.470+0000 7f19013c9700 -1 osd.7 98 *** Got signal Terminated *** 2024-05-09T09:27:59.823 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:59.470+0000 7f19013c9700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:28:00.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:59.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:00.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:59.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:00.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:27:59 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:27:59.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:00.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:59.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:00.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:59.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:00.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:59.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:00.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:27:59.912+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:00.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:59.916+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:28:00.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:59.916+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:28:00.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:27:59 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:27:59.916+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:28:00.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:00.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:00.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:00.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:00.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:00.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:00.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:00 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:00.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:00.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:00.311+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:00.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:00.311+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:00.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:00.311+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:00.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:00.311+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:01.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:00.894+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:01.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:00.894+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:01.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:00 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:00.894+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:01.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:00.963+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:00.963+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:00.963+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:00.963+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.600319+0000 front 2024-05-09T09:27:39.600354+0000 (oldest deadline 2024-05-09T09:28:00.700463+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:00.949+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:00.949+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:00.949+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:01.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:00.949+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:01.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:01.019+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:01.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:01.019+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:01.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:01.019+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:01.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:01 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:01.019+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:01.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:01.322+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:01.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:01.322+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:01.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:01.322+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:01.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:01.322+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:02.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:01.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:02.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:01.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:02.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:01.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:02.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:01 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:01.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:02.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:01.931+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:28:02.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:01.931+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:01.931+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:01.931+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.600319+0000 front 2024-05-09T09:27:39.600354+0000 (oldest deadline 2024-05-09T09:28:00.700463+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:01.972+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:01.972+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:01.972+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:01.972+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:02.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:01 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:01.972+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:02.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:02 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:02.004+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:02.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:02 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:02.004+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:02.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:02 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:02.004+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:02.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:02 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:02.004+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:02.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:02.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:02.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:02.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:02.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:02.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:02.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:02.300+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:03.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:02.950+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:03.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:02.950+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:03.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:02.950+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:03.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:02 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:02.950+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:03.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:02 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:02.914+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:28:03.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:02 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:02.914+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:28:03.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:02 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:02.914+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:28:03.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:02 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:02.915+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.600319+0000 front 2024-05-09T09:27:39.600354+0000 (oldest deadline 2024-05-09T09:28:00.700463+0000) 2024-05-09T09:28:03.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:03.017+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:03.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:03.017+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:03.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:03.017+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:03.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:03.017+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:03.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:03.017+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:03.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:03 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:03.024+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:03.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:03 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:03.024+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:03.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:03 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:03.024+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:03.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:03 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:03.024+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:03.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:03.319+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:03.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:03.319+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:03.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:03.319+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:03.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:03.319+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:04.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:03.971+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:04.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:03.971+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:04.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:03.971+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:04.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:03 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:03.971+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:04.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:03.926+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.195878+0000 front 2024-05-09T09:27:12.196032+0000 (oldest deadline 2024-05-09T09:27:35.096213+0000) 2024-05-09T09:28:04.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:03.926+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:22.098022+0000 front 2024-05-09T09:27:22.097809+0000 (oldest deadline 2024-05-09T09:27:43.797882+0000) 2024-05-09T09:28:04.323 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:03.926+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:27.898889+0000 front 2024-05-09T09:27:27.898913+0000 (oldest deadline 2024-05-09T09:27:53.199011+0000) 2024-05-09T09:28:04.324 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:03 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7[52045]: 2024-05-09T09:28:03.926+0000 7f18fd9bd700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.600319+0000 front 2024-05-09T09:27:39.600354+0000 (oldest deadline 2024-05-09T09:28:00.700463+0000) 2024-05-09T09:28:04.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:04.043+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:04.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:04.043+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:04.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:04.043+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:04.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:04.043+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:04.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:04.043+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:04.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:04.054+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:04.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:04.054+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:04.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:04.054+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:04.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:04 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:04.054+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:04.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:04 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:04.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:04.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:04 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:04.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:04.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:04 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:04.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:04.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:04 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:04.317+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:05.340 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.018+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:05.340 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.018+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:05.340 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.018+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:05.340 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.018+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:05.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:05.020+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:05.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:05.020+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:05.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:05.020+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:05.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:05.020+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:05.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:05 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:05.020+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:05.472 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:05 smithi139 podman[70704]: 2024-05-09 09:28:05.186080129 +0000 UTC m=+5.832942620 container died db2fb58726df8ed78afb41dd84998ed5d074febe04c5eb7e20046d3945884485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T09:28:05.472 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:05 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:05.078+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:05.472 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:05 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:05.078+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:05.472 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:05 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:05.078+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:05.472 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:05 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:05.078+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:05.472 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:05 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:05.078+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:05.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:05.340+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:05.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:05.340+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:05.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:05.340+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:05.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:05.340+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:05.823 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:05 smithi139 podman[70704]: 2024-05-09 09:28:05.537252928 +0000 UTC m=+6.184115411 container cleanup db2fb58726df8ed78afb41dd84998ed5d074febe04c5eb7e20046d3945884485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T09:28:05.823 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:05 smithi139 bash[70704]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7 2024-05-09T09:28:06.175 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:05 smithi139 podman[70721]: 2024-05-09 09:28:05.846813968 +0000 UTC m=+0.658265515 container remove db2fb58726df8ed78afb41dd84998ed5d074febe04c5eb7e20046d3945884485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T09:28:06.175 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:06 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:06.042+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:06.175 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:06 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:06.042+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:06.175 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:06 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:06.042+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:06.175 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:06 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:06.042+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:06.176 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:06 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:06.042+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:06.374 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.997+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:06.374 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.997+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:06.374 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.997+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:06.374 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:05 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:05.997+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:06.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:06.049+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:06.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:06.049+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:06.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:06.049+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:06.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:06.049+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:06.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:06 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:06.049+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:06.449 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:06 smithi139 podman[70811]: 2024-05-09 09:28:06.079336271 +0000 UTC m=+0.018241270 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:28:06.449 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:06 smithi139 podman[70811]: 2024-05-09 09:28:06.214407859 +0000 UTC m=+0.153312844 container create f5738e0770a83c64c92f662a9014e1ee0a968baf788ac9bc5bdfc4308963cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:28:06.449 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:06 smithi139 podman[70811]: 2024-05-09 09:28:06.449970746 +0000 UTC m=+0.388875731 container init f5738e0770a83c64c92f662a9014e1ee0a968baf788ac9bc5bdfc4308963cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T09:28:06.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:06.375+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:06.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:06.375+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:06.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:06.375+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:06.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:06.375+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:06.747 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:06 smithi139 podman[70811]: 2024-05-09 09:28:06.453539334 +0000 UTC m=+0.392444327 container start f5738e0770a83c64c92f662a9014e1ee0a968baf788ac9bc5bdfc4308963cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc) 2024-05-09T09:28:06.747 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:06 smithi139 podman[70811]: 2024-05-09 09:28:06.533229594 +0000 UTC m=+0.472134596 container attach f5738e0770a83c64c92f662a9014e1ee0a968baf788ac9bc5bdfc4308963cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T09:28:07.035 INFO:journalctl@ceph.osd.7.smithi139.stdout:May 09 09:28:06 smithi139 podman[70811]: 2024-05-09 09:28:06.74784255 +0000 UTC m=+0.686747538 container died f5738e0770a83c64c92f662a9014e1ee0a968baf788ac9bc5bdfc4308963cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:28:07.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:07.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:07.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:07.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:07.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:07.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:07.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:07.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:07.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:07 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:07.037+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:07.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:06.951+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:07.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:06.951+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:07.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:06.951+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:07.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:06 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:06.951+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:07.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:07 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:07.091+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:07.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:07 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:07.091+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:07.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:07 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:07.091+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:07.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:07 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:07.091+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:07.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:07 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:07.091+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:07.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.7.service' 2024-05-09T09:28:07.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:07.460 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T09:28:07.460 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T09:28:07.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.8 2024-05-09T09:28:07.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.376+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:07.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.376+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:07.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.376+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:07.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.376+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:07.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 systemd[1]: Stopping Ceph osd.8 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.784+0000 7f3dce6db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.784+0000 7f3dce6db700 -1 osd.8 98 *** Got signal Terminated *** 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:07.784+0000 7f3dce6db700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:07.940+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:07.940+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:07.940+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:08.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:07 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:07.940+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:08.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:08.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:08.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:08.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:08.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:08.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:08.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:08.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:08.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:08 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:08.064+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:08.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:08.086+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:08.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:08.086+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:08.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:08.086+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:08.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:08.086+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:08.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:08.086+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:08.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:08.349+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:08.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:08.349+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:08.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:08.349+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:08.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:08.349+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:09.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:08.924+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:09.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:08.924+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:09.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:08.924+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:09.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:08 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:08.924+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:09.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:09 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:09.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:09.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:09 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:09.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:09.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:09 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:09.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:09.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:09 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:09.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:09.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:09 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:09.084+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:09.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:09.126+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:09.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:09.126+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:09.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:09.126+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:09.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:09.126+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:09.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:09 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:09.126+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:09.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:09.352+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:09.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:09.352+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:09.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:09.352+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:09.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:09.352+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:10.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:09.886+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:10.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:09.886+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:10.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:09.886+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:10.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:09 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:09.886+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:10.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:10 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:10.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:10.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:10 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:10.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:10.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:10 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:10.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:10.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:10 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:10.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:10.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:10 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:10.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:10.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:10 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:10.104+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:10.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:10.127+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:10.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:10.127+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:10.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:10.127+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:10.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:10.127+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:10.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:10 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:10.127+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:10.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:10.333+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:10.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:10.333+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:10.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:10.333+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:10.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:10.333+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:11.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:10.864+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:11.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:10.864+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:11.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:10.864+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:11.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:10 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:10.864+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:11.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:11 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:11.109+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:11.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:11 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:11.109+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:11.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:11 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:11.109+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:11.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:11 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:11.109+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:11.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:11 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:11.109+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:11.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:11 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:11.109+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:11.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:11 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:11.081+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:11.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:11 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:11.081+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:11.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:11 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:11.081+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:11.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:11 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:11.081+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:11.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:11 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:11.081+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:11.729 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:11.299+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:11.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:11.299+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:11.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:11.299+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:11.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:11.299+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:12.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:11.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:12.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:11.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:12.231 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:11.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:12.231 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:11.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:12.231 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:11.861+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:12.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:12 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:12.081+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:12.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:12 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:12.081+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:12.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:12 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:12.081+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:12.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:12 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:12.081+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:12.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:12 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:12.081+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:12.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:12 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:12.081+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:12.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:12 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:12.112+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:12.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:12 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:12.112+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:12.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:12 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:12.112+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:12.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:12 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:12.112+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:12.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:12 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:12.112+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:12.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:12 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:12.112+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:12.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:12.269+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.348376+0000 front 2024-05-09T09:27:03.348237+0000 (oldest deadline 2024-05-09T09:27:28.648406+0000) 2024-05-09T09:28:12.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:12.269+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:12.050381+0000 front 2024-05-09T09:27:12.050210+0000 (oldest deadline 2024-05-09T09:27:36.750213+0000) 2024-05-09T09:28:12.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:12.269+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.951990+0000 front 2024-05-09T09:27:21.951903+0000 (oldest deadline 2024-05-09T09:27:43.651963+0000) 2024-05-09T09:28:12.730 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8[52546]: 2024-05-09T09:28:12.269+0000 7f3dc6cc7700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.453497+0000 front 2024-05-09T09:27:29.453441+0000 (oldest deadline 2024-05-09T09:27:55.353319+0000) 2024-05-09T09:28:13.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:12.902+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:13.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:12.902+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:13.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:12.902+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:13.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:12.902+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:13.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:12 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:12.902+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:13.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:13.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:13.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:13.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:13.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:13.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:13.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:13.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:13.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:13.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:13.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:13 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:13.039+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:13.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:13 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:13.101+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:13.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:13 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:13.101+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:13.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:13 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:13.101+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:13.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:13 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:13.101+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:13.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:13 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:13.101+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:13.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:13 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:13.101+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:13.627 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:13 smithi179 podman[70954]: 2024-05-09 09:28:13.322630292 +0000 UTC m=+5.665793532 container died 3766cdb5154c0d16c824d8d917807a6f0010abca4854e1ca9528e8c406fa830c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T09:28:13.903 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:13 smithi179 podman[70954]: 2024-05-09 09:28:13.727282883 +0000 UTC m=+6.070446125 container cleanup 3766cdb5154c0d16c824d8d917807a6f0010abca4854e1ca9528e8c406fa830c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True) 2024-05-09T09:28:13.903 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:13 smithi179 bash[70954]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8 2024-05-09T09:28:14.229 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[70971]: 2024-05-09 09:28:14.145493196 +0000 UTC m=+0.821736279 container remove 3766cdb5154c0d16c824d8d917807a6f0010abca4854e1ca9528e8c406fa830c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:28:14.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:13 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:13.902+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:14.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:13 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:13.903+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:14.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:13 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:13.903+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:14.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:13 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:13.903+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:14.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:13 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:13.903+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:14.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:14 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:14.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:14.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:14 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:14.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:14.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:14 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:14.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:14.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:14 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:14.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:14.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:14 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:14.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:14.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:14 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:14.032+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:14.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:14.132+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:14.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:14.132+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:14.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:14.132+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:14.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:14.132+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:14.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:14.132+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:14.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:14 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:14.132+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:14.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[71063]: 2024-05-09 09:28:14.429661126 +0000 UTC m=+0.099099468 container create 001c8a1d97a790bf44b9d40bca8cc07b693091b9c91dcd696bf60a4050d14b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T09:28:14.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[71063]: 2024-05-09 09:28:14.348480161 +0000 UTC m=+0.017918516 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:28:14.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[71063]: 2024-05-09 09:28:14.63226292 +0000 UTC m=+0.301701261 container init 001c8a1d97a790bf44b9d40bca8cc07b693091b9c91dcd696bf60a4050d14b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T09:28:14.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[71063]: 2024-05-09 09:28:14.635594177 +0000 UTC m=+0.305032524 container start 001c8a1d97a790bf44b9d40bca8cc07b693091b9c91dcd696bf60a4050d14b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T09:28:14.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[71063]: 2024-05-09 09:28:14.723882867 +0000 UTC m=+0.393321218 container attach 001c8a1d97a790bf44b9d40bca8cc07b693091b9c91dcd696bf60a4050d14b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T09:28:14.980 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 09 09:28:14 smithi179 podman[71063]: 2024-05-09 09:28:14.929036067 +0000 UTC m=+0.598474413 container died 001c8a1d97a790bf44b9d40bca8cc07b693091b9c91dcd696bf60a4050d14b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:28:14.980 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:14 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:14.876+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:14.980 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:14 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:14.876+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:14.981 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:14 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:14.876+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:14.981 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:14 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:14.876+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:14.981 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:14 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:14.876+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:15.408 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.015+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:15.408 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.016+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:15.408 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.016+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:15.408 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.016+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:15.409 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.016+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:15.409 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.016+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:15.565 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.8.service' 2024-05-09T09:28:15.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:15 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:15.113+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:15.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:15 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:15.113+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:15.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:15 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:15.113+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:15.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:15 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:15.113+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:15.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:15 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:15.113+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:15.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:15 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:15.113+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:15.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:15.605 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T09:28:15.605 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T09:28:15.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.9 2024-05-09T09:28:15.902 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 systemd[1]: Stopping Ceph osd.9 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:16.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.903+0000 7f40c48c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:16.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.903+0000 7f40c48c2700 -1 osd.9 98 *** Got signal Terminated *** 2024-05-09T09:28:16.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.903+0000 7f40c48c2700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:28:16.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.970+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:16.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.970+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:16.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.970+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:16.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.970+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:16.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.970+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:16.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:15 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:15.970+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:16.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:15 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:15.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:16.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:15 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:15.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:16.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:15 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:15.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:16.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:15 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:15.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:16.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:15 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:15.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:16.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:16 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:16.095+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:16.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:16 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:16.095+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:16.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:16 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:16.095+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:16.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:16 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:16.095+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:16.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:16 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:16.095+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:16.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:16 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:16.095+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:17.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:16.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:17.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:16.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:17.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:16.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:17.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:16.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:17.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:16.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:17.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:16 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:16.929+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:17.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:16 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:16.929+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:17.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:16 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:16.929+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:17.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:16 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:16.929+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:17.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:16 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:16.929+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:17.404 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:16 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:16.929+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:17.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:17 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:17.102+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:17.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:17 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:17.102+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:17.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:17 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:17.102+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:17.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:17 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:17.102+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:17.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:17 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:17.102+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:17.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:17 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:17.102+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:18.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:17.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:18.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:17.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:18.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:17.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:18.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:17.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:18.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:17 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:17.942+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:18.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:17 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:17.903+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:18.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:17 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:17.903+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:18.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:17 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:17.903+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:18.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:17 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:17.903+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:18.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:17 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:17.903+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:18.403 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:17 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:17.903+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:18.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:18 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:18.099+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:18.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:18 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:18.099+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:18.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:18 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:18.099+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:18.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:18 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:18.099+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:18.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:18 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:18.099+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:18.575 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:18 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:18.099+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:19.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:18 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:18.885+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:19.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:18 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:18.885+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:19.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:18 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:18.885+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:19.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:18 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:18.885+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:19.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:18 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:18.885+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:19.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:18 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:18.885+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:19.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:18 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:18.962+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:19.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:18 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:18.962+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:19.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:18 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:18.962+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:19.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:18 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:18.962+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:19.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:18 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:18.962+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:19.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:19.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:19.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:19.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:19.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:19.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:19.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:19 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:19.061+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:20.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:19 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:19.868+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:02.788925+0000 front 2024-05-09T09:27:02.788982+0000 (oldest deadline 2024-05-09T09:27:27.489135+0000) 2024-05-09T09:28:20.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:19 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:19.868+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.590183+0000 front 2024-05-09T09:27:11.590147+0000 (oldest deadline 2024-05-09T09:27:35.690131+0000) 2024-05-09T09:28:20.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:19 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:19.868+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:20.991282+0000 front 2024-05-09T09:27:20.991462+0000 (oldest deadline 2024-05-09T09:27:45.691391+0000) 2024-05-09T09:28:20.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:19 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:19.868+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:29.792242+0000 front 2024-05-09T09:27:29.792275+0000 (oldest deadline 2024-05-09T09:27:53.892330+0000) 2024-05-09T09:28:20.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:19 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:19.868+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:38.593364+0000 front 2024-05-09T09:27:38.593469+0000 (oldest deadline 2024-05-09T09:28:04.493115+0000) 2024-05-09T09:28:20.154 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:19 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9[63181]: 2024-05-09T09:28:19.868+0000 7f40bceae700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:44.493578+0000 front 2024-05-09T09:27:44.493615+0000 (oldest deadline 2024-05-09T09:28:09.793525+0000) 2024-05-09T09:28:20.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:19 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:19.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:20.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:19 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:19.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:20.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:19 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:19.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:20.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:19 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:19.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:20.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:19 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:19.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:20.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:19 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:19.927+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:20.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:20.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:20.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:20.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:20.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:20.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:20.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:20 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:20.041+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:21.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:20.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:21.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:20.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:21.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:20.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:21.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:20.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:21.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:20.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:21.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:20.892+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:21.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:21.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:21.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:21.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:21.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:21.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:21.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:21 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:21.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:21.795 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:21 smithi064 podman[111550]: 2024-05-09 09:28:21.488085509 +0000 UTC m=+5.681826599 container died f6c8d63bf25b94a068ef66fb19f2e101ad5416ab156409b33d811e7f03cbc76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T09:28:22.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111550]: 2024-05-09 09:28:22.011653628 +0000 UTC m=+6.205394717 container cleanup f6c8d63bf25b94a068ef66fb19f2e101ad5416ab156409b33d811e7f03cbc76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T09:28:22.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 bash[111550]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9 2024-05-09T09:28:22.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:21 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:21.921+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:22.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:21 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:21.921+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:22.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:21 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:21.921+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:22.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:21 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:21.921+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:22.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:21 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:21.921+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:22.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:21 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:21.921+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:22.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:22.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:22.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:22.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:22.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:22.325 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:22.325 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:22 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:22.035+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:22.626 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111569]: 2024-05-09 09:28:22.221160413 +0000 UTC m=+0.733775048 container remove f6c8d63bf25b94a068ef66fb19f2e101ad5416ab156409b33d811e7f03cbc76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:28:22.877 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111663]: 2024-05-09 09:28:22.530678528 +0000 UTC m=+0.018327399 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:28:22.877 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111663]: 2024-05-09 09:28:22.655341264 +0000 UTC m=+0.142990175 container create afca9dabac66642efe8ecc89e25f394f63ec5137618ff2de8013f7212dbd8b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T09:28:22.877 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111663]: 2024-05-09 09:28:22.87479882 +0000 UTC m=+0.362447697 container init afca9dabac66642efe8ecc89e25f394f63ec5137618ff2de8013f7212dbd8b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_CLEAN=True) 2024-05-09T09:28:22.877 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111663]: 2024-05-09 09:28:22.878335945 +0000 UTC m=+0.365984817 container start afca9dabac66642efe8ecc89e25f394f63ec5137618ff2de8013f7212dbd8b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T09:28:23.153 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:22 smithi064 podman[111663]: 2024-05-09 09:28:22.966048067 +0000 UTC m=+0.453696970 container attach afca9dabac66642efe8ecc89e25f394f63ec5137618ff2de8013f7212dbd8b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T09:28:23.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:22 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:22.878+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:23.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:22 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:22.878+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:23.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:22 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:22.878+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:23.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:22 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:22.878+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:23.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:22 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:22.878+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:23.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:22 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:22.878+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:23.474 INFO:journalctl@ceph.osd.9.smithi064.stdout:May 09 09:28:23 smithi064 podman[111663]: 2024-05-09 09:28:23.182848403 +0000 UTC m=+0.670497264 container died afca9dabac66642efe8ecc89e25f394f63ec5137618ff2de8013f7212dbd8b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T09:28:23.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:23.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:23.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:23.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:23.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:23.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:23.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:23.074+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:23.843 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.9.service' 2024-05-09T09:28:23.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:23.884 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T09:28:23.885 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T09:28:23.885 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.10 2024-05-09T09:28:24.204 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:23 smithi139 systemd[1]: Stopping Ceph osd.10 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:24.204 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:24.204 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:24.204 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:24.205 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:24.205 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:24.205 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:24.205 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.045+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:24.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:23 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:23.866+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:24.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:23 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:23.866+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:24.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:23 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:23.866+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:24.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:23 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:23.866+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:24.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:23 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:23.866+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:24.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:23 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:23.866+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:24.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.204+0000 7fc1e48f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:24.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.204+0000 7fc1e48f4700 -1 osd.10 98 *** Got signal Terminated *** 2024-05-09T09:28:24.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:24 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:24.204+0000 7fc1e48f4700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:28:25.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:24.838+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:25.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:24.838+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:25.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:24.838+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:25.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:24.838+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:25.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:24.838+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:25.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:24.838+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:25.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:25.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:25.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:25.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:25.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:25.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:25.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:25 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:25.020+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:26.131 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:25 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:25.823+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:26.131 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:25 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:25.823+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:26.131 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:25 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:25.823+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:26.131 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:25 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:25.823+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:26.131 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:25 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:25.823+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:26.131 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:25 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:25.823+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:26.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:26.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:26.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:26.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:26.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:26.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:26.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:26 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:26.033+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:27.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:26.847+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:27.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:26.847+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:27.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:26.847+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:27.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:26.847+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:27.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:26.847+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:27.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:26 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:26.847+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:27.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:27.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:27.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:27.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:27.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:27.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:27.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:27 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:27.055+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:28.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:27 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:27.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:28.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:28.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:28.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:28.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:28.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:28.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:28.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:28 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:28.025+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:29.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:29.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:29.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:29.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:29.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:29.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:29.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:28 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:28.791+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:29.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:00.838397+0000 front 2024-05-09T09:27:00.838507+0000 (oldest deadline 2024-05-09T09:27:26.039241+0000) 2024-05-09T09:28:29.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6804 osd.1 since back 2024-05-09T09:27:11.940540+0000 front 2024-05-09T09:27:11.940472+0000 (oldest deadline 2024-05-09T09:27:37.240689+0000) 2024-05-09T09:28:29.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:17.241292+0000 front 2024-05-09T09:27:17.241349+0000 (oldest deadline 2024-05-09T09:27:47.841583+0000) 2024-05-09T09:28:29.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:30.142884+0000 front 2024-05-09T09:27:30.143006+0000 (oldest deadline 2024-05-09T09:27:54.842618+0000) 2024-05-09T09:28:29.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:39.245239+0000 front 2024-05-09T09:27:39.245256+0000 (oldest deadline 2024-05-09T09:28:01.545374+0000) 2024-05-09T09:28:29.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.746778+0000 front 2024-05-09T09:27:46.746887+0000 (oldest deadline 2024-05-09T09:28:12.046652+0000) 2024-05-09T09:28:29.324 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10[59074]: 2024-05-09T09:28:29.012+0000 7fc1e0ee8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:53.147762+0000 front 2024-05-09T09:27:53.147749+0000 (oldest deadline 2024-05-09T09:28:18.447727+0000) 2024-05-09T09:28:30.073 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 podman[70920]: 2024-05-09 09:28:29.823208483 +0000 UTC m=+5.741259414 container stop 22a8c622025713c3714aa2f173f27cce9c75b2e9a61d06e3ddef9fc2e603e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T09:28:30.073 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:29 smithi139 podman[70920]: 2024-05-09 09:28:29.84878432 +0000 UTC m=+5.766835239 container died 22a8c622025713c3714aa2f173f27cce9c75b2e9a61d06e3ddef9fc2e603e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc) 2024-05-09T09:28:30.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:30.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:30.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:30.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:30.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:30.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:30.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:29 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:29.747+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:30.573 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:30 smithi139 podman[70920]: 2024-05-09 09:28:30.170617948 +0000 UTC m=+6.088668866 container cleanup 22a8c622025713c3714aa2f173f27cce9c75b2e9a61d06e3ddef9fc2e603e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T09:28:30.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:30 smithi139 bash[70920]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10 2024-05-09T09:28:30.574 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:30 smithi139 podman[70937]: 2024-05-09 09:28:30.346901576 +0000 UTC m=+0.517499479 container remove 22a8c622025713c3714aa2f173f27cce9c75b2e9a61d06e3ddef9fc2e603e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T09:28:31.036 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:30 smithi139 podman[71031]: 2024-05-09 09:28:30.688182879 +0000 UTC m=+0.017706835 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:28:31.036 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:30 smithi139 podman[71031]: 2024-05-09 09:28:30.806036875 +0000 UTC m=+0.135560856 container create 6762810d674a30cd19fb8d254d9ab8d04afb8b3f432f1a41b541c8ee0fbf7ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:28:31.036 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:31 smithi139 podman[71031]: 2024-05-09 09:28:31.033118881 +0000 UTC m=+0.362642826 container init 6762810d674a30cd19fb8d254d9ab8d04afb8b3f432f1a41b541c8ee0fbf7ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T09:28:31.036 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:31 smithi139 podman[71031]: 2024-05-09 09:28:31.036394071 +0000 UTC m=+0.365918006 container start 6762810d674a30cd19fb8d254d9ab8d04afb8b3f432f1a41b541c8ee0fbf7ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10-deactivate, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T09:28:31.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:31.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:31.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:31.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:31.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:31.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:31.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:30 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:30.789+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:31.323 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:31 smithi139 podman[71031]: 2024-05-09 09:28:31.124690418 +0000 UTC m=+0.454214364 container attach 6762810d674a30cd19fb8d254d9ab8d04afb8b3f432f1a41b541c8ee0fbf7ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T09:28:31.659 INFO:journalctl@ceph.osd.10.smithi139.stdout:May 09 09:28:31 smithi139 podman[71031]: 2024-05-09 09:28:31.326478891 +0000 UTC m=+0.656002838 container died 6762810d674a30cd19fb8d254d9ab8d04afb8b3f432f1a41b541c8ee0fbf7ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T09:28:31.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.10.service' 2024-05-09T09:28:31.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:31.980 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T09:28:31.980 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T09:28:31.980 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.11 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:31 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:31.819+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:32.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 systemd[1]: Stopping Ceph osd.11 for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:32.729 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.288+0000 7fef5f9cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:32.730 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.288+0000 7fef5f9cf700 -1 osd.11 98 *** Got signal Terminated *** 2024-05-09T09:28:32.730 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.288+0000 7fef5f9cf700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:33.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:32 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:32.812+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:34.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:34.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:33 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:33.794+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-05-09T09:28:07.577292+0000 front 2024-05-09T09:28:07.577261+0000 (oldest deadline 2024-05-09T09:28:33.377615+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:35.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:34 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:34.833+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-05-09T09:28:07.577292+0000 front 2024-05-09T09:28:07.577261+0000 (oldest deadline 2024-05-09T09:28:33.377615+0000) 2024-05-09T09:28:36.229 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:36.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:35 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:35.860+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-05-09T09:28:07.577292+0000 front 2024-05-09T09:28:07.577261+0000 (oldest deadline 2024-05-09T09:28:33.377615+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-05-09T09:27:03.165043+0000 front 2024-05-09T09:27:03.165030+0000 (oldest deadline 2024-05-09T09:27:26.665004+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-09T09:27:21.069343+0000 front 2024-05-09T09:27:21.069263+0000 (oldest deadline 2024-05-09T09:27:46.969390+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-05-09T09:27:31.070574+0000 front 2024-05-09T09:27:31.070548+0000 (oldest deadline 2024-05-09T09:27:56.970625+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6812 osd.4 since back 2024-05-09T09:27:36.971495+0000 front 2024-05-09T09:27:36.971512+0000 (oldest deadline 2024-05-09T09:28:01.671571+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-09T09:27:46.773303+0000 front 2024-05-09T09:27:46.773364+0000 (oldest deadline 2024-05-09T09:28:10.873421+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-05-09T09:27:56.075187+0000 front 2024-05-09T09:27:56.075159+0000 (oldest deadline 2024-05-09T09:28:19.475440+0000) 2024-05-09T09:28:37.230 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.139:6820 osd.7 since back 2024-05-09T09:28:04.076861+0000 front 2024-05-09T09:28:04.076943+0000 (oldest deadline 2024-05-09T09:28:27.576709+0000) 2024-05-09T09:28:37.231 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:36 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11[59398]: 2024-05-09T09:28:36.910+0000 7fef57fbb700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-05-09T09:28:07.577292+0000 front 2024-05-09T09:28:07.577261+0000 (oldest deadline 2024-05-09T09:28:33.377615+0000) 2024-05-09T09:28:38.211 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:37 smithi179 podman[71183]: 2024-05-09 09:28:37.887879618 +0000 UTC m=+5.712015902 container died 9d87f069b0ad09ab527a3a1746cca8ea677ac7f35a44cf8944fca87aefe399d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:28:38.211 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:38 smithi179 podman[71183]: 2024-05-09 09:28:38.212712039 +0000 UTC m=+6.036848321 container cleanup 9d87f069b0ad09ab527a3a1746cca8ea677ac7f35a44cf8944fca87aefe399d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T09:28:38.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:38 smithi179 bash[71183]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11 2024-05-09T09:28:38.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:38 smithi179 podman[71200]: 2024-05-09 09:28:38.430310373 +0000 UTC m=+0.534498616 container remove 9d87f069b0ad09ab527a3a1746cca8ea677ac7f35a44cf8944fca87aefe399d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T09:28:39.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:38 smithi179 podman[71289]: 2024-05-09 09:28:38.822818027 +0000 UTC m=+0.103799437 container create 8db64b08788e3d643aab5ef20591c5a8482c1f3947c518019cecd52aa70c1b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5) 2024-05-09T09:28:39.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:38 smithi179 podman[71289]: 2024-05-09 09:28:38.737094295 +0000 UTC m=+0.018075675 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T09:28:39.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 podman[71289]: 2024-05-09 09:28:39.033312174 +0000 UTC m=+0.314293553 container init 8db64b08788e3d643aab5ef20591c5a8482c1f3947c518019cecd52aa70c1b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:28:39.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 podman[71289]: 2024-05-09 09:28:39.036557346 +0000 UTC m=+0.317538719 container start 8db64b08788e3d643aab5ef20591c5a8482c1f3947c518019cecd52aa70c1b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T09:28:39.116 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 podman[71289]: 2024-05-09 09:28:39.116512233 +0000 UTC m=+0.397493608 container attach 8db64b08788e3d643aab5ef20591c5a8482c1f3947c518019cecd52aa70c1b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-05-09T09:28:39.479 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 podman[71289]: 2024-05-09 09:28:39.326325142 +0000 UTC m=+0.607306511 container died 8db64b08788e3d643aab5ef20591c5a8482c1f3947c518019cecd52aa70c1b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-05-09T09:28:39.897 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.11.service' 2024-05-09T09:28:39.920 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 podman[71315]: 2024-05-09 09:28:39.878170803 +0000 UTC m=+0.546536339 container remove 8db64b08788e3d643aab5ef20591c5a8482c1f3947c518019cecd52aa70c1b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T09:28:39.920 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-09T09:28:39.920 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 systemd[1]: Stopped Ceph osd.11 for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:28:39.920 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 09 09:28:39 smithi179 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@osd.11.service: Consumed 9min 6.232s CPU time. 2024-05-09T09:28:39.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:39.935 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T09:28:39.936 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T09:28:39.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.a 2024-05-09T09:28:40.230 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:39 smithi064 systemd[1]: Stopping Ceph mds.a for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:40.653 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:40 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a[70149]: 2024-05-09T09:28:40.230+0000 7ff9eeb7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:40.654 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:40 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a[70149]: 2024-05-09T09:28:40.230+0000 7ff9eeb7e700 -1 mds.a *** got signal Terminated *** 2024-05-09T09:28:44.403 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:44 smithi064 podman[111781]: 2024-05-09 09:28:44.151475787 +0000 UTC m=+4.009474013 container died 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc) 2024-05-09T09:28:44.744 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:44 smithi064 podman[111781]: 2024-05-09 09:28:44.446149964 +0000 UTC m=+4.304148189 container cleanup 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T09:28:44.744 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:44 smithi064 bash[111781]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a 2024-05-09T09:28:44.745 INFO:journalctl@ceph.mds.a.smithi064.stdout:May 09 09:28:44 smithi064 podman[111796]: 2024-05-09 09:28:44.655729728 +0000 UTC m=+0.498544437 container remove 5f814df7527c4e0e0d88ce42d680af287395b8aa7483de0410551fdd69fea29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0) 2024-05-09T09:28:44.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.a.service' 2024-05-09T09:28:44.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:44.796 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T09:28:44.796 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T09:28:44.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.d 2024-05-09T09:28:45.153 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:28:44 smithi064 systemd[1]: Stopping Ceph mds.d for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:45.653 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:28:45 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-d[69181]: 2024-05-09T09:28:45.156+0000 7f69bb620700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:45.653 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:28:45 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-d[69181]: 2024-05-09T09:28:45.156+0000 7f69bb620700 -1 mds.d *** got signal Terminated *** 2024-05-09T09:28:48.354 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:28:47 smithi064 podman[111903]: 2024-05-09 09:28:47.971424489 +0000 UTC m=+2.951497873 container died d52b83eb665bcaa02bacba4bea79c936a0b50eaa82fb0debc638317959dc5b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T09:28:48.653 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:28:48 smithi064 podman[111903]: 2024-05-09 09:28:48.458222015 +0000 UTC m=+3.438295405 container cleanup d52b83eb665bcaa02bacba4bea79c936a0b50eaa82fb0debc638317959dc5b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T09:28:48.653 INFO:journalctl@ceph.mds.d.smithi064.stdout:May 09 09:28:48 smithi064 bash[111903]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-d 2024-05-09T09:28:48.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.d.service' 2024-05-09T09:28:48.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:48.884 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T09:28:48.884 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T09:28:48.884 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.g 2024-05-09T09:28:49.228 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:28:48 smithi064 systemd[1]: Stopping Ceph mds.g for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:49.653 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:28:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g[69789]: 2024-05-09T09:28:49.228+0000 7f0c73a3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:49.653 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:28:49 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g[69789]: 2024-05-09T09:28:49.228+0000 7f0c73a3c700 -1 mds.g *** got signal Terminated *** 2024-05-09T09:28:53.546 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:28:53 smithi064 podman[112025]: 2024-05-09 09:28:53.242460793 +0000 UTC m=+4.129681806 container died 05109ada3bef95caa97e64f2479b6f01c7c2adb4889cc0fce2f5d811af5ddf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T09:28:53.853 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:28:53 smithi064 podman[112025]: 2024-05-09 09:28:53.62032157 +0000 UTC m=+4.507542584 container cleanup 05109ada3bef95caa97e64f2479b6f01c7c2adb4889cc0fce2f5d811af5ddf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:28:53.854 INFO:journalctl@ceph.mds.g.smithi064.stdout:May 09 09:28:53 smithi064 bash[112025]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-g 2024-05-09T09:28:53.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.g.service' 2024-05-09T09:28:54.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:54.033 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T09:28:54.033 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T09:28:54.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.j 2024-05-09T09:28:54.403 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:54 smithi064 systemd[1]: Stopping Ceph mds.j for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:54.903 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-j[68830]: 2024-05-09T09:28:54.473+0000 7f8bb6831700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:54.903 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:54 smithi064 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-j[68830]: 2024-05-09T09:28:54.473+0000 7f8bb6831700 -1 mds.j *** got signal Terminated *** 2024-05-09T09:28:57.317 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 podman[112148]: 2024-05-09 09:28:57.060648351 +0000 UTC m=+2.795129000 container died 60e87e37b152ce8119b582effaa5b3291d9290943a8e728f53d53dcc22f662c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-j, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T09:28:57.599 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 podman[112148]: 2024-05-09 09:28:57.415506153 +0000 UTC m=+3.149986803 container cleanup 60e87e37b152ce8119b582effaa5b3291d9290943a8e728f53d53dcc22f662c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-j, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T09:28:57.599 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 bash[112148]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-j 2024-05-09T09:28:57.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.j.service' 2024-05-09T09:28:57.864 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 podman[112162]: 2024-05-09 09:28:57.600099258 +0000 UTC m=+0.537888365 container remove 60e87e37b152ce8119b582effaa5b3291d9290943a8e728f53d53dcc22f662c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T09:28:57.864 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.j.service: Deactivated successfully. 2024-05-09T09:28:57.864 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 systemd[1]: Stopped Ceph mds.j for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:28:57.864 INFO:journalctl@ceph.mds.j.smithi064.stdout:May 09 09:28:57 smithi064 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.j.service: Consumed 1.145s CPU time. 2024-05-09T09:28:57.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:28:57.875 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T09:28:57.875 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T09:28:57.876 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.b 2024-05-09T09:28:58.185 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:28:57 smithi139 systemd[1]: Stopping Ceph mds.b for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:28:58.451 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:28:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:28:58.185+0000 7fe52c6f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:28:58.536 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:28:58 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b[64280]: 2024-05-09T09:28:58.185+0000 7fe52c6f0700 -1 mds.b *** got signal Terminated *** 2024-05-09T09:28:59.752 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:28:59 smithi139 podman[71138]: 2024-05-09 09:28:59.459155474 +0000 UTC m=+1.380919580 container died 7b9d727b4326ef2679654afe0b266c522bad469bbb18b075214a76d5394ba3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T09:29:00.073 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:28:59 smithi139 podman[71138]: 2024-05-09 09:28:59.835427394 +0000 UTC m=+1.757191516 container cleanup 7b9d727b4326ef2679654afe0b266c522bad469bbb18b075214a76d5394ba3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T09:29:00.073 INFO:journalctl@ceph.mds.b.smithi139.stdout:May 09 09:28:59 smithi139 bash[71138]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-b 2024-05-09T09:29:00.265 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.b.service' 2024-05-09T09:29:00.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:00.301 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T09:29:00.301 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T09:29:00.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.e 2024-05-09T09:29:00.823 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:00 smithi139 systemd[1]: Stopping Ceph mds.e for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:00.824 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e[63663]: 2024-05-09T09:29:00.630+0000 7f4b7603d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:00.824 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:00 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e[63663]: 2024-05-09T09:29:00.630+0000 7f4b7603d700 -1 mds.e *** got signal Terminated *** 2024-05-09T09:29:03.771 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:03 smithi139 podman[71268]: 2024-05-09 09:29:03.476426424 +0000 UTC m=+2.956322993 container died 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T09:29:04.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:03 smithi139 podman[71268]: 2024-05-09 09:29:03.854898601 +0000 UTC m=+3.334795180 container cleanup 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-05-09T09:29:04.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:03 smithi139 bash[71268]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e 2024-05-09T09:29:04.073 INFO:journalctl@ceph.mds.e.smithi139.stdout:May 09 09:29:03 smithi139 podman[71283]: 2024-05-09 09:29:03.997661282 +0000 UTC m=+0.520880590 container remove 816b3bfd0e59205f53d28b6c93cc1e8fe7d61769fe30a5f4afe4a5d4089aa9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T09:29:04.217 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.e.service' 2024-05-09T09:29:04.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:04.251 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T09:29:04.251 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T09:29:04.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.h 2024-05-09T09:29:04.573 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:04 smithi139 systemd[1]: Stopping Ceph mds.h for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:05.073 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h[63973]: 2024-05-09T09:29:04.574+0000 7fd24ae06700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:05.073 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:04 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h[63973]: 2024-05-09T09:29:04.574+0000 7fd24ae06700 -1 mds.h *** got signal Terminated *** 2024-05-09T09:29:07.733 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:07 smithi139 podman[71389]: 2024-05-09 09:29:07.474929236 +0000 UTC m=+2.997542504 container died 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2) 2024-05-09T09:29:08.073 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:07 smithi139 podman[71389]: 2024-05-09 09:29:07.824863736 +0000 UTC m=+3.347477005 container cleanup 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T09:29:08.339 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:07 smithi139 bash[71389]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h 2024-05-09T09:29:08.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.h.service' 2024-05-09T09:29:08.364 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:08 smithi139 podman[71403]: 2024-05-09 09:29:08.084001137 +0000 UTC m=+0.610367574 container remove 83a4eba0ae3ddb8c1192efdc729fa8dc059d9ae49f39d5e9e65ce98c6fa3d630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T09:29:08.364 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:08 smithi139 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-09T09:29:08.364 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:08 smithi139 systemd[1]: Stopped Ceph mds.h for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:29:08.365 INFO:journalctl@ceph.mds.h.smithi139.stdout:May 09 09:29:08 smithi139 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.h.service: Consumed 3.765s CPU time. 2024-05-09T09:29:08.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:08.376 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T09:29:08.376 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T09:29:08.376 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.k 2024-05-09T09:29:08.694 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:29:08 smithi139 systemd[1]: Stopping Ceph mds.k for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:09.073 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:29:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k[63336]: 2024-05-09T09:29:08.694+0000 7f10100ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:09.073 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:29:08 smithi139 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k[63336]: 2024-05-09T09:29:08.694+0000 7f10100ce700 -1 mds.k *** got signal Terminated *** 2024-05-09T09:29:10.419 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:29:10 smithi139 podman[71511]: 2024-05-09 09:29:10.157959889 +0000 UTC m=+1.563304235 container died 6290d5fab2919328abab466c239c69ce61f31a6703b63bc1fadee39afaf3ff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T09:29:10.761 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:29:10 smithi139 podman[71511]: 2024-05-09 09:29:10.502858863 +0000 UTC m=+1.908203217 container cleanup 6290d5fab2919328abab466c239c69ce61f31a6703b63bc1fadee39afaf3ff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T09:29:10.761 INFO:journalctl@ceph.mds.k.smithi139.stdout:May 09 09:29:10 smithi139 bash[71511]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-k 2024-05-09T09:29:10.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.k.service' 2024-05-09T09:29:10.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:10.958 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T09:29:10.958 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T09:29:10.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.c 2024-05-09T09:29:11.327 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:29:11 smithi179 systemd[1]: Stopping Ceph mds.c for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:11.596 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:29:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c[64381]: 2024-05-09T09:29:11.327+0000 7fd6d2d5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:11.597 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:29:11 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c[64381]: 2024-05-09T09:29:11.327+0000 7fd6d2d5b700 -1 mds.c *** got signal Terminated *** 2024-05-09T09:29:15.677 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:29:15 smithi179 podman[71395]: 2024-05-09 09:29:15.417896322 +0000 UTC m=+4.261925403 container died 0fccc11519eb5f2b56d68b76bcca685cee0e464ae6212f84331449bd81faea2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T09:29:15.980 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:29:15 smithi179 podman[71395]: 2024-05-09 09:29:15.768913929 +0000 UTC m=+4.612943057 container cleanup 0fccc11519eb5f2b56d68b76bcca685cee0e464ae6212f84331449bd81faea2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T09:29:15.980 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 09 09:29:15 smithi179 bash[71395]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-c 2024-05-09T09:29:16.196 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.c.service' 2024-05-09T09:29:16.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:16.232 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T09:29:16.233 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T09:29:16.233 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.f 2024-05-09T09:29:16.547 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:16 smithi179 systemd[1]: Stopping Ceph mds.f for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:16.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f[64064]: 2024-05-09T09:29:16.547+0000 7f4aa2db3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:16.979 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:16 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f[64064]: 2024-05-09T09:29:16.547+0000 7f4aa2db3700 -1 mds.f *** got signal Terminated *** 2024-05-09T09:29:19.729 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:19 smithi179 podman[71526]: 2024-05-09 09:29:19.478092839 +0000 UTC m=+3.024537729 container died 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T09:29:20.106 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:19 smithi179 podman[71526]: 2024-05-09 09:29:19.905918311 +0000 UTC m=+3.452363201 container cleanup 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-09T09:29:20.106 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:19 smithi179 bash[71526]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f 2024-05-09T09:29:20.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.f.service' 2024-05-09T09:29:20.364 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:20 smithi179 podman[71540]: 2024-05-09 09:29:20.106970818 +0000 UTC m=+0.627624270 container remove 9ce10c83ce2b6e94db53f7101689df1a60bdd72026b8d4630c3c742af5556eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T09:29:20.365 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:20 smithi179 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-09T09:29:20.365 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:20 smithi179 systemd[1]: Stopped Ceph mds.f for 2877b902-0de2-11ef-bc98-c7b262605968. 2024-05-09T09:29:20.365 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 09 09:29:20 smithi179 systemd[1]: ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.f.service: Consumed 4.054s CPU time. 2024-05-09T09:29:20.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:20.377 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T09:29:20.377 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T09:29:20.377 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.i 2024-05-09T09:29:20.691 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:20 smithi179 systemd[1]: Stopping Ceph mds.i for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:20.979 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i[64686]: 2024-05-09T09:29:20.692+0000 7f0219cef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:20.979 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:20 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i[64686]: 2024-05-09T09:29:20.692+0000 7f0219cef700 -1 mds.i *** got signal Terminated *** 2024-05-09T09:29:23.729 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:23 smithi179 podman[71650]: 2024-05-09 09:29:23.477068601 +0000 UTC m=+2.876187070 container died d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T09:29:23.992 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:23 smithi179 podman[71650]: 2024-05-09 09:29:23.817392974 +0000 UTC m=+3.216511449 container cleanup d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T09:29:23.992 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:23 smithi179 bash[71650]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i 2024-05-09T09:29:24.245 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 09 09:29:23 smithi179 podman[71664]: 2024-05-09 09:29:23.993534304 +0000 UTC m=+0.511432085 container remove d5f5d76ae01a2c8bfbf67d8816f2e9065e301f1949469a44df204dbb3ca893a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T09:29:24.249 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.i.service' 2024-05-09T09:29:24.282 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:24.282 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T09:29:24.282 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T09:29:24.283 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.l 2024-05-09T09:29:24.603 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:24 smithi179 systemd[1]: Stopping Ceph mds.l for 2877b902-0de2-11ef-bc98-c7b262605968... 2024-05-09T09:29:24.979 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l[63752]: 2024-05-09T09:29:24.604+0000 7fc3ecec9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T09:29:24.979 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:24 smithi179 ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l[63752]: 2024-05-09T09:29:24.604+0000 7fc3ecec9700 -1 mds.l *** got signal Terminated *** 2024-05-09T09:29:27.569 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:27 smithi179 podman[71769]: 2024-05-09 09:29:27.249006957 +0000 UTC m=+2.748523314 container died 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T09:29:27.964 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:27 smithi179 podman[71769]: 2024-05-09 09:29:27.570977684 +0000 UTC m=+3.070494048 container cleanup 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T09:29:27.964 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:27 smithi179 bash[71769]: ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l 2024-05-09T09:29:27.964 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 09 09:29:27 smithi179 podman[71782]: 2024-05-09 09:29:27.77213753 +0000 UTC m=+0.522886322 container remove 7b3cac0093696c40b23e18b27fdae7ba8f4ecee26c74af5d12fdd8b30c2b02ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2877b902-0de2-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T09:29:27.975 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2877b902-0de2-11ef-bc98-c7b262605968@mds.l.service' 2024-05-09T09:29:28.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T09:29:28.007 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T09:29:28.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T09:29:28.210 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:29:30.789 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T09:29:30.984 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:29:33.476 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T09:29:33.669 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:29:36.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:29:36.071 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:29:36.105 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T09:29:36.238 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T09:29:36.273 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi064/crash 2024-05-09T09:29:36.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash -- . 2024-05-09T09:29:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T09:29:36.340 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-05-09T09:29:36.347 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi139/crash 2024-05-09T09:29:36.348 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash -- . 2024-05-09T09:29:36.397 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T09:29:36.397 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-05-09T09:29:36.398 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi179/crash 2024-05-09T09:29:36.400 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash -- . 2024-05-09T09:29:36.447 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/2877b902-0de2-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T09:29:36.447 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-05-09T09:29:36.448 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T09:29:36.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T09:29:36.543 INFO:tasks.cephadm:Compressing logs... 2024-05-09T09:29:36.544 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T09:29:36.546 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T09:29:36.549 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T09:29:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log 2024-05-09T09:29:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-05-09T09:29:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-09T09:29:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-09T09:29:36.577 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-09T09:29:36.578 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T09:29:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-05-09T09:29:36.585 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-09T09:29:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log 2024-05-09T09:29:36.586 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T09:29:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T09:29:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-09T09:29:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-09T09:29:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log 2024-05-09T09:29:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T09:29:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-09T09:29:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-09T09:29:36.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-09T09:29:36.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-09T09:29:36.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-09T09:29:36.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-09T09:29:36.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-09T09:29:36.592 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.a.log: 87.0% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T09:29:36.593 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-09T09:29:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-09T09:29:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-09T09:29:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-09T09:29:36.597 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T09:29:36.643 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 83.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T09:29:36.649 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T09:29:36.665 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-09T09:29:36.718 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-09T09:29:36.724 INFO:teuthology.orchestra.run.smithi179.stderr: 87.0% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T09:29:36.728 INFO:teuthology.orchestra.run.smithi179.stderr: 90.2% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T09:29:36.786 INFO:teuthology.orchestra.run.smithi179.stderr: 93.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T09:29:36.791 INFO:teuthology.orchestra.run.smithi064.stderr: 90.0% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T09:29:36.795 INFO:teuthology.orchestra.run.smithi064.stderr: 91.1% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T09:29:36.799 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T09:29:36.825 INFO:teuthology.orchestra.run.smithi179.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T09:29:36.835 INFO:teuthology.orchestra.run.smithi179.stderr: 91.0% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T09:29:36.859 INFO:teuthology.orchestra.run.smithi179.stderr: 83.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T09:29:36.867 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T09:29:36.890 INFO:teuthology.orchestra.run.smithi139.stderr: 83.4% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T09:29:36.899 INFO:teuthology.orchestra.run.smithi139.stderr: 87.0% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T09:29:36.901 INFO:teuthology.orchestra.run.smithi139.stderr: 90.2% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T09:29:37.039 INFO:teuthology.orchestra.run.smithi139.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T09:29:37.159 INFO:teuthology.orchestra.run.smithi139.stderr: 94.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-09T09:29:37.180 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-09T09:29:38.218 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-09T09:29:38.392 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T09:29:38.659 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-09T09:29:38.662 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-09T09:29:39.805 INFO:teuthology.orchestra.run.smithi179.stderr: 91.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-09T09:29:41.924 INFO:teuthology.orchestra.run.smithi139.stderr: 94.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-09T09:29:41.950 INFO:teuthology.orchestra.run.smithi139.stderr: 95.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-09T09:29:48.465 INFO:teuthology.orchestra.run.smithi139.stderr: 91.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-09T09:29:49.235 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-09T09:29:49.983 INFO:teuthology.orchestra.run.smithi179.stderr: 94.4% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-09T09:30:08.054 INFO:teuthology.orchestra.run.smithi179.stderr: 95.3% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-09T09:30:20.116 INFO:teuthology.orchestra.run.smithi139.stderr: 93.6% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-09T09:31:03.466 INFO:teuthology.orchestra.run.smithi179.stderr: 94.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-09T09:31:39.210 INFO:teuthology.orchestra.run.smithi064.stderr: 90.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-09T09:31:45.403 INFO:teuthology.orchestra.run.smithi179.stderr: 90.9% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-09T09:33:02.238 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-09T09:34:44.825 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-09T09:34:58.645 INFO:teuthology.orchestra.run.smithi179.stderr: 90.7% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-09T09:37:09.687 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-09T09:37:14.870 INFO:teuthology.orchestra.run.smithi179.stderr: 90.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-09T09:37:34.808 INFO:teuthology.orchestra.run.smithi179.stderr: 90.9% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-09T09:37:34.809 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-05-09T09:37:34.810 INFO:teuthology.orchestra.run.smithi179.stderr:real 7m58.245s 2024-05-09T09:37:34.810 INFO:teuthology.orchestra.run.smithi179.stderr:user 6m45.113s 2024-05-09T09:37:34.810 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m15.784s 2024-05-09T09:37:36.061 INFO:teuthology.orchestra.run.smithi139.stderr: 90.7% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-09T09:37:36.063 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-05-09T09:37:36.063 INFO:teuthology.orchestra.run.smithi139.stderr:real 7m59.499s 2024-05-09T09:37:36.063 INFO:teuthology.orchestra.run.smithi139.stderr:user 6m32.600s 2024-05-09T09:37:36.063 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m15.055s 2024-05-09T09:46:11.863 INFO:teuthology.orchestra.run.smithi064.stderr: 90.8% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-09T09:49:33.273 INFO:teuthology.orchestra.run.smithi064.stderr: 90.7% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-09T09:50:13.106 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-09T09:50:26.024 INFO:teuthology.orchestra.run.smithi064.stderr: 90.7% -- replaced with /var/log/ceph/2877b902-0de2-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-09T09:50:26.037 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-05-09T09:50:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:real 20m49.476s 2024-05-09T09:50:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:user 9m38.460s 2024-05-09T09:50:26.038 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m23.474s 2024-05-09T09:50:26.038 INFO:tasks.cephadm:Archiving logs... 2024-05-09T09:50:26.039 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi064/log 2024-05-09T09:50:26.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:52:00.133 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi139/log 2024-05-09T09:52:00.136 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:53:13.705 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi179/log 2024-05-09T09:53:13.707 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:54:29.544 INFO:tasks.cephadm:Removing cluster... 2024-05-09T09:54:29.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --force 2024-05-09T09:54:29.876 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:54:30.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --force 2024-05-09T09:54:30.946 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:54:31.744 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2877b902-0de2-11ef-bc98-c7b262605968 --force 2024-05-09T09:54:32.339 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 2877b902-0de2-11ef-bc98-c7b262605968 2024-05-09T09:54:33.155 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T09:54:33.155 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:54:33.176 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:54:33.195 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:54:33.230 INFO:tasks.cephadm:Teardown complete 2024-05-09T09:54:33.231 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T09:54:33.242 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T09:54:33.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:54:33.245 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:54:33.248 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:54:33.463 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T09:54:33.474 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T09:54:33.475 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:54:33.478 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:54:33.480 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:54:33.493 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:54:33.495 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:54:33.495 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:54:33.501 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:54:33.501 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-05-09T09:54:33.501 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +406us[ +425us] +/- 50ms 2024-05-09T09:54:33.501 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -959us[ -941us] +/- 73ms 2024-05-09T09:54:33.501 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +312us[ +331us] +/- 74ms 2024-05-09T09:54:33.501 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:54:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:54:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-09T09:54:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +426us[ +436us] +/- 50ms 2024-05-09T09:54:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 -932us[ -923us] +/- 73ms 2024-05-09T09:54:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +356us[ +366us] +/- 74ms 2024-05-09T09:54:33.504 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:54:33.543 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:54:33.543 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-05-09T09:54:33.543 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +459us[ +460us] +/- 50ms 2024-05-09T09:54:33.543 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -945us[ -944us] +/- 73ms 2024-05-09T09:54:33.543 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +337us[ +337us] +/- 74ms 2024-05-09T09:54:33.543 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:54:33.544 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T09:54:33.556 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T09:54:33.556 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T09:54:33.566 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:54:33.730 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:54:33.810 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:54:33.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:54:33.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:54:33.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:54:33.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:54:33.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:54:33.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:54:33.985 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T09:54:33.998 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T09:54:34.008 INFO:teuthology.task.internal:Duration was 3797.235509 seconds 2024-05-09T09:54:34.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T09:54:34.019 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T09:54:34.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:54:34.022 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:54:34.025 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:54:34.070 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:54:34.087 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:54:34.106 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:54:34.376 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T09:54:34.376 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-05-09T09:54:34.377 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:54:34.410 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-05-09T09:54:34.411 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:54:34.439 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-05-09T09:54:34.440 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:54:34.468 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T09:54:34.468 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:54:34.471 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:54:34.483 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:54:34.535 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T09:54:34.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:54:34.541 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:54:34.551 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:54:34.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T09:54:34.981 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T09:54:34.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:54:35.016 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:54:35.043 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:54:35.082 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T09:54:35.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:54:35.096 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:54:35.098 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:54:35.121 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-05-09T09:54:35.131 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-05-09T09:54:35.151 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-05-09T09:54:35.167 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:54:35.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:54:35.197 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:54:35.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:54:35.213 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:54:35.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:54:35.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T09:54:35.239 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T09:54:35.240 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi064 2024-05-09T09:54:35.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:54:35.286 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi139 2024-05-09T09:54:35.287 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:54:35.324 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698710/remote/smithi179 2024-05-09T09:54:35.324 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:54:35.362 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T09:54:35.362 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:54:35.364 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:54:35.367 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:54:35.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T09:54:35.422 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T09:54:35.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T09:54:35.433 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T09:54:35.433 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:54:35.436 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:54:35.439 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:54:35.449 INFO:teuthology.orchestra.run.smithi064.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:54 /home/ubuntu/cephtest 2024-05-09T09:54:35.451 INFO:teuthology.orchestra.run.smithi139.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:54 /home/ubuntu/cephtest 2024-05-09T09:54:35.464 INFO:teuthology.orchestra.run.smithi179.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:54 /home/ubuntu/cephtest 2024-05-09T09:54:35.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T09:54:35.478 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T09:54:35.508 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 3797.235508918762 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T09:54:35.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T09:54:35.589 INFO:teuthology.run:pass